public inbox for [email protected]
help / color / mirror / Atom feedFrom: Peter Eisentraut <[email protected]>
To: pgsql-hackers <[email protected]>
Subject: Re: Transparent column encryption
Date: Tue, 5 Jul 2022 12:54:13 +0200
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
<[email protected]>
Rebased patch, no new functionality.
On 29.06.22 01:29, Peter Eisentraut wrote:
> Here is a new version of this patch. See also the original description
> quoted below. I have done a significant amount of work on this over the
> last few months. Some important news include:
>
> - The cryptography has been improved. It now uses an AEAD scheme, and
> for deterministic encryption a proper SIV construction.
>
> - The OpenSSL-specific parts have been moved to a separate file in
> libpq. Non-OpenSSL builds compile and work (without functionality, of
> course).
>
> - libpq handles multiple CEKs and CMKs, including changing keys on the fly.
>
> - libpq supports a mode to force encryption of certain values.
>
> - libpq supports a flexible configuration system for looking up CMKs,
> including support for external key management systems.
>
> - psql has a new \gencr command that allows passing in bind parameters
> for (potential) encryption.
>
> - There is some more pg_dump and psql support.
>
> - The new data types for storing encrypted data have been renamed for
> clarity.
>
> - Various changes to the protocol compared to the previous patch.
>
> - The patch contains full documentation of the protocol changes,
> glossary entries, and more new documentation.
>
> The major pieces that are still missing are:
>
> - DDL support for registering keys
>
> - Protocol versioning or feature flags
>
> Other than that it's pretty complete in my mind.
>
> For interested reviewers, I have organized the patch so that you can
> start reading it top to bottom: The documentation comes first, then the
> tests, then the code changes. Even some feedback on the first or first
> two aspects would be valuable to me.
>
> Old news follows:
>
> On 03.12.21 22:32, Peter Eisentraut wrote:
>> I want to present my proof-of-concept patch for the transparent column
>> encryption feature. (Some might also think of it as automatic
>> client-side encryption or similar, but I like my name.) This feature
>> enables the {automatic,transparent} encryption and decryption of
>> particular columns in the client. The data for those columns then
>> only ever appears in ciphertext on the server, so it is protected from
>> the "prying eyes" of DBAs, sysadmins, cloud operators, etc. The
>> canonical use case for this feature is storing credit card numbers
>> encrypted, in accordance with PCI DSS, as well as similar situations
>> involving social security numbers etc. Of course, you can't do any
>> computations with encrypted values on the server, but for these use
>> cases, that is not necessary. This feature does support deterministic
>> encryption as an alternative to the default randomized encryption, so
>> in that mode you can do equality lookups, at the cost of some
>> security.
>>
>> This functionality also exists in other SQL database products, so the
>> overall concepts weren't invented by me by any means.
>>
>> Also, this feature has nothing to do with the on-disk encryption
>> feature being contemplated in parallel. Both can exist independently.
>>
>> The attached patch has all the necessary pieces in place to make this
>> work, so you can have an idea how the overall system works. It
>> contains some documentation and tests to help illustrate the
>> functionality. But it's missing the remaining 90% of the work,
>> including additional DDL support, error handling, robust memory
>> management, protocol versioning, forward and backward compatibility,
>> pg_dump support, psql \d support, refinement of the cryptography, and
>> so on. But I think obvious solutions exist to all of those things, so
>> it isn't that interesting to focus on them for now.
>>
>> ------
>>
>> Now to the explanation of how it works.
>>
>> You declare a column as encrypted in a CREATE TABLE statement. The
>> column value is encrypted by a symmetric key called the column
>> encryption key (CEK). The CEK is a catalog object. The CEK key
>> material is in turn encrypted by an assymmetric key called the column
>> master key (CMK). The CMK is not stored in the database but somewhere
>> where the client can get to it, for example in a file or in a key
>> management system. When a server sends rows containing encrypted
>> column values to the client, it first sends the required CMK and CEK
>> information (new protocol messages), which the client needs to record.
>> Then, the client can use this information to automatically decrypt the
>> incoming row data and forward it in plaintext to the application.
>>
>> For the CMKs, the catalog object specifies a "provider" and generic
>> options. Right now, libpq has a "file" provider hardcoded, and it
>> takes a "filename" option. Via some mechanism to be determined,
>> additional providers could be loaded and then talk to key management
>> systems via http or whatever. I have left some comments in the libpq
>> code where the hook points for this could be.
>>
>> The general idea would be for an application to have one CMK per area
>> of secret stuff, for example, for credit card data. The CMK can be
>> rotated: each CEK can be represented multiple times in the database,
>> encrypted by a different CMK. (The CEK can't be rotated easily, since
>> that would require reading out all the data from a table/column and
>> reencrypting it. We could/should add some custom tooling for that,
>> but it wouldn't be a routine operation.)
>>
>> The encryption algorithms are mostly hardcoded right now, but there
>> are facilities for picking algorithms and adding new ones that will be
>> expanded. The CMK process uses RSA-OAEP. The CEK process uses
>> AES-128-CBC right now; a more complete solution should probably
>> involve some HMAC thrown in.
>>
>> In the server, the encrypted datums are stored in types called
>> encryptedr and encryptedd (for randomized and deterministic
>> encryption). These are essentially cousins of bytea. For the rest of
>> the database system below the protocol handling, there is nothing
>> special about those. For example, encryptedr has no operators at all,
>> encryptedd has only an equality operator. pg_attribute has a new
>> column attrealtypid that stores the original type of the data in the
>> column. This is only used for providing it to clients, so that
>> higher-level clients can convert the decrypted value to their
>> appropriate data types in their environments.
>>
>> Some protocol extensions are required. These should be guarded by
>> some _pq_... setting, but this is not done in this patch yet. As
>> mentioned above, extra messages are added for sending the CMKs and
>> CEKs. In the RowDescription message, I have commandeered the format
>> field to add a bit that indicates that the field is encrypted. This
>> could be made a separate field, and there should probably be
>> additional fields to indicate the algorithm and CEK name, but this was
>> easiest for now. The ParameterDescription message is extended to
>> contain format fields for each parameter, for the same purpose.
>> Again, this could be done differently.
>>
>> Speaking of parameter descriptions, the trickiest part of this whole
>> thing appears to be how to get transparently encrypted data into the
>> database (as opposed to reading it out). It is required to use
>> protocol-level prepared statements (i.e., extended query) for this.
>> The client must first prepare a statement, then describe the statement
>> to get parameter metadata, which indicates which parameters are to be
>> encrypted and how. So this will require some care by applications
>> that want to do this, but, well, they probably should be careful
>> anyway. In libpq, the existing APIs make this difficult, because
>> there is no way to pass the result of a describe-statement call back
>> into execute-statement-with-parameters. I added new functions that do
>> this, so you then essentially do
>>
>> res0 = PQdescribePrepared(conn, "");
>> res = PQexecPrepared2(conn, "", 2, values, NULL, NULL, 0, res0);
>>
>> (The name could obviously be improved.) Other client APIs that have a
>> "statement handle" concept could do this more elegantly and probably
>> without any API changes.
>>
>> Another challenge is that the parse analysis must check which
>> underlying column a parameter corresponds to. This is similar to
>> resorigtbl and resorigcol in the opposite direction. The current
>> implementation of this works for the test cases, but I know it has
>> some problems, so I'll continue working in this. This functionality
>> is in principle available to all prepared-statement variants, not only
>> protocol-level. So you can see in the tests that I expanded the
>> pg_prepared_statements view to show this information as well, which
>> also provides an easy way to test and debug this functionality
>> independent of column encryption.
>>
>> And also, psql doesn't use prepared statements, so writing into
>> encrypted columns currently doesn't work at all via psql. (Reading
>> works no problem.) All the test code currently uses custom libpq C
>> programs. We should think about a way to enable prepared statements
>> in psql, perhaps something like
>>
>> INSERT INTO t1 VALUES ($1, $2) \gg 'val1' 'val2'
>>
>> (\gexec and \gx are already taken.)
>>
>> ------
>>
>> This is not targeting PostgreSQL 15. But I'd appreciate some feedback
>> on the direction. As I mentioned above, a lot of the remaining work
>> is arguably mostly straightforward. Some closer examination of the
>> issues surrounding the libpq API changes and psql would be useful.
>> Perhaps there are other projects where that kind of functionality
>> would also be useful.
From bcb9df8eaafdd1b5a05d8ea4bb5b2af001f6403f Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <[email protected]>
Date: Tue, 5 Jul 2022 12:48:18 +0200
Subject: [PATCH v3] Transparent column encryption
---
doc/src/sgml/acronyms.sgml | 18 +
doc/src/sgml/catalogs.sgml | 275 +++++++
doc/src/sgml/datatype.sgml | 47 ++
doc/src/sgml/ddl.sgml | 73 ++
doc/src/sgml/glossary.sgml | 23 +
doc/src/sgml/libpq.sgml | 257 +++++-
doc/src/sgml/protocol.sgml | 346 ++++++++
doc/src/sgml/ref/create_table.sgml | 42 +-
doc/src/sgml/ref/psql-ref.sgml | 33 +
src/test/Makefile | 3 +-
src/test/column_encryption/.gitignore | 3 +
src/test/column_encryption/Makefile | 29 +
.../t/001_column_encryption.pl | 206 +++++
.../column_encryption/t/002_cmk_rotation.pl | 144 ++++
src/test/column_encryption/test_client.c | 210 +++++
.../column_encryption/test_run_decrypt.pl | 41 +
.../traces/disallowed_in_pipeline.trace | 2 +-
.../traces/multi_pipelines.trace | 4 +-
.../libpq_pipeline/traces/nosync.trace | 20 +-
.../traces/pipeline_abort.trace | 4 +-
.../libpq_pipeline/traces/prepared.trace | 6 +-
.../traces/simple_pipeline.trace | 2 +-
.../libpq_pipeline/traces/singlerow.trace | 6 +-
.../libpq_pipeline/traces/transaction.trace | 2 +-
.../regress/expected/column_encryption.out | 59 ++
src/test/regress/expected/oidjoins.out | 6 +
src/test/regress/expected/opr_sanity.out | 12 +-
src/test/regress/expected/prepare.out | 38 +-
src/test/regress/expected/psql.out | 25 +
src/test/regress/expected/rules.out | 4 +-
src/test/regress/expected/type_sanity.out | 20 +-
src/test/regress/parallel_schedule | 3 +
src/test/regress/pg_regress_main.c | 2 +-
src/test/regress/sql/column_encryption.sql | 50 ++
src/test/regress/sql/prepare.sql | 2 +-
src/test/regress/sql/psql.sql | 17 +
src/test/regress/sql/type_sanity.sql | 4 +-
src/include/access/printtup.h | 2 +
src/include/catalog/pg_amop.dat | 5 +
src/include/catalog/pg_amproc.dat | 5 +
src/include/catalog/pg_attribute.h | 9 +
src/include/catalog/pg_cast.dat | 6 +
src/include/catalog/pg_colenckey.h | 55 ++
src/include/catalog/pg_colenckeydata.h | 46 ++
src/include/catalog/pg_colmasterkey.h | 45 ++
src/include/catalog/pg_opclass.dat | 2 +
src/include/catalog/pg_operator.dat | 10 +
src/include/catalog/pg_opfamily.dat | 2 +
src/include/catalog/pg_proc.dat | 13 +-
src/include/catalog/pg_type.dat | 12 +
src/include/catalog/pg_type.h | 1 +
src/include/nodes/parsenodes.h | 1 +
src/include/parser/analyze.h | 4 +-
src/include/parser/parse_node.h | 2 +
src/include/parser/parse_param.h | 3 +-
src/include/tcop/tcopprot.h | 2 +
src/include/utils/plancache.h | 6 +
src/include/utils/syscache.h | 4 +
src/backend/access/common/printsimple.c | 2 +
src/backend/access/common/printtup.c | 162 ++++
src/backend/access/common/tupdesc.c | 12 +
src/backend/access/hash/hashvalidate.c | 2 +-
src/backend/catalog/Makefile | 3 +-
src/backend/catalog/heap.c | 3 +
src/backend/commands/prepare.c | 74 +-
src/backend/commands/tablecmds.c | 74 ++
src/backend/executor/spi.c | 4 +
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/analyze.c | 3 +-
src/backend/parser/gram.y | 13 +-
src/backend/parser/parse_param.c | 43 +-
src/backend/parser/parse_target.c | 6 +
src/backend/tcop/postgres.c | 54 +-
src/backend/utils/adt/arrayfuncs.c | 1 +
src/backend/utils/cache/plancache.c | 10 +
src/backend/utils/cache/syscache.c | 39 +
src/bin/pg_dump/pg_dump.c | 26 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/psql/command.c | 31 +
src/bin/psql/common.c | 39 +-
src/bin/psql/describe.c | 29 +-
src/bin/psql/help.c | 3 +
src/bin/psql/settings.h | 5 +
src/bin/psql/startup.c | 10 +
src/bin/psql/tab-complete.c | 2 +-
src/interfaces/libpq/Makefile | 1 +
src/interfaces/libpq/exports.txt | 4 +
src/interfaces/libpq/fe-connect.c | 20 +
src/interfaces/libpq/fe-encrypt-openssl.c | 758 ++++++++++++++++++
src/interfaces/libpq/fe-encrypt.h | 33 +
src/interfaces/libpq/fe-exec.c | 594 +++++++++++++-
src/interfaces/libpq/fe-protocol3.c | 123 ++-
src/interfaces/libpq/fe-trace.c | 7 +
src/interfaces/libpq/libpq-fe.h | 22 +
src/interfaces/libpq/libpq-int.h | 34 +
src/interfaces/libpq/nls.mk | 2 +-
src/interfaces/libpq/test/.gitignore | 1 +
src/interfaces/libpq/test/Makefile | 7 +
.../postgres_fdw/expected/postgres_fdw.out | 2 +-
102 files changed, 4407 insertions(+), 130 deletions(-)
create mode 100644 src/test/column_encryption/.gitignore
create mode 100644 src/test/column_encryption/Makefile
create mode 100644 src/test/column_encryption/t/001_column_encryption.pl
create mode 100644 src/test/column_encryption/t/002_cmk_rotation.pl
create mode 100644 src/test/column_encryption/test_client.c
create mode 100755 src/test/column_encryption/test_run_decrypt.pl
create mode 100644 src/test/regress/expected/column_encryption.out
create mode 100644 src/test/regress/sql/column_encryption.sql
create mode 100644 src/include/catalog/pg_colenckey.h
create mode 100644 src/include/catalog/pg_colenckeydata.h
create mode 100644 src/include/catalog/pg_colmasterkey.h
create mode 100644 src/interfaces/libpq/fe-encrypt-openssl.c
create mode 100644 src/interfaces/libpq/fe-encrypt.h
diff --git a/doc/src/sgml/acronyms.sgml b/doc/src/sgml/acronyms.sgml
index 9ed148ab84..e21c9bcce3 100644
--- a/doc/src/sgml/acronyms.sgml
+++ b/doc/src/sgml/acronyms.sgml
@@ -56,6 +56,15 @@ <title>Acronyms</title>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><acronym>CEK</acronym></term>
+ <listitem>
+ <para>
+ Column Encryption Key
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><acronym>CIDR</acronym></term>
<listitem>
@@ -67,6 +76,15 @@ <title>Acronyms</title>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><acronym>CMK</acronym></term>
+ <listitem>
+ <para>
+ Column Master Key
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><acronym>CPAN</acronym></term>
<listitem>
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 4f3f375a84..c5e30f8029 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -105,6 +105,21 @@ <title>System Catalogs</title>
<entry>collations (locale information)</entry>
</row>
+ <row>
+ <entry><link linkend="catalog-pg-colenckey"><structname>pg_colenckey</structname></link></entry>
+ <entry>column encryption keys</entry>
+ </row>
+
+ <row>
+ <entry><link linkend="catalog-pg-colenckeydata"><structname>pg_colenckeydata</structname></link></entry>
+ <entry>column encryption key data</entry>
+ </row>
+
+ <row>
+ <entry><link linkend="catalog-pg-colmasterkey"><structname>pg_colmasterkey</structname></link></entry>
+ <entry>column master keys</entry>
+ </row>
+
<row>
<entry><link linkend="catalog-pg-constraint"><structname>pg_constraint</structname></link></entry>
<entry>check constraints, unique constraints, primary key constraints, foreign key constraints</entry>
@@ -1360,6 +1375,40 @@ <title><structname>pg_attribute</structname> Columns</title>
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>attcek</structfield> <type>oid</type>
+ (references <link linkend="catalog-pg-colenckey"><structname>pg_colenckey</structname></link>.<structfield>oid</structfield>)
+ </para>
+ <para>
+ If the column is encrypted, a reference to the column encryption key, else 0.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>attrealtypid</structfield> <type>oid</type>
+ (references <link linkend="catalog-pg-type"><structname>pg_type</structname></link>.<structfield>oid</structfield>)
+ </para>
+ <para>
+ If the column is encrypted, then this column indicates the type of the
+ encrypted data that is reported to the client. For encrypted columns,
+ the field <structfield>atttypid</structfield> is either
+ <type>pg_encrypted_det</type> or <type>pg_encrypted_rnd</type>. If the
+ column is not encrypted, then 0.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>attencalg</structfield> <type>int16</type>
+ </para>
+ <para>
+ If the column is encrypted, the identifier of the encryption algorithm;
+ see <xref linkend="protocol-cek"/> for possible values.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>attinhcount</structfield> <type>int4</type>
@@ -2437,6 +2486,232 @@ <title><structname>pg_collation</structname> Columns</title>
</para>
</sect1>
+ <sect1 id="catalog-pg-colenckey">
+ <title><structname>pg_colenckey</structname></title>
+
+ <indexterm zone="catalog-pg-colenckey">
+ <primary>pg_colenckey</primary>
+ </indexterm>
+
+ <para>
+ The catalog <structname>pg_colenckey</structname> contains information
+ about the column encryption keys in the database. The actual key material
+ of the column encryption keys is in the catalog <link
+ linkend="catalog-pg-colenckeydata"><structname>pg_colenckeydata</structname></link>.
+ </para>
+
+ <table>
+ <title><structname>pg_colenckey</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>oid</structfield> <type>oid</type>
+ </para>
+ <para>
+ Row identifier
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>cekname</structfield> <type>name</type>
+ </para>
+ <para>
+ Column encryption key name
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>cekowner</structfield> <type>oid</type>
+ (references <link linkend="catalog-pg-authid"><structname>pg_authid</structname></link>.<structfield>oid</structfield>)
+ </para>
+ <para>
+ Owner of the column encryption key
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="catalog-pg-colenckeydata">
+ <title><structname>pg_colenckeydata</structname></title>
+
+ <indexterm zone="catalog-pg-colenckeydata">
+ <primary>pg_colenckeydata</primary>
+ </indexterm>
+
+ <para>
+ The catalog <structname>pg_colenckeydata</structname> contains the key
+ material of column encryption keys. Each column encryption key object can
+ contain several versions of the key material, each encrypted with a
+ different column master key. That allows the gradual rotation of the
+ column master keys. Thus, <literal>(ckdcekid, ckdcmkid)</literal> is a
+ unique key of this table.
+ </para>
+
+ <para>
+ The key material of column encryption keys should never be decrypted inside
+ the database instance. It is meant to be sent as-is to the client, where
+ it is decrypted using the associated column master key, and then used to
+ encrypt or decrypt column values.
+ </para>
+
+ <table>
+ <title><structname>pg_colenckeydata</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>oid</structfield> <type>oid</type>
+ </para>
+ <para>
+ Row identifier
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>ckdcekid</structfield> <type>oid</type>
+ (references <link linkend="catalog-pg-colenckey"><structname>pg_colenckey</structname></link>.<structfield>oid</structfield>)
+ </para>
+ <para>
+ The column encryption key this entry belongs to
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>ckdcmkid</structfield> <type>oid</type>
+ (references <link linkend="catalog-pg-colmasterkey"><structname>pg_colmasterkey</structname></link>.<structfield>oid</structfield>)
+ </para>
+ <para>
+ The column master key that the key material is encrypted with
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>ckdcmkalg</structfield> <type>int16</type>
+ </para>
+ <para>
+ The encryption algorithm used for encrypting the key material; see
+ <xref linkend="protocol-cmk"/> for possible values.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>ckdencval</structfield> <type>bytea</type>
+ </para>
+ <para>
+ The key material of this column encryption key, encrypted using the
+ referenced column master key
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="catalog-pg-colmasterkey">
+ <title><structname>pg_colmasterkey</structname></title>
+
+ <indexterm zone="catalog-pg-colmasterkey">
+ <primary>pg_colmasterkey</primary>
+ </indexterm>
+
+ <para>
+ The catalog <structname>pg_colmasterkey</structname> contains information
+ about column master keys. The keys themselves are not stored in the
+ database. The catalog entry only contains information that is used by
+ clients to locate the keys, for example in a file or in a key management
+ system.
+ </para>
+
+ <table>
+ <title><structname>pg_colmasterkey</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>oid</structfield> <type>oid</type>
+ </para>
+ <para>
+ Row identifier
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>cmkname</structfield> <type>name</type>
+ </para>
+ <para>
+ Column master key name
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>cmkowner</structfield> <type>oid</type>
+ (references <link linkend="catalog-pg-authid"><structname>pg_authid</structname></link>.<structfield>oid</structfield>)
+ </para>
+ <para>
+ Owner of the column master key
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>cmkrealm</structfield> <type>text</type>
+ </para>
+ <para>
+ A <quote>realm</quote> associated with this column master key. This is
+ a freely chosen string that is used by clients to determine how to look
+ up the key. A typical configuration would put all CMKs that are looked
+ up in the same way into the same realm.
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
<sect1 id="catalog-pg-constraint">
<title><structname>pg_constraint</structname></title>
diff --git a/doc/src/sgml/datatype.sgml b/doc/src/sgml/datatype.sgml
index 8e30b82273..cdd0530eeb 100644
--- a/doc/src/sgml/datatype.sgml
+++ b/doc/src/sgml/datatype.sgml
@@ -5342,4 +5342,51 @@ <title>Pseudo-Types</title>
</sect1>
+ <sect1 id="datatype-encrypted">
+ <title>Types Related to Encryption</title>
+
+ <para>
+ An encrypted column value (see <xref linkend="ddl-column-encryption"/>) is
+ internally stored using the types
+ <type>pg_encrypted_rnd</type> (for randomized encryption) or
+ <type>pg_encrypted_det</type> (for deterministic encryption); see <xref
+ linkend="datatype-encrypted-table"/>. Most of the database system treats
+ this as normal types. For example, the type <type>pg_encrypted_det</type> has
+ an equals operator that allows lookup of encrypted values. The external
+ representation of these types is the same as the <type>bytea</type> type.
+ Thus, clients that don't support transparent column encryption or have
+ disabled it will see the encrypted values as byte arrays. Clients that
+ support transparent data encryption will not see these types in result
+ sets, as the protocol layer will translate them back to declared
+ underlying type in the table definition.
+ </para>
+
+ <table id="datatype-encrypted-table">
+ <title>Types Related to Encryption</title>
+ <tgroup cols="3">
+ <colspec colname="col1" colwidth="1*"/>
+ <colspec colname="col2" colwidth="3*"/>
+ <colspec colname="col3" colwidth="2*"/>
+ <thead>
+ <row>
+ <entry>Name</entry>
+ <entry>Storage Size</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry><type>pg_encrypted_det</type></entry>
+ <entry>1 or 4 bytes plus the actual binary string</entry>
+ <entry>encrypted column value, deterministic encryption</entry>
+ </row>
+ <row>
+ <entry><type>pg_encrypted_rnd</type></entry>
+ <entry>1 or 4 bytes plus the actual binary string</entry>
+ <entry>encrypted column value, randomized encryption</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
</chapter>
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index b01e3ad544..bb8503584e 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -1211,6 +1211,79 @@ <title>Exclusion Constraints</title>
</sect2>
</sect1>
+ <sect1 id="ddl-column-encryption">
+ <title>Transparent Column Encryption</title>
+
+ <para>
+ With <firstterm>transparent column encryption</firstterm>, columns can be
+ stored encrypted in the database. The encryption and decryption happens on
+ the client, so that the plaintext value is never seen in the database
+ instance or on the server hosting the database. The drawback is that most
+ operations, such as function calls or sorting, are not possible on
+ encrypted values.
+ </para>
+
+ <para>
+ Tranparent column encryption uses two levels of cryptographic keys. The
+ actual column value is encrypted using a symmetric algorithm, such as AES,
+ using a <firstterm>column encryption key</firstterm>
+ (<acronym>CEK</acronym>). The column encryption key is in turn encrypted
+ using an assymmetric algorithm, such as RSA, using a <firstterm>column
+ master key</firstterm> (<acronym>CMK</acronym>). The encrypted CEK is
+ stored in the database system. The CMK is not stored in the database
+ system; it is stored on the client or somewhere where the client can access
+ it, such as in a local file or in a key management system. The database
+ system only records where the CMK is stored and provides this information
+ to the client. When rows containing encrypted columns are sent to the
+ client, the server first sends any necessary CMK information, followed by
+ any required CEK. The client then looks up the CMK and uses that to
+ decrypt the CEK. Then it decrypts incoming row data using the CEK and
+ provides the decrypted row data to the application.
+ </para>
+
+ <para>
+ Here is an example declaring a column as encrypted:
+<programlisting>
+CREATE TABLE customers (
+ id int PRIMARY KEY,
+ name text NOT NULL,
+ ...
+ creditcard_num text <emphasis>ENCRYPTED WITH (column_encryption_key = cek1)</emphasis>
+);
+</programlisting>
+ </para>
+
+ <para>
+ Column encryption supports <firstterm>randomized</firstterm>
+ (also known as <firstterm>probabilistic</firstterm>) and
+ <firstterm>deterministic</firstterm> encryption. The above example uses
+ randomized encryption, which is the default. Randomized encryption uses a
+ random initialization vector for each encryption, so that even if the
+ plaintext of two rows is equal, the encrypted values will be different.
+ This prevents someone with direct access to the database server to make
+ computations such as distinct counts on the encrypted values.
+ Deterministic encryption uses a fixed initialization vector. This reduces
+ security, but it allows equality searches on encrypted values. The
+ following example declares a column with deterministic encryption:
+<programlisting>
+CREATE TABLE employees (
+ id int PRIMARY KEY,
+ name text NOT NULL,
+ ...
+ ssn text ENCRYPTED WITH (
+ column_encryption_key = cek1, <emphasis>encryption_type = deterministic</emphasis>)
+);
+</programlisting>
+ </para>
+
+ <para>
+ Null values are not encrypted by transparent column encryption; null values
+ sent by the client are visible as null values in the database. If the fact
+ that a value is null needs to be hidden from the server, this information
+ needs to be encoded into a nonnull value in the client somehow.
+ </para>
+ </sect1>
+
<sect1 id="ddl-system-columns">
<title>System Columns</title>
diff --git a/doc/src/sgml/glossary.sgml b/doc/src/sgml/glossary.sgml
index d6d0a3a814..10507e4391 100644
--- a/doc/src/sgml/glossary.sgml
+++ b/doc/src/sgml/glossary.sgml
@@ -353,6 +353,29 @@ <title>Glossary</title>
</glossdef>
</glossentry>
+ <glossentry id="glossary-column-encryption-key">
+ <glossterm>Column encryption key</glossterm>
+ <glossdef>
+ <para>
+ A cryptographic key used to encrypt column values when using transparent
+ column encryption. Column encryption keys are stored in the database
+ encrypted by another key, the column master key.
+ </para>
+ </glossdef>
+ </glossentry>
+
+ <glossentry id="glossary-column-master-key">
+ <glossterm>Column master key</glossterm>
+ <glossdef>
+ <para>
+ A cryptographic key used to encrypt column encryption keys. (So the
+ column master key is a <firstterm>key encryption key</firstterm>.)
+ Column master keys are stored outside the database system, for example in
+ a key management system.
+ </para>
+ </glossdef>
+ </glossentry>
+
<glossentry id="glossary-commit">
<glossterm>Commit</glossterm>
<glossdef>
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 74456aa69d..310aa49e37 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -1974,6 +1974,110 @@ <title>Parameter Key Words</title>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="libpq-connect-cmklookup" xreflabel="cmklookup">
+ <term><literal>cmklookup</literal></term>
+ <listitem>
+ <para>
+ This specifies how libpq should look up column master keys (CMKs) in
+ order to decrypt the column encryption keys (CEKs).
+ The value is a list of <literal>key=value</literal> entries separated
+ by semicolons. Each key is the name of a key realm, or
+ <literal>*</literal> to match all realms. The value is a
+ <literal>scheme:data</literal> specification. The scheme specifies
+ the method to look up the key, the remaining data is specific to the
+ scheme. Placeholders are replaced in the remaining data as follows:
+
+ <variablelist>
+ <varlistentry>
+ <term><literal>%a</literal></term>
+ <listitem>
+ <para>
+ The CMK algorithm name (see <xref linkend="protocol-cmk-table"/>)
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>%b</literal></term>
+ <listitem>
+ <para>
+ The encrypted CEK data in Base64 encoding (only for the <literal>run</literal> scheme)
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>%k</literal></term>
+ <listitem>
+ <para>
+ The CMK key name
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>%r</literal></term>
+ <listitem>
+ <para>
+ The realm name
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ Available schemes are:
+ <variablelist>
+ <varlistentry>
+ <term><literal>file</literal></term>
+ <listitem>
+ <para>
+ Load the key material from a file. The remaining data is the file
+ name. Use this if the CMKs are kept in a file on the file system.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>run</literal></term>
+ <listitem>
+ <para>
+ Run the specified command to decrypt the CEK. The remaining data
+ is a shell command. Use this with key management systems that
+ perform the decryption themselves. The command must print the
+ decrypted plaintext in Base64 format on the standard output.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ The default value is empty.
+ </para>
+
+ <para>
+ Example:
+<programlisting>
+cmklookup="r1=file:/some/where/secrets/%k.pem;*=file:/else/where/%r/%k.pem"
+</programlisting>
+ This specification says, for keys in realm <quote>r1</quote>, load
+ them from the specified file, replacing <literal>%k</literal> by the
+ key name. For keys in other realms, load them from the file,
+ replacing realm and key names as specified.
+ </para>
+
+ <para>
+ An example for interacting with a (hypothetical) key management
+ system:
+<programlisting>
+cmklookup="*=run:acmekms decrypt --key %k --alg %a --blob '%b'"
+</programlisting>
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
</para>
</sect2>
@@ -3022,6 +3126,57 @@ <title>Main Functions</title>
</listitem>
</varlistentry>
+ <varlistentry id="libpq-PQexecPrepared2">
+ <term><function>PQexecPrepared2</function><indexterm><primary>PQexecPrepared2</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Sends a request to execute a prepared statement with given
+ parameters, and waits for the result, with support for encrypted columns.
+<synopsis>
+PGresult *PQexecPrepared2(PGconn *conn,
+ const char *stmtName,
+ int nParams,
+ const char * const *paramValues,
+ const int *paramLengths,
+ const int *paramFormats,
+ const int *paramForceColumnEncryptions,
+ int resultFormat,
+ PGresult *paramDesc);
+</synopsis>
+ </para>
+
+ <para>
+ <xref linkend="libpq-PQexecPrepared2"/> is like <xref
+ linkend="libpq-PQexecPrepared"/> with additional support for encrypted
+ columns. The parameter <parameter>paramDesc</parameter> must be a
+ result set obtained from <xref linkend="libpq-PQdescribePrepared"/> on
+ the same prepared statement.
+ </para>
+
+ <para>
+ This function must be used if a statement parameter corresponds to an
+ underlying encrypted column. In that situation, the prepared
+ statement needs to be described first so that libpq can obtain the
+ necessary key and other information from the server. When that is
+ done, the parameters corresponding to encrypted columns are
+ automatically encrypted appropriately before being sent to the server.
+ </para>
+
+ <para>
+ The parameter <parameter>paramForceColumnEncryptions</parameter>
+ specifies whether encryption should be forced for a parameter. If
+ encryption is forced for a parameter but it does not correspond to an
+ encrypted column on the server, then the call will fail and the
+ parameter will not be sent. This can be used for additional security
+ against a comprimised server. (The drawback is that application code
+ then needs to be kept up to date with knowledge about which columns
+ are encrypted rather than letting the server specify this.) If the
+ array pointer is null then encryption is not forced for any parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQdescribePrepared">
<term><function>PQdescribePrepared</function><indexterm><primary>PQdescribePrepared</primary></indexterm></term>
@@ -3872,6 +4027,28 @@ <title>Retrieving Query Result Information</title>
</listitem>
</varlistentry>
+ <varlistentry id="libpq-PQfisencrypted">
+ <term><function>PQfisencrypted</function><indexterm><primary>PQfisencrypted</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Returns whether the value for the given column came from an encrypted
+ column. Column numbers start at 0.
+<synopsis>
+int PQfisencrypted(const PGresult *res,
+ int column_number);
+</synopsis>
+ </para>
+
+ <para>
+ Encrypted column values are automatically decrypted, so this function
+ is not necessary to access the column value. It can be used for extra
+ security to check whether the value was stored encrypted when one
+ thought it should be.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQfsize">
<term><function>PQfsize</function><indexterm><primary>PQfsize</primary></indexterm></term>
@@ -4047,12 +4224,37 @@ <title>Retrieving Query Result Information</title>
<para>
This function is only useful when inspecting the result of
- <xref linkend="libpq-PQdescribePrepared"/>. For other types of queries it
+ <xref linkend="libpq-PQdescribePrepared"/>. For other types of results it
will return zero.
</para>
</listitem>
</varlistentry>
+ <varlistentry id="libpq-PQparamisencrypted">
+ <term><function>PQparamisencrypted</function><indexterm><primary>PQparamisencrypted</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Returns whether the value for the given parameter is destined for an
+ encrypted column. Parameter numbers start at 0.
+<synopsis>
+int PQparamisencrypted(const PGresult *res, int param_number);
+</synopsis>
+ </para>
+
+ <para>
+ Values for parameters destined for encrypted columns are automatically
+ encrypted, so this function is not necessary to prepare the parameter
+ value. It can be used for extra security to check whether the value
+ will be stored encrypted when one thought it should be. (But see also
+ at <xref linkend="libpq-PQexecPrepared2"/> for another way to do that.)
+ This function is only useful when inspecting the result of <xref
+ linkend="libpq-PQdescribePrepared"/>. For other types of results it
+ will return false.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQprint">
<term><function>PQprint</function><indexterm><primary>PQprint</primary></indexterm></term>
@@ -4578,6 +4780,7 @@ <title>Asynchronous Command Processing</title>
<xref linkend="libpq-PQsendQueryParams"/>,
<xref linkend="libpq-PQsendPrepare"/>,
<xref linkend="libpq-PQsendQueryPrepared"/>,
+ <xref linkend="libpq-PQsendQueryPrepared2"/>,
<xref linkend="libpq-PQsendDescribePrepared"/>, and
<xref linkend="libpq-PQsendDescribePortal"/>,
which can be used with <xref linkend="libpq-PQgetResult"/> to duplicate
@@ -4585,6 +4788,7 @@ <title>Asynchronous Command Processing</title>
<xref linkend="libpq-PQexecParams"/>,
<xref linkend="libpq-PQprepare"/>,
<xref linkend="libpq-PQexecPrepared"/>,
+ <xref linkend="libpq-PQexecPrepared2"/>,
<xref linkend="libpq-PQdescribePrepared"/>, and
<xref linkend="libpq-PQdescribePortal"/>
respectively.
@@ -4696,6 +4900,46 @@ <title>Asynchronous Command Processing</title>
</listitem>
</varlistentry>
+ <varlistentry id="libpq-PQsendQueryPrepared2">
+ <term><function>PQsendQueryPrepared2</function><indexterm><primary>PQsendQueryPrepared2</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Sends a request to execute a prepared statement with given
+ parameters, without waiting for the result(s), with support for encrypted columns.
+<synopsis>
+int PQsendQueryPrepared2(PGconn *conn,
+ const char *stmtName,
+ int nParams,
+ const char * const *paramValues,
+ const int *paramLengths,
+ const int *paramFormats,
+ const int *paramForceColumnEncryptions,
+ int resultFormat,
+ PGresult *paramDesc);
+</synopsis>
+ </para>
+
+ <para>
+ <xref linkend="libpq-PQsendQueryPrepared2"/> is like <xref
+ linkend="libpq-PQsendQueryPrepared"/> with additional support for encrypted
+ columns. The parameter <parameter>paramDesc</parameter> must be a
+ result set obtained from <xref linkend="libpq-PQsendDescribePrepared"/> on
+ the same prepared statement.
+ </para>
+
+ <para>
+ This function must be used if a statement parameter corresponds to an
+ underlying encrypted column. In that situation, the prepared
+ statement needs to be described first so that libpq can obtain the
+ necessary key and other information from the server. When that is
+ done, the parameters corresponding to encrypted columns are
+ automatically encrypted appropriately before being sent to the server.
+ See also under <xref linkend="libpq-PQexecPrepared2"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQsendDescribePrepared">
<term><function>PQsendDescribePrepared</function><indexterm><primary>PQsendDescribePrepared</primary></indexterm></term>
@@ -4746,6 +4990,7 @@ <title>Asynchronous Command Processing</title>
<xref linkend="libpq-PQsendQueryParams"/>,
<xref linkend="libpq-PQsendPrepare"/>,
<xref linkend="libpq-PQsendQueryPrepared"/>,
+ <xref linkend="libpq-PQsendQueryPrepared2"/>,
<xref linkend="libpq-PQsendDescribePrepared"/>,
<xref linkend="libpq-PQsendDescribePortal"/>, or
<xref linkend="libpq-PQpipelineSync"/>
@@ -7776,6 +8021,16 @@ <title>Environment Variables</title>
</para>
</listitem>
+ <listitem>
+ <para>
+ <indexterm>
+ <primary><envar>PGCMKLOOKUP</envar></primary>
+ </indexterm>
+ <envar>PGCMKLOOKUP</envar> behaves the same as the <xref
+ linkend="libpq-connect-cmklookup"/> connection parameter.
+ </para>
+ </listitem>
+
<listitem>
<para>
<indexterm>
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index c0b89a3c01..60595c33b1 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -1050,6 +1050,52 @@ <title>Extended Query</title>
</note>
</sect2>
+ <sect2>
+ <title>Transparent Column Encryption</title>
+
+ <para>
+ When transparent column encryption is enabled, the messages
+ ColumnMasterKey and ColumnEncryptionKey can appear before RowDescription
+ and ParameterDescription messages. Clients should collect the information
+ in these messages and keep them for the duration of the connection. A
+ server is not required to resend the key information for each statement
+ cycle if it was already sent during this connection. If a server resends
+ a key that the client has already stored (that is, a key having an ID
+ equal to one already stored), the new information should replace the old.
+ (This could happen, for example, if the key was altered by server-side DDL
+ commands.)
+ </para>
+
+ <para>
+ A client supporting transparent column encryption should automatically
+ decrypt the column value fields of DataRow messages corresponding to
+ encrypted columns, and it should automatically encrypt the parameter value
+ fields of Bind messages corresponding to encrypted columns.
+ </para>
+
+ <para>
+ When column encryption is used, the plaintext is always in text format
+ (not binary format).
+ </para>
+
+ <para>
+ When deterministic encryption is used, clients need to take care to
+ represent plaintext to be encrypted in a consistent form. For example,
+ encrypting an integer represented by the string <literal>100</literal> and
+ an integer represented by the string <literal>+100</literal> would result
+ in two different ciphertexts, thus defeating the main point of
+ deterministic encryption. This protocol specification requires the
+ plaintext to be in <quote>canonical</quote> form, which is the form that
+ is produced by the server when it outputs a particular value in text
+ format.
+ </para>
+
+ <para>
+ The cryptographic operations used for transparent column encryption are
+ described in <xref linkend="protocol-transparent-column-encryption"/>.
+ </para>
+ </sect2>
+
<sect2>
<title>Function Call</title>
@@ -3991,6 +4037,128 @@ <title>Message Formats</title>
</listitem>
</varlistentry>
+ <varlistentry id="protocol-message-formats-ColumnEncryptionKey">
+ <term>ColumnEncryptionKey (B)</term>
+ <listitem>
+ <variablelist>
+ <varlistentry>
+ <term>Byte1('Y')</term>
+ <listitem>
+ <para>
+ Identifies the message as a column encryption key message.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Int32</term>
+ <listitem>
+ <para>
+ Length of message contents in bytes, including self.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Int32</term>
+ <listitem>
+ <para>
+ The session-specific identifier of the key.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Int32</term>
+ <listitem>
+ <para>
+ The identifier of the master key used to encrypt this key.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Int16</term>
+ <listitem>
+ <para>
+ The identifier of the algorithm used to encrypt this key.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Int32</term>
+ <listitem>
+ <para>
+ The length of the following key material.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Byte<replaceable>n</replaceable></term>
+ <listitem>
+ <para>
+ The key material, encrypted with the master key referenced above.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="protocol-message-formats-ColumnMasterKey">
+ <term>ColumnMasterKey (B)</term>
+ <listitem>
+ <variablelist>
+ <varlistentry>
+ <term>Byte1('y')</term>
+ <listitem>
+ <para>
+ Identifies the message as a column master key message.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Int32</term>
+ <listitem>
+ <para>
+ Length of message contents in bytes, including self.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Int32</term>
+ <listitem>
+ <para>
+ The session-specific identifier of the key.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>String</term>
+ <listitem>
+ <para>
+ The name of the key.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>String</term>
+ <listitem>
+ <para>
+ The key's realm.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="protocol-message-formats-CommandComplete">
<term>CommandComplete (B)</term>
<listitem>
@@ -5086,6 +5254,37 @@ <title>Message Formats</title>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term>Int32</term>
+ <listitem>
+ <para>
+ If this parameter is to be encrypted, this specifies the
+ identifier of the column encryption key to use, else zero.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Int16</term>
+ <listitem>
+ <para>
+ If this parameter is to be encrypted, this specifies the
+ identifier of the encryption algorithm, else zero.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Int16</term>
+ <listitem>
+ <para>
+ This is used as a bit field of flags. If the column is
+ encrypted and bit 0x01 is set, the column uses deterministic
+ encryption, otherwise randomized encryption.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
</listitem>
</varlistentry>
@@ -5474,6 +5673,26 @@ <title>Message Formats</title>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term>Int32</term>
+ <listitem>
+ <para>
+ If this parameter is to be encrypted, this specifies the
+ identifier of the column encryption key to use, else zero.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Int16</term>
+ <listitem>
+ <para>
+ If this parameter is to be encrypted, this specifies the
+ identifier of the encrypt algorithm, else zero.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
</listitem>
</varlistentry>
@@ -7278,6 +7497,133 @@ <title>Logical Replication Message Formats</title>
</variablelist>
</sect1>
+ <sect1 id="protocol-transparent-column-encryption">
+ <title>Transparent Column Encryption Cryptography</title>
+
+ <para>
+ This section describes the cryptographic operations used by transparent
+ column encryption. A client that supports transparent column encryption
+ needs to implement these operations as specified here in order to be able
+ to interoperate with other clients.
+ </para>
+
+ <para>
+ Column encryption key algorithms and column master key algorithms are
+ identified by integers in the protocol messages and the system catalogs.
+ Additional algorithms may be added to this protocol specification without a
+ change in the protocol version number. Clients should implement support
+ for all the algorithms specified here. If a client encounters an algorithm
+ identifier it does not recognize or does not support, it must raise an
+ error. A suitable error message should be provided to the application or
+ user.
+ </para>
+
+ <sect2 id="protocol-cmk">
+ <title>Column Master Keys</title>
+
+ <para>
+ The currently defined algorithms for column master keys are listed in
+ <xref linkend="protocol-cmk-table"/>.
+ </para>
+
+ <table id="protocol-cmk-table">
+ <title>Column Master Key Algorithms</title>
+ <tgroup cols="3">
+ <thead>
+ <row>
+ <entry>ID</entry>
+ <entry>Name</entry>
+ <entry>Reference</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>1</entry>
+ <entry><literal>RSAES_OAEP_SHA_1</literal></entry>
+ <entry><ulink url="https://datatracker.ietf.org/doc/html/rfc8017">RFC 8017</ulink> (PKCS #1)</entry>
+ </row>
+ <row>
+ <entry>2</entry>
+ <entry><literal>RSAES_OAEP_SHA_256</literal></entry>
+ <entry><ulink url="https://datatracker.ietf.org/doc/html/rfc8017">RFC 8017</ulink> (PKCS #1)</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+
+ <sect2 id="protocol-cek">
+ <title>Column Encryption Keys</title>
+
+ <para>
+ The currently defined algorithms for column encryption keys are listed in
+ <xref linkend="protocol-cek-table"/>.
+ </para>
+
+ <table id="protocol-cek-table">
+ <title>Column Encryption Key Algorithms</title>
+ <tgroup cols="3">
+ <thead>
+ <row>
+ <entry>ID</entry>
+ <entry>Name</entry>
+ <entry>Reference</entry>
+ <entry>Key length (octets)</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>130</entry>
+ <entry><literal>AEAD_AES_128_CBC_HMAC_SHA_256</literal></entry>
+ <entry><ulink url="https://datatracker.ietf.org/doc/html/draft-mcgrew-aead-aes-cbc-hmac-sha2-05"/></entry;
+ <entry>32</entry>
+ </row>
+ <row>
+ <entry>131</entry>
+ <entry><literal>AEAD_AES_192_CBC_HMAC_SHA_384</literal></entry>
+ <entry><ulink url="https://datatracker.ietf.org/doc/html/draft-mcgrew-aead-aes-cbc-hmac-sha2-05"/></entry;
+ <entry>48</entry>
+ </row>
+ <row>
+ <entry>132</entry>
+ <entry><literal>AEAD_AES_256_CBC_HMAC_SHA_384</literal></entry>
+ <entry><ulink url="https://datatracker.ietf.org/doc/html/draft-mcgrew-aead-aes-cbc-hmac-sha2-05"/></entry;
+ <entry>56</entry>
+ </row>
+ <row>
+ <entry>133</entry>
+ <entry><literal>AEAD_AES_256_CBC_HMAC_SHA_512</literal></entry>
+ <entry><ulink url="https://datatracker.ietf.org/doc/html/draft-mcgrew-aead-aes-cbc-hmac-sha2-05"/></entry;
+ <entry>64</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ The <quote>associated data</quote> in these algorithms consists of 4
+ bytes: The ASCII letters <literal>P</literal> and <literal>G</literal>
+ (byte values 80 and 71), followed by the algorithm ID as a 16-bit unsigned
+ integer in network byte order.
+ </para>
+
+ <para>
+ The length of the initialization vector is 16 octets for all CEK algorithm
+ variants. For randomized encryption, the initialization vector should be
+ (cryptographically strong) random bytes. For deterministic encryption,
+ the initialization vector is constructed as
+<programlisting>
+SUBSTRING(<replaceable>HMAC</replaceable>(<replaceable>K</replaceable>, <replaceable>P</replaceable>) FOR <replaceable>IVLEN</replaceable>)
+</programlisting>
+ where <replaceable>HMAC</replaceable> is the HMAC function associated with
+ the algorithm, <replaceable>K</replaceable> is the prefix of the CEK of
+ the length required by the HMAC function, and <replaceable>P</replaceable>
+ is the plaintext to be encrypted. (This is the same portion of the CEK
+ that the AEAD_AES_*_HMAC_* algorithms use for the MAC part.)
+ </para>
+ </sect2>
+ </sect1>
+
<sect1 id="protocol-changes">
<title>Summary of Changes since Protocol 2.0</title>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 6c9918b0a1..7a1d03f868 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -22,7 +22,7 @@
<refsynopsisdiv>
<synopsis>
CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] <replaceable class="parameter">table_name</replaceable> ( [
- { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COMPRESSION <replaceable>compression_method</replaceable> ] [ COLLATE <replaceable>collation</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
+ { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COMPRESSION <replaceable>compression_method</replaceable> ] [ ENCRYPTED WITH ( <replaceable>encryption_options</replaceable> ) ] [ COLLATE <replaceable>collation</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
| <replaceable>table_constraint</replaceable>
| LIKE <replaceable>source_table</replaceable> [ <replaceable>like_option</replaceable> ... ] }
[, ... ]
@@ -322,6 +322,46 @@ <title>Parameters</title>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ENCRYPTED WITH ( <replaceable>encryption_options</replaceable> )</literal></term>
+ <listitem>
+ <para>
+ Enables transparent column encryption for the column.
+ <replaceable>encryption_options</replaceable> are comma-separated
+ <literal>key=value</literal> specifications. The following options are
+ available:
+ <variablelist>
+ <varlistentry>
+ <term><literal>column_encryption_key</literal></term>
+ <listitem>
+ <para>
+ Specifies the name of the column encryption key to use. Specifying
+ this is mandatory.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>encryption_type</literal></term>
+ <listitem>
+ <para>
+ <literal>randomized</literal> (the default) or <literal>deterministic</literal>
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>algorithm</literal></term>
+ <listitem>
+ <para>
+ The encryption algorithm to use. The default is
+ <literal>AEAD_AES_128_CBC_HMAC_SHA_256</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>INHERITS ( <replaceable>parent_table</replaceable> [, ... ] )</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 65bb0a6a3f..e6c93f1b13 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -2255,6 +2255,28 @@ <title>Meta-Commands</title>
</varlistentry>
+ <varlistentry>
+ <term><literal>\gencr</literal> [ <replaceable class="parameter">parameter</replaceable> ] ... </term>
+
+ <listitem>
+ <para>
+ Sends the current query buffer to the server for execution, as with
+ <literal>\g</literal>, with the specified parameters passed for any
+ parameter placeholders (<literal>$1</literal> etc.). This command
+ ensures that any parameters corresponding to encrypted columns are
+ sent to the server encrypted.
+ </para>
+
+ <para>
+ Example:
+<programlisting>
+INSERT INTO tbl1 VALUES ($1, $2) \gencr 'first value' 'second value'
+</programlisting>
+ </para>
+ </listitem>
+ </varlistentry>
+
+
<varlistentry>
<term><literal>\getenv <replaceable class="parameter">psql_var</replaceable> <replaceable class="parameter">env_var</replaceable></literal></term>
@@ -3945,6 +3967,17 @@ <title>Variables</title>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><varname>HIDE_COLUMN_ENCRYPTION</varname></term>
+ <listitem>
+ <para>
+ If this variable is set to <literal>true</literal>, column encryption
+ details are not displayed. This is mainly useful for regression
+ tests.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><varname>HIDE_TOAST_COMPRESSION</varname></term>
<listitem>
diff --git a/src/test/Makefile b/src/test/Makefile
index 69ef074d75..2300be8332 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -32,6 +32,7 @@ SUBDIRS += ldap
endif
endif
ifeq ($(with_ssl),openssl)
+SUBDIRS += column_encryption
ifneq (,$(filter ssl,$(PG_TEST_EXTRA)))
SUBDIRS += ssl
endif
@@ -41,7 +42,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos icu ldap ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos icu ldap ssl column_encryption)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/column_encryption/.gitignore b/src/test/column_encryption/.gitignore
new file mode 100644
index 0000000000..456dbf69d2
--- /dev/null
+++ b/src/test/column_encryption/.gitignore
@@ -0,0 +1,3 @@
+/test_client
+# Generated by test suite
+/tmp_check/
diff --git a/src/test/column_encryption/Makefile b/src/test/column_encryption/Makefile
new file mode 100644
index 0000000000..7aca84b17a
--- /dev/null
+++ b/src/test/column_encryption/Makefile
@@ -0,0 +1,29 @@
+#-------------------------------------------------------------------------
+#
+# Makefile for src/test/column_encryption
+#
+# Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/test/column_encryption/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/test/column_encryption
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
+
+all: test_client
+
+check: all
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
+
+clean distclean maintainer-clean:
+ rm -f test_client.o test_client
+ rm -rf tmp_check
diff --git a/src/test/column_encryption/t/001_column_encryption.pl b/src/test/column_encryption/t/001_column_encryption.pl
new file mode 100644
index 0000000000..7ab98981d8
--- /dev/null
+++ b/src/test/column_encryption/t/001_column_encryption.pl
@@ -0,0 +1,206 @@
+# Copyright (c) 2021-2022, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $node = PostgreSQL::Test::Cluster->new('node');
+$node->init;
+$node->start;
+
+
+sub create_cmk
+{
+ my ($cmkname) = @_;
+
+ my $cmkfilename = "${PostgreSQL::Test::Utils::tmp_check}/${cmkname}.pem";
+
+ system_or_bail 'openssl', 'genpkey', '-algorithm', 'rsa', '-out', $cmkfilename;
+
+ $node->safe_psql('postgres', qq{
+INSERT INTO pg_colmasterkey (oid, cmkname, cmkowner, cmkrealm) VALUES (
+ pg_nextoid('pg_catalog.pg_colmasterkey', 'oid', 'pg_catalog.pg_colmasterkey_oid_index'),
+ '${cmkname}',
+ (select oid from pg_roles where rolname = current_user),
+ ''
+);
+});
+
+ return $cmkfilename;
+}
+
+sub create_cek
+{
+ my ($cekname, $bytes, $cmkname, $cmkfilename) = @_;
+
+ # generate random bytes
+ system_or_bail 'openssl', 'rand', '-out', "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin", $bytes;
+
+ # encrypt CEK using CMK
+ system_or_bail 'openssl', 'pkeyutl', '-encrypt',
+ '-inkey', $cmkfilename,
+ '-pkeyopt', 'rsa_padding_mode:oaep',
+ '-in', "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin",
+ '-out', "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin.enc";
+
+ my $cekenchex = unpack('H*', slurp_file "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin.enc");
+
+ # create CEK in database
+ $node->safe_psql('postgres', qq{
+INSERT INTO pg_colenckey (oid, cekname, cekowner) VALUES (
+ pg_nextoid('pg_catalog.pg_colenckey', 'oid', 'pg_catalog.pg_colenckey_oid_index'),
+ '${cekname}',
+ (select oid from pg_roles where rolname = current_user)
+);
+});
+ $node->safe_psql('postgres', qq{
+INSERT INTO pg_colenckeydata (oid, ckdcekid, ckdcmkid, ckdcmkalg, ckdencval) VALUES (
+ pg_nextoid('pg_catalog.pg_colenckeydata', 'oid', 'pg_catalog.pg_colenckeydata_oid_index'),
+ (SELECT oid FROM pg_colenckey WHERE cekname = '${cekname}'),
+ (SELECT oid FROM pg_colmasterkey WHERE cmkname = '${cmkname}'),
+ 1,
+ '\\x${cekenchex}'
+);
+});
+
+ return;
+}
+
+
+my $cmk1filename = create_cmk('cmk1');
+my $cmk2filename = create_cmk('cmk2');
+create_cek('cek1', 32, 'cmk1', $cmk1filename);
+create_cek('cek2', 48, 'cmk2', $cmk2filename);
+
+$ENV{'PGCMKLOOKUP'} = '*=file:' . ${PostgreSQL::Test::Utils::tmp_check} . '/%k.pem';
+
+
+$node->safe_psql('postgres', qq{
+CREATE TABLE tbl1 (
+ a int,
+ b text ENCRYPTED WITH (column_encryption_key = cek1)
+);
+});
+
+$node->safe_psql('postgres', qq{
+CREATE TABLE tbl2 (
+ a int,
+ b text ENCRYPTED WITH (encryption_type = deterministic, column_encryption_key = cek1)
+);
+});
+
+$node->safe_psql('postgres', qq{
+CREATE TABLE tbl3 (
+ a int,
+ b text ENCRYPTED WITH (column_encryption_key = cek1),
+ c text ENCRYPTED WITH (column_encryption_key = cek2, algorithm = 'AEAD_AES_192_CBC_HMAC_SHA_384')
+);
+});
+
+$node->safe_psql('postgres', q{
+INSERT INTO tbl1 (a, b) VALUES (1, $1) \gencr 'val1'
+INSERT INTO tbl1 (a, b) VALUES (2, $1) \gencr 'val2'
+});
+
+# Expected ciphertext length is 2 blocks of AES output (2 * 16) plus
+# half SHA-256 output (16) in hex encoding: (2 * 16 + 16) * 2 = 96
+like($node->safe_psql('postgres', q{COPY (SELECT * FROM tbl1) TO STDOUT}),
+ qr/1\t\\\\x[0-9a-f]{96}\n2\t\\\\x[0-9a-f]{96}/,
+ 'inserted data is encrypted');
+
+my $result;
+
+$result = $node->safe_psql('postgres', q{SELECT a, b FROM tbl1 \gdesc});
+is($result,
+ q(a|integer
+b|text),
+ 'query result description has original type');
+
+$result = $node->safe_psql('postgres', q{SELECT a, b FROM tbl1});
+is($result,
+ q(1|val1
+2|val2),
+ 'decrypted query result');
+
+{
+ local $ENV{'PGCMKLOOKUP'} = '*=run:broken %k "%b"';
+ $result = $node->psql('postgres', q{SELECT a, b FROM tbl1});
+ isnt($result, 0, 'query fails with broken cmklookup run setting');
+}
+
+{
+ local $ENV{'PGCMKLOOKUP'} = "*=run:perl ./test_run_decrypt.pl '${PostgreSQL::Test::Utils::tmp_check}' %k %a '%b'";
+
+ $result = $node->safe_psql('postgres', q{SELECT a, b FROM tbl1});
+ is($result,
+ q(1|val1
+2|val2),
+ 'decrypted query result with cmklookup run');
+}
+
+$node->safe_psql('postgres', q{
+INSERT INTO tbl3 (a, b, c) VALUES (1, $1, $2) \gencr 'valB1' 'valC1'
+});
+
+$result = $node->safe_psql('postgres', q{SELECT a, b, c FROM tbl3});
+is($result,
+ q(1|valB1|valC1),
+ 'decrypted query result multiple keys');
+
+
+$node->command_fails_like(['test_client', 'test1'], qr/not encrypted/, 'test client fails because parameters not encrypted');
+
+$result = $node->safe_psql('postgres', q{SELECT a, b FROM tbl1});
+is($result,
+ q(1|val1
+2|val2),
+ 'decrypted query result after test client insert');
+
+$node->command_ok(['test_client', 'test2'], 'test client test 2');
+
+$result = $node->safe_psql('postgres', q{SELECT a, b FROM tbl1});
+is($result,
+ q(1|val1
+2|val2
+3|val3),
+ 'decrypted query result after test client insert 2');
+
+like($node->safe_psql('postgres', q{COPY (SELECT * FROM tbl1 WHERE a = 3) TO STDOUT}),
+ qr/3\t\\\\x[0-9a-f]{96}/,
+ 'inserted data is encrypted');
+
+$node->command_ok(['test_client', 'test3'], 'test client test 3');
+
+$result = $node->safe_psql('postgres', q{SELECT a, b FROM tbl1});
+is($result,
+ q(1|val1
+2|val2
+3|val3upd),
+ 'decrypted query result after test client insert 3');
+
+$node->command_ok(['test_client', 'test4'], 'test client test 4');
+
+$result = $node->safe_psql('postgres', q{SELECT a, b FROM tbl2});
+is($result,
+ q(1|valA
+2|valB
+3|valA),
+ 'decrypted query result after test client insert 4');
+
+is($node->safe_psql('postgres', q{SELECT b, count(*) FROM tbl2 GROUP BY b ORDER BY 2}),
+ q(valB|1
+valA|2),
+ 'group by deterministically encrypted column');
+
+$node->command_ok(['test_client', 'test5'], 'test client test 5');
+
+$result = $node->safe_psql('postgres', q{SELECT a, b, c FROM tbl3});
+is($result,
+ q(1|valB1|valC1
+2|valB2|valC2
+3|valB3|valC3),
+ 'decrypted query result multiple keys after test client insert 5');
+
+done_testing();
diff --git a/src/test/column_encryption/t/002_cmk_rotation.pl b/src/test/column_encryption/t/002_cmk_rotation.pl
new file mode 100644
index 0000000000..5a13a206eb
--- /dev/null
+++ b/src/test/column_encryption/t/002_cmk_rotation.pl
@@ -0,0 +1,144 @@
+# Copyright (c) 2021-2022, PostgreSQL Global Development Group
+
+# Test column master key rotation. First, we generate CMK1 and a CEK
+# encrypted with it. Then we add a CMK2 and encrypt the CEK with it
+# as well. (Recall that a CEK can be associated with multiple CMKs,
+# for this reason. That's why pg_colenckeydata is split out from
+# pg_colenckey.) Then we remove CMK1. We test that we can get
+# decrypted query results at each step.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $node = PostgreSQL::Test::Cluster->new('node');
+$node->init;
+$node->start;
+
+
+sub create_cmk
+{
+ my ($cmkname) = @_;
+
+ my $cmkfilename = "${PostgreSQL::Test::Utils::tmp_check}/${cmkname}.pem";
+
+ system_or_bail 'openssl', 'genpkey', '-algorithm', 'rsa', '-out', $cmkfilename;
+
+ $node->safe_psql('postgres', qq{
+INSERT INTO pg_colmasterkey (oid, cmkname, cmkowner, cmkrealm) VALUES (
+ pg_nextoid('pg_catalog.pg_colmasterkey', 'oid', 'pg_catalog.pg_colmasterkey_oid_index'),
+ '${cmkname}',
+ (select oid from pg_roles where rolname = current_user),
+ ''
+);
+});
+
+ return $cmkfilename;
+}
+
+
+my $cmk1filename = create_cmk('cmk1');
+
+# create CEK
+my ($cekname, $bytes) = ('cek1', 16+16);
+
+# generate random bytes
+system_or_bail 'openssl', 'rand', '-out', "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin", $bytes;
+
+# encrypt CEK using CMK
+system_or_bail 'openssl', 'pkeyutl', '-encrypt',
+ '-inkey', $cmk1filename,
+ '-pkeyopt', 'rsa_padding_mode:oaep',
+ '-in', "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin",
+ '-out', "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin.enc";
+
+my $cekenchex = unpack('H*', slurp_file "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin.enc");
+
+# create CEK in database
+$node->safe_psql('postgres', qq{
+INSERT INTO pg_colenckey (oid, cekname, cekowner) VALUES (
+ pg_nextoid('pg_catalog.pg_colenckey', 'oid', 'pg_catalog.pg_colenckey_oid_index'),
+ '${cekname}',
+ (select oid from pg_roles where rolname = current_user)
+);
+});
+
+$node->safe_psql('postgres', qq{
+INSERT INTO pg_colenckeydata (oid, ckdcekid, ckdcmkid, ckdcmkalg, ckdencval) VALUES (
+ pg_nextoid('pg_catalog.pg_colenckeydata', 'oid', 'pg_catalog.pg_colenckeydata_oid_index'),
+ (SELECT oid FROM pg_colenckey WHERE cekname = '${cekname}'),
+ (SELECT oid FROM pg_colmasterkey WHERE cmkname = 'cmk1'),
+ 1,
+ '\\x${cekenchex}'
+);
+});
+
+$ENV{'PGCMKLOOKUP'} = '*=file:' . ${PostgreSQL::Test::Utils::tmp_check} . '/%k.pem';
+
+$node->safe_psql('postgres', qq{
+CREATE TABLE tbl1 (
+ a int,
+ b text ENCRYPTED WITH (column_encryption_key = cek1)
+);
+});
+
+$node->safe_psql('postgres', q{
+INSERT INTO tbl1 (a, b) VALUES (1, $1) \gencr 'val1'
+INSERT INTO tbl1 (a, b) VALUES (2, $1) \gencr 'val2'
+});
+
+is($node->safe_psql('postgres', q{SELECT a, b FROM tbl1}),
+ q(1|val1
+2|val2),
+ 'decrypted query result with one CMK');
+
+
+# create new CMK
+my $cmk2filename = create_cmk('cmk2');
+
+# encrypt CEK using new CMK
+#
+# (Here, we still have the plaintext of the CEK available from
+# earlier. In reality, one would decrypt the CEK with the first CMK
+# and then re-encrypt it with the second CMK.)
+system_or_bail 'openssl', 'pkeyutl', '-encrypt',
+ '-inkey', $cmk2filename,
+ '-pkeyopt', 'rsa_padding_mode:oaep',
+ '-in', "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin",
+ '-out', "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin.enc";
+
+$cekenchex = unpack('H*', slurp_file "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin.enc");
+
+# add new data record for CEK in database
+$node->safe_psql('postgres', qq{
+INSERT INTO pg_colenckeydata (oid, ckdcekid, ckdcmkid, ckdcmkalg, ckdencval) VALUES (
+ pg_nextoid('pg_catalog.pg_colenckeydata', 'oid', 'pg_catalog.pg_colenckeydata_oid_index'),
+ (SELECT oid FROM pg_colenckey WHERE cekname = '${cekname}'),
+ (SELECT oid FROM pg_colmasterkey WHERE cmkname = 'cmk2'),
+ 1,
+ '\\x${cekenchex}'
+);
+});
+
+
+is($node->safe_psql('postgres', q{SELECT a, b FROM tbl1}),
+ q(1|val1
+2|val2),
+ 'decrypted query result with two CMKs');
+
+
+# delete CEK record for first CMK
+$node->safe_psql('postgres', qq{
+DELETE FROM pg_colenckeydata WHERE ckdcmkid = (SELECT oid FROM pg_colmasterkey WHERE cmkname = 'cmk1');
+});
+
+
+is($node->safe_psql('postgres', q{SELECT a, b FROM tbl1}),
+ q(1|val1
+2|val2),
+ 'decrypted query result with only new CMK');
+
+
+done_testing();
diff --git a/src/test/column_encryption/test_client.c b/src/test/column_encryption/test_client.c
new file mode 100644
index 0000000000..add9fe40a6
--- /dev/null
+++ b/src/test/column_encryption/test_client.c
@@ -0,0 +1,210 @@
+/*
+ * Copyright (c) 2021-2022, PostgreSQL Global Development Group
+ */
+
+#include "postgres_fe.h"
+
+#include "libpq-fe.h"
+
+
+static int
+test1(PGconn *conn)
+{
+ PGresult *res;
+ const char *values[] = {"3", "val3"};
+
+ res = PQexecParams(conn, "INSERT INTO tbl1 (a, b) VALUES ($1, $2)",
+ 2, NULL, values, NULL, NULL, 0);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQexecParams() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ return 0;
+}
+
+static int
+test2(PGconn *conn)
+{
+ PGresult *res,
+ *res2;
+ const char *values[] = {"3", "val3"};
+ int forces[] = {false, true};
+
+ res = PQprepare(conn, "", "INSERT INTO tbl1 (a, b) VALUES ($1, $2)",
+ 2, NULL);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQprepare() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ res2 = PQdescribePrepared(conn, "");
+ if (PQresultStatus(res2) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQdescribePrepared() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ if (!(!PQparamisencrypted(res2, 0) &&
+ PQparamisencrypted(res2, 1)))
+ {
+ fprintf(stderr, "wrong results from PQparamisencrypted()\n");
+ return 1;
+ }
+
+ res = PQexecPrepared2(conn, "", 2, values, NULL, NULL, forces, 0, res2);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQexecPrepared() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ return 0;
+}
+
+static int
+test3(PGconn *conn)
+{
+ PGresult *res,
+ *res2;
+ const char *values[] = {"3", "val3upd"};
+
+ res = PQprepare(conn, "", "UPDATE tbl1 SET b = $2 WHERE a = $1",
+ 2, NULL);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQprepare() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ res2 = PQdescribePrepared(conn, "");
+ if (PQresultStatus(res2) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQdescribePrepared() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ res = PQexecPrepared2(conn, "", 2, values, NULL, NULL, NULL, 0, res2);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQexecPrepared() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ return 0;
+}
+
+static int
+test4(PGconn *conn)
+{
+ PGresult *res,
+ *res2;
+ const char *values[] = {"1", "valA", "2", "valB", "3", "valA"};
+
+ res = PQprepare(conn, "", "INSERT INTO tbl2 (a, b) VALUES ($1, $2), ($3, $4), ($5, $6)",
+ 6, NULL);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQprepare() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ res2 = PQdescribePrepared(conn, "");
+ if (PQresultStatus(res2) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQdescribePrepared() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ res = PQexecPrepared2(conn, "", 6, values, NULL, NULL, NULL, 0, res2);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQexecPrepared() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ return 0;
+}
+
+static int
+test5(PGconn *conn)
+{
+ PGresult *res,
+ *res2;
+ const char *values[] = {
+ "2", "valB2", "valC2",
+ "3", "valB3", "valC3"
+ };
+
+ res = PQprepare(conn, "", "INSERT INTO tbl3 (a, b, c) VALUES ($1, $2, $3), ($4, $5, $6)",
+ 6, NULL);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQprepare() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ res2 = PQdescribePrepared(conn, "");
+ if (PQresultStatus(res2) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQdescribePrepared() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ res = PQexecPrepared2(conn, "", 6, values, NULL, NULL, NULL, 0, res2);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQexecPrepared() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ return 0;
+}
+
+int
+main(int argc, char **argv)
+{
+ PGconn *conn;
+ int ret = 0;
+
+ conn = PQconnectdb("");
+ if (PQstatus(conn) != CONNECTION_OK)
+ {
+ fprintf(stderr, "Connection to database failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ if (argc < 2 || argv[1] == NULL)
+ return 87;
+ else if (strcmp(argv[1], "test1") == 0)
+ ret = test1(conn);
+ else if (strcmp(argv[1], "test2") == 0)
+ ret = test2(conn);
+ else if (strcmp(argv[1], "test3") == 0)
+ ret = test3(conn);
+ else if (strcmp(argv[1], "test4") == 0)
+ ret = test4(conn);
+ else if (strcmp(argv[1], "test5") == 0)
+ ret = test5(conn);
+ else
+ ret = 88;
+
+ PQfinish(conn);
+ return ret;
+}
diff --git a/src/test/column_encryption/test_run_decrypt.pl b/src/test/column_encryption/test_run_decrypt.pl
new file mode 100755
index 0000000000..9664349f14
--- /dev/null
+++ b/src/test/column_encryption/test_run_decrypt.pl
@@ -0,0 +1,41 @@
+#!/usr/bin/perl
+
+# Test/sample command for libpq cmklookup run scheme
+#
+# This just places the data into temporary files and runs the openssl
+# command on it. (In practice, this could more simply be written as a
+# shell script, but this way it's more portable.)
+
+# Copyright (c) 2021-2022, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use MIME::Base64;
+
+my ($tmpdir, $cmkname, $alg, $b64data) = @ARGV;
+
+die unless $alg eq 'RSAES_OAEP_SHA_1';
+
+open my $fh, '>:raw', "${tmpdir}/input.tmp" or die;
+print $fh decode_base64($b64data);
+close $fh;
+
+system('openssl', 'pkeyutl', '-decrypt',
+ '-inkey', "${tmpdir}/${cmkname}.pem", '-pkeyopt', 'rsa_padding_mode:oaep',
+ '-in', "${tmpdir}/input.tmp", '-out', "${tmpdir}/output.tmp") == 0 or die;
+
+open $fh, '<:raw', "${tmpdir}/output.tmp" or die;
+my $data = '';
+
+while (1) {
+ my $success = read $fh, $data, 100, length($data);
+ die $! if not defined $success;
+ last if not $success;
+}
+
+close $fh;
+
+unlink "${tmpdir}/input.tmp", "${tmpdir}/output.tmp";
+
+print encode_base64($data);
diff --git a/src/test/modules/libpq_pipeline/traces/disallowed_in_pipeline.trace b/src/test/modules/libpq_pipeline/traces/disallowed_in_pipeline.trace
index dd6df03f1e..5fc9c0346d 100644
--- a/src/test/modules/libpq_pipeline/traces/disallowed_in_pipeline.trace
+++ b/src/test/modules/libpq_pipeline/traces/disallowed_in_pipeline.trace
@@ -1,5 +1,5 @@
F 13 Query "SELECT 1"
-B 33 RowDescription 1 "?column?" NNNN 0 NNNN 4 -1 0
+B 39 RowDescription 1 "?column?" NNNN 0 NNNN 4 -1 0 NNNN 0
B 11 DataRow 1 1 '1'
B 13 CommandComplete "SELECT 1"
B 5 ReadyForQuery I
diff --git a/src/test/modules/libpq_pipeline/traces/multi_pipelines.trace b/src/test/modules/libpq_pipeline/traces/multi_pipelines.trace
index 4b9ab07ca4..700b0b6519 100644
--- a/src/test/modules/libpq_pipeline/traces/multi_pipelines.trace
+++ b/src/test/modules/libpq_pipeline/traces/multi_pipelines.trace
@@ -10,13 +10,13 @@ F 9 Execute "" 0
F 4 Sync
B 4 ParseComplete
B 4 BindComplete
-B 33 RowDescription 1 "?column?" NNNN 0 NNNN 4 -1 0
+B 39 RowDescription 1 "?column?" NNNN 0 NNNN 4 -1 0 NNNN 0
B 11 DataRow 1 1 '1'
B 13 CommandComplete "SELECT 1"
B 5 ReadyForQuery I
B 4 ParseComplete
B 4 BindComplete
-B 33 RowDescription 1 "?column?" NNNN 0 NNNN 4 -1 0
+B 39 RowDescription 1 "?column?" NNNN 0 NNNN 4 -1 0 NNNN 0
B 11 DataRow 1 1 '1'
B 13 CommandComplete "SELECT 1"
B 5 ReadyForQuery I
diff --git a/src/test/modules/libpq_pipeline/traces/nosync.trace b/src/test/modules/libpq_pipeline/traces/nosync.trace
index d99aac649d..7b1dfa1c15 100644
--- a/src/test/modules/libpq_pipeline/traces/nosync.trace
+++ b/src/test/modules/libpq_pipeline/traces/nosync.trace
@@ -41,52 +41,52 @@ F 9 Execute "" 0
F 4 Flush
B 4 ParseComplete
B 4 BindComplete
-B 31 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0
+B 37 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0 NNNN 0
B 70 DataRow 1 60 'xyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxz'
B 13 CommandComplete "SELECT 1"
B 4 ParseComplete
B 4 BindComplete
-B 31 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0
+B 37 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0 NNNN 0
B 70 DataRow 1 60 'xyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxz'
B 13 CommandComplete "SELECT 1"
B 4 ParseComplete
B 4 BindComplete
-B 31 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0
+B 37 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0 NNNN 0
B 70 DataRow 1 60 'xyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxz'
B 13 CommandComplete "SELECT 1"
B 4 ParseComplete
B 4 BindComplete
-B 31 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0
+B 37 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0 NNNN 0
B 70 DataRow 1 60 'xyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxz'
B 13 CommandComplete "SELECT 1"
B 4 ParseComplete
B 4 BindComplete
-B 31 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0
+B 37 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0 NNNN 0
B 70 DataRow 1 60 'xyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxz'
B 13 CommandComplete "SELECT 1"
B 4 ParseComplete
B 4 BindComplete
-B 31 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0
+B 37 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0 NNNN 0
B 70 DataRow 1 60 'xyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxz'
B 13 CommandComplete "SELECT 1"
B 4 ParseComplete
B 4 BindComplete
-B 31 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0
+B 37 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0 NNNN 0
B 70 DataRow 1 60 'xyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxz'
B 13 CommandComplete "SELECT 1"
B 4 ParseComplete
B 4 BindComplete
-B 31 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0
+B 37 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0 NNNN 0
B 70 DataRow 1 60 'xyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxz'
B 13 CommandComplete "SELECT 1"
B 4 ParseComplete
B 4 BindComplete
-B 31 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0
+B 37 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0 NNNN 0
B 70 DataRow 1 60 'xyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxz'
B 13 CommandComplete "SELECT 1"
B 4 ParseComplete
B 4 BindComplete
-B 31 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0
+B 37 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0 NNNN 0
B 70 DataRow 1 60 'xyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxz'
B 13 CommandComplete "SELECT 1"
F 4 Terminate
diff --git a/src/test/modules/libpq_pipeline/traces/pipeline_abort.trace b/src/test/modules/libpq_pipeline/traces/pipeline_abort.trace
index 3fce548b99..6e4309cadd 100644
--- a/src/test/modules/libpq_pipeline/traces/pipeline_abort.trace
+++ b/src/test/modules/libpq_pipeline/traces/pipeline_abort.trace
@@ -50,14 +50,14 @@ F 6 Close P ""
F 4 Sync
B 4 ParseComplete
B 4 BindComplete
-B 33 RowDescription 1 "?column?" NNNN 0 NNNN 65535 -1 0
+B 39 RowDescription 1 "?column?" NNNN 0 NNNN 65535 -1 0 NNNN 0
B 32 DataRow 1 22 '0.33333333333333333333'
B 32 DataRow 1 22 '0.50000000000000000000'
B 32 DataRow 1 22 '1.00000000000000000000'
B NN ErrorResponse S "ERROR" V "ERROR" C "22012" M "division by zero" F "SSSS" L "SSSS" R "SSSS" \x00
B 5 ReadyForQuery I
F 40 Query "SELECT itemno FROM pq_pipeline_demo"
-B 31 RowDescription 1 "itemno" NNNN 2 NNNN 4 -1 0
+B 37 RowDescription 1 "itemno" NNNN 2 NNNN 4 -1 0 NNNN 0
B 11 DataRow 1 1 '3'
B 13 CommandComplete "SELECT 1"
B 5 ReadyForQuery I
diff --git a/src/test/modules/libpq_pipeline/traces/prepared.trace b/src/test/modules/libpq_pipeline/traces/prepared.trace
index 1a7de5c3e6..8d45bbc0ce 100644
--- a/src/test/modules/libpq_pipeline/traces/prepared.trace
+++ b/src/test/modules/libpq_pipeline/traces/prepared.trace
@@ -2,8 +2,8 @@ F 68 Parse "select_one" "SELECT $1, '42', $1::numeric, interval '1 sec'" 1 NNNN
F 16 Describe S "select_one"
F 4 Sync
B 4 ParseComplete
-B 10 ParameterDescription 1 NNNN
-B 113 RowDescription 4 "?column?" NNNN 0 NNNN 4 -1 0 "?column?" NNNN 0 NNNN 65535 -1 0 "numeric" NNNN 0 NNNN 65535 -1 0 "interval" NNNN 0 NNNN 16 -1 0
+B 18 ParameterDescription 1 NNNN NNNN 0 0
+B 137 RowDescription 4 "?column?" NNNN 0 NNNN 4 -1 0 NNNN 0 "?column?" NNNN 0 NNNN 65535 -1 0 NNNN 0 "numeric" NNNN 0 NNNN 65535 -1 0 NNNN 0 "interval" NNNN 0 NNNN 16 -1 0 NNNN 0
B 5 ReadyForQuery I
F 10 Query "BEGIN"
B 10 CommandComplete "BEGIN"
@@ -13,6 +13,6 @@ B 19 CommandComplete "DECLARE CURSOR"
B 5 ReadyForQuery T
F 16 Describe P "cursor_one"
F 4 Sync
-B 33 RowDescription 1 "?column?" NNNN 0 NNNN 4 -1 0
+B 39 RowDescription 1 "?column?" NNNN 0 NNNN 4 -1 0 NNNN 0
B 5 ReadyForQuery T
F 4 Terminate
diff --git a/src/test/modules/libpq_pipeline/traces/simple_pipeline.trace b/src/test/modules/libpq_pipeline/traces/simple_pipeline.trace
index 5c94749bc1..5f4849425d 100644
--- a/src/test/modules/libpq_pipeline/traces/simple_pipeline.trace
+++ b/src/test/modules/libpq_pipeline/traces/simple_pipeline.trace
@@ -5,7 +5,7 @@ F 9 Execute "" 0
F 4 Sync
B 4 ParseComplete
B 4 BindComplete
-B 33 RowDescription 1 "?column?" NNNN 0 NNNN 4 -1 0
+B 39 RowDescription 1 "?column?" NNNN 0 NNNN 4 -1 0 NNNN 0
B 11 DataRow 1 1 '1'
B 13 CommandComplete "SELECT 1"
B 5 ReadyForQuery I
diff --git a/src/test/modules/libpq_pipeline/traces/singlerow.trace b/src/test/modules/libpq_pipeline/traces/singlerow.trace
index 9de99befcc..199df4d4f4 100644
--- a/src/test/modules/libpq_pipeline/traces/singlerow.trace
+++ b/src/test/modules/libpq_pipeline/traces/singlerow.trace
@@ -13,14 +13,14 @@ F 9 Execute "" 0
F 4 Sync
B 4 ParseComplete
B 4 BindComplete
-B 40 RowDescription 1 "generate_series" NNNN 0 NNNN 4 -1 0
+B 46 RowDescription 1 "generate_series" NNNN 0 NNNN 4 -1 0 NNNN 0
B 12 DataRow 1 2 '42'
B 12 DataRow 1 2 '43'
B 12 DataRow 1 2 '44'
B 13 CommandComplete "SELECT 3"
B 4 ParseComplete
B 4 BindComplete
-B 40 RowDescription 1 "generate_series" NNNN 0 NNNN 4 -1 0
+B 46 RowDescription 1 "generate_series" NNNN 0 NNNN 4 -1 0 NNNN 0
B 12 DataRow 1 2 '42'
B 12 DataRow 1 2 '43'
B 12 DataRow 1 2 '44'
@@ -28,7 +28,7 @@ B 12 DataRow 1 2 '45'
B 13 CommandComplete "SELECT 4"
B 4 ParseComplete
B 4 BindComplete
-B 40 RowDescription 1 "generate_series" NNNN 0 NNNN 4 -1 0
+B 46 RowDescription 1 "generate_series" NNNN 0 NNNN 4 -1 0 NNNN 0
B 12 DataRow 1 2 '42'
B 12 DataRow 1 2 '43'
B 12 DataRow 1 2 '44'
diff --git a/src/test/modules/libpq_pipeline/traces/transaction.trace b/src/test/modules/libpq_pipeline/traces/transaction.trace
index 1dcc2373c0..a6869c4a5b 100644
--- a/src/test/modules/libpq_pipeline/traces/transaction.trace
+++ b/src/test/modules/libpq_pipeline/traces/transaction.trace
@@ -54,7 +54,7 @@ B 15 CommandComplete "INSERT 0 1"
B 5 ReadyForQuery I
B 5 ReadyForQuery I
F 34 Query "SELECT * FROM pq_pipeline_tst"
-B 27 RowDescription 1 "id" NNNN 1 NNNN 4 -1 0
+B 33 RowDescription 1 "id" NNNN 1 NNNN 4 -1 0 NNNN 0
B 11 DataRow 1 1 '3'
B 13 CommandComplete "SELECT 1"
B 5 ReadyForQuery I
diff --git a/src/test/regress/expected/column_encryption.out b/src/test/regress/expected/column_encryption.out
new file mode 100644
index 0000000000..f8db4e17ff
--- /dev/null
+++ b/src/test/regress/expected/column_encryption.out
@@ -0,0 +1,59 @@
+\set HIDE_COLUMN_ENCRYPTION false
+/* Imagine:
+CREATE COLUMN MASTER KEY cmk1 (
+ realm = 'test'
+);
+*/
+INSERT INTO pg_colmasterkey (oid, cmkname, cmkowner, cmkrealm) VALUES (
+ pg_nextoid('pg_catalog.pg_colmasterkey', 'oid', 'pg_catalog.pg_colmasterkey_oid_index'),
+ 'cmk1',
+ (select oid from pg_roles where rolname = current_user),
+ 'test'
+);
+/* Imagine:
+CREATE COLUMN ENCRYPTION KEY cek1 (
+ column_master_key = cmk1,
+ algorithm = '...',
+ encrypted_value = '...'
+);
+*/
+INSERT INTO pg_colenckey (oid, cekname, cekowner) VALUES (
+ pg_nextoid('pg_catalog.pg_colenckey', 'oid', 'pg_catalog.pg_colenckey_oid_index'),
+ 'cek1',
+ (select oid from pg_roles where rolname = current_user)
+);
+INSERT INTO pg_colenckeydata (oid, ckdcekid, ckdcmkid, ckdcmkalg, ckdencval) VALUES (
+ pg_nextoid('pg_catalog.pg_colenckeydata', 'oid', 'pg_catalog.pg_colenckeydata_oid_index'),
+ (SELECT oid FROM pg_colenckey WHERE cekname = 'cek1'),
+ (SELECT oid FROM pg_colmasterkey WHERE cmkname = 'cmk1'),
+ 1,
+ '\xDEADBEEF'
+);
+CREATE TABLE tbl_fail (
+ a int,
+ b text,
+ c text ENCRYPTED WITH (column_encryption_key = notexist)
+);
+ERROR: column encryption key "notexist" does not exist
+CREATE TABLE tbl_29f3 (
+ a int,
+ b text,
+ c text ENCRYPTED WITH (column_encryption_key = cek1)
+);
+\d tbl_29f3
+ Table "public.tbl_29f3"
+ Column | Type | Collation | Nullable | Default
+--------+---------+-----------+----------+---------
+ a | integer | | |
+ b | text | | |
+ c | text | | |
+
+\d+ tbl_29f3
+ Table "public.tbl_29f3"
+ Column | Type | Collation | Nullable | Default | Storage | Encryption | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | | |
+ c | text | | | | extended | cek1 | |
+
+DROP TABLE tbl_29f3; -- FIXME: needs pg_dump support for pg_upgrade tests
diff --git a/src/test/regress/expected/oidjoins.out b/src/test/regress/expected/oidjoins.out
index 215eb899be..2aa0e16323 100644
--- a/src/test/regress/expected/oidjoins.out
+++ b/src/test/regress/expected/oidjoins.out
@@ -73,6 +73,8 @@ NOTICE: checking pg_type {typbasetype} => pg_type {oid}
NOTICE: checking pg_type {typcollation} => pg_collation {oid}
NOTICE: checking pg_attribute {attrelid} => pg_class {oid}
NOTICE: checking pg_attribute {atttypid} => pg_type {oid}
+NOTICE: checking pg_attribute {attcek} => pg_colenckey {oid}
+NOTICE: checking pg_attribute {attrealtypid} => pg_type {oid}
NOTICE: checking pg_attribute {attcollation} => pg_collation {oid}
NOTICE: checking pg_class {relnamespace} => pg_namespace {oid}
NOTICE: checking pg_class {reltype} => pg_type {oid}
@@ -266,3 +268,7 @@ NOTICE: checking pg_subscription {subdbid} => pg_database {oid}
NOTICE: checking pg_subscription {subowner} => pg_authid {oid}
NOTICE: checking pg_subscription_rel {srsubid} => pg_subscription {oid}
NOTICE: checking pg_subscription_rel {srrelid} => pg_class {oid}
+NOTICE: checking pg_colmasterkey {cmkowner} => pg_authid {oid}
+NOTICE: checking pg_colenckey {cekowner} => pg_authid {oid}
+NOTICE: checking pg_colenckeydata {ckdcekid} => pg_colenckey {oid}
+NOTICE: checking pg_colenckeydata {ckdcmkid} => pg_colmasterkey {oid}
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 86d755aa44..4a366074da 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -175,13 +175,14 @@ WHERE p1.oid != p2.oid AND
ORDER BY 1, 2;
proargtypes | proargtypes
-----------------------------+--------------------------
+ bytea | pg_encrypted_det
bigint | xid8
text | character
text | character varying
timestamp without time zone | timestamp with time zone
bit | bit varying
txid_snapshot | pg_snapshot
-(6 rows)
+(7 rows)
SELECT DISTINCT p1.proargtypes[1]::regtype, p2.proargtypes[1]::regtype
FROM pg_proc AS p1, pg_proc AS p2
@@ -197,12 +198,13 @@ WHERE p1.oid != p2.oid AND
ORDER BY 1, 2;
proargtypes | proargtypes
-----------------------------+--------------------------
+ bytea | pg_encrypted_det
integer | xid
timestamp without time zone | timestamp with time zone
bit | bit varying
txid_snapshot | pg_snapshot
anyrange | anymultirange
-(5 rows)
+(6 rows)
SELECT DISTINCT p1.proargtypes[2]::regtype, p2.proargtypes[2]::regtype
FROM pg_proc AS p1, pg_proc AS p2
@@ -841,6 +843,8 @@ xid8ge(xid8,xid8)
xid8eq(xid8,xid8)
xid8ne(xid8,xid8)
xid8cmp(xid8,xid8)
+pg_encrypted_det_eq(pg_encrypted_det,pg_encrypted_det)
+pg_encrypted_det_ne(pg_encrypted_det,pg_encrypted_det)
-- restore normal output mode
\a\t
-- List of functions used by libpq's fe-lobj.c
@@ -988,7 +992,9 @@ WHERE c.castmethod = 'b' AND
xml | text | 0 | a
xml | character varying | 0 | a
xml | character | 0 | a
-(10 rows)
+ bytea | pg_encrypted_det | 0 | e
+ bytea | pg_encrypted_rnd | 0 | e
+(12 rows)
-- **************** pg_conversion ****************
-- Look for illegal values in pg_conversion fields.
diff --git a/src/test/regress/expected/prepare.out b/src/test/regress/expected/prepare.out
index 5815e17b39..4482a65d24 100644
--- a/src/test/regress/expected/prepare.out
+++ b/src/test/regress/expected/prepare.out
@@ -162,26 +162,26 @@ PREPARE q7(unknown) AS
-- DML statements
PREPARE q8 AS
UPDATE tenk1 SET stringu1 = $2 WHERE unique1 = $1;
-SELECT name, statement, parameter_types, result_types FROM pg_prepared_statements
+SELECT name, statement, parameter_types, parameter_orig_tables, parameter_orig_columns, result_types FROM pg_prepared_statements
ORDER BY name;
- name | statement | parameter_types | result_types
-------+------------------------------------------------------------------+----------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------
- q2 | PREPARE q2(text) AS +| {text} | {name,boolean,boolean}
- | SELECT datname, datistemplate, datallowconn +| |
- | FROM pg_database WHERE datname = $1; | |
- q3 | PREPARE q3(text, int, float, boolean, smallint) AS +| {text,integer,"double precision",boolean,smallint} | {integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,name,name,name}
- | SELECT * FROM tenk1 WHERE string4 = $1 AND (four = $2 OR+| |
- | ten = $3::bigint OR true = $4 OR odd = $5::int) +| |
- | ORDER BY unique1; | |
- q5 | PREPARE q5(int, text) AS +| {integer,text} | {integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,name,name,name}
- | SELECT * FROM tenk1 WHERE unique1 = $1 OR stringu1 = $2 +| |
- | ORDER BY unique1; | |
- q6 | PREPARE q6 AS +| {integer,name} | {integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,name,name,name}
- | SELECT * FROM tenk1 WHERE unique1 = $1 AND stringu1 = $2; | |
- q7 | PREPARE q7(unknown) AS +| {path} | {text,path}
- | SELECT * FROM road WHERE thepath = $1; | |
- q8 | PREPARE q8 AS +| {integer,name} |
- | UPDATE tenk1 SET stringu1 = $2 WHERE unique1 = $1; | |
+ name | statement | parameter_types | parameter_orig_tables | parameter_orig_columns | result_types
+------+------------------------------------------------------------------+----------------------------------------------------+-----------------------+------------------------+--------------------------------------------------------------------------------------------------------------------------
+ q2 | PREPARE q2(text) AS +| {text} | {-} | {0} | {name,boolean,boolean}
+ | SELECT datname, datistemplate, datallowconn +| | | |
+ | FROM pg_database WHERE datname = $1; | | | |
+ q3 | PREPARE q3(text, int, float, boolean, smallint) AS +| {text,integer,"double precision",boolean,smallint} | {-,-,-,-,-} | {0,0,0,0,0} | {integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,name,name,name}
+ | SELECT * FROM tenk1 WHERE string4 = $1 AND (four = $2 OR+| | | |
+ | ten = $3::bigint OR true = $4 OR odd = $5::int) +| | | |
+ | ORDER BY unique1; | | | |
+ q5 | PREPARE q5(int, text) AS +| {integer,text} | {-,-} | {0,0} | {integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,name,name,name}
+ | SELECT * FROM tenk1 WHERE unique1 = $1 OR stringu1 = $2 +| | | |
+ | ORDER BY unique1; | | | |
+ q6 | PREPARE q6 AS +| {integer,name} | {-,-} | {0,0} | {integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,name,name,name}
+ | SELECT * FROM tenk1 WHERE unique1 = $1 AND stringu1 = $2; | | | |
+ q7 | PREPARE q7(unknown) AS +| {path} | {-} | {0} | {text,path}
+ | SELECT * FROM road WHERE thepath = $1; | | | |
+ q8 | PREPARE q8 AS +| {integer,name} | {-,tenk1} | {0,14} |
+ | UPDATE tenk1 SET stringu1 = $2 WHERE unique1 = $1; | | | |
(6 rows)
-- test DEALLOCATE ALL;
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 60acbd1241..33956f6993 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -237,6 +237,31 @@ SELECT 3 AS x, 'Hello', 4 AS y, true AS "dirty\name" \gdesc \g
3 | Hello | 4 | t
(1 row)
+-- \gencr
+-- (This just tests the parameter passing; there is no encryption here.)
+CREATE TABLE test_gencr (a int, b text);
+INSERT INTO test_gencr VALUES (1, 'one') \gencr
+SELECT * FROM test_gencr WHERE a = 1 \gencr
+ a | b
+---+-----
+ 1 | one
+(1 row)
+
+INSERT INTO test_gencr VALUES ($1, $2) \gencr 2 'two'
+SELECT * FROM test_gencr WHERE a IN ($1, $2) \gencr 2 3
+ a | b
+---+-----
+ 2 | two
+(1 row)
+
+-- test parse error
+SELECT * FROM test_gencr WHERE a = x \gencr
+ERROR: column "x" does not exist
+LINE 1: SELECT * FROM test_gencr WHERE a = x
+ ^
+-- test bind error
+SELECT * FROM test_gencr WHERE a = $1 \gencr
+ERROR: bind message supplies 0 parameters, but prepared statement "" requires 1
-- \gexec
create temporary table gexec_test(a int, b text, c date, d float);
select format('create index on gexec_test(%I)', attname)
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 7ec3d2688f..42e38d66d1 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1423,11 +1423,13 @@ pg_prepared_statements| SELECT p.name,
p.statement,
p.prepare_time,
p.parameter_types,
+ p.parameter_orig_tables,
+ p.parameter_orig_columns,
p.result_types,
p.from_sql,
p.generic_plans,
p.custom_plans
- FROM pg_prepared_statement() p(name, statement, prepare_time, parameter_types, result_types, from_sql, generic_plans, custom_plans);
+ FROM pg_prepared_statement() p(name, statement, prepare_time, parameter_types, parameter_orig_tables, parameter_orig_columns, result_types, from_sql, generic_plans, custom_plans);
pg_prepared_xacts| SELECT p.transaction,
p.gid,
p.prepared,
diff --git a/src/test/regress/expected/type_sanity.out b/src/test/regress/expected/type_sanity.out
index d3ac08c9ee..fc0c5896e4 100644
--- a/src/test/regress/expected/type_sanity.out
+++ b/src/test/regress/expected/type_sanity.out
@@ -17,7 +17,7 @@ SELECT t1.oid, t1.typname
FROM pg_type as t1
WHERE t1.typnamespace = 0 OR
(t1.typlen <= 0 AND t1.typlen != -1 AND t1.typlen != -2) OR
- (t1.typtype not in ('b', 'c', 'd', 'e', 'm', 'p', 'r')) OR
+ (t1.typtype not in ('b', 'c', 'd', 'e', 'm', 'p', 'r', 'y')) OR
NOT t1.typisdefined OR
(t1.typalign not in ('c', 's', 'i', 'd')) OR
(t1.typstorage not in ('p', 'x', 'e', 'm'));
@@ -75,7 +75,9 @@ ORDER BY t1.oid;
4600 | pg_brin_bloom_summary
4601 | pg_brin_minmax_multi_summary
5017 | pg_mcv_list
-(6 rows)
+ 8243 | pg_encrypted_det
+ 8244 | pg_encrypted_rnd
+(8 rows)
-- Make sure typarray points to a "true" array type of our own base
SELECT t1.oid, t1.typname as basetype, t2.typname as arraytype,
@@ -210,7 +212,8 @@ ORDER BY 1;
e | enum_in
m | multirange_in
r | range_in
-(5 rows)
+ y | byteain
+(6 rows)
-- Check for bogus typoutput routines
-- As of 8.0, this check finds refcursor, which is borrowing
@@ -255,7 +258,8 @@ ORDER BY 1;
e | enum_out
m | multirange_out
r | range_out
-(4 rows)
+ y | byteaout
+(5 rows)
-- Domains should have same typoutput as their base types
SELECT t1.oid, t1.typname, t2.oid, t2.typname
@@ -335,7 +339,8 @@ ORDER BY 1;
e | enum_recv
m | multirange_recv
r | range_recv
-(5 rows)
+ y | bytearecv
+(6 rows)
-- Check for bogus typsend routines
-- As of 7.4, this check finds refcursor, which is borrowing
@@ -380,7 +385,8 @@ ORDER BY 1;
e | enum_send
m | multirange_send
r | range_send
-(4 rows)
+ y | byteasend
+(5 rows)
-- Domains should have same typsend as their base types
SELECT t1.oid, t1.typname, t2.oid, t2.typname
@@ -707,6 +713,8 @@ CREATE TABLE tab_core_types AS SELECT
'txt'::text,
true::bool,
E'\\xDEADBEEF'::bytea,
+ E'\\xDEADBEEF'::pg_encrypted_rnd,
+ E'\\xDEADBEEF'::pg_encrypted_det,
B'10001'::bit,
B'10001'::varbit AS varbit,
'12.34'::money,
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 103e11483d..ffca206c6f 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -129,6 +129,9 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression memoize stats
+# WIP
+test: column_encryption
+
# event_trigger cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
test: event_trigger oidjoins
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index a4b354c9e6..8ad1f458d5 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -82,7 +82,7 @@ psql_start_test(const char *testname,
bindir ? bindir : "",
bindir ? "/" : "",
dblist->str,
- "-v HIDE_TABLEAM=on -v HIDE_TOAST_COMPRESSION=on",
+ "-v HIDE_TABLEAM=on -v HIDE_TOAST_COMPRESSION=on -v HIDE_COLUMN_ENCRYPTION=on",
infile,
outfile);
if (offset >= sizeof(psql_cmd))
diff --git a/src/test/regress/sql/column_encryption.sql b/src/test/regress/sql/column_encryption.sql
new file mode 100644
index 0000000000..37f0b23d10
--- /dev/null
+++ b/src/test/regress/sql/column_encryption.sql
@@ -0,0 +1,50 @@
+\set HIDE_COLUMN_ENCRYPTION false
+
+/* Imagine:
+CREATE COLUMN MASTER KEY cmk1 (
+ realm = 'test'
+);
+*/
+INSERT INTO pg_colmasterkey (oid, cmkname, cmkowner, cmkrealm) VALUES (
+ pg_nextoid('pg_catalog.pg_colmasterkey', 'oid', 'pg_catalog.pg_colmasterkey_oid_index'),
+ 'cmk1',
+ (select oid from pg_roles where rolname = current_user),
+ 'test'
+);
+
+/* Imagine:
+CREATE COLUMN ENCRYPTION KEY cek1 (
+ column_master_key = cmk1,
+ algorithm = '...',
+ encrypted_value = '...'
+);
+*/
+INSERT INTO pg_colenckey (oid, cekname, cekowner) VALUES (
+ pg_nextoid('pg_catalog.pg_colenckey', 'oid', 'pg_catalog.pg_colenckey_oid_index'),
+ 'cek1',
+ (select oid from pg_roles where rolname = current_user)
+);
+INSERT INTO pg_colenckeydata (oid, ckdcekid, ckdcmkid, ckdcmkalg, ckdencval) VALUES (
+ pg_nextoid('pg_catalog.pg_colenckeydata', 'oid', 'pg_catalog.pg_colenckeydata_oid_index'),
+ (SELECT oid FROM pg_colenckey WHERE cekname = 'cek1'),
+ (SELECT oid FROM pg_colmasterkey WHERE cmkname = 'cmk1'),
+ 1,
+ '\xDEADBEEF'
+);
+
+CREATE TABLE tbl_fail (
+ a int,
+ b text,
+ c text ENCRYPTED WITH (column_encryption_key = notexist)
+);
+
+CREATE TABLE tbl_29f3 (
+ a int,
+ b text,
+ c text ENCRYPTED WITH (column_encryption_key = cek1)
+);
+
+\d tbl_29f3
+\d+ tbl_29f3
+
+DROP TABLE tbl_29f3; -- FIXME: needs pg_dump support for pg_upgrade tests
diff --git a/src/test/regress/sql/prepare.sql b/src/test/regress/sql/prepare.sql
index c6098dc95c..7db788735c 100644
--- a/src/test/regress/sql/prepare.sql
+++ b/src/test/regress/sql/prepare.sql
@@ -75,7 +75,7 @@ CREATE TEMPORARY TABLE q5_prep_nodata AS EXECUTE q5(200, 'DTAAAA')
PREPARE q8 AS
UPDATE tenk1 SET stringu1 = $2 WHERE unique1 = $1;
-SELECT name, statement, parameter_types, result_types FROM pg_prepared_statements
+SELECT name, statement, parameter_types, parameter_orig_tables, parameter_orig_columns, result_types FROM pg_prepared_statements
ORDER BY name;
-- test DEALLOCATE ALL;
diff --git a/src/test/regress/sql/psql.sql b/src/test/regress/sql/psql.sql
index 1149c6a839..e88c70d302 100644
--- a/src/test/regress/sql/psql.sql
+++ b/src/test/regress/sql/psql.sql
@@ -119,6 +119,23 @@ CREATE TABLE bububu(a int) \gdesc
-- all on one line
SELECT 3 AS x, 'Hello', 4 AS y, true AS "dirty\name" \gdesc \g
+
+-- \gencr
+-- (This just tests the parameter passing; there is no encryption here.)
+
+CREATE TABLE test_gencr (a int, b text);
+INSERT INTO test_gencr VALUES (1, 'one') \gencr
+SELECT * FROM test_gencr WHERE a = 1 \gencr
+
+INSERT INTO test_gencr VALUES ($1, $2) \gencr 2 'two'
+SELECT * FROM test_gencr WHERE a IN ($1, $2) \gencr 2 3
+
+-- test parse error
+SELECT * FROM test_gencr WHERE a = x \gencr
+-- test bind error
+SELECT * FROM test_gencr WHERE a = $1 \gencr
+
+
-- \gexec
create temporary table gexec_test(a int, b text, c date, d float);
diff --git a/src/test/regress/sql/type_sanity.sql b/src/test/regress/sql/type_sanity.sql
index 5edc1f1f6e..34dd19456d 100644
--- a/src/test/regress/sql/type_sanity.sql
+++ b/src/test/regress/sql/type_sanity.sql
@@ -20,7 +20,7 @@
FROM pg_type as t1
WHERE t1.typnamespace = 0 OR
(t1.typlen <= 0 AND t1.typlen != -1 AND t1.typlen != -2) OR
- (t1.typtype not in ('b', 'c', 'd', 'e', 'm', 'p', 'r')) OR
+ (t1.typtype not in ('b', 'c', 'd', 'e', 'm', 'p', 'r', 'y')) OR
NOT t1.typisdefined OR
(t1.typalign not in ('c', 's', 'i', 'd')) OR
(t1.typstorage not in ('p', 'x', 'e', 'm'));
@@ -529,6 +529,8 @@ CREATE TABLE tab_core_types AS SELECT
'txt'::text,
true::bool,
E'\\xDEADBEEF'::bytea,
+ E'\\xDEADBEEF'::pg_encrypted_rnd,
+ E'\\xDEADBEEF'::pg_encrypted_det,
B'10001'::bit,
B'10001'::varbit AS varbit,
'12.34'::money,
diff --git a/src/include/access/printtup.h b/src/include/access/printtup.h
index 971a74cf22..db1f3b8811 100644
--- a/src/include/access/printtup.h
+++ b/src/include/access/printtup.h
@@ -20,6 +20,8 @@ extern DestReceiver *printtup_create_DR(CommandDest dest);
extern void SetRemoteDestReceiverParams(DestReceiver *self, Portal portal);
+extern void MaybeSendColumnEncryptionKeyMessage(Oid attcek);
+
extern void SendRowDescriptionMessage(StringInfo buf,
TupleDesc typeinfo, List *targetlist, int16 *formats);
diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat
index 61cd591430..a0bd708132 100644
--- a/src/include/catalog/pg_amop.dat
+++ b/src/include/catalog/pg_amop.dat
@@ -1028,6 +1028,11 @@
amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)',
amopmethod => 'hash' },
+# pg_encrypted_det_ops
+{ amopfamily => 'hash/pg_encrypted_det_ops', amoplefttype => 'pg_encrypted_det',
+ amoprighttype => 'pg_encrypted_det', amopstrategy => '1', amopopr => '=(pg_encrypted_det,pg_encrypted_det)',
+ amopmethod => 'hash' },
+
# xid_ops
{ amopfamily => 'hash/xid_ops', amoplefttype => 'xid', amoprighttype => 'xid',
amopstrategy => '1', amopopr => '=(xid,xid)', amopmethod => 'hash' },
diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat
index 4cc129bebd..dddb27113f 100644
--- a/src/include/catalog/pg_amproc.dat
+++ b/src/include/catalog/pg_amproc.dat
@@ -402,6 +402,11 @@
{ amprocfamily => 'hash/bytea_ops', amproclefttype => 'bytea',
amprocrighttype => 'bytea', amprocnum => '2',
amproc => 'hashvarlenaextended' },
+{ amprocfamily => 'hash/pg_encrypted_det_ops', amproclefttype => 'pg_encrypted_det',
+ amprocrighttype => 'pg_encrypted_det', amprocnum => '1', amproc => 'hashvarlena' },
+{ amprocfamily => 'hash/pg_encrypted_det_ops', amproclefttype => 'pg_encrypted_det',
+ amprocrighttype => 'pg_encrypted_det', amprocnum => '2',
+ amproc => 'hashvarlenaextended' },
{ amprocfamily => 'hash/xid_ops', amproclefttype => 'xid',
amprocrighttype => 'xid', amprocnum => '1', amproc => 'hashint4' },
{ amprocfamily => 'hash/xid_ops', amproclefttype => 'xid',
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 053294c99f..550a53649b 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -164,6 +164,15 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
*/
bool attislocal BKI_DEFAULT(t);
+ /* column encryption key */
+ Oid attcek BKI_DEFAULT(0) BKI_LOOKUP_OPT(pg_colenckey);
+
+ /* real type if encrypted */
+ Oid attrealtypid BKI_DEFAULT(0) BKI_LOOKUP_OPT(pg_type);
+
+ /* encryption algorithm (PG_CEK_* values) */
+ int16 attencalg BKI_DEFAULT(0);
+
/* Number of times inherited from direct parent relation(s) */
int32 attinhcount BKI_DEFAULT(0);
diff --git a/src/include/catalog/pg_cast.dat b/src/include/catalog/pg_cast.dat
index 4471eb6bbe..878b0bcbbf 100644
--- a/src/include/catalog/pg_cast.dat
+++ b/src/include/catalog/pg_cast.dat
@@ -546,4 +546,10 @@
{ castsource => 'tstzrange', casttarget => 'tstzmultirange',
castfunc => 'tstzmultirange(tstzrange)', castcontext => 'e',
castmethod => 'f' },
+
+{ castsource => 'bytea', casttarget => 'pg_encrypted_det', castfunc => '0',
+ castcontext => 'e', castmethod => 'b' },
+{ castsource => 'bytea', casttarget => 'pg_encrypted_rnd', castfunc => '0',
+ castcontext => 'e', castmethod => 'b' },
+
]
diff --git a/src/include/catalog/pg_colenckey.h b/src/include/catalog/pg_colenckey.h
new file mode 100644
index 0000000000..095ed712b0
--- /dev/null
+++ b/src/include/catalog/pg_colenckey.h
@@ -0,0 +1,55 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_colenckey.h
+ * definition of the "column encryption key" system catalog
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_colenkey.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COLENCKEY_H
+#define PG_COLENCKEY_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_colenckey_d.h"
+
+/* ----------------
+ * pg_colenckey definition. cpp turns this into
+ * typedef struct FormData_pg_colenckey
+ * ----------------
+ */
+CATALOG(pg_colenckey,8234,ColumnEncKeyRelationId)
+{
+ Oid oid;
+ NameData cekname;
+ Oid cekowner BKI_LOOKUP(pg_authid);
+} FormData_pg_colenckey;
+
+typedef FormData_pg_colenckey *Form_pg_colenckey;
+
+DECLARE_UNIQUE_INDEX_PKEY(pg_colenckey_oid_index, 8240, ColumnEncKeyOidIndexId, on pg_colenckey using btree(oid oid_ops));
+DECLARE_UNIQUE_INDEX(pg_colenckey_cekname_index, 8242, ColumnEncKeyNameIndexId, on pg_colenckey using btree(cekname name_ops));
+
+/*
+ * Constants for CMK and CEK algorithms. Note that these are part of the
+ * protocol. For clarity, the assigned numbers are not reused between CMKs
+ * and CEKs, but that is not technically required. In either case, don't
+ * assign zero, so that that can be used as an invalid value.
+ */
+
+#define PG_CMK_RSAES_OAEP_SHA_1 1
+#define PG_CMK_RSAES_OAEP_SHA_256 2
+
+#define PG_CEK_AEAD_AES_128_CBC_HMAC_SHA_256 130
+#define PG_CEK_AEAD_AES_192_CBC_HMAC_SHA_384 131
+#define PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_384 132
+#define PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_512 133
+
+#endif
diff --git a/src/include/catalog/pg_colenckeydata.h b/src/include/catalog/pg_colenckeydata.h
new file mode 100644
index 0000000000..3e4dea7218
--- /dev/null
+++ b/src/include/catalog/pg_colenckeydata.h
@@ -0,0 +1,46 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_colenckeydata.h
+ * definition of the "column encryption key data" system catalog
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_colenkeydata.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COLENCKEYDATA_H
+#define PG_COLENCKEYDATA_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_colenckeydata_d.h"
+
+/* ----------------
+ * pg_colenckeydata definition. cpp turns this into
+ * typedef struct FormData_pg_colenckeydata
+ * ----------------
+ */
+CATALOG(pg_colenckeydata,8250,ColumnEncKeyDataRelationId)
+{
+ Oid oid;
+ Oid ckdcekid BKI_LOOKUP(pg_colenckey);
+ Oid ckdcmkid BKI_LOOKUP(pg_colmasterkey);
+ int16 ckdcmkalg; /* PG_CMK_* values */
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ bytea ckdencval BKI_FORCE_NOT_NULL;
+#endif
+} FormData_pg_colenckeydata;
+
+typedef FormData_pg_colenckeydata *Form_pg_colenckeydata;
+
+DECLARE_TOAST(pg_colenckeydata, 8237, 8238);
+
+DECLARE_UNIQUE_INDEX_PKEY(pg_colenckeydata_oid_index, 8251, ColumnEncKeyDataOidIndexId, on pg_colenckeydata using btree(oid oid_ops));
+DECLARE_UNIQUE_INDEX(pg_colenckeydata_ckdcekid_ckdcmkid_index, 8252, ColumnEncKeyCekidCmkidIndexId, on pg_colenckeydata using btree(ckdcekid oid_ops, ckdcmkid oid_ops));
+
+#endif
diff --git a/src/include/catalog/pg_colmasterkey.h b/src/include/catalog/pg_colmasterkey.h
new file mode 100644
index 0000000000..0344cc4201
--- /dev/null
+++ b/src/include/catalog/pg_colmasterkey.h
@@ -0,0 +1,45 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_colmasterkey.h
+ * definition of the "column master key" system catalog
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_colmasterkey.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COLMASTERKEY_H
+#define PG_COLMASTERKEY_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_colmasterkey_d.h"
+
+/* ----------------
+ * pg_colmasterkey definition. cpp turns this into
+ * typedef struct FormData_pg_colmasterkey
+ * ----------------
+ */
+CATALOG(pg_colmasterkey,8233,ColumnMasterKeyRelationId)
+{
+ Oid oid;
+ NameData cmkname;
+ Oid cmkowner BKI_LOOKUP(pg_authid);
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text cmkrealm BKI_FORCE_NOT_NULL;
+#endif
+} FormData_pg_colmasterkey;
+
+typedef FormData_pg_colmasterkey *Form_pg_colmasterkey;
+
+DECLARE_TOAST(pg_colmasterkey, 8235, 8236);
+
+DECLARE_UNIQUE_INDEX_PKEY(pg_colmasterkey_oid_index, 8239, ColumnMasterKeyOidIndexId, on pg_colmasterkey using btree(oid oid_ops));
+DECLARE_UNIQUE_INDEX(pg_colmasterkey_cmkname_index, 8241, ColumnMasterKeyNameIndexId, on pg_colmasterkey using btree(cmkname name_ops));
+
+#endif
diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat
index dbcae7ffdd..0ca401ffe4 100644
--- a/src/include/catalog/pg_opclass.dat
+++ b/src/include/catalog/pg_opclass.dat
@@ -166,6 +166,8 @@
opcintype => 'bool' },
{ opcmethod => 'hash', opcname => 'bytea_ops', opcfamily => 'hash/bytea_ops',
opcintype => 'bytea' },
+{ opcmethod => 'hash', opcname => 'pg_encrypted_det_ops', opcfamily => 'hash/pg_encrypted_det_ops',
+ opcintype => 'pg_encrypted_det' },
{ opcmethod => 'btree', opcname => 'tid_ops', opcfamily => 'btree/tid_ops',
opcintype => 'tid' },
{ opcmethod => 'hash', opcname => 'xid_ops', opcfamily => 'hash/xid_ops',
diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat
index bc5f8213f3..4737a7f9ed 100644
--- a/src/include/catalog/pg_operator.dat
+++ b/src/include/catalog/pg_operator.dat
@@ -3458,4 +3458,14 @@
oprcode => 'multirange_after_multirange', oprrest => 'multirangesel',
oprjoin => 'scalargtjoinsel' },
+{ oid => '8247', descr => 'equal',
+ oprname => '=', oprcanmerge => 'f', oprcanhash => 't', oprleft => 'pg_encrypted_det',
+ oprright => 'pg_encrypted_det', oprresult => 'bool', oprcom => '=(pg_encrypted_det,pg_encrypted_det)',
+ oprnegate => '<>(pg_encrypted_det,pg_encrypted_det)', oprcode => 'pg_encrypted_det_eq', oprrest => 'eqsel',
+ oprjoin => 'eqjoinsel' },
+{ oid => '8248', descr => 'not equal',
+ oprname => '<>', oprleft => 'pg_encrypted_det', oprright => 'pg_encrypted_det', oprresult => 'bool',
+ oprcom => '<>(pg_encrypted_det,pg_encrypted_det)', oprnegate => '=(pg_encrypted_det,pg_encrypted_det)',
+ oprcode => 'pg_encrypted_det_ne', oprrest => 'neqsel', oprjoin => 'neqjoinsel' },
+
]
diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat
index b3b6a7e616..5343580b06 100644
--- a/src/include/catalog/pg_opfamily.dat
+++ b/src/include/catalog/pg_opfamily.dat
@@ -108,6 +108,8 @@
opfmethod => 'hash', opfname => 'bool_ops' },
{ oid => '2223',
opfmethod => 'hash', opfname => 'bytea_ops' },
+{ oid => '8249',
+ opfmethod => 'hash', opfname => 'pg_encrypted_det_ops' },
{ oid => '2789',
opfmethod => 'btree', opfname => 'tid_ops' },
{ oid => '2225',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 2e41f4d9e8..2fa26a2222 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8025,9 +8025,9 @@
proname => 'pg_prepared_statement', prorows => '1000', proretset => 't',
provolatile => 's', proparallel => 'r', prorettype => 'record',
proargtypes => '',
- proallargtypes => '{text,text,timestamptz,_regtype,_regtype,bool,int8,int8}',
- proargmodes => '{o,o,o,o,o,o,o,o}',
- proargnames => '{name,statement,prepare_time,parameter_types,result_types,from_sql,generic_plans,custom_plans}',
+ proallargtypes => '{text,text,timestamptz,_regtype,_regclass,_int2,_regtype,bool,int8,int8}',
+ proargmodes => '{o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{name,statement,prepare_time,parameter_types,parameter_orig_tables,parameter_orig_columns,result_types,from_sql,generic_plans,custom_plans}',
prosrc => 'pg_prepared_statement' },
{ oid => '2511', descr => 'get the open cursors for this session',
proname => 'pg_cursor', prorows => '1000', proretset => 't',
@@ -11886,4 +11886,11 @@
prorettype => 'bytea', proargtypes => 'pg_brin_minmax_multi_summary',
prosrc => 'brin_minmax_multi_summary_send' },
+{ oid => '8245',
+ proname => 'pg_encrypted_det_eq', proleakproof => 't', prorettype => 'bool',
+ proargtypes => 'pg_encrypted_det pg_encrypted_det', prosrc => 'byteaeq' },
+{ oid => '8246',
+ proname => 'pg_encrypted_det_ne', proleakproof => 't', prorettype => 'bool',
+ proargtypes => 'pg_encrypted_det pg_encrypted_det', prosrc => 'byteane' },
+
]
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index df45879463..df1fa3e393 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -692,4 +692,16 @@
typreceive => 'brin_minmax_multi_summary_recv',
typsend => 'brin_minmax_multi_summary_send', typalign => 'i',
typstorage => 'x', typcollation => 'default' },
+
+{ oid => '8243', descr => 'encrypted column (deterministic)',
+ typname => 'pg_encrypted_det', typlen => '-1', typbyval => 'f', typtype => 'y',
+ typcategory => 'Y', typinput => 'byteain', typoutput => 'byteaout',
+ typreceive => 'bytearecv', typsend => 'byteasend', typalign => 'i',
+ typstorage => 'e' },
+{ oid => '8244', descr => 'encrypted column (randomized)',
+ typname => 'pg_encrypted_rnd', typlen => '-1', typbyval => 'f', typtype => 'y',
+ typcategory => 'Y', typinput => 'byteain', typoutput => 'byteaout',
+ typreceive => 'bytearecv', typsend => 'byteasend', typalign => 'i',
+ typstorage => 'e' },
+
]
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 48a2559137..11aff9ff83 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -277,6 +277,7 @@ DECLARE_UNIQUE_INDEX(pg_type_typname_nsp_index, 2704, TypeNameNspIndexId, on pg_
#define TYPTYPE_MULTIRANGE 'm' /* multirange type */
#define TYPTYPE_PSEUDO 'p' /* pseudo-type */
#define TYPTYPE_RANGE 'r' /* range type */
+#define TYPTYPE_ENCRYPTED 'y' /* encrypted column value */
#define TYPCATEGORY_INVALID '\0' /* not an allowed category */
#define TYPCATEGORY_ARRAY 'A'
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index f93d866548..8c08bd845f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -679,6 +679,7 @@ typedef struct ColumnDef
char *colname; /* name of column */
TypeName *typeName; /* type of column */
char *compression; /* compression method for column */
+ List *encryption; /* encryption info for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
diff --git a/src/include/parser/analyze.h b/src/include/parser/analyze.h
index dc379547c7..c310e124f1 100644
--- a/src/include/parser/analyze.h
+++ b/src/include/parser/analyze.h
@@ -28,7 +28,9 @@ extern PGDLLIMPORT post_parse_analyze_hook_type post_parse_analyze_hook;
extern Query *parse_analyze_fixedparams(RawStmt *parseTree, const char *sourceText,
const Oid *paramTypes, int numParams, QueryEnvironment *queryEnv);
extern Query *parse_analyze_varparams(RawStmt *parseTree, const char *sourceText,
- Oid **paramTypes, int *numParams, QueryEnvironment *queryEnv);
+ Oid **paramTypes, int *numParams,
+ Oid **paramOrigTbls, AttrNumber **paramOrigCols,
+ QueryEnvironment *queryEnv);
extern Query *parse_analyze_withcb(RawStmt *parseTree, const char *sourceText,
ParserSetupHook parserSetup,
void *parserSetupArg,
diff --git a/src/include/parser/parse_node.h b/src/include/parser/parse_node.h
index cf9c759025..225a2a8733 100644
--- a/src/include/parser/parse_node.h
+++ b/src/include/parser/parse_node.h
@@ -93,6 +93,7 @@ typedef Node *(*ParseParamRefHook) (ParseState *pstate, ParamRef *pref);
typedef Node *(*CoerceParamHook) (ParseState *pstate, Param *param,
Oid targetTypeId, int32 targetTypeMod,
int location);
+typedef void (*ParamAssignOrigHook) (ParseState *pstate, Param *param, Oid origtbl, AttrNumber origcol);
/*
@@ -222,6 +223,7 @@ struct ParseState
PostParseColumnRefHook p_post_columnref_hook;
ParseParamRefHook p_paramref_hook;
CoerceParamHook p_coerce_param_hook;
+ ParamAssignOrigHook p_param_assign_orig_hook;
void *p_ref_hook_state; /* common passthrough link for above */
};
diff --git a/src/include/parser/parse_param.h b/src/include/parser/parse_param.h
index df1ee660d8..da202b28a7 100644
--- a/src/include/parser/parse_param.h
+++ b/src/include/parser/parse_param.h
@@ -18,7 +18,8 @@
extern void setup_parse_fixed_parameters(ParseState *pstate,
const Oid *paramTypes, int numParams);
extern void setup_parse_variable_parameters(ParseState *pstate,
- Oid **paramTypes, int *numParams);
+ Oid **paramTypes, int *numParams,
+ Oid **paramOrigTbls, AttrNumber **paramOrigCols);
extern void check_variable_parameters(ParseState *pstate, Query *query);
extern bool query_contains_extern_params(Query *query);
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index 70d9dab25b..3038ceb522 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -53,6 +53,8 @@ extern List *pg_analyze_and_rewrite_varparams(RawStmt *parsetree,
const char *query_string,
Oid **paramTypes,
int *numParams,
+ Oid **paramOrigTbls,
+ AttrNumber **paramOrigCols,
QueryEnvironment *queryEnv);
extern List *pg_analyze_and_rewrite_withcb(RawStmt *parsetree,
const char *query_string,
diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h
index 0499635f59..9a7cf794ce 100644
--- a/src/include/utils/plancache.h
+++ b/src/include/utils/plancache.h
@@ -101,6 +101,10 @@ typedef struct CachedPlanSource
CommandTag commandTag; /* 'nuff said */
Oid *param_types; /* array of parameter type OIDs, or NULL */
int num_params; /* length of param_types array */
+ Oid *param_origtbls; /* array of underlying tables of parameters,
+ * or NULL */
+ AttrNumber *param_origcols; /* array of underlying columns of parameters,
+ * or NULL */
ParserSetupHook parserSetup; /* alternative parameter spec method */
void *parserSetupArg;
int cursor_options; /* cursor options used for planning */
@@ -199,6 +203,8 @@ extern void CompleteCachedPlan(CachedPlanSource *plansource,
MemoryContext querytree_context,
Oid *param_types,
int num_params,
+ Oid *param_origtbls,
+ AttrNumber *param_origcols,
ParserSetupHook parserSetup,
void *parserSetupArg,
int cursor_options,
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 4463ea66be..4e8a940590 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -44,8 +44,12 @@ enum SysCacheIdentifier
AUTHNAME,
AUTHOID,
CASTSOURCETARGET,
+ CEKDATAOID,
+ CEKNAME,
+ CEKOID,
CLAAMNAMENSP,
CLAOID,
+ CMKOID,
COLLNAMEENCNSP,
COLLOID,
CONDEFAULT,
diff --git a/src/backend/access/common/printsimple.c b/src/backend/access/common/printsimple.c
index e99aa279f6..79154f8aab 100644
--- a/src/backend/access/common/printsimple.c
+++ b/src/backend/access/common/printsimple.c
@@ -46,6 +46,8 @@ printsimple_startup(DestReceiver *self, int operation, TupleDesc tupdesc)
pq_sendint16(&buf, attr->attlen);
pq_sendint32(&buf, attr->atttypmod);
pq_sendint16(&buf, 0); /* format code */
+ pq_sendint32(&buf, 0); /* CEK */
+ pq_sendint16(&buf, 0); /* CEK alg */
}
pq_endmessage(&buf);
diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c
index d2f3b57288..28b437e5c7 100644
--- a/src/backend/access/common/printtup.c
+++ b/src/backend/access/common/printtup.c
@@ -15,13 +15,26 @@
*/
#include "postgres.h"
+#include "access/genam.h"
#include "access/printtup.h"
+#include "access/skey.h"
+#include "access/table.h"
+#include "catalog/pg_colenckey.h"
+#include "catalog/pg_colenckeydata.h"
+#include "catalog/pg_colmasterkey.h"
#include "libpq/libpq.h"
#include "libpq/pqformat.h"
#include "tcop/pquery.h"
+#include "utils/array.h"
+#include "utils/arrayaccess.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/inval.h"
#include "utils/lsyscache.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
static void printtup_startup(DestReceiver *self, int operation,
@@ -151,6 +164,135 @@ printtup_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
*/
}
+/*
+ * Send ColumnMasterKey message, unless it's already been sent in this session
+ * for this key.
+ */
+List *cmk_sent = NIL;
+
+static void
+cmk_change_cb(Datum arg, int cacheid, uint32 hashvalue)
+{
+ list_free(cmk_sent);
+ cmk_sent = NIL;
+}
+
+static void
+MaybeSendColumnMasterKeyMessage(Oid cmkid)
+{
+ HeapTuple tuple;
+ Form_pg_colmasterkey cmkform;
+ Datum datum;
+ bool isnull;
+ StringInfoData buf;
+ static bool registered_inval = false;
+ MemoryContext oldcontext;
+
+ if (list_member_oid(cmk_sent, cmkid))
+ return;
+
+ tuple = SearchSysCache1(CMKOID, ObjectIdGetDatum(cmkid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for column master key %u", cmkid);
+ cmkform = (Form_pg_colmasterkey) GETSTRUCT(tuple);
+
+ pq_beginmessage(&buf, 'y'); /* ColumnMasterKey */
+ pq_sendint32(&buf, cmkform->oid);
+ pq_sendstring(&buf, NameStr(cmkform->cmkname));
+ datum = SysCacheGetAttr(CMKOID, tuple, Anum_pg_colmasterkey_cmkrealm, &isnull);
+ Assert(!isnull);
+ pq_sendstring(&buf, TextDatumGetCString(datum));
+ pq_endmessage(&buf);
+
+ ReleaseSysCache(tuple);
+
+ if (!registered_inval)
+ {
+ CacheRegisterSyscacheCallback(CMKOID, cmk_change_cb, (Datum) 0);
+ registered_inval = true;
+ }
+
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+ cmk_sent = lappend_oid(cmk_sent, cmkid);
+ MemoryContextSwitchTo(oldcontext);
+}
+
+/*
+ * Send ColumnEncryptionKey message, unless it's already been sent in this
+ * session for this key.
+ */
+List *cek_sent = NIL;
+
+static void
+cek_change_cb(Datum arg, int cacheid, uint32 hashvalue)
+{
+ list_free(cek_sent);
+ cek_sent = NIL;
+}
+
+void
+MaybeSendColumnEncryptionKeyMessage(Oid attcek)
+{
+ HeapTuple tuple;
+ ScanKeyData skey[1];
+ SysScanDesc sd;
+ Relation rel;
+ bool found = false;
+ static bool registered_inval = false;
+ MemoryContext oldcontext;
+
+ if (list_member_oid(cek_sent, attcek))
+ return;
+
+ ScanKeyInit(&skey[0],
+ Anum_pg_colenckeydata_ckdcekid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attcek));
+ rel = table_open(ColumnEncKeyDataRelationId, AccessShareLock);
+ sd = systable_beginscan(rel, ColumnEncKeyCekidCmkidIndexId, true, NULL, 1, skey);
+
+ while ((tuple = systable_getnext(sd)))
+ {
+ Form_pg_colenckeydata ckdform = (Form_pg_colenckeydata) GETSTRUCT(tuple);
+ Datum datum;
+ bool isnull;
+ bytea *ba;
+ StringInfoData buf;
+
+ MaybeSendColumnMasterKeyMessage(ckdform->ckdcmkid);
+
+ datum = heap_getattr(tuple, Anum_pg_colenckeydata_ckdencval, RelationGetDescr(rel), &isnull);
+ Assert(!isnull);
+ ba = pg_detoast_datum_packed((bytea *) DatumGetPointer(datum));
+
+ pq_beginmessage(&buf, 'Y'); /* ColumnEncryptionKey */
+ pq_sendint32(&buf, ckdform->ckdcekid);
+ pq_sendint32(&buf, ckdform->ckdcmkid);
+ pq_sendint16(&buf, ckdform->ckdcmkalg);
+ pq_sendint32(&buf, VARSIZE_ANY_EXHDR(ba));
+ pq_sendbytes(&buf, VARDATA_ANY(ba), VARSIZE_ANY_EXHDR(ba));
+ pq_endmessage(&buf);
+
+ found = true;
+ }
+
+ if (!found)
+ elog(ERROR, "lookup failed for column encryption key data %u", attcek);
+
+ systable_endscan(sd);
+ table_close(rel, NoLock);
+
+ if (!registered_inval)
+ {
+ CacheRegisterSyscacheCallback(CEKDATAOID, cek_change_cb, (Datum) 0);
+ registered_inval = true;
+ }
+
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+ cek_sent = lappend_oid(cek_sent, attcek);
+ MemoryContextSwitchTo(oldcontext);
+}
+
/*
* SendRowDescriptionMessage --- send a RowDescription message to the frontend
*
@@ -200,6 +342,8 @@ SendRowDescriptionMessage(StringInfo buf, TupleDesc typeinfo,
Oid resorigtbl;
AttrNumber resorigcol;
int16 format;
+ Oid attcekid = InvalidOid;
+ int16 attencalg = 0;
/*
* If column is a domain, send the base type and typmod instead.
@@ -231,6 +375,22 @@ SendRowDescriptionMessage(StringInfo buf, TupleDesc typeinfo,
else
format = 0;
+ if (get_typtype(atttypid) == TYPTYPE_ENCRYPTED)
+ {
+ HeapTuple tp;
+ Form_pg_attribute orig_att;
+
+ tp = SearchSysCache2(ATTNUM, ObjectIdGetDatum(resorigtbl), Int16GetDatum(resorigcol));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u", resorigcol, resorigtbl);
+ orig_att = (Form_pg_attribute) GETSTRUCT(tp);
+ MaybeSendColumnEncryptionKeyMessage(orig_att->attcek);
+ atttypid = orig_att->attrealtypid;
+ attcekid = orig_att->attcek;
+ attencalg = orig_att->attencalg;
+ ReleaseSysCache(tp);
+ }
+
pq_writestring(buf, NameStr(att->attname));
pq_writeint32(buf, resorigtbl);
pq_writeint16(buf, resorigcol);
@@ -238,6 +398,8 @@ SendRowDescriptionMessage(StringInfo buf, TupleDesc typeinfo,
pq_writeint16(buf, att->attlen);
pq_writeint32(buf, atttypmod);
pq_writeint16(buf, format);
+ pq_writeint32(buf, attcekid);
+ pq_writeint16(buf, attencalg);
}
pq_endmessage_reuse(buf);
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index d6fb261e20..e3ecc64ea7 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -459,6 +459,12 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attislocal != attr2->attislocal)
return false;
+ if (attr1->attcek != attr2->attcek)
+ return false;
+ if (attr1->attrealtypid != attr2->attrealtypid)
+ return false;
+ if (attr1->attencalg != attr2->attencalg)
+ return false;
if (attr1->attinhcount != attr2->attinhcount)
return false;
if (attr1->attcollation != attr2->attcollation)
@@ -629,6 +635,9 @@ TupleDescInitEntry(TupleDesc desc,
att->attgenerated = '\0';
att->attisdropped = false;
att->attislocal = true;
+ att->attcek = 0;
+ att->attrealtypid = 0;
+ att->attencalg = 0;
att->attinhcount = 0;
/* variable-length fields are not present in tupledescs */
@@ -690,6 +699,9 @@ TupleDescInitBuiltinEntry(TupleDesc desc,
att->attgenerated = '\0';
att->attisdropped = false;
att->attislocal = true;
+ att->attcek = 0;
+ att->attrealtypid = 0;
+ att->attencalg = 0;
att->attinhcount = 0;
/* variable-length fields are not present in tupledescs */
diff --git a/src/backend/access/hash/hashvalidate.c b/src/backend/access/hash/hashvalidate.c
index 10bf26ce7c..cea91d4a88 100644
--- a/src/backend/access/hash/hashvalidate.c
+++ b/src/backend/access/hash/hashvalidate.c
@@ -331,7 +331,7 @@ check_hash_func_signature(Oid funcid, int16 amprocnum, Oid argtype)
argtype == BOOLOID)
/* okay, allowed use of hashchar() */ ;
else if ((funcid == F_HASHVARLENA || funcid == F_HASHVARLENAEXTENDED) &&
- argtype == BYTEAOID)
+ (argtype == BYTEAOID || argtype == PG_ENCRYPTED_DETOID))
/* okay, allowed use of hashvarlena() */ ;
else
result = false;
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 89a0221ec9..7e1a91b974 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -72,7 +72,8 @@ CATALOG_HEADERS := \
pg_collation.h pg_parameter_acl.h pg_partitioned_table.h \
pg_range.h pg_transform.h \
pg_sequence.h pg_publication.h pg_publication_namespace.h \
- pg_publication_rel.h pg_subscription.h pg_subscription_rel.h
+ pg_publication_rel.h pg_subscription.h pg_subscription_rel.h \
+ pg_colmasterkey.h pg_colenckey.h pg_colenckeydata.h
GENERATED_HEADERS := $(CATALOG_HEADERS:%.h=%_d.h) schemapg.h system_fk_info.h
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 1803194db9..0ea58239b4 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -746,6 +746,9 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attgenerated - 1] = CharGetDatum(attrs->attgenerated);
slot[slotCount]->tts_values[Anum_pg_attribute_attisdropped - 1] = BoolGetDatum(attrs->attisdropped);
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcek - 1] = ObjectIdGetDatum(attrs->attcek);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attrealtypid - 1] = ObjectIdGetDatum(attrs->attrealtypid);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attencalg - 1] = Int16GetDatum(attrs->attencalg);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
if (attoptions && attoptions[natts] != (Datum) 0)
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 2333aae467..94578350bd 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -50,7 +50,6 @@ static void InitQueryHashTable(void);
static ParamListInfo EvaluateParams(ParseState *pstate,
PreparedStatement *pstmt, List *params,
EState *estate);
-static Datum build_regtype_array(Oid *param_types, int num_params);
/*
* Implements the 'PREPARE' utility statement.
@@ -62,6 +61,8 @@ PrepareQuery(ParseState *pstate, PrepareStmt *stmt,
RawStmt *rawstmt;
CachedPlanSource *plansource;
Oid *argtypes = NULL;
+ Oid *argorigtbls = NULL;
+ AttrNumber *argorigcols = NULL;
int nargs;
List *query_list;
@@ -108,6 +109,9 @@ PrepareQuery(ParseState *pstate, PrepareStmt *stmt,
argtypes[i++] = toid;
}
+
+ argorigtbls = (Oid *) palloc0(nargs * sizeof(Oid));
+ argorigcols = (AttrNumber *) palloc0(nargs * sizeof(AttrNumber));
}
/*
@@ -117,7 +121,9 @@ PrepareQuery(ParseState *pstate, PrepareStmt *stmt,
* Rewrite the query. The result could be 0, 1, or many queries.
*/
query_list = pg_analyze_and_rewrite_varparams(rawstmt, pstate->p_sourcetext,
- &argtypes, &nargs, NULL);
+ &argtypes, &nargs,
+ &argorigtbls, &argorigcols,
+ NULL);
/* Finish filling in the CachedPlanSource */
CompleteCachedPlan(plansource,
@@ -125,6 +131,8 @@ PrepareQuery(ParseState *pstate, PrepareStmt *stmt,
NULL,
argtypes,
nargs,
+ argorigtbls,
+ argorigcols,
NULL,
NULL,
CURSOR_OPT_PARALLEL_OK, /* allow parallel mode */
@@ -683,9 +691,11 @@ pg_prepared_statement(PG_FUNCTION_ARGS)
hash_seq_init(&hash_seq, prepared_queries);
while ((prep_stmt = hash_seq_search(&hash_seq)) != NULL)
{
+ int num_params = prep_stmt->plansource->num_params;
TupleDesc result_desc;
- Datum values[8];
- bool nulls[8];
+ Datum *tmp_ary;
+ Datum values[10];
+ bool nulls[10];
result_desc = prep_stmt->plansource->resultDesc;
@@ -694,25 +704,38 @@ pg_prepared_statement(PG_FUNCTION_ARGS)
values[0] = CStringGetTextDatum(prep_stmt->stmt_name);
values[1] = CStringGetTextDatum(prep_stmt->plansource->query_string);
values[2] = TimestampTzGetDatum(prep_stmt->prepare_time);
- values[3] = build_regtype_array(prep_stmt->plansource->param_types,
- prep_stmt->plansource->num_params);
+
+ tmp_ary = (Datum *) palloc(num_params * sizeof(Datum));
+ for (int i = 0; i < num_params; i++)
+ tmp_ary[i] = ObjectIdGetDatum(prep_stmt->plansource->param_types[i]);
+ values[3] = PointerGetDatum(construct_array_builtin(tmp_ary, num_params, REGTYPEOID));
+
+ tmp_ary = (Datum *) palloc(num_params * sizeof(Datum));
+ for (int i = 0; i < num_params; i++)
+ tmp_ary[i] = ObjectIdGetDatum(prep_stmt->plansource->param_origtbls[i]);
+ values[4] = PointerGetDatum(construct_array_builtin(tmp_ary, num_params, REGCLASSOID));
+
+ tmp_ary = (Datum *) palloc(num_params * sizeof(Datum));
+ for (int i = 0; i < num_params; i++)
+ tmp_ary[i] = Int16GetDatum(prep_stmt->plansource->param_origcols[i]);
+ values[5] = PointerGetDatum(construct_array_builtin(tmp_ary, num_params, INT2OID));
+
if (result_desc)
{
- Oid *result_types;
-
- result_types = (Oid *) palloc(result_desc->natts * sizeof(Oid));
+ tmp_ary = (Datum *) palloc(result_desc->natts * sizeof(Datum));
for (int i = 0; i < result_desc->natts; i++)
- result_types[i] = result_desc->attrs[i].atttypid;
- values[4] = build_regtype_array(result_types, result_desc->natts);
+ tmp_ary[i] = ObjectIdGetDatum(result_desc->attrs[i].atttypid);
+ values[6] = PointerGetDatum(construct_array_builtin(tmp_ary, result_desc->natts, REGTYPEOID));
}
else
{
/* no result descriptor (for example, DML statement) */
- nulls[4] = true;
+ nulls[6] = true;
}
- values[5] = BoolGetDatum(prep_stmt->from_sql);
- values[6] = Int64GetDatumFast(prep_stmt->plansource->num_generic_plans);
- values[7] = Int64GetDatumFast(prep_stmt->plansource->num_custom_plans);
+
+ values[7] = BoolGetDatum(prep_stmt->from_sql);
+ values[8] = Int64GetDatumFast(prep_stmt->plansource->num_generic_plans);
+ values[9] = Int64GetDatumFast(prep_stmt->plansource->num_custom_plans);
tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc,
values, nulls);
@@ -721,24 +744,3 @@ pg_prepared_statement(PG_FUNCTION_ARGS)
return (Datum) 0;
}
-
-/*
- * This utility function takes a C array of Oids, and returns a Datum
- * pointing to a one-dimensional Postgres array of regtypes. An empty
- * array is returned as a zero-element array, not NULL.
- */
-static Datum
-build_regtype_array(Oid *param_types, int num_params)
-{
- Datum *tmp_ary;
- ArrayType *result;
- int i;
-
- tmp_ary = (Datum *) palloc(num_params * sizeof(Datum));
-
- for (i = 0; i < num_params; i++)
- tmp_ary[i] = ObjectIdGetDatum(param_types[i]);
-
- result = construct_array_builtin(tmp_ary, num_params, REGTYPEOID);
- return PointerGetDatum(result);
-}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 2de0ebacec..a0a9c6b59a 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -35,6 +35,7 @@
#include "catalog/partition.h"
#include "catalog/pg_am.h"
#include "catalog/pg_attrdef.h"
+#include "catalog/pg_colenckey.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
@@ -931,6 +932,76 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->compression)
attr->attcompression = GetAttributeCompression(attr->atttypid,
colDef->compression);
+
+ if (colDef->encryption)
+ {
+ ListCell *lc;
+ char *cek = NULL;
+ Oid cekoid;
+ bool encdet = false;
+ int alg = PG_CEK_AEAD_AES_128_CBC_HMAC_SHA_256;
+
+ foreach(lc, colDef->encryption)
+ {
+ DefElem *el = lfirst_node(DefElem, lc);
+
+ if (strcmp(el->defname, "column_encryption_key") == 0)
+ cek = strVal(linitial(castNode(TypeName, el->arg)->names));
+ else if (strcmp(el->defname, "encryption_type") == 0)
+ {
+ char *val = strVal(linitial(castNode(TypeName, el->arg)->names));
+
+ if (strcmp(val, "deterministic") == 0)
+ encdet = true;
+ else if (strcmp(val, "randomized") == 0)
+ encdet = false;
+ else
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unrecognized encryption type: %s", val));
+ }
+ else if (strcmp(el->defname, "algorithm") == 0)
+ {
+ char *val = strVal(el->arg);
+
+ if (strcmp(val, "AEAD_AES_128_CBC_HMAC_SHA_256") == 0)
+ alg = PG_CEK_AEAD_AES_128_CBC_HMAC_SHA_256;
+ else if (strcmp(val, "AEAD_AES_192_CBC_HMAC_SHA_384") == 0)
+ alg = PG_CEK_AEAD_AES_192_CBC_HMAC_SHA_384;
+ else if (strcmp(val, "AEAD_AES_256_CBC_HMAC_SHA_384") == 0)
+ alg = PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_384;
+ else if (strcmp(val, "AEAD_AES_256_CBC_HMAC_SHA_512") == 0)
+ alg = PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_512;
+ else
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unrecognized encryption algorithm: %s", val));
+ }
+ else
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unrecognized column encryption parameter: %s", el->defname));
+ }
+
+ if (!cek)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_COLUMN_DEFINITION),
+ errmsg("column encryption key must be specified"));
+
+ cekoid = GetSysCacheOid1(CEKNAME, Anum_pg_colenckey_oid, PointerGetDatum(cek));
+ if (!cekoid)
+ ereport(ERROR,
+ errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("column encryption key \"%s\" does not exist", cek));
+
+ attr->attcek = cekoid;
+ attr->attrealtypid = attr->atttypid;
+ if (encdet)
+ attr->atttypid = PG_ENCRYPTED_DETOID;
+ else
+ attr->atttypid = PG_ENCRYPTED_RNDOID;
+ attr->attencalg = alg;
+ }
}
/*
@@ -6829,6 +6900,9 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attgenerated = colDef->generated;
attribute.attisdropped = false;
attribute.attislocal = colDef->is_local;
+ attribute.attcek = 0; // TODO
+ attribute.attrealtypid = 0; // TODO
+ attribute.attencalg = 0; // TODO
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 29bc26669b..cde5b0e087 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -2279,6 +2279,8 @@ _SPI_prepare_plan(const char *src, SPIPlanPtr plan)
NULL,
plan->argtypes,
plan->nargs,
+ NULL,
+ NULL,
plan->parserSetup,
plan->parserSetupArg,
plan->cursor_options,
@@ -2516,6 +2518,8 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
NULL,
plan->argtypes,
plan->nargs,
+ NULL,
+ NULL,
plan->parserSetup,
plan->parserSetupArg,
plan->cursor_options,
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 51d630fa89..c901cfbb86 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3555,6 +3555,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(encryption);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e747e1667d..69a3adee7c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -3045,6 +3045,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(encryption);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 4cb1744da6..8fbe0f5f8f 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -4288,6 +4288,8 @@ raw_expression_tree_walker(Node *node,
return true;
if (walker(coldef->compression, context))
return true;
+ if (walker(coldef->encryption, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 4315c53080..777eac7f62 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -3116,6 +3116,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(encryption);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 1bcb875507..edc9fb4a28 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -145,6 +145,7 @@ parse_analyze_fixedparams(RawStmt *parseTree, const char *sourceText,
Query *
parse_analyze_varparams(RawStmt *parseTree, const char *sourceText,
Oid **paramTypes, int *numParams,
+ Oid **paramOrigTbls, AttrNumber **paramOrigCols,
QueryEnvironment *queryEnv)
{
ParseState *pstate = make_parsestate(NULL);
@@ -155,7 +156,7 @@ parse_analyze_varparams(RawStmt *parseTree, const char *sourceText,
pstate->p_sourcetext = sourceText;
- setup_parse_variable_parameters(pstate, paramTypes, numParams);
+ setup_parse_variable_parameters(pstate, paramTypes, numParams, paramOrigTbls, paramOrigCols);
pstate->p_queryEnv = queryEnv;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 969c9c158f..a918eaf427 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <node> TableConstraint TableLikeClause
%type <ival> TableLikeOptionList TableLikeOption
%type <str> column_compression opt_column_compression
+%type <list> opt_column_encryption
%type <list> ColQualList
%type <node> ColConstraint ColConstraintElem ConstraintAttr
%type <ival> key_match
@@ -3778,13 +3779,14 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename opt_column_compression create_generic_options ColQualList
+columnDef: ColId Typename opt_column_compression opt_column_encryption create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
n->compression = $3;
+ n->encryption = $4;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3793,8 +3795,8 @@ columnDef: ColId Typename opt_column_compression create_generic_options ColQualL
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $4;
- SplitColQualList($5, &n->constraints, &n->collClause,
+ n->fdwoptions = $5;
+ SplitColQualList($6, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *) n;
@@ -3851,6 +3853,11 @@ opt_column_compression:
| /*EMPTY*/ { $$ = NULL; }
;
+opt_column_encryption:
+ ENCRYPTED WITH '(' def_list ')' { $$ = $4; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
diff --git a/src/backend/parser/parse_param.c b/src/backend/parser/parse_param.c
index f668abfcb3..b45bf4c438 100644
--- a/src/backend/parser/parse_param.c
+++ b/src/backend/parser/parse_param.c
@@ -49,6 +49,8 @@ typedef struct VarParamState
{
Oid **paramTypes; /* array of parameter type OIDs */
int *numParams; /* number of array entries */
+ Oid **paramOrigTbls; /* underlying tables (0 if none) */
+ AttrNumber **paramOrigCols; /* underlying columns (0 if none) */
} VarParamState;
static Node *fixed_paramref_hook(ParseState *pstate, ParamRef *pref);
@@ -56,6 +58,7 @@ static Node *variable_paramref_hook(ParseState *pstate, ParamRef *pref);
static Node *variable_coerce_param_hook(ParseState *pstate, Param *param,
Oid targetTypeId, int32 targetTypeMod,
int location);
+static void variable_param_assign_orig_hook(ParseState *pstate, Param *param, Oid origtbl, AttrNumber origcol);
static bool check_parameter_resolution_walker(Node *node, ParseState *pstate);
static bool query_contains_extern_params_walker(Node *node, void *context);
@@ -81,15 +84,19 @@ setup_parse_fixed_parameters(ParseState *pstate,
*/
void
setup_parse_variable_parameters(ParseState *pstate,
- Oid **paramTypes, int *numParams)
+ Oid **paramTypes, int *numParams,
+ Oid **paramOrigTbls, AttrNumber **paramOrigCols)
{
VarParamState *parstate = palloc(sizeof(VarParamState));
parstate->paramTypes = paramTypes;
parstate->numParams = numParams;
+ parstate->paramOrigTbls = paramOrigTbls;
+ parstate->paramOrigCols = paramOrigCols;
pstate->p_ref_hook_state = (void *) parstate;
pstate->p_paramref_hook = variable_paramref_hook;
pstate->p_coerce_param_hook = variable_coerce_param_hook;
+ pstate->p_param_assign_orig_hook = variable_param_assign_orig_hook;
}
/*
@@ -145,14 +152,36 @@ variable_paramref_hook(ParseState *pstate, ParamRef *pref)
{
/* Need to enlarge param array */
if (*parstate->paramTypes)
+ {
*parstate->paramTypes = (Oid *) repalloc(*parstate->paramTypes,
paramno * sizeof(Oid));
+ if (parstate->paramOrigTbls)
+ *parstate->paramOrigTbls = repalloc(*parstate->paramOrigTbls,
+ paramno * sizeof(Oid));
+ if (parstate->paramOrigCols)
+ *parstate->paramOrigCols = repalloc(*parstate->paramOrigCols,
+ paramno * sizeof(AttrNumber));
+ }
else
+ {
*parstate->paramTypes = (Oid *) palloc(paramno * sizeof(Oid));
+ if (parstate->paramOrigTbls)
+ *parstate->paramOrigTbls = palloc(paramno * sizeof(Oid));
+ if (parstate->paramOrigCols)
+ *parstate->paramOrigCols = palloc(paramno * sizeof(AttrNumber));
+ }
/* Zero out the previously-unreferenced slots */
MemSet(*parstate->paramTypes + *parstate->numParams,
0,
(paramno - *parstate->numParams) * sizeof(Oid));
+ if (parstate->paramOrigTbls)
+ MemSet(*parstate->paramOrigTbls + *parstate->numParams,
+ 0,
+ (paramno - *parstate->numParams) * sizeof(Oid));
+ if (parstate->paramOrigCols)
+ MemSet(*parstate->paramOrigCols + *parstate->numParams,
+ 0,
+ (paramno - *parstate->numParams) * sizeof(AttrNumber));
*parstate->numParams = paramno;
}
@@ -260,6 +289,18 @@ variable_coerce_param_hook(ParseState *pstate, Param *param,
return NULL;
}
+static void
+variable_param_assign_orig_hook(ParseState *pstate, Param *param, Oid origtbl, AttrNumber origcol)
+{
+ VarParamState *parstate = (VarParamState *) pstate->p_ref_hook_state;
+ int paramno = param->paramid;
+
+ if (parstate->paramOrigTbls)
+ (*parstate->paramOrigTbls)[paramno - 1] = origtbl;
+ if (parstate->paramOrigCols)
+ (*parstate->paramOrigCols)[paramno - 1] = origcol;
+}
+
/*
* Check for consistent assignment of variable parameters after completion
* of parsing with parse_variable_parameters.
diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c
index 2a1d44b813..b964088044 100644
--- a/src/backend/parser/parse_target.c
+++ b/src/backend/parser/parse_target.c
@@ -595,6 +595,12 @@ transformAssignedExpr(ParseState *pstate,
parser_errposition(pstate, exprLocation(orig_expr))));
}
+ if (IsA(expr, Param))
+ {
+ if (pstate->p_param_assign_orig_hook)
+ pstate->p_param_assign_orig_hook(pstate, castNode(Param, expr), RelationGetRelid(pstate->p_target_relation), attrno);
+ }
+
pstate->p_expr_kind = sv_expr_kind;
return expr;
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 495cbf2006..82168235e8 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -79,6 +79,7 @@
#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
#include "utils/timeout.h"
#include "utils/timestamp.h"
@@ -680,6 +681,8 @@ pg_analyze_and_rewrite_varparams(RawStmt *parsetree,
const char *query_string,
Oid **paramTypes,
int *numParams,
+ Oid **paramOrigTbls,
+ AttrNumber **paramOrigCols,
QueryEnvironment *queryEnv)
{
Query *query;
@@ -694,7 +697,7 @@ pg_analyze_and_rewrite_varparams(RawStmt *parsetree,
ResetUsage();
query = parse_analyze_varparams(parsetree, query_string, paramTypes, numParams,
- queryEnv);
+ paramOrigTbls, paramOrigCols, queryEnv);
/*
* Check all parameter types got determined.
@@ -1371,6 +1374,8 @@ exec_parse_message(const char *query_string, /* string to execute */
bool is_named;
bool save_log_statement_stats = log_statement_stats;
char msec_str[32];
+ Oid *paramOrigTbls = palloc(numParams * sizeof(Oid));
+ AttrNumber *paramOrigCols = palloc(numParams * sizeof(AttrNumber));
/*
* Report query to various monitoring facilities.
@@ -1491,6 +1496,8 @@ exec_parse_message(const char *query_string, /* string to execute */
query_string,
¶mTypes,
&numParams,
+ ¶mOrigTbls,
+ ¶mOrigCols,
NULL);
/* Done with the snapshot used for parsing */
@@ -1521,6 +1528,8 @@ exec_parse_message(const char *query_string, /* string to execute */
unnamed_stmt_context,
paramTypes,
numParams,
+ paramOrigTbls,
+ paramOrigCols,
NULL,
NULL,
CURSOR_OPT_PARALLEL_OK, /* allow parallel mode */
@@ -1818,6 +1827,16 @@ exec_bind_message(StringInfo input_message)
else
pformat = 0; /* default = text */
+ if (get_typtype(ptype) == TYPTYPE_ENCRYPTED)
+ {
+ if (pformat & 0xF0)
+ pformat &= ~0xF0;
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("parameter corresponds to an encrypted column, but the parameter value was not encrypted")));
+ }
+
if (pformat == 0) /* text mode */
{
Oid typinput;
@@ -2614,8 +2633,41 @@ exec_describe_statement_message(const char *stmt_name)
for (int i = 0; i < psrc->num_params; i++)
{
Oid ptype = psrc->param_types[i];
+ Oid pcekid = InvalidOid;
+ int pcekalg = 0;
+ int16 pflags = 0;
+
+ if (get_typtype(ptype) == TYPTYPE_ENCRYPTED)
+ {
+ Oid porigtbl = psrc->param_origtbls[i];
+ AttrNumber porigcol = psrc->param_origcols[i];
+ HeapTuple tp;
+ Form_pg_attribute orig_att;
+
+ if (porigtbl == InvalidOid || porigcol <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("parameter %d corresponds to an encrypted column, but an underlying table and column could not be determined", i)));
+
+ tp = SearchSysCache2(ATTNUM, ObjectIdGetDatum(porigtbl), Int16GetDatum(porigcol));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u", porigcol, porigtbl);
+ orig_att = (Form_pg_attribute) GETSTRUCT(tp);
+ ptype = orig_att->attrealtypid;
+ pcekid = orig_att->attcek;
+ pcekalg = orig_att->attencalg;
+ ReleaseSysCache(tp);
+
+ if (psrc->param_types[i] == PG_ENCRYPTED_DETOID)
+ pflags |= 0x01;
+
+ MaybeSendColumnEncryptionKeyMessage(pcekid);
+ }
pq_sendint32(&row_description_buf, (int) ptype);
+ pq_sendint32(&row_description_buf, (int) pcekid);
+ pq_sendint16(&row_description_buf, pcekalg);
+ pq_sendint16(&row_description_buf, pflags);
}
pq_endmessage_reuse(&row_description_buf);
diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c
index b0c37ede87..5683731188 100644
--- a/src/backend/utils/adt/arrayfuncs.c
+++ b/src/backend/utils/adt/arrayfuncs.c
@@ -3387,6 +3387,7 @@ construct_array_builtin(Datum *elems, int nelems, Oid elmtype)
break;
case OIDOID:
+ case REGCLASSOID:
case REGTYPEOID:
elmlen = sizeof(Oid);
elmbyval = true;
diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c
index 0d6a295674..f6893dd8b7 100644
--- a/src/backend/utils/cache/plancache.c
+++ b/src/backend/utils/cache/plancache.c
@@ -340,6 +340,8 @@ CompleteCachedPlan(CachedPlanSource *plansource,
MemoryContext querytree_context,
Oid *param_types,
int num_params,
+ Oid *param_origtbls,
+ AttrNumber *param_origcols,
ParserSetupHook parserSetup,
void *parserSetupArg,
int cursor_options,
@@ -419,6 +421,14 @@ CompleteCachedPlan(CachedPlanSource *plansource,
{
plansource->param_types = (Oid *) palloc(num_params * sizeof(Oid));
memcpy(plansource->param_types, param_types, num_params * sizeof(Oid));
+
+ plansource->param_origtbls = (Oid *) palloc0(num_params * sizeof(Oid));
+ if (param_origtbls)
+ memcpy(plansource->param_origtbls, param_origtbls, num_params * sizeof(Oid));
+
+ plansource->param_origcols = (AttrNumber *) palloc0(num_params * sizeof(AttrNumber));
+ if (param_origcols)
+ memcpy(plansource->param_origcols, param_origcols, num_params * sizeof(AttrNumber));
}
else
plansource->param_types = NULL;
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 1912b12146..468a6873e4 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -29,7 +29,10 @@
#include "catalog/pg_auth_members.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
+#include "catalog/pg_colenckey.h"
+#include "catalog/pg_colenckeydata.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_colmasterkey.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -267,6 +270,33 @@ static const struct cachedesc cacheinfo[] = {
},
256
},
+ {
+ ColumnEncKeyDataRelationId, /* CEKDATAOID */
+ ColumnEncKeyDataOidIndexId,
+ 1,
+ {
+ Anum_pg_colenckeydata_oid,
+ },
+ 8
+ },
+ {
+ ColumnEncKeyRelationId, /* CEKNAME */
+ ColumnEncKeyNameIndexId,
+ 1,
+ {
+ Anum_pg_colenckey_cekname,
+ },
+ 8
+ },
+ {
+ ColumnEncKeyRelationId, /* CEKOID */
+ ColumnEncKeyOidIndexId,
+ 1,
+ {
+ Anum_pg_colenckey_oid,
+ },
+ 8
+ },
{OperatorClassRelationId, /* CLAAMNAMENSP */
OpclassAmNameNspIndexId,
3,
@@ -289,6 +319,15 @@ static const struct cachedesc cacheinfo[] = {
},
8
},
+ {
+ ColumnMasterKeyRelationId, /* CMKOID */
+ ColumnMasterKeyOidIndexId,
+ 1,
+ {
+ Anum_pg_colmasterkey_oid,
+ },
+ 8
+ },
{CollationRelationId, /* COLLNAMEENCNSP */
CollationNameEncNspIndexId,
3,
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index c871cb727d..962947c78d 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -8066,6 +8066,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_typstorage;
int i_attidentity;
int i_attgenerated;
+ int i_attcekname;
int i_attisdropped;
int i_attlen;
int i_attalign;
@@ -8175,17 +8176,24 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
if (fout->remoteVersion >= 120000)
appendPQExpBufferStr(q,
- "a.attgenerated\n");
+ "a.attgenerated,\n");
else
appendPQExpBufferStr(q,
- "'' AS attgenerated\n");
+ "'' AS attgenerated,\n");
+
+ if (fout->remoteVersion >= 160000)
+ appendPQExpBufferStr(q,
+ "(SELECT cekname FROM pg_colenckey cek WHERE cek.oid = a.attcek ) AS attcekname\n");
+ else
+ appendPQExpBufferStr(q,
+ "NULL AS attcekname\n");
/* need left join to pg_type to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM unnest('%s'::pg_catalog.oid[]) AS src(tbloid)\n"
"JOIN pg_catalog.pg_attribute a ON (src.tbloid = a.attrelid) "
"LEFT JOIN pg_catalog.pg_type t "
- "ON (a.atttypid = t.oid)\n"
+ "ON (CASE WHEN a.attrealtypid <> 0 THEN a.attrealtypid ELSE a.atttypid END = t.oid)\n"
"WHERE a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attrelid, a.attnum",
tbloids->data);
@@ -8204,6 +8212,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_typstorage = PQfnumber(res, "typstorage");
i_attidentity = PQfnumber(res, "attidentity");
i_attgenerated = PQfnumber(res, "attgenerated");
+ i_attcekname = PQfnumber(res, "attcekname");
i_attisdropped = PQfnumber(res, "attisdropped");
i_attlen = PQfnumber(res, "attlen");
i_attalign = PQfnumber(res, "attalign");
@@ -8265,6 +8274,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->typstorage = (char *) pg_malloc(numatts * sizeof(char));
tbinfo->attidentity = (char *) pg_malloc(numatts * sizeof(char));
tbinfo->attgenerated = (char *) pg_malloc(numatts * sizeof(char));
+ tbinfo->attcekname = (char **) pg_malloc(numatts * sizeof(char *));
tbinfo->attisdropped = (bool *) pg_malloc(numatts * sizeof(bool));
tbinfo->attlen = (int *) pg_malloc(numatts * sizeof(int));
tbinfo->attalign = (char *) pg_malloc(numatts * sizeof(char));
@@ -8293,6 +8303,10 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attidentity[j] = *(PQgetvalue(res, r, i_attidentity));
tbinfo->attgenerated[j] = *(PQgetvalue(res, r, i_attgenerated));
tbinfo->needs_override = tbinfo->needs_override || (tbinfo->attidentity[j] == ATTRIBUTE_IDENTITY_ALWAYS);
+ if (!PQgetisnull(res, r, i_attcekname))
+ tbinfo->attcekname[j] = pg_strdup(PQgetvalue(res, r, i_attcekname));
+ else
+ tbinfo->attcekname[j] = NULL;
tbinfo->attisdropped[j] = (PQgetvalue(res, r, i_attisdropped)[0] == 't');
tbinfo->attlen[j] = atoi(PQgetvalue(res, r, i_attlen));
tbinfo->attalign[j] = *(PQgetvalue(res, r, i_attalign));
@@ -15260,6 +15274,12 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ if (tbinfo->attcekname[j])
+ {
+ appendPQExpBuffer(q, " ENCRYPTED WITH (column_encryption_key = %s)",
+ fmtId(tbinfo->attcekname[j]));
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1d21c2906f..2c176ed13c 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -333,6 +333,7 @@ typedef struct _tableInfo
bool *attisdropped; /* true if attr is dropped; don't dump it */
char *attidentity;
char *attgenerated;
+ char **attcekname;
int *attlen; /* attribute length, used by binary_upgrade */
char *attalign; /* attribute align, used by binary_upgrade */
bool *attislocal; /* true if attr has local definition */
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index c562c04afe..72e973d2de 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -98,6 +98,7 @@ static backslashResult process_command_g_options(char *first_option,
bool active_branch,
const char *cmd);
static backslashResult exec_command_gdesc(PsqlScanState scan_state, bool active_branch);
+static backslashResult exec_command_gencr(PsqlScanState scan_state, bool active_branch);
static backslashResult exec_command_getenv(PsqlScanState scan_state, bool active_branch,
const char *cmd);
static backslashResult exec_command_gexec(PsqlScanState scan_state, bool active_branch);
@@ -350,6 +351,8 @@ exec_command(const char *cmd,
status = exec_command_g(scan_state, active_branch, cmd);
else if (strcmp(cmd, "gdesc") == 0)
status = exec_command_gdesc(scan_state, active_branch);
+ else if (strcmp(cmd, "gencr") == 0)
+ status = exec_command_gencr(scan_state, active_branch);
else if (strcmp(cmd, "getenv") == 0)
status = exec_command_getenv(scan_state, active_branch, cmd);
else if (strcmp(cmd, "gexec") == 0)
@@ -1492,6 +1495,34 @@ exec_command_gdesc(PsqlScanState scan_state, bool active_branch)
return status;
}
+/*
+ * \gencr -- send query, with support for parameter encryption
+ */
+static backslashResult
+exec_command_gencr(PsqlScanState scan_state, bool active_branch)
+{
+ backslashResult status = PSQL_CMD_SKIP_LINE;
+ char *ap;
+
+ pset.num_params = 0;
+ pset.params = NULL;
+ while ((ap = psql_scan_slash_option(scan_state,
+ OT_NORMAL, NULL, true)) != NULL)
+ {
+ pset.num_params++;
+ pset.params = pg_realloc(pset.params, pset.num_params * sizeof(char *));
+ pset.params[pset.num_params - 1] = ap;
+ }
+
+ if (active_branch)
+ {
+ pset.gencr_flag = true;
+ status = PSQL_CMD_SEND;
+ }
+
+ return status;
+}
+
/*
* \getenv -- set variable from environment variable
*/
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index 9f95869eca..8fe055ffed 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1284,6 +1284,14 @@ SendQuery(const char *query)
/* reset \gdesc trigger */
pset.gdesc_flag = false;
+ /* reset \gencr trigger */
+ pset.gencr_flag = false;
+ for (int i = 0; i < pset.num_params; i++)
+ pg_free(pset.params[i]);
+ pg_free(pset.params);
+ pset.params = NULL;
+ pset.num_params = 0;
+
/* reset \gexec trigger */
pset.gexec_flag = false;
@@ -1448,7 +1456,36 @@ ExecQueryAndProcessResults(const char *query, double *elapsed_msec, bool *svpt_g
if (timing)
INSTR_TIME_SET_CURRENT(before);
- success = PQsendQuery(pset.db, query);
+ if (pset.gencr_flag)
+ {
+ PGresult *res1,
+ *res2;
+
+ res1 = PQprepare(pset.db, "", query, pset.num_params, NULL);
+ if (PQresultStatus(res1) != PGRES_COMMAND_OK)
+ {
+ pg_log_info("%s", PQerrorMessage(pset.db));
+ ClearOrSaveResult(res1);
+ return -1;
+ }
+ PQclear(res1);
+
+ res2 = PQdescribePrepared(pset.db, "");
+ if (PQresultStatus(res2) != PGRES_COMMAND_OK)
+ {
+ pg_log_info("%s", PQerrorMessage(pset.db));
+ ClearOrSaveResult(res2);
+ return -1;
+ }
+
+ success = PQsendQueryPrepared2(pset.db, "", pset.num_params, (const char *const *) pset.params, NULL, NULL, NULL, 0, res2);
+
+ PQclear(res2);
+ }
+ else
+ {
+ success = PQsendQuery(pset.db, query);
+ }
if (!success)
{
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 88d92a08ae..fe81547b29 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1498,7 +1498,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[12];
+ char *headers[13];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1514,6 +1514,7 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attcompression_col = -1,
+ attcekname_col = -1,
attstattarget_col = -1,
attdescr_col = -1;
int numrows;
@@ -1812,7 +1813,7 @@ describeOneTableDetails(const char *schemaname,
cols = 0;
printfPQExpBuffer(&buf, "SELECT a.attname");
attname_col = cols++;
- appendPQExpBufferStr(&buf, ",\n pg_catalog.format_type(a.atttypid, a.atttypmod)");
+ appendPQExpBufferStr(&buf, ",\n pg_catalog.format_type(CASE WHEN a.attrealtypid <> 0 THEN a.attrealtypid ELSE a.atttypid END, a.atttypmod)");
atttype_col = cols++;
if (show_column_details)
@@ -1826,7 +1827,7 @@ describeOneTableDetails(const char *schemaname,
attrdef_col = cols++;
attnotnull_col = cols++;
appendPQExpBufferStr(&buf, ",\n (SELECT c.collname FROM pg_catalog.pg_collation c, pg_catalog.pg_type t\n"
- " WHERE c.oid = a.attcollation AND t.oid = a.atttypid AND a.attcollation <> t.typcollation) AS attcollation");
+ " WHERE c.oid = a.attcollation AND t.oid = (CASE WHEN a.attrealtypid <> 0 THEN a.attrealtypid ELSE a.atttypid END) AND a.attcollation <> t.typcollation) AS attcollation");
attcoll_col = cols++;
if (pset.sversion >= 100000)
appendPQExpBufferStr(&buf, ",\n a.attidentity");
@@ -1877,6 +1878,15 @@ describeOneTableDetails(const char *schemaname,
attcompression_col = cols++;
}
+ /* encryption info */
+ if (pset.sversion >= 160000 &&
+ !pset.hide_column_encryption &&
+ (tableinfo.relkind == RELKIND_RELATION))
+ {
+ appendPQExpBufferStr(&buf, ",\n (SELECT cekname FROM pg_colenckey cek WHERE cek.oid = a.attcek) AS attcekname");
+ attcekname_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2000,6 +2010,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("Storage");
if (attcompression_col >= 0)
headers[cols++] = gettext_noop("Compression");
+ if (attcekname_col >= 0)
+ headers[cols++] = gettext_noop("Encryption");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2092,6 +2104,17 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column encryption */
+ if (attcekname_col >= 0)
+ {
+ if (!PQgetisnull(res, i, attcekname_col))
+ printTableAddCell(&cont, PQgetvalue(res, i, attcekname_col),
+ false, false);
+ else
+ printTableAddCell(&cont, "",
+ false, false);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index f8ce1a0706..9f4fce26fd 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -195,6 +195,7 @@ slashUsage(unsigned short int pager)
HELP0(" \\g [(OPTIONS)] [FILE] execute query (and send result to file or |pipe);\n"
" \\g with no arguments is equivalent to a semicolon\n");
HELP0(" \\gdesc describe result of query, without executing it\n");
+ HELP0(" \\gencr [PARAM]... execute query, with support for parameter encryption\n");
HELP0(" \\gexec execute query, then execute each value in its result\n");
HELP0(" \\gset [PREFIX] execute query and store result in psql variables\n");
HELP0(" \\gx [(OPTIONS)] [FILE] as \\g, but forces expanded output mode\n");
@@ -412,6 +413,8 @@ helpVariables(unsigned short int pager)
" true if last query failed, else false\n");
HELP0(" FETCH_COUNT\n"
" the number of result rows to fetch and display at a time (0 = unlimited)\n");
+ HELP0(" HIDE_COLUMN_ENCRYPTION\n"
+ " if set, column encryption details are not displayed\n");
HELP0(" HIDE_TABLEAM\n"
" if set, table access methods are not displayed\n");
HELP0(" HIDE_TOAST_COMPRESSION\n"
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index 2399cffa3f..636729df1d 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -95,6 +95,10 @@ typedef struct _psqlSettings
char *gset_prefix; /* one-shot prefix argument for \gset */
bool gdesc_flag; /* one-shot request to describe query result */
+ bool gencr_flag; /* one-shot request to send query with support
+ * for parameter encryption */
+ int num_params; /* number of query parameters */
+ char **params; /* query parameters */
bool gexec_flag; /* one-shot request to execute query result */
bool crosstab_flag; /* one-shot request to crosstab result */
char *ctv_args[4]; /* \crosstabview arguments */
@@ -134,6 +138,7 @@ typedef struct _psqlSettings
bool quiet;
bool singleline;
bool singlestep;
+ bool hide_column_encryption;
bool hide_compression;
bool hide_tableam;
int fetch_count;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 7c2f555f15..c955222a50 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -1188,6 +1188,13 @@ hide_compression_hook(const char *newval)
&pset.hide_compression);
}
+static bool
+hide_column_encryption_hook(const char *newval)
+{
+ return ParseVariableBool(newval, "HIDE_COLUMN_ENCRYPTION",
+ &pset.hide_column_encryption);
+}
+
static bool
hide_tableam_hook(const char *newval)
{
@@ -1259,6 +1266,9 @@ EstablishVariableSpace(void)
SetVariableHooks(pset.vars, "SHOW_CONTEXT",
show_context_substitute_hook,
show_context_hook);
+ SetVariableHooks(pset.vars, "HIDE_COLUMN_ENCRYPTION",
+ bool_substitute_hook,
+ hide_column_encryption_hook);
SetVariableHooks(pset.vars, "HIDE_TOAST_COMPRESSION",
bool_substitute_hook,
hide_compression_hook);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index c5cafe6f4b..d5877dc5a3 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1692,7 +1692,7 @@ psql_completion(const char *text, int start, int end)
"\\echo", "\\edit", "\\ef", "\\elif", "\\else", "\\encoding",
"\\endif", "\\errverbose", "\\ev",
"\\f",
- "\\g", "\\gdesc", "\\getenv", "\\gexec", "\\gset", "\\gx",
+ "\\g", "\\gdesc", "\\gencr", "\\getenv", "\\gexec", "\\gset", "\\gx",
"\\help", "\\html",
"\\if", "\\include", "\\include_relative", "\\ir",
"\\list", "\\lo_import", "\\lo_export", "\\lo_list", "\\lo_unlink",
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index b5fd72a4ac..5a065eb80b 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -54,6 +54,7 @@ endif
ifeq ($(with_ssl),openssl)
OBJS += \
+ fe-encrypt-openssl.o \
fe-secure-openssl.o
endif
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index e8bcc88370..fabad38ac0 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -186,3 +186,7 @@ PQpipelineStatus 183
PQsetTraceFlags 184
PQmblenBounded 185
PQsendFlushRequest 186
+PQexecPrepared2 187
+PQsendQueryPrepared2 188
+PQfisencrypted 189
+PQparamisencrypted 190
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index dc49387d6c..f3c15ece4d 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -344,6 +344,10 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"Target-Session-Attrs", "", 15, /* sizeof("prefer-standby") = 15 */
offsetof(struct pg_conn, target_session_attrs)},
+ {"cmklookup", "PGCMKLOOKUP", "", NULL,
+ "CMK-Lookup", "", 64,
+ offsetof(struct pg_conn, cmklookup)},
+
/* Terminating entry --- MUST BE LAST */
{NULL, NULL, NULL, NULL,
NULL, NULL, 0}
@@ -4097,6 +4101,22 @@ freePGconn(PGconn *conn)
free(conn->krbsrvname);
free(conn->gsslib);
free(conn->connip);
+ free(conn->cmklookup);
+ for (int i = 0; i < conn->ncmks; i++)
+ {
+ free(conn->cmks[i].cmkname);
+ free(conn->cmks[i].cmkrealm);
+ }
+ free(conn->cmks);
+ for (int i = 0; i < conn->nceks; i++)
+ {
+ if (conn->ceks[i].cekdata)
+ {
+ explicit_bzero(conn->ceks[i].cekdata, conn->ceks[i].cekdatalen);
+ free(conn->ceks[i].cekdata);
+ }
+ }
+ free(conn->ceks);
/* Note that conn->Pfdebug is not ours to close or free */
free(conn->write_err_msg);
free(conn->inBuffer);
diff --git a/src/interfaces/libpq/fe-encrypt-openssl.c b/src/interfaces/libpq/fe-encrypt-openssl.c
new file mode 100644
index 0000000000..524ba8a0f9
--- /dev/null
+++ b/src/interfaces/libpq/fe-encrypt-openssl.c
@@ -0,0 +1,758 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-encrypt-openssl.c
+ * encryption support using OpenSSL
+ *
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-encrypt-openssl.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "fe-encrypt.h"
+#include "libpq-int.h"
+
+#include "catalog/pg_colenckey.h"
+#include "port/pg_bswap.h"
+
+#include <openssl/evp.h>
+
+
+#ifdef TEST_ENCRYPT
+
+/*
+ * Test data from
+ * <https://datatracker.ietf.org/doc/html/draft-mcgrew-aead-aes-cbc-hmac-sha2-05#section-5;
+ */
+
+/*
+ * The different test cases just use different prefixes of K, so one constant
+ * is enough here.
+ */
+static const unsigned char K[] = {
+ 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
+ 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f,
+ 0x20, 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28, 0x29, 0x2a, 0x2b, 0x2c, 0x2d, 0x2e, 0x2f,
+ 0x30, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x3a, 0x3b, 0x3c, 0x3d, 0x3e, 0x3f,
+};
+
+static const unsigned char P[] = {
+ 0x41, 0x20, 0x63, 0x69, 0x70, 0x68, 0x65, 0x72, 0x20, 0x73, 0x79, 0x73, 0x74, 0x65, 0x6d, 0x20,
+ 0x6d, 0x75, 0x73, 0x74, 0x20, 0x6e, 0x6f, 0x74, 0x20, 0x62, 0x65, 0x20, 0x72, 0x65, 0x71, 0x75,
+ 0x69, 0x72, 0x65, 0x64, 0x20, 0x74, 0x6f, 0x20, 0x62, 0x65, 0x20, 0x73, 0x65, 0x63, 0x72, 0x65,
+ 0x74, 0x2c, 0x20, 0x61, 0x6e, 0x64, 0x20, 0x69, 0x74, 0x20, 0x6d, 0x75, 0x73, 0x74, 0x20, 0x62,
+ 0x65, 0x20, 0x61, 0x62, 0x6c, 0x65, 0x20, 0x74, 0x6f, 0x20, 0x66, 0x61, 0x6c, 0x6c, 0x20, 0x69,
+ 0x6e, 0x74, 0x6f, 0x20, 0x74, 0x68, 0x65, 0x20, 0x68, 0x61, 0x6e, 0x64, 0x73, 0x20, 0x6f, 0x66,
+ 0x20, 0x74, 0x68, 0x65, 0x20, 0x65, 0x6e, 0x65, 0x6d, 0x79, 0x20, 0x77, 0x69, 0x74, 0x68, 0x6f,
+ 0x75, 0x74, 0x20, 0x69, 0x6e, 0x63, 0x6f, 0x6e, 0x76, 0x65, 0x6e, 0x69, 0x65, 0x6e, 0x63, 0x65,
+};
+
+static const unsigned char test_IV[] = {
+ 0x1a, 0xf3, 0x8c, 0x2d, 0xc2, 0xb9, 0x6f, 0xfd, 0xd8, 0x66, 0x94, 0x09, 0x23, 0x41, 0xbc, 0x04,
+};
+
+static const unsigned char test_A[] = {
+ 0x54, 0x68, 0x65, 0x20, 0x73, 0x65, 0x63, 0x6f, 0x6e, 0x64, 0x20, 0x70, 0x72, 0x69, 0x6e, 0x63,
+ 0x69, 0x70, 0x6c, 0x65, 0x20, 0x6f, 0x66, 0x20, 0x41, 0x75, 0x67, 0x75, 0x73, 0x74, 0x65, 0x20,
+ 0x4b, 0x65, 0x72, 0x63, 0x6b, 0x68, 0x6f, 0x66, 0x66, 0x73,
+};
+
+
+#define libpq_gettext(x) (x)
+
+#endif /* TEST_ENCRYPT */
+
+
+unsigned char *
+decrypt_cek_from_file(PGconn *conn, FILE *fp, int cmkalg,
+ int fromlen, const unsigned char *from,
+ int *tolen)
+{
+ const EVP_MD *md = NULL;
+ EVP_PKEY *key = NULL;
+ RSA *rsa = NULL;
+ EVP_PKEY_CTX *ctx = NULL;
+ unsigned char *out = NULL;
+ size_t outlen;
+
+ switch (cmkalg)
+ {
+ case PG_CMK_RSAES_OAEP_SHA_1:
+ md = EVP_sha1();
+ break;
+ case PG_CMK_RSAES_OAEP_SHA_256:
+ md = EVP_sha256();
+ break;
+ default:
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("unsupported CMK algorithm ID: %d\n"), cmkalg);
+ goto fail;
+ }
+
+ rsa = RSA_new();
+ if (!rsa)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not allocate RSA structure: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+ rsa = PEM_read_RSAPrivateKey(fp, &rsa, NULL, NULL);
+ if (!rsa)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not read RSA private key: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+
+ key = EVP_PKEY_new();
+ if (!key)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not allocate private key structure: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+
+ if (!EVP_PKEY_assign_RSA(key, rsa))
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not assign private key: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+
+ ctx = EVP_PKEY_CTX_new(key, NULL);
+ if (!ctx)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not allocate public key algorithm context: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+
+ if (EVP_PKEY_decrypt_init(ctx) <= 0)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("decryption initialization failed: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+
+ if (EVP_PKEY_CTX_set_rsa_padding(ctx, RSA_PKCS1_OAEP_PADDING) <= 0 ||
+ EVP_PKEY_CTX_set_rsa_mgf1_md(ctx, md) <= 0 ||
+ EVP_PKEY_CTX_set_rsa_oaep_md(ctx, md) <= 0)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not set RSA parameter: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+
+ if (EVP_PKEY_decrypt(ctx, NULL, &outlen, from, fromlen) <= 0)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("RSA decryption failed: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+
+ out = malloc(outlen);
+ if (!out)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("out of memory\n"));
+ goto fail;
+ }
+
+ if (EVP_PKEY_decrypt(ctx, out, &outlen, from, fromlen) <= 0)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("RSA decryption failed: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ free(out);
+ out = NULL;
+ goto fail;
+ }
+
+ *tolen = outlen;
+
+fail:
+ EVP_PKEY_CTX_free(ctx);
+ EVP_PKEY_free(key);
+
+ return out;
+}
+
+static const EVP_CIPHER *
+pg_cekalg_to_openssl_cipher(int cekalg)
+{
+ const EVP_CIPHER *cipher;
+
+ switch (cekalg)
+ {
+ case PG_CEK_AEAD_AES_128_CBC_HMAC_SHA_256:
+ cipher = EVP_aes_128_cbc();
+ break;
+ case PG_CEK_AEAD_AES_192_CBC_HMAC_SHA_384:
+ cipher = EVP_aes_192_cbc();
+ break;
+ case PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_384:
+ case PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_512:
+ cipher = EVP_aes_256_cbc();
+ break;
+ default:
+ cipher = NULL;
+ }
+
+ return cipher;
+}
+
+static const EVP_MD *
+pg_cekalg_to_openssl_md(int cekalg)
+{
+ const EVP_MD *md;
+
+ switch (cekalg)
+ {
+ case PG_CEK_AEAD_AES_128_CBC_HMAC_SHA_256:
+ md = EVP_sha256();
+ break;
+ case PG_CEK_AEAD_AES_192_CBC_HMAC_SHA_384:
+ case PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_384:
+ md = EVP_sha384();
+ break;
+ case PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_512:
+ md = EVP_sha512();
+ break;
+ default:
+ md = NULL;
+ }
+
+ return md;
+}
+
+static int
+md_key_length(const EVP_MD *md)
+{
+ if (md == EVP_sha256())
+ return 16;
+ else if (md == EVP_sha384())
+ return 24;
+ else if (md == EVP_sha512())
+ return 32;
+ else
+ return -1;
+}
+
+static int
+md_hash_length(const EVP_MD *md)
+{
+ if (md == EVP_sha256())
+ return 32;
+ else if (md == EVP_sha384())
+ return 48;
+ else if (md == EVP_sha512())
+ return 64;
+ else
+ return -1;
+}
+
+#ifndef TEST_ENCRYPT
+#define PG_AD_LEN 4
+#else
+#define PG_AD_LEN sizeof(test_A)
+#endif
+
+static bool
+get_message_auth_tag(const EVP_MD *md,
+ const unsigned char *mac_key, int mac_key_len,
+ const unsigned char *encr, int encrlen,
+ int cekalg,
+ unsigned char *md_value, size_t *md_len_p,
+ const char **errmsgp)
+{
+ static char msgbuf[1024];
+ EVP_MD_CTX *evp_md_ctx = NULL;
+ EVP_PKEY *pkey = NULL;
+ size_t bufsize;
+ unsigned char *buf = NULL;
+ bool result = false;
+
+ evp_md_ctx = EVP_MD_CTX_new();
+
+ pkey = EVP_PKEY_new_raw_private_key(EVP_PKEY_HMAC, NULL, mac_key, mac_key_len);
+ if (!pkey)
+ {
+ snprintf(msgbuf, sizeof(msgbuf),
+ libpq_gettext("could not allocate key for HMAC: %s"),
+ ERR_reason_error_string(ERR_get_error()));
+ *errmsgp = msgbuf;
+ goto fail;
+ }
+
+ bufsize = PG_AD_LEN + encrlen + sizeof(int64);
+ buf = malloc(bufsize);
+ if (!buf)
+ {
+ *errmsgp = libpq_gettext("out out memory");
+ goto fail;
+ }
+#ifndef TEST_ENCRYPT
+ buf[0] = 'P';
+ buf[1] = 'G';
+ *(int16 *) (buf + 2) = pg_hton16(cekalg);
+#else
+ memcpy(buf, test_A, sizeof(test_A));
+#endif
+ memcpy(buf + PG_AD_LEN, encr, encrlen);
+ *(int64 *) (buf + PG_AD_LEN + encrlen) = pg_hton64(PG_AD_LEN * 8);
+
+ if (!EVP_DigestSignInit(evp_md_ctx, NULL, md, NULL, pkey))
+ {
+ snprintf(msgbuf, sizeof(msgbuf),
+ libpq_gettext("digest initialization failed: %s"),
+ ERR_reason_error_string(ERR_get_error()));
+ *errmsgp = msgbuf;
+ goto fail;
+ }
+
+ if (!EVP_DigestSignUpdate(evp_md_ctx, buf, bufsize))
+ {
+ snprintf(msgbuf, sizeof(msgbuf),
+ libpq_gettext("digest signing failed: %s"),
+ ERR_reason_error_string(ERR_get_error()));
+ *errmsgp = msgbuf;
+ goto fail;
+ }
+
+ if (!EVP_DigestSignFinal(evp_md_ctx, md_value, md_len_p))
+ {
+ snprintf(msgbuf, sizeof(msgbuf),
+ libpq_gettext("digest signing failed: %s"),
+ ERR_reason_error_string(ERR_get_error()));
+ *errmsgp = msgbuf;
+ goto fail;
+ }
+ Assert(*md_len_p == md_hash_length(md));
+
+ /* truncate output to half the length, per spec */
+ *md_len_p /= 2;
+
+ result = true;
+fail:
+ free(buf);
+ EVP_PKEY_free(pkey);
+ EVP_MD_CTX_free(evp_md_ctx);
+ return result;
+}
+
+unsigned char *
+decrypt_value(PGresult *res, const PGCEK *cek, int cekalg, const unsigned char *input, int inputlen, const char **errmsgp)
+{
+ static char msgbuf[1024];
+
+ const unsigned char *iv = NULL;
+ size_t ivlen;
+
+ const EVP_CIPHER *cipher;
+ const EVP_MD *md;
+ EVP_CIPHER_CTX *evp_cipher_ctx = NULL;
+ int enc_key_len;
+ int mac_key_len;
+ int key_len;
+ const unsigned char *enc_key;
+ const unsigned char *mac_key;
+ unsigned char md_value[EVP_MAX_MD_SIZE];
+ size_t md_len = sizeof(md_value);
+ size_t bufsize;
+ unsigned char *buf = NULL;
+ unsigned char *decr;
+ int decrlen,
+ decrlen2;
+
+ unsigned char *result = NULL;
+
+ cipher = pg_cekalg_to_openssl_cipher(cekalg);
+ if (!cipher)
+ {
+ snprintf(msgbuf, sizeof(msgbuf),
+ libpq_gettext("unrecognized encryption algorithm identifier: %d"), cekalg);
+ *errmsgp = msgbuf;
+ goto fail;
+ }
+
+ md = pg_cekalg_to_openssl_md(cekalg);
+ if (!md)
+ {
+ snprintf(msgbuf, sizeof(msgbuf),
+ libpq_gettext("unrecognized digest algorithm identifier: %d"), cekalg);
+ *errmsgp = msgbuf;
+ goto fail;
+ }
+
+ evp_cipher_ctx = EVP_CIPHER_CTX_new();
+
+ if (!EVP_DecryptInit_ex(evp_cipher_ctx, cipher, NULL, NULL, NULL))
+ {
+ snprintf(msgbuf, sizeof(msgbuf),
+ libpq_gettext("decryption initialization failed: %s"),
+ ERR_reason_error_string(ERR_get_error()));
+ *errmsgp = msgbuf;
+ goto fail;
+ }
+
+ enc_key_len = EVP_CIPHER_CTX_key_length(evp_cipher_ctx);
+ mac_key_len = md_key_length(md);
+ key_len = mac_key_len + enc_key_len;
+
+ if (cek->cekdatalen != key_len)
+ {
+ snprintf(msgbuf, sizeof(msgbuf),
+ libpq_gettext("column encryption key has wrong length for algorithm (has: %zu, required: %d)"),
+ cek->cekdatalen, key_len);
+ *errmsgp = msgbuf;
+ goto fail;
+ }
+
+ enc_key = cek->cekdata + mac_key_len;
+ mac_key = cek->cekdata;
+
+ if (!get_message_auth_tag(md, mac_key, mac_key_len,
+ input, inputlen - (md_hash_length(md) / 2),
+ cekalg,
+ md_value, &md_len,
+ errmsgp))
+ {
+ goto fail;
+ }
+
+ if (memcmp(input + (inputlen - md_len), md_value, md_len) != 0)
+ {
+ *errmsgp = libpq_gettext("MAC mismatch");
+ goto fail;
+ }
+
+ ivlen = EVP_CIPHER_CTX_iv_length(evp_cipher_ctx);
+ iv = input;
+ input += ivlen;
+ inputlen -= ivlen;
+ if (!EVP_DecryptInit_ex(evp_cipher_ctx, NULL, NULL, enc_key, iv))
+ {
+ snprintf(msgbuf, sizeof(msgbuf),
+ libpq_gettext("decryption initialization failed: %s"),
+ ERR_reason_error_string(ERR_get_error()));
+ *errmsgp = msgbuf;
+ goto fail;
+ }
+
+ bufsize = inputlen + EVP_CIPHER_CTX_block_size(evp_cipher_ctx) + 1;
+#ifndef TEST_ENCRYPT
+ buf = pqResultAlloc(res, bufsize, false);
+#else
+ buf = malloc(bufsize);
+#endif
+ if (!buf)
+ {
+ *errmsgp = libpq_gettext("out out memory");
+ goto fail;
+ }
+ decr = buf;
+ if (!EVP_DecryptUpdate(evp_cipher_ctx, decr, &decrlen, input, inputlen - md_len))
+ {
+ snprintf(msgbuf, sizeof(msgbuf),
+ libpq_gettext("decryption failed: %s"),
+ ERR_reason_error_string(ERR_get_error()));
+ *errmsgp = msgbuf;
+ goto fail;
+ }
+ if (!EVP_DecryptFinal_ex(evp_cipher_ctx, decr + decrlen, &decrlen2))
+ {
+ snprintf(msgbuf, sizeof(msgbuf),
+ libpq_gettext("decryption failed: %s"),
+ ERR_reason_error_string(ERR_get_error()));
+ *errmsgp = msgbuf;
+ goto fail;
+ }
+ decrlen += decrlen2;
+ Assert(decrlen < bufsize);
+ decr[decrlen] = '\0';
+ result = decr;
+
+fail:
+ EVP_CIPHER_CTX_free(evp_cipher_ctx);
+
+ return result;
+}
+
+#ifndef TEST_ENCRYPT
+static bool
+make_siv(PGconn *conn,
+ unsigned char *iv, size_t ivlen,
+ const EVP_MD *md,
+ const unsigned char *iv_key, int iv_key_len,
+ const unsigned char *plaintext, int plaintext_len)
+{
+ EVP_MD_CTX *evp_md_ctx = NULL;
+ EVP_PKEY *pkey = NULL;
+ unsigned char md_value[EVP_MAX_MD_SIZE];
+ size_t md_len = sizeof(md_value);
+ bool result = false;
+
+ evp_md_ctx = EVP_MD_CTX_new();
+
+ pkey = EVP_PKEY_new_raw_private_key(EVP_PKEY_HMAC, NULL, iv_key, iv_key_len);
+ if (!pkey)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not allocate key for HMAC: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+
+ if (!EVP_DigestSignInit(evp_md_ctx, NULL, md, NULL, pkey))
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("digest initialization failed: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+
+ if (!EVP_DigestSignUpdate(evp_md_ctx, plaintext, plaintext_len))
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("digest signing failed: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+
+ if (!EVP_DigestSignFinal(evp_md_ctx, md_value, &md_len))
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("digest signing failed: %s"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+ Assert(md_len == md_hash_length(md));
+ memcpy(iv, md_value, ivlen);
+
+ result = true;
+fail:
+ EVP_PKEY_free(pkey);
+ EVP_MD_CTX_free(evp_md_ctx);
+ return result;
+}
+#endif /* TEST_ENCRYPT */
+
+unsigned char *
+encrypt_value(PGconn *conn, const PGCEK *cek, int cekalg, const unsigned char *value, int *nbytesp, bool enc_det)
+{
+ int nbytes = *nbytesp;
+ unsigned char iv[EVP_MAX_IV_LENGTH];
+ size_t ivlen;
+ const EVP_CIPHER *cipher;
+ const EVP_MD *md;
+ EVP_CIPHER_CTX *evp_cipher_ctx = NULL;
+ int enc_key_len;
+ int mac_key_len;
+ int key_len;
+ const unsigned char *enc_key;
+ const unsigned char *mac_key;
+ size_t bufsize;
+ unsigned char *buf = NULL;
+ unsigned char *encr;
+ int encrlen,
+ encrlen2;
+
+ const char *errmsg;
+ unsigned char md_value[EVP_MAX_MD_SIZE];
+ size_t md_len = sizeof(md_value);
+ size_t buf2size;
+ unsigned char *buf2 = NULL;
+
+ unsigned char *result = NULL;
+
+ cipher = pg_cekalg_to_openssl_cipher(cekalg);
+ if (!cipher)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("unrecognized encryption algorithm identifier: %d\n"), cekalg);
+ goto fail;
+ }
+
+ md = pg_cekalg_to_openssl_md(cekalg);
+ if (!md)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("unrecognized digest algorithm identifier: %d\n"), cekalg);
+ goto fail;
+ }
+
+ evp_cipher_ctx = EVP_CIPHER_CTX_new();
+
+ if (!EVP_EncryptInit_ex(evp_cipher_ctx, cipher, NULL, NULL, NULL))
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("encryption initialization failed: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+
+ enc_key_len = EVP_CIPHER_CTX_key_length(evp_cipher_ctx);
+ mac_key_len = md_key_length(md);
+ key_len = mac_key_len + enc_key_len;
+
+ if (cek->cekdatalen != key_len)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("column encryption key has wrong length for algorithm (has: %zu, required: %d)\n"),
+ cek->cekdatalen, key_len);
+ goto fail;
+ }
+
+ enc_key = cek->cekdata + mac_key_len;
+ mac_key = cek->cekdata;
+
+ ivlen = EVP_CIPHER_CTX_iv_length(evp_cipher_ctx);
+ Assert(ivlen <= sizeof(iv));
+ if (enc_det)
+ {
+#ifndef TEST_ENCRYPT
+ make_siv(conn, iv, ivlen, md, mac_key, mac_key_len, value, nbytes);
+#else
+ memcpy(iv, test_IV, ivlen);
+#endif
+ }
+ else
+ pg_strong_random(iv, ivlen);
+ if (!EVP_EncryptInit_ex(evp_cipher_ctx, NULL, NULL, enc_key, iv))
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("encryption initialization failed: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+
+ bufsize = ivlen + (nbytes + 2 * EVP_CIPHER_CTX_block_size(evp_cipher_ctx) - 1);
+ buf = malloc(bufsize);
+ if (!buf)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("out of memory\n"));
+ goto fail;
+ }
+ memcpy(buf, iv, ivlen);
+ encr = buf + ivlen;
+ if (!EVP_EncryptUpdate(evp_cipher_ctx, encr, &encrlen, value, nbytes))
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("encryption failed: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+ if (!EVP_EncryptFinal_ex(evp_cipher_ctx, encr + encrlen, &encrlen2))
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("encryption failed: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+ encrlen += encrlen2;
+
+ encr -= ivlen;
+ encrlen += ivlen;
+
+ Assert(encrlen <= bufsize);
+
+ if (!get_message_auth_tag(md, mac_key, mac_key_len,
+ encr, encrlen,
+ cekalg,
+ md_value, &md_len,
+ &errmsg))
+ {
+ appendPQExpBuffer(&conn->errorMessage, "%s\n", errmsg);
+ goto fail;
+ }
+
+ buf2size = encrlen + md_len;
+ buf2 = malloc(buf2size);
+ if (!buf2)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("out of memory\n"));
+ goto fail;
+ }
+ memcpy(buf2, encr, encrlen);
+ memcpy(buf2 + encrlen, md_value, md_len);
+
+ result = buf2;
+ nbytes = buf2size;
+
+fail:
+ free(buf);
+ EVP_CIPHER_CTX_free(evp_cipher_ctx);
+
+ *nbytesp = nbytes;
+ return result;
+}
+
+
+/*
+ * Run test cases
+ */
+#ifdef TEST_ENCRYPT
+
+static void
+debug_print_hex(const char *name, const unsigned char *val, int len)
+{
+ printf("%s =", name);
+ for (int i = 0; i < len; i++)
+ {
+ if (i % 16 == 0)
+ printf("\n");
+ else
+ printf(" ");
+ printf("%02x", val[i]);
+ }
+ printf("\n");
+}
+
+static void
+test_case(int alg, const unsigned char *K, size_t K_len, const unsigned char *P, size_t P_len)
+{
+ unsigned char *C;
+ int nbytes;
+ PGCEK cek;
+
+ nbytes = P_len;
+ cek.cekdata = unconstify(unsigned char *, K);
+ cek.cekdatalen = K_len;
+
+ C = encrypt_value(NULL, &cek, alg, P, &nbytes, true);
+ debug_print_hex("C", C, nbytes);
+}
+
+int
+main(int argc, char **argv)
+{
+ printf("5.1\n");
+ test_case(PG_CEK_AEAD_AES_128_CBC_HMAC_SHA_256, K, 32, P, sizeof(P));
+ printf("5.2\n");
+ test_case(PG_CEK_AEAD_AES_192_CBC_HMAC_SHA_384, K, 48, P, sizeof(P));
+ printf("5.3\n");
+ test_case(PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_384, K, 56, P, sizeof(P));
+ printf("5.4\n");
+ test_case(PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_512, K, 64, P, sizeof(P));
+
+ return 0;
+}
+
+#endif /* TEST_ENCRYPT */
diff --git a/src/interfaces/libpq/fe-encrypt.h b/src/interfaces/libpq/fe-encrypt.h
new file mode 100644
index 0000000000..b0093dc527
--- /dev/null
+++ b/src/interfaces/libpq/fe-encrypt.h
@@ -0,0 +1,33 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-encrypt.h
+ *
+ * encryption support
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-encrypt.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef FE_ENCRYPT_H
+#define FE_ENCRYPT_H
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+
+extern unsigned char *decrypt_cek_from_file(PGconn *conn, FILE *fp, int cmkalg,
+ int fromlen, const unsigned char *from,
+ int *tolen);
+
+extern unsigned char *decrypt_value(PGresult *res, const PGCEK *cek, int cekalg,
+ const unsigned char *input, int inputlen,
+ const char **errmsgp);
+
+extern unsigned char *encrypt_value(PGconn *conn, const PGCEK *cek, int cekalg,
+ const unsigned char *value, int *nbytesp, bool enc_det);
+
+#endif /* FE_ENCRYPT_H */
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index 51e9a362f6..2e3c0295eb 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -24,6 +24,9 @@
#include <unistd.h>
#endif
+#include "catalog/pg_colenckey.h"
+#include "common/base64.h"
+#include "fe-encrypt.h"
#include "libpq-fe.h"
#include "libpq-int.h"
#include "mb/pg_wchar.h"
@@ -72,7 +75,9 @@ static int PQsendQueryGuts(PGconn *conn,
const char *const *paramValues,
const int *paramLengths,
const int *paramFormats,
- int resultFormat);
+ const int *paramForceColumnEncryptions,
+ int resultFormat,
+ PGresult *paramDesc);
static void parseInput(PGconn *conn);
static PGresult *getCopyResult(PGconn *conn, ExecStatusType copytype);
static bool PQexecStart(PGconn *conn);
@@ -1185,6 +1190,381 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
}
}
+int
+pqSaveColumnMasterKey(PGconn *conn, int keyid, const char *keyname,
+ const char *keyrealm)
+{
+ char *keyname_copy;
+ char *keyrealm_copy;
+ bool found;
+
+ keyname_copy = strdup(keyname);
+ if (!keyname_copy)
+ return EOF;
+ keyrealm_copy = strdup(keyrealm);
+ if (!keyrealm_copy)
+ {
+ free(keyname_copy);
+ return EOF;
+ }
+
+ found = false;
+ for (int i = 0; i < conn->ncmks; i++)
+ {
+ struct pg_cmk *checkcmk = &conn->cmks[i];
+
+ /* replace existing? */
+ if (checkcmk->cmkid == keyid)
+ {
+ free(checkcmk->cmkname);
+ free(checkcmk->cmkrealm);
+ checkcmk->cmkname = keyname_copy;
+ checkcmk->cmkrealm = keyrealm_copy;
+ found = true;
+ break;
+ }
+ }
+
+ /* append new? */
+ if (!found)
+ {
+ int newncmks;
+ struct pg_cmk *newcmks;
+ struct pg_cmk *newcmk;
+
+ newncmks = conn->ncmks + 1;
+ if (newncmks <= 0)
+ return EOF;
+ newcmks = realloc(conn->cmks, newncmks * sizeof(struct pg_cmk));
+ if (!newcmks)
+ {
+ free(keyname_copy);
+ free(keyrealm_copy);
+ return EOF;
+ }
+
+ newcmk = &newcmks[newncmks - 1];
+ newcmk->cmkid = keyid;
+ newcmk->cmkname = keyname_copy;
+ newcmk->cmkrealm = keyrealm_copy;
+
+ conn->ncmks = newncmks;
+ conn->cmks = newcmks;
+ }
+
+ return 0;
+}
+
+/*
+ * Replace placeholders in input string. Return value malloc'ed.
+ */
+static char *
+replace_cmk_placeholders(const char *in, const char *cmkname, const char *cmkrealm, int cmkalg, const char *b64data)
+{
+ PQExpBufferData buf;
+
+ initPQExpBuffer(&buf);
+
+ for (const char *p = in; *p; p++)
+ {
+ if (p[0] == '%')
+ {
+ switch (p[1])
+ {
+ case 'a':
+ {
+ const char *s;
+
+ switch (cmkalg)
+ {
+ case PG_CMK_RSAES_OAEP_SHA_1:
+ s = "RSAES_OAEP_SHA_1";
+ break;
+ case PG_CMK_RSAES_OAEP_SHA_256:
+ s = "RSAES_OAEP_SHA_256";
+ break;
+ default:
+ s = "INVALID";
+ }
+ appendPQExpBufferStr(&buf, s);
+ }
+ p++;
+ break;
+ case 'b':
+ appendPQExpBufferStr(&buf, b64data);
+ p++;
+ break;
+ case 'k':
+ appendPQExpBufferStr(&buf, cmkname);
+ p++;
+ break;
+ case 'r':
+ appendPQExpBufferStr(&buf, cmkrealm);
+ p++;
+ break;
+ default:
+ appendPQExpBufferChar(&buf, p[0]);
+ }
+ }
+ else
+ appendPQExpBufferChar(&buf, p[0]);
+ }
+
+ return buf.data;
+}
+
+#ifndef USE_SSL
+/*
+ * Dummy implementation for non-SSL builds
+ */
+unsigned char *
+decrypt_cek_from_file(PGconn *conn, FILE *fp, int cmkalg,
+ int fromlen, const unsigned char *from,
+ int *tolen)
+{
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("column encryption not supported by this build\n"));
+ return NULL;
+}
+#endif
+
+/*
+ * Decrypt a CEK using the given CMK. The ciphertext is passed in
+ * "from" and "fromlen". Return the decrypted value in a malloc'ed area, its
+ * length via "tolen". Return NULL on error; add error messages directly to
+ * "conn".
+ */
+static unsigned char *
+decrypt_cek(PGconn *conn, const PGCMK *cmk, int cmkalg,
+ int fromlen, const unsigned char *from,
+ int *tolen)
+{
+ char *cmklookup;
+ bool found = false;
+ unsigned char *result = NULL;
+
+ cmklookup = strdup(conn->cmklookup ? conn->cmklookup : "");
+
+ if (!cmklookup)
+ {
+ appendPQExpBuffer(&conn->errorMessage, libpq_gettext("out of memory\n"));
+ return NULL;
+ }
+
+ /*
+ * Analyze semicolon-separated list
+ */
+ for (char *s = strtok(cmklookup, ";"); s; s = strtok(NULL, ";"))
+ {
+ char *sep;
+
+ /* split found token at '=' */
+ sep = strchr(s, '=');
+ if (!sep)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("syntax error in CMK lookup specification, missing \"%c\": %s\n"), '=', s);
+ break;
+ }
+
+ /* matching realm? */
+ if (strncmp(s, "*", sep - s) == 0 || strncmp(s, cmk->cmkrealm, sep - s) == 0)
+ {
+ char *sep2;
+
+ found = true;
+
+ sep2 = strchr(sep, ':');
+ if (!sep2)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("syntax error in CMK lookup specification, missing \"%c\": %s\n"), ':', s);
+ goto fail;
+ }
+
+ if (strncmp(sep + 1, "file", sep2 - (sep + 1)) == 0)
+ {
+ char *cmkfilename;
+ FILE *fp;
+
+ cmkfilename = replace_cmk_placeholders(sep2 + 1, cmk->cmkname, cmk->cmkrealm, cmkalg, "INVALID");
+ fp = fopen(cmkfilename, "rb");
+ if (fp)
+ {
+ result = decrypt_cek_from_file(conn, fp, cmkalg, fromlen, from, tolen);
+ fclose(fp);
+ }
+ else
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not open file \"%s\": %m\n"), cmkfilename);
+ free(cmkfilename);
+ goto fail;
+ }
+ free(cmkfilename);
+ }
+ else if (strncmp(sep + 1, "run", sep2 - (sep + 1)) == 0)
+ {
+ int enclen;
+ char *enc;
+ char *command;
+ FILE *fp;
+ char buf[4096];
+
+ enclen = pg_b64_enc_len(fromlen);
+ enc = malloc(enclen + 1);
+ if (!enc)
+ {
+ appendPQExpBuffer(&conn->errorMessage, libpq_gettext("out of memory\n"));
+ goto fail;
+ }
+ enclen = pg_b64_encode((const char *) from, fromlen, enc, enclen);
+ if (enclen < 0)
+ {
+ appendPQExpBuffer(&conn->errorMessage, libpq_gettext("base64 encoding failed\n"));
+ free(enc);
+ goto fail;
+ }
+ command = replace_cmk_placeholders(sep2 + 1, cmk->cmkname, cmk->cmkrealm, cmkalg, enc);
+ free(enc);
+ fp = popen(command, "r");
+ free(command);
+ if (!fp)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not run command \"%s\": %m\n"), command);
+ goto fail;
+ }
+ if (fgets(buf, sizeof(buf), fp))
+ {
+ int linelen;
+ int declen;
+ char *dec;
+
+ linelen = strlen(buf);
+ if (buf[linelen - 1] == '\n')
+ {
+ buf[linelen - 1] = '\0';
+ linelen--;
+ }
+ declen = pg_b64_dec_len(linelen);
+ dec = malloc(declen);
+ if (!dec)
+ {
+ appendPQExpBuffer(&conn->errorMessage, libpq_gettext("out of memory\n"));
+ goto fail;
+ }
+ declen = pg_b64_decode(buf, linelen, dec, declen);
+ if (declen < 0)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("base64 decoding failed\n"));
+ free(dec);
+ goto fail;
+ }
+ result = (unsigned char *) dec;
+ *tolen = declen;
+ }
+ else
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not read from command: %m\n"));
+ pclose(fp);
+ goto fail;
+ }
+ pclose(fp);
+ }
+ else
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("CMK lookup scheme \"%s\" not recognized\n"), sep + 1);
+ goto fail;
+ }
+ }
+ }
+
+ if (!found)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("no CMK lookup found for realm \"%s\"\n"), cmk->cmkrealm);
+ }
+
+fail:
+ free(cmklookup);
+ return result;
+}
+
+int
+pqSaveColumnEncryptionKey(PGconn *conn, int keyid, int cmkid, int cmkalg, const unsigned char *value, int len)
+{
+ PGCMK *cmk = NULL;
+ unsigned char *plainval = NULL;
+ int plainvallen = 0;
+ bool found;
+
+ for (int i = 0; i < conn->ncmks; i++)
+ {
+ if (conn->cmks[i].cmkid == cmkid)
+ {
+ cmk = &conn->cmks[i];
+ break;
+ }
+ }
+
+ if (!cmk)
+ return EOF;
+
+ plainval = decrypt_cek(conn, cmk, cmkalg, len, value, &plainvallen);
+ if (!plainval)
+ return EOF;
+
+ found = false;
+ for (int i = 0; i < conn->nceks; i++)
+ {
+ struct pg_cek *checkcek = &conn->ceks[i];
+
+ /* replace existing? */
+ if (checkcek->cekid == keyid)
+ {
+ free(checkcek->cekdata);
+ checkcek->cekdata = plainval;
+ checkcek->cekdatalen = plainvallen;
+ found = true;
+ break;
+ }
+ }
+
+ /* append new? */
+ if (!found)
+ {
+ int newnceks;
+ struct pg_cek *newceks;
+ struct pg_cek *newcek;
+
+ newnceks = conn->nceks + 1;
+ if (newnceks <= 0)
+ {
+ free(plainval);
+ return EOF;
+ }
+ newceks = realloc(conn->ceks, newnceks * sizeof(struct pg_cek));
+ if (!newceks)
+ {
+ free(plainval);
+ return EOF;
+ }
+
+ newcek = &newceks[newnceks - 1];
+ newcek->cekid = keyid;
+ newcek->cekdata = plainval;
+ newcek->cekdatalen = plainvallen;
+
+ conn->nceks = newnceks;
+ conn->ceks = newceks;
+ }
+
+ return 0;
+}
/*
* pqRowProcessor
@@ -1250,9 +1630,58 @@ pqRowProcessor(PGconn *conn, const char **errmsgp)
}
else
{
- bool isbinary = (res->attDescs[i].format != 0);
+ bool isbinary = ((res->attDescs[i].format & 0x0F) != 0);
char *val;
+ if (res->attDescs[i].cekid)
+ {
+#ifdef USE_SSL
+ PGCEK *cek = NULL;
+
+ for (int j = 0; j < conn->nceks; j++)
+ {
+ if (conn->ceks[j].cekid == res->attDescs[i].cekid)
+ {
+ cek = &conn->ceks[j];
+ break;
+ }
+ }
+ if (!cek)
+ {
+ *errmsgp = libpq_gettext("column encryption key not found");
+ goto fail;
+ }
+
+ if (isbinary)
+ {
+ val = (char *) decrypt_value(res, cek, res->attDescs[i].cekalg,
+ (const unsigned char *) columns[i].value, clen, errmsgp);
+ }
+ else
+ {
+ unsigned char *buf;
+ unsigned char *enccolval;
+ size_t enccolvallen;
+
+ buf = malloc(clen + 1);
+ memcpy(buf, columns[i].value, clen);
+ buf[clen] = '\0';
+ enccolval = PQunescapeBytea(buf, &enccolvallen);
+ val = (char *) decrypt_value(res, cek, res->attDescs[i].cekalg,
+ enccolval, enccolvallen, errmsgp);
+ free(enccolval);
+ free(buf);
+ }
+
+ if (val == NULL)
+ goto fail;
+#else
+ *errmsgp = libpq_gettext("column encryption not supported by this build");
+ goto fail;
+#endif
+ }
+ else
+ {
val = (char *) pqResultAlloc(res, clen + 1, isbinary);
if (val == NULL)
goto fail;
@@ -1260,6 +1689,7 @@ pqRowProcessor(PGconn *conn, const char **errmsgp)
/* copy and zero-terminate the data (even if it's binary) */
memcpy(val, columns[i].value, clen);
val[clen] = '\0';
+ }
tup[i].len = clen;
tup[i].value = val;
@@ -1564,7 +1994,9 @@ PQsendQueryParams(PGconn *conn,
paramValues,
paramLengths,
paramFormats,
- resultFormat);
+ NULL,
+ resultFormat,
+ NULL);
}
/*
@@ -1682,6 +2114,20 @@ PQsendQueryPrepared(PGconn *conn,
const int *paramLengths,
const int *paramFormats,
int resultFormat)
+{
+ return PQsendQueryPrepared2(conn, stmtName, nParams, paramValues, paramLengths, paramFormats, NULL, resultFormat, NULL);
+}
+
+int
+PQsendQueryPrepared2(PGconn *conn,
+ const char *stmtName,
+ int nParams,
+ const char *const *paramValues,
+ const int *paramLengths,
+ const int *paramFormats,
+ const int *paramForceColumnEncryptions,
+ int resultFormat,
+ PGresult *paramDesc)
{
if (!PQsendQueryStart(conn, true))
return 0;
@@ -1709,7 +2155,9 @@ PQsendQueryPrepared(PGconn *conn,
paramValues,
paramLengths,
paramFormats,
- resultFormat);
+ paramForceColumnEncryptions,
+ resultFormat,
+ paramDesc);
}
/*
@@ -1808,7 +2256,9 @@ PQsendQueryGuts(PGconn *conn,
const char *const *paramValues,
const int *paramLengths,
const int *paramFormats,
- int resultFormat)
+ const int *paramForceColumnEncryptions,
+ int resultFormat,
+ PGresult *paramDesc)
{
int i;
PGcmdQueueEntry *entry;
@@ -1855,14 +2305,53 @@ PQsendQueryGuts(PGconn *conn,
pqPuts(stmtName, conn) < 0)
goto sendFailed;
+ /* Check force column encryption */
+ if (nParams > 0 && paramForceColumnEncryptions)
+ {
+ for (i = 0; i < nParams; i++)
+ {
+ if (paramForceColumnEncryptions[i])
+ {
+ if (!(paramDesc &&
+ paramDesc->paramDescs &&
+ paramDesc->paramDescs[i].cekid))
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("parameter with forced encryption is not to be encrypted\n"));
+ goto sendFailed;
+ }
+ }
+ }
+ }
+
/* Send parameter formats */
- if (nParams > 0 && paramFormats)
+ if (nParams > 0 && (paramFormats || (paramDesc && paramDesc->paramDescs)))
{
if (pqPutInt(nParams, 2, conn) < 0)
goto sendFailed;
+
for (i = 0; i < nParams; i++)
{
- if (pqPutInt(paramFormats[i], 2, conn) < 0)
+ int format = paramFormats ? paramFormats[i] : 0;
+
+ if (paramDesc && paramDesc->paramDescs)
+ {
+ PGresParamDesc *pd = ¶mDesc->paramDescs[i];
+
+ if (pd->cekid)
+ {
+ if (format != 0)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("format must be text for encrypted parameter\n"));
+ goto sendFailed;
+ }
+ format = 1;
+ format |= 0x10;
+ }
+ }
+
+ if (pqPutInt(format, 2, conn) < 0)
goto sendFailed;
}
}
@@ -1881,6 +2370,7 @@ PQsendQueryGuts(PGconn *conn,
if (paramValues && paramValues[i])
{
int nbytes;
+ const char *paramValue;
if (paramFormats && paramFormats[i] != 0)
{
@@ -1899,9 +2389,54 @@ PQsendQueryGuts(PGconn *conn,
/* text parameter, do not use paramLengths */
nbytes = strlen(paramValues[i]);
}
+
+ paramValue = paramValues[i];
+
+ if (paramDesc && paramDesc->paramDescs && paramDesc->paramDescs[i].cekid)
+ {
+#ifdef USE_SSL
+ bool enc_det = (paramDesc->paramDescs[i].flags & 0x01) != 0;
+ PGCEK *cek = NULL;
+ char *enc_paramValue;
+ int enc_nbytes = nbytes;
+
+ for (int j = 0; j < conn->nceks; j++)
+ {
+ if (conn->ceks[j].cekid == paramDesc->paramDescs[i].cekid)
+ {
+ cek = &conn->ceks[j];
+ break;
+ }
+ }
+ if (!cek)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("column encryption key not found\n"));
+ goto sendFailed;
+ }
+
+ enc_paramValue = (char *) encrypt_value(conn, cek, paramDesc->paramDescs[i].cekalg,
+ (const unsigned char *) paramValue, &enc_nbytes, enc_det);
+ if (!enc_paramValue)
+ goto sendFailed;
+
+ if (pqPutInt(enc_nbytes, 4, conn) < 0 ||
+ pqPutnchar(enc_paramValue, enc_nbytes, conn) < 0)
+ goto sendFailed;
+
+ free(enc_paramValue);
+#else
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("column encryption not supported by this build\n"));
+ goto sendFailed;
+#endif
+ }
+ else
+ {
if (pqPutInt(nbytes, 4, conn) < 0 ||
- pqPutnchar(paramValues[i], nbytes, conn) < 0)
+ pqPutnchar(paramValue, nbytes, conn) < 0)
goto sendFailed;
+ }
}
else
{
@@ -2334,12 +2869,27 @@ PQexecPrepared(PGconn *conn,
const int *paramLengths,
const int *paramFormats,
int resultFormat)
+{
+ return PQexecPrepared2(conn, stmtName, nParams, paramValues, paramLengths, paramFormats, NULL, resultFormat, NULL);
+}
+
+PGresult *
+PQexecPrepared2(PGconn *conn,
+ const char *stmtName,
+ int nParams,
+ const char *const *paramValues,
+ const int *paramLengths,
+ const int *paramFormats,
+ const int *paramForceColumnEncryptions,
+ int resultFormat,
+ PGresult *paramDesc)
{
if (!PQexecStart(conn))
return NULL;
- if (!PQsendQueryPrepared(conn, stmtName,
- nParams, paramValues, paramLengths,
- paramFormats, resultFormat))
+ if (!PQsendQueryPrepared2(conn, stmtName,
+ nParams, paramValues, paramLengths,
+ paramFormats, paramForceColumnEncryptions,
+ resultFormat, paramDesc))
return NULL;
return PQexecFinish(conn);
}
@@ -3602,6 +4152,17 @@ PQfmod(const PGresult *res, int field_num)
return 0;
}
+int
+PQfisencrypted(const PGresult *res, int field_num)
+{
+ if (!check_field_number(res, field_num))
+ return false;
+ if (res->attDescs)
+ return (res->attDescs[field_num].cekid != 0);
+ else
+ return false;
+}
+
char *
PQcmdStatus(PGresult *res)
{
@@ -3787,6 +4348,17 @@ PQparamtype(const PGresult *res, int param_num)
return InvalidOid;
}
+int
+PQparamisencrypted(const PGresult *res, int param_num)
+{
+ if (!check_param_number(res, param_num))
+ return false;
+ if (res->paramDescs)
+ return (res->paramDescs[param_num].cekid != 0);
+ else
+ return false;
+}
+
/* PQsetnonblocking:
* sets the PGconn's database connection non-blocking if the arg is true
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index 10c76daf6e..9461c04bbe 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -45,6 +45,8 @@ static int getRowDescriptions(PGconn *conn, int msgLength);
static int getParamDescriptions(PGconn *conn, int msgLength);
static int getAnotherTuple(PGconn *conn, int msgLength);
static int getParameterStatus(PGconn *conn);
+static int getColumnMasterKey(PGconn *conn);
+static int getColumnEncryptionKey(PGconn *conn);
static int getNotify(PGconn *conn);
static int getCopyStart(PGconn *conn, ExecStatusType copytype);
static int getReadyForQuery(PGconn *conn);
@@ -315,6 +317,22 @@ pqParseInput3(PGconn *conn)
if (pqGetInt(&(conn->be_key), 4, conn))
return;
break;
+ case 'y': /* Column Master Key */
+ if (getColumnMasterKey(conn))
+ {
+ // TODO: review error handling here
+ pqSaveErrorResult(conn);
+ pqInternalNotice(&conn->noticeHooks, "%s", conn->errorMessage.data);
+ }
+ break;
+ case 'Y': /* Column Encryption Key */
+ if (getColumnEncryptionKey(conn))
+ {
+ // TODO: review error handling here
+ pqSaveErrorResult(conn);
+ pqInternalNotice(&conn->noticeHooks, "%s", conn->errorMessage.data);
+ }
+ break;
case 'T': /* Row Description */
if (conn->error_result ||
(conn->result != NULL &&
@@ -572,6 +590,8 @@ getRowDescriptions(PGconn *conn, int msgLength)
int typlen;
int atttypmod;
int format;
+ int cekid;
+ int cekalg;
if (pqGets(&conn->workBuffer, conn) ||
pqGetInt(&tableid, 4, conn) ||
@@ -579,7 +599,9 @@ getRowDescriptions(PGconn *conn, int msgLength)
pqGetInt(&typid, 4, conn) ||
pqGetInt(&typlen, 2, conn) ||
pqGetInt(&atttypmod, 4, conn) ||
- pqGetInt(&format, 2, conn))
+ pqGetInt(&format, 2, conn) ||
+ pqGetInt(&cekid, 4, conn) ||
+ pqGetInt(&cekalg, 2, conn))
{
/* We should not run out of data here, so complain */
errmsg = libpq_gettext("insufficient data in \"T\" message");
@@ -607,8 +629,10 @@ getRowDescriptions(PGconn *conn, int msgLength)
result->attDescs[i].typid = typid;
result->attDescs[i].typlen = typlen;
result->attDescs[i].atttypmod = atttypmod;
+ result->attDescs[i].cekid = cekid;
+ result->attDescs[i].cekalg = cekalg;
- if (format != 1)
+ if ((format & 0x0F) != 1)
result->binary = 0;
}
@@ -710,10 +734,22 @@ getParamDescriptions(PGconn *conn, int msgLength)
for (i = 0; i < nparams; i++)
{
int typid;
+ int cekid;
+ int cekalg;
+ int flags;
if (pqGetInt(&typid, 4, conn))
goto not_enough_data;
result->paramDescs[i].typid = typid;
+ if (pqGetInt(&cekid, 4, conn))
+ goto not_enough_data;
+ result->paramDescs[i].cekid = cekid;
+ if (pqGetInt(&cekalg, 2, conn))
+ goto not_enough_data;
+ result->paramDescs[i].cekalg = cekalg;
+ if (pqGetInt(&flags, 2, conn))
+ goto not_enough_data;
+ result->paramDescs[i].flags = flags;
}
/* Success! */
@@ -1439,6 +1475,89 @@ getParameterStatus(PGconn *conn)
return 0;
}
+/*
+ * Attempt to read a ColumnMasterKey message.
+ * Entry: 'y' message type and length have already been consumed.
+ * Exit: returns 0 if successfully consumed message.
+ * returns EOF if not enough data.
+ */
+static int
+getColumnMasterKey(PGconn *conn)
+{
+ int keyid;
+ char *keyname;
+ char *keyrealm;
+ int ret;
+
+ /* Get the key ID */
+ if (pqGetInt(&keyid, 4, conn) != 0)
+ return EOF;
+ /* Get the key name */
+ if (pqGets(&conn->workBuffer, conn) != 0)
+ return EOF;
+ keyname = strdup(conn->workBuffer.data);
+ if (!keyname)
+ return EOF;
+ /* Get the key realm */
+ if (pqGets(&conn->workBuffer, conn) != 0)
+ return EOF;
+ keyrealm = strdup(conn->workBuffer.data);
+ if (!keyrealm)
+ return EOF;
+ /* And save it */
+ ret = pqSaveColumnMasterKey(conn, keyid, keyname, keyrealm);
+
+ free(keyname);
+ free(keyrealm);
+
+ return ret;
+}
+
+/*
+ * Attempt to read a ColumnEncryptionKey message.
+ * Entry: 'Y' message type and length have already been consumed.
+ * Exit: returns 0 if successfully consumed message.
+ * returns EOF if not enough data.
+ */
+static int
+getColumnEncryptionKey(PGconn *conn)
+{
+ int keyid;
+ int cmkid;
+ int cmkalg;
+ char *buf;
+ int vallen;
+
+ /* Get the key ID */
+ if (pqGetInt(&keyid, 4, conn) != 0)
+ return EOF;
+ /* Get the CMK ID */
+ if (pqGetInt(&cmkid, 4, conn) != 0)
+ return EOF;
+ /* Get the CMK algorithm */
+ if (pqGetInt(&cmkalg, 2, conn) != 0)
+ return EOF;
+ /* Get the key data len */
+ if (pqGetInt(&vallen, 4, conn) != 0)
+ return EOF;
+ /* Get the key data */
+ buf = malloc(vallen);
+ if (!buf)
+ return EOF;
+ if (pqGetnchar(buf, vallen, conn) != 0)
+ {
+ free(buf);
+ return EOF;
+ }
+ /* And save it */
+ if (pqSaveColumnEncryptionKey(conn, keyid, cmkid, cmkalg, (unsigned char *) buf, vallen) != 0)
+ {
+ free(buf);
+ return EOF;
+ }
+ free(buf);
+ return 0;
+}
/*
* Attempt to read a Notify response message.
diff --git a/src/interfaces/libpq/fe-trace.c b/src/interfaces/libpq/fe-trace.c
index 5d68cf2eb3..e08cbcff33 100644
--- a/src/interfaces/libpq/fe-trace.c
+++ b/src/interfaces/libpq/fe-trace.c
@@ -458,7 +458,12 @@ pqTraceOutputt(FILE *f, const char *message, int *cursor, bool regress)
nfields = pqTraceOutputInt16(f, message, cursor);
for (int i = 0; i < nfields; i++)
+ {
+ pqTraceOutputInt32(f, message, cursor, regress);
pqTraceOutputInt32(f, message, cursor, regress);
+ pqTraceOutputInt16(f, message, cursor);
+ pqTraceOutputInt16(f, message, cursor);
+ }
}
/* RowDescription */
@@ -479,6 +484,8 @@ pqTraceOutputT(FILE *f, const char *message, int *cursor, bool regress)
pqTraceOutputInt16(f, message, cursor);
pqTraceOutputInt32(f, message, cursor, false);
pqTraceOutputInt16(f, message, cursor);
+ pqTraceOutputInt32(f, message, cursor, regress);
+ pqTraceOutputInt16(f, message, cursor);
}
}
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index 7986445f1a..42fbe0cd49 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -267,6 +267,8 @@ typedef struct pgresAttDesc
Oid typid; /* type id */
int typlen; /* type size */
int atttypmod; /* type-specific modifier info */
+ Oid cekid;
+ int cekalg;
} PGresAttDesc;
/* ----------------
@@ -438,6 +440,15 @@ extern PGresult *PQexecPrepared(PGconn *conn,
const int *paramLengths,
const int *paramFormats,
int resultFormat);
+extern PGresult *PQexecPrepared2(PGconn *conn,
+ const char *stmtName,
+ int nParams,
+ const char *const *paramValues,
+ const int *paramLengths,
+ const int *paramFormats,
+ const int *paramForceColumnEncryptions,
+ int resultFormat,
+ PGresult *paramDesc);
/* Interface for multiple-result or asynchronous queries */
#define PQ_QUERY_PARAM_MAX_LIMIT 65535
@@ -461,6 +472,15 @@ extern int PQsendQueryPrepared(PGconn *conn,
const int *paramLengths,
const int *paramFormats,
int resultFormat);
+extern int PQsendQueryPrepared2(PGconn *conn,
+ const char *stmtName,
+ int nParams,
+ const char *const *paramValues,
+ const int *paramLengths,
+ const int *paramFormats,
+ const int *paramForceColumnEncryptions,
+ int resultFormat,
+ PGresult *paramDesc);
extern int PQsetSingleRowMode(PGconn *conn);
extern PGresult *PQgetResult(PGconn *conn);
@@ -531,6 +551,7 @@ extern int PQfformat(const PGresult *res, int field_num);
extern Oid PQftype(const PGresult *res, int field_num);
extern int PQfsize(const PGresult *res, int field_num);
extern int PQfmod(const PGresult *res, int field_num);
+extern int PQfisencrypted(const PGresult *res, int field_num);
extern char *PQcmdStatus(PGresult *res);
extern char *PQoidStatus(const PGresult *res); /* old and ugly */
extern Oid PQoidValue(const PGresult *res); /* new and improved */
@@ -540,6 +561,7 @@ extern int PQgetlength(const PGresult *res, int tup_num, int field_num);
extern int PQgetisnull(const PGresult *res, int tup_num, int field_num);
extern int PQnparams(const PGresult *res);
extern Oid PQparamtype(const PGresult *res, int param_num);
+extern int PQparamisencrypted(const PGresult *res, int param_num);
/* Describe prepared statements and portals */
extern PGresult *PQdescribePrepared(PGconn *conn, const char *stmt);
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 3db6a17db4..0a4c012aa6 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -111,6 +111,9 @@ union pgresult_data
typedef struct pgresParamDesc
{
Oid typid; /* type id */
+ Oid cekid;
+ int cekalg;
+ int flags;
} PGresParamDesc;
/*
@@ -340,6 +343,26 @@ typedef struct pg_conn_host
* found in password file. */
} pg_conn_host;
+/*
+ * Column encryption support data
+ */
+
+/* column master key */
+typedef struct pg_cmk
+{
+ Oid cmkid;
+ char *cmkname;
+ char *cmkrealm;
+} PGCMK;
+
+/* column encryption key */
+typedef struct pg_cek
+{
+ Oid cekid;
+ unsigned char *cekdata; /* (decrypted) */
+ size_t cekdatalen;
+} PGCEK;
+
/*
* PGconn stores all the state data associated with a single connection
* to a backend.
@@ -393,6 +416,7 @@ struct pg_conn
char *ssl_min_protocol_version; /* minimum TLS protocol version */
char *ssl_max_protocol_version; /* maximum TLS protocol version */
char *target_session_attrs; /* desired session properties */
+ char *cmklookup; /* CMK lookup specification */
/* Optional file to write trace info to */
FILE *Pfdebug;
@@ -475,6 +499,12 @@ struct pg_conn
PGContextVisibility show_context; /* whether to show CONTEXT field */
PGlobjfuncs *lobjfuncs; /* private state for large-object access fns */
+ /* Column encryption support data */
+ int ncmks;
+ PGCMK *cmks;
+ int nceks;
+ PGCEK *ceks;
+
/* Buffer for data received from backend and not yet processed */
char *inBuffer; /* currently allocated buffer */
int inBufSize; /* allocated size of buffer */
@@ -670,6 +700,10 @@ extern void pqSaveMessageField(PGresult *res, char code,
const char *value);
extern void pqSaveParameterStatus(PGconn *conn, const char *name,
const char *value);
+extern int pqSaveColumnMasterKey(PGconn *conn, int keyid, const char *keyname,
+ const char *keyrealm);
+extern int pqSaveColumnEncryptionKey(PGconn *conn, int keyid, int cmkid, int cmkalg,
+ const unsigned char *value, int len);
extern int pqRowProcessor(PGconn *conn, const char **errmsgp);
extern void pqCommandQueueAdvance(PGconn *conn);
extern int PQsendQueryContinue(PGconn *conn, const char *query);
diff --git a/src/interfaces/libpq/nls.mk b/src/interfaces/libpq/nls.mk
index 1f62ba1b57..844e085e6d 100644
--- a/src/interfaces/libpq/nls.mk
+++ b/src/interfaces/libpq/nls.mk
@@ -1,6 +1,6 @@
# src/interfaces/libpq/nls.mk
CATALOG_NAME = libpq
AVAIL_LANGUAGES = cs de el es fr he it ja ko pl pt_BR ru sv tr uk zh_CN zh_TW
-GETTEXT_FILES = fe-auth.c fe-auth-scram.c fe-connect.c fe-exec.c fe-gssapi-common.c fe-lobj.c fe-misc.c fe-protocol3.c fe-secure.c fe-secure-common.c fe-secure-gssapi.c fe-secure-openssl.c win32.c ../../port/thread.c
+GETTEXT_FILES = fe-auth.c fe-auth-scram.c fe-connect.c fe-encrypt-openssl.c fe-exec.c fe-gssapi-common.c fe-lobj.c fe-misc.c fe-protocol3.c fe-secure.c fe-secure-common.c fe-secure-gssapi.c fe-secure-openssl.c win32.c ../../port/thread.c
GETTEXT_TRIGGERS = libpq_gettext pqInternalNotice:2
GETTEXT_FLAGS = libpq_gettext:1:pass-c-format pqInternalNotice:2:c-format
diff --git a/src/interfaces/libpq/test/.gitignore b/src/interfaces/libpq/test/.gitignore
index 6ba78adb67..1846594ec5 100644
--- a/src/interfaces/libpq/test/.gitignore
+++ b/src/interfaces/libpq/test/.gitignore
@@ -1,2 +1,3 @@
+/libpq_test_encrypt
/libpq_testclient
/libpq_uri_regress
diff --git a/src/interfaces/libpq/test/Makefile b/src/interfaces/libpq/test/Makefile
index 75ac08f943..b1ebab90d4 100644
--- a/src/interfaces/libpq/test/Makefile
+++ b/src/interfaces/libpq/test/Makefile
@@ -15,10 +15,17 @@ override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += $(libpq_pgport)
PROGS = libpq_testclient libpq_uri_regress
+ifeq ($(with_ssl),openssl)
+PROGS += libpq_test_encrypt
+endif
+
all: $(PROGS)
$(PROGS): $(WIN32RES)
+libpq_test_encrypt: ../fe-encrypt-openssl.c
+ $(CC) $(CPPFLAGS) -DTEST_ENCRYPT $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+
clean distclean maintainer-clean:
rm -f $(PROGS) *.o
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index 44457f930c..c5f3153094 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -9529,7 +9529,7 @@ DO $d$
END;
$d$;
ERROR: invalid option "password"
-HINT: Valid options in this context are: service, passfile, channel_binding, connect_timeout, dbname, host, hostaddr, port, options, application_name, keepalives, keepalives_idle, keepalives_interval, keepalives_count, tcp_user_timeout, sslmode, sslcompression, sslcert, sslkey, sslrootcert, sslcrl, sslcrldir, sslsni, requirepeer, ssl_min_protocol_version, ssl_max_protocol_version, gssencmode, krbsrvname, gsslib, target_session_attrs, use_remote_estimate, fdw_startup_cost, fdw_tuple_cost, extensions, updatable, truncatable, fetch_size, batch_size, async_capable, parallel_commit, keep_connections
+HINT: Valid options in this context are: service, passfile, channel_binding, connect_timeout, dbname, host, hostaddr, port, options, application_name, keepalives, keepalives_idle, keepalives_interval, keepalives_count, tcp_user_timeout, sslmode, sslcompression, sslcert, sslkey, sslrootcert, sslcrl, sslcrldir, sslsni, requirepeer, ssl_min_protocol_version, ssl_max_protocol_version, gssencmode, krbsrvname, gsslib, target_session_attrs, cmklookup, use_remote_estimate, fdw_startup_cost, fdw_tuple_cost, extensions, updatable, truncatable, fetch_size, batch_size, async_capable, parallel_commit, keep_connections
CONTEXT: SQL statement "ALTER SERVER loopback_nopw OPTIONS (ADD password 'dummypw')"
PL/pgSQL function inline_code_block line 3 at EXECUTE
-- If we add a password for our user mapping instead, we should get a different
base-commit: 6ffff0fd225432fe2ae4bd5abb7ff6113e255418
--
2.36.1
Attachments:
[text/plain] v3-0001-Transparent-column-encryption.patch (224.5K, ../[email protected]/2-v3-0001-Transparent-column-encryption.patch)
download | inline diff:
From bcb9df8eaafdd1b5a05d8ea4bb5b2af001f6403f Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <[email protected]>
Date: Tue, 5 Jul 2022 12:48:18 +0200
Subject: [PATCH v3] Transparent column encryption
---
doc/src/sgml/acronyms.sgml | 18 +
doc/src/sgml/catalogs.sgml | 275 +++++++
doc/src/sgml/datatype.sgml | 47 ++
doc/src/sgml/ddl.sgml | 73 ++
doc/src/sgml/glossary.sgml | 23 +
doc/src/sgml/libpq.sgml | 257 +++++-
doc/src/sgml/protocol.sgml | 346 ++++++++
doc/src/sgml/ref/create_table.sgml | 42 +-
doc/src/sgml/ref/psql-ref.sgml | 33 +
src/test/Makefile | 3 +-
src/test/column_encryption/.gitignore | 3 +
src/test/column_encryption/Makefile | 29 +
.../t/001_column_encryption.pl | 206 +++++
.../column_encryption/t/002_cmk_rotation.pl | 144 ++++
src/test/column_encryption/test_client.c | 210 +++++
.../column_encryption/test_run_decrypt.pl | 41 +
.../traces/disallowed_in_pipeline.trace | 2 +-
.../traces/multi_pipelines.trace | 4 +-
.../libpq_pipeline/traces/nosync.trace | 20 +-
.../traces/pipeline_abort.trace | 4 +-
.../libpq_pipeline/traces/prepared.trace | 6 +-
.../traces/simple_pipeline.trace | 2 +-
.../libpq_pipeline/traces/singlerow.trace | 6 +-
.../libpq_pipeline/traces/transaction.trace | 2 +-
.../regress/expected/column_encryption.out | 59 ++
src/test/regress/expected/oidjoins.out | 6 +
src/test/regress/expected/opr_sanity.out | 12 +-
src/test/regress/expected/prepare.out | 38 +-
src/test/regress/expected/psql.out | 25 +
src/test/regress/expected/rules.out | 4 +-
src/test/regress/expected/type_sanity.out | 20 +-
src/test/regress/parallel_schedule | 3 +
src/test/regress/pg_regress_main.c | 2 +-
src/test/regress/sql/column_encryption.sql | 50 ++
src/test/regress/sql/prepare.sql | 2 +-
src/test/regress/sql/psql.sql | 17 +
src/test/regress/sql/type_sanity.sql | 4 +-
src/include/access/printtup.h | 2 +
src/include/catalog/pg_amop.dat | 5 +
src/include/catalog/pg_amproc.dat | 5 +
src/include/catalog/pg_attribute.h | 9 +
src/include/catalog/pg_cast.dat | 6 +
src/include/catalog/pg_colenckey.h | 55 ++
src/include/catalog/pg_colenckeydata.h | 46 ++
src/include/catalog/pg_colmasterkey.h | 45 ++
src/include/catalog/pg_opclass.dat | 2 +
src/include/catalog/pg_operator.dat | 10 +
src/include/catalog/pg_opfamily.dat | 2 +
src/include/catalog/pg_proc.dat | 13 +-
src/include/catalog/pg_type.dat | 12 +
src/include/catalog/pg_type.h | 1 +
src/include/nodes/parsenodes.h | 1 +
src/include/parser/analyze.h | 4 +-
src/include/parser/parse_node.h | 2 +
src/include/parser/parse_param.h | 3 +-
src/include/tcop/tcopprot.h | 2 +
src/include/utils/plancache.h | 6 +
src/include/utils/syscache.h | 4 +
src/backend/access/common/printsimple.c | 2 +
src/backend/access/common/printtup.c | 162 ++++
src/backend/access/common/tupdesc.c | 12 +
src/backend/access/hash/hashvalidate.c | 2 +-
src/backend/catalog/Makefile | 3 +-
src/backend/catalog/heap.c | 3 +
src/backend/commands/prepare.c | 74 +-
src/backend/commands/tablecmds.c | 74 ++
src/backend/executor/spi.c | 4 +
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/analyze.c | 3 +-
src/backend/parser/gram.y | 13 +-
src/backend/parser/parse_param.c | 43 +-
src/backend/parser/parse_target.c | 6 +
src/backend/tcop/postgres.c | 54 +-
src/backend/utils/adt/arrayfuncs.c | 1 +
src/backend/utils/cache/plancache.c | 10 +
src/backend/utils/cache/syscache.c | 39 +
src/bin/pg_dump/pg_dump.c | 26 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/psql/command.c | 31 +
src/bin/psql/common.c | 39 +-
src/bin/psql/describe.c | 29 +-
src/bin/psql/help.c | 3 +
src/bin/psql/settings.h | 5 +
src/bin/psql/startup.c | 10 +
src/bin/psql/tab-complete.c | 2 +-
src/interfaces/libpq/Makefile | 1 +
src/interfaces/libpq/exports.txt | 4 +
src/interfaces/libpq/fe-connect.c | 20 +
src/interfaces/libpq/fe-encrypt-openssl.c | 758 ++++++++++++++++++
src/interfaces/libpq/fe-encrypt.h | 33 +
src/interfaces/libpq/fe-exec.c | 594 +++++++++++++-
src/interfaces/libpq/fe-protocol3.c | 123 ++-
src/interfaces/libpq/fe-trace.c | 7 +
src/interfaces/libpq/libpq-fe.h | 22 +
src/interfaces/libpq/libpq-int.h | 34 +
src/interfaces/libpq/nls.mk | 2 +-
src/interfaces/libpq/test/.gitignore | 1 +
src/interfaces/libpq/test/Makefile | 7 +
.../postgres_fdw/expected/postgres_fdw.out | 2 +-
102 files changed, 4407 insertions(+), 130 deletions(-)
create mode 100644 src/test/column_encryption/.gitignore
create mode 100644 src/test/column_encryption/Makefile
create mode 100644 src/test/column_encryption/t/001_column_encryption.pl
create mode 100644 src/test/column_encryption/t/002_cmk_rotation.pl
create mode 100644 src/test/column_encryption/test_client.c
create mode 100755 src/test/column_encryption/test_run_decrypt.pl
create mode 100644 src/test/regress/expected/column_encryption.out
create mode 100644 src/test/regress/sql/column_encryption.sql
create mode 100644 src/include/catalog/pg_colenckey.h
create mode 100644 src/include/catalog/pg_colenckeydata.h
create mode 100644 src/include/catalog/pg_colmasterkey.h
create mode 100644 src/interfaces/libpq/fe-encrypt-openssl.c
create mode 100644 src/interfaces/libpq/fe-encrypt.h
diff --git a/doc/src/sgml/acronyms.sgml b/doc/src/sgml/acronyms.sgml
index 9ed148ab84..e21c9bcce3 100644
--- a/doc/src/sgml/acronyms.sgml
+++ b/doc/src/sgml/acronyms.sgml
@@ -56,6 +56,15 @@ <title>Acronyms</title>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><acronym>CEK</acronym></term>
+ <listitem>
+ <para>
+ Column Encryption Key
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><acronym>CIDR</acronym></term>
<listitem>
@@ -67,6 +76,15 @@ <title>Acronyms</title>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><acronym>CMK</acronym></term>
+ <listitem>
+ <para>
+ Column Master Key
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><acronym>CPAN</acronym></term>
<listitem>
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 4f3f375a84..c5e30f8029 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -105,6 +105,21 @@ <title>System Catalogs</title>
<entry>collations (locale information)</entry>
</row>
+ <row>
+ <entry><link linkend="catalog-pg-colenckey"><structname>pg_colenckey</structname></link></entry>
+ <entry>column encryption keys</entry>
+ </row>
+
+ <row>
+ <entry><link linkend="catalog-pg-colenckeydata"><structname>pg_colenckeydata</structname></link></entry>
+ <entry>column encryption key data</entry>
+ </row>
+
+ <row>
+ <entry><link linkend="catalog-pg-colmasterkey"><structname>pg_colmasterkey</structname></link></entry>
+ <entry>column master keys</entry>
+ </row>
+
<row>
<entry><link linkend="catalog-pg-constraint"><structname>pg_constraint</structname></link></entry>
<entry>check constraints, unique constraints, primary key constraints, foreign key constraints</entry>
@@ -1360,6 +1375,40 @@ <title><structname>pg_attribute</structname> Columns</title>
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>attcek</structfield> <type>oid</type>
+ (references <link linkend="catalog-pg-colenckey"><structname>pg_colenckey</structname></link>.<structfield>oid</structfield>)
+ </para>
+ <para>
+ If the column is encrypted, a reference to the column encryption key, else 0.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>attrealtypid</structfield> <type>oid</type>
+ (references <link linkend="catalog-pg-type"><structname>pg_type</structname></link>.<structfield>oid</structfield>)
+ </para>
+ <para>
+ If the column is encrypted, then this column indicates the type of the
+ encrypted data that is reported to the client. For encrypted columns,
+ the field <structfield>atttypid</structfield> is either
+ <type>pg_encrypted_det</type> or <type>pg_encrypted_rnd</type>. If the
+ column is not encrypted, then 0.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>attencalg</structfield> <type>int16</type>
+ </para>
+ <para>
+ If the column is encrypted, the identifier of the encryption algorithm;
+ see <xref linkend="protocol-cek"/> for possible values.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>attinhcount</structfield> <type>int4</type>
@@ -2437,6 +2486,232 @@ <title><structname>pg_collation</structname> Columns</title>
</para>
</sect1>
+ <sect1 id="catalog-pg-colenckey">
+ <title><structname>pg_colenckey</structname></title>
+
+ <indexterm zone="catalog-pg-colenckey">
+ <primary>pg_colenckey</primary>
+ </indexterm>
+
+ <para>
+ The catalog <structname>pg_colenckey</structname> contains information
+ about the column encryption keys in the database. The actual key material
+ of the column encryption keys is in the catalog <link
+ linkend="catalog-pg-colenckeydata"><structname>pg_colenckeydata</structname></link>.
+ </para>
+
+ <table>
+ <title><structname>pg_colenckey</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>oid</structfield> <type>oid</type>
+ </para>
+ <para>
+ Row identifier
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>cekname</structfield> <type>name</type>
+ </para>
+ <para>
+ Column encryption key name
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>cekowner</structfield> <type>oid</type>
+ (references <link linkend="catalog-pg-authid"><structname>pg_authid</structname></link>.<structfield>oid</structfield>)
+ </para>
+ <para>
+ Owner of the column encryption key
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="catalog-pg-colenckeydata">
+ <title><structname>pg_colenckeydata</structname></title>
+
+ <indexterm zone="catalog-pg-colenckeydata">
+ <primary>pg_colenckeydata</primary>
+ </indexterm>
+
+ <para>
+ The catalog <structname>pg_colenckeydata</structname> contains the key
+ material of column encryption keys. Each column encryption key object can
+ contain several versions of the key material, each encrypted with a
+ different column master key. That allows the gradual rotation of the
+ column master keys. Thus, <literal>(ckdcekid, ckdcmkid)</literal> is a
+ unique key of this table.
+ </para>
+
+ <para>
+ The key material of column encryption keys should never be decrypted inside
+ the database instance. It is meant to be sent as-is to the client, where
+ it is decrypted using the associated column master key, and then used to
+ encrypt or decrypt column values.
+ </para>
+
+ <table>
+ <title><structname>pg_colenckeydata</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>oid</structfield> <type>oid</type>
+ </para>
+ <para>
+ Row identifier
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>ckdcekid</structfield> <type>oid</type>
+ (references <link linkend="catalog-pg-colenckey"><structname>pg_colenckey</structname></link>.<structfield>oid</structfield>)
+ </para>
+ <para>
+ The column encryption key this entry belongs to
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>ckdcmkid</structfield> <type>oid</type>
+ (references <link linkend="catalog-pg-colmasterkey"><structname>pg_colmasterkey</structname></link>.<structfield>oid</structfield>)
+ </para>
+ <para>
+ The column master key that the key material is encrypted with
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>ckdcmkalg</structfield> <type>int16</type>
+ </para>
+ <para>
+ The encryption algorithm used for encrypting the key material; see
+ <xref linkend="protocol-cmk"/> for possible values.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>ckdencval</structfield> <type>bytea</type>
+ </para>
+ <para>
+ The key material of this column encryption key, encrypted using the
+ referenced column master key
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
+ <sect1 id="catalog-pg-colmasterkey">
+ <title><structname>pg_colmasterkey</structname></title>
+
+ <indexterm zone="catalog-pg-colmasterkey">
+ <primary>pg_colmasterkey</primary>
+ </indexterm>
+
+ <para>
+ The catalog <structname>pg_colmasterkey</structname> contains information
+ about column master keys. The keys themselves are not stored in the
+ database. The catalog entry only contains information that is used by
+ clients to locate the keys, for example in a file or in a key management
+ system.
+ </para>
+
+ <table>
+ <title><structname>pg_colmasterkey</structname> Columns</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ Column Type
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>oid</structfield> <type>oid</type>
+ </para>
+ <para>
+ Row identifier
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>cmkname</structfield> <type>name</type>
+ </para>
+ <para>
+ Column master key name
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>cmkowner</structfield> <type>oid</type>
+ (references <link linkend="catalog-pg-authid"><structname>pg_authid</structname></link>.<structfield>oid</structfield>)
+ </para>
+ <para>
+ Owner of the column master key
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>cmkrealm</structfield> <type>text</type>
+ </para>
+ <para>
+ A <quote>realm</quote> associated with this column master key. This is
+ a freely chosen string that is used by clients to determine how to look
+ up the key. A typical configuration would put all CMKs that are looked
+ up in the same way into the same realm.
+ </para></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+
<sect1 id="catalog-pg-constraint">
<title><structname>pg_constraint</structname></title>
diff --git a/doc/src/sgml/datatype.sgml b/doc/src/sgml/datatype.sgml
index 8e30b82273..cdd0530eeb 100644
--- a/doc/src/sgml/datatype.sgml
+++ b/doc/src/sgml/datatype.sgml
@@ -5342,4 +5342,51 @@ <title>Pseudo-Types</title>
</sect1>
+ <sect1 id="datatype-encrypted">
+ <title>Types Related to Encryption</title>
+
+ <para>
+ An encrypted column value (see <xref linkend="ddl-column-encryption"/>) is
+ internally stored using the types
+ <type>pg_encrypted_rnd</type> (for randomized encryption) or
+ <type>pg_encrypted_det</type> (for deterministic encryption); see <xref
+ linkend="datatype-encrypted-table"/>. Most of the database system treats
+ this as normal types. For example, the type <type>pg_encrypted_det</type> has
+ an equals operator that allows lookup of encrypted values. The external
+ representation of these types is the same as the <type>bytea</type> type.
+ Thus, clients that don't support transparent column encryption or have
+ disabled it will see the encrypted values as byte arrays. Clients that
+ support transparent data encryption will not see these types in result
+ sets, as the protocol layer will translate them back to declared
+ underlying type in the table definition.
+ </para>
+
+ <table id="datatype-encrypted-table">
+ <title>Types Related to Encryption</title>
+ <tgroup cols="3">
+ <colspec colname="col1" colwidth="1*"/>
+ <colspec colname="col2" colwidth="3*"/>
+ <colspec colname="col3" colwidth="2*"/>
+ <thead>
+ <row>
+ <entry>Name</entry>
+ <entry>Storage Size</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry><type>pg_encrypted_det</type></entry>
+ <entry>1 or 4 bytes plus the actual binary string</entry>
+ <entry>encrypted column value, deterministic encryption</entry>
+ </row>
+ <row>
+ <entry><type>pg_encrypted_rnd</type></entry>
+ <entry>1 or 4 bytes plus the actual binary string</entry>
+ <entry>encrypted column value, randomized encryption</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
</chapter>
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index b01e3ad544..bb8503584e 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -1211,6 +1211,79 @@ <title>Exclusion Constraints</title>
</sect2>
</sect1>
+ <sect1 id="ddl-column-encryption">
+ <title>Transparent Column Encryption</title>
+
+ <para>
+ With <firstterm>transparent column encryption</firstterm>, columns can be
+ stored encrypted in the database. The encryption and decryption happens on
+ the client, so that the plaintext value is never seen in the database
+ instance or on the server hosting the database. The drawback is that most
+ operations, such as function calls or sorting, are not possible on
+ encrypted values.
+ </para>
+
+ <para>
+ Tranparent column encryption uses two levels of cryptographic keys. The
+ actual column value is encrypted using a symmetric algorithm, such as AES,
+ using a <firstterm>column encryption key</firstterm>
+ (<acronym>CEK</acronym>). The column encryption key is in turn encrypted
+ using an assymmetric algorithm, such as RSA, using a <firstterm>column
+ master key</firstterm> (<acronym>CMK</acronym>). The encrypted CEK is
+ stored in the database system. The CMK is not stored in the database
+ system; it is stored on the client or somewhere where the client can access
+ it, such as in a local file or in a key management system. The database
+ system only records where the CMK is stored and provides this information
+ to the client. When rows containing encrypted columns are sent to the
+ client, the server first sends any necessary CMK information, followed by
+ any required CEK. The client then looks up the CMK and uses that to
+ decrypt the CEK. Then it decrypts incoming row data using the CEK and
+ provides the decrypted row data to the application.
+ </para>
+
+ <para>
+ Here is an example declaring a column as encrypted:
+<programlisting>
+CREATE TABLE customers (
+ id int PRIMARY KEY,
+ name text NOT NULL,
+ ...
+ creditcard_num text <emphasis>ENCRYPTED WITH (column_encryption_key = cek1)</emphasis>
+);
+</programlisting>
+ </para>
+
+ <para>
+ Column encryption supports <firstterm>randomized</firstterm>
+ (also known as <firstterm>probabilistic</firstterm>) and
+ <firstterm>deterministic</firstterm> encryption. The above example uses
+ randomized encryption, which is the default. Randomized encryption uses a
+ random initialization vector for each encryption, so that even if the
+ plaintext of two rows is equal, the encrypted values will be different.
+ This prevents someone with direct access to the database server to make
+ computations such as distinct counts on the encrypted values.
+ Deterministic encryption uses a fixed initialization vector. This reduces
+ security, but it allows equality searches on encrypted values. The
+ following example declares a column with deterministic encryption:
+<programlisting>
+CREATE TABLE employees (
+ id int PRIMARY KEY,
+ name text NOT NULL,
+ ...
+ ssn text ENCRYPTED WITH (
+ column_encryption_key = cek1, <emphasis>encryption_type = deterministic</emphasis>)
+);
+</programlisting>
+ </para>
+
+ <para>
+ Null values are not encrypted by transparent column encryption; null values
+ sent by the client are visible as null values in the database. If the fact
+ that a value is null needs to be hidden from the server, this information
+ needs to be encoded into a nonnull value in the client somehow.
+ </para>
+ </sect1>
+
<sect1 id="ddl-system-columns">
<title>System Columns</title>
diff --git a/doc/src/sgml/glossary.sgml b/doc/src/sgml/glossary.sgml
index d6d0a3a814..10507e4391 100644
--- a/doc/src/sgml/glossary.sgml
+++ b/doc/src/sgml/glossary.sgml
@@ -353,6 +353,29 @@ <title>Glossary</title>
</glossdef>
</glossentry>
+ <glossentry id="glossary-column-encryption-key">
+ <glossterm>Column encryption key</glossterm>
+ <glossdef>
+ <para>
+ A cryptographic key used to encrypt column values when using transparent
+ column encryption. Column encryption keys are stored in the database
+ encrypted by another key, the column master key.
+ </para>
+ </glossdef>
+ </glossentry>
+
+ <glossentry id="glossary-column-master-key">
+ <glossterm>Column master key</glossterm>
+ <glossdef>
+ <para>
+ A cryptographic key used to encrypt column encryption keys. (So the
+ column master key is a <firstterm>key encryption key</firstterm>.)
+ Column master keys are stored outside the database system, for example in
+ a key management system.
+ </para>
+ </glossdef>
+ </glossentry>
+
<glossentry id="glossary-commit">
<glossterm>Commit</glossterm>
<glossdef>
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 74456aa69d..310aa49e37 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -1974,6 +1974,110 @@ <title>Parameter Key Words</title>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="libpq-connect-cmklookup" xreflabel="cmklookup">
+ <term><literal>cmklookup</literal></term>
+ <listitem>
+ <para>
+ This specifies how libpq should look up column master keys (CMKs) in
+ order to decrypt the column encryption keys (CEKs).
+ The value is a list of <literal>key=value</literal> entries separated
+ by semicolons. Each key is the name of a key realm, or
+ <literal>*</literal> to match all realms. The value is a
+ <literal>scheme:data</literal> specification. The scheme specifies
+ the method to look up the key, the remaining data is specific to the
+ scheme. Placeholders are replaced in the remaining data as follows:
+
+ <variablelist>
+ <varlistentry>
+ <term><literal>%a</literal></term>
+ <listitem>
+ <para>
+ The CMK algorithm name (see <xref linkend="protocol-cmk-table"/>)
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>%b</literal></term>
+ <listitem>
+ <para>
+ The encrypted CEK data in Base64 encoding (only for the <literal>run</literal> scheme)
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>%k</literal></term>
+ <listitem>
+ <para>
+ The CMK key name
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>%r</literal></term>
+ <listitem>
+ <para>
+ The realm name
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ Available schemes are:
+ <variablelist>
+ <varlistentry>
+ <term><literal>file</literal></term>
+ <listitem>
+ <para>
+ Load the key material from a file. The remaining data is the file
+ name. Use this if the CMKs are kept in a file on the file system.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>run</literal></term>
+ <listitem>
+ <para>
+ Run the specified command to decrypt the CEK. The remaining data
+ is a shell command. Use this with key management systems that
+ perform the decryption themselves. The command must print the
+ decrypted plaintext in Base64 format on the standard output.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ The default value is empty.
+ </para>
+
+ <para>
+ Example:
+<programlisting>
+cmklookup="r1=file:/some/where/secrets/%k.pem;*=file:/else/where/%r/%k.pem"
+</programlisting>
+ This specification says, for keys in realm <quote>r1</quote>, load
+ them from the specified file, replacing <literal>%k</literal> by the
+ key name. For keys in other realms, load them from the file,
+ replacing realm and key names as specified.
+ </para>
+
+ <para>
+ An example for interacting with a (hypothetical) key management
+ system:
+<programlisting>
+cmklookup="*=run:acmekms decrypt --key %k --alg %a --blob '%b'"
+</programlisting>
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
</para>
</sect2>
@@ -3022,6 +3126,57 @@ <title>Main Functions</title>
</listitem>
</varlistentry>
+ <varlistentry id="libpq-PQexecPrepared2">
+ <term><function>PQexecPrepared2</function><indexterm><primary>PQexecPrepared2</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Sends a request to execute a prepared statement with given
+ parameters, and waits for the result, with support for encrypted columns.
+<synopsis>
+PGresult *PQexecPrepared2(PGconn *conn,
+ const char *stmtName,
+ int nParams,
+ const char * const *paramValues,
+ const int *paramLengths,
+ const int *paramFormats,
+ const int *paramForceColumnEncryptions,
+ int resultFormat,
+ PGresult *paramDesc);
+</synopsis>
+ </para>
+
+ <para>
+ <xref linkend="libpq-PQexecPrepared2"/> is like <xref
+ linkend="libpq-PQexecPrepared"/> with additional support for encrypted
+ columns. The parameter <parameter>paramDesc</parameter> must be a
+ result set obtained from <xref linkend="libpq-PQdescribePrepared"/> on
+ the same prepared statement.
+ </para>
+
+ <para>
+ This function must be used if a statement parameter corresponds to an
+ underlying encrypted column. In that situation, the prepared
+ statement needs to be described first so that libpq can obtain the
+ necessary key and other information from the server. When that is
+ done, the parameters corresponding to encrypted columns are
+ automatically encrypted appropriately before being sent to the server.
+ </para>
+
+ <para>
+ The parameter <parameter>paramForceColumnEncryptions</parameter>
+ specifies whether encryption should be forced for a parameter. If
+ encryption is forced for a parameter but it does not correspond to an
+ encrypted column on the server, then the call will fail and the
+ parameter will not be sent. This can be used for additional security
+ against a comprimised server. (The drawback is that application code
+ then needs to be kept up to date with knowledge about which columns
+ are encrypted rather than letting the server specify this.) If the
+ array pointer is null then encryption is not forced for any parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQdescribePrepared">
<term><function>PQdescribePrepared</function><indexterm><primary>PQdescribePrepared</primary></indexterm></term>
@@ -3872,6 +4027,28 @@ <title>Retrieving Query Result Information</title>
</listitem>
</varlistentry>
+ <varlistentry id="libpq-PQfisencrypted">
+ <term><function>PQfisencrypted</function><indexterm><primary>PQfisencrypted</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Returns whether the value for the given column came from an encrypted
+ column. Column numbers start at 0.
+<synopsis>
+int PQfisencrypted(const PGresult *res,
+ int column_number);
+</synopsis>
+ </para>
+
+ <para>
+ Encrypted column values are automatically decrypted, so this function
+ is not necessary to access the column value. It can be used for extra
+ security to check whether the value was stored encrypted when one
+ thought it should be.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQfsize">
<term><function>PQfsize</function><indexterm><primary>PQfsize</primary></indexterm></term>
@@ -4047,12 +4224,37 @@ <title>Retrieving Query Result Information</title>
<para>
This function is only useful when inspecting the result of
- <xref linkend="libpq-PQdescribePrepared"/>. For other types of queries it
+ <xref linkend="libpq-PQdescribePrepared"/>. For other types of results it
will return zero.
</para>
</listitem>
</varlistentry>
+ <varlistentry id="libpq-PQparamisencrypted">
+ <term><function>PQparamisencrypted</function><indexterm><primary>PQparamisencrypted</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Returns whether the value for the given parameter is destined for an
+ encrypted column. Parameter numbers start at 0.
+<synopsis>
+int PQparamisencrypted(const PGresult *res, int param_number);
+</synopsis>
+ </para>
+
+ <para>
+ Values for parameters destined for encrypted columns are automatically
+ encrypted, so this function is not necessary to prepare the parameter
+ value. It can be used for extra security to check whether the value
+ will be stored encrypted when one thought it should be. (But see also
+ at <xref linkend="libpq-PQexecPrepared2"/> for another way to do that.)
+ This function is only useful when inspecting the result of <xref
+ linkend="libpq-PQdescribePrepared"/>. For other types of results it
+ will return false.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQprint">
<term><function>PQprint</function><indexterm><primary>PQprint</primary></indexterm></term>
@@ -4578,6 +4780,7 @@ <title>Asynchronous Command Processing</title>
<xref linkend="libpq-PQsendQueryParams"/>,
<xref linkend="libpq-PQsendPrepare"/>,
<xref linkend="libpq-PQsendQueryPrepared"/>,
+ <xref linkend="libpq-PQsendQueryPrepared2"/>,
<xref linkend="libpq-PQsendDescribePrepared"/>, and
<xref linkend="libpq-PQsendDescribePortal"/>,
which can be used with <xref linkend="libpq-PQgetResult"/> to duplicate
@@ -4585,6 +4788,7 @@ <title>Asynchronous Command Processing</title>
<xref linkend="libpq-PQexecParams"/>,
<xref linkend="libpq-PQprepare"/>,
<xref linkend="libpq-PQexecPrepared"/>,
+ <xref linkend="libpq-PQexecPrepared2"/>,
<xref linkend="libpq-PQdescribePrepared"/>, and
<xref linkend="libpq-PQdescribePortal"/>
respectively.
@@ -4696,6 +4900,46 @@ <title>Asynchronous Command Processing</title>
</listitem>
</varlistentry>
+ <varlistentry id="libpq-PQsendQueryPrepared2">
+ <term><function>PQsendQueryPrepared2</function><indexterm><primary>PQsendQueryPrepared2</primary></indexterm></term>
+
+ <listitem>
+ <para>
+ Sends a request to execute a prepared statement with given
+ parameters, without waiting for the result(s), with support for encrypted columns.
+<synopsis>
+int PQsendQueryPrepared2(PGconn *conn,
+ const char *stmtName,
+ int nParams,
+ const char * const *paramValues,
+ const int *paramLengths,
+ const int *paramFormats,
+ const int *paramForceColumnEncryptions,
+ int resultFormat,
+ PGresult *paramDesc);
+</synopsis>
+ </para>
+
+ <para>
+ <xref linkend="libpq-PQsendQueryPrepared2"/> is like <xref
+ linkend="libpq-PQsendQueryPrepared"/> with additional support for encrypted
+ columns. The parameter <parameter>paramDesc</parameter> must be a
+ result set obtained from <xref linkend="libpq-PQsendDescribePrepared"/> on
+ the same prepared statement.
+ </para>
+
+ <para>
+ This function must be used if a statement parameter corresponds to an
+ underlying encrypted column. In that situation, the prepared
+ statement needs to be described first so that libpq can obtain the
+ necessary key and other information from the server. When that is
+ done, the parameters corresponding to encrypted columns are
+ automatically encrypted appropriately before being sent to the server.
+ See also under <xref linkend="libpq-PQexecPrepared2"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="libpq-PQsendDescribePrepared">
<term><function>PQsendDescribePrepared</function><indexterm><primary>PQsendDescribePrepared</primary></indexterm></term>
@@ -4746,6 +4990,7 @@ <title>Asynchronous Command Processing</title>
<xref linkend="libpq-PQsendQueryParams"/>,
<xref linkend="libpq-PQsendPrepare"/>,
<xref linkend="libpq-PQsendQueryPrepared"/>,
+ <xref linkend="libpq-PQsendQueryPrepared2"/>,
<xref linkend="libpq-PQsendDescribePrepared"/>,
<xref linkend="libpq-PQsendDescribePortal"/>, or
<xref linkend="libpq-PQpipelineSync"/>
@@ -7776,6 +8021,16 @@ <title>Environment Variables</title>
</para>
</listitem>
+ <listitem>
+ <para>
+ <indexterm>
+ <primary><envar>PGCMKLOOKUP</envar></primary>
+ </indexterm>
+ <envar>PGCMKLOOKUP</envar> behaves the same as the <xref
+ linkend="libpq-connect-cmklookup"/> connection parameter.
+ </para>
+ </listitem>
+
<listitem>
<para>
<indexterm>
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index c0b89a3c01..60595c33b1 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -1050,6 +1050,52 @@ <title>Extended Query</title>
</note>
</sect2>
+ <sect2>
+ <title>Transparent Column Encryption</title>
+
+ <para>
+ When transparent column encryption is enabled, the messages
+ ColumnMasterKey and ColumnEncryptionKey can appear before RowDescription
+ and ParameterDescription messages. Clients should collect the information
+ in these messages and keep them for the duration of the connection. A
+ server is not required to resend the key information for each statement
+ cycle if it was already sent during this connection. If a server resends
+ a key that the client has already stored (that is, a key having an ID
+ equal to one already stored), the new information should replace the old.
+ (This could happen, for example, if the key was altered by server-side DDL
+ commands.)
+ </para>
+
+ <para>
+ A client supporting transparent column encryption should automatically
+ decrypt the column value fields of DataRow messages corresponding to
+ encrypted columns, and it should automatically encrypt the parameter value
+ fields of Bind messages corresponding to encrypted columns.
+ </para>
+
+ <para>
+ When column encryption is used, the plaintext is always in text format
+ (not binary format).
+ </para>
+
+ <para>
+ When deterministic encryption is used, clients need to take care to
+ represent plaintext to be encrypted in a consistent form. For example,
+ encrypting an integer represented by the string <literal>100</literal> and
+ an integer represented by the string <literal>+100</literal> would result
+ in two different ciphertexts, thus defeating the main point of
+ deterministic encryption. This protocol specification requires the
+ plaintext to be in <quote>canonical</quote> form, which is the form that
+ is produced by the server when it outputs a particular value in text
+ format.
+ </para>
+
+ <para>
+ The cryptographic operations used for transparent column encryption are
+ described in <xref linkend="protocol-transparent-column-encryption"/>.
+ </para>
+ </sect2>
+
<sect2>
<title>Function Call</title>
@@ -3991,6 +4037,128 @@ <title>Message Formats</title>
</listitem>
</varlistentry>
+ <varlistentry id="protocol-message-formats-ColumnEncryptionKey">
+ <term>ColumnEncryptionKey (B)</term>
+ <listitem>
+ <variablelist>
+ <varlistentry>
+ <term>Byte1('Y')</term>
+ <listitem>
+ <para>
+ Identifies the message as a column encryption key message.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Int32</term>
+ <listitem>
+ <para>
+ Length of message contents in bytes, including self.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Int32</term>
+ <listitem>
+ <para>
+ The session-specific identifier of the key.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Int32</term>
+ <listitem>
+ <para>
+ The identifier of the master key used to encrypt this key.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Int16</term>
+ <listitem>
+ <para>
+ The identifier of the algorithm used to encrypt this key.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Int32</term>
+ <listitem>
+ <para>
+ The length of the following key material.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Byte<replaceable>n</replaceable></term>
+ <listitem>
+ <para>
+ The key material, encrypted with the master key referenced above.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="protocol-message-formats-ColumnMasterKey">
+ <term>ColumnMasterKey (B)</term>
+ <listitem>
+ <variablelist>
+ <varlistentry>
+ <term>Byte1('y')</term>
+ <listitem>
+ <para>
+ Identifies the message as a column master key message.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Int32</term>
+ <listitem>
+ <para>
+ Length of message contents in bytes, including self.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Int32</term>
+ <listitem>
+ <para>
+ The session-specific identifier of the key.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>String</term>
+ <listitem>
+ <para>
+ The name of the key.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>String</term>
+ <listitem>
+ <para>
+ The key's realm.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="protocol-message-formats-CommandComplete">
<term>CommandComplete (B)</term>
<listitem>
@@ -5086,6 +5254,37 @@ <title>Message Formats</title>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term>Int32</term>
+ <listitem>
+ <para>
+ If this parameter is to be encrypted, this specifies the
+ identifier of the column encryption key to use, else zero.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Int16</term>
+ <listitem>
+ <para>
+ If this parameter is to be encrypted, this specifies the
+ identifier of the encryption algorithm, else zero.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Int16</term>
+ <listitem>
+ <para>
+ This is used as a bit field of flags. If the column is
+ encrypted and bit 0x01 is set, the column uses deterministic
+ encryption, otherwise randomized encryption.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
</listitem>
</varlistentry>
@@ -5474,6 +5673,26 @@ <title>Message Formats</title>
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term>Int32</term>
+ <listitem>
+ <para>
+ If this parameter is to be encrypted, this specifies the
+ identifier of the column encryption key to use, else zero.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>Int16</term>
+ <listitem>
+ <para>
+ If this parameter is to be encrypted, this specifies the
+ identifier of the encrypt algorithm, else zero.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
</listitem>
</varlistentry>
@@ -7278,6 +7497,133 @@ <title>Logical Replication Message Formats</title>
</variablelist>
</sect1>
+ <sect1 id="protocol-transparent-column-encryption">
+ <title>Transparent Column Encryption Cryptography</title>
+
+ <para>
+ This section describes the cryptographic operations used by transparent
+ column encryption. A client that supports transparent column encryption
+ needs to implement these operations as specified here in order to be able
+ to interoperate with other clients.
+ </para>
+
+ <para>
+ Column encryption key algorithms and column master key algorithms are
+ identified by integers in the protocol messages and the system catalogs.
+ Additional algorithms may be added to this protocol specification without a
+ change in the protocol version number. Clients should implement support
+ for all the algorithms specified here. If a client encounters an algorithm
+ identifier it does not recognize or does not support, it must raise an
+ error. A suitable error message should be provided to the application or
+ user.
+ </para>
+
+ <sect2 id="protocol-cmk">
+ <title>Column Master Keys</title>
+
+ <para>
+ The currently defined algorithms for column master keys are listed in
+ <xref linkend="protocol-cmk-table"/>.
+ </para>
+
+ <table id="protocol-cmk-table">
+ <title>Column Master Key Algorithms</title>
+ <tgroup cols="3">
+ <thead>
+ <row>
+ <entry>ID</entry>
+ <entry>Name</entry>
+ <entry>Reference</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>1</entry>
+ <entry><literal>RSAES_OAEP_SHA_1</literal></entry>
+ <entry><ulink url="https://datatracker.ietf.org/doc/html/rfc8017">RFC 8017</ulink> (PKCS #1)</entry>
+ </row>
+ <row>
+ <entry>2</entry>
+ <entry><literal>RSAES_OAEP_SHA_256</literal></entry>
+ <entry><ulink url="https://datatracker.ietf.org/doc/html/rfc8017">RFC 8017</ulink> (PKCS #1)</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+
+ <sect2 id="protocol-cek">
+ <title>Column Encryption Keys</title>
+
+ <para>
+ The currently defined algorithms for column encryption keys are listed in
+ <xref linkend="protocol-cek-table"/>.
+ </para>
+
+ <table id="protocol-cek-table">
+ <title>Column Encryption Key Algorithms</title>
+ <tgroup cols="3">
+ <thead>
+ <row>
+ <entry>ID</entry>
+ <entry>Name</entry>
+ <entry>Reference</entry>
+ <entry>Key length (octets)</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>130</entry>
+ <entry><literal>AEAD_AES_128_CBC_HMAC_SHA_256</literal></entry>
+ <entry><ulink url="https://datatracker.ietf.org/doc/html/draft-mcgrew-aead-aes-cbc-hmac-sha2-05"/></entry>
+ <entry>32</entry>
+ </row>
+ <row>
+ <entry>131</entry>
+ <entry><literal>AEAD_AES_192_CBC_HMAC_SHA_384</literal></entry>
+ <entry><ulink url="https://datatracker.ietf.org/doc/html/draft-mcgrew-aead-aes-cbc-hmac-sha2-05"/></entry>
+ <entry>48</entry>
+ </row>
+ <row>
+ <entry>132</entry>
+ <entry><literal>AEAD_AES_256_CBC_HMAC_SHA_384</literal></entry>
+ <entry><ulink url="https://datatracker.ietf.org/doc/html/draft-mcgrew-aead-aes-cbc-hmac-sha2-05"/></entry>
+ <entry>56</entry>
+ </row>
+ <row>
+ <entry>133</entry>
+ <entry><literal>AEAD_AES_256_CBC_HMAC_SHA_512</literal></entry>
+ <entry><ulink url="https://datatracker.ietf.org/doc/html/draft-mcgrew-aead-aes-cbc-hmac-sha2-05"/></entry>
+ <entry>64</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ <para>
+ The <quote>associated data</quote> in these algorithms consists of 4
+ bytes: The ASCII letters <literal>P</literal> and <literal>G</literal>
+ (byte values 80 and 71), followed by the algorithm ID as a 16-bit unsigned
+ integer in network byte order.
+ </para>
+
+ <para>
+ The length of the initialization vector is 16 octets for all CEK algorithm
+ variants. For randomized encryption, the initialization vector should be
+ (cryptographically strong) random bytes. For deterministic encryption,
+ the initialization vector is constructed as
+<programlisting>
+SUBSTRING(<replaceable>HMAC</replaceable>(<replaceable>K</replaceable>, <replaceable>P</replaceable>) FOR <replaceable>IVLEN</replaceable>)
+</programlisting>
+ where <replaceable>HMAC</replaceable> is the HMAC function associated with
+ the algorithm, <replaceable>K</replaceable> is the prefix of the CEK of
+ the length required by the HMAC function, and <replaceable>P</replaceable>
+ is the plaintext to be encrypted. (This is the same portion of the CEK
+ that the AEAD_AES_*_HMAC_* algorithms use for the MAC part.)
+ </para>
+ </sect2>
+ </sect1>
+
<sect1 id="protocol-changes">
<title>Summary of Changes since Protocol 2.0</title>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 6c9918b0a1..7a1d03f868 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -22,7 +22,7 @@
<refsynopsisdiv>
<synopsis>
CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] <replaceable class="parameter">table_name</replaceable> ( [
- { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COMPRESSION <replaceable>compression_method</replaceable> ] [ COLLATE <replaceable>collation</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
+ { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COMPRESSION <replaceable>compression_method</replaceable> ] [ ENCRYPTED WITH ( <replaceable>encryption_options</replaceable> ) ] [ COLLATE <replaceable>collation</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
| <replaceable>table_constraint</replaceable>
| LIKE <replaceable>source_table</replaceable> [ <replaceable>like_option</replaceable> ... ] }
[, ... ]
@@ -322,6 +322,46 @@ <title>Parameters</title>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>ENCRYPTED WITH ( <replaceable>encryption_options</replaceable> )</literal></term>
+ <listitem>
+ <para>
+ Enables transparent column encryption for the column.
+ <replaceable>encryption_options</replaceable> are comma-separated
+ <literal>key=value</literal> specifications. The following options are
+ available:
+ <variablelist>
+ <varlistentry>
+ <term><literal>column_encryption_key</literal></term>
+ <listitem>
+ <para>
+ Specifies the name of the column encryption key to use. Specifying
+ this is mandatory.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>encryption_type</literal></term>
+ <listitem>
+ <para>
+ <literal>randomized</literal> (the default) or <literal>deterministic</literal>
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>algorithm</literal></term>
+ <listitem>
+ <para>
+ The encryption algorithm to use. The default is
+ <literal>AEAD_AES_128_CBC_HMAC_SHA_256</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>INHERITS ( <replaceable>parent_table</replaceable> [, ... ] )</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 65bb0a6a3f..e6c93f1b13 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -2255,6 +2255,28 @@ <title>Meta-Commands</title>
</varlistentry>
+ <varlistentry>
+ <term><literal>\gencr</literal> [ <replaceable class="parameter">parameter</replaceable> ] ... </term>
+
+ <listitem>
+ <para>
+ Sends the current query buffer to the server for execution, as with
+ <literal>\g</literal>, with the specified parameters passed for any
+ parameter placeholders (<literal>$1</literal> etc.). This command
+ ensures that any parameters corresponding to encrypted columns are
+ sent to the server encrypted.
+ </para>
+
+ <para>
+ Example:
+<programlisting>
+INSERT INTO tbl1 VALUES ($1, $2) \gencr 'first value' 'second value'
+</programlisting>
+ </para>
+ </listitem>
+ </varlistentry>
+
+
<varlistentry>
<term><literal>\getenv <replaceable class="parameter">psql_var</replaceable> <replaceable class="parameter">env_var</replaceable></literal></term>
@@ -3945,6 +3967,17 @@ <title>Variables</title>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><varname>HIDE_COLUMN_ENCRYPTION</varname></term>
+ <listitem>
+ <para>
+ If this variable is set to <literal>true</literal>, column encryption
+ details are not displayed. This is mainly useful for regression
+ tests.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><varname>HIDE_TOAST_COMPRESSION</varname></term>
<listitem>
diff --git a/src/test/Makefile b/src/test/Makefile
index 69ef074d75..2300be8332 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -32,6 +32,7 @@ SUBDIRS += ldap
endif
endif
ifeq ($(with_ssl),openssl)
+SUBDIRS += column_encryption
ifneq (,$(filter ssl,$(PG_TEST_EXTRA)))
SUBDIRS += ssl
endif
@@ -41,7 +42,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos icu ldap ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos icu ldap ssl column_encryption)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/column_encryption/.gitignore b/src/test/column_encryption/.gitignore
new file mode 100644
index 0000000000..456dbf69d2
--- /dev/null
+++ b/src/test/column_encryption/.gitignore
@@ -0,0 +1,3 @@
+/test_client
+# Generated by test suite
+/tmp_check/
diff --git a/src/test/column_encryption/Makefile b/src/test/column_encryption/Makefile
new file mode 100644
index 0000000000..7aca84b17a
--- /dev/null
+++ b/src/test/column_encryption/Makefile
@@ -0,0 +1,29 @@
+#-------------------------------------------------------------------------
+#
+# Makefile for src/test/column_encryption
+#
+# Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/test/column_encryption/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/test/column_encryption
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
+
+all: test_client
+
+check: all
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
+
+clean distclean maintainer-clean:
+ rm -f test_client.o test_client
+ rm -rf tmp_check
diff --git a/src/test/column_encryption/t/001_column_encryption.pl b/src/test/column_encryption/t/001_column_encryption.pl
new file mode 100644
index 0000000000..7ab98981d8
--- /dev/null
+++ b/src/test/column_encryption/t/001_column_encryption.pl
@@ -0,0 +1,206 @@
+# Copyright (c) 2021-2022, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $node = PostgreSQL::Test::Cluster->new('node');
+$node->init;
+$node->start;
+
+
+sub create_cmk
+{
+ my ($cmkname) = @_;
+
+ my $cmkfilename = "${PostgreSQL::Test::Utils::tmp_check}/${cmkname}.pem";
+
+ system_or_bail 'openssl', 'genpkey', '-algorithm', 'rsa', '-out', $cmkfilename;
+
+ $node->safe_psql('postgres', qq{
+INSERT INTO pg_colmasterkey (oid, cmkname, cmkowner, cmkrealm) VALUES (
+ pg_nextoid('pg_catalog.pg_colmasterkey', 'oid', 'pg_catalog.pg_colmasterkey_oid_index'),
+ '${cmkname}',
+ (select oid from pg_roles where rolname = current_user),
+ ''
+);
+});
+
+ return $cmkfilename;
+}
+
+sub create_cek
+{
+ my ($cekname, $bytes, $cmkname, $cmkfilename) = @_;
+
+ # generate random bytes
+ system_or_bail 'openssl', 'rand', '-out', "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin", $bytes;
+
+ # encrypt CEK using CMK
+ system_or_bail 'openssl', 'pkeyutl', '-encrypt',
+ '-inkey', $cmkfilename,
+ '-pkeyopt', 'rsa_padding_mode:oaep',
+ '-in', "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin",
+ '-out', "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin.enc";
+
+ my $cekenchex = unpack('H*', slurp_file "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin.enc");
+
+ # create CEK in database
+ $node->safe_psql('postgres', qq{
+INSERT INTO pg_colenckey (oid, cekname, cekowner) VALUES (
+ pg_nextoid('pg_catalog.pg_colenckey', 'oid', 'pg_catalog.pg_colenckey_oid_index'),
+ '${cekname}',
+ (select oid from pg_roles where rolname = current_user)
+);
+});
+ $node->safe_psql('postgres', qq{
+INSERT INTO pg_colenckeydata (oid, ckdcekid, ckdcmkid, ckdcmkalg, ckdencval) VALUES (
+ pg_nextoid('pg_catalog.pg_colenckeydata', 'oid', 'pg_catalog.pg_colenckeydata_oid_index'),
+ (SELECT oid FROM pg_colenckey WHERE cekname = '${cekname}'),
+ (SELECT oid FROM pg_colmasterkey WHERE cmkname = '${cmkname}'),
+ 1,
+ '\\x${cekenchex}'
+);
+});
+
+ return;
+}
+
+
+my $cmk1filename = create_cmk('cmk1');
+my $cmk2filename = create_cmk('cmk2');
+create_cek('cek1', 32, 'cmk1', $cmk1filename);
+create_cek('cek2', 48, 'cmk2', $cmk2filename);
+
+$ENV{'PGCMKLOOKUP'} = '*=file:' . ${PostgreSQL::Test::Utils::tmp_check} . '/%k.pem';
+
+
+$node->safe_psql('postgres', qq{
+CREATE TABLE tbl1 (
+ a int,
+ b text ENCRYPTED WITH (column_encryption_key = cek1)
+);
+});
+
+$node->safe_psql('postgres', qq{
+CREATE TABLE tbl2 (
+ a int,
+ b text ENCRYPTED WITH (encryption_type = deterministic, column_encryption_key = cek1)
+);
+});
+
+$node->safe_psql('postgres', qq{
+CREATE TABLE tbl3 (
+ a int,
+ b text ENCRYPTED WITH (column_encryption_key = cek1),
+ c text ENCRYPTED WITH (column_encryption_key = cek2, algorithm = 'AEAD_AES_192_CBC_HMAC_SHA_384')
+);
+});
+
+$node->safe_psql('postgres', q{
+INSERT INTO tbl1 (a, b) VALUES (1, $1) \gencr 'val1'
+INSERT INTO tbl1 (a, b) VALUES (2, $1) \gencr 'val2'
+});
+
+# Expected ciphertext length is 2 blocks of AES output (2 * 16) plus
+# half SHA-256 output (16) in hex encoding: (2 * 16 + 16) * 2 = 96
+like($node->safe_psql('postgres', q{COPY (SELECT * FROM tbl1) TO STDOUT}),
+ qr/1\t\\\\x[0-9a-f]{96}\n2\t\\\\x[0-9a-f]{96}/,
+ 'inserted data is encrypted');
+
+my $result;
+
+$result = $node->safe_psql('postgres', q{SELECT a, b FROM tbl1 \gdesc});
+is($result,
+ q(a|integer
+b|text),
+ 'query result description has original type');
+
+$result = $node->safe_psql('postgres', q{SELECT a, b FROM tbl1});
+is($result,
+ q(1|val1
+2|val2),
+ 'decrypted query result');
+
+{
+ local $ENV{'PGCMKLOOKUP'} = '*=run:broken %k "%b"';
+ $result = $node->psql('postgres', q{SELECT a, b FROM tbl1});
+ isnt($result, 0, 'query fails with broken cmklookup run setting');
+}
+
+{
+ local $ENV{'PGCMKLOOKUP'} = "*=run:perl ./test_run_decrypt.pl '${PostgreSQL::Test::Utils::tmp_check}' %k %a '%b'";
+
+ $result = $node->safe_psql('postgres', q{SELECT a, b FROM tbl1});
+ is($result,
+ q(1|val1
+2|val2),
+ 'decrypted query result with cmklookup run');
+}
+
+$node->safe_psql('postgres', q{
+INSERT INTO tbl3 (a, b, c) VALUES (1, $1, $2) \gencr 'valB1' 'valC1'
+});
+
+$result = $node->safe_psql('postgres', q{SELECT a, b, c FROM tbl3});
+is($result,
+ q(1|valB1|valC1),
+ 'decrypted query result multiple keys');
+
+
+$node->command_fails_like(['test_client', 'test1'], qr/not encrypted/, 'test client fails because parameters not encrypted');
+
+$result = $node->safe_psql('postgres', q{SELECT a, b FROM tbl1});
+is($result,
+ q(1|val1
+2|val2),
+ 'decrypted query result after test client insert');
+
+$node->command_ok(['test_client', 'test2'], 'test client test 2');
+
+$result = $node->safe_psql('postgres', q{SELECT a, b FROM tbl1});
+is($result,
+ q(1|val1
+2|val2
+3|val3),
+ 'decrypted query result after test client insert 2');
+
+like($node->safe_psql('postgres', q{COPY (SELECT * FROM tbl1 WHERE a = 3) TO STDOUT}),
+ qr/3\t\\\\x[0-9a-f]{96}/,
+ 'inserted data is encrypted');
+
+$node->command_ok(['test_client', 'test3'], 'test client test 3');
+
+$result = $node->safe_psql('postgres', q{SELECT a, b FROM tbl1});
+is($result,
+ q(1|val1
+2|val2
+3|val3upd),
+ 'decrypted query result after test client insert 3');
+
+$node->command_ok(['test_client', 'test4'], 'test client test 4');
+
+$result = $node->safe_psql('postgres', q{SELECT a, b FROM tbl2});
+is($result,
+ q(1|valA
+2|valB
+3|valA),
+ 'decrypted query result after test client insert 4');
+
+is($node->safe_psql('postgres', q{SELECT b, count(*) FROM tbl2 GROUP BY b ORDER BY 2}),
+ q(valB|1
+valA|2),
+ 'group by deterministically encrypted column');
+
+$node->command_ok(['test_client', 'test5'], 'test client test 5');
+
+$result = $node->safe_psql('postgres', q{SELECT a, b, c FROM tbl3});
+is($result,
+ q(1|valB1|valC1
+2|valB2|valC2
+3|valB3|valC3),
+ 'decrypted query result multiple keys after test client insert 5');
+
+done_testing();
diff --git a/src/test/column_encryption/t/002_cmk_rotation.pl b/src/test/column_encryption/t/002_cmk_rotation.pl
new file mode 100644
index 0000000000..5a13a206eb
--- /dev/null
+++ b/src/test/column_encryption/t/002_cmk_rotation.pl
@@ -0,0 +1,144 @@
+# Copyright (c) 2021-2022, PostgreSQL Global Development Group
+
+# Test column master key rotation. First, we generate CMK1 and a CEK
+# encrypted with it. Then we add a CMK2 and encrypt the CEK with it
+# as well. (Recall that a CEK can be associated with multiple CMKs,
+# for this reason. That's why pg_colenckeydata is split out from
+# pg_colenckey.) Then we remove CMK1. We test that we can get
+# decrypted query results at each step.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $node = PostgreSQL::Test::Cluster->new('node');
+$node->init;
+$node->start;
+
+
+sub create_cmk
+{
+ my ($cmkname) = @_;
+
+ my $cmkfilename = "${PostgreSQL::Test::Utils::tmp_check}/${cmkname}.pem";
+
+ system_or_bail 'openssl', 'genpkey', '-algorithm', 'rsa', '-out', $cmkfilename;
+
+ $node->safe_psql('postgres', qq{
+INSERT INTO pg_colmasterkey (oid, cmkname, cmkowner, cmkrealm) VALUES (
+ pg_nextoid('pg_catalog.pg_colmasterkey', 'oid', 'pg_catalog.pg_colmasterkey_oid_index'),
+ '${cmkname}',
+ (select oid from pg_roles where rolname = current_user),
+ ''
+);
+});
+
+ return $cmkfilename;
+}
+
+
+my $cmk1filename = create_cmk('cmk1');
+
+# create CEK
+my ($cekname, $bytes) = ('cek1', 16+16);
+
+# generate random bytes
+system_or_bail 'openssl', 'rand', '-out', "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin", $bytes;
+
+# encrypt CEK using CMK
+system_or_bail 'openssl', 'pkeyutl', '-encrypt',
+ '-inkey', $cmk1filename,
+ '-pkeyopt', 'rsa_padding_mode:oaep',
+ '-in', "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin",
+ '-out', "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin.enc";
+
+my $cekenchex = unpack('H*', slurp_file "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin.enc");
+
+# create CEK in database
+$node->safe_psql('postgres', qq{
+INSERT INTO pg_colenckey (oid, cekname, cekowner) VALUES (
+ pg_nextoid('pg_catalog.pg_colenckey', 'oid', 'pg_catalog.pg_colenckey_oid_index'),
+ '${cekname}',
+ (select oid from pg_roles where rolname = current_user)
+);
+});
+
+$node->safe_psql('postgres', qq{
+INSERT INTO pg_colenckeydata (oid, ckdcekid, ckdcmkid, ckdcmkalg, ckdencval) VALUES (
+ pg_nextoid('pg_catalog.pg_colenckeydata', 'oid', 'pg_catalog.pg_colenckeydata_oid_index'),
+ (SELECT oid FROM pg_colenckey WHERE cekname = '${cekname}'),
+ (SELECT oid FROM pg_colmasterkey WHERE cmkname = 'cmk1'),
+ 1,
+ '\\x${cekenchex}'
+);
+});
+
+$ENV{'PGCMKLOOKUP'} = '*=file:' . ${PostgreSQL::Test::Utils::tmp_check} . '/%k.pem';
+
+$node->safe_psql('postgres', qq{
+CREATE TABLE tbl1 (
+ a int,
+ b text ENCRYPTED WITH (column_encryption_key = cek1)
+);
+});
+
+$node->safe_psql('postgres', q{
+INSERT INTO tbl1 (a, b) VALUES (1, $1) \gencr 'val1'
+INSERT INTO tbl1 (a, b) VALUES (2, $1) \gencr 'val2'
+});
+
+is($node->safe_psql('postgres', q{SELECT a, b FROM tbl1}),
+ q(1|val1
+2|val2),
+ 'decrypted query result with one CMK');
+
+
+# create new CMK
+my $cmk2filename = create_cmk('cmk2');
+
+# encrypt CEK using new CMK
+#
+# (Here, we still have the plaintext of the CEK available from
+# earlier. In reality, one would decrypt the CEK with the first CMK
+# and then re-encrypt it with the second CMK.)
+system_or_bail 'openssl', 'pkeyutl', '-encrypt',
+ '-inkey', $cmk2filename,
+ '-pkeyopt', 'rsa_padding_mode:oaep',
+ '-in', "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin",
+ '-out', "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin.enc";
+
+$cekenchex = unpack('H*', slurp_file "${PostgreSQL::Test::Utils::tmp_check}/${cekname}.bin.enc");
+
+# add new data record for CEK in database
+$node->safe_psql('postgres', qq{
+INSERT INTO pg_colenckeydata (oid, ckdcekid, ckdcmkid, ckdcmkalg, ckdencval) VALUES (
+ pg_nextoid('pg_catalog.pg_colenckeydata', 'oid', 'pg_catalog.pg_colenckeydata_oid_index'),
+ (SELECT oid FROM pg_colenckey WHERE cekname = '${cekname}'),
+ (SELECT oid FROM pg_colmasterkey WHERE cmkname = 'cmk2'),
+ 1,
+ '\\x${cekenchex}'
+);
+});
+
+
+is($node->safe_psql('postgres', q{SELECT a, b FROM tbl1}),
+ q(1|val1
+2|val2),
+ 'decrypted query result with two CMKs');
+
+
+# delete CEK record for first CMK
+$node->safe_psql('postgres', qq{
+DELETE FROM pg_colenckeydata WHERE ckdcmkid = (SELECT oid FROM pg_colmasterkey WHERE cmkname = 'cmk1');
+});
+
+
+is($node->safe_psql('postgres', q{SELECT a, b FROM tbl1}),
+ q(1|val1
+2|val2),
+ 'decrypted query result with only new CMK');
+
+
+done_testing();
diff --git a/src/test/column_encryption/test_client.c b/src/test/column_encryption/test_client.c
new file mode 100644
index 0000000000..add9fe40a6
--- /dev/null
+++ b/src/test/column_encryption/test_client.c
@@ -0,0 +1,210 @@
+/*
+ * Copyright (c) 2021-2022, PostgreSQL Global Development Group
+ */
+
+#include "postgres_fe.h"
+
+#include "libpq-fe.h"
+
+
+static int
+test1(PGconn *conn)
+{
+ PGresult *res;
+ const char *values[] = {"3", "val3"};
+
+ res = PQexecParams(conn, "INSERT INTO tbl1 (a, b) VALUES ($1, $2)",
+ 2, NULL, values, NULL, NULL, 0);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQexecParams() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ return 0;
+}
+
+static int
+test2(PGconn *conn)
+{
+ PGresult *res,
+ *res2;
+ const char *values[] = {"3", "val3"};
+ int forces[] = {false, true};
+
+ res = PQprepare(conn, "", "INSERT INTO tbl1 (a, b) VALUES ($1, $2)",
+ 2, NULL);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQprepare() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ res2 = PQdescribePrepared(conn, "");
+ if (PQresultStatus(res2) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQdescribePrepared() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ if (!(!PQparamisencrypted(res2, 0) &&
+ PQparamisencrypted(res2, 1)))
+ {
+ fprintf(stderr, "wrong results from PQparamisencrypted()\n");
+ return 1;
+ }
+
+ res = PQexecPrepared2(conn, "", 2, values, NULL, NULL, forces, 0, res2);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQexecPrepared() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ return 0;
+}
+
+static int
+test3(PGconn *conn)
+{
+ PGresult *res,
+ *res2;
+ const char *values[] = {"3", "val3upd"};
+
+ res = PQprepare(conn, "", "UPDATE tbl1 SET b = $2 WHERE a = $1",
+ 2, NULL);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQprepare() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ res2 = PQdescribePrepared(conn, "");
+ if (PQresultStatus(res2) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQdescribePrepared() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ res = PQexecPrepared2(conn, "", 2, values, NULL, NULL, NULL, 0, res2);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQexecPrepared() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ return 0;
+}
+
+static int
+test4(PGconn *conn)
+{
+ PGresult *res,
+ *res2;
+ const char *values[] = {"1", "valA", "2", "valB", "3", "valA"};
+
+ res = PQprepare(conn, "", "INSERT INTO tbl2 (a, b) VALUES ($1, $2), ($3, $4), ($5, $6)",
+ 6, NULL);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQprepare() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ res2 = PQdescribePrepared(conn, "");
+ if (PQresultStatus(res2) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQdescribePrepared() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ res = PQexecPrepared2(conn, "", 6, values, NULL, NULL, NULL, 0, res2);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQexecPrepared() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ return 0;
+}
+
+static int
+test5(PGconn *conn)
+{
+ PGresult *res,
+ *res2;
+ const char *values[] = {
+ "2", "valB2", "valC2",
+ "3", "valB3", "valC3"
+ };
+
+ res = PQprepare(conn, "", "INSERT INTO tbl3 (a, b, c) VALUES ($1, $2, $3), ($4, $5, $6)",
+ 6, NULL);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQprepare() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ res2 = PQdescribePrepared(conn, "");
+ if (PQresultStatus(res2) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQdescribePrepared() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ res = PQexecPrepared2(conn, "", 6, values, NULL, NULL, NULL, 0, res2);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, "PQexecPrepared() failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ return 0;
+}
+
+int
+main(int argc, char **argv)
+{
+ PGconn *conn;
+ int ret = 0;
+
+ conn = PQconnectdb("");
+ if (PQstatus(conn) != CONNECTION_OK)
+ {
+ fprintf(stderr, "Connection to database failed: %s\n",
+ PQerrorMessage(conn));
+ return 1;
+ }
+
+ if (argc < 2 || argv[1] == NULL)
+ return 87;
+ else if (strcmp(argv[1], "test1") == 0)
+ ret = test1(conn);
+ else if (strcmp(argv[1], "test2") == 0)
+ ret = test2(conn);
+ else if (strcmp(argv[1], "test3") == 0)
+ ret = test3(conn);
+ else if (strcmp(argv[1], "test4") == 0)
+ ret = test4(conn);
+ else if (strcmp(argv[1], "test5") == 0)
+ ret = test5(conn);
+ else
+ ret = 88;
+
+ PQfinish(conn);
+ return ret;
+}
diff --git a/src/test/column_encryption/test_run_decrypt.pl b/src/test/column_encryption/test_run_decrypt.pl
new file mode 100755
index 0000000000..9664349f14
--- /dev/null
+++ b/src/test/column_encryption/test_run_decrypt.pl
@@ -0,0 +1,41 @@
+#!/usr/bin/perl
+
+# Test/sample command for libpq cmklookup run scheme
+#
+# This just places the data into temporary files and runs the openssl
+# command on it. (In practice, this could more simply be written as a
+# shell script, but this way it's more portable.)
+
+# Copyright (c) 2021-2022, PostgreSQL Global Development Group
+
+use strict;
+use warnings;
+
+use MIME::Base64;
+
+my ($tmpdir, $cmkname, $alg, $b64data) = @ARGV;
+
+die unless $alg eq 'RSAES_OAEP_SHA_1';
+
+open my $fh, '>:raw', "${tmpdir}/input.tmp" or die;
+print $fh decode_base64($b64data);
+close $fh;
+
+system('openssl', 'pkeyutl', '-decrypt',
+ '-inkey', "${tmpdir}/${cmkname}.pem", '-pkeyopt', 'rsa_padding_mode:oaep',
+ '-in', "${tmpdir}/input.tmp", '-out', "${tmpdir}/output.tmp") == 0 or die;
+
+open $fh, '<:raw', "${tmpdir}/output.tmp" or die;
+my $data = '';
+
+while (1) {
+ my $success = read $fh, $data, 100, length($data);
+ die $! if not defined $success;
+ last if not $success;
+}
+
+close $fh;
+
+unlink "${tmpdir}/input.tmp", "${tmpdir}/output.tmp";
+
+print encode_base64($data);
diff --git a/src/test/modules/libpq_pipeline/traces/disallowed_in_pipeline.trace b/src/test/modules/libpq_pipeline/traces/disallowed_in_pipeline.trace
index dd6df03f1e..5fc9c0346d 100644
--- a/src/test/modules/libpq_pipeline/traces/disallowed_in_pipeline.trace
+++ b/src/test/modules/libpq_pipeline/traces/disallowed_in_pipeline.trace
@@ -1,5 +1,5 @@
F 13 Query "SELECT 1"
-B 33 RowDescription 1 "?column?" NNNN 0 NNNN 4 -1 0
+B 39 RowDescription 1 "?column?" NNNN 0 NNNN 4 -1 0 NNNN 0
B 11 DataRow 1 1 '1'
B 13 CommandComplete "SELECT 1"
B 5 ReadyForQuery I
diff --git a/src/test/modules/libpq_pipeline/traces/multi_pipelines.trace b/src/test/modules/libpq_pipeline/traces/multi_pipelines.trace
index 4b9ab07ca4..700b0b6519 100644
--- a/src/test/modules/libpq_pipeline/traces/multi_pipelines.trace
+++ b/src/test/modules/libpq_pipeline/traces/multi_pipelines.trace
@@ -10,13 +10,13 @@ F 9 Execute "" 0
F 4 Sync
B 4 ParseComplete
B 4 BindComplete
-B 33 RowDescription 1 "?column?" NNNN 0 NNNN 4 -1 0
+B 39 RowDescription 1 "?column?" NNNN 0 NNNN 4 -1 0 NNNN 0
B 11 DataRow 1 1 '1'
B 13 CommandComplete "SELECT 1"
B 5 ReadyForQuery I
B 4 ParseComplete
B 4 BindComplete
-B 33 RowDescription 1 "?column?" NNNN 0 NNNN 4 -1 0
+B 39 RowDescription 1 "?column?" NNNN 0 NNNN 4 -1 0 NNNN 0
B 11 DataRow 1 1 '1'
B 13 CommandComplete "SELECT 1"
B 5 ReadyForQuery I
diff --git a/src/test/modules/libpq_pipeline/traces/nosync.trace b/src/test/modules/libpq_pipeline/traces/nosync.trace
index d99aac649d..7b1dfa1c15 100644
--- a/src/test/modules/libpq_pipeline/traces/nosync.trace
+++ b/src/test/modules/libpq_pipeline/traces/nosync.trace
@@ -41,52 +41,52 @@ F 9 Execute "" 0
F 4 Flush
B 4 ParseComplete
B 4 BindComplete
-B 31 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0
+B 37 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0 NNNN 0
B 70 DataRow 1 60 'xyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxz'
B 13 CommandComplete "SELECT 1"
B 4 ParseComplete
B 4 BindComplete
-B 31 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0
+B 37 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0 NNNN 0
B 70 DataRow 1 60 'xyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxz'
B 13 CommandComplete "SELECT 1"
B 4 ParseComplete
B 4 BindComplete
-B 31 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0
+B 37 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0 NNNN 0
B 70 DataRow 1 60 'xyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxz'
B 13 CommandComplete "SELECT 1"
B 4 ParseComplete
B 4 BindComplete
-B 31 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0
+B 37 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0 NNNN 0
B 70 DataRow 1 60 'xyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxz'
B 13 CommandComplete "SELECT 1"
B 4 ParseComplete
B 4 BindComplete
-B 31 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0
+B 37 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0 NNNN 0
B 70 DataRow 1 60 'xyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxz'
B 13 CommandComplete "SELECT 1"
B 4 ParseComplete
B 4 BindComplete
-B 31 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0
+B 37 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0 NNNN 0
B 70 DataRow 1 60 'xyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxz'
B 13 CommandComplete "SELECT 1"
B 4 ParseComplete
B 4 BindComplete
-B 31 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0
+B 37 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0 NNNN 0
B 70 DataRow 1 60 'xyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxz'
B 13 CommandComplete "SELECT 1"
B 4 ParseComplete
B 4 BindComplete
-B 31 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0
+B 37 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0 NNNN 0
B 70 DataRow 1 60 'xyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxz'
B 13 CommandComplete "SELECT 1"
B 4 ParseComplete
B 4 BindComplete
-B 31 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0
+B 37 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0 NNNN 0
B 70 DataRow 1 60 'xyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxz'
B 13 CommandComplete "SELECT 1"
B 4 ParseComplete
B 4 BindComplete
-B 31 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0
+B 37 RowDescription 1 "repeat" NNNN 0 NNNN 65535 -1 0 NNNN 0
B 70 DataRow 1 60 'xyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxzxyzxz'
B 13 CommandComplete "SELECT 1"
F 4 Terminate
diff --git a/src/test/modules/libpq_pipeline/traces/pipeline_abort.trace b/src/test/modules/libpq_pipeline/traces/pipeline_abort.trace
index 3fce548b99..6e4309cadd 100644
--- a/src/test/modules/libpq_pipeline/traces/pipeline_abort.trace
+++ b/src/test/modules/libpq_pipeline/traces/pipeline_abort.trace
@@ -50,14 +50,14 @@ F 6 Close P ""
F 4 Sync
B 4 ParseComplete
B 4 BindComplete
-B 33 RowDescription 1 "?column?" NNNN 0 NNNN 65535 -1 0
+B 39 RowDescription 1 "?column?" NNNN 0 NNNN 65535 -1 0 NNNN 0
B 32 DataRow 1 22 '0.33333333333333333333'
B 32 DataRow 1 22 '0.50000000000000000000'
B 32 DataRow 1 22 '1.00000000000000000000'
B NN ErrorResponse S "ERROR" V "ERROR" C "22012" M "division by zero" F "SSSS" L "SSSS" R "SSSS" \x00
B 5 ReadyForQuery I
F 40 Query "SELECT itemno FROM pq_pipeline_demo"
-B 31 RowDescription 1 "itemno" NNNN 2 NNNN 4 -1 0
+B 37 RowDescription 1 "itemno" NNNN 2 NNNN 4 -1 0 NNNN 0
B 11 DataRow 1 1 '3'
B 13 CommandComplete "SELECT 1"
B 5 ReadyForQuery I
diff --git a/src/test/modules/libpq_pipeline/traces/prepared.trace b/src/test/modules/libpq_pipeline/traces/prepared.trace
index 1a7de5c3e6..8d45bbc0ce 100644
--- a/src/test/modules/libpq_pipeline/traces/prepared.trace
+++ b/src/test/modules/libpq_pipeline/traces/prepared.trace
@@ -2,8 +2,8 @@ F 68 Parse "select_one" "SELECT $1, '42', $1::numeric, interval '1 sec'" 1 NNNN
F 16 Describe S "select_one"
F 4 Sync
B 4 ParseComplete
-B 10 ParameterDescription 1 NNNN
-B 113 RowDescription 4 "?column?" NNNN 0 NNNN 4 -1 0 "?column?" NNNN 0 NNNN 65535 -1 0 "numeric" NNNN 0 NNNN 65535 -1 0 "interval" NNNN 0 NNNN 16 -1 0
+B 18 ParameterDescription 1 NNNN NNNN 0 0
+B 137 RowDescription 4 "?column?" NNNN 0 NNNN 4 -1 0 NNNN 0 "?column?" NNNN 0 NNNN 65535 -1 0 NNNN 0 "numeric" NNNN 0 NNNN 65535 -1 0 NNNN 0 "interval" NNNN 0 NNNN 16 -1 0 NNNN 0
B 5 ReadyForQuery I
F 10 Query "BEGIN"
B 10 CommandComplete "BEGIN"
@@ -13,6 +13,6 @@ B 19 CommandComplete "DECLARE CURSOR"
B 5 ReadyForQuery T
F 16 Describe P "cursor_one"
F 4 Sync
-B 33 RowDescription 1 "?column?" NNNN 0 NNNN 4 -1 0
+B 39 RowDescription 1 "?column?" NNNN 0 NNNN 4 -1 0 NNNN 0
B 5 ReadyForQuery T
F 4 Terminate
diff --git a/src/test/modules/libpq_pipeline/traces/simple_pipeline.trace b/src/test/modules/libpq_pipeline/traces/simple_pipeline.trace
index 5c94749bc1..5f4849425d 100644
--- a/src/test/modules/libpq_pipeline/traces/simple_pipeline.trace
+++ b/src/test/modules/libpq_pipeline/traces/simple_pipeline.trace
@@ -5,7 +5,7 @@ F 9 Execute "" 0
F 4 Sync
B 4 ParseComplete
B 4 BindComplete
-B 33 RowDescription 1 "?column?" NNNN 0 NNNN 4 -1 0
+B 39 RowDescription 1 "?column?" NNNN 0 NNNN 4 -1 0 NNNN 0
B 11 DataRow 1 1 '1'
B 13 CommandComplete "SELECT 1"
B 5 ReadyForQuery I
diff --git a/src/test/modules/libpq_pipeline/traces/singlerow.trace b/src/test/modules/libpq_pipeline/traces/singlerow.trace
index 9de99befcc..199df4d4f4 100644
--- a/src/test/modules/libpq_pipeline/traces/singlerow.trace
+++ b/src/test/modules/libpq_pipeline/traces/singlerow.trace
@@ -13,14 +13,14 @@ F 9 Execute "" 0
F 4 Sync
B 4 ParseComplete
B 4 BindComplete
-B 40 RowDescription 1 "generate_series" NNNN 0 NNNN 4 -1 0
+B 46 RowDescription 1 "generate_series" NNNN 0 NNNN 4 -1 0 NNNN 0
B 12 DataRow 1 2 '42'
B 12 DataRow 1 2 '43'
B 12 DataRow 1 2 '44'
B 13 CommandComplete "SELECT 3"
B 4 ParseComplete
B 4 BindComplete
-B 40 RowDescription 1 "generate_series" NNNN 0 NNNN 4 -1 0
+B 46 RowDescription 1 "generate_series" NNNN 0 NNNN 4 -1 0 NNNN 0
B 12 DataRow 1 2 '42'
B 12 DataRow 1 2 '43'
B 12 DataRow 1 2 '44'
@@ -28,7 +28,7 @@ B 12 DataRow 1 2 '45'
B 13 CommandComplete "SELECT 4"
B 4 ParseComplete
B 4 BindComplete
-B 40 RowDescription 1 "generate_series" NNNN 0 NNNN 4 -1 0
+B 46 RowDescription 1 "generate_series" NNNN 0 NNNN 4 -1 0 NNNN 0
B 12 DataRow 1 2 '42'
B 12 DataRow 1 2 '43'
B 12 DataRow 1 2 '44'
diff --git a/src/test/modules/libpq_pipeline/traces/transaction.trace b/src/test/modules/libpq_pipeline/traces/transaction.trace
index 1dcc2373c0..a6869c4a5b 100644
--- a/src/test/modules/libpq_pipeline/traces/transaction.trace
+++ b/src/test/modules/libpq_pipeline/traces/transaction.trace
@@ -54,7 +54,7 @@ B 15 CommandComplete "INSERT 0 1"
B 5 ReadyForQuery I
B 5 ReadyForQuery I
F 34 Query "SELECT * FROM pq_pipeline_tst"
-B 27 RowDescription 1 "id" NNNN 1 NNNN 4 -1 0
+B 33 RowDescription 1 "id" NNNN 1 NNNN 4 -1 0 NNNN 0
B 11 DataRow 1 1 '3'
B 13 CommandComplete "SELECT 1"
B 5 ReadyForQuery I
diff --git a/src/test/regress/expected/column_encryption.out b/src/test/regress/expected/column_encryption.out
new file mode 100644
index 0000000000..f8db4e17ff
--- /dev/null
+++ b/src/test/regress/expected/column_encryption.out
@@ -0,0 +1,59 @@
+\set HIDE_COLUMN_ENCRYPTION false
+/* Imagine:
+CREATE COLUMN MASTER KEY cmk1 (
+ realm = 'test'
+);
+*/
+INSERT INTO pg_colmasterkey (oid, cmkname, cmkowner, cmkrealm) VALUES (
+ pg_nextoid('pg_catalog.pg_colmasterkey', 'oid', 'pg_catalog.pg_colmasterkey_oid_index'),
+ 'cmk1',
+ (select oid from pg_roles where rolname = current_user),
+ 'test'
+);
+/* Imagine:
+CREATE COLUMN ENCRYPTION KEY cek1 (
+ column_master_key = cmk1,
+ algorithm = '...',
+ encrypted_value = '...'
+);
+*/
+INSERT INTO pg_colenckey (oid, cekname, cekowner) VALUES (
+ pg_nextoid('pg_catalog.pg_colenckey', 'oid', 'pg_catalog.pg_colenckey_oid_index'),
+ 'cek1',
+ (select oid from pg_roles where rolname = current_user)
+);
+INSERT INTO pg_colenckeydata (oid, ckdcekid, ckdcmkid, ckdcmkalg, ckdencval) VALUES (
+ pg_nextoid('pg_catalog.pg_colenckeydata', 'oid', 'pg_catalog.pg_colenckeydata_oid_index'),
+ (SELECT oid FROM pg_colenckey WHERE cekname = 'cek1'),
+ (SELECT oid FROM pg_colmasterkey WHERE cmkname = 'cmk1'),
+ 1,
+ '\xDEADBEEF'
+);
+CREATE TABLE tbl_fail (
+ a int,
+ b text,
+ c text ENCRYPTED WITH (column_encryption_key = notexist)
+);
+ERROR: column encryption key "notexist" does not exist
+CREATE TABLE tbl_29f3 (
+ a int,
+ b text,
+ c text ENCRYPTED WITH (column_encryption_key = cek1)
+);
+\d tbl_29f3
+ Table "public.tbl_29f3"
+ Column | Type | Collation | Nullable | Default
+--------+---------+-----------+----------+---------
+ a | integer | | |
+ b | text | | |
+ c | text | | |
+
+\d+ tbl_29f3
+ Table "public.tbl_29f3"
+ Column | Type | Collation | Nullable | Default | Storage | Encryption | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | | |
+ c | text | | | | extended | cek1 | |
+
+DROP TABLE tbl_29f3; -- FIXME: needs pg_dump support for pg_upgrade tests
diff --git a/src/test/regress/expected/oidjoins.out b/src/test/regress/expected/oidjoins.out
index 215eb899be..2aa0e16323 100644
--- a/src/test/regress/expected/oidjoins.out
+++ b/src/test/regress/expected/oidjoins.out
@@ -73,6 +73,8 @@ NOTICE: checking pg_type {typbasetype} => pg_type {oid}
NOTICE: checking pg_type {typcollation} => pg_collation {oid}
NOTICE: checking pg_attribute {attrelid} => pg_class {oid}
NOTICE: checking pg_attribute {atttypid} => pg_type {oid}
+NOTICE: checking pg_attribute {attcek} => pg_colenckey {oid}
+NOTICE: checking pg_attribute {attrealtypid} => pg_type {oid}
NOTICE: checking pg_attribute {attcollation} => pg_collation {oid}
NOTICE: checking pg_class {relnamespace} => pg_namespace {oid}
NOTICE: checking pg_class {reltype} => pg_type {oid}
@@ -266,3 +268,7 @@ NOTICE: checking pg_subscription {subdbid} => pg_database {oid}
NOTICE: checking pg_subscription {subowner} => pg_authid {oid}
NOTICE: checking pg_subscription_rel {srsubid} => pg_subscription {oid}
NOTICE: checking pg_subscription_rel {srrelid} => pg_class {oid}
+NOTICE: checking pg_colmasterkey {cmkowner} => pg_authid {oid}
+NOTICE: checking pg_colenckey {cekowner} => pg_authid {oid}
+NOTICE: checking pg_colenckeydata {ckdcekid} => pg_colenckey {oid}
+NOTICE: checking pg_colenckeydata {ckdcmkid} => pg_colmasterkey {oid}
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 86d755aa44..4a366074da 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -175,13 +175,14 @@ WHERE p1.oid != p2.oid AND
ORDER BY 1, 2;
proargtypes | proargtypes
-----------------------------+--------------------------
+ bytea | pg_encrypted_det
bigint | xid8
text | character
text | character varying
timestamp without time zone | timestamp with time zone
bit | bit varying
txid_snapshot | pg_snapshot
-(6 rows)
+(7 rows)
SELECT DISTINCT p1.proargtypes[1]::regtype, p2.proargtypes[1]::regtype
FROM pg_proc AS p1, pg_proc AS p2
@@ -197,12 +198,13 @@ WHERE p1.oid != p2.oid AND
ORDER BY 1, 2;
proargtypes | proargtypes
-----------------------------+--------------------------
+ bytea | pg_encrypted_det
integer | xid
timestamp without time zone | timestamp with time zone
bit | bit varying
txid_snapshot | pg_snapshot
anyrange | anymultirange
-(5 rows)
+(6 rows)
SELECT DISTINCT p1.proargtypes[2]::regtype, p2.proargtypes[2]::regtype
FROM pg_proc AS p1, pg_proc AS p2
@@ -841,6 +843,8 @@ xid8ge(xid8,xid8)
xid8eq(xid8,xid8)
xid8ne(xid8,xid8)
xid8cmp(xid8,xid8)
+pg_encrypted_det_eq(pg_encrypted_det,pg_encrypted_det)
+pg_encrypted_det_ne(pg_encrypted_det,pg_encrypted_det)
-- restore normal output mode
\a\t
-- List of functions used by libpq's fe-lobj.c
@@ -988,7 +992,9 @@ WHERE c.castmethod = 'b' AND
xml | text | 0 | a
xml | character varying | 0 | a
xml | character | 0 | a
-(10 rows)
+ bytea | pg_encrypted_det | 0 | e
+ bytea | pg_encrypted_rnd | 0 | e
+(12 rows)
-- **************** pg_conversion ****************
-- Look for illegal values in pg_conversion fields.
diff --git a/src/test/regress/expected/prepare.out b/src/test/regress/expected/prepare.out
index 5815e17b39..4482a65d24 100644
--- a/src/test/regress/expected/prepare.out
+++ b/src/test/regress/expected/prepare.out
@@ -162,26 +162,26 @@ PREPARE q7(unknown) AS
-- DML statements
PREPARE q8 AS
UPDATE tenk1 SET stringu1 = $2 WHERE unique1 = $1;
-SELECT name, statement, parameter_types, result_types FROM pg_prepared_statements
+SELECT name, statement, parameter_types, parameter_orig_tables, parameter_orig_columns, result_types FROM pg_prepared_statements
ORDER BY name;
- name | statement | parameter_types | result_types
-------+------------------------------------------------------------------+----------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------
- q2 | PREPARE q2(text) AS +| {text} | {name,boolean,boolean}
- | SELECT datname, datistemplate, datallowconn +| |
- | FROM pg_database WHERE datname = $1; | |
- q3 | PREPARE q3(text, int, float, boolean, smallint) AS +| {text,integer,"double precision",boolean,smallint} | {integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,name,name,name}
- | SELECT * FROM tenk1 WHERE string4 = $1 AND (four = $2 OR+| |
- | ten = $3::bigint OR true = $4 OR odd = $5::int) +| |
- | ORDER BY unique1; | |
- q5 | PREPARE q5(int, text) AS +| {integer,text} | {integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,name,name,name}
- | SELECT * FROM tenk1 WHERE unique1 = $1 OR stringu1 = $2 +| |
- | ORDER BY unique1; | |
- q6 | PREPARE q6 AS +| {integer,name} | {integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,name,name,name}
- | SELECT * FROM tenk1 WHERE unique1 = $1 AND stringu1 = $2; | |
- q7 | PREPARE q7(unknown) AS +| {path} | {text,path}
- | SELECT * FROM road WHERE thepath = $1; | |
- q8 | PREPARE q8 AS +| {integer,name} |
- | UPDATE tenk1 SET stringu1 = $2 WHERE unique1 = $1; | |
+ name | statement | parameter_types | parameter_orig_tables | parameter_orig_columns | result_types
+------+------------------------------------------------------------------+----------------------------------------------------+-----------------------+------------------------+--------------------------------------------------------------------------------------------------------------------------
+ q2 | PREPARE q2(text) AS +| {text} | {-} | {0} | {name,boolean,boolean}
+ | SELECT datname, datistemplate, datallowconn +| | | |
+ | FROM pg_database WHERE datname = $1; | | | |
+ q3 | PREPARE q3(text, int, float, boolean, smallint) AS +| {text,integer,"double precision",boolean,smallint} | {-,-,-,-,-} | {0,0,0,0,0} | {integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,name,name,name}
+ | SELECT * FROM tenk1 WHERE string4 = $1 AND (four = $2 OR+| | | |
+ | ten = $3::bigint OR true = $4 OR odd = $5::int) +| | | |
+ | ORDER BY unique1; | | | |
+ q5 | PREPARE q5(int, text) AS +| {integer,text} | {-,-} | {0,0} | {integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,name,name,name}
+ | SELECT * FROM tenk1 WHERE unique1 = $1 OR stringu1 = $2 +| | | |
+ | ORDER BY unique1; | | | |
+ q6 | PREPARE q6 AS +| {integer,name} | {-,-} | {0,0} | {integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,integer,name,name,name}
+ | SELECT * FROM tenk1 WHERE unique1 = $1 AND stringu1 = $2; | | | |
+ q7 | PREPARE q7(unknown) AS +| {path} | {-} | {0} | {text,path}
+ | SELECT * FROM road WHERE thepath = $1; | | | |
+ q8 | PREPARE q8 AS +| {integer,name} | {-,tenk1} | {0,14} |
+ | UPDATE tenk1 SET stringu1 = $2 WHERE unique1 = $1; | | | |
(6 rows)
-- test DEALLOCATE ALL;
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 60acbd1241..33956f6993 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -237,6 +237,31 @@ SELECT 3 AS x, 'Hello', 4 AS y, true AS "dirty\name" \gdesc \g
3 | Hello | 4 | t
(1 row)
+-- \gencr
+-- (This just tests the parameter passing; there is no encryption here.)
+CREATE TABLE test_gencr (a int, b text);
+INSERT INTO test_gencr VALUES (1, 'one') \gencr
+SELECT * FROM test_gencr WHERE a = 1 \gencr
+ a | b
+---+-----
+ 1 | one
+(1 row)
+
+INSERT INTO test_gencr VALUES ($1, $2) \gencr 2 'two'
+SELECT * FROM test_gencr WHERE a IN ($1, $2) \gencr 2 3
+ a | b
+---+-----
+ 2 | two
+(1 row)
+
+-- test parse error
+SELECT * FROM test_gencr WHERE a = x \gencr
+ERROR: column "x" does not exist
+LINE 1: SELECT * FROM test_gencr WHERE a = x
+ ^
+-- test bind error
+SELECT * FROM test_gencr WHERE a = $1 \gencr
+ERROR: bind message supplies 0 parameters, but prepared statement "" requires 1
-- \gexec
create temporary table gexec_test(a int, b text, c date, d float);
select format('create index on gexec_test(%I)', attname)
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 7ec3d2688f..42e38d66d1 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1423,11 +1423,13 @@ pg_prepared_statements| SELECT p.name,
p.statement,
p.prepare_time,
p.parameter_types,
+ p.parameter_orig_tables,
+ p.parameter_orig_columns,
p.result_types,
p.from_sql,
p.generic_plans,
p.custom_plans
- FROM pg_prepared_statement() p(name, statement, prepare_time, parameter_types, result_types, from_sql, generic_plans, custom_plans);
+ FROM pg_prepared_statement() p(name, statement, prepare_time, parameter_types, parameter_orig_tables, parameter_orig_columns, result_types, from_sql, generic_plans, custom_plans);
pg_prepared_xacts| SELECT p.transaction,
p.gid,
p.prepared,
diff --git a/src/test/regress/expected/type_sanity.out b/src/test/regress/expected/type_sanity.out
index d3ac08c9ee..fc0c5896e4 100644
--- a/src/test/regress/expected/type_sanity.out
+++ b/src/test/regress/expected/type_sanity.out
@@ -17,7 +17,7 @@ SELECT t1.oid, t1.typname
FROM pg_type as t1
WHERE t1.typnamespace = 0 OR
(t1.typlen <= 0 AND t1.typlen != -1 AND t1.typlen != -2) OR
- (t1.typtype not in ('b', 'c', 'd', 'e', 'm', 'p', 'r')) OR
+ (t1.typtype not in ('b', 'c', 'd', 'e', 'm', 'p', 'r', 'y')) OR
NOT t1.typisdefined OR
(t1.typalign not in ('c', 's', 'i', 'd')) OR
(t1.typstorage not in ('p', 'x', 'e', 'm'));
@@ -75,7 +75,9 @@ ORDER BY t1.oid;
4600 | pg_brin_bloom_summary
4601 | pg_brin_minmax_multi_summary
5017 | pg_mcv_list
-(6 rows)
+ 8243 | pg_encrypted_det
+ 8244 | pg_encrypted_rnd
+(8 rows)
-- Make sure typarray points to a "true" array type of our own base
SELECT t1.oid, t1.typname as basetype, t2.typname as arraytype,
@@ -210,7 +212,8 @@ ORDER BY 1;
e | enum_in
m | multirange_in
r | range_in
-(5 rows)
+ y | byteain
+(6 rows)
-- Check for bogus typoutput routines
-- As of 8.0, this check finds refcursor, which is borrowing
@@ -255,7 +258,8 @@ ORDER BY 1;
e | enum_out
m | multirange_out
r | range_out
-(4 rows)
+ y | byteaout
+(5 rows)
-- Domains should have same typoutput as their base types
SELECT t1.oid, t1.typname, t2.oid, t2.typname
@@ -335,7 +339,8 @@ ORDER BY 1;
e | enum_recv
m | multirange_recv
r | range_recv
-(5 rows)
+ y | bytearecv
+(6 rows)
-- Check for bogus typsend routines
-- As of 7.4, this check finds refcursor, which is borrowing
@@ -380,7 +385,8 @@ ORDER BY 1;
e | enum_send
m | multirange_send
r | range_send
-(4 rows)
+ y | byteasend
+(5 rows)
-- Domains should have same typsend as their base types
SELECT t1.oid, t1.typname, t2.oid, t2.typname
@@ -707,6 +713,8 @@ CREATE TABLE tab_core_types AS SELECT
'txt'::text,
true::bool,
E'\\xDEADBEEF'::bytea,
+ E'\\xDEADBEEF'::pg_encrypted_rnd,
+ E'\\xDEADBEEF'::pg_encrypted_det,
B'10001'::bit,
B'10001'::varbit AS varbit,
'12.34'::money,
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 103e11483d..ffca206c6f 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -129,6 +129,9 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression memoize stats
+# WIP
+test: column_encryption
+
# event_trigger cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
test: event_trigger oidjoins
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index a4b354c9e6..8ad1f458d5 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -82,7 +82,7 @@ psql_start_test(const char *testname,
bindir ? bindir : "",
bindir ? "/" : "",
dblist->str,
- "-v HIDE_TABLEAM=on -v HIDE_TOAST_COMPRESSION=on",
+ "-v HIDE_TABLEAM=on -v HIDE_TOAST_COMPRESSION=on -v HIDE_COLUMN_ENCRYPTION=on",
infile,
outfile);
if (offset >= sizeof(psql_cmd))
diff --git a/src/test/regress/sql/column_encryption.sql b/src/test/regress/sql/column_encryption.sql
new file mode 100644
index 0000000000..37f0b23d10
--- /dev/null
+++ b/src/test/regress/sql/column_encryption.sql
@@ -0,0 +1,50 @@
+\set HIDE_COLUMN_ENCRYPTION false
+
+/* Imagine:
+CREATE COLUMN MASTER KEY cmk1 (
+ realm = 'test'
+);
+*/
+INSERT INTO pg_colmasterkey (oid, cmkname, cmkowner, cmkrealm) VALUES (
+ pg_nextoid('pg_catalog.pg_colmasterkey', 'oid', 'pg_catalog.pg_colmasterkey_oid_index'),
+ 'cmk1',
+ (select oid from pg_roles where rolname = current_user),
+ 'test'
+);
+
+/* Imagine:
+CREATE COLUMN ENCRYPTION KEY cek1 (
+ column_master_key = cmk1,
+ algorithm = '...',
+ encrypted_value = '...'
+);
+*/
+INSERT INTO pg_colenckey (oid, cekname, cekowner) VALUES (
+ pg_nextoid('pg_catalog.pg_colenckey', 'oid', 'pg_catalog.pg_colenckey_oid_index'),
+ 'cek1',
+ (select oid from pg_roles where rolname = current_user)
+);
+INSERT INTO pg_colenckeydata (oid, ckdcekid, ckdcmkid, ckdcmkalg, ckdencval) VALUES (
+ pg_nextoid('pg_catalog.pg_colenckeydata', 'oid', 'pg_catalog.pg_colenckeydata_oid_index'),
+ (SELECT oid FROM pg_colenckey WHERE cekname = 'cek1'),
+ (SELECT oid FROM pg_colmasterkey WHERE cmkname = 'cmk1'),
+ 1,
+ '\xDEADBEEF'
+);
+
+CREATE TABLE tbl_fail (
+ a int,
+ b text,
+ c text ENCRYPTED WITH (column_encryption_key = notexist)
+);
+
+CREATE TABLE tbl_29f3 (
+ a int,
+ b text,
+ c text ENCRYPTED WITH (column_encryption_key = cek1)
+);
+
+\d tbl_29f3
+\d+ tbl_29f3
+
+DROP TABLE tbl_29f3; -- FIXME: needs pg_dump support for pg_upgrade tests
diff --git a/src/test/regress/sql/prepare.sql b/src/test/regress/sql/prepare.sql
index c6098dc95c..7db788735c 100644
--- a/src/test/regress/sql/prepare.sql
+++ b/src/test/regress/sql/prepare.sql
@@ -75,7 +75,7 @@ CREATE TEMPORARY TABLE q5_prep_nodata AS EXECUTE q5(200, 'DTAAAA')
PREPARE q8 AS
UPDATE tenk1 SET stringu1 = $2 WHERE unique1 = $1;
-SELECT name, statement, parameter_types, result_types FROM pg_prepared_statements
+SELECT name, statement, parameter_types, parameter_orig_tables, parameter_orig_columns, result_types FROM pg_prepared_statements
ORDER BY name;
-- test DEALLOCATE ALL;
diff --git a/src/test/regress/sql/psql.sql b/src/test/regress/sql/psql.sql
index 1149c6a839..e88c70d302 100644
--- a/src/test/regress/sql/psql.sql
+++ b/src/test/regress/sql/psql.sql
@@ -119,6 +119,23 @@ CREATE TABLE bububu(a int) \gdesc
-- all on one line
SELECT 3 AS x, 'Hello', 4 AS y, true AS "dirty\name" \gdesc \g
+
+-- \gencr
+-- (This just tests the parameter passing; there is no encryption here.)
+
+CREATE TABLE test_gencr (a int, b text);
+INSERT INTO test_gencr VALUES (1, 'one') \gencr
+SELECT * FROM test_gencr WHERE a = 1 \gencr
+
+INSERT INTO test_gencr VALUES ($1, $2) \gencr 2 'two'
+SELECT * FROM test_gencr WHERE a IN ($1, $2) \gencr 2 3
+
+-- test parse error
+SELECT * FROM test_gencr WHERE a = x \gencr
+-- test bind error
+SELECT * FROM test_gencr WHERE a = $1 \gencr
+
+
-- \gexec
create temporary table gexec_test(a int, b text, c date, d float);
diff --git a/src/test/regress/sql/type_sanity.sql b/src/test/regress/sql/type_sanity.sql
index 5edc1f1f6e..34dd19456d 100644
--- a/src/test/regress/sql/type_sanity.sql
+++ b/src/test/regress/sql/type_sanity.sql
@@ -20,7 +20,7 @@
FROM pg_type as t1
WHERE t1.typnamespace = 0 OR
(t1.typlen <= 0 AND t1.typlen != -1 AND t1.typlen != -2) OR
- (t1.typtype not in ('b', 'c', 'd', 'e', 'm', 'p', 'r')) OR
+ (t1.typtype not in ('b', 'c', 'd', 'e', 'm', 'p', 'r', 'y')) OR
NOT t1.typisdefined OR
(t1.typalign not in ('c', 's', 'i', 'd')) OR
(t1.typstorage not in ('p', 'x', 'e', 'm'));
@@ -529,6 +529,8 @@ CREATE TABLE tab_core_types AS SELECT
'txt'::text,
true::bool,
E'\\xDEADBEEF'::bytea,
+ E'\\xDEADBEEF'::pg_encrypted_rnd,
+ E'\\xDEADBEEF'::pg_encrypted_det,
B'10001'::bit,
B'10001'::varbit AS varbit,
'12.34'::money,
diff --git a/src/include/access/printtup.h b/src/include/access/printtup.h
index 971a74cf22..db1f3b8811 100644
--- a/src/include/access/printtup.h
+++ b/src/include/access/printtup.h
@@ -20,6 +20,8 @@ extern DestReceiver *printtup_create_DR(CommandDest dest);
extern void SetRemoteDestReceiverParams(DestReceiver *self, Portal portal);
+extern void MaybeSendColumnEncryptionKeyMessage(Oid attcek);
+
extern void SendRowDescriptionMessage(StringInfo buf,
TupleDesc typeinfo, List *targetlist, int16 *formats);
diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat
index 61cd591430..a0bd708132 100644
--- a/src/include/catalog/pg_amop.dat
+++ b/src/include/catalog/pg_amop.dat
@@ -1028,6 +1028,11 @@
amoprighttype => 'bytea', amopstrategy => '1', amopopr => '=(bytea,bytea)',
amopmethod => 'hash' },
+# pg_encrypted_det_ops
+{ amopfamily => 'hash/pg_encrypted_det_ops', amoplefttype => 'pg_encrypted_det',
+ amoprighttype => 'pg_encrypted_det', amopstrategy => '1', amopopr => '=(pg_encrypted_det,pg_encrypted_det)',
+ amopmethod => 'hash' },
+
# xid_ops
{ amopfamily => 'hash/xid_ops', amoplefttype => 'xid', amoprighttype => 'xid',
amopstrategy => '1', amopopr => '=(xid,xid)', amopmethod => 'hash' },
diff --git a/src/include/catalog/pg_amproc.dat b/src/include/catalog/pg_amproc.dat
index 4cc129bebd..dddb27113f 100644
--- a/src/include/catalog/pg_amproc.dat
+++ b/src/include/catalog/pg_amproc.dat
@@ -402,6 +402,11 @@
{ amprocfamily => 'hash/bytea_ops', amproclefttype => 'bytea',
amprocrighttype => 'bytea', amprocnum => '2',
amproc => 'hashvarlenaextended' },
+{ amprocfamily => 'hash/pg_encrypted_det_ops', amproclefttype => 'pg_encrypted_det',
+ amprocrighttype => 'pg_encrypted_det', amprocnum => '1', amproc => 'hashvarlena' },
+{ amprocfamily => 'hash/pg_encrypted_det_ops', amproclefttype => 'pg_encrypted_det',
+ amprocrighttype => 'pg_encrypted_det', amprocnum => '2',
+ amproc => 'hashvarlenaextended' },
{ amprocfamily => 'hash/xid_ops', amproclefttype => 'xid',
amprocrighttype => 'xid', amprocnum => '1', amproc => 'hashint4' },
{ amprocfamily => 'hash/xid_ops', amproclefttype => 'xid',
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 053294c99f..550a53649b 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -164,6 +164,15 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
*/
bool attislocal BKI_DEFAULT(t);
+ /* column encryption key */
+ Oid attcek BKI_DEFAULT(0) BKI_LOOKUP_OPT(pg_colenckey);
+
+ /* real type if encrypted */
+ Oid attrealtypid BKI_DEFAULT(0) BKI_LOOKUP_OPT(pg_type);
+
+ /* encryption algorithm (PG_CEK_* values) */
+ int16 attencalg BKI_DEFAULT(0);
+
/* Number of times inherited from direct parent relation(s) */
int32 attinhcount BKI_DEFAULT(0);
diff --git a/src/include/catalog/pg_cast.dat b/src/include/catalog/pg_cast.dat
index 4471eb6bbe..878b0bcbbf 100644
--- a/src/include/catalog/pg_cast.dat
+++ b/src/include/catalog/pg_cast.dat
@@ -546,4 +546,10 @@
{ castsource => 'tstzrange', casttarget => 'tstzmultirange',
castfunc => 'tstzmultirange(tstzrange)', castcontext => 'e',
castmethod => 'f' },
+
+{ castsource => 'bytea', casttarget => 'pg_encrypted_det', castfunc => '0',
+ castcontext => 'e', castmethod => 'b' },
+{ castsource => 'bytea', casttarget => 'pg_encrypted_rnd', castfunc => '0',
+ castcontext => 'e', castmethod => 'b' },
+
]
diff --git a/src/include/catalog/pg_colenckey.h b/src/include/catalog/pg_colenckey.h
new file mode 100644
index 0000000000..095ed712b0
--- /dev/null
+++ b/src/include/catalog/pg_colenckey.h
@@ -0,0 +1,55 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_colenckey.h
+ * definition of the "column encryption key" system catalog
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_colenkey.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COLENCKEY_H
+#define PG_COLENCKEY_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_colenckey_d.h"
+
+/* ----------------
+ * pg_colenckey definition. cpp turns this into
+ * typedef struct FormData_pg_colenckey
+ * ----------------
+ */
+CATALOG(pg_colenckey,8234,ColumnEncKeyRelationId)
+{
+ Oid oid;
+ NameData cekname;
+ Oid cekowner BKI_LOOKUP(pg_authid);
+} FormData_pg_colenckey;
+
+typedef FormData_pg_colenckey *Form_pg_colenckey;
+
+DECLARE_UNIQUE_INDEX_PKEY(pg_colenckey_oid_index, 8240, ColumnEncKeyOidIndexId, on pg_colenckey using btree(oid oid_ops));
+DECLARE_UNIQUE_INDEX(pg_colenckey_cekname_index, 8242, ColumnEncKeyNameIndexId, on pg_colenckey using btree(cekname name_ops));
+
+/*
+ * Constants for CMK and CEK algorithms. Note that these are part of the
+ * protocol. For clarity, the assigned numbers are not reused between CMKs
+ * and CEKs, but that is not technically required. In either case, don't
+ * assign zero, so that that can be used as an invalid value.
+ */
+
+#define PG_CMK_RSAES_OAEP_SHA_1 1
+#define PG_CMK_RSAES_OAEP_SHA_256 2
+
+#define PG_CEK_AEAD_AES_128_CBC_HMAC_SHA_256 130
+#define PG_CEK_AEAD_AES_192_CBC_HMAC_SHA_384 131
+#define PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_384 132
+#define PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_512 133
+
+#endif
diff --git a/src/include/catalog/pg_colenckeydata.h b/src/include/catalog/pg_colenckeydata.h
new file mode 100644
index 0000000000..3e4dea7218
--- /dev/null
+++ b/src/include/catalog/pg_colenckeydata.h
@@ -0,0 +1,46 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_colenckeydata.h
+ * definition of the "column encryption key data" system catalog
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_colenkeydata.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COLENCKEYDATA_H
+#define PG_COLENCKEYDATA_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_colenckeydata_d.h"
+
+/* ----------------
+ * pg_colenckeydata definition. cpp turns this into
+ * typedef struct FormData_pg_colenckeydata
+ * ----------------
+ */
+CATALOG(pg_colenckeydata,8250,ColumnEncKeyDataRelationId)
+{
+ Oid oid;
+ Oid ckdcekid BKI_LOOKUP(pg_colenckey);
+ Oid ckdcmkid BKI_LOOKUP(pg_colmasterkey);
+ int16 ckdcmkalg; /* PG_CMK_* values */
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ bytea ckdencval BKI_FORCE_NOT_NULL;
+#endif
+} FormData_pg_colenckeydata;
+
+typedef FormData_pg_colenckeydata *Form_pg_colenckeydata;
+
+DECLARE_TOAST(pg_colenckeydata, 8237, 8238);
+
+DECLARE_UNIQUE_INDEX_PKEY(pg_colenckeydata_oid_index, 8251, ColumnEncKeyDataOidIndexId, on pg_colenckeydata using btree(oid oid_ops));
+DECLARE_UNIQUE_INDEX(pg_colenckeydata_ckdcekid_ckdcmkid_index, 8252, ColumnEncKeyCekidCmkidIndexId, on pg_colenckeydata using btree(ckdcekid oid_ops, ckdcmkid oid_ops));
+
+#endif
diff --git a/src/include/catalog/pg_colmasterkey.h b/src/include/catalog/pg_colmasterkey.h
new file mode 100644
index 0000000000..0344cc4201
--- /dev/null
+++ b/src/include/catalog/pg_colmasterkey.h
@@ -0,0 +1,45 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_colmasterkey.h
+ * definition of the "column master key" system catalog
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_colmasterkey.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COLMASTERKEY_H
+#define PG_COLMASTERKEY_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_colmasterkey_d.h"
+
+/* ----------------
+ * pg_colmasterkey definition. cpp turns this into
+ * typedef struct FormData_pg_colmasterkey
+ * ----------------
+ */
+CATALOG(pg_colmasterkey,8233,ColumnMasterKeyRelationId)
+{
+ Oid oid;
+ NameData cmkname;
+ Oid cmkowner BKI_LOOKUP(pg_authid);
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text cmkrealm BKI_FORCE_NOT_NULL;
+#endif
+} FormData_pg_colmasterkey;
+
+typedef FormData_pg_colmasterkey *Form_pg_colmasterkey;
+
+DECLARE_TOAST(pg_colmasterkey, 8235, 8236);
+
+DECLARE_UNIQUE_INDEX_PKEY(pg_colmasterkey_oid_index, 8239, ColumnMasterKeyOidIndexId, on pg_colmasterkey using btree(oid oid_ops));
+DECLARE_UNIQUE_INDEX(pg_colmasterkey_cmkname_index, 8241, ColumnMasterKeyNameIndexId, on pg_colmasterkey using btree(cmkname name_ops));
+
+#endif
diff --git a/src/include/catalog/pg_opclass.dat b/src/include/catalog/pg_opclass.dat
index dbcae7ffdd..0ca401ffe4 100644
--- a/src/include/catalog/pg_opclass.dat
+++ b/src/include/catalog/pg_opclass.dat
@@ -166,6 +166,8 @@
opcintype => 'bool' },
{ opcmethod => 'hash', opcname => 'bytea_ops', opcfamily => 'hash/bytea_ops',
opcintype => 'bytea' },
+{ opcmethod => 'hash', opcname => 'pg_encrypted_det_ops', opcfamily => 'hash/pg_encrypted_det_ops',
+ opcintype => 'pg_encrypted_det' },
{ opcmethod => 'btree', opcname => 'tid_ops', opcfamily => 'btree/tid_ops',
opcintype => 'tid' },
{ opcmethod => 'hash', opcname => 'xid_ops', opcfamily => 'hash/xid_ops',
diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat
index bc5f8213f3..4737a7f9ed 100644
--- a/src/include/catalog/pg_operator.dat
+++ b/src/include/catalog/pg_operator.dat
@@ -3458,4 +3458,14 @@
oprcode => 'multirange_after_multirange', oprrest => 'multirangesel',
oprjoin => 'scalargtjoinsel' },
+{ oid => '8247', descr => 'equal',
+ oprname => '=', oprcanmerge => 'f', oprcanhash => 't', oprleft => 'pg_encrypted_det',
+ oprright => 'pg_encrypted_det', oprresult => 'bool', oprcom => '=(pg_encrypted_det,pg_encrypted_det)',
+ oprnegate => '<>(pg_encrypted_det,pg_encrypted_det)', oprcode => 'pg_encrypted_det_eq', oprrest => 'eqsel',
+ oprjoin => 'eqjoinsel' },
+{ oid => '8248', descr => 'not equal',
+ oprname => '<>', oprleft => 'pg_encrypted_det', oprright => 'pg_encrypted_det', oprresult => 'bool',
+ oprcom => '<>(pg_encrypted_det,pg_encrypted_det)', oprnegate => '=(pg_encrypted_det,pg_encrypted_det)',
+ oprcode => 'pg_encrypted_det_ne', oprrest => 'neqsel', oprjoin => 'neqjoinsel' },
+
]
diff --git a/src/include/catalog/pg_opfamily.dat b/src/include/catalog/pg_opfamily.dat
index b3b6a7e616..5343580b06 100644
--- a/src/include/catalog/pg_opfamily.dat
+++ b/src/include/catalog/pg_opfamily.dat
@@ -108,6 +108,8 @@
opfmethod => 'hash', opfname => 'bool_ops' },
{ oid => '2223',
opfmethod => 'hash', opfname => 'bytea_ops' },
+{ oid => '8249',
+ opfmethod => 'hash', opfname => 'pg_encrypted_det_ops' },
{ oid => '2789',
opfmethod => 'btree', opfname => 'tid_ops' },
{ oid => '2225',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 2e41f4d9e8..2fa26a2222 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -8025,9 +8025,9 @@
proname => 'pg_prepared_statement', prorows => '1000', proretset => 't',
provolatile => 's', proparallel => 'r', prorettype => 'record',
proargtypes => '',
- proallargtypes => '{text,text,timestamptz,_regtype,_regtype,bool,int8,int8}',
- proargmodes => '{o,o,o,o,o,o,o,o}',
- proargnames => '{name,statement,prepare_time,parameter_types,result_types,from_sql,generic_plans,custom_plans}',
+ proallargtypes => '{text,text,timestamptz,_regtype,_regclass,_int2,_regtype,bool,int8,int8}',
+ proargmodes => '{o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{name,statement,prepare_time,parameter_types,parameter_orig_tables,parameter_orig_columns,result_types,from_sql,generic_plans,custom_plans}',
prosrc => 'pg_prepared_statement' },
{ oid => '2511', descr => 'get the open cursors for this session',
proname => 'pg_cursor', prorows => '1000', proretset => 't',
@@ -11886,4 +11886,11 @@
prorettype => 'bytea', proargtypes => 'pg_brin_minmax_multi_summary',
prosrc => 'brin_minmax_multi_summary_send' },
+{ oid => '8245',
+ proname => 'pg_encrypted_det_eq', proleakproof => 't', prorettype => 'bool',
+ proargtypes => 'pg_encrypted_det pg_encrypted_det', prosrc => 'byteaeq' },
+{ oid => '8246',
+ proname => 'pg_encrypted_det_ne', proleakproof => 't', prorettype => 'bool',
+ proargtypes => 'pg_encrypted_det pg_encrypted_det', prosrc => 'byteane' },
+
]
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index df45879463..df1fa3e393 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -692,4 +692,16 @@
typreceive => 'brin_minmax_multi_summary_recv',
typsend => 'brin_minmax_multi_summary_send', typalign => 'i',
typstorage => 'x', typcollation => 'default' },
+
+{ oid => '8243', descr => 'encrypted column (deterministic)',
+ typname => 'pg_encrypted_det', typlen => '-1', typbyval => 'f', typtype => 'y',
+ typcategory => 'Y', typinput => 'byteain', typoutput => 'byteaout',
+ typreceive => 'bytearecv', typsend => 'byteasend', typalign => 'i',
+ typstorage => 'e' },
+{ oid => '8244', descr => 'encrypted column (randomized)',
+ typname => 'pg_encrypted_rnd', typlen => '-1', typbyval => 'f', typtype => 'y',
+ typcategory => 'Y', typinput => 'byteain', typoutput => 'byteaout',
+ typreceive => 'bytearecv', typsend => 'byteasend', typalign => 'i',
+ typstorage => 'e' },
+
]
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 48a2559137..11aff9ff83 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -277,6 +277,7 @@ DECLARE_UNIQUE_INDEX(pg_type_typname_nsp_index, 2704, TypeNameNspIndexId, on pg_
#define TYPTYPE_MULTIRANGE 'm' /* multirange type */
#define TYPTYPE_PSEUDO 'p' /* pseudo-type */
#define TYPTYPE_RANGE 'r' /* range type */
+#define TYPTYPE_ENCRYPTED 'y' /* encrypted column value */
#define TYPCATEGORY_INVALID '\0' /* not an allowed category */
#define TYPCATEGORY_ARRAY 'A'
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index f93d866548..8c08bd845f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -679,6 +679,7 @@ typedef struct ColumnDef
char *colname; /* name of column */
TypeName *typeName; /* type of column */
char *compression; /* compression method for column */
+ List *encryption; /* encryption info for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
diff --git a/src/include/parser/analyze.h b/src/include/parser/analyze.h
index dc379547c7..c310e124f1 100644
--- a/src/include/parser/analyze.h
+++ b/src/include/parser/analyze.h
@@ -28,7 +28,9 @@ extern PGDLLIMPORT post_parse_analyze_hook_type post_parse_analyze_hook;
extern Query *parse_analyze_fixedparams(RawStmt *parseTree, const char *sourceText,
const Oid *paramTypes, int numParams, QueryEnvironment *queryEnv);
extern Query *parse_analyze_varparams(RawStmt *parseTree, const char *sourceText,
- Oid **paramTypes, int *numParams, QueryEnvironment *queryEnv);
+ Oid **paramTypes, int *numParams,
+ Oid **paramOrigTbls, AttrNumber **paramOrigCols,
+ QueryEnvironment *queryEnv);
extern Query *parse_analyze_withcb(RawStmt *parseTree, const char *sourceText,
ParserSetupHook parserSetup,
void *parserSetupArg,
diff --git a/src/include/parser/parse_node.h b/src/include/parser/parse_node.h
index cf9c759025..225a2a8733 100644
--- a/src/include/parser/parse_node.h
+++ b/src/include/parser/parse_node.h
@@ -93,6 +93,7 @@ typedef Node *(*ParseParamRefHook) (ParseState *pstate, ParamRef *pref);
typedef Node *(*CoerceParamHook) (ParseState *pstate, Param *param,
Oid targetTypeId, int32 targetTypeMod,
int location);
+typedef void (*ParamAssignOrigHook) (ParseState *pstate, Param *param, Oid origtbl, AttrNumber origcol);
/*
@@ -222,6 +223,7 @@ struct ParseState
PostParseColumnRefHook p_post_columnref_hook;
ParseParamRefHook p_paramref_hook;
CoerceParamHook p_coerce_param_hook;
+ ParamAssignOrigHook p_param_assign_orig_hook;
void *p_ref_hook_state; /* common passthrough link for above */
};
diff --git a/src/include/parser/parse_param.h b/src/include/parser/parse_param.h
index df1ee660d8..da202b28a7 100644
--- a/src/include/parser/parse_param.h
+++ b/src/include/parser/parse_param.h
@@ -18,7 +18,8 @@
extern void setup_parse_fixed_parameters(ParseState *pstate,
const Oid *paramTypes, int numParams);
extern void setup_parse_variable_parameters(ParseState *pstate,
- Oid **paramTypes, int *numParams);
+ Oid **paramTypes, int *numParams,
+ Oid **paramOrigTbls, AttrNumber **paramOrigCols);
extern void check_variable_parameters(ParseState *pstate, Query *query);
extern bool query_contains_extern_params(Query *query);
diff --git a/src/include/tcop/tcopprot.h b/src/include/tcop/tcopprot.h
index 70d9dab25b..3038ceb522 100644
--- a/src/include/tcop/tcopprot.h
+++ b/src/include/tcop/tcopprot.h
@@ -53,6 +53,8 @@ extern List *pg_analyze_and_rewrite_varparams(RawStmt *parsetree,
const char *query_string,
Oid **paramTypes,
int *numParams,
+ Oid **paramOrigTbls,
+ AttrNumber **paramOrigCols,
QueryEnvironment *queryEnv);
extern List *pg_analyze_and_rewrite_withcb(RawStmt *parsetree,
const char *query_string,
diff --git a/src/include/utils/plancache.h b/src/include/utils/plancache.h
index 0499635f59..9a7cf794ce 100644
--- a/src/include/utils/plancache.h
+++ b/src/include/utils/plancache.h
@@ -101,6 +101,10 @@ typedef struct CachedPlanSource
CommandTag commandTag; /* 'nuff said */
Oid *param_types; /* array of parameter type OIDs, or NULL */
int num_params; /* length of param_types array */
+ Oid *param_origtbls; /* array of underlying tables of parameters,
+ * or NULL */
+ AttrNumber *param_origcols; /* array of underlying columns of parameters,
+ * or NULL */
ParserSetupHook parserSetup; /* alternative parameter spec method */
void *parserSetupArg;
int cursor_options; /* cursor options used for planning */
@@ -199,6 +203,8 @@ extern void CompleteCachedPlan(CachedPlanSource *plansource,
MemoryContext querytree_context,
Oid *param_types,
int num_params,
+ Oid *param_origtbls,
+ AttrNumber *param_origcols,
ParserSetupHook parserSetup,
void *parserSetupArg,
int cursor_options,
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 4463ea66be..4e8a940590 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -44,8 +44,12 @@ enum SysCacheIdentifier
AUTHNAME,
AUTHOID,
CASTSOURCETARGET,
+ CEKDATAOID,
+ CEKNAME,
+ CEKOID,
CLAAMNAMENSP,
CLAOID,
+ CMKOID,
COLLNAMEENCNSP,
COLLOID,
CONDEFAULT,
diff --git a/src/backend/access/common/printsimple.c b/src/backend/access/common/printsimple.c
index e99aa279f6..79154f8aab 100644
--- a/src/backend/access/common/printsimple.c
+++ b/src/backend/access/common/printsimple.c
@@ -46,6 +46,8 @@ printsimple_startup(DestReceiver *self, int operation, TupleDesc tupdesc)
pq_sendint16(&buf, attr->attlen);
pq_sendint32(&buf, attr->atttypmod);
pq_sendint16(&buf, 0); /* format code */
+ pq_sendint32(&buf, 0); /* CEK */
+ pq_sendint16(&buf, 0); /* CEK alg */
}
pq_endmessage(&buf);
diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c
index d2f3b57288..28b437e5c7 100644
--- a/src/backend/access/common/printtup.c
+++ b/src/backend/access/common/printtup.c
@@ -15,13 +15,26 @@
*/
#include "postgres.h"
+#include "access/genam.h"
#include "access/printtup.h"
+#include "access/skey.h"
+#include "access/table.h"
+#include "catalog/pg_colenckey.h"
+#include "catalog/pg_colenckeydata.h"
+#include "catalog/pg_colmasterkey.h"
#include "libpq/libpq.h"
#include "libpq/pqformat.h"
#include "tcop/pquery.h"
+#include "utils/array.h"
+#include "utils/arrayaccess.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/inval.h"
#include "utils/lsyscache.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
static void printtup_startup(DestReceiver *self, int operation,
@@ -151,6 +164,135 @@ printtup_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
*/
}
+/*
+ * Send ColumnMasterKey message, unless it's already been sent in this session
+ * for this key.
+ */
+List *cmk_sent = NIL;
+
+static void
+cmk_change_cb(Datum arg, int cacheid, uint32 hashvalue)
+{
+ list_free(cmk_sent);
+ cmk_sent = NIL;
+}
+
+static void
+MaybeSendColumnMasterKeyMessage(Oid cmkid)
+{
+ HeapTuple tuple;
+ Form_pg_colmasterkey cmkform;
+ Datum datum;
+ bool isnull;
+ StringInfoData buf;
+ static bool registered_inval = false;
+ MemoryContext oldcontext;
+
+ if (list_member_oid(cmk_sent, cmkid))
+ return;
+
+ tuple = SearchSysCache1(CMKOID, ObjectIdGetDatum(cmkid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for column master key %u", cmkid);
+ cmkform = (Form_pg_colmasterkey) GETSTRUCT(tuple);
+
+ pq_beginmessage(&buf, 'y'); /* ColumnMasterKey */
+ pq_sendint32(&buf, cmkform->oid);
+ pq_sendstring(&buf, NameStr(cmkform->cmkname));
+ datum = SysCacheGetAttr(CMKOID, tuple, Anum_pg_colmasterkey_cmkrealm, &isnull);
+ Assert(!isnull);
+ pq_sendstring(&buf, TextDatumGetCString(datum));
+ pq_endmessage(&buf);
+
+ ReleaseSysCache(tuple);
+
+ if (!registered_inval)
+ {
+ CacheRegisterSyscacheCallback(CMKOID, cmk_change_cb, (Datum) 0);
+ registered_inval = true;
+ }
+
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+ cmk_sent = lappend_oid(cmk_sent, cmkid);
+ MemoryContextSwitchTo(oldcontext);
+}
+
+/*
+ * Send ColumnEncryptionKey message, unless it's already been sent in this
+ * session for this key.
+ */
+List *cek_sent = NIL;
+
+static void
+cek_change_cb(Datum arg, int cacheid, uint32 hashvalue)
+{
+ list_free(cek_sent);
+ cek_sent = NIL;
+}
+
+void
+MaybeSendColumnEncryptionKeyMessage(Oid attcek)
+{
+ HeapTuple tuple;
+ ScanKeyData skey[1];
+ SysScanDesc sd;
+ Relation rel;
+ bool found = false;
+ static bool registered_inval = false;
+ MemoryContext oldcontext;
+
+ if (list_member_oid(cek_sent, attcek))
+ return;
+
+ ScanKeyInit(&skey[0],
+ Anum_pg_colenckeydata_ckdcekid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attcek));
+ rel = table_open(ColumnEncKeyDataRelationId, AccessShareLock);
+ sd = systable_beginscan(rel, ColumnEncKeyCekidCmkidIndexId, true, NULL, 1, skey);
+
+ while ((tuple = systable_getnext(sd)))
+ {
+ Form_pg_colenckeydata ckdform = (Form_pg_colenckeydata) GETSTRUCT(tuple);
+ Datum datum;
+ bool isnull;
+ bytea *ba;
+ StringInfoData buf;
+
+ MaybeSendColumnMasterKeyMessage(ckdform->ckdcmkid);
+
+ datum = heap_getattr(tuple, Anum_pg_colenckeydata_ckdencval, RelationGetDescr(rel), &isnull);
+ Assert(!isnull);
+ ba = pg_detoast_datum_packed((bytea *) DatumGetPointer(datum));
+
+ pq_beginmessage(&buf, 'Y'); /* ColumnEncryptionKey */
+ pq_sendint32(&buf, ckdform->ckdcekid);
+ pq_sendint32(&buf, ckdform->ckdcmkid);
+ pq_sendint16(&buf, ckdform->ckdcmkalg);
+ pq_sendint32(&buf, VARSIZE_ANY_EXHDR(ba));
+ pq_sendbytes(&buf, VARDATA_ANY(ba), VARSIZE_ANY_EXHDR(ba));
+ pq_endmessage(&buf);
+
+ found = true;
+ }
+
+ if (!found)
+ elog(ERROR, "lookup failed for column encryption key data %u", attcek);
+
+ systable_endscan(sd);
+ table_close(rel, NoLock);
+
+ if (!registered_inval)
+ {
+ CacheRegisterSyscacheCallback(CEKDATAOID, cek_change_cb, (Datum) 0);
+ registered_inval = true;
+ }
+
+ oldcontext = MemoryContextSwitchTo(TopMemoryContext);
+ cek_sent = lappend_oid(cek_sent, attcek);
+ MemoryContextSwitchTo(oldcontext);
+}
+
/*
* SendRowDescriptionMessage --- send a RowDescription message to the frontend
*
@@ -200,6 +342,8 @@ SendRowDescriptionMessage(StringInfo buf, TupleDesc typeinfo,
Oid resorigtbl;
AttrNumber resorigcol;
int16 format;
+ Oid attcekid = InvalidOid;
+ int16 attencalg = 0;
/*
* If column is a domain, send the base type and typmod instead.
@@ -231,6 +375,22 @@ SendRowDescriptionMessage(StringInfo buf, TupleDesc typeinfo,
else
format = 0;
+ if (get_typtype(atttypid) == TYPTYPE_ENCRYPTED)
+ {
+ HeapTuple tp;
+ Form_pg_attribute orig_att;
+
+ tp = SearchSysCache2(ATTNUM, ObjectIdGetDatum(resorigtbl), Int16GetDatum(resorigcol));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u", resorigcol, resorigtbl);
+ orig_att = (Form_pg_attribute) GETSTRUCT(tp);
+ MaybeSendColumnEncryptionKeyMessage(orig_att->attcek);
+ atttypid = orig_att->attrealtypid;
+ attcekid = orig_att->attcek;
+ attencalg = orig_att->attencalg;
+ ReleaseSysCache(tp);
+ }
+
pq_writestring(buf, NameStr(att->attname));
pq_writeint32(buf, resorigtbl);
pq_writeint16(buf, resorigcol);
@@ -238,6 +398,8 @@ SendRowDescriptionMessage(StringInfo buf, TupleDesc typeinfo,
pq_writeint16(buf, att->attlen);
pq_writeint32(buf, atttypmod);
pq_writeint16(buf, format);
+ pq_writeint32(buf, attcekid);
+ pq_writeint16(buf, attencalg);
}
pq_endmessage_reuse(buf);
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index d6fb261e20..e3ecc64ea7 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -459,6 +459,12 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attislocal != attr2->attislocal)
return false;
+ if (attr1->attcek != attr2->attcek)
+ return false;
+ if (attr1->attrealtypid != attr2->attrealtypid)
+ return false;
+ if (attr1->attencalg != attr2->attencalg)
+ return false;
if (attr1->attinhcount != attr2->attinhcount)
return false;
if (attr1->attcollation != attr2->attcollation)
@@ -629,6 +635,9 @@ TupleDescInitEntry(TupleDesc desc,
att->attgenerated = '\0';
att->attisdropped = false;
att->attislocal = true;
+ att->attcek = 0;
+ att->attrealtypid = 0;
+ att->attencalg = 0;
att->attinhcount = 0;
/* variable-length fields are not present in tupledescs */
@@ -690,6 +699,9 @@ TupleDescInitBuiltinEntry(TupleDesc desc,
att->attgenerated = '\0';
att->attisdropped = false;
att->attislocal = true;
+ att->attcek = 0;
+ att->attrealtypid = 0;
+ att->attencalg = 0;
att->attinhcount = 0;
/* variable-length fields are not present in tupledescs */
diff --git a/src/backend/access/hash/hashvalidate.c b/src/backend/access/hash/hashvalidate.c
index 10bf26ce7c..cea91d4a88 100644
--- a/src/backend/access/hash/hashvalidate.c
+++ b/src/backend/access/hash/hashvalidate.c
@@ -331,7 +331,7 @@ check_hash_func_signature(Oid funcid, int16 amprocnum, Oid argtype)
argtype == BOOLOID)
/* okay, allowed use of hashchar() */ ;
else if ((funcid == F_HASHVARLENA || funcid == F_HASHVARLENAEXTENDED) &&
- argtype == BYTEAOID)
+ (argtype == BYTEAOID || argtype == PG_ENCRYPTED_DETOID))
/* okay, allowed use of hashvarlena() */ ;
else
result = false;
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 89a0221ec9..7e1a91b974 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -72,7 +72,8 @@ CATALOG_HEADERS := \
pg_collation.h pg_parameter_acl.h pg_partitioned_table.h \
pg_range.h pg_transform.h \
pg_sequence.h pg_publication.h pg_publication_namespace.h \
- pg_publication_rel.h pg_subscription.h pg_subscription_rel.h
+ pg_publication_rel.h pg_subscription.h pg_subscription_rel.h \
+ pg_colmasterkey.h pg_colenckey.h pg_colenckeydata.h
GENERATED_HEADERS := $(CATALOG_HEADERS:%.h=%_d.h) schemapg.h system_fk_info.h
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 1803194db9..0ea58239b4 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -746,6 +746,9 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attgenerated - 1] = CharGetDatum(attrs->attgenerated);
slot[slotCount]->tts_values[Anum_pg_attribute_attisdropped - 1] = BoolGetDatum(attrs->attisdropped);
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcek - 1] = ObjectIdGetDatum(attrs->attcek);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attrealtypid - 1] = ObjectIdGetDatum(attrs->attrealtypid);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attencalg - 1] = Int16GetDatum(attrs->attencalg);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
if (attoptions && attoptions[natts] != (Datum) 0)
diff --git a/src/backend/commands/prepare.c b/src/backend/commands/prepare.c
index 2333aae467..94578350bd 100644
--- a/src/backend/commands/prepare.c
+++ b/src/backend/commands/prepare.c
@@ -50,7 +50,6 @@ static void InitQueryHashTable(void);
static ParamListInfo EvaluateParams(ParseState *pstate,
PreparedStatement *pstmt, List *params,
EState *estate);
-static Datum build_regtype_array(Oid *param_types, int num_params);
/*
* Implements the 'PREPARE' utility statement.
@@ -62,6 +61,8 @@ PrepareQuery(ParseState *pstate, PrepareStmt *stmt,
RawStmt *rawstmt;
CachedPlanSource *plansource;
Oid *argtypes = NULL;
+ Oid *argorigtbls = NULL;
+ AttrNumber *argorigcols = NULL;
int nargs;
List *query_list;
@@ -108,6 +109,9 @@ PrepareQuery(ParseState *pstate, PrepareStmt *stmt,
argtypes[i++] = toid;
}
+
+ argorigtbls = (Oid *) palloc0(nargs * sizeof(Oid));
+ argorigcols = (AttrNumber *) palloc0(nargs * sizeof(AttrNumber));
}
/*
@@ -117,7 +121,9 @@ PrepareQuery(ParseState *pstate, PrepareStmt *stmt,
* Rewrite the query. The result could be 0, 1, or many queries.
*/
query_list = pg_analyze_and_rewrite_varparams(rawstmt, pstate->p_sourcetext,
- &argtypes, &nargs, NULL);
+ &argtypes, &nargs,
+ &argorigtbls, &argorigcols,
+ NULL);
/* Finish filling in the CachedPlanSource */
CompleteCachedPlan(plansource,
@@ -125,6 +131,8 @@ PrepareQuery(ParseState *pstate, PrepareStmt *stmt,
NULL,
argtypes,
nargs,
+ argorigtbls,
+ argorigcols,
NULL,
NULL,
CURSOR_OPT_PARALLEL_OK, /* allow parallel mode */
@@ -683,9 +691,11 @@ pg_prepared_statement(PG_FUNCTION_ARGS)
hash_seq_init(&hash_seq, prepared_queries);
while ((prep_stmt = hash_seq_search(&hash_seq)) != NULL)
{
+ int num_params = prep_stmt->plansource->num_params;
TupleDesc result_desc;
- Datum values[8];
- bool nulls[8];
+ Datum *tmp_ary;
+ Datum values[10];
+ bool nulls[10];
result_desc = prep_stmt->plansource->resultDesc;
@@ -694,25 +704,38 @@ pg_prepared_statement(PG_FUNCTION_ARGS)
values[0] = CStringGetTextDatum(prep_stmt->stmt_name);
values[1] = CStringGetTextDatum(prep_stmt->plansource->query_string);
values[2] = TimestampTzGetDatum(prep_stmt->prepare_time);
- values[3] = build_regtype_array(prep_stmt->plansource->param_types,
- prep_stmt->plansource->num_params);
+
+ tmp_ary = (Datum *) palloc(num_params * sizeof(Datum));
+ for (int i = 0; i < num_params; i++)
+ tmp_ary[i] = ObjectIdGetDatum(prep_stmt->plansource->param_types[i]);
+ values[3] = PointerGetDatum(construct_array_builtin(tmp_ary, num_params, REGTYPEOID));
+
+ tmp_ary = (Datum *) palloc(num_params * sizeof(Datum));
+ for (int i = 0; i < num_params; i++)
+ tmp_ary[i] = ObjectIdGetDatum(prep_stmt->plansource->param_origtbls[i]);
+ values[4] = PointerGetDatum(construct_array_builtin(tmp_ary, num_params, REGCLASSOID));
+
+ tmp_ary = (Datum *) palloc(num_params * sizeof(Datum));
+ for (int i = 0; i < num_params; i++)
+ tmp_ary[i] = Int16GetDatum(prep_stmt->plansource->param_origcols[i]);
+ values[5] = PointerGetDatum(construct_array_builtin(tmp_ary, num_params, INT2OID));
+
if (result_desc)
{
- Oid *result_types;
-
- result_types = (Oid *) palloc(result_desc->natts * sizeof(Oid));
+ tmp_ary = (Datum *) palloc(result_desc->natts * sizeof(Datum));
for (int i = 0; i < result_desc->natts; i++)
- result_types[i] = result_desc->attrs[i].atttypid;
- values[4] = build_regtype_array(result_types, result_desc->natts);
+ tmp_ary[i] = ObjectIdGetDatum(result_desc->attrs[i].atttypid);
+ values[6] = PointerGetDatum(construct_array_builtin(tmp_ary, result_desc->natts, REGTYPEOID));
}
else
{
/* no result descriptor (for example, DML statement) */
- nulls[4] = true;
+ nulls[6] = true;
}
- values[5] = BoolGetDatum(prep_stmt->from_sql);
- values[6] = Int64GetDatumFast(prep_stmt->plansource->num_generic_plans);
- values[7] = Int64GetDatumFast(prep_stmt->plansource->num_custom_plans);
+
+ values[7] = BoolGetDatum(prep_stmt->from_sql);
+ values[8] = Int64GetDatumFast(prep_stmt->plansource->num_generic_plans);
+ values[9] = Int64GetDatumFast(prep_stmt->plansource->num_custom_plans);
tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc,
values, nulls);
@@ -721,24 +744,3 @@ pg_prepared_statement(PG_FUNCTION_ARGS)
return (Datum) 0;
}
-
-/*
- * This utility function takes a C array of Oids, and returns a Datum
- * pointing to a one-dimensional Postgres array of regtypes. An empty
- * array is returned as a zero-element array, not NULL.
- */
-static Datum
-build_regtype_array(Oid *param_types, int num_params)
-{
- Datum *tmp_ary;
- ArrayType *result;
- int i;
-
- tmp_ary = (Datum *) palloc(num_params * sizeof(Datum));
-
- for (i = 0; i < num_params; i++)
- tmp_ary[i] = ObjectIdGetDatum(param_types[i]);
-
- result = construct_array_builtin(tmp_ary, num_params, REGTYPEOID);
- return PointerGetDatum(result);
-}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 2de0ebacec..a0a9c6b59a 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -35,6 +35,7 @@
#include "catalog/partition.h"
#include "catalog/pg_am.h"
#include "catalog/pg_attrdef.h"
+#include "catalog/pg_colenckey.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
@@ -931,6 +932,76 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->compression)
attr->attcompression = GetAttributeCompression(attr->atttypid,
colDef->compression);
+
+ if (colDef->encryption)
+ {
+ ListCell *lc;
+ char *cek = NULL;
+ Oid cekoid;
+ bool encdet = false;
+ int alg = PG_CEK_AEAD_AES_128_CBC_HMAC_SHA_256;
+
+ foreach(lc, colDef->encryption)
+ {
+ DefElem *el = lfirst_node(DefElem, lc);
+
+ if (strcmp(el->defname, "column_encryption_key") == 0)
+ cek = strVal(linitial(castNode(TypeName, el->arg)->names));
+ else if (strcmp(el->defname, "encryption_type") == 0)
+ {
+ char *val = strVal(linitial(castNode(TypeName, el->arg)->names));
+
+ if (strcmp(val, "deterministic") == 0)
+ encdet = true;
+ else if (strcmp(val, "randomized") == 0)
+ encdet = false;
+ else
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unrecognized encryption type: %s", val));
+ }
+ else if (strcmp(el->defname, "algorithm") == 0)
+ {
+ char *val = strVal(el->arg);
+
+ if (strcmp(val, "AEAD_AES_128_CBC_HMAC_SHA_256") == 0)
+ alg = PG_CEK_AEAD_AES_128_CBC_HMAC_SHA_256;
+ else if (strcmp(val, "AEAD_AES_192_CBC_HMAC_SHA_384") == 0)
+ alg = PG_CEK_AEAD_AES_192_CBC_HMAC_SHA_384;
+ else if (strcmp(val, "AEAD_AES_256_CBC_HMAC_SHA_384") == 0)
+ alg = PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_384;
+ else if (strcmp(val, "AEAD_AES_256_CBC_HMAC_SHA_512") == 0)
+ alg = PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_512;
+ else
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unrecognized encryption algorithm: %s", val));
+ }
+ else
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unrecognized column encryption parameter: %s", el->defname));
+ }
+
+ if (!cek)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_COLUMN_DEFINITION),
+ errmsg("column encryption key must be specified"));
+
+ cekoid = GetSysCacheOid1(CEKNAME, Anum_pg_colenckey_oid, PointerGetDatum(cek));
+ if (!cekoid)
+ ereport(ERROR,
+ errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("column encryption key \"%s\" does not exist", cek));
+
+ attr->attcek = cekoid;
+ attr->attrealtypid = attr->atttypid;
+ if (encdet)
+ attr->atttypid = PG_ENCRYPTED_DETOID;
+ else
+ attr->atttypid = PG_ENCRYPTED_RNDOID;
+ attr->attencalg = alg;
+ }
}
/*
@@ -6829,6 +6900,9 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attgenerated = colDef->generated;
attribute.attisdropped = false;
attribute.attislocal = colDef->is_local;
+ attribute.attcek = 0; // TODO
+ attribute.attrealtypid = 0; // TODO
+ attribute.attencalg = 0; // TODO
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 29bc26669b..cde5b0e087 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -2279,6 +2279,8 @@ _SPI_prepare_plan(const char *src, SPIPlanPtr plan)
NULL,
plan->argtypes,
plan->nargs,
+ NULL,
+ NULL,
plan->parserSetup,
plan->parserSetupArg,
plan->cursor_options,
@@ -2516,6 +2518,8 @@ _SPI_execute_plan(SPIPlanPtr plan, const SPIExecuteOptions *options,
NULL,
plan->argtypes,
plan->nargs,
+ NULL,
+ NULL,
plan->parserSetup,
plan->parserSetupArg,
plan->cursor_options,
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 51d630fa89..c901cfbb86 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3555,6 +3555,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(encryption);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e747e1667d..69a3adee7c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -3045,6 +3045,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(encryption);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 4cb1744da6..8fbe0f5f8f 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -4288,6 +4288,8 @@ raw_expression_tree_walker(Node *node,
return true;
if (walker(coldef->compression, context))
return true;
+ if (walker(coldef->encryption, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 4315c53080..777eac7f62 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -3116,6 +3116,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(encryption);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 1bcb875507..edc9fb4a28 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -145,6 +145,7 @@ parse_analyze_fixedparams(RawStmt *parseTree, const char *sourceText,
Query *
parse_analyze_varparams(RawStmt *parseTree, const char *sourceText,
Oid **paramTypes, int *numParams,
+ Oid **paramOrigTbls, AttrNumber **paramOrigCols,
QueryEnvironment *queryEnv)
{
ParseState *pstate = make_parsestate(NULL);
@@ -155,7 +156,7 @@ parse_analyze_varparams(RawStmt *parseTree, const char *sourceText,
pstate->p_sourcetext = sourceText;
- setup_parse_variable_parameters(pstate, paramTypes, numParams);
+ setup_parse_variable_parameters(pstate, paramTypes, numParams, paramOrigTbls, paramOrigCols);
pstate->p_queryEnv = queryEnv;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 969c9c158f..a918eaf427 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <node> TableConstraint TableLikeClause
%type <ival> TableLikeOptionList TableLikeOption
%type <str> column_compression opt_column_compression
+%type <list> opt_column_encryption
%type <list> ColQualList
%type <node> ColConstraint ColConstraintElem ConstraintAttr
%type <ival> key_match
@@ -3778,13 +3779,14 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename opt_column_compression create_generic_options ColQualList
+columnDef: ColId Typename opt_column_compression opt_column_encryption create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
n->compression = $3;
+ n->encryption = $4;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3793,8 +3795,8 @@ columnDef: ColId Typename opt_column_compression create_generic_options ColQualL
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $4;
- SplitColQualList($5, &n->constraints, &n->collClause,
+ n->fdwoptions = $5;
+ SplitColQualList($6, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *) n;
@@ -3851,6 +3853,11 @@ opt_column_compression:
| /*EMPTY*/ { $$ = NULL; }
;
+opt_column_encryption:
+ ENCRYPTED WITH '(' def_list ')' { $$ = $4; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
diff --git a/src/backend/parser/parse_param.c b/src/backend/parser/parse_param.c
index f668abfcb3..b45bf4c438 100644
--- a/src/backend/parser/parse_param.c
+++ b/src/backend/parser/parse_param.c
@@ -49,6 +49,8 @@ typedef struct VarParamState
{
Oid **paramTypes; /* array of parameter type OIDs */
int *numParams; /* number of array entries */
+ Oid **paramOrigTbls; /* underlying tables (0 if none) */
+ AttrNumber **paramOrigCols; /* underlying columns (0 if none) */
} VarParamState;
static Node *fixed_paramref_hook(ParseState *pstate, ParamRef *pref);
@@ -56,6 +58,7 @@ static Node *variable_paramref_hook(ParseState *pstate, ParamRef *pref);
static Node *variable_coerce_param_hook(ParseState *pstate, Param *param,
Oid targetTypeId, int32 targetTypeMod,
int location);
+static void variable_param_assign_orig_hook(ParseState *pstate, Param *param, Oid origtbl, AttrNumber origcol);
static bool check_parameter_resolution_walker(Node *node, ParseState *pstate);
static bool query_contains_extern_params_walker(Node *node, void *context);
@@ -81,15 +84,19 @@ setup_parse_fixed_parameters(ParseState *pstate,
*/
void
setup_parse_variable_parameters(ParseState *pstate,
- Oid **paramTypes, int *numParams)
+ Oid **paramTypes, int *numParams,
+ Oid **paramOrigTbls, AttrNumber **paramOrigCols)
{
VarParamState *parstate = palloc(sizeof(VarParamState));
parstate->paramTypes = paramTypes;
parstate->numParams = numParams;
+ parstate->paramOrigTbls = paramOrigTbls;
+ parstate->paramOrigCols = paramOrigCols;
pstate->p_ref_hook_state = (void *) parstate;
pstate->p_paramref_hook = variable_paramref_hook;
pstate->p_coerce_param_hook = variable_coerce_param_hook;
+ pstate->p_param_assign_orig_hook = variable_param_assign_orig_hook;
}
/*
@@ -145,14 +152,36 @@ variable_paramref_hook(ParseState *pstate, ParamRef *pref)
{
/* Need to enlarge param array */
if (*parstate->paramTypes)
+ {
*parstate->paramTypes = (Oid *) repalloc(*parstate->paramTypes,
paramno * sizeof(Oid));
+ if (parstate->paramOrigTbls)
+ *parstate->paramOrigTbls = repalloc(*parstate->paramOrigTbls,
+ paramno * sizeof(Oid));
+ if (parstate->paramOrigCols)
+ *parstate->paramOrigCols = repalloc(*parstate->paramOrigCols,
+ paramno * sizeof(AttrNumber));
+ }
else
+ {
*parstate->paramTypes = (Oid *) palloc(paramno * sizeof(Oid));
+ if (parstate->paramOrigTbls)
+ *parstate->paramOrigTbls = palloc(paramno * sizeof(Oid));
+ if (parstate->paramOrigCols)
+ *parstate->paramOrigCols = palloc(paramno * sizeof(AttrNumber));
+ }
/* Zero out the previously-unreferenced slots */
MemSet(*parstate->paramTypes + *parstate->numParams,
0,
(paramno - *parstate->numParams) * sizeof(Oid));
+ if (parstate->paramOrigTbls)
+ MemSet(*parstate->paramOrigTbls + *parstate->numParams,
+ 0,
+ (paramno - *parstate->numParams) * sizeof(Oid));
+ if (parstate->paramOrigCols)
+ MemSet(*parstate->paramOrigCols + *parstate->numParams,
+ 0,
+ (paramno - *parstate->numParams) * sizeof(AttrNumber));
*parstate->numParams = paramno;
}
@@ -260,6 +289,18 @@ variable_coerce_param_hook(ParseState *pstate, Param *param,
return NULL;
}
+static void
+variable_param_assign_orig_hook(ParseState *pstate, Param *param, Oid origtbl, AttrNumber origcol)
+{
+ VarParamState *parstate = (VarParamState *) pstate->p_ref_hook_state;
+ int paramno = param->paramid;
+
+ if (parstate->paramOrigTbls)
+ (*parstate->paramOrigTbls)[paramno - 1] = origtbl;
+ if (parstate->paramOrigCols)
+ (*parstate->paramOrigCols)[paramno - 1] = origcol;
+}
+
/*
* Check for consistent assignment of variable parameters after completion
* of parsing with parse_variable_parameters.
diff --git a/src/backend/parser/parse_target.c b/src/backend/parser/parse_target.c
index 2a1d44b813..b964088044 100644
--- a/src/backend/parser/parse_target.c
+++ b/src/backend/parser/parse_target.c
@@ -595,6 +595,12 @@ transformAssignedExpr(ParseState *pstate,
parser_errposition(pstate, exprLocation(orig_expr))));
}
+ if (IsA(expr, Param))
+ {
+ if (pstate->p_param_assign_orig_hook)
+ pstate->p_param_assign_orig_hook(pstate, castNode(Param, expr), RelationGetRelid(pstate->p_target_relation), attrno);
+ }
+
pstate->p_expr_kind = sv_expr_kind;
return expr;
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 495cbf2006..82168235e8 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -79,6 +79,7 @@
#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
#include "utils/timeout.h"
#include "utils/timestamp.h"
@@ -680,6 +681,8 @@ pg_analyze_and_rewrite_varparams(RawStmt *parsetree,
const char *query_string,
Oid **paramTypes,
int *numParams,
+ Oid **paramOrigTbls,
+ AttrNumber **paramOrigCols,
QueryEnvironment *queryEnv)
{
Query *query;
@@ -694,7 +697,7 @@ pg_analyze_and_rewrite_varparams(RawStmt *parsetree,
ResetUsage();
query = parse_analyze_varparams(parsetree, query_string, paramTypes, numParams,
- queryEnv);
+ paramOrigTbls, paramOrigCols, queryEnv);
/*
* Check all parameter types got determined.
@@ -1371,6 +1374,8 @@ exec_parse_message(const char *query_string, /* string to execute */
bool is_named;
bool save_log_statement_stats = log_statement_stats;
char msec_str[32];
+ Oid *paramOrigTbls = palloc(numParams * sizeof(Oid));
+ AttrNumber *paramOrigCols = palloc(numParams * sizeof(AttrNumber));
/*
* Report query to various monitoring facilities.
@@ -1491,6 +1496,8 @@ exec_parse_message(const char *query_string, /* string to execute */
query_string,
¶mTypes,
&numParams,
+ ¶mOrigTbls,
+ ¶mOrigCols,
NULL);
/* Done with the snapshot used for parsing */
@@ -1521,6 +1528,8 @@ exec_parse_message(const char *query_string, /* string to execute */
unnamed_stmt_context,
paramTypes,
numParams,
+ paramOrigTbls,
+ paramOrigCols,
NULL,
NULL,
CURSOR_OPT_PARALLEL_OK, /* allow parallel mode */
@@ -1818,6 +1827,16 @@ exec_bind_message(StringInfo input_message)
else
pformat = 0; /* default = text */
+ if (get_typtype(ptype) == TYPTYPE_ENCRYPTED)
+ {
+ if (pformat & 0xF0)
+ pformat &= ~0xF0;
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("parameter corresponds to an encrypted column, but the parameter value was not encrypted")));
+ }
+
if (pformat == 0) /* text mode */
{
Oid typinput;
@@ -2614,8 +2633,41 @@ exec_describe_statement_message(const char *stmt_name)
for (int i = 0; i < psrc->num_params; i++)
{
Oid ptype = psrc->param_types[i];
+ Oid pcekid = InvalidOid;
+ int pcekalg = 0;
+ int16 pflags = 0;
+
+ if (get_typtype(ptype) == TYPTYPE_ENCRYPTED)
+ {
+ Oid porigtbl = psrc->param_origtbls[i];
+ AttrNumber porigcol = psrc->param_origcols[i];
+ HeapTuple tp;
+ Form_pg_attribute orig_att;
+
+ if (porigtbl == InvalidOid || porigcol <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("parameter %d corresponds to an encrypted column, but an underlying table and column could not be determined", i)));
+
+ tp = SearchSysCache2(ATTNUM, ObjectIdGetDatum(porigtbl), Int16GetDatum(porigcol));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u", porigcol, porigtbl);
+ orig_att = (Form_pg_attribute) GETSTRUCT(tp);
+ ptype = orig_att->attrealtypid;
+ pcekid = orig_att->attcek;
+ pcekalg = orig_att->attencalg;
+ ReleaseSysCache(tp);
+
+ if (psrc->param_types[i] == PG_ENCRYPTED_DETOID)
+ pflags |= 0x01;
+
+ MaybeSendColumnEncryptionKeyMessage(pcekid);
+ }
pq_sendint32(&row_description_buf, (int) ptype);
+ pq_sendint32(&row_description_buf, (int) pcekid);
+ pq_sendint16(&row_description_buf, pcekalg);
+ pq_sendint16(&row_description_buf, pflags);
}
pq_endmessage_reuse(&row_description_buf);
diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c
index b0c37ede87..5683731188 100644
--- a/src/backend/utils/adt/arrayfuncs.c
+++ b/src/backend/utils/adt/arrayfuncs.c
@@ -3387,6 +3387,7 @@ construct_array_builtin(Datum *elems, int nelems, Oid elmtype)
break;
case OIDOID:
+ case REGCLASSOID:
case REGTYPEOID:
elmlen = sizeof(Oid);
elmbyval = true;
diff --git a/src/backend/utils/cache/plancache.c b/src/backend/utils/cache/plancache.c
index 0d6a295674..f6893dd8b7 100644
--- a/src/backend/utils/cache/plancache.c
+++ b/src/backend/utils/cache/plancache.c
@@ -340,6 +340,8 @@ CompleteCachedPlan(CachedPlanSource *plansource,
MemoryContext querytree_context,
Oid *param_types,
int num_params,
+ Oid *param_origtbls,
+ AttrNumber *param_origcols,
ParserSetupHook parserSetup,
void *parserSetupArg,
int cursor_options,
@@ -419,6 +421,14 @@ CompleteCachedPlan(CachedPlanSource *plansource,
{
plansource->param_types = (Oid *) palloc(num_params * sizeof(Oid));
memcpy(plansource->param_types, param_types, num_params * sizeof(Oid));
+
+ plansource->param_origtbls = (Oid *) palloc0(num_params * sizeof(Oid));
+ if (param_origtbls)
+ memcpy(plansource->param_origtbls, param_origtbls, num_params * sizeof(Oid));
+
+ plansource->param_origcols = (AttrNumber *) palloc0(num_params * sizeof(AttrNumber));
+ if (param_origcols)
+ memcpy(plansource->param_origcols, param_origcols, num_params * sizeof(AttrNumber));
}
else
plansource->param_types = NULL;
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 1912b12146..468a6873e4 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -29,7 +29,10 @@
#include "catalog/pg_auth_members.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
+#include "catalog/pg_colenckey.h"
+#include "catalog/pg_colenckeydata.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_colmasterkey.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -267,6 +270,33 @@ static const struct cachedesc cacheinfo[] = {
},
256
},
+ {
+ ColumnEncKeyDataRelationId, /* CEKDATAOID */
+ ColumnEncKeyDataOidIndexId,
+ 1,
+ {
+ Anum_pg_colenckeydata_oid,
+ },
+ 8
+ },
+ {
+ ColumnEncKeyRelationId, /* CEKNAME */
+ ColumnEncKeyNameIndexId,
+ 1,
+ {
+ Anum_pg_colenckey_cekname,
+ },
+ 8
+ },
+ {
+ ColumnEncKeyRelationId, /* CEKOID */
+ ColumnEncKeyOidIndexId,
+ 1,
+ {
+ Anum_pg_colenckey_oid,
+ },
+ 8
+ },
{OperatorClassRelationId, /* CLAAMNAMENSP */
OpclassAmNameNspIndexId,
3,
@@ -289,6 +319,15 @@ static const struct cachedesc cacheinfo[] = {
},
8
},
+ {
+ ColumnMasterKeyRelationId, /* CMKOID */
+ ColumnMasterKeyOidIndexId,
+ 1,
+ {
+ Anum_pg_colmasterkey_oid,
+ },
+ 8
+ },
{CollationRelationId, /* COLLNAMEENCNSP */
CollationNameEncNspIndexId,
3,
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index c871cb727d..962947c78d 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -8066,6 +8066,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_typstorage;
int i_attidentity;
int i_attgenerated;
+ int i_attcekname;
int i_attisdropped;
int i_attlen;
int i_attalign;
@@ -8175,17 +8176,24 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
if (fout->remoteVersion >= 120000)
appendPQExpBufferStr(q,
- "a.attgenerated\n");
+ "a.attgenerated,\n");
else
appendPQExpBufferStr(q,
- "'' AS attgenerated\n");
+ "'' AS attgenerated,\n");
+
+ if (fout->remoteVersion >= 160000)
+ appendPQExpBufferStr(q,
+ "(SELECT cekname FROM pg_colenckey cek WHERE cek.oid = a.attcek ) AS attcekname\n");
+ else
+ appendPQExpBufferStr(q,
+ "NULL AS attcekname\n");
/* need left join to pg_type to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM unnest('%s'::pg_catalog.oid[]) AS src(tbloid)\n"
"JOIN pg_catalog.pg_attribute a ON (src.tbloid = a.attrelid) "
"LEFT JOIN pg_catalog.pg_type t "
- "ON (a.atttypid = t.oid)\n"
+ "ON (CASE WHEN a.attrealtypid <> 0 THEN a.attrealtypid ELSE a.atttypid END = t.oid)\n"
"WHERE a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attrelid, a.attnum",
tbloids->data);
@@ -8204,6 +8212,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_typstorage = PQfnumber(res, "typstorage");
i_attidentity = PQfnumber(res, "attidentity");
i_attgenerated = PQfnumber(res, "attgenerated");
+ i_attcekname = PQfnumber(res, "attcekname");
i_attisdropped = PQfnumber(res, "attisdropped");
i_attlen = PQfnumber(res, "attlen");
i_attalign = PQfnumber(res, "attalign");
@@ -8265,6 +8274,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->typstorage = (char *) pg_malloc(numatts * sizeof(char));
tbinfo->attidentity = (char *) pg_malloc(numatts * sizeof(char));
tbinfo->attgenerated = (char *) pg_malloc(numatts * sizeof(char));
+ tbinfo->attcekname = (char **) pg_malloc(numatts * sizeof(char *));
tbinfo->attisdropped = (bool *) pg_malloc(numatts * sizeof(bool));
tbinfo->attlen = (int *) pg_malloc(numatts * sizeof(int));
tbinfo->attalign = (char *) pg_malloc(numatts * sizeof(char));
@@ -8293,6 +8303,10 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attidentity[j] = *(PQgetvalue(res, r, i_attidentity));
tbinfo->attgenerated[j] = *(PQgetvalue(res, r, i_attgenerated));
tbinfo->needs_override = tbinfo->needs_override || (tbinfo->attidentity[j] == ATTRIBUTE_IDENTITY_ALWAYS);
+ if (!PQgetisnull(res, r, i_attcekname))
+ tbinfo->attcekname[j] = pg_strdup(PQgetvalue(res, r, i_attcekname));
+ else
+ tbinfo->attcekname[j] = NULL;
tbinfo->attisdropped[j] = (PQgetvalue(res, r, i_attisdropped)[0] == 't');
tbinfo->attlen[j] = atoi(PQgetvalue(res, r, i_attlen));
tbinfo->attalign[j] = *(PQgetvalue(res, r, i_attalign));
@@ -15260,6 +15274,12 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ if (tbinfo->attcekname[j])
+ {
+ appendPQExpBuffer(q, " ENCRYPTED WITH (column_encryption_key = %s)",
+ fmtId(tbinfo->attcekname[j]));
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1d21c2906f..2c176ed13c 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -333,6 +333,7 @@ typedef struct _tableInfo
bool *attisdropped; /* true if attr is dropped; don't dump it */
char *attidentity;
char *attgenerated;
+ char **attcekname;
int *attlen; /* attribute length, used by binary_upgrade */
char *attalign; /* attribute align, used by binary_upgrade */
bool *attislocal; /* true if attr has local definition */
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index c562c04afe..72e973d2de 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -98,6 +98,7 @@ static backslashResult process_command_g_options(char *first_option,
bool active_branch,
const char *cmd);
static backslashResult exec_command_gdesc(PsqlScanState scan_state, bool active_branch);
+static backslashResult exec_command_gencr(PsqlScanState scan_state, bool active_branch);
static backslashResult exec_command_getenv(PsqlScanState scan_state, bool active_branch,
const char *cmd);
static backslashResult exec_command_gexec(PsqlScanState scan_state, bool active_branch);
@@ -350,6 +351,8 @@ exec_command(const char *cmd,
status = exec_command_g(scan_state, active_branch, cmd);
else if (strcmp(cmd, "gdesc") == 0)
status = exec_command_gdesc(scan_state, active_branch);
+ else if (strcmp(cmd, "gencr") == 0)
+ status = exec_command_gencr(scan_state, active_branch);
else if (strcmp(cmd, "getenv") == 0)
status = exec_command_getenv(scan_state, active_branch, cmd);
else if (strcmp(cmd, "gexec") == 0)
@@ -1492,6 +1495,34 @@ exec_command_gdesc(PsqlScanState scan_state, bool active_branch)
return status;
}
+/*
+ * \gencr -- send query, with support for parameter encryption
+ */
+static backslashResult
+exec_command_gencr(PsqlScanState scan_state, bool active_branch)
+{
+ backslashResult status = PSQL_CMD_SKIP_LINE;
+ char *ap;
+
+ pset.num_params = 0;
+ pset.params = NULL;
+ while ((ap = psql_scan_slash_option(scan_state,
+ OT_NORMAL, NULL, true)) != NULL)
+ {
+ pset.num_params++;
+ pset.params = pg_realloc(pset.params, pset.num_params * sizeof(char *));
+ pset.params[pset.num_params - 1] = ap;
+ }
+
+ if (active_branch)
+ {
+ pset.gencr_flag = true;
+ status = PSQL_CMD_SEND;
+ }
+
+ return status;
+}
+
/*
* \getenv -- set variable from environment variable
*/
diff --git a/src/bin/psql/common.c b/src/bin/psql/common.c
index 9f95869eca..8fe055ffed 100644
--- a/src/bin/psql/common.c
+++ b/src/bin/psql/common.c
@@ -1284,6 +1284,14 @@ SendQuery(const char *query)
/* reset \gdesc trigger */
pset.gdesc_flag = false;
+ /* reset \gencr trigger */
+ pset.gencr_flag = false;
+ for (int i = 0; i < pset.num_params; i++)
+ pg_free(pset.params[i]);
+ pg_free(pset.params);
+ pset.params = NULL;
+ pset.num_params = 0;
+
/* reset \gexec trigger */
pset.gexec_flag = false;
@@ -1448,7 +1456,36 @@ ExecQueryAndProcessResults(const char *query, double *elapsed_msec, bool *svpt_g
if (timing)
INSTR_TIME_SET_CURRENT(before);
- success = PQsendQuery(pset.db, query);
+ if (pset.gencr_flag)
+ {
+ PGresult *res1,
+ *res2;
+
+ res1 = PQprepare(pset.db, "", query, pset.num_params, NULL);
+ if (PQresultStatus(res1) != PGRES_COMMAND_OK)
+ {
+ pg_log_info("%s", PQerrorMessage(pset.db));
+ ClearOrSaveResult(res1);
+ return -1;
+ }
+ PQclear(res1);
+
+ res2 = PQdescribePrepared(pset.db, "");
+ if (PQresultStatus(res2) != PGRES_COMMAND_OK)
+ {
+ pg_log_info("%s", PQerrorMessage(pset.db));
+ ClearOrSaveResult(res2);
+ return -1;
+ }
+
+ success = PQsendQueryPrepared2(pset.db, "", pset.num_params, (const char *const *) pset.params, NULL, NULL, NULL, 0, res2);
+
+ PQclear(res2);
+ }
+ else
+ {
+ success = PQsendQuery(pset.db, query);
+ }
if (!success)
{
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 88d92a08ae..fe81547b29 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1498,7 +1498,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[12];
+ char *headers[13];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1514,6 +1514,7 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attcompression_col = -1,
+ attcekname_col = -1,
attstattarget_col = -1,
attdescr_col = -1;
int numrows;
@@ -1812,7 +1813,7 @@ describeOneTableDetails(const char *schemaname,
cols = 0;
printfPQExpBuffer(&buf, "SELECT a.attname");
attname_col = cols++;
- appendPQExpBufferStr(&buf, ",\n pg_catalog.format_type(a.atttypid, a.atttypmod)");
+ appendPQExpBufferStr(&buf, ",\n pg_catalog.format_type(CASE WHEN a.attrealtypid <> 0 THEN a.attrealtypid ELSE a.atttypid END, a.atttypmod)");
atttype_col = cols++;
if (show_column_details)
@@ -1826,7 +1827,7 @@ describeOneTableDetails(const char *schemaname,
attrdef_col = cols++;
attnotnull_col = cols++;
appendPQExpBufferStr(&buf, ",\n (SELECT c.collname FROM pg_catalog.pg_collation c, pg_catalog.pg_type t\n"
- " WHERE c.oid = a.attcollation AND t.oid = a.atttypid AND a.attcollation <> t.typcollation) AS attcollation");
+ " WHERE c.oid = a.attcollation AND t.oid = (CASE WHEN a.attrealtypid <> 0 THEN a.attrealtypid ELSE a.atttypid END) AND a.attcollation <> t.typcollation) AS attcollation");
attcoll_col = cols++;
if (pset.sversion >= 100000)
appendPQExpBufferStr(&buf, ",\n a.attidentity");
@@ -1877,6 +1878,15 @@ describeOneTableDetails(const char *schemaname,
attcompression_col = cols++;
}
+ /* encryption info */
+ if (pset.sversion >= 160000 &&
+ !pset.hide_column_encryption &&
+ (tableinfo.relkind == RELKIND_RELATION))
+ {
+ appendPQExpBufferStr(&buf, ",\n (SELECT cekname FROM pg_colenckey cek WHERE cek.oid = a.attcek) AS attcekname");
+ attcekname_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2000,6 +2010,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("Storage");
if (attcompression_col >= 0)
headers[cols++] = gettext_noop("Compression");
+ if (attcekname_col >= 0)
+ headers[cols++] = gettext_noop("Encryption");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2092,6 +2104,17 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column encryption */
+ if (attcekname_col >= 0)
+ {
+ if (!PQgetisnull(res, i, attcekname_col))
+ printTableAddCell(&cont, PQgetvalue(res, i, attcekname_col),
+ false, false);
+ else
+ printTableAddCell(&cont, "",
+ false, false);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index f8ce1a0706..9f4fce26fd 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -195,6 +195,7 @@ slashUsage(unsigned short int pager)
HELP0(" \\g [(OPTIONS)] [FILE] execute query (and send result to file or |pipe);\n"
" \\g with no arguments is equivalent to a semicolon\n");
HELP0(" \\gdesc describe result of query, without executing it\n");
+ HELP0(" \\gencr [PARAM]... execute query, with support for parameter encryption\n");
HELP0(" \\gexec execute query, then execute each value in its result\n");
HELP0(" \\gset [PREFIX] execute query and store result in psql variables\n");
HELP0(" \\gx [(OPTIONS)] [FILE] as \\g, but forces expanded output mode\n");
@@ -412,6 +413,8 @@ helpVariables(unsigned short int pager)
" true if last query failed, else false\n");
HELP0(" FETCH_COUNT\n"
" the number of result rows to fetch and display at a time (0 = unlimited)\n");
+ HELP0(" HIDE_COLUMN_ENCRYPTION\n"
+ " if set, column encryption details are not displayed\n");
HELP0(" HIDE_TABLEAM\n"
" if set, table access methods are not displayed\n");
HELP0(" HIDE_TOAST_COMPRESSION\n"
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index 2399cffa3f..636729df1d 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -95,6 +95,10 @@ typedef struct _psqlSettings
char *gset_prefix; /* one-shot prefix argument for \gset */
bool gdesc_flag; /* one-shot request to describe query result */
+ bool gencr_flag; /* one-shot request to send query with support
+ * for parameter encryption */
+ int num_params; /* number of query parameters */
+ char **params; /* query parameters */
bool gexec_flag; /* one-shot request to execute query result */
bool crosstab_flag; /* one-shot request to crosstab result */
char *ctv_args[4]; /* \crosstabview arguments */
@@ -134,6 +138,7 @@ typedef struct _psqlSettings
bool quiet;
bool singleline;
bool singlestep;
+ bool hide_column_encryption;
bool hide_compression;
bool hide_tableam;
int fetch_count;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 7c2f555f15..c955222a50 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -1188,6 +1188,13 @@ hide_compression_hook(const char *newval)
&pset.hide_compression);
}
+static bool
+hide_column_encryption_hook(const char *newval)
+{
+ return ParseVariableBool(newval, "HIDE_COLUMN_ENCRYPTION",
+ &pset.hide_column_encryption);
+}
+
static bool
hide_tableam_hook(const char *newval)
{
@@ -1259,6 +1266,9 @@ EstablishVariableSpace(void)
SetVariableHooks(pset.vars, "SHOW_CONTEXT",
show_context_substitute_hook,
show_context_hook);
+ SetVariableHooks(pset.vars, "HIDE_COLUMN_ENCRYPTION",
+ bool_substitute_hook,
+ hide_column_encryption_hook);
SetVariableHooks(pset.vars, "HIDE_TOAST_COMPRESSION",
bool_substitute_hook,
hide_compression_hook);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index c5cafe6f4b..d5877dc5a3 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1692,7 +1692,7 @@ psql_completion(const char *text, int start, int end)
"\\echo", "\\edit", "\\ef", "\\elif", "\\else", "\\encoding",
"\\endif", "\\errverbose", "\\ev",
"\\f",
- "\\g", "\\gdesc", "\\getenv", "\\gexec", "\\gset", "\\gx",
+ "\\g", "\\gdesc", "\\gencr", "\\getenv", "\\gexec", "\\gset", "\\gx",
"\\help", "\\html",
"\\if", "\\include", "\\include_relative", "\\ir",
"\\list", "\\lo_import", "\\lo_export", "\\lo_list", "\\lo_unlink",
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index b5fd72a4ac..5a065eb80b 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -54,6 +54,7 @@ endif
ifeq ($(with_ssl),openssl)
OBJS += \
+ fe-encrypt-openssl.o \
fe-secure-openssl.o
endif
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index e8bcc88370..fabad38ac0 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -186,3 +186,7 @@ PQpipelineStatus 183
PQsetTraceFlags 184
PQmblenBounded 185
PQsendFlushRequest 186
+PQexecPrepared2 187
+PQsendQueryPrepared2 188
+PQfisencrypted 189
+PQparamisencrypted 190
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index dc49387d6c..f3c15ece4d 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -344,6 +344,10 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"Target-Session-Attrs", "", 15, /* sizeof("prefer-standby") = 15 */
offsetof(struct pg_conn, target_session_attrs)},
+ {"cmklookup", "PGCMKLOOKUP", "", NULL,
+ "CMK-Lookup", "", 64,
+ offsetof(struct pg_conn, cmklookup)},
+
/* Terminating entry --- MUST BE LAST */
{NULL, NULL, NULL, NULL,
NULL, NULL, 0}
@@ -4097,6 +4101,22 @@ freePGconn(PGconn *conn)
free(conn->krbsrvname);
free(conn->gsslib);
free(conn->connip);
+ free(conn->cmklookup);
+ for (int i = 0; i < conn->ncmks; i++)
+ {
+ free(conn->cmks[i].cmkname);
+ free(conn->cmks[i].cmkrealm);
+ }
+ free(conn->cmks);
+ for (int i = 0; i < conn->nceks; i++)
+ {
+ if (conn->ceks[i].cekdata)
+ {
+ explicit_bzero(conn->ceks[i].cekdata, conn->ceks[i].cekdatalen);
+ free(conn->ceks[i].cekdata);
+ }
+ }
+ free(conn->ceks);
/* Note that conn->Pfdebug is not ours to close or free */
free(conn->write_err_msg);
free(conn->inBuffer);
diff --git a/src/interfaces/libpq/fe-encrypt-openssl.c b/src/interfaces/libpq/fe-encrypt-openssl.c
new file mode 100644
index 0000000000..524ba8a0f9
--- /dev/null
+++ b/src/interfaces/libpq/fe-encrypt-openssl.c
@@ -0,0 +1,758 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-encrypt-openssl.c
+ * encryption support using OpenSSL
+ *
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-encrypt-openssl.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "fe-encrypt.h"
+#include "libpq-int.h"
+
+#include "catalog/pg_colenckey.h"
+#include "port/pg_bswap.h"
+
+#include <openssl/evp.h>
+
+
+#ifdef TEST_ENCRYPT
+
+/*
+ * Test data from
+ * <https://datatracker.ietf.org/doc/html/draft-mcgrew-aead-aes-cbc-hmac-sha2-05#section-5>
+ */
+
+/*
+ * The different test cases just use different prefixes of K, so one constant
+ * is enough here.
+ */
+static const unsigned char K[] = {
+ 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
+ 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f,
+ 0x20, 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28, 0x29, 0x2a, 0x2b, 0x2c, 0x2d, 0x2e, 0x2f,
+ 0x30, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x3a, 0x3b, 0x3c, 0x3d, 0x3e, 0x3f,
+};
+
+static const unsigned char P[] = {
+ 0x41, 0x20, 0x63, 0x69, 0x70, 0x68, 0x65, 0x72, 0x20, 0x73, 0x79, 0x73, 0x74, 0x65, 0x6d, 0x20,
+ 0x6d, 0x75, 0x73, 0x74, 0x20, 0x6e, 0x6f, 0x74, 0x20, 0x62, 0x65, 0x20, 0x72, 0x65, 0x71, 0x75,
+ 0x69, 0x72, 0x65, 0x64, 0x20, 0x74, 0x6f, 0x20, 0x62, 0x65, 0x20, 0x73, 0x65, 0x63, 0x72, 0x65,
+ 0x74, 0x2c, 0x20, 0x61, 0x6e, 0x64, 0x20, 0x69, 0x74, 0x20, 0x6d, 0x75, 0x73, 0x74, 0x20, 0x62,
+ 0x65, 0x20, 0x61, 0x62, 0x6c, 0x65, 0x20, 0x74, 0x6f, 0x20, 0x66, 0x61, 0x6c, 0x6c, 0x20, 0x69,
+ 0x6e, 0x74, 0x6f, 0x20, 0x74, 0x68, 0x65, 0x20, 0x68, 0x61, 0x6e, 0x64, 0x73, 0x20, 0x6f, 0x66,
+ 0x20, 0x74, 0x68, 0x65, 0x20, 0x65, 0x6e, 0x65, 0x6d, 0x79, 0x20, 0x77, 0x69, 0x74, 0x68, 0x6f,
+ 0x75, 0x74, 0x20, 0x69, 0x6e, 0x63, 0x6f, 0x6e, 0x76, 0x65, 0x6e, 0x69, 0x65, 0x6e, 0x63, 0x65,
+};
+
+static const unsigned char test_IV[] = {
+ 0x1a, 0xf3, 0x8c, 0x2d, 0xc2, 0xb9, 0x6f, 0xfd, 0xd8, 0x66, 0x94, 0x09, 0x23, 0x41, 0xbc, 0x04,
+};
+
+static const unsigned char test_A[] = {
+ 0x54, 0x68, 0x65, 0x20, 0x73, 0x65, 0x63, 0x6f, 0x6e, 0x64, 0x20, 0x70, 0x72, 0x69, 0x6e, 0x63,
+ 0x69, 0x70, 0x6c, 0x65, 0x20, 0x6f, 0x66, 0x20, 0x41, 0x75, 0x67, 0x75, 0x73, 0x74, 0x65, 0x20,
+ 0x4b, 0x65, 0x72, 0x63, 0x6b, 0x68, 0x6f, 0x66, 0x66, 0x73,
+};
+
+
+#define libpq_gettext(x) (x)
+
+#endif /* TEST_ENCRYPT */
+
+
+unsigned char *
+decrypt_cek_from_file(PGconn *conn, FILE *fp, int cmkalg,
+ int fromlen, const unsigned char *from,
+ int *tolen)
+{
+ const EVP_MD *md = NULL;
+ EVP_PKEY *key = NULL;
+ RSA *rsa = NULL;
+ EVP_PKEY_CTX *ctx = NULL;
+ unsigned char *out = NULL;
+ size_t outlen;
+
+ switch (cmkalg)
+ {
+ case PG_CMK_RSAES_OAEP_SHA_1:
+ md = EVP_sha1();
+ break;
+ case PG_CMK_RSAES_OAEP_SHA_256:
+ md = EVP_sha256();
+ break;
+ default:
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("unsupported CMK algorithm ID: %d\n"), cmkalg);
+ goto fail;
+ }
+
+ rsa = RSA_new();
+ if (!rsa)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not allocate RSA structure: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+ rsa = PEM_read_RSAPrivateKey(fp, &rsa, NULL, NULL);
+ if (!rsa)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not read RSA private key: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+
+ key = EVP_PKEY_new();
+ if (!key)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not allocate private key structure: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+
+ if (!EVP_PKEY_assign_RSA(key, rsa))
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not assign private key: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+
+ ctx = EVP_PKEY_CTX_new(key, NULL);
+ if (!ctx)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not allocate public key algorithm context: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+
+ if (EVP_PKEY_decrypt_init(ctx) <= 0)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("decryption initialization failed: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+
+ if (EVP_PKEY_CTX_set_rsa_padding(ctx, RSA_PKCS1_OAEP_PADDING) <= 0 ||
+ EVP_PKEY_CTX_set_rsa_mgf1_md(ctx, md) <= 0 ||
+ EVP_PKEY_CTX_set_rsa_oaep_md(ctx, md) <= 0)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not set RSA parameter: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+
+ if (EVP_PKEY_decrypt(ctx, NULL, &outlen, from, fromlen) <= 0)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("RSA decryption failed: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+
+ out = malloc(outlen);
+ if (!out)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("out of memory\n"));
+ goto fail;
+ }
+
+ if (EVP_PKEY_decrypt(ctx, out, &outlen, from, fromlen) <= 0)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("RSA decryption failed: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ free(out);
+ out = NULL;
+ goto fail;
+ }
+
+ *tolen = outlen;
+
+fail:
+ EVP_PKEY_CTX_free(ctx);
+ EVP_PKEY_free(key);
+
+ return out;
+}
+
+static const EVP_CIPHER *
+pg_cekalg_to_openssl_cipher(int cekalg)
+{
+ const EVP_CIPHER *cipher;
+
+ switch (cekalg)
+ {
+ case PG_CEK_AEAD_AES_128_CBC_HMAC_SHA_256:
+ cipher = EVP_aes_128_cbc();
+ break;
+ case PG_CEK_AEAD_AES_192_CBC_HMAC_SHA_384:
+ cipher = EVP_aes_192_cbc();
+ break;
+ case PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_384:
+ case PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_512:
+ cipher = EVP_aes_256_cbc();
+ break;
+ default:
+ cipher = NULL;
+ }
+
+ return cipher;
+}
+
+static const EVP_MD *
+pg_cekalg_to_openssl_md(int cekalg)
+{
+ const EVP_MD *md;
+
+ switch (cekalg)
+ {
+ case PG_CEK_AEAD_AES_128_CBC_HMAC_SHA_256:
+ md = EVP_sha256();
+ break;
+ case PG_CEK_AEAD_AES_192_CBC_HMAC_SHA_384:
+ case PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_384:
+ md = EVP_sha384();
+ break;
+ case PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_512:
+ md = EVP_sha512();
+ break;
+ default:
+ md = NULL;
+ }
+
+ return md;
+}
+
+static int
+md_key_length(const EVP_MD *md)
+{
+ if (md == EVP_sha256())
+ return 16;
+ else if (md == EVP_sha384())
+ return 24;
+ else if (md == EVP_sha512())
+ return 32;
+ else
+ return -1;
+}
+
+static int
+md_hash_length(const EVP_MD *md)
+{
+ if (md == EVP_sha256())
+ return 32;
+ else if (md == EVP_sha384())
+ return 48;
+ else if (md == EVP_sha512())
+ return 64;
+ else
+ return -1;
+}
+
+#ifndef TEST_ENCRYPT
+#define PG_AD_LEN 4
+#else
+#define PG_AD_LEN sizeof(test_A)
+#endif
+
+static bool
+get_message_auth_tag(const EVP_MD *md,
+ const unsigned char *mac_key, int mac_key_len,
+ const unsigned char *encr, int encrlen,
+ int cekalg,
+ unsigned char *md_value, size_t *md_len_p,
+ const char **errmsgp)
+{
+ static char msgbuf[1024];
+ EVP_MD_CTX *evp_md_ctx = NULL;
+ EVP_PKEY *pkey = NULL;
+ size_t bufsize;
+ unsigned char *buf = NULL;
+ bool result = false;
+
+ evp_md_ctx = EVP_MD_CTX_new();
+
+ pkey = EVP_PKEY_new_raw_private_key(EVP_PKEY_HMAC, NULL, mac_key, mac_key_len);
+ if (!pkey)
+ {
+ snprintf(msgbuf, sizeof(msgbuf),
+ libpq_gettext("could not allocate key for HMAC: %s"),
+ ERR_reason_error_string(ERR_get_error()));
+ *errmsgp = msgbuf;
+ goto fail;
+ }
+
+ bufsize = PG_AD_LEN + encrlen + sizeof(int64);
+ buf = malloc(bufsize);
+ if (!buf)
+ {
+ *errmsgp = libpq_gettext("out out memory");
+ goto fail;
+ }
+#ifndef TEST_ENCRYPT
+ buf[0] = 'P';
+ buf[1] = 'G';
+ *(int16 *) (buf + 2) = pg_hton16(cekalg);
+#else
+ memcpy(buf, test_A, sizeof(test_A));
+#endif
+ memcpy(buf + PG_AD_LEN, encr, encrlen);
+ *(int64 *) (buf + PG_AD_LEN + encrlen) = pg_hton64(PG_AD_LEN * 8);
+
+ if (!EVP_DigestSignInit(evp_md_ctx, NULL, md, NULL, pkey))
+ {
+ snprintf(msgbuf, sizeof(msgbuf),
+ libpq_gettext("digest initialization failed: %s"),
+ ERR_reason_error_string(ERR_get_error()));
+ *errmsgp = msgbuf;
+ goto fail;
+ }
+
+ if (!EVP_DigestSignUpdate(evp_md_ctx, buf, bufsize))
+ {
+ snprintf(msgbuf, sizeof(msgbuf),
+ libpq_gettext("digest signing failed: %s"),
+ ERR_reason_error_string(ERR_get_error()));
+ *errmsgp = msgbuf;
+ goto fail;
+ }
+
+ if (!EVP_DigestSignFinal(evp_md_ctx, md_value, md_len_p))
+ {
+ snprintf(msgbuf, sizeof(msgbuf),
+ libpq_gettext("digest signing failed: %s"),
+ ERR_reason_error_string(ERR_get_error()));
+ *errmsgp = msgbuf;
+ goto fail;
+ }
+ Assert(*md_len_p == md_hash_length(md));
+
+ /* truncate output to half the length, per spec */
+ *md_len_p /= 2;
+
+ result = true;
+fail:
+ free(buf);
+ EVP_PKEY_free(pkey);
+ EVP_MD_CTX_free(evp_md_ctx);
+ return result;
+}
+
+unsigned char *
+decrypt_value(PGresult *res, const PGCEK *cek, int cekalg, const unsigned char *input, int inputlen, const char **errmsgp)
+{
+ static char msgbuf[1024];
+
+ const unsigned char *iv = NULL;
+ size_t ivlen;
+
+ const EVP_CIPHER *cipher;
+ const EVP_MD *md;
+ EVP_CIPHER_CTX *evp_cipher_ctx = NULL;
+ int enc_key_len;
+ int mac_key_len;
+ int key_len;
+ const unsigned char *enc_key;
+ const unsigned char *mac_key;
+ unsigned char md_value[EVP_MAX_MD_SIZE];
+ size_t md_len = sizeof(md_value);
+ size_t bufsize;
+ unsigned char *buf = NULL;
+ unsigned char *decr;
+ int decrlen,
+ decrlen2;
+
+ unsigned char *result = NULL;
+
+ cipher = pg_cekalg_to_openssl_cipher(cekalg);
+ if (!cipher)
+ {
+ snprintf(msgbuf, sizeof(msgbuf),
+ libpq_gettext("unrecognized encryption algorithm identifier: %d"), cekalg);
+ *errmsgp = msgbuf;
+ goto fail;
+ }
+
+ md = pg_cekalg_to_openssl_md(cekalg);
+ if (!md)
+ {
+ snprintf(msgbuf, sizeof(msgbuf),
+ libpq_gettext("unrecognized digest algorithm identifier: %d"), cekalg);
+ *errmsgp = msgbuf;
+ goto fail;
+ }
+
+ evp_cipher_ctx = EVP_CIPHER_CTX_new();
+
+ if (!EVP_DecryptInit_ex(evp_cipher_ctx, cipher, NULL, NULL, NULL))
+ {
+ snprintf(msgbuf, sizeof(msgbuf),
+ libpq_gettext("decryption initialization failed: %s"),
+ ERR_reason_error_string(ERR_get_error()));
+ *errmsgp = msgbuf;
+ goto fail;
+ }
+
+ enc_key_len = EVP_CIPHER_CTX_key_length(evp_cipher_ctx);
+ mac_key_len = md_key_length(md);
+ key_len = mac_key_len + enc_key_len;
+
+ if (cek->cekdatalen != key_len)
+ {
+ snprintf(msgbuf, sizeof(msgbuf),
+ libpq_gettext("column encryption key has wrong length for algorithm (has: %zu, required: %d)"),
+ cek->cekdatalen, key_len);
+ *errmsgp = msgbuf;
+ goto fail;
+ }
+
+ enc_key = cek->cekdata + mac_key_len;
+ mac_key = cek->cekdata;
+
+ if (!get_message_auth_tag(md, mac_key, mac_key_len,
+ input, inputlen - (md_hash_length(md) / 2),
+ cekalg,
+ md_value, &md_len,
+ errmsgp))
+ {
+ goto fail;
+ }
+
+ if (memcmp(input + (inputlen - md_len), md_value, md_len) != 0)
+ {
+ *errmsgp = libpq_gettext("MAC mismatch");
+ goto fail;
+ }
+
+ ivlen = EVP_CIPHER_CTX_iv_length(evp_cipher_ctx);
+ iv = input;
+ input += ivlen;
+ inputlen -= ivlen;
+ if (!EVP_DecryptInit_ex(evp_cipher_ctx, NULL, NULL, enc_key, iv))
+ {
+ snprintf(msgbuf, sizeof(msgbuf),
+ libpq_gettext("decryption initialization failed: %s"),
+ ERR_reason_error_string(ERR_get_error()));
+ *errmsgp = msgbuf;
+ goto fail;
+ }
+
+ bufsize = inputlen + EVP_CIPHER_CTX_block_size(evp_cipher_ctx) + 1;
+#ifndef TEST_ENCRYPT
+ buf = pqResultAlloc(res, bufsize, false);
+#else
+ buf = malloc(bufsize);
+#endif
+ if (!buf)
+ {
+ *errmsgp = libpq_gettext("out out memory");
+ goto fail;
+ }
+ decr = buf;
+ if (!EVP_DecryptUpdate(evp_cipher_ctx, decr, &decrlen, input, inputlen - md_len))
+ {
+ snprintf(msgbuf, sizeof(msgbuf),
+ libpq_gettext("decryption failed: %s"),
+ ERR_reason_error_string(ERR_get_error()));
+ *errmsgp = msgbuf;
+ goto fail;
+ }
+ if (!EVP_DecryptFinal_ex(evp_cipher_ctx, decr + decrlen, &decrlen2))
+ {
+ snprintf(msgbuf, sizeof(msgbuf),
+ libpq_gettext("decryption failed: %s"),
+ ERR_reason_error_string(ERR_get_error()));
+ *errmsgp = msgbuf;
+ goto fail;
+ }
+ decrlen += decrlen2;
+ Assert(decrlen < bufsize);
+ decr[decrlen] = '\0';
+ result = decr;
+
+fail:
+ EVP_CIPHER_CTX_free(evp_cipher_ctx);
+
+ return result;
+}
+
+#ifndef TEST_ENCRYPT
+static bool
+make_siv(PGconn *conn,
+ unsigned char *iv, size_t ivlen,
+ const EVP_MD *md,
+ const unsigned char *iv_key, int iv_key_len,
+ const unsigned char *plaintext, int plaintext_len)
+{
+ EVP_MD_CTX *evp_md_ctx = NULL;
+ EVP_PKEY *pkey = NULL;
+ unsigned char md_value[EVP_MAX_MD_SIZE];
+ size_t md_len = sizeof(md_value);
+ bool result = false;
+
+ evp_md_ctx = EVP_MD_CTX_new();
+
+ pkey = EVP_PKEY_new_raw_private_key(EVP_PKEY_HMAC, NULL, iv_key, iv_key_len);
+ if (!pkey)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not allocate key for HMAC: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+
+ if (!EVP_DigestSignInit(evp_md_ctx, NULL, md, NULL, pkey))
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("digest initialization failed: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+
+ if (!EVP_DigestSignUpdate(evp_md_ctx, plaintext, plaintext_len))
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("digest signing failed: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+
+ if (!EVP_DigestSignFinal(evp_md_ctx, md_value, &md_len))
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("digest signing failed: %s"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+ Assert(md_len == md_hash_length(md));
+ memcpy(iv, md_value, ivlen);
+
+ result = true;
+fail:
+ EVP_PKEY_free(pkey);
+ EVP_MD_CTX_free(evp_md_ctx);
+ return result;
+}
+#endif /* TEST_ENCRYPT */
+
+unsigned char *
+encrypt_value(PGconn *conn, const PGCEK *cek, int cekalg, const unsigned char *value, int *nbytesp, bool enc_det)
+{
+ int nbytes = *nbytesp;
+ unsigned char iv[EVP_MAX_IV_LENGTH];
+ size_t ivlen;
+ const EVP_CIPHER *cipher;
+ const EVP_MD *md;
+ EVP_CIPHER_CTX *evp_cipher_ctx = NULL;
+ int enc_key_len;
+ int mac_key_len;
+ int key_len;
+ const unsigned char *enc_key;
+ const unsigned char *mac_key;
+ size_t bufsize;
+ unsigned char *buf = NULL;
+ unsigned char *encr;
+ int encrlen,
+ encrlen2;
+
+ const char *errmsg;
+ unsigned char md_value[EVP_MAX_MD_SIZE];
+ size_t md_len = sizeof(md_value);
+ size_t buf2size;
+ unsigned char *buf2 = NULL;
+
+ unsigned char *result = NULL;
+
+ cipher = pg_cekalg_to_openssl_cipher(cekalg);
+ if (!cipher)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("unrecognized encryption algorithm identifier: %d\n"), cekalg);
+ goto fail;
+ }
+
+ md = pg_cekalg_to_openssl_md(cekalg);
+ if (!md)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("unrecognized digest algorithm identifier: %d\n"), cekalg);
+ goto fail;
+ }
+
+ evp_cipher_ctx = EVP_CIPHER_CTX_new();
+
+ if (!EVP_EncryptInit_ex(evp_cipher_ctx, cipher, NULL, NULL, NULL))
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("encryption initialization failed: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+
+ enc_key_len = EVP_CIPHER_CTX_key_length(evp_cipher_ctx);
+ mac_key_len = md_key_length(md);
+ key_len = mac_key_len + enc_key_len;
+
+ if (cek->cekdatalen != key_len)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("column encryption key has wrong length for algorithm (has: %zu, required: %d)\n"),
+ cek->cekdatalen, key_len);
+ goto fail;
+ }
+
+ enc_key = cek->cekdata + mac_key_len;
+ mac_key = cek->cekdata;
+
+ ivlen = EVP_CIPHER_CTX_iv_length(evp_cipher_ctx);
+ Assert(ivlen <= sizeof(iv));
+ if (enc_det)
+ {
+#ifndef TEST_ENCRYPT
+ make_siv(conn, iv, ivlen, md, mac_key, mac_key_len, value, nbytes);
+#else
+ memcpy(iv, test_IV, ivlen);
+#endif
+ }
+ else
+ pg_strong_random(iv, ivlen);
+ if (!EVP_EncryptInit_ex(evp_cipher_ctx, NULL, NULL, enc_key, iv))
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("encryption initialization failed: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+
+ bufsize = ivlen + (nbytes + 2 * EVP_CIPHER_CTX_block_size(evp_cipher_ctx) - 1);
+ buf = malloc(bufsize);
+ if (!buf)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("out of memory\n"));
+ goto fail;
+ }
+ memcpy(buf, iv, ivlen);
+ encr = buf + ivlen;
+ if (!EVP_EncryptUpdate(evp_cipher_ctx, encr, &encrlen, value, nbytes))
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("encryption failed: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+ if (!EVP_EncryptFinal_ex(evp_cipher_ctx, encr + encrlen, &encrlen2))
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("encryption failed: %s\n"),
+ ERR_reason_error_string(ERR_get_error()));
+ goto fail;
+ }
+ encrlen += encrlen2;
+
+ encr -= ivlen;
+ encrlen += ivlen;
+
+ Assert(encrlen <= bufsize);
+
+ if (!get_message_auth_tag(md, mac_key, mac_key_len,
+ encr, encrlen,
+ cekalg,
+ md_value, &md_len,
+ &errmsg))
+ {
+ appendPQExpBuffer(&conn->errorMessage, "%s\n", errmsg);
+ goto fail;
+ }
+
+ buf2size = encrlen + md_len;
+ buf2 = malloc(buf2size);
+ if (!buf2)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("out of memory\n"));
+ goto fail;
+ }
+ memcpy(buf2, encr, encrlen);
+ memcpy(buf2 + encrlen, md_value, md_len);
+
+ result = buf2;
+ nbytes = buf2size;
+
+fail:
+ free(buf);
+ EVP_CIPHER_CTX_free(evp_cipher_ctx);
+
+ *nbytesp = nbytes;
+ return result;
+}
+
+
+/*
+ * Run test cases
+ */
+#ifdef TEST_ENCRYPT
+
+static void
+debug_print_hex(const char *name, const unsigned char *val, int len)
+{
+ printf("%s =", name);
+ for (int i = 0; i < len; i++)
+ {
+ if (i % 16 == 0)
+ printf("\n");
+ else
+ printf(" ");
+ printf("%02x", val[i]);
+ }
+ printf("\n");
+}
+
+static void
+test_case(int alg, const unsigned char *K, size_t K_len, const unsigned char *P, size_t P_len)
+{
+ unsigned char *C;
+ int nbytes;
+ PGCEK cek;
+
+ nbytes = P_len;
+ cek.cekdata = unconstify(unsigned char *, K);
+ cek.cekdatalen = K_len;
+
+ C = encrypt_value(NULL, &cek, alg, P, &nbytes, true);
+ debug_print_hex("C", C, nbytes);
+}
+
+int
+main(int argc, char **argv)
+{
+ printf("5.1\n");
+ test_case(PG_CEK_AEAD_AES_128_CBC_HMAC_SHA_256, K, 32, P, sizeof(P));
+ printf("5.2\n");
+ test_case(PG_CEK_AEAD_AES_192_CBC_HMAC_SHA_384, K, 48, P, sizeof(P));
+ printf("5.3\n");
+ test_case(PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_384, K, 56, P, sizeof(P));
+ printf("5.4\n");
+ test_case(PG_CEK_AEAD_AES_256_CBC_HMAC_SHA_512, K, 64, P, sizeof(P));
+
+ return 0;
+}
+
+#endif /* TEST_ENCRYPT */
diff --git a/src/interfaces/libpq/fe-encrypt.h b/src/interfaces/libpq/fe-encrypt.h
new file mode 100644
index 0000000000..b0093dc527
--- /dev/null
+++ b/src/interfaces/libpq/fe-encrypt.h
@@ -0,0 +1,33 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-encrypt.h
+ *
+ * encryption support
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-encrypt.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef FE_ENCRYPT_H
+#define FE_ENCRYPT_H
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+
+extern unsigned char *decrypt_cek_from_file(PGconn *conn, FILE *fp, int cmkalg,
+ int fromlen, const unsigned char *from,
+ int *tolen);
+
+extern unsigned char *decrypt_value(PGresult *res, const PGCEK *cek, int cekalg,
+ const unsigned char *input, int inputlen,
+ const char **errmsgp);
+
+extern unsigned char *encrypt_value(PGconn *conn, const PGCEK *cek, int cekalg,
+ const unsigned char *value, int *nbytesp, bool enc_det);
+
+#endif /* FE_ENCRYPT_H */
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index 51e9a362f6..2e3c0295eb 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -24,6 +24,9 @@
#include <unistd.h>
#endif
+#include "catalog/pg_colenckey.h"
+#include "common/base64.h"
+#include "fe-encrypt.h"
#include "libpq-fe.h"
#include "libpq-int.h"
#include "mb/pg_wchar.h"
@@ -72,7 +75,9 @@ static int PQsendQueryGuts(PGconn *conn,
const char *const *paramValues,
const int *paramLengths,
const int *paramFormats,
- int resultFormat);
+ const int *paramForceColumnEncryptions,
+ int resultFormat,
+ PGresult *paramDesc);
static void parseInput(PGconn *conn);
static PGresult *getCopyResult(PGconn *conn, ExecStatusType copytype);
static bool PQexecStart(PGconn *conn);
@@ -1185,6 +1190,381 @@ pqSaveParameterStatus(PGconn *conn, const char *name, const char *value)
}
}
+int
+pqSaveColumnMasterKey(PGconn *conn, int keyid, const char *keyname,
+ const char *keyrealm)
+{
+ char *keyname_copy;
+ char *keyrealm_copy;
+ bool found;
+
+ keyname_copy = strdup(keyname);
+ if (!keyname_copy)
+ return EOF;
+ keyrealm_copy = strdup(keyrealm);
+ if (!keyrealm_copy)
+ {
+ free(keyname_copy);
+ return EOF;
+ }
+
+ found = false;
+ for (int i = 0; i < conn->ncmks; i++)
+ {
+ struct pg_cmk *checkcmk = &conn->cmks[i];
+
+ /* replace existing? */
+ if (checkcmk->cmkid == keyid)
+ {
+ free(checkcmk->cmkname);
+ free(checkcmk->cmkrealm);
+ checkcmk->cmkname = keyname_copy;
+ checkcmk->cmkrealm = keyrealm_copy;
+ found = true;
+ break;
+ }
+ }
+
+ /* append new? */
+ if (!found)
+ {
+ int newncmks;
+ struct pg_cmk *newcmks;
+ struct pg_cmk *newcmk;
+
+ newncmks = conn->ncmks + 1;
+ if (newncmks <= 0)
+ return EOF;
+ newcmks = realloc(conn->cmks, newncmks * sizeof(struct pg_cmk));
+ if (!newcmks)
+ {
+ free(keyname_copy);
+ free(keyrealm_copy);
+ return EOF;
+ }
+
+ newcmk = &newcmks[newncmks - 1];
+ newcmk->cmkid = keyid;
+ newcmk->cmkname = keyname_copy;
+ newcmk->cmkrealm = keyrealm_copy;
+
+ conn->ncmks = newncmks;
+ conn->cmks = newcmks;
+ }
+
+ return 0;
+}
+
+/*
+ * Replace placeholders in input string. Return value malloc'ed.
+ */
+static char *
+replace_cmk_placeholders(const char *in, const char *cmkname, const char *cmkrealm, int cmkalg, const char *b64data)
+{
+ PQExpBufferData buf;
+
+ initPQExpBuffer(&buf);
+
+ for (const char *p = in; *p; p++)
+ {
+ if (p[0] == '%')
+ {
+ switch (p[1])
+ {
+ case 'a':
+ {
+ const char *s;
+
+ switch (cmkalg)
+ {
+ case PG_CMK_RSAES_OAEP_SHA_1:
+ s = "RSAES_OAEP_SHA_1";
+ break;
+ case PG_CMK_RSAES_OAEP_SHA_256:
+ s = "RSAES_OAEP_SHA_256";
+ break;
+ default:
+ s = "INVALID";
+ }
+ appendPQExpBufferStr(&buf, s);
+ }
+ p++;
+ break;
+ case 'b':
+ appendPQExpBufferStr(&buf, b64data);
+ p++;
+ break;
+ case 'k':
+ appendPQExpBufferStr(&buf, cmkname);
+ p++;
+ break;
+ case 'r':
+ appendPQExpBufferStr(&buf, cmkrealm);
+ p++;
+ break;
+ default:
+ appendPQExpBufferChar(&buf, p[0]);
+ }
+ }
+ else
+ appendPQExpBufferChar(&buf, p[0]);
+ }
+
+ return buf.data;
+}
+
+#ifndef USE_SSL
+/*
+ * Dummy implementation for non-SSL builds
+ */
+unsigned char *
+decrypt_cek_from_file(PGconn *conn, FILE *fp, int cmkalg,
+ int fromlen, const unsigned char *from,
+ int *tolen)
+{
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("column encryption not supported by this build\n"));
+ return NULL;
+}
+#endif
+
+/*
+ * Decrypt a CEK using the given CMK. The ciphertext is passed in
+ * "from" and "fromlen". Return the decrypted value in a malloc'ed area, its
+ * length via "tolen". Return NULL on error; add error messages directly to
+ * "conn".
+ */
+static unsigned char *
+decrypt_cek(PGconn *conn, const PGCMK *cmk, int cmkalg,
+ int fromlen, const unsigned char *from,
+ int *tolen)
+{
+ char *cmklookup;
+ bool found = false;
+ unsigned char *result = NULL;
+
+ cmklookup = strdup(conn->cmklookup ? conn->cmklookup : "");
+
+ if (!cmklookup)
+ {
+ appendPQExpBuffer(&conn->errorMessage, libpq_gettext("out of memory\n"));
+ return NULL;
+ }
+
+ /*
+ * Analyze semicolon-separated list
+ */
+ for (char *s = strtok(cmklookup, ";"); s; s = strtok(NULL, ";"))
+ {
+ char *sep;
+
+ /* split found token at '=' */
+ sep = strchr(s, '=');
+ if (!sep)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("syntax error in CMK lookup specification, missing \"%c\": %s\n"), '=', s);
+ break;
+ }
+
+ /* matching realm? */
+ if (strncmp(s, "*", sep - s) == 0 || strncmp(s, cmk->cmkrealm, sep - s) == 0)
+ {
+ char *sep2;
+
+ found = true;
+
+ sep2 = strchr(sep, ':');
+ if (!sep2)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("syntax error in CMK lookup specification, missing \"%c\": %s\n"), ':', s);
+ goto fail;
+ }
+
+ if (strncmp(sep + 1, "file", sep2 - (sep + 1)) == 0)
+ {
+ char *cmkfilename;
+ FILE *fp;
+
+ cmkfilename = replace_cmk_placeholders(sep2 + 1, cmk->cmkname, cmk->cmkrealm, cmkalg, "INVALID");
+ fp = fopen(cmkfilename, "rb");
+ if (fp)
+ {
+ result = decrypt_cek_from_file(conn, fp, cmkalg, fromlen, from, tolen);
+ fclose(fp);
+ }
+ else
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not open file \"%s\": %m\n"), cmkfilename);
+ free(cmkfilename);
+ goto fail;
+ }
+ free(cmkfilename);
+ }
+ else if (strncmp(sep + 1, "run", sep2 - (sep + 1)) == 0)
+ {
+ int enclen;
+ char *enc;
+ char *command;
+ FILE *fp;
+ char buf[4096];
+
+ enclen = pg_b64_enc_len(fromlen);
+ enc = malloc(enclen + 1);
+ if (!enc)
+ {
+ appendPQExpBuffer(&conn->errorMessage, libpq_gettext("out of memory\n"));
+ goto fail;
+ }
+ enclen = pg_b64_encode((const char *) from, fromlen, enc, enclen);
+ if (enclen < 0)
+ {
+ appendPQExpBuffer(&conn->errorMessage, libpq_gettext("base64 encoding failed\n"));
+ free(enc);
+ goto fail;
+ }
+ command = replace_cmk_placeholders(sep2 + 1, cmk->cmkname, cmk->cmkrealm, cmkalg, enc);
+ free(enc);
+ fp = popen(command, "r");
+ free(command);
+ if (!fp)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not run command \"%s\": %m\n"), command);
+ goto fail;
+ }
+ if (fgets(buf, sizeof(buf), fp))
+ {
+ int linelen;
+ int declen;
+ char *dec;
+
+ linelen = strlen(buf);
+ if (buf[linelen - 1] == '\n')
+ {
+ buf[linelen - 1] = '\0';
+ linelen--;
+ }
+ declen = pg_b64_dec_len(linelen);
+ dec = malloc(declen);
+ if (!dec)
+ {
+ appendPQExpBuffer(&conn->errorMessage, libpq_gettext("out of memory\n"));
+ goto fail;
+ }
+ declen = pg_b64_decode(buf, linelen, dec, declen);
+ if (declen < 0)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("base64 decoding failed\n"));
+ free(dec);
+ goto fail;
+ }
+ result = (unsigned char *) dec;
+ *tolen = declen;
+ }
+ else
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("could not read from command: %m\n"));
+ pclose(fp);
+ goto fail;
+ }
+ pclose(fp);
+ }
+ else
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("CMK lookup scheme \"%s\" not recognized\n"), sep + 1);
+ goto fail;
+ }
+ }
+ }
+
+ if (!found)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("no CMK lookup found for realm \"%s\"\n"), cmk->cmkrealm);
+ }
+
+fail:
+ free(cmklookup);
+ return result;
+}
+
+int
+pqSaveColumnEncryptionKey(PGconn *conn, int keyid, int cmkid, int cmkalg, const unsigned char *value, int len)
+{
+ PGCMK *cmk = NULL;
+ unsigned char *plainval = NULL;
+ int plainvallen = 0;
+ bool found;
+
+ for (int i = 0; i < conn->ncmks; i++)
+ {
+ if (conn->cmks[i].cmkid == cmkid)
+ {
+ cmk = &conn->cmks[i];
+ break;
+ }
+ }
+
+ if (!cmk)
+ return EOF;
+
+ plainval = decrypt_cek(conn, cmk, cmkalg, len, value, &plainvallen);
+ if (!plainval)
+ return EOF;
+
+ found = false;
+ for (int i = 0; i < conn->nceks; i++)
+ {
+ struct pg_cek *checkcek = &conn->ceks[i];
+
+ /* replace existing? */
+ if (checkcek->cekid == keyid)
+ {
+ free(checkcek->cekdata);
+ checkcek->cekdata = plainval;
+ checkcek->cekdatalen = plainvallen;
+ found = true;
+ break;
+ }
+ }
+
+ /* append new? */
+ if (!found)
+ {
+ int newnceks;
+ struct pg_cek *newceks;
+ struct pg_cek *newcek;
+
+ newnceks = conn->nceks + 1;
+ if (newnceks <= 0)
+ {
+ free(plainval);
+ return EOF;
+ }
+ newceks = realloc(conn->ceks, newnceks * sizeof(struct pg_cek));
+ if (!newceks)
+ {
+ free(plainval);
+ return EOF;
+ }
+
+ newcek = &newceks[newnceks - 1];
+ newcek->cekid = keyid;
+ newcek->cekdata = plainval;
+ newcek->cekdatalen = plainvallen;
+
+ conn->nceks = newnceks;
+ conn->ceks = newceks;
+ }
+
+ return 0;
+}
/*
* pqRowProcessor
@@ -1250,9 +1630,58 @@ pqRowProcessor(PGconn *conn, const char **errmsgp)
}
else
{
- bool isbinary = (res->attDescs[i].format != 0);
+ bool isbinary = ((res->attDescs[i].format & 0x0F) != 0);
char *val;
+ if (res->attDescs[i].cekid)
+ {
+#ifdef USE_SSL
+ PGCEK *cek = NULL;
+
+ for (int j = 0; j < conn->nceks; j++)
+ {
+ if (conn->ceks[j].cekid == res->attDescs[i].cekid)
+ {
+ cek = &conn->ceks[j];
+ break;
+ }
+ }
+ if (!cek)
+ {
+ *errmsgp = libpq_gettext("column encryption key not found");
+ goto fail;
+ }
+
+ if (isbinary)
+ {
+ val = (char *) decrypt_value(res, cek, res->attDescs[i].cekalg,
+ (const unsigned char *) columns[i].value, clen, errmsgp);
+ }
+ else
+ {
+ unsigned char *buf;
+ unsigned char *enccolval;
+ size_t enccolvallen;
+
+ buf = malloc(clen + 1);
+ memcpy(buf, columns[i].value, clen);
+ buf[clen] = '\0';
+ enccolval = PQunescapeBytea(buf, &enccolvallen);
+ val = (char *) decrypt_value(res, cek, res->attDescs[i].cekalg,
+ enccolval, enccolvallen, errmsgp);
+ free(enccolval);
+ free(buf);
+ }
+
+ if (val == NULL)
+ goto fail;
+#else
+ *errmsgp = libpq_gettext("column encryption not supported by this build");
+ goto fail;
+#endif
+ }
+ else
+ {
val = (char *) pqResultAlloc(res, clen + 1, isbinary);
if (val == NULL)
goto fail;
@@ -1260,6 +1689,7 @@ pqRowProcessor(PGconn *conn, const char **errmsgp)
/* copy and zero-terminate the data (even if it's binary) */
memcpy(val, columns[i].value, clen);
val[clen] = '\0';
+ }
tup[i].len = clen;
tup[i].value = val;
@@ -1564,7 +1994,9 @@ PQsendQueryParams(PGconn *conn,
paramValues,
paramLengths,
paramFormats,
- resultFormat);
+ NULL,
+ resultFormat,
+ NULL);
}
/*
@@ -1682,6 +2114,20 @@ PQsendQueryPrepared(PGconn *conn,
const int *paramLengths,
const int *paramFormats,
int resultFormat)
+{
+ return PQsendQueryPrepared2(conn, stmtName, nParams, paramValues, paramLengths, paramFormats, NULL, resultFormat, NULL);
+}
+
+int
+PQsendQueryPrepared2(PGconn *conn,
+ const char *stmtName,
+ int nParams,
+ const char *const *paramValues,
+ const int *paramLengths,
+ const int *paramFormats,
+ const int *paramForceColumnEncryptions,
+ int resultFormat,
+ PGresult *paramDesc)
{
if (!PQsendQueryStart(conn, true))
return 0;
@@ -1709,7 +2155,9 @@ PQsendQueryPrepared(PGconn *conn,
paramValues,
paramLengths,
paramFormats,
- resultFormat);
+ paramForceColumnEncryptions,
+ resultFormat,
+ paramDesc);
}
/*
@@ -1808,7 +2256,9 @@ PQsendQueryGuts(PGconn *conn,
const char *const *paramValues,
const int *paramLengths,
const int *paramFormats,
- int resultFormat)
+ const int *paramForceColumnEncryptions,
+ int resultFormat,
+ PGresult *paramDesc)
{
int i;
PGcmdQueueEntry *entry;
@@ -1855,14 +2305,53 @@ PQsendQueryGuts(PGconn *conn,
pqPuts(stmtName, conn) < 0)
goto sendFailed;
+ /* Check force column encryption */
+ if (nParams > 0 && paramForceColumnEncryptions)
+ {
+ for (i = 0; i < nParams; i++)
+ {
+ if (paramForceColumnEncryptions[i])
+ {
+ if (!(paramDesc &&
+ paramDesc->paramDescs &&
+ paramDesc->paramDescs[i].cekid))
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("parameter with forced encryption is not to be encrypted\n"));
+ goto sendFailed;
+ }
+ }
+ }
+ }
+
/* Send parameter formats */
- if (nParams > 0 && paramFormats)
+ if (nParams > 0 && (paramFormats || (paramDesc && paramDesc->paramDescs)))
{
if (pqPutInt(nParams, 2, conn) < 0)
goto sendFailed;
+
for (i = 0; i < nParams; i++)
{
- if (pqPutInt(paramFormats[i], 2, conn) < 0)
+ int format = paramFormats ? paramFormats[i] : 0;
+
+ if (paramDesc && paramDesc->paramDescs)
+ {
+ PGresParamDesc *pd = ¶mDesc->paramDescs[i];
+
+ if (pd->cekid)
+ {
+ if (format != 0)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("format must be text for encrypted parameter\n"));
+ goto sendFailed;
+ }
+ format = 1;
+ format |= 0x10;
+ }
+ }
+
+ if (pqPutInt(format, 2, conn) < 0)
goto sendFailed;
}
}
@@ -1881,6 +2370,7 @@ PQsendQueryGuts(PGconn *conn,
if (paramValues && paramValues[i])
{
int nbytes;
+ const char *paramValue;
if (paramFormats && paramFormats[i] != 0)
{
@@ -1899,9 +2389,54 @@ PQsendQueryGuts(PGconn *conn,
/* text parameter, do not use paramLengths */
nbytes = strlen(paramValues[i]);
}
+
+ paramValue = paramValues[i];
+
+ if (paramDesc && paramDesc->paramDescs && paramDesc->paramDescs[i].cekid)
+ {
+#ifdef USE_SSL
+ bool enc_det = (paramDesc->paramDescs[i].flags & 0x01) != 0;
+ PGCEK *cek = NULL;
+ char *enc_paramValue;
+ int enc_nbytes = nbytes;
+
+ for (int j = 0; j < conn->nceks; j++)
+ {
+ if (conn->ceks[j].cekid == paramDesc->paramDescs[i].cekid)
+ {
+ cek = &conn->ceks[j];
+ break;
+ }
+ }
+ if (!cek)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("column encryption key not found\n"));
+ goto sendFailed;
+ }
+
+ enc_paramValue = (char *) encrypt_value(conn, cek, paramDesc->paramDescs[i].cekalg,
+ (const unsigned char *) paramValue, &enc_nbytes, enc_det);
+ if (!enc_paramValue)
+ goto sendFailed;
+
+ if (pqPutInt(enc_nbytes, 4, conn) < 0 ||
+ pqPutnchar(enc_paramValue, enc_nbytes, conn) < 0)
+ goto sendFailed;
+
+ free(enc_paramValue);
+#else
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("column encryption not supported by this build\n"));
+ goto sendFailed;
+#endif
+ }
+ else
+ {
if (pqPutInt(nbytes, 4, conn) < 0 ||
- pqPutnchar(paramValues[i], nbytes, conn) < 0)
+ pqPutnchar(paramValue, nbytes, conn) < 0)
goto sendFailed;
+ }
}
else
{
@@ -2334,12 +2869,27 @@ PQexecPrepared(PGconn *conn,
const int *paramLengths,
const int *paramFormats,
int resultFormat)
+{
+ return PQexecPrepared2(conn, stmtName, nParams, paramValues, paramLengths, paramFormats, NULL, resultFormat, NULL);
+}
+
+PGresult *
+PQexecPrepared2(PGconn *conn,
+ const char *stmtName,
+ int nParams,
+ const char *const *paramValues,
+ const int *paramLengths,
+ const int *paramFormats,
+ const int *paramForceColumnEncryptions,
+ int resultFormat,
+ PGresult *paramDesc)
{
if (!PQexecStart(conn))
return NULL;
- if (!PQsendQueryPrepared(conn, stmtName,
- nParams, paramValues, paramLengths,
- paramFormats, resultFormat))
+ if (!PQsendQueryPrepared2(conn, stmtName,
+ nParams, paramValues, paramLengths,
+ paramFormats, paramForceColumnEncryptions,
+ resultFormat, paramDesc))
return NULL;
return PQexecFinish(conn);
}
@@ -3602,6 +4152,17 @@ PQfmod(const PGresult *res, int field_num)
return 0;
}
+int
+PQfisencrypted(const PGresult *res, int field_num)
+{
+ if (!check_field_number(res, field_num))
+ return false;
+ if (res->attDescs)
+ return (res->attDescs[field_num].cekid != 0);
+ else
+ return false;
+}
+
char *
PQcmdStatus(PGresult *res)
{
@@ -3787,6 +4348,17 @@ PQparamtype(const PGresult *res, int param_num)
return InvalidOid;
}
+int
+PQparamisencrypted(const PGresult *res, int param_num)
+{
+ if (!check_param_number(res, param_num))
+ return false;
+ if (res->paramDescs)
+ return (res->paramDescs[param_num].cekid != 0);
+ else
+ return false;
+}
+
/* PQsetnonblocking:
* sets the PGconn's database connection non-blocking if the arg is true
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index 10c76daf6e..9461c04bbe 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -45,6 +45,8 @@ static int getRowDescriptions(PGconn *conn, int msgLength);
static int getParamDescriptions(PGconn *conn, int msgLength);
static int getAnotherTuple(PGconn *conn, int msgLength);
static int getParameterStatus(PGconn *conn);
+static int getColumnMasterKey(PGconn *conn);
+static int getColumnEncryptionKey(PGconn *conn);
static int getNotify(PGconn *conn);
static int getCopyStart(PGconn *conn, ExecStatusType copytype);
static int getReadyForQuery(PGconn *conn);
@@ -315,6 +317,22 @@ pqParseInput3(PGconn *conn)
if (pqGetInt(&(conn->be_key), 4, conn))
return;
break;
+ case 'y': /* Column Master Key */
+ if (getColumnMasterKey(conn))
+ {
+ // TODO: review error handling here
+ pqSaveErrorResult(conn);
+ pqInternalNotice(&conn->noticeHooks, "%s", conn->errorMessage.data);
+ }
+ break;
+ case 'Y': /* Column Encryption Key */
+ if (getColumnEncryptionKey(conn))
+ {
+ // TODO: review error handling here
+ pqSaveErrorResult(conn);
+ pqInternalNotice(&conn->noticeHooks, "%s", conn->errorMessage.data);
+ }
+ break;
case 'T': /* Row Description */
if (conn->error_result ||
(conn->result != NULL &&
@@ -572,6 +590,8 @@ getRowDescriptions(PGconn *conn, int msgLength)
int typlen;
int atttypmod;
int format;
+ int cekid;
+ int cekalg;
if (pqGets(&conn->workBuffer, conn) ||
pqGetInt(&tableid, 4, conn) ||
@@ -579,7 +599,9 @@ getRowDescriptions(PGconn *conn, int msgLength)
pqGetInt(&typid, 4, conn) ||
pqGetInt(&typlen, 2, conn) ||
pqGetInt(&atttypmod, 4, conn) ||
- pqGetInt(&format, 2, conn))
+ pqGetInt(&format, 2, conn) ||
+ pqGetInt(&cekid, 4, conn) ||
+ pqGetInt(&cekalg, 2, conn))
{
/* We should not run out of data here, so complain */
errmsg = libpq_gettext("insufficient data in \"T\" message");
@@ -607,8 +629,10 @@ getRowDescriptions(PGconn *conn, int msgLength)
result->attDescs[i].typid = typid;
result->attDescs[i].typlen = typlen;
result->attDescs[i].atttypmod = atttypmod;
+ result->attDescs[i].cekid = cekid;
+ result->attDescs[i].cekalg = cekalg;
- if (format != 1)
+ if ((format & 0x0F) != 1)
result->binary = 0;
}
@@ -710,10 +734,22 @@ getParamDescriptions(PGconn *conn, int msgLength)
for (i = 0; i < nparams; i++)
{
int typid;
+ int cekid;
+ int cekalg;
+ int flags;
if (pqGetInt(&typid, 4, conn))
goto not_enough_data;
result->paramDescs[i].typid = typid;
+ if (pqGetInt(&cekid, 4, conn))
+ goto not_enough_data;
+ result->paramDescs[i].cekid = cekid;
+ if (pqGetInt(&cekalg, 2, conn))
+ goto not_enough_data;
+ result->paramDescs[i].cekalg = cekalg;
+ if (pqGetInt(&flags, 2, conn))
+ goto not_enough_data;
+ result->paramDescs[i].flags = flags;
}
/* Success! */
@@ -1439,6 +1475,89 @@ getParameterStatus(PGconn *conn)
return 0;
}
+/*
+ * Attempt to read a ColumnMasterKey message.
+ * Entry: 'y' message type and length have already been consumed.
+ * Exit: returns 0 if successfully consumed message.
+ * returns EOF if not enough data.
+ */
+static int
+getColumnMasterKey(PGconn *conn)
+{
+ int keyid;
+ char *keyname;
+ char *keyrealm;
+ int ret;
+
+ /* Get the key ID */
+ if (pqGetInt(&keyid, 4, conn) != 0)
+ return EOF;
+ /* Get the key name */
+ if (pqGets(&conn->workBuffer, conn) != 0)
+ return EOF;
+ keyname = strdup(conn->workBuffer.data);
+ if (!keyname)
+ return EOF;
+ /* Get the key realm */
+ if (pqGets(&conn->workBuffer, conn) != 0)
+ return EOF;
+ keyrealm = strdup(conn->workBuffer.data);
+ if (!keyrealm)
+ return EOF;
+ /* And save it */
+ ret = pqSaveColumnMasterKey(conn, keyid, keyname, keyrealm);
+
+ free(keyname);
+ free(keyrealm);
+
+ return ret;
+}
+
+/*
+ * Attempt to read a ColumnEncryptionKey message.
+ * Entry: 'Y' message type and length have already been consumed.
+ * Exit: returns 0 if successfully consumed message.
+ * returns EOF if not enough data.
+ */
+static int
+getColumnEncryptionKey(PGconn *conn)
+{
+ int keyid;
+ int cmkid;
+ int cmkalg;
+ char *buf;
+ int vallen;
+
+ /* Get the key ID */
+ if (pqGetInt(&keyid, 4, conn) != 0)
+ return EOF;
+ /* Get the CMK ID */
+ if (pqGetInt(&cmkid, 4, conn) != 0)
+ return EOF;
+ /* Get the CMK algorithm */
+ if (pqGetInt(&cmkalg, 2, conn) != 0)
+ return EOF;
+ /* Get the key data len */
+ if (pqGetInt(&vallen, 4, conn) != 0)
+ return EOF;
+ /* Get the key data */
+ buf = malloc(vallen);
+ if (!buf)
+ return EOF;
+ if (pqGetnchar(buf, vallen, conn) != 0)
+ {
+ free(buf);
+ return EOF;
+ }
+ /* And save it */
+ if (pqSaveColumnEncryptionKey(conn, keyid, cmkid, cmkalg, (unsigned char *) buf, vallen) != 0)
+ {
+ free(buf);
+ return EOF;
+ }
+ free(buf);
+ return 0;
+}
/*
* Attempt to read a Notify response message.
diff --git a/src/interfaces/libpq/fe-trace.c b/src/interfaces/libpq/fe-trace.c
index 5d68cf2eb3..e08cbcff33 100644
--- a/src/interfaces/libpq/fe-trace.c
+++ b/src/interfaces/libpq/fe-trace.c
@@ -458,7 +458,12 @@ pqTraceOutputt(FILE *f, const char *message, int *cursor, bool regress)
nfields = pqTraceOutputInt16(f, message, cursor);
for (int i = 0; i < nfields; i++)
+ {
+ pqTraceOutputInt32(f, message, cursor, regress);
pqTraceOutputInt32(f, message, cursor, regress);
+ pqTraceOutputInt16(f, message, cursor);
+ pqTraceOutputInt16(f, message, cursor);
+ }
}
/* RowDescription */
@@ -479,6 +484,8 @@ pqTraceOutputT(FILE *f, const char *message, int *cursor, bool regress)
pqTraceOutputInt16(f, message, cursor);
pqTraceOutputInt32(f, message, cursor, false);
pqTraceOutputInt16(f, message, cursor);
+ pqTraceOutputInt32(f, message, cursor, regress);
+ pqTraceOutputInt16(f, message, cursor);
}
}
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index 7986445f1a..42fbe0cd49 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -267,6 +267,8 @@ typedef struct pgresAttDesc
Oid typid; /* type id */
int typlen; /* type size */
int atttypmod; /* type-specific modifier info */
+ Oid cekid;
+ int cekalg;
} PGresAttDesc;
/* ----------------
@@ -438,6 +440,15 @@ extern PGresult *PQexecPrepared(PGconn *conn,
const int *paramLengths,
const int *paramFormats,
int resultFormat);
+extern PGresult *PQexecPrepared2(PGconn *conn,
+ const char *stmtName,
+ int nParams,
+ const char *const *paramValues,
+ const int *paramLengths,
+ const int *paramFormats,
+ const int *paramForceColumnEncryptions,
+ int resultFormat,
+ PGresult *paramDesc);
/* Interface for multiple-result or asynchronous queries */
#define PQ_QUERY_PARAM_MAX_LIMIT 65535
@@ -461,6 +472,15 @@ extern int PQsendQueryPrepared(PGconn *conn,
const int *paramLengths,
const int *paramFormats,
int resultFormat);
+extern int PQsendQueryPrepared2(PGconn *conn,
+ const char *stmtName,
+ int nParams,
+ const char *const *paramValues,
+ const int *paramLengths,
+ const int *paramFormats,
+ const int *paramForceColumnEncryptions,
+ int resultFormat,
+ PGresult *paramDesc);
extern int PQsetSingleRowMode(PGconn *conn);
extern PGresult *PQgetResult(PGconn *conn);
@@ -531,6 +551,7 @@ extern int PQfformat(const PGresult *res, int field_num);
extern Oid PQftype(const PGresult *res, int field_num);
extern int PQfsize(const PGresult *res, int field_num);
extern int PQfmod(const PGresult *res, int field_num);
+extern int PQfisencrypted(const PGresult *res, int field_num);
extern char *PQcmdStatus(PGresult *res);
extern char *PQoidStatus(const PGresult *res); /* old and ugly */
extern Oid PQoidValue(const PGresult *res); /* new and improved */
@@ -540,6 +561,7 @@ extern int PQgetlength(const PGresult *res, int tup_num, int field_num);
extern int PQgetisnull(const PGresult *res, int tup_num, int field_num);
extern int PQnparams(const PGresult *res);
extern Oid PQparamtype(const PGresult *res, int param_num);
+extern int PQparamisencrypted(const PGresult *res, int param_num);
/* Describe prepared statements and portals */
extern PGresult *PQdescribePrepared(PGconn *conn, const char *stmt);
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 3db6a17db4..0a4c012aa6 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -111,6 +111,9 @@ union pgresult_data
typedef struct pgresParamDesc
{
Oid typid; /* type id */
+ Oid cekid;
+ int cekalg;
+ int flags;
} PGresParamDesc;
/*
@@ -340,6 +343,26 @@ typedef struct pg_conn_host
* found in password file. */
} pg_conn_host;
+/*
+ * Column encryption support data
+ */
+
+/* column master key */
+typedef struct pg_cmk
+{
+ Oid cmkid;
+ char *cmkname;
+ char *cmkrealm;
+} PGCMK;
+
+/* column encryption key */
+typedef struct pg_cek
+{
+ Oid cekid;
+ unsigned char *cekdata; /* (decrypted) */
+ size_t cekdatalen;
+} PGCEK;
+
/*
* PGconn stores all the state data associated with a single connection
* to a backend.
@@ -393,6 +416,7 @@ struct pg_conn
char *ssl_min_protocol_version; /* minimum TLS protocol version */
char *ssl_max_protocol_version; /* maximum TLS protocol version */
char *target_session_attrs; /* desired session properties */
+ char *cmklookup; /* CMK lookup specification */
/* Optional file to write trace info to */
FILE *Pfdebug;
@@ -475,6 +499,12 @@ struct pg_conn
PGContextVisibility show_context; /* whether to show CONTEXT field */
PGlobjfuncs *lobjfuncs; /* private state for large-object access fns */
+ /* Column encryption support data */
+ int ncmks;
+ PGCMK *cmks;
+ int nceks;
+ PGCEK *ceks;
+
/* Buffer for data received from backend and not yet processed */
char *inBuffer; /* currently allocated buffer */
int inBufSize; /* allocated size of buffer */
@@ -670,6 +700,10 @@ extern void pqSaveMessageField(PGresult *res, char code,
const char *value);
extern void pqSaveParameterStatus(PGconn *conn, const char *name,
const char *value);
+extern int pqSaveColumnMasterKey(PGconn *conn, int keyid, const char *keyname,
+ const char *keyrealm);
+extern int pqSaveColumnEncryptionKey(PGconn *conn, int keyid, int cmkid, int cmkalg,
+ const unsigned char *value, int len);
extern int pqRowProcessor(PGconn *conn, const char **errmsgp);
extern void pqCommandQueueAdvance(PGconn *conn);
extern int PQsendQueryContinue(PGconn *conn, const char *query);
diff --git a/src/interfaces/libpq/nls.mk b/src/interfaces/libpq/nls.mk
index 1f62ba1b57..844e085e6d 100644
--- a/src/interfaces/libpq/nls.mk
+++ b/src/interfaces/libpq/nls.mk
@@ -1,6 +1,6 @@
# src/interfaces/libpq/nls.mk
CATALOG_NAME = libpq
AVAIL_LANGUAGES = cs de el es fr he it ja ko pl pt_BR ru sv tr uk zh_CN zh_TW
-GETTEXT_FILES = fe-auth.c fe-auth-scram.c fe-connect.c fe-exec.c fe-gssapi-common.c fe-lobj.c fe-misc.c fe-protocol3.c fe-secure.c fe-secure-common.c fe-secure-gssapi.c fe-secure-openssl.c win32.c ../../port/thread.c
+GETTEXT_FILES = fe-auth.c fe-auth-scram.c fe-connect.c fe-encrypt-openssl.c fe-exec.c fe-gssapi-common.c fe-lobj.c fe-misc.c fe-protocol3.c fe-secure.c fe-secure-common.c fe-secure-gssapi.c fe-secure-openssl.c win32.c ../../port/thread.c
GETTEXT_TRIGGERS = libpq_gettext pqInternalNotice:2
GETTEXT_FLAGS = libpq_gettext:1:pass-c-format pqInternalNotice:2:c-format
diff --git a/src/interfaces/libpq/test/.gitignore b/src/interfaces/libpq/test/.gitignore
index 6ba78adb67..1846594ec5 100644
--- a/src/interfaces/libpq/test/.gitignore
+++ b/src/interfaces/libpq/test/.gitignore
@@ -1,2 +1,3 @@
+/libpq_test_encrypt
/libpq_testclient
/libpq_uri_regress
diff --git a/src/interfaces/libpq/test/Makefile b/src/interfaces/libpq/test/Makefile
index 75ac08f943..b1ebab90d4 100644
--- a/src/interfaces/libpq/test/Makefile
+++ b/src/interfaces/libpq/test/Makefile
@@ -15,10 +15,17 @@ override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += $(libpq_pgport)
PROGS = libpq_testclient libpq_uri_regress
+ifeq ($(with_ssl),openssl)
+PROGS += libpq_test_encrypt
+endif
+
all: $(PROGS)
$(PROGS): $(WIN32RES)
+libpq_test_encrypt: ../fe-encrypt-openssl.c
+ $(CC) $(CPPFLAGS) -DTEST_ENCRYPT $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+
clean distclean maintainer-clean:
rm -f $(PROGS) *.o
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index 44457f930c..c5f3153094 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -9529,7 +9529,7 @@ DO $d$
END;
$d$;
ERROR: invalid option "password"
-HINT: Valid options in this context are: service, passfile, channel_binding, connect_timeout, dbname, host, hostaddr, port, options, application_name, keepalives, keepalives_idle, keepalives_interval, keepalives_count, tcp_user_timeout, sslmode, sslcompression, sslcert, sslkey, sslrootcert, sslcrl, sslcrldir, sslsni, requirepeer, ssl_min_protocol_version, ssl_max_protocol_version, gssencmode, krbsrvname, gsslib, target_session_attrs, use_remote_estimate, fdw_startup_cost, fdw_tuple_cost, extensions, updatable, truncatable, fetch_size, batch_size, async_capable, parallel_commit, keep_connections
+HINT: Valid options in this context are: service, passfile, channel_binding, connect_timeout, dbname, host, hostaddr, port, options, application_name, keepalives, keepalives_idle, keepalives_interval, keepalives_count, tcp_user_timeout, sslmode, sslcompression, sslcert, sslkey, sslrootcert, sslcrl, sslcrldir, sslsni, requirepeer, ssl_min_protocol_version, ssl_max_protocol_version, gssencmode, krbsrvname, gsslib, target_session_attrs, cmklookup, use_remote_estimate, fdw_startup_cost, fdw_tuple_cost, extensions, updatable, truncatable, fetch_size, batch_size, async_capable, parallel_commit, keep_connections
CONTEXT: SQL statement "ALTER SERVER loopback_nopw OPTIONS (ADD password 'dummypw')"
PL/pgSQL function inline_code_block line 3 at EXECUTE
-- If we add a password for our user mapping instead, we should get a different
base-commit: 6ffff0fd225432fe2ae4bd5abb7ff6113e255418
--
2.36.1
view thread (53+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected]
Subject: Re: Transparent column encryption
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox