pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feed"WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
37+ messages / 9 participants
[nested] [flat]
* "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2018-08-13 05:56 Toshi Harada <harada.toshi@po.ntt-tx.co.jp>
0 siblings, 1 reply; 37+ messages in thread
From: Toshi Harada @ 2018-08-13 05:56 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: pgsql-hackers
Hi.
If "data-at-rest-encryption-wip-2018.07.25.patch" is applied to PostgreSQL 11-beta3 released last week,
patch execution will fail as follows.
----
patching file src/backend/replication/logical/reorderbuffer.c
Hunk #9 FAILED at 2464.
1 out of 7 hunks FAILED -- saving rejects to file src/bin/pg_rewind/pg_rewind.c.rej
1 out of 6 hunks FAILED -- saving rejects to file src/bin/pg_upgrade/controldata.c.rej
----
(See the attached file for the entire patch log)
Antonin Houska <ah@cybertec.at> wrote:
> Toshi Harada <harada.toshi@po.ntt-tx.co.jp> wrote:
>
> > Hi.
> >
> > I applied the patch "WIP: Data at rest encryption" to PostgreSQL 11 - beta 2 and I'm working on it.
> >
> > When this patch is applied, the following problem occurs.
> >
> > * An error occurs when CHECKPOINT is executed during two-phase commit.
> > * After an error occurs, if you stop PostgreSQL, it will never start again.
> >
> > (1) First, execute PREPARE TRANSACTION.
> >
> > postgres=# BEGIN;
> > BEGIN
> > postgres=# PREPARE TRANSACTION 'foo';
> > PREPARE TRANSACTION
> > postgres=#
> >
> > (2) Execute the CHECKPOINT command from another terminal.
> > CHEKPOINT command fails.
> >
> > postgres=# CHECKPOINT;
> > ERROR: checkpoint request failed
> > HINT: Consult recent messages in the server log for details.
> > postgres=#
>
> The patch version I posted in
>
> https://www.postgresql.org/message-id/11678.1532519255%40localhost
>
> fixes an issue (unitialized pointer) that caused failure here, but it was
> SEGFAULT rather than ERROR. And the scope of the bug was broader than just
> CHECKPOINT.
>
> Can you please test it again with the new version of the patch?
>
> Anyway, thanks for your reports!
>
>
> --
> Antonin Houska
> Cybertec Schonig & Schonig GmbH
> Grohrmuhlgasse 26, A-2700 Wiener Neustadt
> Web: https://www.cybertec-postgresql.com
>
Attachments:
PG11-beta3+0725patch.log (application/octet-stream, 8.1K) download
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2018-08-13 06:06 Antonin Houska <ah@cybertec.at>
parent: Toshi Harada <harada.toshi@po.ntt-tx.co.jp>
0 siblings, 1 reply; 37+ messages in thread
From: Antonin Houska @ 2018-08-13 06:06 UTC (permalink / raw)
To: Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; +Cc: pgsql-hackers
Toshi Harada <harada.toshi@po.ntt-tx.co.jp> wrote:
> Hi.
>
> If "data-at-rest-encryption-wip-2018.07.25.patch" is applied to PostgreSQL 11-beta3 released last week,
> patch execution will fail as follows.
> ----
> patching file src/backend/replication/logical/reorderbuffer.c
> Hunk #9 FAILED at 2464.
>
> 1 out of 7 hunks FAILED -- saving rejects to file src/bin/pg_rewind/pg_rewind.c.rej
>
> 1 out of 6 hunks FAILED -- saving rejects to file src/bin/pg_upgrade/controldata.c.rej
The patch should be applicable to the master branch.
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2018-08-15 02:09 Toshi Harada <harada.toshi@po.ntt-tx.co.jp>
parent: Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 37+ messages in thread
From: Toshi Harada @ 2018-08-15 02:09 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: pgsql-hackers
Hi.
Even if you apply "data-at-rest-encryption-wip-2018.07.25.patch" to the master branch,
the same patch error will occur.
(Patch error of pg_rewind was eliminated)
----
patching file src/backend/replication/logical/reorderbuffer.c
Hunk #9 FAILED at 2464.
patching file src/bin/pg_upgrade/controldata.c
Hunk #1 FAILED at 58.
----
(See the attached file for the entire patch log)
Antonin Houska <ah@cybertec.at> wrote:
> Toshi Harada <harada.toshi@po.ntt-tx.co.jp> wrote:
>
> > Hi.
> >
> > If "data-at-rest-encryption-wip-2018.07.25.patch" is applied to PostgreSQL 11-beta3 released last week,
> > patch execution will fail as follows.
>
> > ----
> > patching file src/backend/replication/logical/reorderbuffer.c
> > Hunk #9 FAILED at 2464.
> >
> > 1 out of 7 hunks FAILED -- saving rejects to file src/bin/pg_rewind/pg_rewind.c.rej
> >
> > 1 out of 6 hunks FAILED -- saving rejects to file src/bin/pg_upgrade/controldata.c.rej
>
> The patch should be applicable to the master branch.
>
> --
> Antonin Houska
> Cybertec Schonig & Schonig GmbH
> Grohrmuhlgasse 26, A-2700 Wiener Neustadt
> Web: https://www.cybertec-postgresql.com
>
Attachments:
master+patch.log (application/octet-stream, 5.7K) download
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2018-09-14 14:05 Antonin Houska <ah@cybertec.at>
parent: Toshi Harada <harada.toshi@po.ntt-tx.co.jp>
0 siblings, 1 reply; 37+ messages in thread
From: Antonin Houska @ 2018-09-14 14:05 UTC (permalink / raw)
To: Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; +Cc: pgsql-hackers
Toshi Harada <harada.toshi@po.ntt-tx.co.jp> wrote:
> Even if you apply "data-at-rest-encryption-wip-2018.07.25.patch" to the master branch,
> the same patch error will occur.
The version attached to this email should be applicable to the current
branch. Sorry for the delay.
--
Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Web: https://www.cybertec-postgresql.com
Attachments:
data-at-rest-encryption-wip-2018.09.14.patch (text/x-diff, 206.6K) download
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 3025d0badb..49398d2250 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -54,6 +54,7 @@
#include "replication/walreceiver.h"
#include "replication/walsender.h"
#include "storage/bufmgr.h"
+#include "storage/encryption.h"
#include "storage/fd.h"
#include "storage/ipc.h"
#include "storage/large_object.h"
@@ -77,6 +78,7 @@
#include "pg_trace.h"
extern uint32 bootstrap_data_checksum_version;
+extern char *bootstrap_encryption_sample;
/* File path names (all relative to $PGDATA) */
#define RECOVERY_COMMAND_FILE "recovery.conf"
@@ -868,6 +870,7 @@ static void LocalSetXLogInsertAllowed(void);
static void CreateEndOfRecoveryRecord(void);
static void CheckPointGuts(XLogRecPtr checkPointRedo, int flags);
static void KeepLogSeg(XLogRecPtr recptr, XLogSegNo *logSegNo);
+static void XLogWritePages(char *from, int npages);
static XLogRecPtr XLogGetReplicationSlotMinimumLSN(void);
static void AdvanceXLInsertBuffer(XLogRecPtr upto, bool opportunistic);
@@ -1247,7 +1250,7 @@ ReserveXLogInsertLocation(int size, XLogRecPtr *StartPos, XLogRecPtr *EndPos,
uint64 endbytepos;
uint64 prevbytepos;
- size = MAXALIGN(size);
+ size = XLOG_REC_ALIGN(size);
/* All (non xlog-switch) records should contain data. */
Assert(size > SizeOfXLogRecord);
@@ -1301,7 +1304,7 @@ ReserveXLogSwitch(XLogRecPtr *StartPos, XLogRecPtr *EndPos, XLogRecPtr *PrevPtr)
uint64 startbytepos;
uint64 endbytepos;
uint64 prevbytepos;
- uint32 size = MAXALIGN(SizeOfXLogRecord);
+ uint32 size = XLOG_REC_ALIGN(SizeOfXLogRecord);
XLogRecPtr ptr;
uint32 segleft;
@@ -1597,7 +1600,7 @@ CopyXLogRecordToWAL(int write_len, bool isLogSwitch, XLogRecData *rdata,
else
{
/* Align the end position, so that the next record starts aligned */
- CurrPos = MAXALIGN64(CurrPos);
+ CurrPos = XLOG_REC_ALIGN(CurrPos);
}
if (CurrPos != EndPos)
@@ -2481,8 +2484,6 @@ XLogWrite(XLogwrtRqst WriteRqst, bool flexible)
{
char *from;
Size nbytes;
- Size nleft;
- int written;
/* Need to seek in the file? */
if (openLogOff != startoffset)
@@ -2499,30 +2500,40 @@ XLogWrite(XLogwrtRqst WriteRqst, bool flexible)
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
nbytes = npages * (Size) XLOG_BLCKSZ;
- nleft = nbytes;
- do
+ if (data_encrypted)
{
- errno = 0;
- pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
- written = write(openLogFile, from, nleft);
- pgstat_report_wait_end();
- if (written <= 0)
+#ifdef USE_ENCRYPTION
+ int i;
+
+ /*
+ * XXX: use larger encryption buffer to enable larger writes
+ * and reduce number of syscalls?
+ */
+ for (i = 0; i < npages; i++)
{
- if (errno == EINTR)
- continue;
- ereport(PANIC,
- (errcode_for_file_access(),
- errmsg("could not write to log file %s "
- "at offset %u, length %zu: %m",
- XLogFileNameP(ThisTimeLineID, openLogSegNo),
- openLogOff, nbytes)));
- }
- nleft -= written;
- from += written;
- } while (nleft > 0);
+ char buf[XLOG_BLCKSZ];
+ char tweak[TWEAK_SIZE];
+
+ XLogEncryptionTweak(tweak, openLogSegNo, openLogOff);
+ encrypt_block(from, buf, XLOG_BLCKSZ, tweak);
+ pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
+ XLogWritePages(buf, 1);
+ pgstat_report_wait_end();
+
+ from += XLOG_BLCKSZ;
+ openLogOff += XLOG_BLCKSZ;
+ }
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
+ else
+ {
+ XLogWritePages(from, npages);
+ openLogOff += nbytes;
+ }
/* Update state for write */
- openLogOff += nbytes;
npages = 0;
/*
@@ -2638,6 +2649,32 @@ XLogWrite(XLogwrtRqst WriteRqst, bool flexible)
}
}
+static void
+XLogWritePages(char *from, int npages)
+{
+ Size nleft = npages * (Size) XLOG_BLCKSZ;
+ Size written;
+
+ do
+ {
+ errno = 0;
+ written = write(openLogFile, from, nleft);
+ if (written <= 0)
+ {
+ if (errno == EINTR)
+ continue;
+ ereport(PANIC,
+ (errcode_for_file_access(),
+ errmsg("could not write to log file %s "
+ "at offset %u, length %zu: %m",
+ XLogFileNameP(ThisTimeLineID, openLogSegNo),
+ openLogOff, npages * (Size) XLOG_BLCKSZ)));
+ }
+ nleft -= written;
+ from += written;
+ } while (nleft > 0);
+}
+
/*
* Record the LSN for an asynchronous transaction commit/abort
* and nudge the WALWriter if there is work for it to do.
@@ -4756,6 +4793,31 @@ ReadControlFile(void)
ereport(ERROR, (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("\"max_wal_size\" must be at least twice \"wal_segment_size\".")));
+ /*
+ * Initialize encryption, but not if the current backend has already done
+ * that.
+ */
+ if (ControlFile->data_encrypted && !data_encrypted)
+ {
+ char sample[ENCRYPTION_SAMPLE_SIZE];
+
+ setup_encryption(false);
+
+ memset(sample, 0, ENCRYPTION_SAMPLE_SIZE);
+ sample_encryption(sample);
+ if (memcmp(ControlFile->encryption_verification, sample, ENCRYPTION_SAMPLE_SIZE))
+ ereport(FATAL,
+ (errmsg("invalid encryption key"),
+ errdetail("The passed encryption key does not match"
+ " database encryption key.")));
+ }
+ SetConfigOption("data_encryption", DataEncryptionEnabled() ? "yes" : "no",
+ PGC_INTERNAL, PGC_S_OVERRIDE);
+
+ /*
+ * This calculation relies on data_encryption (in particular the header
+ * sizes do), so we could not do it earlier.
+ */
UsableBytesInSegment =
(wal_segment_size / XLOG_BLCKSZ * UsableBytesInPage) -
(SizeOfXLogLongPHD - SizeOfXLogShortPHD);
@@ -4845,6 +4907,16 @@ DataChecksumsEnabled(void)
}
/*
+ * Is this cluster encrypted?
+ */
+bool
+DataEncryptionEnabled(void)
+{
+ Assert(ControlFile != NULL);
+ return (ControlFile->data_encrypted);
+}
+
+/*
* Returns a fake LSN for unlogged relations.
*
* Each call generates an LSN that is greater than any previous value
@@ -5223,6 +5295,18 @@ BootStrapXLOG(void)
use_existent = false;
openLogFile = XLogFileInit(1, &use_existent, false);
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ char tweak[TWEAK_SIZE];
+
+ XLogEncryptionTweak(tweak, 1, 0);
+ encrypt_block((char *) page, (char *) page, XLOG_BLCKSZ, tweak);
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
+
/* Write the first page with the initial record */
errno = 0;
pgstat_report_wait_start(WAIT_EVENT_WAL_BOOTSTRAP_WRITE);
@@ -5272,6 +5356,22 @@ BootStrapXLOG(void)
ControlFile->wal_log_hints = wal_log_hints;
ControlFile->track_commit_timestamp = track_commit_timestamp;
ControlFile->data_checksum_version = bootstrap_data_checksum_version;
+ ControlFile->data_encrypted = data_encrypted;
+
+ if (data_encrypted)
+ {
+ char *sample;
+
+ sample = palloc0(ENCRYPTION_SAMPLE_SIZE);
+ sample_encryption(sample);
+
+ memcpy(ControlFile->encryption_verification, sample,
+ ENCRYPTION_SAMPLE_SIZE);
+ pfree(sample);
+ }
+ else
+ memset(ControlFile->encryption_verification, 0,
+ ENCRYPTION_SAMPLE_SIZE);
/* some additional ControlFile fields are set in WriteControlFile() */
@@ -11862,6 +11962,18 @@ retry:
Assert(targetPageOff == readOff);
Assert(reqLen <= readLen);
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ char tweak[TWEAK_SIZE];
+
+ XLogEncryptionTweak(tweak, readSegNo, readOff);
+ decrypt_block(readBuf, readBuf, XLOG_BLCKSZ, tweak);
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
+
*readTLI = curFileTLI;
/*
diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c
index 34d4db4297..2e190d5074 100644
--- a/src/backend/access/transam/xloginsert.c
+++ b/src/backend/access/transam/xloginsert.c
@@ -28,6 +28,7 @@
#include "miscadmin.h"
#include "replication/origin.h"
#include "storage/bufmgr.h"
+#include "storage/encryption.h"
#include "storage/proc.h"
#include "utils/memutils.h"
#include "pg_trace.h"
diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c
index 0768ca7822..08fc1eec2b 100644
--- a/src/backend/access/transam/xlogreader.c
+++ b/src/backend/access/transam/xlogreader.c
@@ -24,6 +24,7 @@
#include "catalog/pg_control.h"
#include "common/pg_lzcompress.h"
#include "replication/origin.h"
+#include "storage/encryption.h"
#ifndef FRONTEND
#include "utils/memutils.h"
@@ -317,9 +318,9 @@ XLogReadRecord(XLogReaderState *state, XLogRecPtr RecPtr, char **errormsg)
*
* NB: Even though we use an XLogRecord pointer here, the whole record
* header might not fit on this page. xl_tot_len is the first field of the
- * struct, so it must be on this page (the records are MAXALIGNed), but we
- * cannot access any other fields until we've verified that we got the
- * whole header.
+ * struct, so it must be on this page (the records are XLOG_REC_ALIGNed)
+ * but we cannot access any other fields until we've verified that we got
+ * the whole header.
*/
record = (XLogRecord *) (state->readBuf + RecPtr % XLOG_BLCKSZ);
total_len = record->xl_tot_len;
@@ -461,8 +462,8 @@ XLogReadRecord(XLogReaderState *state, XLogRecPtr RecPtr, char **errormsg)
pageHeaderSize = XLogPageHeaderSize((XLogPageHeader) state->readBuf);
state->ReadRecPtr = RecPtr;
- state->EndRecPtr = targetPagePtr + pageHeaderSize
- + MAXALIGN(pageHeader->xlp_rem_len);
+ state->EndRecPtr = targetPagePtr + pageHeaderSize;
+ state->EndRecPtr += XLOG_REC_ALIGN(pageHeader->xlp_rem_len);
}
else
{
@@ -476,7 +477,7 @@ XLogReadRecord(XLogReaderState *state, XLogRecPtr RecPtr, char **errormsg)
if (!ValidXLogRecord(state, record, RecPtr))
goto err;
- state->EndRecPtr = RecPtr + MAXALIGN(total_len);
+ state->EndRecPtr = RecPtr + XLOG_REC_ALIGN(total_len);
state->ReadRecPtr = RecPtr;
memcpy(state->readRecordBuf, record, total_len);
@@ -532,6 +533,15 @@ ReadPageInternal(XLogReaderState *state, XLogRecPtr pageptr, int reqLen)
Assert((pageptr % XLOG_BLCKSZ) == 0);
+ /*
+ * Make sure that we only return data that can be decrypted in a sensible
+ * way. If the valid data ended in the middle of encryption block, then
+ * decryption of that last block would turn the contained data into
+ * garbage.
+ */
+ if (data_encrypted)
+ reqLen = XLOG_REC_ALIGN(reqLen);
+
XLByteToSeg(pageptr, targetSegNo, state->wal_segment_size);
targetPageOff = XLogSegmentOffset(pageptr, state->wal_segment_size);
@@ -954,9 +964,10 @@ XLogFindNextRecord(XLogReaderState *state, XLogRecPtr RecPtr)
* next page header will contain the remaining length of the
* continuation data
*
- * Note that record headers are MAXALIGN'ed
+ * Note that record headers are XLOG_REC_ALIGN'ed
*/
- if (MAXALIGN(header->xlp_rem_len) > (XLOG_BLCKSZ - pageHeaderSize))
+ if (XLOG_REC_ALIGN(header->xlp_rem_len) >
+ (XLOG_BLCKSZ - pageHeaderSize))
tmpRecPtr = targetPagePtr + XLOG_BLCKSZ;
else
{
@@ -965,7 +976,7 @@ XLogFindNextRecord(XLogReaderState *state, XLogRecPtr RecPtr)
* tmpRecPtr to point to the first valid record
*/
tmpRecPtr = targetPagePtr + pageHeaderSize
- + MAXALIGN(header->xlp_rem_len);
+ + XLOG_REC_ALIGN(header->xlp_rem_len);
break;
}
}
diff --git a/src/backend/access/transam/xlogutils.c b/src/backend/access/transam/xlogutils.c
index 4ecdc9220f..aad05de2ef 100644
--- a/src/backend/access/transam/xlogutils.c
+++ b/src/backend/access/transam/xlogutils.c
@@ -25,6 +25,7 @@
#include "access/xlogutils.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "storage/encryption.h"
#include "storage/smgr.h"
#include "utils/guc.h"
#include "utils/hsearch.h"
@@ -653,12 +654,15 @@ XLogTruncateRelation(RelFileNode rnode, ForkNumber forkNum,
* frontend). Probably these should be merged at some point.
*/
static void
-XLogRead(char *buf, int segsize, TimeLineID tli, XLogRecPtr startptr,
- Size count)
+XLogRead(char *buf, int segsize, TimeLineID tli, XLogRecPtr startptr, Size count)
{
- char *p;
XLogRecPtr recptr;
Size nbytes;
+#ifdef USE_ENCRYPTION
+ char *decrypt_p;
+ uint32 decryptOff;
+#endif
+ char *p;
/* state maintained across calls */
static int sendFile = -1;
@@ -669,6 +673,9 @@ XLogRead(char *buf, int segsize, TimeLineID tli, XLogRecPtr startptr,
Assert(segsize == wal_segment_size);
p = buf;
+#ifdef USE_ENCRYPTION
+ decrypt_p = p;
+#endif
recptr = startptr;
nbytes = count;
@@ -679,6 +686,9 @@ XLogRead(char *buf, int segsize, TimeLineID tli, XLogRecPtr startptr,
int readbytes;
startoff = XLogSegmentOffset(recptr, segsize);
+#ifdef USE_ENCRYPTION
+ decryptOff = startoff;
+#endif
/* Do we need to switch to a different xlog segment? */
if (sendFile < 0 || !XLByteInSeg(recptr, sendSegNo, segsize) ||
@@ -758,6 +768,25 @@ XLogRead(char *buf, int segsize, TimeLineID tli, XLogRecPtr startptr,
sendOff += readbytes;
nbytes -= readbytes;
p += readbytes;
+
+ /* Decrypt completed blocks */
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ while (decrypt_p + XLOG_BLCKSZ <= p)
+ {
+ char tweak[TWEAK_SIZE];
+
+ XLogEncryptionTweak(tweak, sendSegNo, decryptOff);
+ decrypt_block(decrypt_p, decrypt_p, XLOG_BLCKSZ, tweak);
+
+ decrypt_p += XLOG_BLCKSZ;
+ decryptOff += XLOG_BLCKSZ;
+ }
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
}
}
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 578af2e66d..0d5013d6c2 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -23,6 +23,7 @@
#include "bootstrap/bootstrap.h"
#include "catalog/index.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_control.h"
#include "catalog/pg_type.h"
#include "common/link-canary.h"
#include "libpq/pqsignal.h"
@@ -37,6 +38,7 @@
#include "storage/bufmgr.h"
#include "storage/bufpage.h"
#include "storage/condition_variable.h"
+#include "storage/encryption.h"
#include "storage/ipc.h"
#include "storage/proc.h"
#include "tcop/tcopprot.h"
@@ -50,7 +52,6 @@
uint32 bootstrap_data_checksum_version = 0; /* No checksum */
-
#define ALLOC(t, c) \
((t *) MemoryContextAllocZero(TopMemoryContext, (unsigned)(c) * sizeof(t)))
@@ -224,7 +225,7 @@ AuxiliaryProcessMain(int argc, char *argv[])
/* If no -x argument, we are a CheckerProcess */
MyAuxProcType = CheckerProcess;
- while ((flag = getopt(argc, argv, "B:c:d:D:Fkr:x:X:-:")) != -1)
+ while ((flag = getopt(argc, argv, "B:c:d:D:FkK:r:x:X:-:")) != -1)
{
switch (flag)
{
@@ -250,6 +251,25 @@ AuxiliaryProcessMain(int argc, char *argv[])
case 'F':
SetConfigOption("fsync", "false", PGC_POSTMASTER, PGC_S_ARGV);
break;
+#ifdef USE_OPENSSL
+ case 'K':
+ encryption_key_command = strdup(optarg);
+
+ /*
+ * When auxiliary process (typically bootstrap) starts, the
+ * control file might not exist yet. In this case we also use
+ * encryption_key_command to indicate that the encryption is
+ * enabled.
+ *
+ * Postmaster should not set this variable. Instead, it just
+ * sets data_encrypted according to the control file and child
+ * processes inherit that.
+ */
+ Assert(!IsUnderPostmaster);
+ data_encrypted = true;
+
+ break;
+#endif /* USE_OPENSSL */
case 'k':
bootstrap_data_checksum_version = PG_DATA_CHECKSUM_VERSION;
break;
@@ -371,6 +391,20 @@ AuxiliaryProcessMain(int argc, char *argv[])
if (!IsUnderPostmaster)
InitializeMaxBackends();
+ /*
+ * If data_encryption is set because of command line argument, do the
+ * setup now. (If set by postmaster, postmaster should have performed the
+ * setup.)
+ *
+ * This should only be useful for the bootstrap process. Anyone else
+ * initializes the encryption via ReadControlFile().
+ */
+ if (data_encrypted && MyAuxProcType == BootstrapProcess)
+ {
+ Assert(!IsUnderPostmaster);
+ setup_encryption(true);
+ }
+
BaseInit();
/*
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index 5342f217c0..0645ac00ba 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -625,7 +625,12 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
*
* We don't need to copy subdirectories
*/
- copydir(srcpath, dstpath, false);
+ {
+ RelFileNode fromNode = {srctablespace, src_dboid, InvalidOid};
+ RelFileNode toNode = {dsttablespace, dboid, InvalidOid};
+
+ copydir(srcpath, dstpath, &fromNode, &toNode);
+ }
/* Record the filesystem change in XLOG */
{
@@ -1254,7 +1259,12 @@ movedb(const char *dbname, const char *tblspcname)
/*
* Copy files from the old tablespace to the new one
*/
- copydir(src_dbpath, dst_dbpath, false);
+ {
+ RelFileNode fromNode = {src_tblspcoid, db_id, InvalidOid};
+ RelFileNode toNode = {dst_tblspcoid, db_id, InvalidOid};
+
+ copydir(src_dbpath, dst_dbpath, &fromNode, &toNode);
+ }
/*
* Record the filesystem change in XLOG
@@ -2116,7 +2126,12 @@ dbase_redo(XLogReaderState *record)
*
* We don't need to copy subdirectories
*/
- copydir(src_path, dst_path, false);
+ {
+ RelFileNode fromNode = {xlrec->src_tablespace_id, xlrec->src_db_id, InvalidOid};
+ RelFileNode toNode = {xlrec->tablespace_id, xlrec->db_id, InvalidOid};
+
+ copydir(src_path, dst_path, &fromNode, &toNode);
+ }
}
else if (info == XLOG_DBASE_DROP)
{
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 8a5b2b3b42..056d975893 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -42,6 +42,7 @@
#include "catalog/pg_database.h"
#include "catalog/pg_proc.h"
#include "common/ip.h"
+#include "lib/stringinfo.h"
#include "libpq/libpq.h"
#include "libpq/pqsignal.h"
#include "mb/pg_wchar.h"
@@ -52,6 +53,7 @@
#include "postmaster/postmaster.h"
#include "replication/walsender.h"
#include "storage/backendid.h"
+#include "storage/encryption.h"
#include "storage/dsm.h"
#include "storage/fd.h"
#include "storage/ipc.h"
@@ -105,6 +107,10 @@
#define PGSTAT_TAB_HASH_SIZE 512
#define PGSTAT_FUNCTION_HASH_SIZE 512
+/*
+ * Size of a buffer used to read stats files from file.
+ */
+#define PGSTAT_FILE_BUFFER_SIZE 1024
/* ----------
* Total number of backends including auxiliary
@@ -309,6 +315,9 @@ static HTAB *pgstat_collect_oids(Oid catalogid);
static PgStat_TableStatus *get_tabstat_entry(Oid rel_id, bool isshared);
static void pgstat_setup_memcxt(void);
+static StringInfo pgstat_setup_serial_buffer(MemoryContext *context);
+static bool pgstat_read_file(StringInfo buf, const char *statfile,
+ FILE *fpin);
static const char *pgstat_get_wait_activity(WaitEventActivity w);
static const char *pgstat_get_wait_client(WaitEventClient w);
@@ -337,6 +346,12 @@ static void pgstat_recv_recoveryconflict(PgStat_MsgRecoveryConflict *msg, int le
static void pgstat_recv_deadlock(PgStat_MsgDeadlock *msg, int len);
static void pgstat_recv_tempfile(PgStat_MsgTempFile *msg, int len);
+static bool readFromStringInfo(StringInfo str, void *data, Size size);
+
+#ifdef USE_ENCRYPTION
+static void pgstat_encryption_tweak(char *tweak, Oid dbid, bool permament);
+#endif
+
/* ------------------------------------------------------------
* Public functions called from postmaster follow
* ------------------------------------------------------------
@@ -3784,6 +3799,15 @@ pgstat_get_wait_io(WaitEventIO w)
case WAIT_EVENT_DSM_FILL_ZERO_WRITE:
event_name = "DSMFillZeroWrite";
break;
+ case WAIT_EVENT_KDF_FILE_READ:
+ event_name = "KDFFileRead";
+ break;
+ case WAIT_EVENT_KDF_FILE_SYNC:
+ event_name = "KDFFileSync";
+ break;
+ case WAIT_EVENT_KDF_FILE_WRITE:
+ event_name = "KDFFileWrite";
+ break;
case WAIT_EVENT_LOCK_FILE_ADDTODATADIR_READ:
event_name = "LockFileAddToDataDirRead";
break;
@@ -4683,6 +4707,8 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
const char *tmpfile = permanent ? PGSTAT_STAT_PERMANENT_TMPFILE : pgstat_stat_tmpname;
const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
int rc;
+ StringInfo buf;
+ MemoryContext serialContext;
elog(DEBUG2, "writing stats file \"%s\"", statfile);
@@ -4705,23 +4731,25 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
globalStats.stats_timestamp = GetCurrentTimestamp();
/*
+ * Memory for data serialization.
+ */
+ buf = pgstat_setup_serial_buffer(&serialContext);
+
+ /*
* Write the file header --- currently just a format ID.
*/
format_id = PGSTAT_FILE_FORMAT_ID;
- rc = fwrite(&format_id, sizeof(format_id), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
+ appendBinaryStringInfo(buf, (char *) &format_id, sizeof(format_id));
/*
* Write global stats struct
*/
- rc = fwrite(&globalStats, sizeof(globalStats), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
+ appendBinaryStringInfo(buf, (char *) &globalStats, sizeof(globalStats));
/*
* Write archiver stats struct
*/
- rc = fwrite(&archiverStats, sizeof(archiverStats), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
+ appendBinaryStringInfo(buf, (char *) &archiverStats, sizeof(archiverStats));
/*
* Walk through the database table.
@@ -4745,18 +4773,53 @@ pgstat_write_statsfiles(bool permanent, bool allDbs)
* Write out the DB entry. We don't write the tables or functions
* pointers, since they're of no use to any other process.
*/
- fputc('D', fpout);
- rc = fwrite(dbentry, offsetof(PgStat_StatDBEntry, tables), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
+ appendBinaryStringInfo(buf, (char *) "D", 1);
+ appendBinaryStringInfo(buf, (char *) dbentry,
+ offsetof(PgStat_StatDBEntry, tables));
+ }
+
+ /*
+ * No more output to be done.
+ */
+ appendBinaryStringInfo(buf, (char *) "E", 1);
+
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ char tweak[TWEAK_SIZE];
+
+ /*
+ * Make sure the data is aligned to ENCRYPTION_BLOCK.
+ */
+ if (buf->len % ENCRYPTION_BLOCK > 0)
+ {
+ char zerobuf[ENCRYPTION_BLOCK];
+
+ memset(zerobuf, 0, ENCRYPTION_BLOCK);
+ appendBinaryStringInfo(buf,
+ zerobuf,
+ ENCRYPTION_BLOCK - buf->len % ENCRYPTION_BLOCK);
+ }
+
+ pgstat_encryption_tweak(tweak, InvalidOid, permanent);
+ encrypt_block(buf->data, buf->data, buf->len, tweak);
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
}
/*
- * No more output to be done. Close the temp file and replace the old
- * pgstat.stat with it. The ferror() check replaces testing for error
- * after each individual fputc or fwrite above.
+ * Write the data to the file.
*/
- fputc('E', fpout);
+ rc = fwrite(buf->data, buf->len, 1, fpout);
+ (void) rc; /* we'll check for error with ferror */
+
+ /* Free the temporary storage. */
+ MemoryContextResetOnly(serialContext);
+ /*
+ * Close the temp file and replace the old pgstat.stat with it.
+ */
if (ferror(fpout))
{
ereport(LOG,
@@ -4837,6 +4900,8 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
int rc;
char tmpfile[MAXPGPATH];
char statfile[MAXPGPATH];
+ StringInfo buf;
+ MemoryContext serialContext;
get_dbstat_filename(permanent, true, dbid, tmpfile, MAXPGPATH);
get_dbstat_filename(permanent, false, dbid, statfile, MAXPGPATH);
@@ -4857,11 +4922,15 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
}
/*
+ * Memory for data serialization.
+ */
+ buf = pgstat_setup_serial_buffer(&serialContext);
+
+ /*
* Write the file header --- currently just a format ID.
*/
format_id = PGSTAT_FILE_FORMAT_ID;
- rc = fwrite(&format_id, sizeof(format_id), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
+ appendBinaryStringInfo(buf, (char *) &format_id, sizeof(format_id));
/*
* Walk through the database's access stats per table.
@@ -4869,9 +4938,8 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
hash_seq_init(&tstat, dbentry->tables);
while ((tabentry = (PgStat_StatTabEntry *) hash_seq_search(&tstat)) != NULL)
{
- fputc('T', fpout);
- rc = fwrite(tabentry, sizeof(PgStat_StatTabEntry), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
+ appendBinaryStringInfo(buf, (char *) "T", 1);
+ appendBinaryStringInfo(buf, (char *) tabentry, sizeof(PgStat_StatTabEntry));
}
/*
@@ -4880,18 +4948,52 @@ pgstat_write_db_statsfile(PgStat_StatDBEntry *dbentry, bool permanent)
hash_seq_init(&fstat, dbentry->functions);
while ((funcentry = (PgStat_StatFuncEntry *) hash_seq_search(&fstat)) != NULL)
{
- fputc('F', fpout);
- rc = fwrite(funcentry, sizeof(PgStat_StatFuncEntry), 1, fpout);
- (void) rc; /* we'll check for error with ferror */
+ appendBinaryStringInfo(buf, (char *) "F", 1);
+ appendBinaryStringInfo(buf, (char *) funcentry, sizeof(PgStat_StatFuncEntry));
}
/*
- * No more output to be done. Close the temp file and replace the old
- * pgstat.stat with it. The ferror() check replaces testing for error
- * after each individual fputc or fwrite above.
+ * No more output to be done.
*/
- fputc('E', fpout);
+ appendBinaryStringInfo(buf, (char *) "E", 1);
+
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ char tweak[TWEAK_SIZE];
+
+ /*
+ * Make sure the data is aligned to ENCRYPTION_BLOCK.
+ */
+ if (buf->len % ENCRYPTION_BLOCK > 0)
+ {
+ char zerobuf[ENCRYPTION_BLOCK];
+ memset(zerobuf, 0, ENCRYPTION_BLOCK);
+ appendBinaryStringInfo(buf,
+ zerobuf,
+ ENCRYPTION_BLOCK - buf->len % ENCRYPTION_BLOCK);
+ }
+
+ pgstat_encryption_tweak(tweak, dbid, permanent);
+ encrypt_block(buf->data, buf->data, buf->len, tweak);
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
+
+ /*
+ * Write the data to the file.
+ */
+ rc = fwrite(buf->data, buf->len, 1, fpout);
+ (void) rc; /* we'll check for error with ferror */
+
+ /* Free the temporary storage. */
+ MemoryContextResetOnly(serialContext);
+
+ /*
+ * Close the temp file and replace the old pgstat.stat with it.
+ */
if (ferror(fpout))
{
ereport(LOG,
@@ -4958,6 +5060,8 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
int32 format_id;
bool found;
const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
+ StringInfo buf;
+ MemoryContext serialContext;
/*
* The tables will live in pgStatLocalContext.
@@ -5008,9 +5112,38 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
}
/*
+ * Memory for data serialization.
+ */
+ buf = pgstat_setup_serial_buffer(&serialContext);
+
+ /*
+ * Read the data into memory.
+ */
+ if (!pgstat_read_file(buf, statfile, fpin))
+ goto done;
+
+ /*
+ * Decrypt the data if it's encrypted.
+ */
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ char tweak[TWEAK_SIZE];
+
+ pgstat_encryption_tweak(tweak, InvalidOid, permanent);
+ decrypt_block(buf->data, buf->data, buf->len, tweak);
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
+
+ /* Prepare for reading. */
+ buf->cursor = 0;
+
+ /*
* Verify it's of the expected format.
*/
- if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
+ if (!readFromStringInfo(buf, &format_id, sizeof(format_id)) ||
format_id != PGSTAT_FILE_FORMAT_ID)
{
ereport(pgStatRunningInCollector ? LOG : WARNING,
@@ -5021,7 +5154,7 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
/*
* Read global stats struct
*/
- if (fread(&globalStats, 1, sizeof(globalStats), fpin) != sizeof(globalStats))
+ if (!readFromStringInfo(buf, &globalStats, sizeof(globalStats)))
{
ereport(pgStatRunningInCollector ? LOG : WARNING,
(errmsg("corrupted statistics file \"%s\"", statfile)));
@@ -5042,7 +5175,7 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
/*
* Read archiver stats struct
*/
- if (fread(&archiverStats, 1, sizeof(archiverStats), fpin) != sizeof(archiverStats))
+ if (!readFromStringInfo(buf, &archiverStats, sizeof(archiverStats)))
{
ereport(pgStatRunningInCollector ? LOG : WARNING,
(errmsg("corrupted statistics file \"%s\"", statfile)));
@@ -5056,15 +5189,24 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
*/
for (;;)
{
- switch (fgetc(fpin))
+ char kind;
+
+ if (!readFromStringInfo(buf, &kind, 1))
+ {
+ ereport(pgStatRunningInCollector ? LOG : WARNING,
+ (errmsg("corrupted statistics file \"%s\"", statfile)));
+ goto done;
+ }
+
+ switch (kind)
{
/*
* 'D' A PgStat_StatDBEntry struct describing a database
* follows.
*/
case 'D':
- if (fread(&dbbuf, 1, offsetof(PgStat_StatDBEntry, tables),
- fpin) != offsetof(PgStat_StatDBEntry, tables))
+ if (!readFromStringInfo(buf, &dbbuf,
+ offsetof(PgStat_StatDBEntry, tables)))
{
ereport(pgStatRunningInCollector ? LOG : WARNING,
(errmsg("corrupted statistics file \"%s\"",
@@ -5154,6 +5296,9 @@ pgstat_read_statsfiles(Oid onlydb, bool permanent, bool deep)
done:
FreeFile(fpin);
+ /* Free the temporary storage. */
+ MemoryContextResetOnly(serialContext);
+
/* If requested to read the permanent file, also get rid of it. */
if (permanent)
{
@@ -5191,6 +5336,8 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
int32 format_id;
bool found;
char statfile[MAXPGPATH];
+ StringInfo buf;
+ MemoryContext serialContext;
get_dbstat_filename(permanent, false, databaseid, statfile, MAXPGPATH);
@@ -5214,9 +5361,39 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
}
/*
+ * Memory for data serialization.
+ */
+ buf = pgstat_setup_serial_buffer(&serialContext);
+
+ /*
+ * Read the data into memory.
+ */
+
+ if (!pgstat_read_file(buf, statfile, fpin))
+ goto done;
+
+ /*
+ * Decrypt the data if it's encrypted.
+ */
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ char tweak[TWEAK_SIZE];
+
+ pgstat_encryption_tweak(tweak, databaseid, permanent);
+ decrypt_block(buf->data, buf->data, buf->len, tweak);
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
+
+ /* Prepare for reading. */
+ buf->cursor = 0;
+
+ /*
* Verify it's of the expected format.
*/
- if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
+ if (!readFromStringInfo(buf, &format_id, sizeof(format_id)) ||
format_id != PGSTAT_FILE_FORMAT_ID)
{
ereport(pgStatRunningInCollector ? LOG : WARNING,
@@ -5230,14 +5407,22 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
*/
for (;;)
{
- switch (fgetc(fpin))
+ char kind;
+
+ if (!readFromStringInfo(buf, &kind, 1))
+ {
+ ereport(pgStatRunningInCollector ? LOG : WARNING,
+ (errmsg("corrupted statistics file \"%s\"", statfile)));
+ goto done;
+ }
+
+ switch (kind)
{
/*
* 'T' A PgStat_StatTabEntry follows.
*/
case 'T':
- if (fread(&tabbuf, 1, sizeof(PgStat_StatTabEntry),
- fpin) != sizeof(PgStat_StatTabEntry))
+ if (!readFromStringInfo(buf, &tabbuf, sizeof(PgStat_StatTabEntry)))
{
ereport(pgStatRunningInCollector ? LOG : WARNING,
(errmsg("corrupted statistics file \"%s\"",
@@ -5270,8 +5455,8 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
* 'F' A PgStat_StatFuncEntry follows.
*/
case 'F':
- if (fread(&funcbuf, 1, sizeof(PgStat_StatFuncEntry),
- fpin) != sizeof(PgStat_StatFuncEntry))
+ if (!readFromStringInfo(buf, &funcbuf,
+ sizeof(PgStat_StatFuncEntry)))
{
ereport(pgStatRunningInCollector ? LOG : WARNING,
(errmsg("corrupted statistics file \"%s\"",
@@ -5317,6 +5502,9 @@ pgstat_read_db_statsfile(Oid databaseid, HTAB *tabhash, HTAB *funchash,
done:
FreeFile(fpin);
+ /* Free the temporary storage. */
+ MemoryContextResetOnly(serialContext);
+
if (permanent)
{
elog(DEBUG2, "removing permanent stats file \"%s\"", statfile);
@@ -5351,6 +5539,8 @@ pgstat_read_db_statsfile_timestamp(Oid databaseid, bool permanent,
FILE *fpin;
int32 format_id;
const char *statfile = permanent ? PGSTAT_STAT_PERMANENT_FILENAME : pgstat_stat_filename;
+ StringInfo buf;
+ MemoryContext serialContext;
/*
* Try to open the stats file. As above, anything but ENOENT is worthy of
@@ -5367,38 +5557,68 @@ pgstat_read_db_statsfile_timestamp(Oid databaseid, bool permanent,
}
/*
+ * Memory for data serialization.
+ */
+ buf = pgstat_setup_serial_buffer(&serialContext);
+
+ /*
+ * Read the data into memory.
+ */
+ if (!pgstat_read_file(buf, statfile, fpin))
+ goto done;
+
+ /*
+ * Decrypt the data if it's encrypted.
+ */
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ char tweak[TWEAK_SIZE];
+
+ pgstat_encryption_tweak(tweak, InvalidOid, permanent);
+ decrypt_block(buf->data, buf->data, buf->len, tweak);
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
+
+ /* Prepare for reading. */
+ buf->cursor = 0;
+
+ /*
* Verify it's of the expected format.
*/
- if (fread(&format_id, 1, sizeof(format_id), fpin) != sizeof(format_id) ||
+ if (!readFromStringInfo(buf, &format_id, sizeof(format_id)) ||
format_id != PGSTAT_FILE_FORMAT_ID)
{
ereport(pgStatRunningInCollector ? LOG : WARNING,
(errmsg("corrupted statistics file \"%s\"", statfile)));
FreeFile(fpin);
+ MemoryContextResetOnly(serialContext);
return false;
}
/*
* Read global stats struct
*/
- if (fread(&myGlobalStats, 1, sizeof(myGlobalStats),
- fpin) != sizeof(myGlobalStats))
+ if (!readFromStringInfo(buf, &myGlobalStats, sizeof(myGlobalStats)))
{
ereport(pgStatRunningInCollector ? LOG : WARNING,
(errmsg("corrupted statistics file \"%s\"", statfile)));
FreeFile(fpin);
+ MemoryContextResetOnly(serialContext);
return false;
}
/*
* Read archiver stats struct
*/
- if (fread(&myArchiverStats, 1, sizeof(myArchiverStats),
- fpin) != sizeof(myArchiverStats))
+ if (!readFromStringInfo(buf, &myArchiverStats, sizeof(myArchiverStats)))
{
ereport(pgStatRunningInCollector ? LOG : WARNING,
(errmsg("corrupted statistics file \"%s\"", statfile)));
FreeFile(fpin);
+ MemoryContextResetOnly(serialContext);
return false;
}
@@ -5411,15 +5631,24 @@ pgstat_read_db_statsfile_timestamp(Oid databaseid, bool permanent,
*/
for (;;)
{
- switch (fgetc(fpin))
+ char kind;
+
+ if (!readFromStringInfo(buf, &kind, 1))
+ {
+ ereport(pgStatRunningInCollector ? LOG : WARNING,
+ (errmsg("corrupted statistics file \"%s\"", statfile)));
+ goto done;
+ }
+
+ switch (kind)
{
/*
* 'D' A PgStat_StatDBEntry struct describing a database
* follows.
*/
case 'D':
- if (fread(&dbentry, 1, offsetof(PgStat_StatDBEntry, tables),
- fpin) != offsetof(PgStat_StatDBEntry, tables))
+ if (!readFromStringInfo(buf, &dbentry,
+ offsetof(PgStat_StatDBEntry, tables)))
{
ereport(pgStatRunningInCollector ? LOG : WARNING,
(errmsg("corrupted statistics file \"%s\"",
@@ -5452,6 +5681,7 @@ pgstat_read_db_statsfile_timestamp(Oid databaseid, bool permanent,
done:
FreeFile(fpin);
+ MemoryContextResetOnly(serialContext);
return true;
}
@@ -5603,6 +5833,61 @@ pgstat_setup_memcxt(void)
ALLOCSET_SMALL_SIZES);
}
+/* ----------
+ * pgstat_setup_serial_buffer() -
+ *
+ * Allocate memory for data serialization and return the containing memory
+ * context.
+ *
+ * ----------
+ */
+static StringInfo
+pgstat_setup_serial_buffer(MemoryContext *context)
+{
+ MemoryContext oldCxt;
+ StringInfo result;
+
+ *context = AllocSetContextCreate(TopMemoryContext,
+ "Statistics encryption",
+ ALLOCSET_DEFAULT_SIZES);
+ oldCxt = MemoryContextSwitchTo(*context);
+ result = makeStringInfo();
+ MemoryContextSwitchTo(oldCxt);
+
+ return result;
+}
+
+/*
+ * Read statistics data from file into memory.
+ */
+static bool
+pgstat_read_file(StringInfo buf, const char *statfile, FILE *fpin)
+{
+ for (;;)
+ {
+ char bufread[PGSTAT_FILE_BUFFER_SIZE];
+ Size bytesread;
+
+ bytesread = fread(bufread, 1, PGSTAT_FILE_BUFFER_SIZE, fpin);
+ appendBinaryStringInfo(buf, bufread, bytesread);
+
+ if (bytesread < PGSTAT_FILE_BUFFER_SIZE)
+ break;
+ }
+
+ /*
+ * Check if the amount of data makes sense.
+ */
+ if (buf->len == 0 || (data_encrypted && buf->len % ENCRYPTION_BLOCK > 0))
+ {
+ ereport(pgStatRunningInCollector ? LOG : WARNING,
+ (errmsg("corrupted statistics file \"%s\"", statfile)));
+ return false;
+ }
+
+ return true;
+}
+
/* ----------
* pgstat_clear_snapshot() -
@@ -6312,6 +6597,35 @@ pgstat_write_statsfile_needed(void)
return false;
}
+/*
+ * Read "size" bytes into "data" from StringInfo.
+ *
+ * Return true iff there was enough data available.
+ *
+ * XXX Consider moving this function to stringinfo.c.
+ */
+static bool
+readFromStringInfo(StringInfo str, void *data, Size size)
+{
+ if (str->cursor + size > str->len)
+ return false;
+
+ memcpy(data, str->data + str->cursor, size);
+ str->cursor += size;
+
+ return true;
+}
+
+#ifdef USE_ENCRYPTION
+static void
+pgstat_encryption_tweak(char *tweak, Oid dbid, bool permanent)
+{
+ memset(tweak, 0, TWEAK_SIZE);
+ memcpy(tweak, &dbid, sizeof(Oid));
+ memcpy(tweak + sizeof(Oid), &permanent, sizeof(bool));
+}
+#endif /* USE_ENCRYPTION */
+
/* ----------
* pgstat_db_requested() -
*
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 305ff36258..0831c74436 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -116,6 +116,7 @@
#include "postmaster/syslogger.h"
#include "replication/logicallauncher.h"
#include "replication/walsender.h"
+#include "storage/encryption.h"
#include "storage/fd.h"
#include "storage/ipc.h"
#include "storage/pg_shmem.h"
@@ -673,7 +674,7 @@ PostmasterMain(int argc, char *argv[])
* tcop/postgres.c (the option sets should not conflict) and with the
* common help() function in main/main.c.
*/
- while ((opt = getopt(argc, argv, "B:bc:C:D:d:EeFf:h:ijk:lN:nOo:Pp:r:S:sTt:W:-:")) != -1)
+ while ((opt = getopt(argc, argv, "B:bc:C:D:d:EeFf:h:ijK:k:lN:nOo:Pp:r:S:sTt:W:-:")) != -1)
{
switch (opt)
{
@@ -731,6 +732,12 @@ PostmasterMain(int argc, char *argv[])
/* only used by interactive backend */
break;
+#ifdef USE_OPENSSL
+ case 'K':
+ encryption_key_command = strdup(optarg);
+ break;
+#endif /* USE_OPENSSL */
+
case 'k':
SetConfigOption("unix_socket_directories", optarg, PGC_POSTMASTER, PGC_S_ARGV);
break;
@@ -1174,6 +1181,11 @@ PostmasterMain(int argc, char *argv[])
/*
* Set up shared memory and semaphores.
+ *
+ * This includes call of setup_encryption() as soon as we realize that
+ * "data_encrypted" field of the control file is set. The encryption must
+ * be initialized at the point so that "encryption_verification" field of
+ * the control file can be checked.
*/
reset_shared(PostPortNumber);
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index 91ae448955..d8c9abb262 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -1102,7 +1102,7 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
/* Exclude all forks for unlogged tables except the init fork */
if (isDbDir &&
parse_filename_for_nontemp_relation(de->d_name, &relOidChars,
- &relForkNum))
+ &relForkNum, NULL))
{
/* Never exclude init forks */
if (relForkNum != INIT_FORKNUM)
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 658fbe6494..65d298ca3c 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -70,6 +70,7 @@
#include "replication/slot.h"
#include "replication/snapbuild.h" /* just for SnapBuildSnapDecRefcount */
#include "storage/bufmgr.h"
+#include "storage/encryption.h"
#include "storage/fd.h"
#include "storage/sinval.h"
#include "utils/builtins.h"
@@ -192,7 +193,7 @@ static void ReorderBufferExecuteInvalidations(ReorderBuffer *rb, ReorderBufferTX
static void ReorderBufferCheckSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn);
static void ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn);
static void ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
- int fd, ReorderBufferChange *change);
+ int fd, XLogSegNo segno, ReorderBufferChange *change);
static Size ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn,
int *fd, XLogSegNo *segno);
static void ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
@@ -218,6 +219,20 @@ static void ReorderBufferToastAppendChunk(ReorderBuffer *rb, ReorderBufferTXN *t
Relation relation, ReorderBufferChange *change);
+/* ----------------------------------------------
+ * encryption / decryption of serialized changes
+ * ----------------------------------------------
+ */
+
+/*
+ * Compute encryption tweak for buffer change.
+ */
+#define REORDER_BUFFER_CHANGE_TWEAK(tweak, segno) \
+ StaticAssertStmt(sizeof(XLogSegNo) <= TWEAK_SIZE, \
+ "XLogSegNo does not fit into encryption tweak"); \
+ memset((tweak), 0, TWEAK_SIZE); \
+ memcpy((tweak), (char *) &segno, sizeof(XLogSegNo))
+
/*
* Allocate a new ReorderBuffer and clean out any old serialized state from
* prior ReorderBuffer instances for the same slot.
@@ -2294,7 +2309,7 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
errmsg("could not open file \"%s\": %m", path)));
}
- ReorderBufferSerializeChange(rb, txn, fd, change);
+ ReorderBufferSerializeChange(rb, txn, fd, curOpenSegNo, change);
dlist_delete(&change->node);
ReorderBufferReturnChange(rb, change);
@@ -2315,12 +2330,26 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
*/
static void
ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
- int fd, ReorderBufferChange *change)
+ int fd, XLogSegNo segno,
+ ReorderBufferChange *change)
{
ReorderBufferDiskChange *ondisk;
- Size sz = sizeof(ReorderBufferDiskChange);
+ Size sz,
+ sz_hdr;
+ char *outbuf;
+
+ sz = sizeof(ReorderBufferDiskChange);
- ReorderBufferSerializeReserve(rb, sz);
+ /*
+ * As the on-disk change has variable length, the header will have to be
+ * de-serialized separate, see ReorderBufferRestoreChanges(). Therefore we
+ * also need to encrypt it separate. Make sure the size is appropriate.
+ */
+ if (data_encrypted)
+ sz = TYPEALIGN(ENCRYPTION_BLOCK, sz);
+
+ sz_hdr = sz;
+ ReorderBufferSerializeReserve(rb, sz_hdr);
ondisk = (ReorderBufferDiskChange *) rb->outbuf;
memcpy(&ondisk->change, change, sizeof(ReorderBufferChange));
@@ -2357,6 +2386,15 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
}
/* make sure we have enough space */
+ if (data_encrypted)
+ {
+ /*
+ * Encryption works with blocks. As the changes are stored
+ * and retrieved separately, we also have to decrypt them
+ * alone.
+ */
+ sz = TYPEALIGN(ENCRYPTION_BLOCK, sz);
+ }
ReorderBufferSerializeReserve(rb, sz);
data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
@@ -2387,12 +2425,15 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
char *data;
Size prefix_size = strlen(change->data.msg.prefix) + 1;
+
sz += prefix_size + change->data.msg.message_size +
sizeof(Size) + sizeof(Size);
+
+ if (data_encrypted)
+ sz = TYPEALIGN(ENCRYPTION_BLOCK, sz);
ReorderBufferSerializeReserve(rb, sz);
data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
-
/* might have been reallocated above */
ondisk = (ReorderBufferDiskChange *) rb->outbuf;
@@ -2425,11 +2466,13 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
;
/* make sure we have enough space */
+ if (data_encrypted)
+ sz = TYPEALIGN(ENCRYPTION_BLOCK, sz);
ReorderBufferSerializeReserve(rb, sz);
+
data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
/* might have been reallocated above */
ondisk = (ReorderBufferDiskChange *) rb->outbuf;
-
memcpy(data, snap, sizeof(SnapshotData));
data += sizeof(SnapshotData);
@@ -2478,9 +2521,51 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
ondisk->size = sz;
+ /*
+ * Encrypt the change if encryption is required.
+ */
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ char tweak[TWEAK_SIZE];
+
+ REORDER_BUFFER_CHANGE_TWEAK(tweak, segno);
+
+ enlarge_encryption_buffer(ondisk->size);
+
+ /*
+ * Encrypt the header and the payload separate as explained at the top
+ * of the function.
+ */
+ encrypt_block((char *) rb->outbuf, encryption_buffer, sz_hdr,
+ tweak);
+
+ if (ondisk->size > sz_hdr)
+ encrypt_block((char *) rb->outbuf + sz_hdr,
+ encryption_buffer + sz_hdr,
+ ondisk->size - sz_hdr, tweak);
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
+
+ /*
+ * Make sure the correct buffer is written to disk.
+ */
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ outbuf = encryption_buffer;
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
+ else
+ outbuf = rb->outbuf;
+
errno = 0;
pgstat_report_wait_start(WAIT_EVENT_REORDER_BUFFER_WRITE);
- if (write(fd, rb->outbuf, ondisk->size) != ondisk->size)
+ if (write(fd, outbuf, ondisk->size) != ondisk->size)
{
int save_errno = errno;
@@ -2508,6 +2593,10 @@ ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn,
Size restored = 0;
XLogSegNo last_segno;
dlist_mutable_iter cleanup_iter;
+ Size sz_hdr;
+#ifdef USE_ENCRYPTION
+ char encryption_tweak[TWEAK_SIZE];
+#endif
Assert(txn->first_lsn != InvalidXLogRecPtr);
Assert(txn->final_lsn != InvalidXLogRecPtr);
@@ -2526,6 +2615,17 @@ ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn,
XLByteToSeg(txn->final_lsn, last_segno, wal_segment_size);
+ /*
+ * See ReorderBufferSerializeChange for explanation.
+ */
+ sz_hdr = sizeof(ReorderBufferDiskChange);
+ if (data_encrypted)
+#ifdef USE_ENCRYPTION
+ sz_hdr = TYPEALIGN(ENCRYPTION_BLOCK, sz_hdr);
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+
while (restored < max_changes_in_memory && *segno <= last_segno)
{
int readBytes;
@@ -2567,9 +2667,9 @@ ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn,
* about the total size. If we couldn't read a record, we're at the
* end of this file.
*/
- ReorderBufferSerializeReserve(rb, sizeof(ReorderBufferDiskChange));
+ ReorderBufferSerializeReserve(rb, sz_hdr);
pgstat_report_wait_start(WAIT_EVENT_REORDER_BUFFER_READ);
- readBytes = read(*fd, rb->outbuf, sizeof(ReorderBufferDiskChange));
+ readBytes = read(*fd, rb->outbuf, sz_hdr);
pgstat_report_wait_end();
/* eof */
@@ -2584,38 +2684,64 @@ ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn,
ereport(ERROR,
(errcode_for_file_access(),
errmsg("could not read from reorderbuffer spill file: %m")));
- else if (readBytes != sizeof(ReorderBufferDiskChange))
+ else if (readBytes != sz_hdr)
ereport(ERROR,
(errcode_for_file_access(),
errmsg("could not read from reorderbuffer spill file: read %d instead of %u bytes",
readBytes,
- (uint32) sizeof(ReorderBufferDiskChange))));
+ (uint32) sz_hdr)));
+
+ /*
+ * Decrypt the header if the change is decrypted, so that we know the
+ * change size.
+ */
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ REORDER_BUFFER_CHANGE_TWEAK(encryption_tweak, *segno);
+ decrypt_block(rb->outbuf, rb->outbuf, sz_hdr, encryption_tweak);
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
ondisk = (ReorderBufferDiskChange *) rb->outbuf;
- ReorderBufferSerializeReserve(rb,
- sizeof(ReorderBufferDiskChange) + ondisk->size);
+ ReorderBufferSerializeReserve(rb, ondisk->size);
ondisk = (ReorderBufferDiskChange *) rb->outbuf;
pgstat_report_wait_start(WAIT_EVENT_REORDER_BUFFER_READ);
- readBytes = read(*fd, rb->outbuf + sizeof(ReorderBufferDiskChange),
- ondisk->size - sizeof(ReorderBufferDiskChange));
+ readBytes = read(*fd, rb->outbuf + sz_hdr, ondisk->size - sz_hdr);
pgstat_report_wait_end();
if (readBytes < 0)
ereport(ERROR,
(errcode_for_file_access(),
errmsg("could not read from reorderbuffer spill file: %m")));
- else if (readBytes != ondisk->size - sizeof(ReorderBufferDiskChange))
+ else if (readBytes != ondisk->size - sz_hdr)
ereport(ERROR,
(errcode_for_file_access(),
errmsg("could not read from reorderbuffer spill file: read %d instead of %u bytes",
readBytes,
- (uint32) (ondisk->size - sizeof(ReorderBufferDiskChange)))));
+ (uint32) (ondisk->size - sz_hdr))));
+
+ /*
+ * ok, read a full change from disk, now decrypt it if it's encrypted
+ * and if there's some data beyond the header.
+ */
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ if (ondisk->size > sz_hdr)
+ decrypt_block(rb->outbuf + sz_hdr, rb->outbuf + sz_hdr,
+ ondisk->size - sz_hdr, encryption_tweak);
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
/*
- * ok, read a full change from disk, now restore it into proper
- * in-memory format
+ * Restore the change into proper in-memory format
*/
ReorderBufferRestoreChange(rb, txn, rb->outbuf);
restored++;
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 370429d746..0dc5edec90 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -77,6 +77,7 @@
#include "replication/walsender.h"
#include "replication/walsender_private.h"
#include "storage/condition_variable.h"
+#include "storage/encryption.h"
#include "storage/fd.h"
#include "storage/ipc.h"
#include "storage/pmsignal.h"
@@ -2338,9 +2339,16 @@ XLogRead(char *buf, XLogRecPtr startptr, Size count)
XLogRecPtr recptr;
Size nbytes;
XLogSegNo segno;
+#ifdef USE_ENCRYPTION
+ char *decrypt_p;
+ uint32 decryptOff;
+#endif
retry:
p = buf;
+#ifdef USE_ENCRYPTION
+ decrypt_p = p;
+#endif
recptr = startptr;
nbytes = count;
@@ -2351,6 +2359,9 @@ retry:
int readbytes;
startoff = XLogSegmentOffset(recptr, wal_segment_size);
+#ifdef USE_ENCRYPTION
+ decryptOff = startoff;
+#endif
if (sendFile < 0 || !XLByteInSeg(recptr, sendSegNo, wal_segment_size))
{
@@ -2466,6 +2477,25 @@ retry:
sendOff += readbytes;
nbytes -= readbytes;
p += readbytes;
+
+ /* Decrypt completed blocks */
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ while (decrypt_p + XLOG_BLCKSZ <= p)
+ {
+ char tweak[TWEAK_SIZE];
+
+ XLogEncryptionTweak(tweak, sendSegNo, decryptOff);
+ decrypt_block(decrypt_p, decrypt_p, XLOG_BLCKSZ, tweak);
+
+ decrypt_p += XLOG_BLCKSZ;
+ decryptOff += XLOG_BLCKSZ;
+ }
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
}
/*
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index e93813d973..9fb202abf7 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -47,12 +47,17 @@
#include "storage/fd.h"
#include "storage/buffile.h"
#include "storage/buf_internals.h"
+#include "storage/encryption.h"
+#include "utils/datetime.h"
#include "utils/resowner.h"
/*
* We break BufFiles into gigabyte-sized segments, regardless of RELSEG_SIZE.
* The reason is that we'd like large BufFiles to be spread across multiple
* tablespaces when available.
+ *
+ * The number of useful bytes is appended to each segment of shared encrypted
+ * file, see BufFile.useful.
*/
#define MAX_PHYSICAL_FILESIZE 0x40000000
#define BUFFILE_SEG_SIZE (MAX_PHYSICAL_FILESIZE / BLCKSZ)
@@ -68,6 +73,35 @@ struct BufFile
/* all files except the last have length exactly MAX_PHYSICAL_FILESIZE */
File *files; /* palloc'd array with numFiles entries */
off_t *offsets; /* palloc'd array with numFiles entries */
+#ifdef USE_ENCRYPTION
+
+ /*
+ * If the file is encrypted, only the whole buffer can be loaded / dumped
+ * --- see BufFileLoadBuffer() for more info --- whether it's space is
+ * used up or not. Therefore we need to keep track of the actual on-disk
+ * size buffer of each component file, as it would be if there was no
+ * encryption.
+ *
+ * List would make coding simpler, however would not contribute to
+ * performance. Random access is important here.
+ */
+ off_t *useful;
+
+ /*
+ * The array may need to be expanded independent from extendBufFile()
+ * (i.e. earlier than the buffer gets dumped), so store the number of
+ * elements separate from numFiles.
+ */
+ int nuseful;
+
+ /*
+ * Segment number is used to compute encryption tweak so we must remember
+ * the original numbers of segments if the file is encrypted and if it was
+ * passed as target to BufFileAppend() at least once. If this field is
+ * NULL, ->curFile is used to compute the tweak.
+ */
+ off_t *segnos;
+#endif
/*
* offsets[i] is the current seek position of files[i]. We use this to
@@ -106,6 +140,10 @@ static void BufFileLoadBuffer(BufFile *file);
static void BufFileDumpBuffer(BufFile *file);
static int BufFileFlush(BufFile *file);
static File MakeNewSharedSegment(BufFile *file, int segment);
+#ifdef USE_ENCRYPTION
+static void BufFileTweak(char *tweak, BufFile *file);
+static void ensureUsefulArraySize(BufFile *file, int required);
+#endif
/*
* Create BufFile and perform the common initialization.
@@ -125,6 +163,28 @@ makeBufFileCommon(int nfiles)
file->pos = 0;
file->nbytes = 0;
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ file->useful = (off_t *) palloc0(sizeof(off_t) * nfiles);
+ file->nuseful = nfiles;
+ file->segnos = NULL;
+
+ /*
+ * The unused (trailing) part of the buffer should not contain
+ * undefined data: if we encrypt such a buffer and flush it to disk,
+ * the encrypted form of that "undefined part" can get zeroed due to
+ * seek and write beyond EOF. If such a buffer gets loaded and
+ * decrypted, the change of the undefined part to zeroes can affect
+ * the valid part if it does not end at block boundary. By setting the
+ * whole buffer to zeroes we ensure that the unused part always
+ * contains zeroes.
+ */
+ MemSet(file->buffer.data, 0, BLCKSZ);
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
return file;
}
@@ -172,6 +232,22 @@ extendBufFile(BufFile *file)
(file->numFiles + 1) * sizeof(File));
file->offsets = (off_t *) repalloc(file->offsets,
(file->numFiles + 1) * sizeof(off_t));
+
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ ensureUsefulArraySize(file, file->numFiles + 1);
+
+ if (file->segnos)
+ {
+ file->segnos = (off_t *) repalloc(file->segnos,
+ (file->numFiles + 1) * sizeof(off_t));
+ file->segnos[file->numFiles] = file->numFiles;
+ }
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
file->files[file->numFiles] = pfile;
file->offsets[file->numFiles] = 0L;
file->numFiles++;
@@ -302,6 +378,7 @@ BufFileOpenShared(SharedFileSet *fileset, const char *name)
files[nfiles] = SharedFileSetOpen(fileset, segment_name);
if (files[nfiles] <= 0)
break;
+
++nfiles;
CHECK_FOR_INTERRUPTS();
@@ -317,7 +394,64 @@ BufFileOpenShared(SharedFileSet *fileset, const char *name)
errmsg("could not open BufFile \"%s\"", name)));
file = makeBufFileCommon(nfiles);
+
+ /*
+ * Shared encrypted segment should contain information on the number of
+ * useful bytes at the end.
+ */
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ off_t pos;
+ int i;
+
+ for (i = 0; i < nfiles; i++)
+ {
+ int nbytes;
+ File segment = files[i];
+
+ pos = FileSeek(segment, -sizeof(off_t), SEEK_END);
+
+ /*
+ * The word must immediately follow the last buffer of the
+ * segment.
+ */
+ if (pos <= 0 || pos % BLCKSZ != 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not find padding info in BufFile \"%s\"",
+ name)));
+
+ nbytes = FileRead(segment, (char *) &file->useful[i],
+ sizeof(off_t), WAIT_EVENT_BUFFILE_READ);
+ if (nbytes != sizeof(off_t))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read padding info from BufFile \"%s\"",
+ name)));
+ Assert(file->useful[i] > 0);
+
+ /*
+ * Keep ->offsets up-to-date.
+ */
+ file->offsets[i] = FileTell(segment);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
+
file->files = files;
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ file->nuseful = nfiles;
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
file->readOnly = true; /* Can't write to files opened this way */
file->fileset = fileset;
file->name = pstrdup(name);
@@ -397,6 +531,17 @@ BufFileClose(BufFile *file)
/* release the buffer space */
pfree(file->files);
pfree(file->offsets);
+
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ if (file->segnos)
+ pfree(file->segnos);
+ pfree(file->useful);
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
pfree(file);
}
@@ -404,7 +549,7 @@ BufFileClose(BufFile *file)
* BufFileLoadBuffer
*
* Load some data into buffer, if possible, starting from curOffset.
- * At call, must have dirty = false, pos and nbytes = 0.
+ * At call, must have dirty = false, nbytes = 0.
* On exit, nbytes is number of bytes loaded.
*/
static void
@@ -413,6 +558,22 @@ BufFileLoadBuffer(BufFile *file)
File thisfile;
/*
+ * Only whole multiple of ENCRYPTION_BLOCK can be encrypted / decrypted,
+ * but we choose to use BLCKSZ (i.e. BufFile buffer) as the unit. The
+ * point is that curOffset is a component of the encryption tweak, and all
+ * data within particular call of encrypt_block() / decrypt_block() must
+ * have the same tweak. So whichever unit we choose we must stick on it
+ * and never encrypt / decrypt multiple units at a time.
+ *
+ * BLCKSZ also seems better choice than ENCRYPTION_BLOCK for performance
+ * purposes. We assume that alignment to BLCKSZ implies alignment to
+ * ENCRYPTION_BLOCK.
+ */
+ Assert((file->curOffset % BLCKSZ == 0 &&
+ file->curOffset % ENCRYPTION_BLOCK == 0) ||
+ !data_encrypted);
+
+ /*
* Advance to next component file if necessary and possible.
*/
if (file->curOffset >= MAX_PHYSICAL_FILESIZE &&
@@ -423,6 +584,20 @@ BufFileLoadBuffer(BufFile *file)
}
/*
+ * See makeBufFile().
+ *
+ * Actually here we only handle the case of FileRead() returning zero
+ * bytes below. In contrast, if the buffer contains any data but it's not
+ * full, it should already have the trailing zeroes (encrypted) on disk.
+ * And as the encrypted buffer is always loaded in its entirety (i.e. EOF
+ * should only appear at buffer boundary if the data is encrypted), all
+ * unused bytes of the buffer should eventually be zeroes after the
+ * decryption.
+ */
+ if (data_encrypted)
+ MemSet(file->buffer.data, 0, BLCKSZ);
+
+ /*
* May need to reposition physical file.
*/
thisfile = file->files[file->curFile];
@@ -445,6 +620,64 @@ BufFileLoadBuffer(BufFile *file)
file->offsets[file->curFile] += file->nbytes;
/* we choose not to advance curOffset here */
+ if (data_encrypted && file->nbytes > 0)
+ {
+#ifdef USE_ENCRYPTION
+ char tweak[TWEAK_SIZE];
+ int nbytes = file->nbytes;
+
+ /*
+ * The encrypted component file can only consist of whole number of
+ * our encryption units. (Only the whole buffers are dumped / loaded.)
+ * The only exception is that we're at the end of segment file and
+ * found the word indicating the number of useful bytes in the
+ * segment. This can only happen for shared file.
+ */
+ if (nbytes % BLCKSZ != 0)
+ {
+ Assert(nbytes == sizeof(off_t) && file->fileset != NULL);
+
+ /*
+ * This special word cannot start within the useful data.
+ */
+ Assert(file->offsets[file->curFile] - nbytes >=
+ file->useful[file->curFile]);
+
+ /*
+ * This metadata his hidden to caller, so all he needs to know
+ * that there's no real data at the end of the file.
+ */
+ file->nbytes = 0;
+ return;
+ }
+
+ BufFileTweak(tweak, file);
+
+ /*
+ * The whole block is encrypted / decrypted at once as explained
+ * above.
+ */
+ decrypt_block(file->buffer.data, file->buffer.data, BLCKSZ, tweak);
+
+#ifdef USE_ASSERT_CHECKING
+
+ /*
+ * The unused part of the buffer which we've read from disk and
+ * decrypted should only contain zeroes, as explained in front of the
+ * MemSet() call.
+ */
+ {
+ int i;
+
+ for (i = file->nbytes; i < BLCKSZ; i++)
+ Assert(file->buffer.data[i] == 0);
+ }
+#endif /* USE_ASSERT_CHECKING */
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
+
if (file->nbytes > 0)
pgBufferUsage.temp_blks_read++;
}
@@ -455,6 +688,9 @@ BufFileLoadBuffer(BufFile *file)
* Dump buffer contents starting at curOffset.
* At call, should have dirty = true, nbytes > 0.
* On exit, dirty is cleared if successful write, and curOffset is advanced.
+ *
+ * XXX Consider separate function for encrypted buffer. (No loop is needed to
+ * dump the encrypted buffer.)
*/
static void
BufFileDumpBuffer(BufFile *file)
@@ -462,6 +698,52 @@ BufFileDumpBuffer(BufFile *file)
int wpos = 0;
int bytestowrite;
File thisfile;
+ char *write_ptr;
+
+ /*
+ * See comments in BufFileLoadBuffer();
+ */
+ Assert((file->curOffset % BLCKSZ == 0 &&
+ file->curOffset % ENCRYPTION_BLOCK == 0) ||
+ !data_encrypted);
+
+ /*
+ * Caller's responsibility.
+ */
+ Assert(file->pos <= file->nbytes);
+
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ char tweak[TWEAK_SIZE];
+
+ BufFileTweak(tweak, file);
+
+ /*
+ * The amount of data encrypted must be a multiple of
+ * ENCRYPTION_BLOCK. We meet this condition simply by encrypting the
+ * whole buffer.
+ *
+ * XXX Alternatively we could get the encrypted chunk length by
+ * rounding file->nbytes up to the nearest multiple of
+ * ENCRYPTION_BLOCK, and for decryption use the
+ * file->useful[file->curFile] value to find out how many blocks
+ * should be decrypted. That would reduce I/O if the buffer is mostly
+ * empty, but (BLCKSZ / ENCRYPTION_BLOCK) calls of encrypt_block()
+ * would be needed for full buffers. See also BufFileLoadBuffer() for
+ * explanation why we must stick on the unit of data amount encrypted
+ * / decrypted.
+ */
+ if (encryption_buf_size < BLCKSZ)
+ enlarge_encryption_buffer(BLCKSZ);
+ encrypt_block(file->buffer.data, encryption_buffer, BLCKSZ, tweak);
+ write_ptr = encryption_buffer;
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
+ else
+ write_ptr = file->buffer.data;
/*
* Unlike BufFileLoadBuffer, we must dump the whole buffer even if it
@@ -469,8 +751,6 @@ BufFileDumpBuffer(BufFile *file)
*/
while (wpos < file->nbytes)
{
- off_t availbytes;
-
/*
* Advance to next component file if necessary and possible.
*/
@@ -482,31 +762,56 @@ BufFileDumpBuffer(BufFile *file)
file->curOffset = 0L;
}
- /*
- * Determine how much we need to write into this file.
- */
- bytestowrite = file->nbytes - wpos;
- availbytes = MAX_PHYSICAL_FILESIZE - file->curOffset;
+ if (!data_encrypted)
+ {
+ off_t availbytes;
+
+ bytestowrite = file->nbytes - wpos;
+ availbytes = MAX_PHYSICAL_FILESIZE - file->curOffset;
- if ((off_t) bytestowrite > availbytes)
- bytestowrite = (int) availbytes;
+ if ((off_t) bytestowrite > availbytes)
+ bytestowrite = (int) availbytes;
+ }
+ else
+ {
+ /*
+ * This condition plus the alignment of curOffset to BLCKSZ
+ * (checked above) ensure that the encrypted buffer never crosses
+ * component file boundary.
+ */
+ StaticAssertStmt((MAX_PHYSICAL_FILESIZE % BLCKSZ) == 0,
+ "BLCKSZ is not whole multiple of MAX_PHYSICAL_FILESIZE");
+
+ /*
+ * Encrypted data is dumped all at once.
+ *
+ * Here we don't have to check availbytes because --- according to
+ * the assertions above --- currOffset should be lower than
+ * MAX_PHYSICAL_FILESIZE by non-zero multiple of BLCKSZ.
+ */
+ bytestowrite = BLCKSZ;
+ }
/*
* May need to reposition physical file.
*/
thisfile = file->files[file->curFile];
+
if (file->curOffset != file->offsets[file->curFile])
{
if (FileSeek(thisfile, file->curOffset, SEEK_SET) != file->curOffset)
return; /* seek failed, give up */
+
file->offsets[file->curFile] = file->curOffset;
}
bytestowrite = FileWrite(thisfile,
- file->buffer.data + wpos,
+ write_ptr + wpos,
bytestowrite,
WAIT_EVENT_BUFFILE_WRITE);
- if (bytestowrite <= 0)
+ if (bytestowrite <= 0 ||
+ (data_encrypted && bytestowrite != BLCKSZ))
return; /* failed to write */
+
file->offsets[file->curFile] += bytestowrite;
file->curOffset += bytestowrite;
wpos += bytestowrite;
@@ -515,25 +820,130 @@ BufFileDumpBuffer(BufFile *file)
}
file->dirty = false;
- /*
- * At this point, curOffset has been advanced to the end of the buffer,
- * ie, its original value + nbytes. We need to make it point to the
- * logical file position, ie, original value + pos, in case that is less
- * (as could happen due to a small backwards seek in a dirty buffer!)
- */
- file->curOffset -= (file->nbytes - file->pos);
- if (file->curOffset < 0) /* handle possible segment crossing */
+ if (!data_encrypted)
{
- file->curFile--;
- Assert(file->curFile >= 0);
- file->curOffset += MAX_PHYSICAL_FILESIZE;
+ /*
+ * At this point, curOffset has been advanced to the end of the
+ * buffer, ie, its original value + nbytes. We need to make it point
+ * to the logical file position, ie, original value + pos, in case
+ * that is less (as could happen due to a small backwards seek in a
+ * dirty buffer!)
+ */
+ file->curOffset -= (file->nbytes - file->pos);
+ if (file->curOffset < 0) /* handle possible segment crossing */
+ {
+ file->curFile--;
+ Assert(file->curFile >= 0);
+ file->curOffset += MAX_PHYSICAL_FILESIZE;
+ }
+
+ /*
+ * Now we can set the buffer empty without changing the logical
+ * position
+ */
+ file->pos = 0;
+ file->nbytes = 0;
}
+ else
+ {
+ /*
+ * curOffset should be at buffer boundary and buffer is the smallest
+ * I/O unit for encrypted data.
+ */
+ Assert(file->curOffset % BLCKSZ == 0);
- /*
- * Now we can set the buffer empty without changing the logical position
- */
- file->pos = 0;
- file->nbytes = 0;
+ /*
+ * The number of useful bytes needs to be written at the end of each
+ * encrypted segment of a shared file so that the other backends know
+ * how many bytes of the last buffer are useful.
+ */
+ if (file->fileset != NULL)
+ {
+ off_t useful;
+
+#ifdef USE_ENCRYPTION
+
+ /*
+ * nuseful may be increased earlier than numFiles but not later,
+ * so the corresponding entry should always exist in ->useful.
+ */
+ Assert(file->curFile < file->nuseful);
+
+ /*
+ * The number of useful bytes in the current segment file.
+ */
+ useful = file->useful[file->curFile];
+#endif
+
+ /*
+ * Have we dumped the last buffer of the segment, i.e. the one
+ * that can contain padding?
+ */
+ if (file->curOffset >= useful)
+ {
+ int bytes_extra;
+
+ /*
+ * Write the number of useful bytes in the segment.
+ */
+ bytes_extra = FileWrite(file->files[file->curFile],
+ (char *) &useful,
+ sizeof(useful),
+ WAIT_EVENT_BUFFILE_WRITE);
+ if (bytes_extra != sizeof(useful))
+ return; /* failed to write */
+
+ /*
+ * Remember the file position at OS level, but do not increase
+ * curOffset. That ensures that the next buffer appended will
+ * overwrite the "useful" value just written, instead of being
+ * appended to it.
+ *
+ * XXX The additional seek backwards at OS level when the next
+ * buffer is gonna be written is hard to avoid by writing this
+ * info higher in the stack (e.g. in BufFileWrite() or
+ * BufFileFlush()). Once we've cleared the "dirty" flag, we
+ * cannot recognize easily later that the number of useful
+ * bytes might need to be updated on disk because of this
+ * buffer. Is it worth introducing another bool field and
+ * making the logic trickier?
+ */
+ file->offsets[file->curFile] += bytes_extra;
+ }
+ }
+
+ if (file->pos >= BLCKSZ)
+ {
+ Assert(file->pos == BLCKSZ);
+
+ /*
+ * curOffset points to the beginning of the next buffer, so just
+ * reset pos and nbytes.
+ */
+ file->pos = 0;
+ file->nbytes = 0;
+
+ /* See makeBufFile() */
+ if (data_encrypted)
+ MemSet(file->buffer.data, 0, BLCKSZ);
+ }
+ else
+ {
+ /*
+ * Move curOffset to the beginning of the just-written buffer and
+ * preserve pos.
+ */
+ file->curOffset -= BLCKSZ;
+
+ /*
+ * At least pos bytes should be written even if the first change
+ * since now appears at pos == nbytes, but in fact the whole
+ * buffer will be written regardless pos. This is the price we pay
+ * for the choosing BLCKSZ as the I/O unit for encrypted data.
+ */
+ file->nbytes = BLCKSZ;
+ }
+ }
}
/*
@@ -556,18 +966,107 @@ BufFileRead(BufFile *file, void *ptr, size_t size)
while (size > 0)
{
+ int avail;
+
if (file->pos >= file->nbytes)
{
+ /*
+ * Neither read nor write nor seek should leave pos greater than
+ * nbytes, regardless the data is encrypted or not. pos can only
+ * be greater if nbytes is zero --- this situation can be caused
+ * by BufFileSeek().
+ */
+ Assert(file->pos == file->nbytes || file->nbytes == 0);
+
+ /*
+ * The Assert() above implies that pos is a whole multiple of
+ * BLCKSZ, so curOffset has meet the same encryption-specific
+ * requirement too.
+ */
+ Assert(file->curOffset % BLCKSZ == 0 || !data_encrypted);
+
/* Try to load more data into buffer. */
- file->curOffset += file->pos;
- file->pos = 0;
- file->nbytes = 0;
- BufFileLoadBuffer(file);
- if (file->nbytes <= 0)
- break; /* no more data available */
+ if (!data_encrypted || file->pos % BLCKSZ == 0)
+ {
+ file->curOffset += file->pos;
+ file->pos = 0;
+ file->nbytes = 0;
+ BufFileLoadBuffer(file);
+ if (file->nbytes <= 0)
+ break; /* no more data available */
+ }
+ else
+ {
+ int nbytes_orig = file->nbytes;
+
+ /*
+ * Given that BLCKSZ is the I/O unit for encrypted data (see
+ * comments in BufFileLoadBuffer()), we cannot add pos to
+ * curOffset because that would make it point outside block
+ * boundary. The only thing we can do is to reload the whole
+ * buffer and see if more data is eventually there than the
+ * previous load has fetched.
+ */
+ BufFileLoadBuffer(file);
+ if (file->nbytes <= nbytes_orig)
+ break; /* no more data available */
+ }
+ }
+
+ avail = file->nbytes;
+ nthistime = avail - file->pos;
+
+ /*
+ * The buffer can contain trailing zeroes because BLCKSZ is the I/O
+ * unit for encrypted data. These are not available for reading.
+ */
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ off_t useful = file->useful[file->curFile];
+
+ /*
+ * The criterion is whether the useful data end within the
+ * currently loaded buffer.
+ */
+ if (useful < file->curOffset + BLCKSZ)
+ {
+ /*
+ * Compute the number of bytes available in the current
+ * buffer.
+ */
+ avail = useful - file->curOffset;
+ Assert(avail >= 0);
+
+ /*
+ * An empty buffer can exist, e.g. after a seek to the end of
+ * the last component file.
+ */
+ if (avail == 0)
+ break;
+
+ /*
+ * Seek beyond the current EOF, which was not followed by
+ * write, could have resulted in position outside the useful
+ * data
+ */
+ if (file->pos > avail)
+ break;
+
+ nthistime = avail - file->pos;
+ Assert(nthistime >= 0);
+
+ /*
+ * Have we reached the end of the valid data?
+ */
+ if (nthistime == 0)
+ break;
+ }
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
}
- nthistime = file->nbytes - file->pos;
if (nthistime > size)
nthistime = size;
Assert(nthistime > 0);
@@ -609,11 +1108,28 @@ BufFileWrite(BufFile *file, void *ptr, size_t size)
}
else
{
- /* Hmm, went directly from reading to writing? */
+ /*
+ * Hmm, went directly from reading to writing?
+ *
+ * As pos should be exactly BLCKSZ, there is nothing special
+ * to do about data_encrypted. Except for zeroing the buffer.
+ */
+ Assert(file->pos == BLCKSZ);
+
file->curOffset += file->pos;
file->pos = 0;
file->nbytes = 0;
+
+ /* See makeBufFile() */
+ if (data_encrypted)
+ MemSet(file->buffer.data, 0, BLCKSZ);
}
+
+ /*
+ * If curOffset changed above, it should still meet the assumption
+ * that buffer is the I/O unit for encrypted data.
+ */
+ Assert(file->curOffset % BLCKSZ == 0 || !data_encrypted);
}
nthistime = BLCKSZ - file->pos;
@@ -627,6 +1143,65 @@ BufFileWrite(BufFile *file, void *ptr, size_t size)
file->pos += nthistime;
if (file->nbytes < file->pos)
file->nbytes = file->pos;
+
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ off_t new_offset;
+ int fileno = file->curFile;
+
+ /*
+ * curFile does not necessarily correspond to the offset: it can
+ * still have the initial value if BufFileSeek() skipped the
+ * previous file w/o dumping anything of it. While curFile will be
+ * fixed during the next dump, we need valid fileno now.
+ */
+ if (file->curOffset >= MAX_PHYSICAL_FILESIZE)
+ {
+ /*
+ * Even BufFileSeek() should not allow curOffset to become
+ * more than MAX_PHYSICAL_FILESIZE (if caller passes higher
+ * offset, curFile gets increased instead).
+ */
+ Assert(file->curOffset == MAX_PHYSICAL_FILESIZE);
+
+ fileno++;
+ }
+
+ /*
+ * fileno can point to a segment that does not exist on disk yet.
+ */
+ ensureUsefulArraySize(file, fileno + 1);
+
+ /*
+ * Update the "useful offset" of the underlying component file if
+ * we've added any useful data.
+ */
+ new_offset = file->curOffset + file->pos;
+
+ /*
+ * Make sure the offset is relative to the correct component file.
+ * It should have been adjusted during sequential write, but if
+ * we've just used BufFileSeek() to jump to segment boundary w/o
+ * writing, the value is relative to the start of the *previous*
+ * segment.
+ */
+ if (file->curOffset % MAX_PHYSICAL_FILESIZE == 0)
+ new_offset %= MAX_PHYSICAL_FILESIZE;
+
+ /*
+ * Adjust the number of useful bytes in the file if needed. This
+ * has to happen immediately, independent from
+ * BufFileDumpBuffer(), so that BufFileRead() works correctly
+ * anytime.
+ */
+ if (new_offset > file->useful[fileno])
+ file->useful[fileno] = new_offset;
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
+
ptr = (void *) ((char *) ptr + nthistime);
size -= nthistime;
nwritten += nthistime;
@@ -741,9 +1316,43 @@ BufFileSeek(BufFile *file, int fileno, off_t offset, int whence)
return EOF;
/* Seek is OK! */
file->curFile = newFile;
- file->curOffset = newOffset;
- file->pos = 0;
- file->nbytes = 0;
+ if (!data_encrypted)
+ {
+ file->curOffset = newOffset;
+ file->pos = 0;
+ file->nbytes = 0;
+ }
+ else
+ {
+ /*
+ * Offset of an encrypted buffer must be a multiple of BLCKSZ.
+ */
+ file->pos = newOffset % BLCKSZ;
+ file->curOffset = newOffset - file->pos;
+
+ /*
+ * BufFileLoadBuffer() will set nbytes iff it can read something.
+ */
+ file->nbytes = 0;
+
+ /*
+ * Load and decrypt the existing part of the buffer.
+ */
+ BufFileLoadBuffer(file);
+ if (file->nbytes == 0)
+ {
+ /*
+ * The data requested is not in the file, but this is not an
+ * error.
+ */
+ return 0;
+ }
+
+ /*
+ * The whole buffer should have been loaded.
+ */
+ Assert(file->nbytes == BLCKSZ);
+ }
return 0;
}
@@ -774,6 +1383,55 @@ BufFileSeekBlock(BufFile *file, long blknum)
SEEK_SET);
}
+#ifdef USE_ENCRYPTION
+static void
+BufFileTweak(char *tweak, BufFile *file)
+{
+ off_t block;
+ int curFile = file->curFile;
+
+ /*
+ * If the file was produced by BufFileAppend(), we need the original
+ * curFile, as it was used originally for encryption.
+ */
+ if (file->segnos)
+ curFile = file->segnos[curFile];
+
+ block = curFile * BUFFILE_SEG_SIZE + file->curOffset / BLCKSZ;
+
+ /*
+ * The unused bytes should always be defined.
+ */
+ memset(tweak, 0, TWEAK_SIZE);
+ StaticAssertStmt(sizeof(block) <= TWEAK_SIZE,
+ "block number does not fit into encryption tweak");
+ memcpy(tweak, &block, sizeof(off_t));
+}
+
+/*
+ * Make sure that BufFile.useful array has the required size.
+ */
+static void
+ensureUsefulArraySize(BufFile *file, int required)
+{
+ /*
+ * Does the array already have enough space?
+ */
+ if (required <= file->nuseful)
+ return;
+
+ /*
+ * It shouldn't be possible to jump beyond the end of the last segment,
+ * i.e. skip more than 1 segment.
+ */
+ Assert(file->nuseful + 1 == required);
+
+ file->useful = (off_t *) repalloc(file->useful, required * sizeof(off_t));
+ file->useful[file->nuseful] = 0L;
+ file->nuseful++;
+}
+#endif
+
#ifdef NOT_USED
/*
* BufFileTellBlock --- block-oriented tell
@@ -803,11 +1461,33 @@ BufFileSize(BufFile *file)
{
off_t lastFileSize;
- /* Get the size of the last physical file by seeking to end. */
- lastFileSize = FileSeek(file->files[file->numFiles - 1], 0, SEEK_END);
- if (lastFileSize < 0)
- return -1;
- file->offsets[file->numFiles - 1] = lastFileSize;
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ /*
+ * "useful" should be initialized even for shared file, see
+ * BufFileOpenShared().
+ */
+ Assert(file->useful != NULL && file->nuseful >= file->numFiles);
+
+ /*
+ * The number of useful bytes in the segment is what caller is
+ * interested in.
+ */
+ lastFileSize = file->useful[file->nuseful - 1];
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
+ else
+ {
+ /* Get the size of the last physical file by seeking to end. */
+ lastFileSize = FileSeek(file->files[file->numFiles - 1], 0, SEEK_END);
+ if (lastFileSize < 0)
+ return -1;
+
+ file->offsets[file->numFiles - 1] = lastFileSize;
+ }
return ((file->numFiles - 1) * (off_t) MAX_PHYSICAL_FILESIZE) +
lastFileSize;
@@ -837,6 +1517,9 @@ BufFileAppend(BufFile *target, BufFile *source)
{
long startBlock = target->numFiles * BUFFILE_SEG_SIZE;
int newNumFiles = target->numFiles + source->numFiles;
+#ifdef USE_ENCRYPTION
+ int newNUseful = target->nuseful + source->nuseful;
+#endif
int i;
Assert(target->fileset != NULL);
@@ -851,11 +1534,75 @@ BufFileAppend(BufFile *target, BufFile *source)
repalloc(target->files, sizeof(File) * newNumFiles);
target->offsets = (off_t *)
repalloc(target->offsets, sizeof(off_t) * newNumFiles);
+
for (i = target->numFiles; i < newNumFiles; i++)
{
- target->files[i] = source->files[i - target->numFiles];
- target->offsets[i] = source->offsets[i - target->numFiles];
+ int n = i - target->numFiles;
+
+ target->files[i] = source->files[n];
+ target->offsets[i] = source->offsets[n];
}
+
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ /*
+ * XXX As the typical use case is that parallel workers expose file to
+ * the leader, can we expect both target and source to have been
+ * exported, i.e. flushed? In such a case "nuseful" would have to be
+ * equal to "numFiles" for both input files and the code could get a
+ * bit simpler. It seems that at least source should be flushed, as
+ * source->readOnly is expected to be true above.
+ */
+ target->useful = (off_t *)
+ repalloc(target->useful, sizeof(off_t) * newNUseful);
+
+ for (i = target->nuseful; i < newNUseful; i++)
+ target->useful[i] = source->useful[i - target->nuseful];
+ target->nuseful = newNUseful;
+
+ /*
+ * File segments can appear at different position due to
+ * concatenation, so make sure we remember the original positions for
+ * the sake of encryption tweak.
+ */
+ if (target->segnos == NULL)
+ {
+ /*
+ * If the target does not have the array yet, allocate it for both
+ * target and source and initialize the target part.
+ */
+ target->segnos = (off_t *) palloc(newNumFiles * sizeof(off_t));
+ for (i = 0; i < target->numFiles; i++)
+ target->segnos[i] = i;
+ }
+ else
+ {
+ /*
+ * Use the existing target part and add space for the source part.
+ */
+ target->segnos = (off_t *) repalloc(target->segnos,
+ newNumFiles * sizeof(off_t));
+ }
+
+ /*
+ * The source segment number either equals to (0-based) index of the
+ * segment, or to an element of an already existing array.
+ */
+ for (i = target->numFiles; i < newNumFiles; i++)
+ {
+ off_t segno = i - target->numFiles;
+
+ if (source->segnos == NULL)
+ target->segnos[i] = segno;
+ else
+ target->segnos[i] = source->segnos[segno];
+ }
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
+
target->numFiles = newNumFiles;
return startBlock;
diff --git a/src/backend/storage/file/copydir.c b/src/backend/storage/file/copydir.c
index 4a0d23b11e..eb4ad205e1 100644
--- a/src/backend/storage/file/copydir.c
+++ b/src/backend/storage/file/copydir.c
@@ -23,24 +23,30 @@
#include <sys/stat.h>
#include "storage/copydir.h"
+#include "storage/encryption.h"
#include "storage/fd.h"
+#include "storage/reinit.h"
+#include "storage/smgr.h"
#include "miscadmin.h"
#include "pgstat.h"
/*
* copydir: copy a directory
*
- * If recurse is false, subdirectories are ignored. Anything that's not
- * a directory or a regular file is ignored.
+ * RelFileNode values must specify tablespace and database oids for source
+ * and target to support re-encryption if necessary. relNode value in provided
+ * structs will be clobbered.
*/
void
-copydir(char *fromdir, char *todir, bool recurse)
+copydir(char *fromdir, char *todir, RelFileNode *fromNode, RelFileNode *toNode)
{
DIR *xldir;
struct dirent *xlde;
char fromfile[MAXPGPATH * 2];
char tofile[MAXPGPATH * 2];
+ Assert(!data_encrypted || (fromNode != NULL && toNode != NULL));
+
if (MakePGDirectory(todir) != 0)
ereport(ERROR,
(errcode_for_file_access(),
@@ -67,14 +73,32 @@ copydir(char *fromdir, char *todir, bool recurse)
(errcode_for_file_access(),
errmsg("could not stat file \"%s\": %m", fromfile)));
- if (S_ISDIR(fst.st_mode))
+ if (S_ISREG(fst.st_mode))
{
- /* recurse to handle subdirectories */
- if (recurse)
- copydir(fromfile, tofile, true);
+ int oidchars;
+ ForkNumber forkNum;
+ int segment;
+
+ /*
+ * For encrypted databases we need to reencrypt files with new
+ * tweaks.
+ */
+ if (data_encrypted &&
+ parse_filename_for_nontemp_relation(xlde->d_name,
+ &oidchars, &forkNum, &segment))
+ {
+ char oidbuf[OIDCHARS + 1];
+
+ memcpy(oidbuf, xlde->d_name, oidchars);
+ oidbuf[oidchars] = '\0';
+
+ /* We scribble over the provided RelFileNodes here */
+ fromNode->relNode = toNode->relNode = atol(oidbuf);
+ copy_file(fromfile, tofile, fromNode, toNode, forkNum, forkNum, segment);
+ }
+ else
+ copy_file(fromfile, tofile, NULL, NULL, 0, 0, 0);
}
- else if (S_ISREG(fst.st_mode))
- copy_file(fromfile, tofile);
}
FreeDir(xldir);
@@ -121,17 +145,24 @@ copydir(char *fromdir, char *todir, bool recurse)
}
/*
- * copy one file
+ * copy one file. If decryption and reencryption may be needed specify
+ * relfilenodes for source and target.
*/
void
-copy_file(char *fromfile, char *tofile)
+copy_file(char *fromfile, char *tofile, RelFileNode *fromNode,
+ RelFileNode *toNode, ForkNumber fromForkNum, ForkNumber toForkNum,
+ int segment)
{
char *buffer;
int srcfd;
int dstfd;
int nbytes;
+ int bytesread;
off_t offset;
off_t flush_offset;
+#ifdef USE_ENCRYPTION
+ BlockNumber blockNum = segment * RELSEG_SIZE;
+#endif
/* Size of copy buffer (read and write requests) */
#define COPY_BUF_SIZE (8 * BLCKSZ)
@@ -186,16 +217,55 @@ copy_file(char *fromfile, char *tofile)
flush_offset = offset;
}
- pgstat_report_wait_start(WAIT_EVENT_COPY_FILE_READ);
- nbytes = read(srcfd, buffer, COPY_BUF_SIZE);
- pgstat_report_wait_end();
- if (nbytes < 0)
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not read file \"%s\": %m", fromfile)));
+ /*
+ * Try to read as much as we fit in the buffer so we can deal with
+ * complete blocks if we need to reencrypt.
+ */
+ nbytes = 0;
+ while (nbytes < COPY_BUF_SIZE)
+ {
+ pgstat_report_wait_start(WAIT_EVENT_COPY_FILE_READ);
+ bytesread = read(srcfd, buffer + nbytes, COPY_BUF_SIZE - nbytes);
+ pgstat_report_wait_end();
+ if (bytesread < 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read file \"%s\": %m", fromfile)));
+ nbytes += bytesread;
+ if (bytesread == 0)
+ break;
+ }
if (nbytes == 0)
break;
+
+ /*
+ * If encryption is in place, only complete blocks should have been
+ * read. Thus we do not have to care whether any encryption block is
+ * affected by the previous buffer contents.
+ */
+ Assert(nbytes % BLCKSZ == 0 || (fromNode == NULL && toNode == NULL));
+
errno = 0;
+
+ /*
+ * If the database is encrypted we need to decrypt the data here and
+ * reencrypt it to adjust the tweak values of blocks.
+ */
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ if (fromNode != NULL)
+ {
+ Assert(toNode != NULL);
+ blockNum = ReencryptBlock(buffer, nbytes / BLCKSZ,
+ fromNode, toNode, fromForkNum,
+ toForkNum, blockNum);
+ }
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
+
pgstat_report_wait_start(WAIT_EVENT_COPY_FILE_WRITE);
if ((int) write(dstfd, buffer, nbytes) != nbytes)
{
diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c
index 8dd51f1767..b3e3a50d30 100644
--- a/src/backend/storage/file/fd.c
+++ b/src/backend/storage/file/fd.c
@@ -2163,10 +2163,6 @@ FileSeek(File file, off_t offset, int whence)
return vfdP->seekPos;
}
-/*
- * XXX not actually used but here for completeness
- */
-#ifdef NOT_USED
off_t
FileTell(File file)
{
@@ -2175,7 +2171,6 @@ FileTell(File file)
file, VfdCache[file].fileName));
return VfdCache[file].seekPos;
}
-#endif
int
FileTruncate(File file, off_t offset, uint32 wait_event_info)
diff --git a/src/backend/storage/file/reinit.c b/src/backend/storage/file/reinit.c
index 74ff6c359b..70e138503e 100644
--- a/src/backend/storage/file/reinit.c
+++ b/src/backend/storage/file/reinit.c
@@ -16,6 +16,7 @@
#include <unistd.h>
+#include "catalog/pg_tablespace.h"
#include "common/relpath.h"
#include "storage/copydir.h"
#include "storage/fd.h"
@@ -24,9 +25,9 @@
#include "utils/memutils.h"
static void ResetUnloggedRelationsInTablespaceDir(const char *tsdirname,
- int op);
+ int op, Oid spcOid);
static void ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname,
- int op);
+ int op, Oid spcOid, Oid dbOid);
typedef struct
{
@@ -68,7 +69,7 @@ ResetUnloggedRelations(int op)
/*
* First process unlogged files in pg_default ($PGDATA/base)
*/
- ResetUnloggedRelationsInTablespaceDir("base", op);
+ ResetUnloggedRelationsInTablespaceDir("base", op, DEFAULTTABLESPACE_OID);
/*
* Cycle through directories for all non-default tablespaces.
@@ -77,13 +78,16 @@ ResetUnloggedRelations(int op)
while ((spc_de = ReadDir(spc_dir, "pg_tblspc")) != NULL)
{
+ Oid spcOid;
+
if (strcmp(spc_de->d_name, ".") == 0 ||
strcmp(spc_de->d_name, "..") == 0)
continue;
snprintf(temp_path, sizeof(temp_path), "pg_tblspc/%s/%s",
spc_de->d_name, TABLESPACE_VERSION_DIRECTORY);
- ResetUnloggedRelationsInTablespaceDir(temp_path, op);
+ spcOid = atoi(spc_de->d_name);
+ ResetUnloggedRelationsInTablespaceDir(temp_path, op, spcOid);
}
FreeDir(spc_dir);
@@ -99,7 +103,8 @@ ResetUnloggedRelations(int op)
* Process one tablespace directory for ResetUnloggedRelations
*/
static void
-ResetUnloggedRelationsInTablespaceDir(const char *tsdirname, int op)
+ResetUnloggedRelationsInTablespaceDir(const char *tsdirname, int op,
+ Oid spcOid)
{
DIR *ts_dir;
struct dirent *de;
@@ -126,6 +131,8 @@ ResetUnloggedRelationsInTablespaceDir(const char *tsdirname, int op)
while ((de = ReadDir(ts_dir, tsdirname)) != NULL)
{
+ Oid dbOid;
+
/*
* We're only interested in the per-database directories, which have
* numeric names. Note that this code will also (properly) ignore "."
@@ -134,9 +141,10 @@ ResetUnloggedRelationsInTablespaceDir(const char *tsdirname, int op)
if (strspn(de->d_name, "0123456789") != strlen(de->d_name))
continue;
+ dbOid = atoi(de->d_name);
snprintf(dbspace_path, sizeof(dbspace_path), "%s/%s",
tsdirname, de->d_name);
- ResetUnloggedRelationsInDbspaceDir(dbspace_path, op);
+ ResetUnloggedRelationsInDbspaceDir(dbspace_path, op, spcOid, dbOid);
}
FreeDir(ts_dir);
@@ -146,7 +154,8 @@ ResetUnloggedRelationsInTablespaceDir(const char *tsdirname, int op)
* Process one per-dbspace directory for ResetUnloggedRelations
*/
static void
-ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op)
+ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op,
+ Oid spcOid, Oid dbOid)
{
DIR *dbspace_dir;
struct dirent *de;
@@ -187,7 +196,7 @@ ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op)
/* Skip anything that doesn't look like a relation data file. */
if (!parse_filename_for_nontemp_relation(de->d_name, &oidchars,
- &forkNum))
+ &forkNum, NULL))
continue;
/* Also skip it unless this is the init fork. */
@@ -229,7 +238,7 @@ ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op)
/* Skip anything that doesn't look like a relation data file. */
if (!parse_filename_for_nontemp_relation(de->d_name, &oidchars,
- &forkNum))
+ &forkNum, NULL))
continue;
/* We never remove the init fork. */
@@ -279,13 +288,14 @@ ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op)
{
ForkNumber forkNum;
int oidchars;
+ int segment;
char oidbuf[OIDCHARS + 1];
char srcpath[MAXPGPATH * 2];
char dstpath[MAXPGPATH];
/* Skip anything that doesn't look like a relation data file. */
if (!parse_filename_for_nontemp_relation(de->d_name, &oidchars,
- &forkNum))
+ &forkNum, &segment))
continue;
/* Also skip it unless this is the init fork. */
@@ -305,7 +315,13 @@ ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op)
/* OK, we're ready to perform the actual copy. */
elog(DEBUG2, "copying %s to %s", srcpath, dstpath);
- copy_file(srcpath, dstpath);
+ {
+ RelFileNode srcNode = {spcOid, dbOid, atol(oidbuf)};
+ RelFileNode dstNode = srcNode;
+
+ copy_file(srcpath, dstpath, &srcNode, &dstNode,
+ INIT_FORKNUM, MAIN_FORKNUM, segment);
+ }
}
FreeDir(dbspace_dir);
@@ -327,7 +343,7 @@ ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op)
/* Skip anything that doesn't look like a relation data file. */
if (!parse_filename_for_nontemp_relation(de->d_name, &oidchars,
- &forkNum))
+ &forkNum, NULL))
continue;
/* Also skip it unless this is the init fork. */
@@ -372,9 +388,10 @@ ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op)
*/
bool
parse_filename_for_nontemp_relation(const char *name, int *oidchars,
- ForkNumber *fork)
+ ForkNumber *fork, int *segment)
{
int pos;
+ int segstart = 0;
/* Look for a non-empty string of digits (that isn't too long). */
for (pos = 0; isdigit((unsigned char) name[pos]); ++pos)
@@ -401,6 +418,7 @@ parse_filename_for_nontemp_relation(const char *name, int *oidchars,
{
int segchar;
+ segstart = pos + 1;
for (segchar = 1; isdigit((unsigned char) name[pos + segchar]); ++segchar)
;
if (segchar <= 1)
@@ -411,5 +429,14 @@ parse_filename_for_nontemp_relation(const char *name, int *oidchars,
/* Now we should be at the end. */
if (name[pos] != '\0')
return false;
+
+ if (segment != NULL)
+ {
+ if (segstart == 0)
+ *segment = 0;
+ else
+ *segment = atoi(name + segstart);
+ }
+
return true;
}
diff --git a/src/backend/storage/page/bufpage.c b/src/backend/storage/page/bufpage.c
index dfbda5458f..1c3d9463d4 100644
--- a/src/backend/storage/page/bufpage.c
+++ b/src/backend/storage/page/bufpage.c
@@ -17,6 +17,7 @@
#include "access/htup_details.h"
#include "access/itup.h"
#include "access/xlog.h"
+#include "common/string.h"
#include "storage/checksum.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
@@ -81,11 +82,8 @@ bool
PageIsVerified(Page page, BlockNumber blkno)
{
PageHeader p = (PageHeader) page;
- size_t *pagebytes;
- int i;
bool checksum_failure = false;
bool header_sane = false;
- bool all_zeroes = false;
uint16 checksum = 0;
/*
@@ -118,26 +116,8 @@ PageIsVerified(Page page, BlockNumber blkno)
return true;
}
- /*
- * Check all-zeroes case. Luckily BLCKSZ is guaranteed to always be a
- * multiple of size_t - and it's much faster to compare memory using the
- * native word size.
- */
- StaticAssertStmt(BLCKSZ == (BLCKSZ / sizeof(size_t)) * sizeof(size_t),
- "BLCKSZ has to be a multiple of sizeof(size_t)");
-
- all_zeroes = true;
- pagebytes = (size_t *) page;
- for (i = 0; i < (BLCKSZ / sizeof(size_t)); i++)
- {
- if (pagebytes[i] != 0)
- {
- all_zeroes = false;
- break;
- }
- }
-
- if (all_zeroes)
+ /* Check all-zeroes case */
+ if (IsAllZero((char *) page, BLCKSZ))
return true;
/*
diff --git a/src/backend/storage/smgr/Makefile b/src/backend/storage/smgr/Makefile
index 2b95cb0df1..0601ea7894 100644
--- a/src/backend/storage/smgr/Makefile
+++ b/src/backend/storage/smgr/Makefile
@@ -12,6 +12,6 @@ subdir = src/backend/storage/smgr
top_builddir = ../../../..
include $(top_builddir)/src/Makefile.global
-OBJS = md.o smgr.o smgrtype.o
+OBJS = encryption.o md.o smgr.o smgrtype.o
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/smgr/encryption.c b/src/backend/storage/smgr/encryption.c
new file mode 100644
index 0000000000..9adea3959e
--- /dev/null
+++ b/src/backend/storage/smgr/encryption.c
@@ -0,0 +1,814 @@
+/*-------------------------------------------------------------------------
+ *
+ * encryption.c
+ * This code handles encryption and decryption of data.
+ *
+ * Encryption is done by extension modules loaded by encryption_library GUC.
+ * The extension module must register itself and provide a cryptography
+ * implementation. Key setup is left to the extension module.
+ *
+ *
+ * Copyright (c) 2016, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/storage/smgr/encryption.c
+ *
+ * NOTES
+ * This file is compiled as both front-end and backend code, so it
+ * may not use ereport, server-defined static variables, etc.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include <unistd.h>
+
+#include "common/fe_memutils.h"
+#include "common/sha2.h"
+#include "common/string.h"
+#include "catalog/pg_control.h"
+#ifndef FRONTEND
+#include "pgstat.h"
+#endif /* FRONTEND */
+#include "storage/bufpage.h"
+#include "storage/encryption.h"
+#include "storage/fd.h"
+#include "utils/memutils.h"
+#include "miscadmin.h"
+#include "fmgr.h"
+#include "port.h"
+
+#ifdef USE_ENCRYPTION
+#include <openssl/conf.h>
+#include <openssl/evp.h>
+#include <openssl/err.h>
+#endif
+
+#ifdef USE_ENCRYPTION
+unsigned char *encryption_key = NULL;
+const char *encryption_key_prefix = "encryption_key=";
+const char *encryption_pwd_prefix = "encryption_password=";
+#endif
+
+bool data_encrypted = false;
+
+#ifdef USE_ENCRYPTION
+char *encryption_key_command = NULL;
+char *encryption_buffer = NULL;
+Size encryption_buf_size = 0;
+
+static bool initialized = false;
+
+static void setup_encryption_internal(void);
+static char *run_encryption_key_command(bool *is_key_p, size_t *len_p);
+static void evp_error(void);
+
+/*
+ * Pointer to the KDF parameters.
+ *
+ * XXX Rename this and the write / read functions so they contain the
+ * 'keysetup' string?
+ */
+KDFParamsData *KDFParams = NULL;
+
+/*
+ * Initialize KDFParamsData and write it to a file.
+ *
+ * This is very similar to WriteControlFile().
+ */
+#ifndef FRONTEND
+extern void
+write_kdf_file(void)
+{
+ KDFParamsPBKDF2 *params;
+ int i,
+ fd;
+
+ StaticAssertStmt(sizeof(KDFParamsData) <= KDF_PARAMS_FILE_SIZE,
+ "kdf file is too large for atomic disk writes");
+
+ /*
+ * The initialization should not be repeated.
+ */
+ Assert(KDFParams == NULL);
+
+ KDFParams = MemoryContextAllocZero(TopMemoryContext,
+ KDF_PARAMS_FILE_SIZE);
+ KDFParams->function = KDF_OPENSSL_PKCS5_PBKDF2_HMAC_SHA;
+ params = &KDFParams->data.pbkdf2;
+ params->niter = ENCRYPTION_KDF_NITER;
+ for (i = 0; i < ENCRYPTION_KDF_SALT_LEN; i++)
+ params->salt[i] = (unsigned char) random();
+
+ /* Contents are protected with a CRC */
+ INIT_CRC32C(KDFParams->crc);
+ COMP_CRC32C(KDFParams->crc,
+ (char *) KDFParams,
+ offsetof(KDFParamsData, crc));
+ FIN_CRC32C(KDFParams->crc);
+
+ fd = BasicOpenFile(KDF_PARAMS_FILE,
+ O_RDWR | O_CREAT | O_EXCL | PG_BINARY);
+ if (fd < 0)
+ ereport(PANIC,
+ (errcode_for_file_access(),
+ errmsg("could not create key setup file \"%s\": %m",
+ KDF_PARAMS_FILE)));
+
+ pgstat_report_wait_start(WAIT_EVENT_KDF_FILE_WRITE);
+ if (write(fd, KDFParams, KDF_PARAMS_FILE_SIZE) != KDF_PARAMS_FILE_SIZE)
+ {
+ /* if write didn't set errno, assume problem is no disk space */
+ if (errno == 0)
+ errno = ENOSPC;
+ ereport(PANIC,
+ (errcode_for_file_access(),
+ errmsg("could not write to key setup file: %m")));
+ }
+ pgstat_report_wait_end();
+
+ pgstat_report_wait_start(WAIT_EVENT_KDF_FILE_SYNC);
+ if (pg_fsync(fd) != 0)
+ ereport(PANIC,
+ (errcode_for_file_access(),
+ errmsg("could not fsync key setup file: %m")));
+ pgstat_report_wait_end();
+
+ if (close(fd))
+ ereport(PANIC,
+ (errcode_for_file_access(),
+ errmsg("could not close key setup file: %m")));
+}
+
+/*
+ * Read KDFParamsData from file and store it in local memory.
+ *
+ * postmaster should call the function early enough for any other process to
+ * inherit valid pointer to the data.
+ */
+extern void
+read_kdf_file(void)
+{
+ pg_crc32c crc;
+ int fd;
+
+ KDFParams = MemoryContextAllocZero(TopMemoryContext,
+ KDF_PARAMS_FILE_SIZE);
+
+ fd = BasicOpenFile(KDF_PARAMS_FILE, O_RDONLY | PG_BINARY);
+
+ if (fd < 0)
+ {
+ ereport(PANIC,
+ (errcode_for_file_access(),
+ errmsg("could not open key setup file \"%s\": %m",
+ KDF_PARAMS_FILE)));
+ }
+
+ pgstat_report_wait_start(WAIT_EVENT_KDF_FILE_READ);
+
+ if (read(fd, KDFParams, sizeof(KDFParamsData)) != sizeof(KDFParamsData))
+ {
+ ereport(PANIC,
+ (errcode_for_file_access(),
+ errmsg("could not read from key setup file: %m")));
+ }
+
+ pgstat_report_wait_end();
+
+ close(fd);
+
+ /* Now check the CRC. */
+ INIT_CRC32C(crc);
+ COMP_CRC32C(crc,
+ (char *) KDFParams,
+ offsetof(KDFParamsData, crc));
+ FIN_CRC32C(crc);
+
+ if (!EQ_CRC32C(crc, KDFParams->crc))
+ ereport(FATAL,
+ (errmsg("incorrect checksum in key setup file")));
+
+ if (KDFParams->function != KDF_OPENSSL_PKCS5_PBKDF2_HMAC_SHA)
+ ereport(FATAL,
+ (errmsg("unsupported KDF function %d", KDFParams->function)));
+}
+#endif /* FRONTEND */
+#endif /* USE_ENCRYPTION */
+
+/*
+ * Encrypts a fixed value into *buf to verify that encryption key is correct.
+ * Caller provided buf needs to be able to hold at least ENCRYPTION_SAMPLE_SIZE
+ * bytes.
+ */
+void
+sample_encryption(char *buf)
+{
+#ifdef USE_ENCRYPTION
+ char tweak[TWEAK_SIZE];
+ int i;
+
+ for (i = 0; i < TWEAK_SIZE; i++)
+ tweak[i] = i;
+
+ encrypt_block("postgresqlcrypt", buf, ENCRYPTION_SAMPLE_SIZE, tweak);
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+}
+
+/*
+ * Encrypts one block of data with a specified tweak value. May only be called
+ * when encryption_enabled is true.
+ *
+ * Input and output buffer may point to the same location.
+ *
+ * "size" must be a (non-zero) multiple of ENCRYPTION_BLOCK.
+ *
+ * "tweak" value must be TWEAK_SIZE bytes long.
+ *
+ * All-zero blocks are not encrypted to correctly handle relation extension,
+ * and also to simplify handling of holes created by seek past EOF and
+ * consequent write (see buffile.c).
+ */
+void
+encrypt_block(const char *input, char *output, Size size, const char *tweak)
+{
+#ifdef USE_ENCRYPTION
+ Assert(size >= ENCRYPTION_BLOCK && size % ENCRYPTION_BLOCK == 0);
+ Assert(initialized);
+
+ /*
+ * The EVP API does not seem to expect the output buffer to be equal to
+ * the input. Ensure that we pass separate pointers.
+ */
+ if (input == output)
+ {
+ if (size > encryption_buf_size)
+ enlarge_encryption_buffer(size);
+
+ memcpy(encryption_buffer, input, size);
+ input = encryption_buffer;
+ }
+
+ if (IsAllZero(input, size))
+ memset(output, 0, size);
+ else
+ {
+ int out_size;
+ EVP_CIPHER_CTX *ctx;
+
+ if ((ctx = EVP_CIPHER_CTX_new()) == NULL)
+ evp_error();
+
+ if (EVP_EncryptInit_ex(ctx, EVP_aes_256_xts(), NULL, encryption_key,
+ (unsigned char *) tweak) != 1)
+ evp_error();
+
+ /*
+ * No padding is needed, the input block size should already be a
+ * multiple of ENCRYPTION_BLOCK_OPENSSL.
+ */
+ EVP_CIPHER_CTX_set_padding(ctx, 0);
+
+ Assert(EVP_CIPHER_CTX_block_size(ctx) == ENCRYPTION_BLOCK_OPENSSL);
+ Assert(EVP_CIPHER_CTX_iv_length(ctx) == TWEAK_SIZE);
+ Assert(EVP_CIPHER_CTX_key_length(ctx) == ENCRYPTION_KEY_LENGTH);
+
+ /*
+ * Do the actual encryption. As the padding is disabled,
+ * EVP_EncryptFinal_ex() won't be needed.
+ */
+ if (EVP_EncryptUpdate(ctx, (unsigned char *) output, &out_size,
+ (unsigned char *) input, size) != 1)
+ evp_error();
+
+ /*
+ * The input size is a multiple of ENCRYPTION_BLOCK_OPENSSL, so the
+ * output of AES-XTS should meet this condition.
+ */
+ Assert(out_size == size);
+
+ if (EVP_CIPHER_CTX_cleanup(ctx) != 1)
+ evp_error();
+ }
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+}
+
+/*
+ * Decrypts one block of data with a specified tweak value. May only be called
+ * when encryption_enabled is true.
+ *
+ * Input and output buffer may point to the same location.
+ *
+ * "size" must be a (non-zero) multiple of ENCRYPTION_BLOCK.
+ *
+ * "tweak" value must be TWEAK_SIZE bytes long.
+ *
+ * All-zero blocks are not decrypted to correctly handle relation extension,
+ * and also to simplify handling of holes created by seek past EOF and
+ * consequent write (see buffile.c).
+ */
+void
+decrypt_block(const char *input, char *output, Size size, const char *tweak)
+{
+#ifdef USE_ENCRYPTION
+ Assert(size >= ENCRYPTION_BLOCK && size % ENCRYPTION_BLOCK == 0);
+ Assert(initialized);
+
+ /*
+ * The EVP API does not seem to expect the output buffer to be equal to
+ * the input. Ensure that we pass separate pointers.
+ */
+ if (input == output)
+ {
+ if (size > encryption_buf_size)
+ enlarge_encryption_buffer(size);
+
+ memcpy(encryption_buffer, input, size);
+ input = encryption_buffer;
+ }
+
+ if (IsAllZero(input, size))
+ memset(output, 0, size);
+ else
+ {
+ int out_size;
+ EVP_CIPHER_CTX *ctx;
+
+ if ((ctx = EVP_CIPHER_CTX_new()) == NULL)
+ evp_error();
+
+ if (EVP_DecryptInit_ex(ctx, EVP_aes_256_xts(), NULL, encryption_key,
+ (unsigned char *) tweak) != 1)
+ evp_error();
+
+ /* The same considerations apply below as those in encrypt_block(). */
+ EVP_CIPHER_CTX_set_padding(ctx, 0);
+ Assert(EVP_CIPHER_CTX_block_size(ctx) == ENCRYPTION_BLOCK_OPENSSL);
+ Assert(EVP_CIPHER_CTX_iv_length(ctx) == TWEAK_SIZE);
+ Assert(EVP_CIPHER_CTX_key_length(ctx) == ENCRYPTION_KEY_LENGTH);
+
+ if (EVP_DecryptUpdate(ctx, (unsigned char *) output, &out_size,
+ (unsigned char *) input, size) != 1)
+ evp_error();
+
+ Assert(out_size == size);
+
+ if (EVP_CIPHER_CTX_cleanup(ctx) != 1)
+ evp_error();
+ }
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+}
+
+/*
+ * Report an error in an universal way so that caller does not have to care
+ * whether it executes in backend or front-end.
+ */
+void
+encryption_error(bool fatal, char *message)
+{
+#ifndef FRONTEND
+ elog(fatal ? FATAL : INFO, "%s", message);
+#else
+ fprintf(stderr, "%s\n", message);
+ if (fatal)
+ exit(EXIT_FAILURE);
+#endif
+}
+
+/*
+ * Initialize encryption subsystem for use. Must be called before any
+ * encryptable data is read from or written to data directory.
+ */
+void
+setup_encryption(bool bootstrap)
+{
+#ifdef USE_ENCRYPTION
+ char *credentials;
+ bool is_key;
+ size_t len;
+
+ credentials = run_encryption_key_command(&is_key, &len);
+
+ /*
+ * Setup KDF if we need to derive the key from a password.
+ *
+ * Front-ends always need the key because some of them are not aware of
+ * the data directory and thus they'd need one more command line option to
+ * find the key setup file.
+ */
+#ifndef FRONTEND
+ if (!is_key)
+ {
+ if (bootstrap)
+ {
+ write_kdf_file();
+ }
+ else
+ read_kdf_file();
+ }
+#endif /* FRONTEND */
+
+ setup_encryption_key(credentials, is_key, len);
+ pfree(credentials);
+ setup_encryption_internal();
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+}
+
+#ifdef USE_ENCRYPTION
+static void
+setup_encryption_internal(void)
+{
+ /*
+ * Setup OpenSSL.
+ *
+ * None of these functions should return a value or raise error.
+ */
+ ERR_load_crypto_strings();
+ OpenSSL_add_all_algorithms();
+ OPENSSL_config(NULL);
+
+ /*
+ * It makes no sense to initialize the encryption multiple times.
+ */
+ Assert(!initialized);
+
+ initialized = true;
+}
+#endif /* USE_ENCRYPTION */
+
+/*
+ * If credentials is a key, just copy it to encryption_key. If it's a
+ * password, derive the key from it.
+ */
+void
+setup_encryption_key(char *credentials, bool is_key, size_t len)
+{
+#ifdef USE_ENCRYPTION
+ Assert(credentials != NULL);
+ Assert(encryption_key == NULL);
+
+ /*
+ * Although the function should be called in the TopMemoryContext, we
+ * should be pretty sure that the key does not become garbage due to
+ * pfree(). The cluster should crash in such a case, but if it did not
+ * happen immediately, some data could be encrypted using an invalid key
+ * and therefore become lost.
+ */
+#ifndef FRONTEND
+ encryption_key = (unsigned char *) MemoryContextAllocZero(TopMemoryContext,
+ ENCRYPTION_KEY_LENGTH);
+#else
+ encryption_key = (unsigned char *) palloc(ENCRYPTION_KEY_LENGTH);
+#endif /* FRONTEND */
+ if (is_key)
+ {
+ Assert(len = ENCRYPTION_KEY_LENGTH);
+ memcpy(encryption_key, credentials, len);
+ }
+ else
+ {
+ KDFParamsPBKDF2 *params;
+ int rc;
+
+ /*
+ * The file contains password so we need the KDF parameters to turn it
+ * to key.
+ */
+ if (KDFParams == NULL)
+ {
+#ifndef FRONTEND
+ ereport(FATAL,
+ (errmsg("this instance does not accept encryption password"),
+ errdetail("Encryption key was probably used to initialize the instance.")));
+#else
+ encryption_error(true,
+ "this instance does not accept encryption password.\n"
+ "Encryption key was probably used to initialize the instance.\n");
+#endif /* FRONTEND */
+ }
+
+ /*
+ * Turn the password into the encryption key.
+ */
+ params = &KDFParams->data.pbkdf2;
+ rc = PKCS5_PBKDF2_HMAC(credentials,
+ len,
+ params->salt,
+ ENCRYPTION_KDF_SALT_LEN,
+ params->niter,
+ EVP_sha(),
+ ENCRYPTION_KEY_LENGTH,
+ encryption_key);
+
+ if (rc != 1)
+ {
+#ifndef FRONTEND
+ ereport(FATAL,
+ (errmsg("failed to derive key from password")));
+#else
+ encryption_error(true, "failed to derive key from password");
+#endif /* FRONTEND */
+ }
+ }
+
+#else /* USE_ENCRYPTION */
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+}
+
+#ifdef USE_ENCRYPTION
+void
+enlarge_encryption_buffer(Size new_size)
+{
+ Assert(new_size > 0);
+
+ /*
+ * Shrinkage is not the use case for this routine.
+ */
+ if (new_size <= encryption_buf_size)
+ return;
+
+ /*
+ * Allocate a new chunk if nothing is there yet, else reallocate the
+ * existing one.
+ */
+ if (encryption_buf_size == 0)
+#ifndef FRONTEND
+ encryption_buffer = (char *) MemoryContextAlloc(TopMemoryContext,
+ new_size);
+#else
+ encryption_buffer = (char *) palloc(new_size);
+#endif /* FRONTEND */
+ else
+ encryption_buffer = (char *) repalloc(encryption_buffer, new_size);
+ encryption_buf_size = new_size;
+}
+
+/*
+ * Run the command stored in encryption_key_command and return the key or
+ * password.
+ *
+ * *is_key_p receives true if the command returns key, false if it's
+ * password. *len_p receives length of the data.
+ */
+static char *
+run_encryption_key_command(bool *is_key_p, size_t *len_p)
+{
+ FILE *fp;
+ char *buf,
+ *result;
+ bool is_key = false;
+ size_t key_pref_len,
+ pwd_pref_len,
+ read_len,
+ bytes_read;
+ size_t buf_size,
+ result_size;
+ int i;
+
+ if (encryption_key_command == NULL ||
+ !strlen(encryption_key_command))
+ {
+ /*
+ * encryption_key_command should have been set by initdb. It's weird
+ * if it was not, but there's no better recommendation we can give the
+ * user.
+ */
+#ifndef FRONTEND
+ ereport(FATAL,
+ (errmsg("encryption key not provided"),
+ errdetail("The database cluster was initialized with encryption"
+ " but the server was started without an encryption key."),
+ errhint("Set the encryption_key_command configuration variable.")));
+#else /* FRONTEND */
+ encryption_error(true,
+ "The database cluster was initialized with encryption"
+ " but the server was started without an encryption key. "
+ "Set the encryption_key_command configuration variable.\n");
+#endif /* FRONTEND */
+ }
+
+ encryption_error(false,
+ psprintf("Executing \"%s\" to set up encryption key",
+ encryption_key_command));
+
+ fp = popen(encryption_key_command, "r");
+ if (fp == NULL)
+ encryption_error(true,
+ psprintf("Failed to execute encryption_key_command \"%s\"",
+ encryption_key_command));
+
+ /*
+ * Check which prefix the file starts with.
+ *
+ * The prefixes probably won't change after the release but they might
+ * change during development. The reading logic should be generic so that
+ * change of prefix length requires no additional coding.
+ */
+ key_pref_len = strlen(encryption_key_prefix);
+ pwd_pref_len = strlen(encryption_pwd_prefix);
+
+ /*
+ * The buffer must accommodate either prefix.
+ */
+ buf_size = Max(key_pref_len, pwd_pref_len);
+ buf = (char *) palloc(buf_size);
+
+ /*
+ * Read as few bytes as necessary so that we don't have to move back in
+ * the buffer if the first comparison does not match.
+ */
+ read_len = Min(key_pref_len, pwd_pref_len);
+
+ if (fread(buf, 1, read_len, fp) != read_len)
+ encryption_error(true, "Not enough data received from encryption_key_command");
+
+ if (read_len == key_pref_len &&
+ strncmp(buf, encryption_key_prefix, key_pref_len) == 0)
+ is_key = true;
+ else if (read_len == pwd_pref_len &&
+ strncmp(buf, encryption_pwd_prefix, pwd_pref_len) == 0)
+ is_key = false;
+ else if (buf_size > read_len)
+ {
+ size_t len_diff;
+
+ /*
+ * Read enough data so that one of the prefixes must match.
+ */
+ len_diff = buf_size - read_len;
+ if (fread(buf + read_len, 1, len_diff, fp) != len_diff)
+ encryption_error(true,
+ "Not enough data received from encryption_key_command");
+ read_len += len_diff;
+
+ /*
+ * Try to match the prefixes again.
+ */
+ if (read_len == key_pref_len &&
+ strncmp(buf, encryption_key_prefix, key_pref_len) == 0)
+ is_key = true;
+ else if (read_len == pwd_pref_len &&
+ strncmp(buf, encryption_pwd_prefix, pwd_pref_len) == 0)
+ is_key = false;
+ else
+ encryption_error(true,
+ "Unknown data received from encryption_key_command");
+ }
+
+ *is_key_p = is_key;
+
+ /*
+ * Read the actual credentials.
+ */
+ read_len = is_key ? ENCRYPTION_KEY_LENGTH * 2 : ENCRYPTION_PWD_MAX_LENGTH;
+
+ /*
+ * One extra character can be read and this should be nothing but line
+ * delimiter, see below.
+ */
+ read_len++;
+
+ if (read_len > buf_size)
+ {
+ buf = (char *) repalloc(buf, read_len);
+ buf_size = read_len;
+ }
+
+ bytes_read = fread(buf, 1, read_len, fp);
+
+ if ((is_key && bytes_read < read_len) ||
+ (!is_key && bytes_read < (ENCRYPTION_PWD_MIN_LENGTH + 1)))
+ {
+ if (feof(fp))
+ encryption_error(true,
+ "Not enough data provided by encryption_key_command");
+ else
+ encryption_error(true,
+ psprintf("encryption_key_command returned error code %d",
+ ferror(fp)));
+ }
+ Assert(bytes_read > 0);
+
+ if (buf[bytes_read - 1] != '\n')
+ encryption_error(true, "Encryption key is too long.");
+
+ /*
+ * The credentials must not contain line delimiter.
+ *
+ * If that was allowed, the line delimiter would have to terminate
+ * password even if more characters followed, however those extra
+ * characters would be useless. Since distinguishing of the special (still
+ * legal) case "password\n<EOF>" would make the checks less simple and
+ * since the delimiter itself is useless in the case of a key (because the
+ * key has constant length), let's prohibit the line delimiters anywhere
+ * except for the last position.
+ */
+ for (i = 0; i < bytes_read; i++)
+ if (buf[i] == '\n' && i < (bytes_read - 1))
+ encryption_error(true,
+ "Neither password nor key may contain line break.");
+
+ /*
+ * For a key the result size is different from the amount of data read.
+ */
+ result_size = is_key ? ENCRYPTION_KEY_LENGTH : ENCRYPTION_PWD_MAX_LENGTH;
+
+ result = (char *) palloc(result_size);
+
+ if (is_key)
+ {
+ int i;
+
+ for (i = 0; i < ENCRYPTION_KEY_LENGTH; i++)
+ {
+ if (sscanf(buf + 2 * i, "%2hhx", result + i) == 0)
+ encryption_error(true,
+ psprintf("Invalid character in encryption key at position %d",
+ 2 * i));
+ }
+ *len_p = ENCRYPTION_KEY_LENGTH;
+ }
+ else
+ {
+ /*
+ * Ignore the line delimiter.
+ */
+ *len_p = bytes_read - 1;
+ memcpy(result, buf, *len_p);
+
+ }
+
+ /*
+ * No extra data is allowed.
+ */
+ if (fread(buf, 1, 1, fp) > 0)
+ encryption_error(true,
+ "Credentials are followed by useless data");
+
+
+ pfree(buf);
+ pclose(fp);
+
+ return result;
+}
+#endif /* USE_ENCRYPTION */
+
+/*
+ * Error callback for openssl.
+ */
+#ifdef USE_ENCRYPTION
+static void
+evp_error(void)
+{
+ ERR_print_errors_fp(stderr);
+#ifndef FRONTEND
+
+ /*
+ * FATAL is the appropriate level because backend can hardly fix anything
+ * if encryption / decryption has failed.
+ *
+ * XXX Do we yet need EVP_CIPHER_CTX_cleanup() here?
+ */
+ elog(FATAL, "OpenSSL encountered error during encryption or decryption.");
+#else
+ fprintf(stderr,
+ "OpenSSL encountered error during encryption or decryption.");
+ exit(EXIT_FAILURE);
+#endif /* FRONTEND */
+}
+#endif /* USE_ENCRYPTION */
+
+/*
+ * Xlog is encrypted page at a time. Each xlog page gets a unique tweak via
+ * segment and offset. Unfortunately we can't include timeline because
+ * exitArchiveRecovery() can copy part of the last segment of the old timeline
+ * into the first segment of the new timeline.
+ *
+ * TODO Consider teaching exitArchiveRecovery() to decrypt the copied pages
+ * and encrypt them using a tweak that mentions the new timeline.
+ *
+ * The function is located here rather than some of the xlog*.c modules so
+ * that front-end applications can easily use it too.
+ */
+void
+XLogEncryptionTweak(char *tweak, XLogSegNo segment, uint32 offset)
+{
+#ifdef USE_ENCRYPTION
+ memset(tweak, 0, TWEAK_SIZE);
+ memcpy(tweak, &segment, sizeof(XLogSegNo));
+ memcpy(tweak + sizeof(XLogSegNo), &offset, sizeof(offset));
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+}
diff --git a/src/backend/storage/smgr/md.c b/src/backend/storage/smgr/md.c
index f4374d077b..92f7b36c40 100644
--- a/src/backend/storage/smgr/md.c
+++ b/src/backend/storage/smgr/md.c
@@ -33,6 +33,7 @@
#include "postmaster/bgwriter.h"
#include "storage/fd.h"
#include "storage/bufmgr.h"
+#include "storage/encryption.h"
#include "storage/relfilenode.h"
#include "storage/smgr.h"
#include "utils/hsearch.h"
@@ -113,7 +114,14 @@ typedef struct _MdfdVec
} MdfdVec;
static MemoryContext MdCxt; /* context for all MdfdVec objects */
-
+#ifdef USE_ENCRYPTION
+/*
+ * encryption_buffer from encryption.h is not used here because of the special
+ * memory context.
+ */
+static char *md_encryption_buffer;
+static char *md_encryption_tweak;
+#endif
/*
* In some contexts (currently, standalone backends and the checkpointer)
@@ -198,6 +206,11 @@ static MdfdVec *_mdfd_getseg(SMgrRelation reln, ForkNumber forkno,
static BlockNumber _mdnblocks(SMgrRelation reln, ForkNumber forknum,
MdfdVec *seg);
+#ifdef USE_ENCRYPTION
+static void mdtweak(char *tweak, RelFileNode *relnode, ForkNumber forknum, BlockNumber blocknum);
+static void mdencrypt(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum, char *buffer);
+static void mddecrypt(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum, char *buffer);
+#endif
/*
* mdinit() -- Initialize private state for magnetic disk storage manager.
@@ -242,6 +255,11 @@ mdinit(void)
HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
pendingUnlinks = NIL;
}
+
+#ifdef USE_ENCRYPTION
+ md_encryption_buffer = MemoryContextAllocZero(MdCxt, BLCKSZ);
+ md_encryption_tweak = MemoryContextAllocZero(MdCxt, TWEAK_SIZE);
+#endif
}
/*
@@ -537,6 +555,16 @@ mdextend(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
errmsg("could not seek to block %u in file \"%s\": %m",
blocknum, FilePathName(v->mdfd_vfd))));
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ mdencrypt(reln, forknum, blocknum, buffer);
+ buffer = md_encryption_buffer;
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
+
if ((nbytes = FileWrite(v->mdfd_vfd, buffer, BLCKSZ, WAIT_EVENT_DATA_FILE_EXTEND)) != BLCKSZ)
{
if (nbytes < 0)
@@ -734,6 +762,7 @@ mdread(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
off_t seekpos;
int nbytes;
MdfdVec *v;
+ char *buffer_read = buffer;
TRACE_POSTGRESQL_SMGR_MD_READ_START(forknum, blocknum,
reln->smgr_rnode.node.spcNode,
@@ -754,7 +783,15 @@ mdread(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
errmsg("could not seek to block %u in file \"%s\": %m",
blocknum, FilePathName(v->mdfd_vfd))));
- nbytes = FileRead(v->mdfd_vfd, buffer, BLCKSZ, WAIT_EVENT_DATA_FILE_READ);
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ buffer_read = md_encryption_buffer;
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
+ nbytes = FileRead(v->mdfd_vfd, buffer_read, BLCKSZ, WAIT_EVENT_DATA_FILE_READ);
TRACE_POSTGRESQL_SMGR_MD_READ_DONE(forknum, blocknum,
reln->smgr_rnode.node.spcNode,
@@ -789,6 +826,14 @@ mdread(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
blocknum, FilePathName(v->mdfd_vfd),
nbytes, BLCKSZ)));
}
+ else if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ mddecrypt(reln, forknum, blocknum, buffer);
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
}
/*
@@ -830,6 +875,15 @@ mdwrite(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
errmsg("could not seek to block %u in file \"%s\": %m",
blocknum, FilePathName(v->mdfd_vfd))));
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ mdencrypt(reln, forknum, blocknum, buffer);
+ buffer = md_encryption_buffer;
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
nbytes = FileWrite(v->mdfd_vfd, buffer, BLCKSZ, WAIT_EVENT_DATA_FILE_WRITE);
TRACE_POSTGRESQL_SMGR_MD_WRITE_DONE(forknum, blocknum,
@@ -1988,3 +2042,58 @@ _mdnblocks(SMgrRelation reln, ForkNumber forknum, MdfdVec *seg)
/* note that this calculation will ignore any partial block at EOF */
return (BlockNumber) (len / BLCKSZ);
}
+
+#ifdef USE_ENCRYPTION
+/*
+ * md files are encrypted block at a time. Tweak will alias higher numbered
+ * forks for huge tables.
+ */
+static void
+mdtweak(char *tweak, RelFileNode *relnode, ForkNumber forknum, BlockNumber blocknum)
+{
+ uint32 fork_and_block = (forknum << 24) ^ blocknum;
+
+ memcpy(tweak, relnode, sizeof(RelFileNode));
+ memcpy(tweak + sizeof(RelFileNode), &fork_and_block, 4);
+}
+
+static void
+mdencrypt(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum, char *buffer)
+{
+ mdtweak(md_encryption_tweak, &(reln->smgr_rnode.node), forknum, blocknum);
+ encrypt_block(buffer, md_encryption_buffer, BLCKSZ, md_encryption_tweak);
+}
+
+static void
+mddecrypt(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum, char *dest)
+{
+ mdtweak(md_encryption_tweak, &(reln->smgr_rnode.node), forknum, blocknum);
+ decrypt_block(md_encryption_buffer, dest, BLCKSZ, md_encryption_tweak);
+}
+
+/*
+ * Copying relations between tablespaces/databases means that the tweak values
+ * of each block will change. This function transcodes a series of blocks with
+ * new tweak values. Returns the new block number for convenience.
+ */
+BlockNumber
+ReencryptBlock(char *buffer, int blocks,
+ RelFileNode *srcNode, RelFileNode *dstNode,
+ ForkNumber srcForkNum, ForkNumber dstForkNum,
+ BlockNumber blockNum)
+{
+ char *cur;
+ char srcTweak[TWEAK_SIZE];
+ char dstTweak[TWEAK_SIZE];
+
+ for (cur = buffer; cur < buffer + blocks * BLCKSZ; cur += BLCKSZ)
+ {
+ mdtweak(srcTweak, srcNode, srcForkNum, blockNum);
+ mdtweak(dstTweak, dstNode, dstForkNum, blockNum);
+ decrypt_block(cur, cur, BLCKSZ, srcTweak);
+ encrypt_block(cur, cur, BLCKSZ, dstTweak);
+ blockNum++;
+ }
+ return blockNum;
+}
+#endif /* USE_ENCRYPTION */
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 7a9ada2c71..d53eed6d44 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -63,6 +63,7 @@
#include "replication/walsender.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/encryption.h"
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
@@ -3398,7 +3399,7 @@ process_postgres_switches(int argc, char *argv[], GucContext ctx,
* postmaster/postmaster.c (the option sets should not conflict) and with
* the common help() function in main/main.c.
*/
- while ((flag = getopt(argc, argv, "B:bc:C:D:d:EeFf:h:ijk:lN:nOo:Pp:r:S:sTt:v:W:-:")) != -1)
+ while ((flag = getopt(argc, argv, "B:bc:C:D:d:EeFf:h:ijK:k:lN:nOo:Pp:r:S:sTt:v:W:-:")) != -1)
{
switch (flag)
{
@@ -3456,6 +3457,12 @@ process_postgres_switches(int argc, char *argv[], GucContext ctx,
UseSemiNewlineNewline = true;
break;
+#ifdef USE_OPENSSL
+ case 'K':
+ encryption_key_command = strdup(optarg);
+ break;
+#endif /* USE_OPENSSL */
+
case 'k':
SetConfigOption("unix_socket_directories", optarg, ctx, gucsource);
break;
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index 865119d272..d2ab0aed23 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -1522,6 +1522,7 @@ ValidatePgVersion(const char *path)
* GUC variables: lists of library names to be preloaded at postmaster
* start and at backend start
*/
+char *encryption_library_string = NULL;
char *session_preload_libraries_string = NULL;
char *shared_preload_libraries_string = NULL;
char *local_preload_libraries_string = NULL;
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 0625eff219..582e34fcf6 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -69,6 +69,7 @@
#include "replication/walsender.h"
#include "storage/bufmgr.h"
#include "storage/dsm_impl.h"
+#include "storage/encryption.h"
#include "storage/standby.h"
#include "storage/fd.h"
#include "storage/large_object.h"
@@ -194,6 +195,9 @@ static bool check_cluster_name(char **newval, void **extra, GucSource source);
static const char *show_unix_socket_permissions(void);
static const char *show_log_file_mode(void);
static const char *show_data_directory_mode(void);
+#ifdef USE_OPENSSL
+static const char *show_encryption_key_command(void);
+#endif /* USE_OPENSSL */
/* Private functions in guc-file.l that need to be called from guc.c */
static ConfigVariable *ProcessConfigFileInternal(GucContext context,
@@ -1725,6 +1729,17 @@ static struct config_bool ConfigureNamesBool[] =
},
{
+ {"data_encryption", PGC_INTERNAL, PRESET_OPTIONS,
+ gettext_noop("Shows whether data encryption is turned on for this cluster."),
+ NULL,
+ GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE
+ },
+ &data_encrypted,
+ false,
+ NULL, NULL, NULL
+ },
+
+ {
{"syslog_sequence_numbers", PGC_SIGHUP, LOGGING_WHERE,
gettext_noop("Add sequence number to syslog messages to avoid duplicate suppression."),
NULL
@@ -3905,6 +3920,19 @@ static struct config_string ConfigureNamesString[] =
NULL, NULL, NULL
},
+#ifdef USE_OPENSSL
+ {
+ {"encryption_key_command", PGC_POSTMASTER, 0,
+ gettext_noop("Sets the shell command that will be called to fetch database encryption key."),
+ NULL,
+ GUC_NOT_IN_SAMPLE | GUC_SUPERUSER_ONLY | GUC_IS_NAME
+ },
+ &encryption_key_command,
+ NULL,
+ NULL, NULL, show_encryption_key_command
+ },
+#endif /* USE_OPENSSL */
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, NULL, NULL, NULL, NULL
@@ -10814,4 +10842,15 @@ show_data_directory_mode(void)
return buf;
}
+#ifdef USE_OPENSSL
+static const char *
+show_encryption_key_command(void)
+{
+ if (encryption_key_command)
+ return encryption_key_command;
+ else
+ return "(disabled)";
+}
+#endif /* USE_OPENSSL */
+
#include "guc-file.c"
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 7486d20a34..e20a4ad919 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -88,6 +88,7 @@
#authentication_timeout = 1min # 1s-600s
#password_encryption = md5 # md5 or scram-sha-256
#db_user_namespace = off
+#encryption_key_command = ''
# GSSAPI using Kerberos
#krb_server_keyfile = ''
diff --git a/src/bin/Makefile b/src/bin/Makefile
index 8c11060a2f..431c406a65 100644
--- a/src/bin/Makefile
+++ b/src/bin/Makefile
@@ -21,6 +21,7 @@ SUBDIRS = \
pg_controldata \
pg_ctl \
pg_dump \
+ pg_keysetup \
pg_resetwal \
pg_rewind \
pg_test_fsync \
diff --git a/src/bin/initdb/Makefile b/src/bin/initdb/Makefile
index 8c23941930..ea7af9144f 100644
--- a/src/bin/initdb/Makefile
+++ b/src/bin/initdb/Makefile
@@ -26,7 +26,7 @@ ifneq (,$(with_system_tzdata))
override CPPFLAGS += '-DSYSTEMTZDIR="$(with_system_tzdata)"'
endif
-OBJS= initdb.o findtimezone.o localtime.o encnames.o $(WIN32RES)
+OBJS= initdb.o findtimezone.o localtime.o encnames.o encryption.o $(WIN32RES)
all: initdb
@@ -45,6 +45,9 @@ encnames.c: % : $(top_srcdir)/src/backend/utils/mb/%
localtime.c: % : $(top_srcdir)/src/timezone/%
rm -f $@ && $(LN_S) $< .
+encryption.c: % : $(top_srcdir)/src/backend/storage/smgr/%
+ rm -f $@ && $(LN_S) $< .
+
install: all installdirs
$(INSTALL_PROGRAM) initdb$(X) '$(DESTDIR)$(bindir)/initdb$(X)'
@@ -55,7 +58,7 @@ uninstall:
rm -f '$(DESTDIR)$(bindir)/initdb$(X)'
clean distclean maintainer-clean:
- rm -f initdb$(X) $(OBJS) encnames.c localtime.c
+ rm -f initdb$(X) $(OBJS) encnames.c localtime.c encryption.c
rm -rf tmp_check
# ensure that changes in datadir propagate into object file
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index b53d6eb9cc..47234aec7a 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -68,6 +68,9 @@
#include "common/restricted_token.h"
#include "common/username.h"
#include "fe_utils/string_utils.h"
+#include "storage/encryption.h"
+#include "lib/ilist.h"
+#include "mb/pg_wchar.h"
#include "getaddrinfo.h"
#include "getopt_long.h"
#include "mb/pg_wchar.h"
@@ -77,6 +80,12 @@
/* Ideally this would be in a .h file, but it hardly seems worth the trouble */
extern const char *select_default_timezone(const char *share_path);
+typedef struct
+{
+ dlist_node list_node;
+ char *value;
+} extra_option;
+
static const char *const auth_methods_host[] = {
"trust", "reject", "scram-sha-256", "md5", "password", "ident", "radius",
#ifdef ENABLE_GSS
@@ -141,9 +150,11 @@ static bool do_sync = true;
static bool sync_only = false;
static bool show_setting = false;
static bool data_checksums = false;
+static char *encr_key_cmd_str = NULL;
static char *xlog_dir = NULL;
static char *str_wal_segment_size_mb = NULL;
static int wal_segment_size_mb;
+static dlist_head extra_options = DLIST_STATIC_INIT(extra_options);
/* internal vars */
@@ -538,6 +549,7 @@ readfile(const char *path)
fclose(infile);
free(buffer);
+
result[n] = NULL;
return result;
@@ -1091,6 +1103,39 @@ pretty_wal_size(int segment_count)
return result;
}
+static void
+append_extra_options(char ***conflines)
+{
+ dlist_iter iter;
+ int n_extra = 0;
+ int n_current = 0;
+ int i = 0;
+ char **new_conflines;
+
+ dlist_foreach(iter, &extra_options)
+ {
+ n_extra++;
+ }
+ while ((*conflines)[i++] != NULL)
+ n_current++;
+
+ new_conflines = (char **) pg_malloc((n_current + n_extra + 1) * sizeof(char *));
+ for (i = 0; i < n_current; i++)
+ new_conflines[i] = (*conflines)[i];
+
+ dlist_foreach(iter, &extra_options)
+ {
+ extra_option *opt = dlist_container(extra_option, list_node, iter.cur);
+
+ new_conflines[i++] = opt->value;
+ }
+
+ new_conflines[i] = NULL;
+ pg_free(*conflines);
+ *conflines = new_conflines;
+}
+
+
/*
* set up all the config files
*/
@@ -1109,6 +1154,8 @@ setup_config(void)
conflines = readfile(conf_file);
+ append_extra_options(&conflines);
+
snprintf(repltok, sizeof(repltok), "max_connections = %d", n_connections);
conflines = replace_token(conflines, "#max_connections = 100", repltok);
@@ -1250,6 +1297,17 @@ setup_config(void)
"log_file_mode = 0640");
}
+#ifdef USE_OPENSSL
+ if (encryption_key_command)
+ {
+ snprintf(repltok, sizeof(repltok), "encryption_key_command = '%s'",
+ encryption_key_command);
+ conflines = replace_token(conflines,
+ "#encryption_key_command = ''", repltok);
+ }
+#endif /* USE_OPENSSL */
+
+
snprintf(path, sizeof(path), "%s/postgresql.conf", pg_data);
writefile(path, conflines);
@@ -1467,11 +1525,29 @@ bootstrap_template1(void)
/* Also ensure backend isn't confused by this environment var: */
unsetenv("PGCLIENTENCODING");
+#ifdef USE_OPENSSL
+ /* Prepare the -K option for the backend. */
+ if (encryption_key_command)
+ {
+ size_t len;
+
+ len = 3 + strlen(encryption_key_command) + 1;
+ encr_key_cmd_str = (char *) pg_malloc(len);
+ snprintf(encr_key_cmd_str, len, "-K %s", encryption_key_command);
+ }
+ else
+ {
+ encr_key_cmd_str = (char *) pg_malloc(1);
+ encr_key_cmd_str[0] = '\0';
+ }
+#endif /* USE_OPENSSL */
+
snprintf(cmd, sizeof(cmd),
- "\"%s\" --boot -x1 -X %u %s %s %s",
+ "\"%s\" --boot -x1 -X %u %s %s %s %s",
backend_exec,
wal_segment_size_mb * (1024 * 1024),
data_checksums ? "-k" : "",
+ encr_key_cmd_str,
boot_options,
debug ? "-d 5" : "");
@@ -2422,6 +2498,10 @@ usage(const char *progname)
printf(_("\nLess commonly used options:\n"));
printf(_(" -d, --debug generate lots of debugging output\n"));
printf(_(" -k, --data-checksums use data page checksums\n"));
+#ifdef USE_OPENSSL
+ printf(_(" -K, --encryption-key-command\n"
+ " command that returns encryption key\n"));
+#endif /* USE_OPENSSL */
printf(_(" -L DIRECTORY where to find the input files\n"));
printf(_(" -n, --no-clean do not clean up after errors\n"));
printf(_(" -N, --no-sync do not wait for changes to be written safely to disk\n"));
@@ -2528,7 +2608,6 @@ setup_pgdata(void)
putenv(pgdata_set_env);
}
-
void
setup_bin_paths(const char *argv0)
{
@@ -3041,8 +3120,8 @@ initialize_data_directory(void)
fflush(stdout);
snprintf(cmd, sizeof(cmd),
- "\"%s\" %s template1 >%s",
- backend_exec, backend_options,
+ "\"%s\" %s %s template1 >%s",
+ backend_exec, backend_options, encr_key_cmd_str,
DEVNULL);
PG_CMD_OPEN;
@@ -3083,6 +3162,23 @@ initialize_data_directory(void)
check_ok();
}
+static void
+parse_extra_option_arg(char *optarg)
+{
+ extra_option *opt;
+
+ if (!strchr(optarg, '='))
+ {
+ fprintf(stderr, _("Option value is not in key=value format"));
+ exit(1);
+ }
+
+ opt = malloc(sizeof(extra_option));
+ if (asprintf(&opt->value, "%s\n", optarg) < 0)
+ exit(1);
+
+ dlist_push_tail(&extra_options, &opt->list_node);
+}
int
main(int argc, char *argv[])
@@ -3117,6 +3213,9 @@ main(int argc, char *argv[])
{"waldir", required_argument, NULL, 'X'},
{"wal-segsize", required_argument, NULL, 12},
{"data-checksums", no_argument, NULL, 'k'},
+#ifdef USE_OPENSSL
+ {"encryption-key-command", required_argument, NULL, 'K'},
+#endif /* USE_OPENSSL */
{"allow-group-access", no_argument, NULL, 'g'},
{NULL, 0, NULL, 0}
};
@@ -3159,7 +3258,7 @@ main(int argc, char *argv[])
/* process command-line options */
- while ((c = getopt_long(argc, argv, "dD:E:kL:nNU:WA:sST:X:g", long_options, &option_index)) != -1)
+ while ((c = getopt_long(argc, argv, "dD:E:kK:L:nNU:WA:sST:X:g", long_options, &option_index)) != -1)
{
switch (c)
{
@@ -3211,6 +3310,14 @@ main(int argc, char *argv[])
case 'k':
data_checksums = true;
break;
+#ifdef USE_OPENSSL
+ case 'K':
+ encryption_key_command = pg_strdup(optarg);
+ break;
+#endif /* USE_OPENSSL */
+ case 'c':
+ parse_extra_option_arg(optarg);
+ break;
case 'L':
share_path = pg_strdup(optarg);
break;
@@ -3384,7 +3491,12 @@ main(int argc, char *argv[])
if (pwprompt || pwfilename)
get_su_pwd();
- printf("\n");
+#ifdef USE_OPENSSL
+ if (encryption_key_command)
+ printf(_("Data encryption is enabled.\n"));
+ else
+ printf(_("Data encryption is disabled.\n"));
+#endif /* USE_OPENSSL */
initialize_data_directory();
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 895a51f89d..32c702a978 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -340,5 +340,14 @@ main(int argc, char *argv[])
ControlFile->data_checksum_version);
printf(_("Mock authentication nonce: %s\n"),
mock_auth_nonce_str);
+ printf(_("Data encryption: %s\n"),
+ ControlFile->data_encrypted ? _("on") : _("off"));
+ if (ControlFile->data_encrypted)
+ printf(_("Data encryption fingerprint: %08X%08X%08X%08X\n"),
+ htonl(((uint32 *) ControlFile->encryption_verification)[0]),
+ htonl(((uint32 *) ControlFile->encryption_verification)[1]),
+ htonl(((uint32 *) ControlFile->encryption_verification)[2]),
+ htonl(((uint32 *) ControlFile->encryption_verification)[3])
+ );
return 0;
}
diff --git a/src/bin/pg_keysetup/Makefile b/src/bin/pg_keysetup/Makefile
new file mode 100644
index 0000000000..c9fe1064d0
--- /dev/null
+++ b/src/bin/pg_keysetup/Makefile
@@ -0,0 +1,32 @@
+# src/bin/pg_keysetup/Makefile
+
+PGFILEDESC = "pg_keysetup - generate encryption key from a password"
+PGAPPICON=win32
+
+subdir = src/bin/pg_keysetup
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = pg_keysetup.o $(RMGRDESCOBJS) encryption.o $(WIN32RES)
+
+override CPPFLAGS := -DFRONTEND $(CPPFLAGS)
+
+all: pg_keysetup
+
+pg_keysetup: $(OBJS) | submake-libpgport
+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+
+encryption.c: % : $(top_srcdir)/src/backend/storage/smgr/%
+ rm -f $@ && $(LN_S) $< .
+
+install: all installdirs
+ $(INSTALL_PROGRAM) pg_keysetup$(X) '$(DESTDIR)$(bindir)/pg_keysetup$(X)'
+
+installdirs:
+ $(MKDIR_P) '$(DESTDIR)$(bindir)'
+
+uninstall:
+ rm -f '$(DESTDIR)$(bindir)/pg_keysetup$(X)'
+
+clean distclean maintainer-clean:
+ rm -f pg_keysetup$(X) $(OBJS) encryption.c
diff --git a/src/bin/pg_keysetup/pg_keysetup.c b/src/bin/pg_keysetup/pg_keysetup.c
new file mode 100644
index 0000000000..68c33b26f7
--- /dev/null
+++ b/src/bin/pg_keysetup/pg_keysetup.c
@@ -0,0 +1,208 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_keysetup.c - Turn password into encryption key.
+ *
+ * Copyright (c) 2013-2017, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/bin/pg_keysetup/pg_keysetup.c
+ *-------------------------------------------------------------------------
+ */
+
+#define FRONTEND 1
+#include "postgres.h"
+
+#include <dirent.h>
+#include <unistd.h>
+
+#include "port/pg_crc32c.h"
+#include "storage/encryption.h"
+#include "getopt_long.h"
+
+#ifdef USE_ENCRYPTION
+static const char *progname;
+
+static void
+usage(const char *progname)
+{
+ printf(_("%s derives encryption key from a password.\n\n"),
+ progname);
+ printf(_("Usage:\n"));
+ printf(_(" %s [OPTION]...\n"), progname);
+ printf(_("\nOptions:\n"));
+ printf(_(" [-D] DATADIR data directory\n"));
+ printf(_(" -?, --help show this help, then exit\n\n"));
+ printf(_("Password is read from stdin and the key is sent to stdout\n"));
+}
+
+static void fatal_error(const char *fmt,...) pg_attribute_printf(1, 2);
+
+/*
+ * Big red button to push when things go horribly wrong.
+ */
+static void
+fatal_error(const char *fmt,...)
+{
+ va_list args;
+
+ fflush(stdout);
+
+ fprintf(stderr, _("%s: FATAL: "), progname);
+ va_start(args, fmt);
+ vfprintf(stderr, _(fmt), args);
+ va_end(args);
+ fputc('\n', stderr);
+
+ exit(EXIT_FAILURE);
+}
+
+/*
+ * This function does the same as read_kdf_file() in the backend. However most
+ * of the calls look different on frontend side, it seems better to write it
+ * here from scratch than to use too many #ifdef-#else-#endif constructs.
+ */
+static void
+read_kdf_file(char *path)
+{
+ pg_crc32c crc;
+ int fd;
+
+ KDFParams = palloc(KDF_PARAMS_FILE_SIZE);
+
+ fd = open(path, O_RDONLY | PG_BINARY, S_IRUSR);
+
+ if (fd < 0)
+ fatal_error("could not open key setup file \"%s\": %m",
+ KDF_PARAMS_FILE);
+
+ if (read(fd, KDFParams, sizeof(KDFParamsData)) != sizeof(KDFParamsData))
+ fatal_error("could not read from key setup file: %m");
+
+ close(fd);
+
+ /* Now check the CRC. */
+ INIT_CRC32C(crc);
+ COMP_CRC32C(crc,
+ (char *) KDFParams,
+ offsetof(KDFParamsData, crc));
+ FIN_CRC32C(crc);
+
+ if (!EQ_CRC32C(crc, KDFParams->crc))
+ fatal_error("incorrect checksum in key setup file");
+
+ if (KDFParams->function != KDF_OPENSSL_PKCS5_PBKDF2_HMAC_SHA)
+ fatal_error("unsupported KDF function %d", KDFParams->function);
+}
+#endif /* USE_ENCRYPTION */
+
+int
+main(int argc, char **argv)
+{
+#ifdef USE_ENCRYPTION
+ int c;
+ char *DataDir = NULL;
+ char fpath[MAXPGPATH];
+ int fd;
+ char password[ENCRYPTION_PWD_MAX_LENGTH];
+ size_t pwd_len,
+ i;
+
+ progname = get_progname(argv[0]);
+
+ if (argc > 1)
+ {
+ if (strcmp(argv[1], "--help") == 0 || strcmp(argv[1], "-?") == 0)
+ {
+ usage(progname);
+ exit(0);
+ }
+ if (strcmp(argv[1], "--version") == 0 || strcmp(argv[1], "-V") == 0)
+ {
+ puts("pg_controldata (PostgreSQL) " PG_VERSION);
+ exit(0);
+ }
+ }
+
+ while ((c = getopt(argc, argv, "D:")) != -1)
+ {
+ switch (c)
+ {
+ case 'D':
+ DataDir = optarg;
+ break;
+
+ default:
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
+ exit(1);
+ }
+ }
+
+ if (DataDir == NULL)
+ {
+ if (optind < argc)
+ DataDir = argv[optind++];
+ else
+ DataDir = getenv("PGDATA");
+ }
+
+ /* Complain if any arguments remain */
+ if (optind < argc)
+ {
+ fprintf(stderr, _("%s: too many command-line arguments (first is \"%s\")\n"),
+ progname, argv[optind]);
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+ if (DataDir == NULL)
+ {
+ fprintf(stderr, _("%s: no data directory specified\n"), progname);
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
+ exit(1);
+ }
+
+ snprintf(fpath, MAXPGPATH, "%s/%s", DataDir, KDF_PARAMS_FILE);
+ fd = open(fpath, O_RDONLY | PG_BINARY, 0);
+ if (fd < 0)
+ fatal_error("could not open file \"%s\"", fpath);
+
+ read_kdf_file(fpath);
+ close(fd);
+
+ /*
+ * Read the password.
+ */
+ pwd_len = 0;
+ while (true)
+ {
+ int c = getchar();
+
+ if (c == EOF || c == '\n')
+ break;
+
+ if (pwd_len >= ENCRYPTION_PWD_MAX_LENGTH)
+ fatal_error("The password is too long");
+
+ password[pwd_len++] = c;
+ }
+
+ if (pwd_len < ENCRYPTION_PWD_MIN_LENGTH)
+ fatal_error("The password is too short");
+
+ /*
+ * Run the key derivation function.
+ */
+ setup_encryption_key(password, false, pwd_len);
+
+ /*
+ * Finally print the encryption key.
+ */
+ for (i = 0; i < ENCRYPTION_KEY_LENGTH; i++)
+ printf("%.2x", encryption_key[i]);
+ printf("\n");
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ return 0;
+}
diff --git a/src/bin/pg_resetwal/Makefile b/src/bin/pg_resetwal/Makefile
index e49d52b928..6ea9c515f7 100644
--- a/src/bin/pg_resetwal/Makefile
+++ b/src/bin/pg_resetwal/Makefile
@@ -15,13 +15,18 @@ subdir = src/bin/pg_resetwal
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-OBJS= pg_resetwal.o $(WIN32RES)
+override CPPFLAGS := -DFRONTEND $(CPPFLAGS)
+
+OBJS= pg_resetwal.o encryption.o $(WIN32RES)
all: pg_resetwal
pg_resetwal: $(OBJS) | submake-libpgport
$(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+encryption.c: % : $(top_srcdir)/src/backend/storage/smgr/%
+ rm -f $@ && $(LN_S) $< .
+
install: all installdirs
$(INSTALL_PROGRAM) pg_resetwal$(X) '$(DESTDIR)$(bindir)/pg_resetwal$(X)'
@@ -32,7 +37,7 @@ uninstall:
rm -f '$(DESTDIR)$(bindir)/pg_resetwal$(X)'
clean distclean maintainer-clean:
- rm -f pg_resetwal$(X) $(OBJS)
+ rm -f pg_resetwal$(X) $(OBJS) encryption.c
rm -rf tmp_check
check:
diff --git a/src/bin/pg_resetwal/pg_resetwal.c b/src/bin/pg_resetwal/pg_resetwal.c
index 6fb403a5a8..ab7cce4bad 100644
--- a/src/bin/pg_resetwal/pg_resetwal.c
+++ b/src/bin/pg_resetwal/pg_resetwal.c
@@ -54,11 +54,11 @@
#include "common/fe_memutils.h"
#include "common/file_perm.h"
#include "common/restricted_token.h"
+#include "storage/encryption.h"
#include "storage/large_object.h"
#include "pg_getopt.h"
#include "getopt_long.h"
-
static ControlFileData ControlFile; /* pg_control values */
static XLogSegNo newXlogSegNo; /* new XLOG segment # */
static bool guessed = false; /* T if we had to guess at any values */
@@ -135,7 +135,7 @@ main(int argc, char *argv[])
}
- while ((c = getopt_long(argc, argv, "c:D:e:fl:m:no:O:x:", long_options, NULL)) != -1)
+ while ((c = getopt_long(argc, argv, "c:D:e:fK:l:m:no:O:x:", long_options, NULL)) != -1)
{
switch (c)
{
@@ -278,6 +278,12 @@ main(int argc, char *argv[])
}
break;
+#ifdef USE_ENCRYPTION
+ case 'K':
+ encryption_key_command = strdup(optarg);
+ break;
+#endif /* USE_ENCRYPTION */
+
case 'l':
if (strspn(optarg, "01234567890ABCDEFabcdef") != XLOG_FNAME_LEN)
{
@@ -415,6 +421,16 @@ main(int argc, char *argv[])
XLogFromFileName(log_fname, &minXlogTli, &minXlogSegNo, WalSegSz);
/*
+ * If the data is encrypted, we'll also have to encrypt the XLOG record
+ * below.
+ */
+ if (ControlFile.data_encrypted)
+ {
+ setup_encryption(false);
+ data_encrypted = true;
+ }
+
+ /*
* Also look at existing segment files to set up newXlogSegNo
*/
FindEndOfXLOG();
@@ -1253,6 +1269,18 @@ WriteEmptyXLOG(void)
FIN_CRC32C(crc);
record->xl_crc = crc;
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ char tweak[TWEAK_SIZE];
+
+ XLogEncryptionTweak(tweak, newXlogSegNo, 0);
+ encrypt_block(buffer.data, buffer.data, XLOG_BLCKSZ, tweak);
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
+
/* Write the first page */
XLogFilePath(path, ControlFile.checkPointCopy.ThisTimeLineID,
newXlogSegNo, WalSegSz);
@@ -1316,6 +1344,10 @@ usage(void)
printf(_(" [-D, --pgdata=]DATADIR data directory\n"));
printf(_(" -e, --epoch=XIDEPOCH set next transaction ID epoch\n"));
printf(_(" -f, --force force update to be done\n"));
+#ifdef USE_ENCRYPTION
+ printf(_(" -K, --encryption-key-command\n"
+ " command that returns encryption key\n"));
+#endif /* USE_ENCRYPTION */
printf(_(" -l, --next-wal-file=WALFILE set minimum starting location for new WAL\n"));
printf(_(" -m, --multixact-ids=MXID,MXID set next and oldest multitransaction ID\n"));
printf(_(" -n, --dry-run no update, just show what would be done\n"));
diff --git a/src/bin/pg_rewind/Makefile b/src/bin/pg_rewind/Makefile
index 2bcfcc61af..50263c0438 100644
--- a/src/bin/pg_rewind/Makefile
+++ b/src/bin/pg_rewind/Makefile
@@ -20,7 +20,7 @@ LDFLAGS_INTERNAL += $(libpq_pgport)
OBJS = pg_rewind.o parsexlog.o xlogreader.o datapagemap.o timeline.o \
fetch.o file_ops.o copy_fetch.o libpq_fetch.o filemap.o logging.o \
- $(WIN32RES)
+ encryption.o $(WIN32RES)
EXTRA_CLEAN = xlogreader.c
@@ -32,6 +32,9 @@ pg_rewind: $(OBJS) | submake-libpq submake-libpgport
xlogreader.c: % : $(top_srcdir)/src/backend/access/transam/%
rm -f $@ && $(LN_S) $< .
+encryption.c: % : $(top_srcdir)/src/backend/storage/smgr/%
+ rm -f $@ && $(LN_S) $< .
+
install: all installdirs
$(INSTALL_PROGRAM) pg_rewind$(X) '$(DESTDIR)$(bindir)/pg_rewind$(X)'
@@ -42,7 +45,7 @@ uninstall:
rm -f '$(DESTDIR)$(bindir)/pg_rewind$(X)'
clean distclean maintainer-clean:
- rm -f pg_rewind$(X) $(OBJS) xlogreader.c
+ rm -f pg_rewind$(X) $(OBJS) xlogreader.c encryption.c
rm -rf tmp_check
check:
diff --git a/src/bin/pg_rewind/parsexlog.c b/src/bin/pg_rewind/parsexlog.c
index 40028471bf..72a0d07f84 100644
--- a/src/bin/pg_rewind/parsexlog.c
+++ b/src/bin/pg_rewind/parsexlog.c
@@ -23,7 +23,7 @@
#include "catalog/pg_control.h"
#include "catalog/storage_xlog.h"
#include "commands/dbcommands_xlog.h"
-
+#include "storage/encryption.h"
/*
* RmgrNames is an array of resource manager names, to make error messages
@@ -324,6 +324,18 @@ SimpleXLogPageRead(XLogReaderState *xlogreader, XLogRecPtr targetPagePtr,
return -1;
}
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ char tweak[TWEAK_SIZE];
+
+ XLogEncryptionTweak(tweak, xlogreadsegno, targetPageOff);
+ decrypt_block(readBuf, readBuf, XLOG_BLCKSZ, tweak);
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
+
Assert(targetSegNo == xlogreadsegno);
*pageTLI = targetHistory[private->tliIndex].tli;
diff --git a/src/bin/pg_rewind/pg_rewind.c b/src/bin/pg_rewind/pg_rewind.c
index 9653106386..5f94559c64 100644
--- a/src/bin/pg_rewind/pg_rewind.c
+++ b/src/bin/pg_rewind/pg_rewind.c
@@ -29,6 +29,7 @@
#include "common/restricted_token.h"
#include "getopt_long.h"
#include "storage/bufpage.h"
+#include "storage/encryption.h"
static void usage(const char *progname);
@@ -71,6 +72,10 @@ usage(const char *progname)
printf(_(" -D, --target-pgdata=DIRECTORY existing data directory to modify\n"));
printf(_(" --source-pgdata=DIRECTORY source data directory to synchronize with\n"));
printf(_(" --source-server=CONNSTR source server to synchronize with\n"));
+#ifdef USE_OPENSSL
+ printf(_(" -K, --encryption-key-command=COMMAND\n"
+ " command that returns encryption key\n"));
+#endif /* USE_OPENSSL */
printf(_(" -n, --dry-run stop before modifying anything\n"));
printf(_(" -N, --no-sync do not wait for changes to be written\n"));
printf(_(" safely to disk\n"));
@@ -95,6 +100,9 @@ main(int argc, char **argv)
{"no-sync", no_argument, NULL, 'N'},
{"progress", no_argument, NULL, 'P'},
{"debug", no_argument, NULL, 3},
+#ifdef USE_OPENSSL
+ {"encryption-key-command", required_argument, NULL, 'K'},
+#endif /* USE_OPENSSL */
{NULL, 0, NULL, 0}
};
int option_index;
@@ -129,7 +137,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "D:nNP", long_options, &option_index)) != -1)
+ while ((c = getopt_long(argc, argv, "D:K:nNP", long_options, &option_index)) != -1)
{
switch (c)
{
@@ -163,6 +171,12 @@ main(int argc, char **argv)
case 2: /* --source-server */
connstr_source = pg_strdup(optarg);
break;
+#ifdef USE_OPENSSL
+ case 4: /* --encryption-key-command */
+ case 'K':
+ encryption_key_command = strdup(optarg);
+ break;
+#endif /* USE_OPENSSL */
}
}
@@ -242,6 +256,16 @@ main(int argc, char **argv)
sanityChecks();
/*
+ * Setup encryption if it's obvious that we'll have to deal with encrypted
+ * XLOG.
+ */
+ if (ControlFile_target.data_encrypted)
+ {
+ setup_encryption(false);
+ data_encrypted = true;
+ }
+
+ /*
* If both clusters are already on the same timeline, there's nothing to
* do.
*/
@@ -434,6 +458,24 @@ sanityChecks(void)
ControlFile_source.state != DB_SHUTDOWNED &&
ControlFile_source.state != DB_SHUTDOWNED_IN_RECOVERY)
pg_fatal("source data directory must be shut down cleanly\n");
+
+ /*
+ * Since slave receives XLOG stream encrypted by master, handling
+ * differently encrypted clusters is not the typical use case for
+ * pg_rewind. Yet we should check the encryption.
+ */
+ if (ControlFile_source.data_encrypted ||
+ ControlFile_target.data_encrypted)
+ {
+ if (ControlFile_source.data_encrypted !=
+ ControlFile_target.data_encrypted)
+ pg_fatal("source and target server must be both unencrypted or both encrypted\n");
+
+ if (memcmp(ControlFile_source.encryption_verification,
+ ControlFile_target.encryption_verification,
+ ENCRYPTION_SAMPLE_SIZE))
+ pg_fatal("both source and target server must use the same encryption key");
+ }
}
/*
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index 5a78d603dc..c37ad0bb65 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -267,6 +267,23 @@ check_cluster_versions(void)
GET_MAJOR_VERSION(new_cluster.bin_version))
pg_fatal("New cluster data and binary directories are from different major versions.\n");
+ /*
+ * As this is the first version that supports encryption, we actually do
+ * not expect the old cluster to be encrypted.
+ *
+ * TODO Change this logic as soon as we merge the encryption into the
+ * next, even minor version (pg_upgrade seems to accept clusters that
+ * differ only in the minor version), or for the PG core patch. Eventually
+ * we'll support the upgrade if either both clusters are unencrypted, or
+ * if both are encrypted using the same key. In the latter case, we need
+ * to ensure that the PGENCRYPTIONKEY environment variable is passed to
+ * both clusters.
+ */
+ if (old_cluster.controldata.data_encrypted)
+ pg_fatal("Old cluster is encrypted.\n");
+ if (new_cluster.controldata.data_encrypted)
+ pg_fatal("New cluster is encrypted.\n");
+
check_ok();
}
diff --git a/src/bin/pg_upgrade/controldata.c b/src/bin/pg_upgrade/controldata.c
index c2962043c2..6fcf2e4d3c 100644
--- a/src/bin/pg_upgrade/controldata.c
+++ b/src/bin/pg_upgrade/controldata.c
@@ -58,6 +58,7 @@ get_control_data(ClusterInfo *cluster, bool live_check)
bool got_large_object = false;
bool got_date_is_int = false;
bool got_data_checksum_version = false;
+ bool got_data_encrypted = false;
bool got_cluster_state = false;
char *lc_collate = NULL;
char *lc_ctype = NULL;
@@ -202,6 +203,13 @@ get_control_data(ClusterInfo *cluster, bool live_check)
got_data_checksum_version = true;
}
+ /* Only in <= 9.6 */
+ if (GET_MAJOR_VERSION(cluster->major_version) <= 906)
+ {
+ cluster->controldata.data_encrypted = false;
+ got_data_encrypted = true;
+ }
+
/* we have the result of cmd in "output". so parse it line by line now */
while (fgets(bufin, sizeof(bufin), output))
{
@@ -485,6 +493,34 @@ get_control_data(ClusterInfo *cluster, bool live_check)
cluster->controldata.data_checksum_version = str2uint(p);
got_data_checksum_version = true;
}
+ else if ((p = strstr(bufin, "encryption fingerprint")) != NULL)
+ {
+ int i;
+
+ p = strchr(p, ':');
+
+ if (p == NULL || strlen(p) <= 1)
+ pg_fatal("%d: controldata retrieval problem\n", __LINE__);
+
+ cluster->controldata.data_encrypted = true;
+
+ /* Skip the colon and any whitespace after it */
+ p = strchr(p, ':');
+ if (p == NULL || strlen(p) <= 1)
+ pg_fatal("%d: controldata retrieval problem\n", __LINE__);
+ p = strpbrk(p, "01234567890ABCDEF");
+ if (p == NULL || strlen(p) <= 1)
+ pg_fatal("%d: controldata retrieval problem\n", __LINE__);
+
+ /* Make sure it looks like a valid finerprint */
+ if (strspn(p, "0123456789ABCDEF") != 32)
+ pg_fatal("%d: controldata retrieval problem\n", __LINE__);
+
+ for (i = 0; i < 16; i++)
+ sscanf(p + 2 * i, "%2hhx",
+ cluster->controldata.encryption_verification + i);
+ got_data_encrypted = true;
+ }
}
pclose(output);
@@ -539,7 +575,7 @@ get_control_data(ClusterInfo *cluster, bool live_check)
!got_index || !got_toast ||
(!got_large_object &&
cluster->controldata.ctrl_ver >= LARGE_OBJECT_SIZE_PG_CONTROL_VER) ||
- !got_date_is_int || !got_data_checksum_version)
+ !got_date_is_int || !got_data_checksum_version || !got_data_encrypted)
{
if (cluster == &old_cluster)
pg_log(PG_REPORT,
@@ -605,6 +641,10 @@ get_control_data(ClusterInfo *cluster, bool live_check)
if (!got_data_checksum_version)
pg_log(PG_REPORT, " data checksum version\n");
+ /* value added in Postgres 10 */
+ if (!got_data_encrypted)
+ pg_log(PG_REPORT, " data encryption status\n");
+
pg_fatal("Cannot continue without required control information, terminating\n");
}
}
@@ -669,6 +709,18 @@ check_control_data(ControlData *oldctrl,
pg_fatal("old cluster uses data checksums but the new one does not\n");
else if (oldctrl->data_checksum_version != newctrl->data_checksum_version)
pg_fatal("old and new cluster pg_controldata checksum versions do not match\n");
+
+ if (oldctrl->data_encrypted && !newctrl->data_encrypted)
+ pg_fatal("old cluster is encrypted, but the new one is not\n");
+ else if (!oldctrl->data_encrypted && newctrl->data_encrypted)
+ pg_fatal("old cluster is not encrypted, but the new one is\n");
+ else if (oldctrl->data_encrypted && newctrl->data_encrypted)
+ {
+ if (oldctrl->encryption_verification != newctrl->encryption_verification)
+ pg_fatal("encryption keys do not match between old and new cluster\n");
+ else
+ pg_fatal("upgrading encrypted databases is not implemented yet\n"); /* TODO */
+ }
}
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index f83a3eeb67..46eff5fd02 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -227,6 +227,8 @@ typedef struct
bool date_is_int;
bool float8_pass_by_value;
bool data_checksum_version;
+ bool data_encrypted;
+ uint8 encryption_verification[16];
} ControlData;
/*
diff --git a/src/bin/pg_waldump/Makefile b/src/bin/pg_waldump/Makefile
index f5957bd75a..5a8bd1fba9 100644
--- a/src/bin/pg_waldump/Makefile
+++ b/src/bin/pg_waldump/Makefile
@@ -7,8 +7,8 @@ subdir = src/bin/pg_waldump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-OBJS = pg_waldump.o compat.o xlogreader.o rmgrdesc.o \
- $(RMGRDESCOBJS) $(WIN32RES)
+OBJS = pg_waldump.o compat.o xlogreader.o rmgrdesc.o encryption.o \
+ $(RMGRDESCOBJS) encryption.o $(WIN32RES)
override CPPFLAGS := -DFRONTEND $(CPPFLAGS)
@@ -27,6 +27,9 @@ xlogreader.c: % : $(top_srcdir)/src/backend/access/transam/%
$(RMGRDESCSOURCES): % : $(top_srcdir)/src/backend/access/rmgrdesc/%
rm -f $@ && $(LN_S) $< .
+encryption.c: % : $(top_srcdir)/src/backend/storage/smgr/%
+ rm -f $@ && $(LN_S) $< .
+
install: all installdirs
$(INSTALL_PROGRAM) pg_waldump$(X) '$(DESTDIR)$(bindir)/pg_waldump$(X)'
@@ -37,4 +40,5 @@ uninstall:
rm -f '$(DESTDIR)$(bindir)/pg_waldump$(X)'
clean distclean maintainer-clean:
- rm -f pg_waldump$(X) $(OBJS) $(RMGRDESCSOURCES) xlogreader.c
+ rm -f pg_waldump$(X) $(OBJS) $(RMGRDESCSOURCES) xlogreader.c \
+encryption.c
diff --git a/src/bin/pg_waldump/pg_waldump.c b/src/bin/pg_waldump/pg_waldump.c
index ad28333a1e..904ceb6f9a 100644
--- a/src/bin/pg_waldump/pg_waldump.c
+++ b/src/bin/pg_waldump/pg_waldump.c
@@ -21,12 +21,15 @@
#include "access/xlog_internal.h"
#include "access/transam.h"
#include "common/fe_memutils.h"
+#include "storage/encryption.h"
#include "getopt_long.h"
#include "rmgrdesc.h"
static const char *progname;
+static char *str_wal_segment_size_mb = NULL;
+static bool encr_inited = false;
static int WalSegSz;
typedef struct XLogDumpPrivate
@@ -215,8 +218,85 @@ search_directory(const char *directory, const char *fname)
r = read(fd, buf.data, XLOG_BLCKSZ);
if (r == XLOG_BLCKSZ)
{
- XLogLongPageHeader longhdr = (XLogLongPageHeader) buf.data;
+ XLogLongPageHeader longhdr;
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ uint32 tli;
+ char tweak[TWEAK_SIZE];
+ XLogSegNo segNo;
+ int wal_segment_size;
+ XLogPageHeader hdr;
+
+ /*
+ * Segment size is contained in the encryption tweak so we
+ * cannot get the size from the header until the page is
+ * decrypted. User needs to pass the size if he uses
+ * non-default value.
+ *
+ * TODO The processing of wal-segsize is almost copy & pasted
+ * from initdb.c. Reuse in a better way.
+ */
+ /* set wal segment size */
+ if (str_wal_segment_size_mb == NULL)
+ wal_segment_size = DEFAULT_XLOG_SEG_SIZE;
+ else
+ {
+ char *endptr;
+
+ /* check that the argument is a number */
+ wal_segment_size = strtol(str_wal_segment_size_mb, &endptr, 10);
+
+ /* verify that wal segment size is valid */
+ if (endptr == str_wal_segment_size_mb || *endptr != '\0')
+ {
+ fprintf(stderr,
+ _("%s: argument of --wal-segsize must be a number\n"),
+ progname);
+ exit(1);
+ }
+
+ wal_segment_size *= 1024 * 1024;
+ if (!IsValidWalSegSize(wal_segment_size))
+ {
+ fprintf(stderr,
+ _("%s: argument of --wal-segsize must be a power of 2 between 1 and 1024\n"),
+ progname);
+ exit(1);
+ }
+ }
+
+ XLogFromFileName(fname, &tli, &segNo, wal_segment_size);
+ XLogEncryptionTweak(tweak, segNo, 0);
+ setup_encryption(false);
+ encr_inited = true;
+ decrypt_block(buf.data, buf.data, XLOG_BLCKSZ, tweak);
+
+ hdr = (XLogPageHeader) buf.data;
+ if (hdr->xlp_magic != XLOG_PAGE_MAGIC)
+ {
+ fprintf(stderr,
+ _("invalid magic number %04X. Make sure you've passed correct argument for --wal-segsize"),
+ hdr->xlp_magic);
+ exit(1);
+ }
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
+ else
+ {
+ if (str_wal_segment_size_mb != NULL)
+ {
+ fprintf(stderr,
+ _("%s: --wal-segsize is only accepted for encrypted WAL\n"),
+ progname);
+ exit(1);
+ }
+ }
+
+ longhdr = (XLogLongPageHeader) buf.data;
WalSegSz = longhdr->xlp_seg_size;
if (!IsValidWalSegSize(WalSegSz))
@@ -462,6 +542,23 @@ XLogDumpReadPage(XLogReaderState *state, XLogRecPtr targetPagePtr, int reqLen,
XLogDumpXLogRead(private->inpath, private->timeline, targetPagePtr,
readBuff, count);
+ if (data_encrypted)
+ {
+#ifdef USE_ENCRYPTION
+ char tweak[TWEAK_SIZE];
+ XLogSegNo readSegNo;
+ uint32 readSegOff;
+
+ XLByteToSeg(targetPagePtr, readSegNo, WalSegSz);
+ readSegOff = targetPagePtr % WalSegSz;
+
+ XLogEncryptionTweak(tweak, readSegNo, readSegOff);
+ decrypt_block(readBuff, readBuff, count, tweak);
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ }
+
return count;
}
@@ -810,6 +907,12 @@ usage(void)
printf(_(" -b, --bkp-details output detailed information about backup blocks\n"));
printf(_(" -e, --end=RECPTR stop reading at WAL location RECPTR\n"));
printf(_(" -f, --follow keep retrying after reaching end of WAL\n"));
+#ifdef USE_ENCRYPTION
+ printf(_(" -K, --encryption-key-command=COMMAND\n"
+ " command that returns encryption key\n"));
+ printf(_(" -w, --wal-segsize=SIZE\n"
+ " size of WAL segments, in megabytes\n"));
+#endif /* USE_ENCRYPTION */
printf(_(" -n, --limit=N number of records to display\n"));
printf(_(" -p, --path=PATH directory in which to find log segment files or a\n"
" directory with a ./pg_wal that contains such files\n"
@@ -844,6 +947,7 @@ main(int argc, char **argv)
{"end", required_argument, NULL, 'e'},
{"follow", no_argument, NULL, 'f'},
{"help", no_argument, NULL, '?'},
+ {"encryption-key-command", required_argument, NULL, 'K'},
{"limit", required_argument, NULL, 'n'},
{"path", required_argument, NULL, 'p'},
{"rmgr", required_argument, NULL, 'r'},
@@ -851,6 +955,7 @@ main(int argc, char **argv)
{"timeline", required_argument, NULL, 't'},
{"xid", required_argument, NULL, 'x'},
{"version", no_argument, NULL, 'V'},
+ {"wal-segsize", required_argument, NULL, 'w'},
{"stats", optional_argument, NULL, 'z'},
{NULL, 0, NULL, 0}
};
@@ -886,7 +991,7 @@ main(int argc, char **argv)
goto bad_argument;
}
- while ((option = getopt_long(argc, argv, "be:?fn:p:r:s:t:Vx:z",
+ while ((option = getopt_long(argc, argv, "be:?fK:n:p:r:s:t:Vx:z",
long_options, &optindex)) != -1)
{
switch (option)
@@ -910,6 +1015,12 @@ main(int argc, char **argv)
usage();
exit(EXIT_SUCCESS);
break;
+#ifdef USE_ENCRYPTION
+ case 'K':
+ encryption_key_command = pg_strdup(optarg);
+ data_encrypted = true;
+ break;
+#endif /* USE_ENCRYPTION */
case 'n':
if (sscanf(optarg, "%d", &config.stop_after_records) != 1)
{
@@ -970,6 +1081,9 @@ main(int argc, char **argv)
puts("pg_waldump (PostgreSQL) " PG_VERSION);
exit(EXIT_SUCCESS);
break;
+ case 'w':
+ str_wal_segment_size_mb = pg_strdup(optarg);
+ break;
case 'x':
if (sscanf(optarg, "%u", &config.filter_by_xid) != 1)
{
@@ -1117,6 +1231,12 @@ main(int argc, char **argv)
/* done with argument parsing, do the actual work */
+ if (data_encrypted && !encr_inited)
+ {
+ setup_encryption(false);
+ encr_inited = false;
+ }
+
/* we have everything we need, start reading */
xlogreader_state = XLogReaderAllocate(WalSegSz, XLogDumpReadPage,
&private);
diff --git a/src/common/string.c b/src/common/string.c
index 3260d37a84..57ede1dd14 100644
--- a/src/common/string.c
+++ b/src/common/string.c
@@ -42,6 +42,45 @@ pg_str_endswith(const char *str, const char *end)
return strcmp(str, end) == 0;
}
+/*
+ * Helper function to check if a page is completely empty.
+ *
+ * TODO Invent name that is more consistent with that of the other function(s)
+ * in this module.
+ */
+bool
+IsAllZero(const char *input, Size size)
+{
+ const char *pos = input;
+ const char *aligned_start = (char *) MAXALIGN64(input);
+ const char *end = input + size;
+
+ /* Check 1 byte at a time until pos is 8 byte aligned */
+ while (pos < aligned_start)
+ if (*pos++ != 0)
+ return false;
+
+ /*
+ * Run 8 parallel 8 byte checks in one iteration. On 2016 hardware
+ * slightly faster than 4 parallel checks.
+ */
+ while (pos + 8 * sizeof(uint64) <= end)
+ {
+ uint64 *p = (uint64 *) pos;
+
+ if ((p[0] | p[1] | p[2] | p[3] | p[4] | p[5] | p[6] | p[7]) != 0)
+ return false;
+ pos += 8 * sizeof(uint64);
+ }
+
+ /* Handle unaligned tail. */
+ while (pos < end)
+ if (*pos++ != 0)
+ return false;
+
+ return true;
+}
+
/*
* strtoint --- just like strtol, but returns int not long
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 421ba6d775..c7fea0f6cb 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -258,6 +258,7 @@ extern void UpdateControlFile(void);
extern uint64 GetSystemIdentifier(void);
extern char *GetMockAuthenticationNonce(void);
extern bool DataChecksumsEnabled(void);
+extern bool DataEncryptionEnabled(void);
extern XLogRecPtr GetFakeLSNForUnloggedRel(void);
extern Size XLOGShmemSize(void);
extern void XLOGShmemInit(void);
diff --git a/src/include/access/xlog_internal.h b/src/include/access/xlog_internal.h
index 30610b3ea9..4990c0bc97 100644
--- a/src/include/access/xlog_internal.h
+++ b/src/include/access/xlog_internal.h
@@ -52,7 +52,7 @@ typedef struct XLogPageHeaderData
uint32 xlp_rem_len; /* total len of remaining data for record */
} XLogPageHeaderData;
-#define SizeOfXLogShortPHD MAXALIGN(sizeof(XLogPageHeaderData))
+#define SizeOfXLogShortPHD XLOG_REC_ALIGN(sizeof(XLogPageHeaderData))
typedef XLogPageHeaderData *XLogPageHeader;
@@ -69,7 +69,7 @@ typedef struct XLogLongPageHeaderData
uint32 xlp_xlog_blcksz; /* just as a cross-check */
} XLogLongPageHeaderData;
-#define SizeOfXLogLongPHD MAXALIGN(sizeof(XLogLongPageHeaderData))
+#define SizeOfXLogLongPHD XLOG_REC_ALIGN(sizeof(XLogLongPageHeaderData))
typedef XLogLongPageHeaderData *XLogLongPageHeader;
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 773d9e6eba..ce4386d592 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -94,6 +94,11 @@ typedef enum DBState
} DBState;
/*
+ * Number of bytes reserved to store encryption sample in ControlFileData.
+ */
+#define ENCRYPTION_SAMPLE_SIZE 16
+
+/*
* Contents of pg_control.
*/
@@ -227,6 +232,11 @@ typedef struct ControlFileData
*/
char mock_authentication_nonce[MOCK_AUTH_NONCE_LEN];
+ /* Is data directory encrypted? */
+ bool data_encrypted;
+ /* Sample value for encryption key verification */
+ uint8 encryption_verification[ENCRYPTION_SAMPLE_SIZE];
+
/* CRC of all above ... MUST BE LAST! */
pg_crc32c crc;
} ControlFileData;
diff --git a/src/include/common/string.h b/src/include/common/string.h
index 78a450192e..26fbfdc51e 100644
--- a/src/include/common/string.h
+++ b/src/include/common/string.h
@@ -13,5 +13,5 @@
extern bool pg_str_endswith(const char *str, const char *end);
extern int strtoint(const char *pg_restrict str, char **pg_restrict endptr,
int base);
-
+extern bool IsAllZero(const char *input, Size size);
#endif /* COMMON_STRING_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index e167ee8fcb..bc01ace87a 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -429,6 +429,7 @@ extern void BaseInit(void);
/* in utils/init/miscinit.c */
extern bool IgnoreSystemIndexes;
extern PGDLLIMPORT bool process_shared_preload_libraries_in_progress;
+extern char *encryption_library_string;
extern char *session_preload_libraries_string;
extern char *shared_preload_libraries_string;
extern char *local_preload_libraries_string;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index d59c24ae23..10d4f3423f 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -874,6 +874,9 @@ typedef enum
WAIT_EVENT_DATA_FILE_TRUNCATE,
WAIT_EVENT_DATA_FILE_WRITE,
WAIT_EVENT_DSM_FILL_ZERO_WRITE,
+ WAIT_EVENT_KDF_FILE_READ,
+ WAIT_EVENT_KDF_FILE_SYNC,
+ WAIT_EVENT_KDF_FILE_WRITE,
WAIT_EVENT_LOCK_FILE_ADDTODATADIR_READ,
WAIT_EVENT_LOCK_FILE_ADDTODATADIR_SYNC,
WAIT_EVENT_LOCK_FILE_ADDTODATADIR_WRITE,
diff --git a/src/include/storage/copydir.h b/src/include/storage/copydir.h
index 4fef3e2107..8aea3eb2ed 100644
--- a/src/include/storage/copydir.h
+++ b/src/include/storage/copydir.h
@@ -13,7 +13,9 @@
#ifndef COPYDIR_H
#define COPYDIR_H
-extern void copydir(char *fromdir, char *todir, bool recurse);
-extern void copy_file(char *fromfile, char *tofile);
+#include "storage/relfilenode.h"
+
+extern void copydir(char *fromdir, char *todir, RelFileNode *fromNode, RelFileNode *toNode);
+extern void copy_file(char *fromfile, char *tofile, RelFileNode *fromNode, RelFileNode *toNode, ForkNumber fromForkNum, ForkNumber toForkNum, int segment);
#endif /* COPYDIR_H */
diff --git a/src/include/storage/encryption.h b/src/include/storage/encryption.h
new file mode 100644
index 0000000000..0a49f5a949
--- /dev/null
+++ b/src/include/storage/encryption.h
@@ -0,0 +1,181 @@
+/*-------------------------------------------------------------------------
+ *
+ * encryption.h
+ * Full database encryption support
+ *
+ *
+ * Portions Copyright (c) 1996-2015, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/storage/encryption.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef ENCRYPTION_H
+#define ENCRYPTION_H
+
+#include "access/xlogdefs.h"
+#include "port/pg_crc32c.h"
+
+/*
+ * OpenSSL is currently the only implementation of encryption we use.
+ */
+#ifdef USE_OPENSSL
+#define USE_ENCRYPTION
+#endif
+
+/*
+ * Common error message issued when particular code path cannot be executed
+ * due to absence of the OpenSSL library.
+ */
+#define ENCRYPTION_NOT_SUPPORTED_MSG \
+ encryption_error(true, \
+ "data encryption cannot be used because SSL is not supported by this build\n" \
+ "Compile with --with-openssl to use SSL connections.")
+#ifdef USE_ENCRYPTION
+/*
+ * Full database encryption key.
+ *
+ * EVP_aes_256_xts() needs the key twice as long as AES would do in general.
+ */
+#define ENCRYPTION_KEY_LENGTH 64
+
+/*
+ * TODO Tune these values.
+ */
+#define ENCRYPTION_PWD_MIN_LENGTH 8
+#define ENCRYPTION_PWD_MAX_LENGTH 16
+#define ENCRYPTION_KDF_NITER 1048576
+#define ENCRYPTION_KDF_SALT_LEN 8
+
+extern unsigned char *encryption_key;
+#endif /* USE_ENCRYPTION */
+
+/*
+ * The encrypted data is a series of blocks of size
+ * ENCRYPTION_BLOCK. Currently we use the EVP_aes_256_xts implementation. Make
+ * sure the following constants match if adopting another algorithm.
+ */
+#define ENCRYPTION_BLOCK 16
+
+#ifdef USE_ENCRYPTION
+/*
+ * The openssl EVP API refers to a block in terms of padding of the output
+ * chunk. That's the purpose of this constant. However the openssl
+ * implementation of AES XTS still uses the 16-byte block internally, as
+ * defined by ENCRYPTION_BLOCK.
+ */
+#define ENCRYPTION_BLOCK_OPENSSL 1
+
+#define TWEAK_SIZE 16
+
+#define KDF_PARAMS_FILE "global/pg_keysetup"
+#define KDF_PARAMS_FILE_SIZE 512
+
+/*
+ * Key derivation function.
+ */
+typedef enum KDFKind
+{
+ KDF_OPENSSL_PKCS5_PBKDF2_HMAC_SHA = 0
+} KFDKind;
+
+typedef struct KDFParamsPBKDF2
+{
+ unsigned long int niter;
+ unsigned char salt[ENCRYPTION_KDF_SALT_LEN];
+} KDFParamsPBKDF2;
+
+/*
+ * Parameters of the key derivation function.
+ *
+ * The parameters are generated by initdb and stored into a file, which is
+ * then read during PG startup. This is similar to storing various settings in
+ * pg_control. However an existing KDF file is read only, so it does not have
+ * to be stored in shared memory.
+ */
+typedef struct KDFParamsData
+{
+ KFDKind function;
+
+ /*
+ * Function-specific parameters.
+ */
+ union
+ {
+ KDFParamsPBKDF2 pbkdf2;
+ } data;
+
+ /* CRC of all above ... MUST BE LAST! */
+ pg_crc32c crc;
+} KDFParamsData;
+
+extern KDFParamsData *KDFParams;
+#endif /* USE_ENCRYPTION */
+
+extern PGDLLIMPORT bool data_encrypted;
+
+#ifdef USE_ENCRYPTION
+#ifndef FRONTEND
+extern void write_kdf_file(void);
+extern void read_kdf_file(void);
+#endif /* FRONTEND */
+
+extern PGDLLIMPORT char *encryption_key_command;
+
+extern char *encryption_buffer;
+extern Size encryption_buf_size;
+
+extern void enlarge_encryption_buffer(Size new_size);
+#endif /* USE_ENCRYPTION */
+
+/*
+ * If one XLOG record ended and the following one started in the same block,
+ * we'd have to either encrypt and decrypt both records together, or encrypt
+ * (after having zeroed the part of the block occupied by the other record)
+ * and decrypt them separate. Neither approach is compatible with streaming
+ * replication. In the first case we can't ask standby not to decrypt the
+ * first record until the second has been streamed. The second approach would
+ * imply streaming of two different versions of the same block two times.
+ *
+ * We avoid this problem by aligning XLOG records to the encryption block
+ * size. This way no adjacent XLOG records should appear in the same block.
+ *
+ * For similar reasons, the alignment to ENCRYPTION_BLOCK also has to be
+ * applied when storing changes to disk in reorderbuffer.c. Another module
+ * that takes the block into account is buffile.c.
+ *
+ * TODO If the configuration allows walsender to decrypt the XLOG stream
+ * before sending it, adjust this expression so that the additional padding is
+ * not added to XLOG records. (Since the XLOG alignment cannot change without
+ * initdb, the same would apply to the configuration variable that makes
+ * walsender perform the decryption. Does such a variable make sense?)
+ */
+#define DO_ENCRYPTION_BLOCK_ALIGN data_encrypted
+
+/*
+ * Use TYPEALIGN64 since besides record size we also need to align XLogRecPtr.
+ */
+#define ENCRYPTION_BLOCK_ALIGN(LEN) TYPEALIGN64(ENCRYPTION_BLOCK, (LEN))
+
+/*
+ * Universal computation of XLOG record alignment.
+ */
+#define XLOG_REC_ALIGN(LEN) ((DO_ENCRYPTION_BLOCK_ALIGN) ?\
+ ENCRYPTION_BLOCK_ALIGN(LEN) : MAXALIGN64(LEN))
+
+extern void setup_encryption(bool bootstrap);
+extern void setup_encryption_key(char *credentials, bool is_key, size_t len);
+extern void sample_encryption(char *buf);
+extern void encrypt_block(const char *input, char *output, Size size,
+ const char *tweak);
+extern void decrypt_block(const char *input, char *output, Size size,
+ const char *tweak);
+#ifdef USE_ENCRYPTION
+extern void encryption_error(bool fatal, char *message);
+#endif
+
+extern void XLogEncryptionTweak(char *tweak, XLogSegNo segment,
+ uint32 offset);
+
+#endif /* ENCRYPTION_H */
diff --git a/src/include/storage/fd.h b/src/include/storage/fd.h
index 8e7c9728f4..0567c68c0c 100644
--- a/src/include/storage/fd.h
+++ b/src/include/storage/fd.h
@@ -72,6 +72,7 @@ extern int FileRead(File file, char *buffer, int amount, uint32 wait_event_info)
extern int FileWrite(File file, char *buffer, int amount, uint32 wait_event_info);
extern int FileSync(File file, uint32 wait_event_info);
extern off_t FileSeek(File file, off_t offset, int whence);
+extern off_t FileTell(File file);
extern int FileTruncate(File file, off_t offset, uint32 wait_event_info);
extern void FileWriteback(File file, off_t offset, off_t nbytes, uint32 wait_event_info);
extern char *FilePathName(File file);
diff --git a/src/include/storage/reinit.h b/src/include/storage/reinit.h
index a62703c647..5efd101d03 100644
--- a/src/include/storage/reinit.h
+++ b/src/include/storage/reinit.h
@@ -20,7 +20,8 @@
extern void ResetUnloggedRelations(int op);
extern bool parse_filename_for_nontemp_relation(const char *name,
- int *oidchars, ForkNumber *fork);
+ int *oidchars, ForkNumber *fork,
+ int *segment);
#define UNLOGGED_RELATION_CLEANUP 0x0001
#define UNLOGGED_RELATION_INIT 0x0002
diff --git a/src/include/storage/smgr.h b/src/include/storage/smgr.h
index c843bbc969..d69611934c 100644
--- a/src/include/storage/smgr.h
+++ b/src/include/storage/smgr.h
@@ -145,4 +145,10 @@ extern void ForgetRelationFsyncRequests(RelFileNode rnode, ForkNumber forknum);
extern void ForgetDatabaseFsyncRequests(Oid dbid);
extern void DropRelationFiles(RelFileNode *delrels, int ndelrels, bool isRedo);
+#ifdef USE_ENCRYPTION
+extern BlockNumber ReencryptBlock(char *buffer, int blocks,
+ RelFileNode *srcNode, RelFileNode *dstNode,
+ ForkNumber srcForkNum, ForkNumber dstForkNum,
+ BlockNumber blockNum);
+#endif
#endif /* SMGR_H */
diff --git a/src/test/modules/buffile/Makefile b/src/test/modules/buffile/Makefile
new file mode 100644
index 0000000000..b658c0dd8d
--- /dev/null
+++ b/src/test/modules/buffile/Makefile
@@ -0,0 +1,21 @@
+PG_CONFIG ?= pg_config
+MODULE_big = buffile_test
+OBJS = buffile.o $(WIN32RES)
+PGFILEDESC = "buffile_test"
+
+EXTENSION = buffile
+DATA = buffile--1.0.sql
+
+REGRESS = test_00 test_01 test_02 test_03 test_04 test_05 test_06 test_07 \
+test_08 test_09 test_10 test_11 test_12
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/buffile
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/buffile/README b/src/test/modules/buffile/README
new file mode 100644
index 0000000000..9c80787797
--- /dev/null
+++ b/src/test/modules/buffile/README
@@ -0,0 +1,11 @@
+This extension was written to check if changes introduced due to cluster
+encryption do not break buffile.c.
+
+Caution: To make the test cheaper, it was decided to adjust the segment sizeq,
+see
+
+#define MAX_PHYSICAL_FILESIZE (4 * BLCKSZ)
+
+in buffile.c. BLCKSZ is 8192 (the default). All the tests rely on this
+value. So if you haven't compiled Postgres with this value, the tests will
+create 1 GB files and they will fail.
diff --git a/src/test/modules/buffile/buffile--1.0.sql b/src/test/modules/buffile/buffile--1.0.sql
new file mode 100644
index 0000000000..82baa043f7
--- /dev/null
+++ b/src/test/modules/buffile/buffile--1.0.sql
@@ -0,0 +1,39 @@
+CREATE FUNCTION buffile_create()
+RETURNS void
+AS 'MODULE_PATHNAME', 'buffile_create'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_close()
+RETURNS void
+AS 'MODULE_PATHNAME', 'buffile_close'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_write(text)
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'buffile_write'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_read(bigint)
+RETURNS bytea
+AS 'MODULE_PATHNAME', 'buffile_read'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_seek(int, bigint)
+RETURNS int
+AS 'MODULE_PATHNAME', 'buffile_seek'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_assert_fileno(int)
+RETURNS void
+AS 'MODULE_PATHNAME', 'buffile_assert_fileno'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_test_shared()
+RETURNS void
+AS 'MODULE_PATHNAME', 'buffile_test_shared'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_test_shared_append()
+RETURNS void
+AS 'MODULE_PATHNAME', 'buffile_test_shared_append'
+LANGUAGE C;
diff --git a/src/test/modules/buffile/buffile.c b/src/test/modules/buffile/buffile.c
new file mode 100644
index 0000000000..5cc953c529
--- /dev/null
+++ b/src/test/modules/buffile/buffile.c
@@ -0,0 +1,351 @@
+#include "postgres.h"
+#include "fmgr.h"
+#include "lib/stringinfo.h"
+#include "storage/buffile.h"
+#include "utils/array.h"
+#include "utils/builtins.h"
+#include "utils/lsyscache.h"
+#include "utils/memutils.h"
+#include "utils/resowner.h"
+
+PG_MODULE_MAGIC;
+
+/*
+ * To cover various corner cases, the tests assume MAX_PHYSICAL_FILESIZE to be
+ * exactly MAX_PHYSICAL_FILESIZE_TEST.
+ */
+#define MAX_PHYSICAL_FILESIZE_TEST (4 * BLCKSZ)
+
+static BufFile *bf = NULL;
+
+static void check_file(void);
+
+extern Datum buffile_create(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_create);
+Datum
+buffile_create(PG_FUNCTION_ARGS)
+{
+ MemoryContext old_cxt;
+ ResourceOwner old_ro;
+
+ if (bf != NULL)
+ elog(ERROR, "file already exists");
+
+ old_cxt = MemoryContextSwitchTo(TopMemoryContext);
+
+ /*
+ * Make sure the file is not deleted across function calls.
+ */
+ old_ro = CurrentResourceOwner;
+ CurrentResourceOwner = TopTransactionResourceOwner;
+
+ bf = BufFileCreateTemp(false);
+
+ CurrentResourceOwner = old_ro;
+ MemoryContextSwitchTo(old_cxt);
+
+ PG_RETURN_VOID();
+}
+
+extern Datum buffile_close(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_close);
+Datum
+buffile_close(PG_FUNCTION_ARGS)
+{
+ if (bf == NULL)
+ elog(ERROR, "there's no file to close");
+
+ BufFileClose(bf);
+ bf = NULL;
+
+ PG_RETURN_VOID();
+}
+
+extern Datum buffile_write(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_write);
+Datum
+buffile_write(PG_FUNCTION_ARGS)
+{
+ Datum d = PG_GETARG_DATUM(0);
+ char *s = TextDatumGetCString(d);
+ size_t res;
+
+ check_file();
+ res = BufFileWrite(bf, s, strlen(s));
+
+ PG_RETURN_INT64(res);
+}
+
+extern Datum buffile_read(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_read);
+Datum
+buffile_read(PG_FUNCTION_ARGS)
+{
+ int64 size = PG_GETARG_INT64(0);
+ StringInfo buf = makeStringInfo();
+ size_t res_size;
+ bytea *result;
+
+ check_file();
+
+ enlargeStringInfo(buf, size);
+ res_size = BufFileRead(bf, buf->data, size);
+ buf->len = res_size;
+
+ result = DatumGetByteaPP(DirectFunctionCall1(bytearecv,
+ PointerGetDatum(buf)));
+ PG_RETURN_BYTEA_P(result);
+}
+
+extern Datum buffile_seek(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_seek);
+Datum
+buffile_seek(PG_FUNCTION_ARGS)
+{
+ int32 fileno = PG_GETARG_INT32(0);
+ int64 offset = PG_GETARG_INT64(1);
+ int32 res;
+
+ check_file();
+ res = BufFileSeek(bf, fileno, offset, SEEK_SET);
+
+ PG_RETURN_INT32(res);
+}
+
+extern Datum buffile_assert_fileno(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_assert_fileno);
+Datum
+buffile_assert_fileno(PG_FUNCTION_ARGS)
+{
+ int32 fileno_expected = PG_GETARG_INT32(0);
+ int32 fileno;
+ off_t offset;
+
+ check_file();
+ BufFileTell(bf, &fileno, &offset);
+
+ if (fileno != fileno_expected)
+ {
+ /*
+ * Bring the backend down so that the following tests have no chance
+ * to create the 1GB files.
+ */
+ elog(FATAL, "file number does not match");
+ }
+
+ PG_RETURN_VOID();
+}
+
+static void
+check_file(void)
+{
+ if (bf == NULL)
+ elog(ERROR, "the file is not opened");
+}
+
+/*
+ * This test is especially important for shared encrypted files, see the
+ * comments below.
+ */
+extern Datum buffile_test_shared(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_test_shared);
+Datum
+buffile_test_shared(PG_FUNCTION_ARGS)
+{
+ dsm_segment *seg;
+ SharedFileSet *fileset;
+ BufFile *bf_1,
+ *bf_2;
+ char *data_1,
+ *data_2,
+ *data;
+ Size chunk_size_1,
+ chunk_size_2;
+ int fileno,
+ i;
+ off_t offset,
+ res,
+ total_size;
+
+ /*
+ * The size is not important, we actually do not need the shared memory.
+ * The segment is only needed to initialize the fileset.
+ */
+ seg = dsm_create(1024, 0);
+
+ /*
+ * The fileset must survive error handling, so that dsm_detach works fine.
+ * (The typical use case is that the fileset is in shared memory.)
+ */
+ fileset = (SharedFileSet *) MemoryContextAlloc(TopTransactionContext,
+ sizeof(SharedFileSet));
+ SharedFileSetInit(fileset, seg);
+
+ bf_1 = BufFileCreateShared(fileset, "file_1");
+
+ /*
+ * Write more data than the buffer size, so that we can check that the
+ * number of "useful bytes" word is only appended at the end of the
+ * segment, not after each buffer.
+ */
+ chunk_size_1 = BLCKSZ + 256;
+ data_1 = (char *) palloc(chunk_size_1);
+ memset(data_1, 1, chunk_size_1);
+ if (BufFileWrite(bf_1, data_1, chunk_size_1) != chunk_size_1)
+ elog(ERROR, "Failed to write data");
+ pfree(data_1);
+
+ /*
+ * Enforce buffer flush (The BufFileFlush() function is not exported).
+ * Thus the "useful bytes" metadata should appear at the current end the
+ * first file segment. The next write will have to seek back to overwrite
+ * the metadata.
+ */
+ BufFileTell(bf_1, &fileno, &offset);
+ if (BufFileSeek(bf_1, 0, 0, SEEK_SET) != 0)
+ elog(ERROR, "seek failed");
+ if (BufFileSeek(bf_1, fileno, offset, SEEK_SET) != 0)
+ elog(ERROR, "seek failed");
+
+ /*
+ * Write another chunk that does not fit into the first segment file. Thus
+ * the "useful bytes" metadata should appear at the end of both segments.
+ */
+ chunk_size_2 = 3 * BLCKSZ;
+ data_2 = (char *) palloc(chunk_size_2);
+ memset(data_2, 1, chunk_size_2);
+ if (BufFileWrite(bf_1, data_2, chunk_size_2) != chunk_size_2)
+ elog(ERROR, "Failed to write data");
+ pfree(data_2);
+ BufFileClose(bf_1);
+
+ /*
+ * The word indicating the number of "useful bytes" (i.e. the actual data
+ * w/o padding to buffer size) is stored at the end of each segment file.
+ * Check that this metadata is read correctly.
+ */
+ bf_2 = BufFileOpenShared(fileset, "file_1");
+ total_size = BufFileSize(bf_2);
+ if (total_size != (chunk_size_1 + chunk_size_2))
+ elog(ERROR, "Incorrect file size: %zu", total_size);
+
+ data = (char *) palloc(total_size);
+ res = BufFileRead(bf_2, data, total_size);
+ if (res != total_size)
+ elog(ERROR, "Incorrect chunk size read: %zu", res);
+ for (i = 0; i < total_size; i++)
+ if (data[i] != 1)
+ elog(ERROR, "Unexpected data read from the file");
+ pfree(data);
+ BufFileClose(bf_2);
+
+ dsm_detach(seg);
+
+ PG_RETURN_VOID();
+}
+
+
+/*
+ * Test BufFileAppend().
+ */
+extern Datum buffile_test_shared_append(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_test_shared_append);
+Datum
+buffile_test_shared_append(PG_FUNCTION_ARGS)
+{
+ dsm_segment *seg;
+ SharedFileSet *fileset;
+ BufFile *bf_1,
+ *bf_2,
+ *bf_3;
+ char *data;
+ Size chunk_size;
+ int i;
+ off_t res,
+ total_size;
+
+ seg = dsm_create(1024, 0);
+
+ fileset = (SharedFileSet *) MemoryContextAlloc(TopTransactionContext,
+ sizeof(SharedFileSet));
+ SharedFileSetInit(fileset, seg);
+
+ /*
+ * XXX Does the chunk size matter much?
+ */
+ chunk_size = 8;
+ data = (char *) palloc(chunk_size);
+ memset(data, 1, chunk_size);
+
+ bf_1 = BufFileCreateShared(fileset, "file_1");
+ if (BufFileWrite(bf_1, data, chunk_size) != chunk_size)
+ elog(ERROR, "Failed to write data");
+
+ bf_2 = BufFileCreateShared(fileset, "file_2");
+ if (BufFileWrite(bf_2, data, chunk_size) != chunk_size)
+ elog(ERROR, "Failed to write data");
+
+ /*
+ * Make sure it's read-only so that BufFileAppend() can accept it as
+ * source.
+ */
+ BufFileClose(bf_2);
+ bf_2 = BufFileOpenShared(fileset, "file_2");
+
+ bf_3 = BufFileCreateShared(fileset, "file_3");
+ if (BufFileWrite(bf_3, data, chunk_size) != chunk_size)
+ elog(ERROR, "Failed to write data");
+ BufFileClose(bf_3);
+ bf_3 = BufFileOpenShared(fileset, "file_3");
+
+ BufFileAppend(bf_1, bf_2);
+ BufFileAppend(bf_1, bf_3);
+
+ total_size = BufFileSize(bf_1);
+
+ /*
+ * The result should contain complete segments of bf_1 and bf_2 and the
+ * valid part of bf_3.
+ */
+ if (total_size != (2 * MAX_PHYSICAL_FILESIZE_TEST + chunk_size))
+ elog(ERROR, "Incorrect total size of the appended data: %zu",
+ total_size);
+
+ /*
+ * Check that data of the 2nd segment was decrypted correctly.
+ */
+ if (BufFileSeek(bf_1, 1, 0, SEEK_SET) != 0)
+ elog(ERROR, "seek failed");
+ res = BufFileRead(bf_1, data, chunk_size);
+ if (res != chunk_size)
+ elog(ERROR, "Incorrect chunk size read: %zu", res);
+ for (i = 0; i < chunk_size; i++)
+ if (data[i] != 1)
+ elog(ERROR, "Unexpected data read from the file");
+
+ /*
+ * And the same for the 3rd segment.
+ *
+ * TODO Reuse the code above by putting it into a function.
+ */
+ if (BufFileSeek(bf_1, 2, 0, SEEK_SET) != 0)
+ elog(ERROR, "seek failed");
+ res = BufFileRead(bf_1, data, chunk_size);
+ if (res != chunk_size)
+ elog(ERROR, "Incorrect chunk size read: %zu", res);
+ for (i = 0; i < chunk_size; i++)
+ if (data[i] != 1)
+ elog(ERROR, "Unexpected data read from the file");
+
+ BufFileClose(bf_1);
+ dsm_detach(seg);
+ PG_RETURN_VOID();
+}
diff --git a/src/test/modules/buffile/buffile.control b/src/test/modules/buffile/buffile.control
new file mode 100644
index 0000000000..8472c5a348
--- /dev/null
+++ b/src/test/modules/buffile/buffile.control
@@ -0,0 +1,5 @@
+# buffile_test extension
+comment = 'buffile_test'
+default_version = '1.0'
+module_pathname = '$libdir/buffile_test'
+relocatable = true
diff --git a/src/test/modules/buffile/expected/test_00.out b/src/test/modules/buffile/expected/test_00.out
new file mode 100644
index 0000000000..637c33689f
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_00.out
@@ -0,0 +1,47 @@
+CREATE EXTENSION IF NOT EXISTS buffile;
+-- This test only verifies that PG is compiled with a component file size of
+-- 32 kB (i.e. 4 buffers of 8 kB) instead of 1 GB. That seems appropriate for
+-- testing. Some other tests may rely on it.
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+-- Skip the first component file.
+SELECT buffile_seek(0, 32768);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+-- Write the first byte of the second component file. We can't simply
+-- buffile_seek() beyond the position 32768 as it would return EOF.
+SELECT buffile_write('a');
+ buffile_write
+---------------
+ 1
+(1 row)
+
+-- Enforce BufFileFlush(), which actually adds the component file.
+SELECT buffile_read(1);
+ buffile_read
+--------------
+ \x
+(1 row)
+
+-- Check that we're in the 2nd file, i.e. the file size is as expected.
+SELECT buffile_assert_fileno(1);
+ buffile_assert_fileno
+-----------------------
+
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_01.out b/src/test/modules/buffile/expected/test_01.out
new file mode 100644
index 0000000000..448bac171c
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_01.out
@@ -0,0 +1,59 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+SELECT buffile_seek(0, 1);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_write('abc');
+ buffile_write
+---------------
+ 3
+(1 row)
+
+SELECT buffile_seek(0, 0);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+-- Check that the trailing zeroes are not fetched.
+SELECT buffile_read(16);
+ buffile_read
+--------------
+ \x00616263
+(1 row)
+
+-- Adjust the number of useful bytes.
+SELECT buffile_write('abc');
+ buffile_write
+---------------
+ 3
+(1 row)
+
+-- ... and check again.
+SELECT buffile_seek(0, 0);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(16);
+ buffile_read
+------------------
+ \x00616263616263
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_02.out b/src/test/modules/buffile/expected/test_02.out
new file mode 100644
index 0000000000..f783d0cb24
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_02.out
@@ -0,0 +1,48 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+SELECT buffile_seek(0, 8189);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+-- Initialize the last 3 positions of the first buffer and the initial 3
+-- positions of the 2nd buffer.
+SELECT buffile_write('abcdef');
+ buffile_write
+---------------
+ 6
+(1 row)
+
+SELECT buffile_seek(0, 0);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+-- Read the first buffer.
+SELECT length(buffile_read(8192));
+ length
+--------
+ 8192
+(1 row)
+
+-- Only 3 bytes of the 2nd buffer should be fetched.
+SELECT length(buffile_read(8192));
+ length
+--------
+ 3
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_03.out b/src/test/modules/buffile/expected/test_03.out
new file mode 100644
index 0000000000..e899fa3b38
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_03.out
@@ -0,0 +1,27 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+-- Read from an empty file.
+SELECT buffile_seek(0, 8);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(16);
+ buffile_read
+--------------
+ \x
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_04.out b/src/test/modules/buffile/expected/test_04.out
new file mode 100644
index 0000000000..1f8eeabe48
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_04.out
@@ -0,0 +1,84 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+-- Write something near the end of the first buffer, but leave some trailing
+-- space.
+SELECT buffile_seek(0, 8184);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_write('abcd');
+ buffile_write
+---------------
+ 4
+(1 row)
+
+-- Leave the 2nd buffer empty, as well as a few leading bytes. Thus we should
+-- get a hole that spans the whole 2nd buffer as well as a few adjacent bytes
+-- on each side.
+SELECT buffile_seek(0, 2 * 8192 + 4);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_write('efgh');
+ buffile_write
+---------------
+ 4
+(1 row)
+
+-- Check the initial part of the hole, which crosses the boundary of the 1st
+-- and the 2nd buffer.
+SELECT buffile_seek(0, 8184);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(16);
+ buffile_read
+------------------------------------
+ \x61626364000000000000000000000000
+(1 row)
+
+-- Check the trailing part of the whole, which crosses the boundary of the 2nd
+-- and the 3rd buffer.
+SELECT buffile_seek(0, 2 * 8192 - 8);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(16);
+ buffile_read
+------------------------------------
+ \x00000000000000000000000065666768
+(1 row)
+
+-- Check that the hole contains nothing but zeroes.
+SELECT buffile_seek(0, 8192 - 4);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT btrim(buffile_read(8192 + 8), '\x00');
+ btrim
+-------
+ \x
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_05.out b/src/test/modules/buffile/expected/test_05.out
new file mode 100644
index 0000000000..6a73147711
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_05.out
@@ -0,0 +1,33 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+-- Seek does not extend the file if it's not followed by write.
+SELECT buffile_seek(0, 1);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_seek(0, 0);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(2);
+ buffile_read
+--------------
+ \x
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_06.out b/src/test/modules/buffile/expected/test_06.out
new file mode 100644
index 0000000000..def9af5112
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_06.out
@@ -0,0 +1,41 @@
+-- This test shows that the first component file (segment) stays empty, read
+-- stops prematurely even if it starts on that segment, even though it'd
+-- otherwise receive some data from the following one.
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+SELECT buffile_seek(0, 32768);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_write('a');
+ buffile_write
+---------------
+ 1
+(1 row)
+
+SELECT buffile_seek(0, 32767);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(2);
+ buffile_read
+--------------
+ \x
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_07.out b/src/test/modules/buffile/expected/test_07.out
new file mode 100644
index 0000000000..215280f0b0
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_07.out
@@ -0,0 +1,39 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+-- Write data at component file boundary and try to read it.
+SELECT buffile_seek(0, 32768);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_write('abcd');
+ buffile_write
+---------------
+ 4
+(1 row)
+
+SELECT buffile_seek(0, 32768);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(8);
+ buffile_read
+--------------
+ \x61626364
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_08.out b/src/test/modules/buffile/expected/test_08.out
new file mode 100644
index 0000000000..7c1069930c
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_08.out
@@ -0,0 +1,39 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+-- Write data across component file boundary and try to read it.
+SELECT buffile_seek(0, 32766);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_write('abcd');
+ buffile_write
+---------------
+ 4
+(1 row)
+
+SELECT buffile_seek(0, 32766);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(8);
+ buffile_read
+--------------
+ \x61626364
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_09.out b/src/test/modules/buffile/expected/test_09.out
new file mode 100644
index 0000000000..b9d325b676
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_09.out
@@ -0,0 +1,39 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+-- Write data across buffer boundary and try to read it.
+SELECT buffile_seek(0, 8190);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_write('abcd');
+ buffile_write
+---------------
+ 4
+(1 row)
+
+SELECT buffile_seek(0, 8190);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(8);
+ buffile_read
+--------------
+ \x61626364
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_10.out b/src/test/modules/buffile/expected/test_10.out
new file mode 100644
index 0000000000..8e457fdcda
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_10.out
@@ -0,0 +1,76 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+-- Write some data at the end of the buffer.
+SELECT buffile_seek(0, 8188);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_write('abcd');
+ buffile_write
+---------------
+ 4
+(1 row)
+
+SELECT buffile_seek(0, 8189);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+-- Enforce flush with the write position not at the end of the buffer. This is
+-- special by not moving curOffset to the next buffer.
+SELECT buffile_read(1);
+ buffile_read
+--------------
+ \x62
+(1 row)
+
+-- Therefore the next writes should eventually affect the original data. (Here
+-- we also test going directly from read to write and vice versa.)
+SELECT buffile_write('x');
+ buffile_write
+---------------
+ 1
+(1 row)
+
+SELECT buffile_read(1);
+ buffile_read
+--------------
+ \x64
+(1 row)
+
+-- Start a new buffer, i.e. force flushing of the previous one.
+SELECT buffile_write('z');
+ buffile_write
+---------------
+ 1
+(1 row)
+
+-- Check that the 'x' and 'y' letters are in the first buffer, not in the
+-- 2nd. (We read enough data to find any non-zero bytes in the 2nd buffer.)
+SELECT buffile_seek(0, 8188);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(4 + 8192);
+ buffile_read
+--------------
+ \x616278647a
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_11.out b/src/test/modules/buffile/expected/test_11.out
new file mode 100644
index 0000000000..c6804d08af
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_11.out
@@ -0,0 +1,34 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+SELECT buffile_write('abcd');
+ buffile_write
+---------------
+ 4
+(1 row)
+
+-- Seek beyond EOF not followed by write.
+SELECT buffile_seek(0, 5);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+-- Nothing should be fetched.
+SELECT buffile_read(8);
+ buffile_read
+--------------
+ \x
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_12.out b/src/test/modules/buffile/expected/test_12.out
new file mode 100644
index 0000000000..3dce7eaf0f
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_12.out
@@ -0,0 +1,12 @@
+SELECT buffile_test_shared();
+ buffile_test_shared
+---------------------
+
+(1 row)
+
+SELECT buffile_test_shared_append();
+ buffile_test_shared_append
+----------------------------
+
+(1 row)
+
diff --git a/src/test/modules/buffile/sql/test_00.sql b/src/test/modules/buffile/sql/test_00.sql
new file mode 100644
index 0000000000..c907a5e195
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_00.sql
@@ -0,0 +1,18 @@
+CREATE EXTENSION IF NOT EXISTS buffile;
+
+-- This test only verifies that PG is compiled with a component file size of
+-- 32 kB (i.e. 4 buffers of 8 kB) instead of 1 GB. That seems appropriate for
+-- testing. Some other tests may rely on it.
+BEGIN;
+SELECT buffile_create();
+-- Skip the first component file.
+SELECT buffile_seek(0, 32768);
+-- Write the first byte of the second component file. We can't simply
+-- buffile_seek() beyond the position 32768 as it would return EOF.
+SELECT buffile_write('a');
+-- Enforce BufFileFlush(), which actually adds the component file.
+SELECT buffile_read(1);
+-- Check that we're in the 2nd file, i.e. the file size is as expected.
+SELECT buffile_assert_fileno(1);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_01.sql b/src/test/modules/buffile/sql/test_01.sql
new file mode 100644
index 0000000000..8cc87de841
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_01.sql
@@ -0,0 +1,14 @@
+BEGIN;
+SELECT buffile_create();
+SELECT buffile_seek(0, 1);
+SELECT buffile_write('abc');
+SELECT buffile_seek(0, 0);
+-- Check that the trailing zeroes are not fetched.
+SELECT buffile_read(16);
+-- Adjust the number of useful bytes.
+SELECT buffile_write('abc');
+-- ... and check again.
+SELECT buffile_seek(0, 0);
+SELECT buffile_read(16);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_02.sql b/src/test/modules/buffile/sql/test_02.sql
new file mode 100644
index 0000000000..35d34722d7
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_02.sql
@@ -0,0 +1,13 @@
+BEGIN;
+SELECT buffile_create();
+SELECT buffile_seek(0, 8189);
+-- Initialize the last 3 positions of the first buffer and the initial 3
+-- positions of the 2nd buffer.
+SELECT buffile_write('abcdef');
+SELECT buffile_seek(0, 0);
+-- Read the first buffer.
+SELECT length(buffile_read(8192));
+-- Only 3 bytes of the 2nd buffer should be fetched.
+SELECT length(buffile_read(8192));
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_03.sql b/src/test/modules/buffile/sql/test_03.sql
new file mode 100644
index 0000000000..a95391f7c3
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_03.sql
@@ -0,0 +1,7 @@
+BEGIN;
+SELECT buffile_create();
+-- Read from an empty file.
+SELECT buffile_seek(0, 8);
+SELECT buffile_read(16);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_04.sql b/src/test/modules/buffile/sql/test_04.sql
new file mode 100644
index 0000000000..64e8d39f94
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_04.sql
@@ -0,0 +1,25 @@
+BEGIN;
+SELECT buffile_create();
+-- Write something near the end of the first buffer, but leave some trailing
+-- space.
+SELECT buffile_seek(0, 8184);
+SELECT buffile_write('abcd');
+-- Leave the 2nd buffer empty, as well as a few leading bytes. Thus we should
+-- get a hole that spans the whole 2nd buffer as well as a few adjacent bytes
+-- on each side.
+SELECT buffile_seek(0, 2 * 8192 + 4);
+SELECT buffile_write('efgh');
+-- Check the initial part of the hole, which crosses the boundary of the 1st
+-- and the 2nd buffer.
+SELECT buffile_seek(0, 8184);
+SELECT buffile_read(16);
+-- Check the trailing part of the whole, which crosses the boundary of the 2nd
+-- and the 3rd buffer.
+SELECT buffile_seek(0, 2 * 8192 - 8);
+SELECT buffile_read(16);
+-- Check that the hole contains nothing but zeroes.
+SELECT buffile_seek(0, 8192 - 4);
+SELECT btrim(buffile_read(8192 + 8), '\x00');
+
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_05.sql b/src/test/modules/buffile/sql/test_05.sql
new file mode 100644
index 0000000000..5fd642e558
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_05.sql
@@ -0,0 +1,8 @@
+BEGIN;
+SELECT buffile_create();
+-- Seek does not extend the file if it's not followed by write.
+SELECT buffile_seek(0, 1);
+SELECT buffile_seek(0, 0);
+SELECT buffile_read(2);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_06.sql b/src/test/modules/buffile/sql/test_06.sql
new file mode 100644
index 0000000000..bbf506ec0e
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_06.sql
@@ -0,0 +1,11 @@
+-- This test shows that the first component file (segment) stays empty, read
+-- stops prematurely even if it starts on that segment, even though it'd
+-- otherwise receive some data from the following one.
+BEGIN;
+SELECT buffile_create();
+SELECT buffile_seek(0, 32768);
+SELECT buffile_write('a');
+SELECT buffile_seek(0, 32767);
+SELECT buffile_read(2);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_07.sql b/src/test/modules/buffile/sql/test_07.sql
new file mode 100644
index 0000000000..ca78506ecb
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_07.sql
@@ -0,0 +1,9 @@
+BEGIN;
+SELECT buffile_create();
+-- Write data at component file boundary and try to read it.
+SELECT buffile_seek(0, 32768);
+SELECT buffile_write('abcd');
+SELECT buffile_seek(0, 32768);
+SELECT buffile_read(8);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_08.sql b/src/test/modules/buffile/sql/test_08.sql
new file mode 100644
index 0000000000..0e5b1de383
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_08.sql
@@ -0,0 +1,9 @@
+BEGIN;
+SELECT buffile_create();
+-- Write data across component file boundary and try to read it.
+SELECT buffile_seek(0, 32766);
+SELECT buffile_write('abcd');
+SELECT buffile_seek(0, 32766);
+SELECT buffile_read(8);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_09.sql b/src/test/modules/buffile/sql/test_09.sql
new file mode 100644
index 0000000000..cc7060932e
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_09.sql
@@ -0,0 +1,9 @@
+BEGIN;
+SELECT buffile_create();
+-- Write data across buffer boundary and try to read it.
+SELECT buffile_seek(0, 8190);
+SELECT buffile_write('abcd');
+SELECT buffile_seek(0, 8190);
+SELECT buffile_read(8);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_10.sql b/src/test/modules/buffile/sql/test_10.sql
new file mode 100644
index 0000000000..63af760d9f
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_10.sql
@@ -0,0 +1,25 @@
+BEGIN;
+SELECT buffile_create();
+-- Write some data at the end of the buffer.
+SELECT buffile_seek(0, 8188);
+SELECT buffile_write('abcd');
+SELECT buffile_seek(0, 8189);
+-- Enforce flush with the write position not at the end of the buffer. This is
+-- special by not moving curOffset to the next buffer.
+SELECT buffile_read(1);
+
+-- Therefore the next writes should eventually affect the original data. (Here
+-- we also test going directly from read to write and vice versa.)
+SELECT buffile_write('x');
+SELECT buffile_read(1);
+
+-- Start a new buffer, i.e. force flushing of the previous one.
+SELECT buffile_write('z');
+
+-- Check that the 'x' and 'y' letters are in the first buffer, not in the
+-- 2nd. (We read enough data to find any non-zero bytes in the 2nd buffer.)
+SELECT buffile_seek(0, 8188);
+SELECT buffile_read(4 + 8192);
+
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_11.sql b/src/test/modules/buffile/sql/test_11.sql
new file mode 100644
index 0000000000..94300d253f
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_11.sql
@@ -0,0 +1,9 @@
+BEGIN;
+SELECT buffile_create();
+SELECT buffile_write('abcd');
+-- Seek beyond EOF not followed by write.
+SELECT buffile_seek(0, 5);
+-- Nothing should be fetched.
+SELECT buffile_read(8);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_12.sql b/src/test/modules/buffile/sql/test_12.sql
new file mode 100644
index 0000000000..a28ae395ef
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_12.sql
@@ -0,0 +1,2 @@
+SELECT buffile_test_shared();
+SELECT buffile_test_shared_append();
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-03-01 21:17 Robert Haas <robertmhaas@gmail.com>
parent: Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 37+ messages in thread
From: Robert Haas @ 2019-03-01 21:17 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
On Fri, Sep 14, 2018 at 10:04 AM Antonin Houska <ah@cybertec.at> wrote:
> Toshi Harada <harada.toshi@po.ntt-tx.co.jp> wrote:
> > Even if you apply "data-at-rest-encryption-wip-2018.07.25.patch" to the master branch,
> > the same patch error will occur.
>
> The version attached to this email should be applicable to the current
> branch. Sorry for the delay.
I have a few comments on this patch. It doesn't seem useful to go
through and make really detailed review comments on everything at this
stage, because it's pretty far from being committable, at least IMHO.
However, I have a few general thoughts.
First of all, this is an impressive piece of work. It's obvious even
from looking at this quickly that a lot of energy has gone into making
this work, and it touches a lot of stuff, not without reason. For
example, it's quite impressive that this touches things like the
write-ahead log, logical decoding, and the stats collector, all of
which write to disk.
However, this is also quite invasive. It changes a lot of code and it
doesn't do so in a very predictable way. It's not like you went
through and replaced every call to write() with a call to
SpecialEncyptionMagicWrite(). Rather, there are new #ifdef
USE_ENCRYPTION blocks all over the code base. I think it will be
important to look for ways to refactor this functionality to reduce
that sort of thing to the absolute minimum possible. Encryption needs
to be something that the code needs to know about here and there, but
not everywhere.
Some of the changes are things that could perhaps be done as separate,
preparatory patches. For instance, pgstat.c gets a heavy refactoring
to make it do I/O in bigger chunks. There's no reason that you
couldn't separate some of those changes out, clean them up, and get
them committed separately. It would be more efficient even as things
stand. OK, well, there is one reason: we may be getting a
shared-memory stats collector soon, and if we do, then the need for
all of this will go away. But the general point is valid: whatever is
a useful cleanup apart from this patch should be done separately from
this patch.
As far as possible, we need to try to do things in the same way in the
encrypted and non-encrypted cases. For example, it's pretty hard to
feel good about code like this:
+ sz_hdr = sizeof(ReorderBufferDiskChange);
+ if (data_encrypted)
+#ifdef USE_ENCRYPTION
+ sz_hdr = TYPEALIGN(ENCRYPTION_BLOCK, sz_hdr);
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
You won't be able to have much confidence that this code works both
with and without encryption unless you test it both ways, because the
file format is different in those two cases, and not just by being
encrypted. That means that anyone who modifies this code in the
future has two ways of breaking it. They can break the normal case,
or they can break the encrypted case. If encryption were available as
a sort of service that everyone could use, then that would probably be
fine, but like I said above, I think something as invasive as this
currently is will lead to a lot of complaints.
The code needs a lot more documentation, not only SGML documentation
but also code comments and probably a README file someplace. There is
a lot of discussion in the comments here of encryption tweaks, but
there's no general introduction to the topic (what's a tweak?) or --
and I think this is important -- what the idea between the choice of
various tweaks in different places actually is. There's probably some
motivating philosophy behind the way the tweaks are chosen that should
be explained someplace. Think about it this way: if some hapless
developer, let's say me, wants to add a new module to PostgreSQL which
happens to write data to disk, that person needs an easy way to
understand what they need to do to make that new code play nice with
encryption. Right now, the code for every existing module is a little
different (uggh) and there's this encryption tweak stuff that you
about which you have to do something intelligent, but there's nothing
that tells you how to be intelligent about it, at least not that I
see. People are going to need something that looks more like a
formula that they can just copy, and a good introduction to the
principles in some appropriate place, too.
The interaction of this capability with certain tricks that PostgreSQL
plays needs some thought -- and documentation, at least developer
documentation, maybe user documentation. One area is recovery.
Writing WAL relies on the fact that if you are in the midst of
rewriting a block and the power fails, bytes that are the same in both
the old and new block images will be undisturbed; encryption will make
that false. How's that going to work? Hint bits also rely on this
principle. I thought there might be some interaction between this
work and wal_log_hints for this reason, but I see nothing of that sort
in the patch. Full page writes seem related too; I don't know how
something like this can be safe without both full_page_writes and
wal_log_hints forced on. I don't know whether using encryption should
forcibly override those settings or whether it should just refuse to
start unless they are set properly, but I think it probably needs to
do one or the other.
Hope this helps in some way. I think it would be good if this effort
went forward in some way.
Thanks,
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-03-14 13:28 Antonin Houska <ah@cybertec.at>
parent: Robert Haas <robertmhaas@gmail.com>
0 siblings, 1 reply; 37+ messages in thread
From: Antonin Houska @ 2019-03-14 13:28 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
Thanks for feedback!
Robert Haas <robertmhaas@gmail.com> wrote:
> However, this is also quite invasive. It changes a lot of code and it
> doesn't do so in a very predictable way. It's not like you went
> through and replaced every call to write() with a call to
> SpecialEncyptionMagicWrite(). Rather, there are new #ifdef
> USE_ENCRYPTION blocks all over the code base. I think it will be
> important to look for ways to refactor this functionality to reduce
> that sort of thing to the absolute minimum possible. Encryption needs
> to be something that the code needs to know about here and there, but
> not everywhere.
The point of #ifdef USE_ENCRYPTION is that we rely on OpenSSL, so the code
needs to compile even w/o OpenSSL (of course the encryption won't be enabled
in that case). I'll try to reduce the use of this construct only to the code
blocks that really reference the OpenSSL functions.
> Some of the changes are things that could perhaps be done as separate,
> preparatory patches. For instance, pgstat.c gets a heavy refactoring
> to make it do I/O in bigger chunks. There's no reason that you
> couldn't separate some of those changes out, clean them up, and get
> them committed separately. It would be more efficient even as things
> stand. OK, well, there is one reason: we may be getting a
> shared-memory stats collector soon, and if we do, then the need for
> all of this will go away. But the general point is valid: whatever is
> a useful cleanup apart from this patch should be done separately from
> this patch.
We'll omit the stats collector related changes so far because the patch [1] is
under active development, and that would require us to rebase our patch
often. And if the stats files will eventually go away, we won't need to
encrypt the statistics.
> As far as possible, we need to try to do things in the same way in the
> encrypted and non-encrypted cases. For example, it's pretty hard to
> feel good about code like this:
>
> + sz_hdr = sizeof(ReorderBufferDiskChange);
> + if (data_encrypted)
> +#ifdef USE_ENCRYPTION
> + sz_hdr = TYPEALIGN(ENCRYPTION_BLOCK, sz_hdr);
> +#else
> + ENCRYPTION_NOT_SUPPORTED_MSG;
> +#endif /* USE_ENCRYPTION */
>
> You won't be able to have much confidence that this code works both
> with and without encryption unless you test it both ways, because the
> file format is different in those two cases, and not just by being
> encrypted. That means that anyone who modifies this code in the
> future has two ways of breaking it. They can break the normal case,
> or they can break the encrypted case. If encryption were available as
> a sort of service that everyone could use, then that would probably be
> fine, but like I said above, I think something as invasive as this
> currently is will lead to a lot of complaints.
The problem I tried to solve here is that the whole encryption block (16
bytes) needs to be read and decrypted even if you need only part of its
data. In the snippet above I tried to ensure that the whole blocks are always
read, but I agree this approach is fragile.
Since buffile.c needs to handle this kind of problem (and it already does in
the last patch version), I think even other than temporary files could be
handled by this module. The patch below adds functions BufFileOpenTransient(),
BufFileWriteTransient(), etc. that can replace OpenTransientFile, write(),
etc. respectively. Once we implement the encryption, these functions will also
hide handling of the encryption blocks from user. In this (preliminary) patch
As an example, I applied this approach to ReorderBufferSerializeChange() and
the function seems a bit simpler now. (ReorderBufferRestoreChange() would need
more work to adopt this approach.)
> The code needs a lot more documentation, not only SGML documentation
> but also code comments and probably a README file someplace.
ok, we'll do that.
> The interaction of this capability with certain tricks that PostgreSQL
> plays needs some thought -- and documentation, at least developer
> documentation, maybe user documentation. One area is recovery.
> Writing WAL relies on the fact that if you are in the midst of
> rewriting a block and the power fails, bytes that are the same in both
> the old and new block images will be undisturbed; encryption will make
> that false. How's that going to work?
Yes, if only a single bit is different, decryption will turn the whole
encryption block (16 bytes) into garbage. So we need to enforce
full_page_writes=on if encryption is enabled. The last version of the patch
does not do that.
> Hint bits also rely on this principle. I thought there might be some
> interaction between this work and wal_log_hints for this reason, but I see
> nothing of that sort in the patch.
I'm not sure if the hint bit is still a problem if we first copy the shared
buffer to backend-local memory and encrypt it there. That's what the patch
does.
> Full page writes seem related too; I don't know how something like this can
> be safe without both full_page_writes and wal_log_hints forced on. I don't
> know whether using encryption should forcibly override those settings or
> whether it should just refuse to start unless they are set properly, but I
> think it probably needs to do one or the other.
Maybe it's more polite to refuse to start so that user knows what's going
on. I'm not sure if PG ever changes any configuration variable forcibly.
> Hope this helps in some way. I think it would be good if this effort
> went forward in some way.
Sure, it helps! Thanks.
[1] https://commitfest.postgresql.org/22/1708/
--
Antonin Houska
https://www.cybertec-postgresql.com
Attachments:
buf_file_transient.patch (text/x-diff, 25.4K) download
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 2fbfadd9f0..9fc00bde9e 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3874,12 +3874,6 @@ pgstat_get_wait_io(WaitEventIO w)
case WAIT_EVENT_RELATION_MAP_WRITE:
event_name = "RelationMapWrite";
break;
- case WAIT_EVENT_REORDER_BUFFER_READ:
- event_name = "ReorderBufferRead";
- break;
- case WAIT_EVENT_REORDER_BUFFER_WRITE:
- event_name = "ReorderBufferWrite";
- break;
case WAIT_EVENT_REORDER_LOGICAL_MAPPING_READ:
event_name = "ReorderLogicalMappingRead";
break;
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 2cfdf1c9ac..f958d27761 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -71,6 +71,7 @@
#include "replication/slot.h"
#include "replication/snapbuild.h" /* just for SnapBuildSnapDecRefcount */
#include "storage/bufmgr.h"
+#include "storage/buffile.h"
#include "storage/fd.h"
#include "storage/sinval.h"
#include "utils/builtins.h"
@@ -109,7 +110,7 @@ typedef struct ReorderBufferIterTXNEntry
XLogRecPtr lsn;
ReorderBufferChange *change;
ReorderBufferTXN *txn;
- int fd;
+ TransientBufFile *file;
XLogSegNo segno;
} ReorderBufferIterTXNEntry;
@@ -192,9 +193,11 @@ static void ReorderBufferExecuteInvalidations(ReorderBuffer *rb, ReorderBufferTX
static void ReorderBufferCheckSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn);
static void ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn);
static void ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
- int fd, ReorderBufferChange *change);
+ TransientBufFile *file, ReorderBufferChange *change);
+static void ReorderBufferWriteData(TransientBufFile *file, void *ptr, size_t size,
+ ReorderBufferTXN *txn);
static Size ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn,
- int *fd, XLogSegNo *segno);
+ TransientBufFile **file, XLogSegNo *segno);
static void ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
char *change);
static void ReorderBufferRestoreCleanup(ReorderBuffer *rb, ReorderBufferTXN *txn);
@@ -468,8 +471,8 @@ ReorderBufferReturnTupleBuf(ReorderBuffer *rb, ReorderBufferTupleBuf *tuple)
Oid *
ReorderBufferGetRelids(ReorderBuffer *rb, int nrelids)
{
- Oid *relids;
- Size alloc_len;
+ Oid *relids;
+ Size alloc_len;
alloc_len = sizeof(Oid) * nrelids;
@@ -988,7 +991,7 @@ ReorderBufferIterTXNInit(ReorderBuffer *rb, ReorderBufferTXN *txn)
for (off = 0; off < state->nr_txns; off++)
{
- state->entries[off].fd = -1;
+ state->entries[off].file = NULL;
state->entries[off].segno = 0;
}
@@ -1013,7 +1016,7 @@ ReorderBufferIterTXNInit(ReorderBuffer *rb, ReorderBufferTXN *txn)
{
/* serialize remaining changes */
ReorderBufferSerializeTXN(rb, txn);
- ReorderBufferRestoreChanges(rb, txn, &state->entries[off].fd,
+ ReorderBufferRestoreChanges(rb, txn, &state->entries[off].file,
&state->entries[off].segno);
}
@@ -1043,7 +1046,7 @@ ReorderBufferIterTXNInit(ReorderBuffer *rb, ReorderBufferTXN *txn)
/* serialize remaining changes */
ReorderBufferSerializeTXN(rb, cur_txn);
ReorderBufferRestoreChanges(rb, cur_txn,
- &state->entries[off].fd,
+ &state->entries[off].file,
&state->entries[off].segno);
}
cur_change = dlist_head_element(ReorderBufferChange, node,
@@ -1124,7 +1127,7 @@ ReorderBufferIterTXNNext(ReorderBuffer *rb, ReorderBufferIterTXNState *state)
dlist_delete(&change->node);
dlist_push_tail(&state->old_change, &change->node);
- if (ReorderBufferRestoreChanges(rb, entry->txn, &entry->fd,
+ if (ReorderBufferRestoreChanges(rb, entry->txn, &entry->file,
&state->entries[off].segno))
{
/* successfully restored changes from disk */
@@ -1163,8 +1166,8 @@ ReorderBufferIterTXNFinish(ReorderBuffer *rb,
for (off = 0; off < state->nr_txns; off++)
{
- if (state->entries[off].fd != -1)
- CloseTransientFile(state->entries[off].fd);
+ if (state->entries[off].file)
+ BufFileCloseTransient(state->entries[off].file);
}
/* free memory we might have "leaked" in the last *Next call */
@@ -1327,8 +1330,8 @@ ReorderBufferBuildTupleCidHash(ReorderBuffer *rb, ReorderBufferTXN *txn)
else
{
/*
- * Maybe we already saw this tuple before in this transaction,
- * but if so it must have the same cmin.
+ * Maybe we already saw this tuple before in this transaction, but
+ * if so it must have the same cmin.
*/
Assert(ent->cmin == change->data.tuplecid.cmin);
@@ -2254,7 +2257,7 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
{
dlist_iter subtxn_i;
dlist_mutable_iter change_i;
- int fd = -1;
+ TransientBufFile *file = NULL;
XLogSegNo curOpenSegNo = 0;
Size spilled = 0;
@@ -2281,13 +2284,13 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
* store in segment in which it belongs by start lsn, don't split over
* multiple segments tho
*/
- if (fd == -1 ||
+ if (file == NULL ||
!XLByteInSeg(change->lsn, curOpenSegNo, wal_segment_size))
{
char path[MAXPGPATH];
- if (fd != -1)
- CloseTransientFile(fd);
+ if (file)
+ BufFileCloseTransient(file);
XLByteToSeg(change->lsn, curOpenSegNo, wal_segment_size);
@@ -2299,16 +2302,11 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
curOpenSegNo);
/* open segment, create it if necessary */
- fd = OpenTransientFile(path,
- O_CREAT | O_WRONLY | O_APPEND | PG_BINARY);
-
- if (fd < 0)
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not open file \"%s\": %m", path)));
+ file = BufFileOpenTransient(path,
+ O_CREAT | O_WRONLY | O_APPEND | PG_BINARY);
}
- ReorderBufferSerializeChange(rb, txn, fd, change);
+ ReorderBufferSerializeChange(rb, txn, file, change);
dlist_delete(&change->node);
ReorderBufferReturnChange(rb, change);
@@ -2320,8 +2318,8 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
txn->nentries_mem = 0;
txn->serialized = true;
- if (fd != -1)
- CloseTransientFile(fd);
+ if (file)
+ BufFileCloseTransient(file);
}
/*
@@ -2329,15 +2327,13 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
*/
static void
ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
- int fd, ReorderBufferChange *change)
+ TransientBufFile *file, ReorderBufferChange *change)
{
- ReorderBufferDiskChange *ondisk;
+ ReorderBufferDiskChange hdr;
Size sz = sizeof(ReorderBufferDiskChange);
- ReorderBufferSerializeReserve(rb, sz);
-
- ondisk = (ReorderBufferDiskChange *) rb->outbuf;
- memcpy(&ondisk->change, change, sizeof(ReorderBufferChange));
+ memcpy((char *) &hdr + offsetof(ReorderBufferDiskChange, change),
+ change, sizeof(ReorderBufferChange));
switch (change->action)
{
@@ -2347,7 +2343,6 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
case REORDER_BUFFER_CHANGE_DELETE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT:
{
- char *data;
ReorderBufferTupleBuf *oldtup,
*newtup;
Size oldlen = 0;
@@ -2370,66 +2365,55 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
sz += newlen;
}
- /* make sure we have enough space */
- ReorderBufferSerializeReserve(rb, sz);
-
- data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
- /* might have been reallocated above */
- ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+ hdr.size = sz;
+ ReorderBufferWriteData(file, &hdr, sizeof(ReorderBufferDiskChange),
+ txn);
if (oldlen)
{
- memcpy(data, &oldtup->tuple, sizeof(HeapTupleData));
- data += sizeof(HeapTupleData);
-
- memcpy(data, oldtup->tuple.t_data, oldlen);
- data += oldlen;
+ ReorderBufferWriteData(file, &oldtup->tuple,
+ sizeof(HeapTupleData), txn);
+ ReorderBufferWriteData(file, oldtup->tuple.t_data, oldlen,
+ txn);
}
if (newlen)
{
- memcpy(data, &newtup->tuple, sizeof(HeapTupleData));
- data += sizeof(HeapTupleData);
-
- memcpy(data, newtup->tuple.t_data, newlen);
- data += newlen;
+ ReorderBufferWriteData(file, &newtup->tuple,
+ sizeof(HeapTupleData), txn);
+ ReorderBufferWriteData(file, newtup->tuple.t_data, newlen,
+ txn);
}
break;
}
case REORDER_BUFFER_CHANGE_MESSAGE:
{
- char *data;
Size prefix_size = strlen(change->data.msg.prefix) + 1;
sz += prefix_size + change->data.msg.message_size +
sizeof(Size) + sizeof(Size);
- ReorderBufferSerializeReserve(rb, sz);
-
- data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
- /* might have been reallocated above */
- ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+ hdr.size = sz;
+ ReorderBufferWriteData(file, &hdr,
+ sizeof(ReorderBufferDiskChange),
+ txn);
/* write the prefix including the size */
- memcpy(data, &prefix_size, sizeof(Size));
- data += sizeof(Size);
- memcpy(data, change->data.msg.prefix,
- prefix_size);
- data += prefix_size;
+ ReorderBufferWriteData(file, &prefix_size, sizeof(Size), txn);
+ ReorderBufferWriteData(file, change->data.msg.prefix,
+ prefix_size, txn);
/* write the message including the size */
- memcpy(data, &change->data.msg.message_size, sizeof(Size));
- data += sizeof(Size);
- memcpy(data, change->data.msg.message,
- change->data.msg.message_size);
- data += change->data.msg.message_size;
+ ReorderBufferWriteData(file, &change->data.msg.message_size,
+ sizeof(Size), txn);
+ ReorderBufferWriteData(file, change->data.msg.message,
+ change->data.msg.message_size, txn);
break;
}
case REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT:
{
Snapshot snap;
- char *data;
snap = change->data.snapshot;
@@ -2438,49 +2422,37 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
sizeof(TransactionId) * snap->subxcnt
;
- /* make sure we have enough space */
- ReorderBufferSerializeReserve(rb, sz);
- data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
- /* might have been reallocated above */
- ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+ hdr.size = sz;
+ ReorderBufferWriteData(file, &hdr,
+ sizeof(ReorderBufferDiskChange), txn);
- memcpy(data, snap, sizeof(SnapshotData));
- data += sizeof(SnapshotData);
+ ReorderBufferWriteData(file, snap, sizeof(SnapshotData), txn);
if (snap->xcnt)
- {
- memcpy(data, snap->xip,
- sizeof(TransactionId) * snap->xcnt);
- data += sizeof(TransactionId) * snap->xcnt;
- }
+ ReorderBufferWriteData(file, snap->xip,
+ sizeof(TransactionId) * snap->xcnt,
+ txn);
if (snap->subxcnt)
- {
- memcpy(data, snap->subxip,
- sizeof(TransactionId) * snap->subxcnt);
- data += sizeof(TransactionId) * snap->subxcnt;
- }
+ ReorderBufferWriteData(file, snap->subxip,
+ sizeof(TransactionId) * snap->subxcnt,
+ txn);
break;
}
case REORDER_BUFFER_CHANGE_TRUNCATE:
{
- Size size;
- char *data;
+ Size size;
/* account for the OIDs of truncated relations */
size = sizeof(Oid) * change->data.truncate.nrelids;
sz += size;
- /* make sure we have enough space */
- ReorderBufferSerializeReserve(rb, sz);
-
- data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
- /* might have been reallocated above */
- ondisk = (ReorderBufferDiskChange *) rb->outbuf;
-
- memcpy(data, change->data.truncate.relids, size);
- data += size;
+ hdr.size = sz;
+ ReorderBufferWriteData(file, &hdr, sizeof(ReorderBufferDiskChange),
+ txn);
+ ReorderBufferWriteData(file, change->data.truncate.relids, size,
+ txn);
break;
}
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
@@ -2489,27 +2461,21 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
/* ReorderBufferChange contains everything important */
break;
}
+}
- ondisk->size = sz;
-
- errno = 0;
- pgstat_report_wait_start(WAIT_EVENT_REORDER_BUFFER_WRITE);
- if (write(fd, rb->outbuf, ondisk->size) != ondisk->size)
- {
- int save_errno = errno;
-
- CloseTransientFile(fd);
-
- /* if write didn't set errno, assume problem is no disk space */
- errno = save_errno ? save_errno : ENOSPC;
+/*
+ * Wrapper for BufFileWriteTransient() that raises ERROR if the whole chunk
+ * was not written. XXX Should this be a macro?
+ */
+static void
+ReorderBufferWriteData(TransientBufFile *file, void *ptr, size_t size,
+ ReorderBufferTXN *txn)
+{
+ if (BufFileWriteTransient(file, ptr, size) != size)
ereport(ERROR,
(errcode_for_file_access(),
errmsg("could not write to data file for XID %u: %m",
txn->xid)));
- }
- pgstat_report_wait_end();
-
- Assert(ondisk->change.action == change->action);
}
/*
@@ -2517,7 +2483,7 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
*/
static Size
ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn,
- int *fd, XLogSegNo *segno)
+ TransientBufFile **file, XLogSegNo *segno)
{
Size restored = 0;
XLogSegNo last_segno;
@@ -2545,7 +2511,7 @@ ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn,
int readBytes;
ReorderBufferDiskChange *ondisk;
- if (*fd == -1)
+ if (*file == NULL)
{
char path[MAXPGPATH];
@@ -2562,18 +2528,13 @@ ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferSerializedPath(path, MyReplicationSlot, txn->xid,
*segno);
- *fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
- if (*fd < 0 && errno == ENOENT)
+ *file = BufFileOpenTransient(path, O_RDONLY | PG_BINARY);
+ if (*file == NULL)
{
- *fd = -1;
+ Assert(errno == ENOENT);
(*segno)++;
continue;
}
- else if (*fd < 0)
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not open file \"%s\": %m",
- path)));
}
/*
@@ -2582,22 +2543,16 @@ ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn,
* end of this file.
*/
ReorderBufferSerializeReserve(rb, sizeof(ReorderBufferDiskChange));
- pgstat_report_wait_start(WAIT_EVENT_REORDER_BUFFER_READ);
- readBytes = read(*fd, rb->outbuf, sizeof(ReorderBufferDiskChange));
- pgstat_report_wait_end();
+ readBytes = BufFileReadTransient(*file, rb->outbuf, sizeof(ReorderBufferDiskChange));
/* eof */
if (readBytes == 0)
{
- CloseTransientFile(*fd);
- *fd = -1;
+ BufFileCloseTransient(*file);
+ *file = NULL;
(*segno)++;
continue;
}
- else if (readBytes < 0)
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not read from reorderbuffer spill file: %m")));
else if (readBytes != sizeof(ReorderBufferDiskChange))
ereport(ERROR,
(errcode_for_file_access(),
@@ -2611,16 +2566,10 @@ ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn,
sizeof(ReorderBufferDiskChange) + ondisk->size);
ondisk = (ReorderBufferDiskChange *) rb->outbuf;
- pgstat_report_wait_start(WAIT_EVENT_REORDER_BUFFER_READ);
- readBytes = read(*fd, rb->outbuf + sizeof(ReorderBufferDiskChange),
- ondisk->size - sizeof(ReorderBufferDiskChange));
- pgstat_report_wait_end();
+ readBytes = BufFileReadTransient(*file, rb->outbuf + sizeof(ReorderBufferDiskChange),
+ ondisk->size - sizeof(ReorderBufferDiskChange));
- if (readBytes < 0)
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not read from reorderbuffer spill file: %m")));
- else if (readBytes != ondisk->size - sizeof(ReorderBufferDiskChange))
+ if (readBytes != ondisk->size - sizeof(ReorderBufferDiskChange))
ereport(ERROR,
(errcode_for_file_access(),
errmsg("could not read from reorderbuffer spill file: read %d instead of %u bytes",
@@ -2767,7 +2716,7 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
/* the base struct contains all the data, easy peasy */
case REORDER_BUFFER_CHANGE_TRUNCATE:
{
- Oid *relids;
+ Oid *relids;
relids = ReorderBufferGetRelids(rb,
change->data.truncate.nrelids);
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index c2c445dbf4..6c426c0509 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -41,6 +41,8 @@
#include "postgres.h"
+#include <unistd.h>
+
#include "executor/instrument.h"
#include "miscadmin.h"
#include "pgstat.h"
@@ -93,6 +95,27 @@ struct BufFile
PGAlignedBlock buffer;
};
+/*
+ * Buffered variant of a transient file. Unlike BufFile this is simpler in
+ * several ways: 1) it's not split into segments, 2) there's no need of seek,
+ * 3) there's no need to combine read and write access.
+ */
+struct TransientBufFile
+{
+ /* The underlying file. */
+ char *path;
+ int fileFlags;
+ int fd;
+
+ off_t offset; /* next read/write position in file */
+ int pos; /* next read/write position in buffer */
+ int nbytes; /* total # of valid bytes in buffer */
+
+ bool dirty; /* unsaved data in the buffer? */
+
+ PGAlignedBlock buffer;
+};
+
static BufFile *makeBufFileCommon(int nfiles);
static BufFile *makeBufFile(File firstfile);
static void extendBufFile(BufFile *file);
@@ -101,6 +124,9 @@ static void BufFileDumpBuffer(BufFile *file);
static int BufFileFlush(BufFile *file);
static File MakeNewSharedSegment(BufFile *file, int segment);
+static void BufFileLoadBufferTransient(TransientBufFile *file);
+static void BufFileDumpBufferTransient(TransientBufFile *file);
+
/*
* Create BufFile and perform the common initialization.
*/
@@ -831,3 +857,267 @@ BufFileAppend(BufFile *target, BufFile *source)
return startBlock;
}
+
+/*
+ * Open TransientBufFile at given path or create one if it does not
+ * exist. User will be allowed either to write to the file or to read from it,
+ * according to fileFlags, but not both.
+ */
+TransientBufFile *
+BufFileOpenTransient(const char *path, int fileFlags)
+{
+ TransientBufFile *file;
+ int fd;
+
+ /* Either read or write mode, but not both. */
+ Assert((fileFlags & O_RDWR) == 0);
+
+ fd = OpenTransientFile(path, fileFlags);
+ if (fd < 0)
+ {
+ /*
+ * If caller wants to read from file and the file is not there, he
+ * should be able to handle the condition on his own.
+ *
+ * XXX Shouldn't we always let caller evaluate errno?
+ */
+ if (errno == ENOENT && (fileFlags & O_RDONLY))
+ return NULL;
+
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not open file \"%s\": %m", path)));
+ }
+
+ file = (TransientBufFile *) palloc(sizeof(TransientBufFile));
+
+ file->path = pstrdup(path);
+ file->fileFlags = fileFlags;
+ file->fd = fd;
+
+ file->dirty = false;
+
+ file->pos = 0;
+ file->nbytes = 0;
+
+ if (file->fileFlags & O_APPEND)
+ {
+ /* Position the buffer at the end of the file. */
+ errno = 0;
+ file->offset = lseek(file->fd, 0, SEEK_END);
+ }
+ else
+ {
+ /* Load the initial part of the file. */
+ file->offset = 0L;
+ BufFileLoadBufferTransient(file);
+ }
+
+ if (errno > 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not initialize TransientBufFile for file \"%s\": %m",
+ path)));
+
+ return file;
+}
+
+/*
+ * Close a TransientBufFile.
+ */
+void
+BufFileCloseTransient(TransientBufFile *file)
+{
+ /* Flush any unwritten data. */
+ if (file->fileFlags & O_WRONLY && file->dirty && file->nbytes > 0)
+ {
+ BufFileDumpBufferTransient(file);
+
+ /*
+ * Caller of BufFileWriteTransient() recognizes the failure to flush
+ * buffer by the returned value, however this function has no return
+ * code.
+ */
+ if (file->dirty)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not flush file \"%s\": %m", file->path)));
+ }
+
+ /*
+ * XXX Should we raise ERROR if non-zero code is returned here, or is it
+ * enough if the failure cause WARNING during transaction commit?
+ */
+ CloseTransientFile(file->fd);
+
+ pfree(file->path);
+ pfree(file);
+}
+
+/*
+ * Like BufFileWrite() except it receives pointer to TransientBufFile.
+ *
+ * TODO Reuse the code of BufFileWrite() in a better way than copy & paste.
+ */
+size_t
+BufFileWriteTransient(TransientBufFile *file, void *ptr, size_t size)
+{
+ size_t nwritten = 0;
+ size_t nthistime;
+
+ Assert((file->fileFlags & O_WRONLY));
+
+ while (size > 0)
+ {
+ if (file->pos >= BLCKSZ)
+ {
+ Assert(file->dirty);
+
+ /* Buffer full, dump it out */
+ BufFileDumpBufferTransient(file);
+ if (file->dirty)
+ break; /* I/O error */
+ }
+
+ nthistime = BLCKSZ - file->pos;
+ if (nthistime > size)
+ nthistime = size;
+ Assert(nthistime > 0);
+
+ memcpy(file->buffer.data + file->pos, ptr, nthistime);
+
+ file->dirty = true;
+ file->pos += nthistime;
+ if (file->nbytes < file->pos)
+ file->nbytes = file->pos;
+ ptr = (void *) ((char *) ptr + nthistime);
+ size -= nthistime;
+ nwritten += nthistime;
+ }
+
+ return nwritten;
+}
+
+/*
+ * Like BufFileRead() except it receives pointer to TransientBufFile.
+ *
+ * TODO Reuse the code of BufFileRead() in a better way than copy & paste.
+ */
+size_t
+BufFileReadTransient(TransientBufFile *file, void *ptr, size_t size)
+{
+ size_t nread = 0;
+ size_t nthistime;
+
+ Assert((file->fileFlags & O_WRONLY) == 0);
+
+ while (size > 0)
+ {
+ if (file->pos >= file->nbytes)
+ {
+ /* Try to load more data into buffer. */
+ file->offset += file->pos;
+ file->pos = 0;
+ file->nbytes = 0;
+ BufFileLoadBufferTransient(file);
+ if (file->nbytes <= 0)
+ break; /* no more data available */
+ }
+
+ nthistime = file->nbytes - file->pos;
+ if (nthistime > size)
+ nthistime = size;
+ Assert(nthistime > 0);
+
+ memcpy(ptr, file->buffer.data + file->pos, nthistime);
+
+ file->pos += nthistime;
+ ptr = (void *) ((char *) ptr + nthistime);
+ size -= nthistime;
+ nread += nthistime;
+ }
+
+ return nread;
+}
+
+/*
+ * Load some data into buffer, if possible, starting from file->offset. At
+ * call, must have dirty = false, pos and nbytes = 0. On exit, nbytes is
+ * number of bytes loaded.
+ */
+static void
+BufFileLoadBufferTransient(TransientBufFile *file)
+{
+ Assert(!file->dirty);
+ Assert(file->pos == 0 && file->nbytes == 0);
+
+retry:
+
+ /*
+ * Read whatever we can get, up to a full bufferload.
+ */
+ errno = 0;
+ pgstat_report_wait_start(WAIT_EVENT_BUFFILE_READ);
+ file->nbytes = pg_pread(file->fd, file->buffer.data,
+ sizeof(file->buffer), file->offset);
+ pgstat_report_wait_end();
+
+ if (file->nbytes < 0)
+ {
+ /* TODO The W32 specific code, see FileWrite. */
+
+ /* OK to retry if interrupted */
+ if (errno == EINTR)
+ goto retry;
+
+ return; /* failed to write */
+ }
+
+ if (file->nbytes < 0)
+ file->nbytes = 0;
+ /* we choose not to advance offset here */
+}
+
+/*
+ * Write buffer contents to disk.
+ */
+static void
+BufFileDumpBufferTransient(TransientBufFile *file)
+{
+ int nwritten;
+
+ /* This function should only be needed during write access ... */
+ Assert(file->fileFlags & O_WRONLY);
+
+ /* ... and if there's some work to do. */
+ Assert(file->dirty);
+ Assert(file->nbytes > 0);
+
+retry:
+ errno = 0;
+ pgstat_report_wait_start(WAIT_EVENT_BUFFILE_WRITE);
+ nwritten = pg_pwrite(file->fd, file->buffer.data, file->nbytes,
+ file->offset);
+ pgstat_report_wait_end();
+
+ /* if write didn't set errno, assume problem is no disk space */
+ if (nwritten != file->nbytes && errno == 0)
+ errno = ENOSPC;
+
+ if (nwritten < 0)
+ {
+ /* TODO The W32 specific code, see FileWrite. */
+
+ /* OK to retry if interrupted */
+ if (errno == EINTR)
+ goto retry;
+
+ return; /* failed to write */
+ }
+
+ file->dirty = false;
+
+ file->offset += nwritten;
+ file->pos = 0;
+ file->nbytes = 0;
+}
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index ea6cc8b560..cbaeccced5 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -907,8 +907,6 @@ typedef enum
WAIT_EVENT_RELATION_MAP_READ,
WAIT_EVENT_RELATION_MAP_SYNC,
WAIT_EVENT_RELATION_MAP_WRITE,
- WAIT_EVENT_REORDER_BUFFER_READ,
- WAIT_EVENT_REORDER_BUFFER_WRITE,
WAIT_EVENT_REORDER_LOGICAL_MAPPING_READ,
WAIT_EVENT_REPLICATION_SLOT_READ,
WAIT_EVENT_REPLICATION_SLOT_RESTORE_SYNC,
diff --git a/src/include/storage/buffile.h b/src/include/storage/buffile.h
index 1fba404fe2..8b2845e33f 100644
--- a/src/include/storage/buffile.h
+++ b/src/include/storage/buffile.h
@@ -28,9 +28,13 @@
#include "storage/sharedfileset.h"
-/* BufFile is an opaque type whose details are not known outside buffile.c. */
+/*
+ * BufFile and TransientBufFile are opaque types whose details are not known
+ * outside buffile.c.
+ */
typedef struct BufFile BufFile;
+typedef struct TransientBufFile TransientBufFile;
/*
* prototypes for functions in buffile.c
@@ -51,4 +55,11 @@ extern void BufFileExportShared(BufFile *file);
extern BufFile *BufFileOpenShared(SharedFileSet *fileset, const char *name);
extern void BufFileDeleteShared(SharedFileSet *fileset, const char *name);
+extern TransientBufFile *BufFileOpenTransient(const char *path, int fileFlags);
+extern void BufFileCloseTransient(TransientBufFile *file);
+extern size_t BufFileWriteTransient(TransientBufFile *file, void *ptr,
+ size_t size);
+extern size_t BufFileReadTransient(TransientBufFile *file, void *ptr,
+ size_t size);
+
#endif /* BUFFILE_H */
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-03-15 15:34 Robert Haas <robertmhaas@gmail.com>
parent: Antonin Houska <ah@cybertec.at>
0 siblings, 2 replies; 37+ messages in thread
From: Robert Haas @ 2019-03-15 15:34 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
On Thu, Mar 14, 2019 at 9:26 AM Antonin Houska <ah@cybertec.at> wrote:
> The point of #ifdef USE_ENCRYPTION is that we rely on OpenSSL, so the code
> needs to compile even w/o OpenSSL (of course the encryption won't be enabled
> in that case). I'll try to reduce the use of this construct only to the code
> blocks that really reference the OpenSSL functions.
That sounds like a good place to start. The other part will be
avoiding referencing the OpenSSL functions from too many places. For
example, right now we have secure_read() -- which I know some people
hate, or at least hate the name of] -- which lets you send data to the
client without having to care whether that data is going to be
SSL-encrypted on its way over the network. One can argue about
whether that particular abstraction is properly-designed, but as Tom
recently felt the urge to remind me on another thread, the value of
abstraction in general is not in doubt. I think we need one here, so
that the SSL-specific bits can be isolated in a relatively small
number of files, and then clients can ask to write a file, or write a
block to a file, or whatever, and ignore the question of whether that
is going to get encrypted somehow behind the scenes.
> Since buffile.c needs to handle this kind of problem (and it already does in
> the last patch version), I think even other than temporary files could be
> handled by this module. The patch below adds functions BufFileOpenTransient(),
> BufFileWriteTransient(), etc. that can replace OpenTransientFile, write(),
> etc. respectively. Once we implement the encryption, these functions will also
> hide handling of the encryption blocks from user. In this (preliminary) patch
> As an example, I applied this approach to ReorderBufferSerializeChange() and
> the function seems a bit simpler now. (ReorderBufferRestoreChange() would need
> more work to adopt this approach.)
Yeah, I don't have time to look at this in detail right now, but this
kind of thing seems like a promising approach to me.
> > The interaction of this capability with certain tricks that PostgreSQL
> > plays needs some thought -- and documentation, at least developer
> > documentation, maybe user documentation. One area is recovery.
> > Writing WAL relies on the fact that if you are in the midst of
> > rewriting a block and the power fails, bytes that are the same in both
> > the old and new block images will be undisturbed; encryption will make
> > that false. How's that going to work?
>
> Yes, if only a single bit is different, decryption will turn the whole
> encryption block (16 bytes) into garbage. So we need to enforce
> full_page_writes=on if encryption is enabled. The last version of the patch
> does not do that.
I believe that we need full_page_writes=on and wal_log_hints=on, but
they don't fix the problem that I'm describing. Suppose that the very
last page in the write-ahead log contains 3.5kB of good data and 7kB
of zeroes. Now, somebody comes along and writes a new write-ahead log
record that is 1k in size. In memory, we update the block: it now
contains 4.5kB of good data and 6.5kB of zeroes. Now, as we're
writing the block to do disk, the power is removed. Consequently, the
first 4kB hits the platter and the other 4kB disappears into the void.
If the WAL is not encrypted, the state at this point is that the 3.5kB
of good data that we had previously is still there, and the first 500
bytes of the new WAL record are also there, and the rest is missing.
Recovery will detect that there's a valid-looking record at end of
WAL, but when it tries to checksum that record, it will fail, and so
that trailing half-record will just get ignored. And that's fine.
If the WAL *is* encrypted, the state at this point is that the block
is unreadable, because the first 4kB of the block is the first half of
the bits that resulted from encrypting 8kB of data that includes the
new record, and the second 4kB of the block is the second half of the
bits that resulted from encrypting 8kB of data that did not include
the new record, and since the new record perturbed every bit on the
page with probability ~50%, what that means is you now have garbage.
That means that not only did we lose the new record, but we also lost
the 3.5kB of good data which the page previously contained. That's
NOT ok. Some of the changes associated with those WAL records may
have been flushed to disk already, or there may be commits in there
that were acknowledged to the client, and we can't just lose them.
One idea I have for fixing this problem is to encrypt the individual
WAL records rather than the blocks. That leaks some information, of
course, but maybe not in a way that really matters.
> > Hint bits also rely on this principle. I thought there might be some
> > interaction between this work and wal_log_hints for this reason, but I see
> > nothing of that sort in the patch.
>
> I'm not sure if the hint bit is still a problem if we first copy the shared
> buffer to backend-local memory and encrypt it there. That's what the patch
> does.
That has the same problem that I described in the previous paragraph,
except for the relation data files rather than the WAL itself. See
the manual's description of wal_log_hints:
<para>
When this parameter is <literal>on</literal>, the
<productname>PostgreSQL</productname>
server writes the entire content of each disk page to WAL during the
first modification of that page after a checkpoint, even for
non-critical modifications of so-called hint bits.
</para>
For encryption to work, you need that. A hint bit write might convert
the page to garbage, just as in the previous example, and this option
make sure that if that happens, there will be an FPW, allowing you to
recover... unless of course you also hit the problem I described
above, but assuming that's fixed somehow you still have this issue.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-03-15 21:10 Tom Lane <tgl@sss.pgh.pa.us>
parent: Robert Haas <robertmhaas@gmail.com>
1 sibling, 2 replies; 37+ messages in thread
From: Tom Lane @ 2019-03-15 21:10 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
Robert Haas <robertmhaas@gmail.com> writes:
> If the WAL *is* encrypted, the state at this point is that the block
> is unreadable, because the first 4kB of the block is the first half of
> the bits that resulted from encrypting 8kB of data that includes the
> new record, and the second 4kB of the block is the second half of the
> bits that resulted from encrypting 8kB of data that did not include
> the new record, and since the new record perturbed every bit on the
> page with probability ~50%, what that means is you now have garbage.
> That means that not only did we lose the new record, but we also lost
> the 3.5kB of good data which the page previously contained. That's
> NOT ok. Some of the changes associated with those WAL records may
> have been flushed to disk already, or there may be commits in there
> that were acknowledged to the client, and we can't just lose them.
ISTM that this is only a problem if you choose the wrong encryption
method. One not-wrong encryption method is to use a stream cipher
--- maybe that's not the exact right technical term, but anyway
I'm talking about a method which notionally XOR's the cleartext
data with a random bit stream generated from the encryption key
(probably along with other knowable inputs such as the block number).
In such a method, corruption of individual on-disk bytes doesn't
prevent you from getting the correct decryption of on-disk bytes
that aren't corrupted.
regards, tom lane
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-03-15 21:48 Robert Haas <robertmhaas@gmail.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 0 replies; 37+ messages in thread
From: Robert Haas @ 2019-03-15 21:48 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Antonin Houska <ah@cybertec.at>; Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
On Fri, Mar 15, 2019 at 5:10 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> ISTM that this is only a problem if you choose the wrong encryption
> method. One not-wrong encryption method is to use a stream cipher
> --- maybe that's not the exact right technical term, but anyway
> I'm talking about a method which notionally XOR's the cleartext
> data with a random bit stream generated from the encryption key
> (probably along with other knowable inputs such as the block number).
> In such a method, corruption of individual on-disk bytes doesn't
> prevent you from getting the correct decryption of on-disk bytes
> that aren't corrupted.
Oh, that seems like it might be a really good idea.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-03-21 11:46 Antonin Houska <ah@cybertec.at>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 1 reply; 37+ messages in thread
From: Antonin Houska @ 2019-03-21 11:46 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Robert Haas <robertmhaas@gmail.com>; Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
> > If the WAL *is* encrypted, the state at this point is that the block
> > is unreadable, because the first 4kB of the block is the first half of
> > the bits that resulted from encrypting 8kB of data that includes the
> > new record, and the second 4kB of the block is the second half of the
> > bits that resulted from encrypting 8kB of data that did not include
> > the new record, and since the new record perturbed every bit on the
> > page with probability ~50%, what that means is you now have garbage.
> > That means that not only did we lose the new record, but we also lost
> > the 3.5kB of good data which the page previously contained. That's
> > NOT ok. Some of the changes associated with those WAL records may
> > have been flushed to disk already, or there may be commits in there
> > that were acknowledged to the client, and we can't just lose them.
>
> ISTM that this is only a problem if you choose the wrong encryption
> method. One not-wrong encryption method is to use a stream cipher
> --- maybe that's not the exact right technical term, but anyway
> I'm talking about a method which notionally XOR's the cleartext
> data with a random bit stream generated from the encryption key
> (probably along with other knowable inputs such as the block number).
> In such a method, corruption of individual on-disk bytes doesn't
> prevent you from getting the correct decryption of on-disk bytes
> that aren't corrupted.
We actually use a block cipher (with block size 16 bytes), as opposed to
stream cipher. It's true that partial write is a problem because if a single
bit of the cipher text changed, decryption will produce 16 bytes of
garbage. However I'm not sure if partial write can affect as small unit as 16
bytes.
Nevertheless, with the current version of our patch, PG should be resistant
against such a partial write anyway because we chose to align XLOG records to
16 bytes (as long as the encryption is enabled) for the following reasons:
If one XLOG record ends and the following one starts in the same encryption
block, both records can get corrupted during streaming replication. The
scenario looks like: 1) the first record is written on master (the unused part
of the block contains zeroes), 2) the block is encrypted and its initial part
(i.e. the number of bytes occupied by the first record in the plain text) is
streamed to slave, 3) the second record is written on master, 4) the
containing encryption block is encrypted again and the trailing part (i.e. the
number of bytes occupied by the second record) is streamed, 5) decryption of
the block on slave will produce garbage and thus corrupt both records. This is
because the trailing part of the block was filled with zeroes during
encryption, but it contains different data at decryption time.
Alternative approach to this replication problem is that walsender decrypts
the stream and walreceiver encrypts it again. While this can provide us with
the advantage to have master and slave encrypted with different keys, this
approach brings some additional complexity. For example, pg_basebackup would
need to deal with encryption.
This design decision can be changed, but there's one more thing to consider:
if the XLOG stream is decrypted, the decryption cannot be disabled unless the
XLOG records are aligned to 16 bytes (and in turn, the XLOG alignment cannot
be enabled w/o initdb).
--
Antonin Houska
https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-03-21 20:07 Robert Haas <robertmhaas@gmail.com>
parent: Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 37+ messages in thread
From: Robert Haas @ 2019-03-21 20:07 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
On Thu, Mar 21, 2019 at 7:46 AM Antonin Houska <ah@cybertec.at> wrote:
> Nevertheless, with the current version of our patch, PG should be resistant
> against such a partial write anyway because we chose to align XLOG records to
> 16 bytes (as long as the encryption is enabled) for the following reasons:
>
> If one XLOG record ends and the following one starts in the same encryption
> block, both records can get corrupted during streaming replication. The
> scenario looks like: 1) the first record is written on master (the unused part
> of the block contains zeroes), 2) the block is encrypted and its initial part
> (i.e. the number of bytes occupied by the first record in the plain text) is
> streamed to slave, 3) the second record is written on master, 4) the
> containing encryption block is encrypted again and the trailing part (i.e. the
> number of bytes occupied by the second record) is streamed, 5) decryption of
> the block on slave will produce garbage and thus corrupt both records. This is
> because the trailing part of the block was filled with zeroes during
> encryption, but it contains different data at decryption time.
Wouldn't Tom's proposal to use a stream cipher fix all this?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-04-04 13:57 Antonin Houska <ah@cybertec.at>
parent: Robert Haas <robertmhaas@gmail.com>
1 sibling, 1 reply; 37+ messages in thread
From: Antonin Houska @ 2019-04-04 13:57 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
Robert Haas <robertmhaas@gmail.com> wrote:
> On Thu, Mar 14, 2019 at 9:26 AM Antonin Houska <ah@cybertec.at> wrote:
> > > Hint bits also rely on this principle. I thought there might be some
> > > interaction between this work and wal_log_hints for this reason, but I see
> > > nothing of that sort in the patch.
> >
> > I'm not sure if the hint bit is still a problem if we first copy the shared
> > buffer to backend-local memory and encrypt it there. That's what the patch
> > does.
>
> That has the same problem that I described in the previous paragraph,
> except for the relation data files rather than the WAL itself. See
> the manual's description of wal_log_hints:
>
> <para>
> When this parameter is <literal>on</literal>, the
> <productname>PostgreSQL</productname>
> server writes the entire content of each disk page to WAL during the
> first modification of that page after a checkpoint, even for
> non-critical modifications of so-called hint bits.
> </para>
>
> For encryption to work, you need that. A hint bit write might convert
> the page to garbage, just as in the previous example, and this option
> make sure that if that happens, there will be an FPW, allowing you to
> recover...
I think I finally understand. Originally I thought the question is how to
compute correct page checksum while the hint bits can be changed w/o exclusive
lock on the buffer. Now I realize that it's more about *recovery*: if the hint
bit change is followed by a torn page write, the hint bit can get changed on
disk but the checksum might not get updated. The wrong checksum is detected
during recovery, but if XLOG does not contain the corresponding full page
image, we're not able to recover.
And with encryption, the consequence is even worse because torn page write
causes not only wrong checksum of otherwise useful page, but really damaged
page.
I'll enforce the FPW in the next version of the patch.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-04-04 14:55 Robert Haas <robertmhaas@gmail.com>
parent: Antonin Houska <ah@cybertec.at>
0 siblings, 2 replies; 37+ messages in thread
From: Robert Haas @ 2019-04-04 14:55 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
On Thu, Apr 4, 2019 at 9:57 AM Antonin Houska <ah@cybertec.at> wrote:
> I think I finally understand. Originally I thought the question is how to
> compute correct page checksum while the hint bits can be changed w/o exclusive
> lock on the buffer. Now I realize that it's more about *recovery*: if the hint
> bit change is followed by a torn page write, the hint bit can get changed on
> disk but the checksum might not get updated. The wrong checksum is detected
> during recovery, but if XLOG does not contain the corresponding full page
> image, we're not able to recover.
>
> And with encryption, the consequence is even worse because torn page write
> causes not only wrong checksum of otherwise useful page, but really damaged
> page.
Correct.
> I'll enforce the FPW in the next version of the patch.
Cool.
I'm willing to put some effort into trying to get this into v13 if
you're willing to keep hacking on it, but there's probably a fair
amount to do and a year can go by in a hurry.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-04-04 15:52 Antonin Houska <ah@cybertec.at>
parent: Robert Haas <robertmhaas@gmail.com>
1 sibling, 2 replies; 37+ messages in thread
From: Antonin Houska @ 2019-04-04 15:52 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
Robert Haas <robertmhaas@gmail.com> wrote:
> I'm willing to put some effort into trying to get this into v13 if
> you're willing to keep hacking on it, but there's probably a fair
> amount to do and a year can go by in a hurry.
That'd be appreciated, especially by my boss. It doesn't seem likely that in
this situation I'll be assigned many other tasks of higher priority. So yes, I
expect to have quite some time for this patch. Thanks.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-04-04 18:09 PostgreSQL - Hans-Jürgen Schönig <postgres@cybertec.at>
parent: Antonin Houska <ah@cybertec.at>
1 sibling, 0 replies; 37+ messages in thread
From: PostgreSQL - Hans-Jürgen Schönig @ 2019-04-04 18:09 UTC (permalink / raw)
To: pgsql-hackers@lists.postgresql.org
On 4/4/19 5:52 PM, Antonin Houska wrote:
> Robert Haas <robertmhaas@gmail.com> wrote:
>
>> I'm willing to put some effort into trying to get this into v13 if
>> you're willing to keep hacking on it, but there's probably a fair
>> amount to do and a year can go by in a hurry.
> That'd be appreciated, especially by my boss. It doesn't seem likely that in
> this situation I'll be assigned many other tasks of higher priority. So yes, I
> expect to have quite some time for this patch. Thanks.
>
if there is any chance of having this committed into core, we will of
course provide the capacity to hack on this.
the same of course applies to the optimizer patches tony has been
working on.
many thanks
hans
--
Hans-Jürgen Schönig
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-04-04 19:47 Robert Haas <robertmhaas@gmail.com>
parent: Antonin Houska <ah@cybertec.at>
1 sibling, 0 replies; 37+ messages in thread
From: Robert Haas @ 2019-04-04 19:47 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
On Thu, Apr 4, 2019 at 11:52 AM Antonin Houska <ah@cybertec.at> wrote:
> Robert Haas <robertmhaas@gmail.com> wrote:
> > I'm willing to put some effort into trying to get this into v13 if
> > you're willing to keep hacking on it, but there's probably a fair
> > amount to do and a year can go by in a hurry.
>
> That'd be appreciated, especially by my boss. It doesn't seem likely that in
> this situation I'll be assigned many other tasks of higher priority. So yes, I
> expect to have quite some time for this patch. Thanks.
Great. I think it will be appreciated by my boss, too.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-04-05 15:22 Antonin Houska <ah@cybertec.at>
parent: Robert Haas <robertmhaas@gmail.com>
0 siblings, 1 reply; 37+ messages in thread
From: Antonin Houska @ 2019-04-05 15:22 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
Robert Haas <robertmhaas@gmail.com> wrote:
> On Thu, Mar 21, 2019 at 7:46 AM Antonin Houska <ah@cybertec.at> wrote:
> > Nevertheless, with the current version of our patch, PG should be resistant
> > against such a partial write anyway because we chose to align XLOG records to
> > 16 bytes (as long as the encryption is enabled) for the following reasons:
> >
> > If one XLOG record ends and the following one starts in the same encryption
> > block, both records can get corrupted during streaming replication. The
> > scenario looks like: 1) the first record is written on master (the unused part
> > of the block contains zeroes), 2) the block is encrypted and its initial part
> > (i.e. the number of bytes occupied by the first record in the plain text) is
> > streamed to slave, 3) the second record is written on master, 4) the
> > containing encryption block is encrypted again and the trailing part (i.e. the
> > number of bytes occupied by the second record) is streamed, 5) decryption of
> > the block on slave will produce garbage and thus corrupt both records. This is
> > because the trailing part of the block was filled with zeroes during
> > encryption, but it contains different data at decryption time.
>
> Wouldn't Tom's proposal to use a stream cipher fix all this?
Yes it would make the extra alignment unnecessary, but our solution tries to
meet specific requirements of disk encryption. Stream cipher appears to be
incompatible with these requirements:
https://en.wikipedia.org/wiki/Disk_encryption_theory
Currently we try to use the CBC-ESSIV mode. It's worth to mention that in this
mode, if the page is encrypted twice and if the plain data in the encryption
block N (i.e. block of 16 bytes) changes before the 2nd encryption, the
encrypted data will only change in blocks >= N. So the problem of losing
already flushed WAL records should not happen.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-04-05 19:57 Robert Haas <robertmhaas@gmail.com>
parent: Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 37+ messages in thread
From: Robert Haas @ 2019-04-05 19:57 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
On Fri, Apr 5, 2019 at 11:22 AM Antonin Houska <ah@cybertec.at> wrote:
> > Wouldn't Tom's proposal to use a stream cipher fix all this?
>
> Yes it would make the extra alignment unnecessary, but our solution tries to
> meet specific requirements of disk encryption. Stream cipher appears to be
> incompatible with these requirements:
>
> https://en.wikipedia.org/wiki/Disk_encryption_theory
Hmm. Well, I don't know what to do about that, but I think this patch
is going to be facing an uphill battle if the encrypted and
unencrypted WAL formats use different alignment.
> Currently we try to use the CBC-ESSIV mode. It's worth to mention that in this
> mode, if the page is encrypted twice and if the plain data in the encryption
> block N (i.e. block of 16 bytes) changes before the 2nd encryption, the
> encrypted data will only change in blocks >= N. So the problem of losing
> already flushed WAL records should not happen.
Well, this is just a question of alignment. If WAL records are at
least 16-byte aligned, then it should be fine. But I have a feeling
they may just be MAXALIGN'd.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-04-06 13:29 Antonin Houska <ah@cybertec.at>
parent: Robert Haas <robertmhaas@gmail.com>
0 siblings, 1 reply; 37+ messages in thread
From: Antonin Houska @ 2019-04-06 13:29 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
Robert Haas <robertmhaas@gmail.com> wrote:
> On Fri, Apr 5, 2019 at 11:22 AM Antonin Houska <ah@cybertec.at> wrote:
> > > Wouldn't Tom's proposal to use a stream cipher fix all this?
> >
> > Yes it would make the extra alignment unnecessary, but our solution tries to
> > meet specific requirements of disk encryption. Stream cipher appears to be
> > incompatible with these requirements:
> >
> > https://en.wikipedia.org/wiki/Disk_encryption_theory
>
> Hmm. Well, I don't know what to do about that, but I think this patch
> is going to be facing an uphill battle if the encrypted and
> unencrypted WAL formats use different alignment.
Once you said this could be a problem, I thought about it a bit more. The link
above tells that stream cipher is not good for disk encryption because it's
risky to encrypt different data (e.g. old an new version of disk sector) with
the same key (or rather with the same combination of key and initialization
vector).
However WAL is a special case because here we should never need to change the
data at given position. The only exception is that unused space (sequence of
zeroes) becomes a valid record. So if we avoid encryption of the unused space,
it might be o.k. to use the stream cipher for WAL.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-04-12 20:17 Bruce Momjian <bruce@momjian.us>
parent: Antonin Houska <ah@cybertec.at>
0 siblings, 0 replies; 37+ messages in thread
From: Bruce Momjian @ 2019-04-12 20:17 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Robert Haas <robertmhaas@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
On Sat, Apr 6, 2019 at 03:29:13PM +0200, Antonin Houska wrote:
> Robert Haas <robertmhaas@gmail.com> wrote:
>
> > On Fri, Apr 5, 2019 at 11:22 AM Antonin Houska <ah@cybertec.at> wrote:
> > > > Wouldn't Tom's proposal to use a stream cipher fix all this?
> > >
> > > Yes it would make the extra alignment unnecessary, but our solution tries to
> > > meet specific requirements of disk encryption. Stream cipher appears to be
> > > incompatible with these requirements:
> > >
> > > https://en.wikipedia.org/wiki/Disk_encryption_theory
> >
> > Hmm. Well, I don't know what to do about that, but I think this patch
> > is going to be facing an uphill battle if the encrypted and
> > unencrypted WAL formats use different alignment.
>
> Once you said this could be a problem, I thought about it a bit more. The link
> above tells that stream cipher is not good for disk encryption because it's
> risky to encrypt different data (e.g. old an new version of disk sector) with
> the same key (or rather with the same combination of key and initialization
> vector).
>
> However WAL is a special case because here we should never need to change the
> data at given position. The only exception is that unused space (sequence of
> zeroes) becomes a valid record. So if we avoid encryption of the unused space,
> it might be o.k. to use the stream cipher for WAL.
Agreed. Requirement #2 is not needed for WAL:
https://en.wikipedia.org/wiki/Disk_encryption_theory#Problem_definition
2. Data retrieval and storage should both be fast operations, no
matter where on the disk the data is stored.
because you are not writing into random parts of the WAL.
Using the same stream cipher system that SSH uses would make sense,
since it is byte-oriented, and it only encrypts the tail.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-04-24 14:48 Antonin Houska <ah@cybertec.at>
parent: Robert Haas <robertmhaas@gmail.com>
1 sibling, 1 reply; 37+ messages in thread
From: Antonin Houska @ 2019-04-24 14:48 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
Robert Haas <robertmhaas@gmail.com> wrote:
> On Thu, Apr 4, 2019 at 9:57 AM Antonin Houska <ah@cybertec.at> wrote:
> > I'll enforce the FPW in the next version of the patch.
>
> Cool.
>
> I'm willing to put some effort into trying to get this into v13 if
> you're willing to keep hacking on it, but there's probably a fair
> amount to do and a year can go by in a hurry.
Attached is the next version. It tries to address various problems pointed out
upthread, including documentation.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
Attachments: download all (3 files, .tar.gz)
v03-001-BufFileTransient.patch (text/x-diff, 48.0K) download
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index cdf87bae32..51ff2cfa65 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3926,12 +3926,6 @@ pgstat_get_wait_io(WaitEventIO w)
case WAIT_EVENT_RELATION_MAP_WRITE:
event_name = "RelationMapWrite";
break;
- case WAIT_EVENT_REORDER_BUFFER_READ:
- event_name = "ReorderBufferRead";
- break;
- case WAIT_EVENT_REORDER_BUFFER_WRITE:
- event_name = "ReorderBufferWrite";
- break;
case WAIT_EVENT_REORDER_LOGICAL_MAPPING_READ:
event_name = "ReorderLogicalMappingRead";
break;
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 2cfdf1c9ac..6fe8ea57e1 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -71,6 +71,7 @@
#include "replication/slot.h"
#include "replication/snapbuild.h" /* just for SnapBuildSnapDecRefcount */
#include "storage/bufmgr.h"
+#include "storage/buffile.h"
#include "storage/fd.h"
#include "storage/sinval.h"
#include "utils/builtins.h"
@@ -109,7 +110,7 @@ typedef struct ReorderBufferIterTXNEntry
XLogRecPtr lsn;
ReorderBufferChange *change;
ReorderBufferTXN *txn;
- int fd;
+ TransientBufFile *file;
XLogSegNo segno;
} ReorderBufferIterTXNEntry;
@@ -192,11 +193,17 @@ static void ReorderBufferExecuteInvalidations(ReorderBuffer *rb, ReorderBufferTX
static void ReorderBufferCheckSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn);
static void ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn);
static void ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
- int fd, ReorderBufferChange *change);
+ TransientBufFile *file, ReorderBufferChange *change);
+static void ReorderBufferWriteData(TransientBufFile *file, void *ptr, size_t size,
+ ReorderBufferTXN *txn);
static Size ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn,
- int *fd, XLogSegNo *segno);
+ TransientBufFile **file, XLogSegNo *segno);
static void ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
- char *change);
+ TransientBufFile **file);
+static ReorderBufferTupleBuf *ReorderBufferRestoreTuple(ReorderBuffer *rb,
+ TransientBufFile *file);
+static void ReorderBufferReadData(TransientBufFile *file, void *ptr, size_t size,
+ bool *no_data_p);
static void ReorderBufferRestoreCleanup(ReorderBuffer *rb, ReorderBufferTXN *txn);
static void ReorderBufferCleanupSerializedTXNs(const char *slotname);
static void ReorderBufferSerializedPath(char *path, ReplicationSlot *slot,
@@ -267,9 +274,6 @@ ReorderBufferAllocate(void)
buffer->by_txn_last_xid = InvalidTransactionId;
buffer->by_txn_last_txn = NULL;
- buffer->outbuf = NULL;
- buffer->outbufsize = 0;
-
buffer->current_restart_decoding_lsn = InvalidXLogRecPtr;
dlist_init(&buffer->toplevel_by_lsn);
@@ -988,7 +992,7 @@ ReorderBufferIterTXNInit(ReorderBuffer *rb, ReorderBufferTXN *txn)
for (off = 0; off < state->nr_txns; off++)
{
- state->entries[off].fd = -1;
+ state->entries[off].file = NULL;
state->entries[off].segno = 0;
}
@@ -1013,7 +1017,7 @@ ReorderBufferIterTXNInit(ReorderBuffer *rb, ReorderBufferTXN *txn)
{
/* serialize remaining changes */
ReorderBufferSerializeTXN(rb, txn);
- ReorderBufferRestoreChanges(rb, txn, &state->entries[off].fd,
+ ReorderBufferRestoreChanges(rb, txn, &state->entries[off].file,
&state->entries[off].segno);
}
@@ -1043,7 +1047,7 @@ ReorderBufferIterTXNInit(ReorderBuffer *rb, ReorderBufferTXN *txn)
/* serialize remaining changes */
ReorderBufferSerializeTXN(rb, cur_txn);
ReorderBufferRestoreChanges(rb, cur_txn,
- &state->entries[off].fd,
+ &state->entries[off].file,
&state->entries[off].segno);
}
cur_change = dlist_head_element(ReorderBufferChange, node,
@@ -1124,7 +1128,7 @@ ReorderBufferIterTXNNext(ReorderBuffer *rb, ReorderBufferIterTXNState *state)
dlist_delete(&change->node);
dlist_push_tail(&state->old_change, &change->node);
- if (ReorderBufferRestoreChanges(rb, entry->txn, &entry->fd,
+ if (ReorderBufferRestoreChanges(rb, entry->txn, &entry->file,
&state->entries[off].segno))
{
/* successfully restored changes from disk */
@@ -1163,8 +1167,8 @@ ReorderBufferIterTXNFinish(ReorderBuffer *rb,
for (off = 0; off < state->nr_txns; off++)
{
- if (state->entries[off].fd != -1)
- CloseTransientFile(state->entries[off].fd);
+ if (state->entries[off].file)
+ BufFileCloseTransient(state->entries[off].file);
}
/* free memory we might have "leaked" in the last *Next call */
@@ -2212,24 +2216,6 @@ ReorderBufferXidHasBaseSnapshot(ReorderBuffer *rb, TransactionId xid)
*/
/*
- * Ensure the IO buffer is >= sz.
- */
-static void
-ReorderBufferSerializeReserve(ReorderBuffer *rb, Size sz)
-{
- if (!rb->outbufsize)
- {
- rb->outbuf = MemoryContextAlloc(rb->context, sz);
- rb->outbufsize = sz;
- }
- else if (rb->outbufsize < sz)
- {
- rb->outbuf = repalloc(rb->outbuf, sz);
- rb->outbufsize = sz;
- }
-}
-
-/*
* Check whether the transaction tx should spill its data to disk.
*/
static void
@@ -2254,7 +2240,7 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
{
dlist_iter subtxn_i;
dlist_mutable_iter change_i;
- int fd = -1;
+ TransientBufFile *file = NULL;
XLogSegNo curOpenSegNo = 0;
Size spilled = 0;
@@ -2281,13 +2267,13 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
* store in segment in which it belongs by start lsn, don't split over
* multiple segments tho
*/
- if (fd == -1 ||
+ if (file == NULL ||
!XLByteInSeg(change->lsn, curOpenSegNo, wal_segment_size))
{
char path[MAXPGPATH];
- if (fd != -1)
- CloseTransientFile(fd);
+ if (file)
+ BufFileCloseTransient(file);
XLByteToSeg(change->lsn, curOpenSegNo, wal_segment_size);
@@ -2299,16 +2285,11 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
curOpenSegNo);
/* open segment, create it if necessary */
- fd = OpenTransientFile(path,
- O_CREAT | O_WRONLY | O_APPEND | PG_BINARY);
-
- if (fd < 0)
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not open file \"%s\": %m", path)));
+ file = BufFileOpenTransient(path,
+ O_CREAT | O_WRONLY | O_APPEND | PG_BINARY);
}
- ReorderBufferSerializeChange(rb, txn, fd, change);
+ ReorderBufferSerializeChange(rb, txn, file, change);
dlist_delete(&change->node);
ReorderBufferReturnChange(rb, change);
@@ -2320,8 +2301,8 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
txn->nentries_mem = 0;
txn->serialized = true;
- if (fd != -1)
- CloseTransientFile(fd);
+ if (file)
+ BufFileCloseTransient(file);
}
/*
@@ -2329,15 +2310,13 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
*/
static void
ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
- int fd, ReorderBufferChange *change)
+ TransientBufFile *file, ReorderBufferChange *change)
{
- ReorderBufferDiskChange *ondisk;
+ ReorderBufferDiskChange hdr;
Size sz = sizeof(ReorderBufferDiskChange);
- ReorderBufferSerializeReserve(rb, sz);
-
- ondisk = (ReorderBufferDiskChange *) rb->outbuf;
- memcpy(&ondisk->change, change, sizeof(ReorderBufferChange));
+ memcpy((char *) &hdr + offsetof(ReorderBufferDiskChange, change),
+ change, sizeof(ReorderBufferChange));
switch (change->action)
{
@@ -2347,7 +2326,6 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
case REORDER_BUFFER_CHANGE_DELETE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT:
{
- char *data;
ReorderBufferTupleBuf *oldtup,
*newtup;
Size oldlen = 0;
@@ -2370,66 +2348,55 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
sz += newlen;
}
- /* make sure we have enough space */
- ReorderBufferSerializeReserve(rb, sz);
-
- data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
- /* might have been reallocated above */
- ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+ hdr.size = sz;
+ ReorderBufferWriteData(file, &hdr, sizeof(ReorderBufferDiskChange),
+ txn);
if (oldlen)
{
- memcpy(data, &oldtup->tuple, sizeof(HeapTupleData));
- data += sizeof(HeapTupleData);
-
- memcpy(data, oldtup->tuple.t_data, oldlen);
- data += oldlen;
+ ReorderBufferWriteData(file, &oldtup->tuple,
+ sizeof(HeapTupleData), txn);
+ ReorderBufferWriteData(file, oldtup->tuple.t_data, oldlen,
+ txn);
}
if (newlen)
{
- memcpy(data, &newtup->tuple, sizeof(HeapTupleData));
- data += sizeof(HeapTupleData);
-
- memcpy(data, newtup->tuple.t_data, newlen);
- data += newlen;
+ ReorderBufferWriteData(file, &newtup->tuple,
+ sizeof(HeapTupleData), txn);
+ ReorderBufferWriteData(file, newtup->tuple.t_data, newlen,
+ txn);
}
break;
}
case REORDER_BUFFER_CHANGE_MESSAGE:
{
- char *data;
Size prefix_size = strlen(change->data.msg.prefix) + 1;
sz += prefix_size + change->data.msg.message_size +
sizeof(Size) + sizeof(Size);
- ReorderBufferSerializeReserve(rb, sz);
- data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
-
- /* might have been reallocated above */
- ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+ hdr.size = sz;
+ ReorderBufferWriteData(file, &hdr,
+ sizeof(ReorderBufferDiskChange),
+ txn);
/* write the prefix including the size */
- memcpy(data, &prefix_size, sizeof(Size));
- data += sizeof(Size);
- memcpy(data, change->data.msg.prefix,
- prefix_size);
- data += prefix_size;
+ ReorderBufferWriteData(file, &prefix_size, sizeof(Size), txn);
+ ReorderBufferWriteData(file, change->data.msg.prefix,
+ prefix_size, txn);
/* write the message including the size */
- memcpy(data, &change->data.msg.message_size, sizeof(Size));
- data += sizeof(Size);
- memcpy(data, change->data.msg.message,
- change->data.msg.message_size);
- data += change->data.msg.message_size;
+ ReorderBufferWriteData(file, &change->data.msg.message_size,
+ sizeof(Size), txn);
+ ReorderBufferWriteData(file, change->data.msg.message,
+ change->data.msg.message_size, txn);
break;
}
case REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT:
{
Snapshot snap;
- char *data;
snap = change->data.snapshot;
@@ -2438,49 +2405,37 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
sizeof(TransactionId) * snap->subxcnt
;
- /* make sure we have enough space */
- ReorderBufferSerializeReserve(rb, sz);
- data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
- /* might have been reallocated above */
- ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+ hdr.size = sz;
+ ReorderBufferWriteData(file, &hdr,
+ sizeof(ReorderBufferDiskChange), txn);
- memcpy(data, snap, sizeof(SnapshotData));
- data += sizeof(SnapshotData);
+ ReorderBufferWriteData(file, snap, sizeof(SnapshotData), txn);
if (snap->xcnt)
- {
- memcpy(data, snap->xip,
- sizeof(TransactionId) * snap->xcnt);
- data += sizeof(TransactionId) * snap->xcnt;
- }
+ ReorderBufferWriteData(file, snap->xip,
+ sizeof(TransactionId) * snap->xcnt,
+ txn);
if (snap->subxcnt)
- {
- memcpy(data, snap->subxip,
- sizeof(TransactionId) * snap->subxcnt);
- data += sizeof(TransactionId) * snap->subxcnt;
- }
+ ReorderBufferWriteData(file, snap->subxip,
+ sizeof(TransactionId) * snap->subxcnt,
+ txn);
break;
}
case REORDER_BUFFER_CHANGE_TRUNCATE:
{
- Size size;
- char *data;
+ Size size;
/* account for the OIDs of truncated relations */
size = sizeof(Oid) * change->data.truncate.nrelids;
sz += size;
- /* make sure we have enough space */
- ReorderBufferSerializeReserve(rb, sz);
-
- data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
- /* might have been reallocated above */
- ondisk = (ReorderBufferDiskChange *) rb->outbuf;
-
- memcpy(data, change->data.truncate.relids, size);
- data += size;
+ hdr.size = sz;
+ ReorderBufferWriteData(file, &hdr, sizeof(ReorderBufferDiskChange),
+ txn);
+ ReorderBufferWriteData(file, change->data.truncate.relids, size,
+ txn);
break;
}
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
@@ -2489,27 +2444,21 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
/* ReorderBufferChange contains everything important */
break;
}
+}
- ondisk->size = sz;
-
- errno = 0;
- pgstat_report_wait_start(WAIT_EVENT_REORDER_BUFFER_WRITE);
- if (write(fd, rb->outbuf, ondisk->size) != ondisk->size)
- {
- int save_errno = errno;
-
- CloseTransientFile(fd);
-
- /* if write didn't set errno, assume problem is no disk space */
- errno = save_errno ? save_errno : ENOSPC;
+/*
+ * Wrapper for BufFileWriteTransient() that raises ERROR if the whole chunk
+ * was not written. XXX Should this be a macro?
+ */
+static void
+ReorderBufferWriteData(TransientBufFile *file, void *ptr, size_t size,
+ ReorderBufferTXN *txn)
+{
+ if (BufFileWriteTransient(file, ptr, size) != size)
ereport(ERROR,
(errcode_for_file_access(),
errmsg("could not write to data file for XID %u: %m",
txn->xid)));
- }
- pgstat_report_wait_end();
-
- Assert(ondisk->change.action == change->action);
}
/*
@@ -2517,7 +2466,7 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
*/
static Size
ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn,
- int *fd, XLogSegNo *segno)
+ TransientBufFile **file, XLogSegNo *segno)
{
Size restored = 0;
XLogSegNo last_segno;
@@ -2542,10 +2491,7 @@ ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn,
while (restored < max_changes_in_memory && *segno <= last_segno)
{
- int readBytes;
- ReorderBufferDiskChange *ondisk;
-
- if (*fd == -1)
+ if (*file == NULL)
{
char path[MAXPGPATH];
@@ -2562,77 +2508,24 @@ ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferSerializedPath(path, MyReplicationSlot, txn->xid,
*segno);
- *fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
- if (*fd < 0 && errno == ENOENT)
+ *file = BufFileOpenTransient(path, O_RDONLY | PG_BINARY);
+ if (*file == NULL)
{
- *fd = -1;
+ Assert(errno == ENOENT);
(*segno)++;
continue;
}
- else if (*fd < 0)
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not open file \"%s\": %m",
- path)));
}
- /*
- * Read the statically sized part of a change which has information
- * about the total size. If we couldn't read a record, we're at the
- * end of this file.
- */
- ReorderBufferSerializeReserve(rb, sizeof(ReorderBufferDiskChange));
- pgstat_report_wait_start(WAIT_EVENT_REORDER_BUFFER_READ);
- readBytes = read(*fd, rb->outbuf, sizeof(ReorderBufferDiskChange));
- pgstat_report_wait_end();
-
- /* eof */
- if (readBytes == 0)
+ ReorderBufferRestoreChange(rb, txn, file);
+ if (*file)
+ restored++;
+ else
{
- CloseTransientFile(*fd);
- *fd = -1;
+ /* No data could be restored. */
(*segno)++;
continue;
}
- else if (readBytes < 0)
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not read from reorderbuffer spill file: %m")));
- else if (readBytes != sizeof(ReorderBufferDiskChange))
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not read from reorderbuffer spill file: read %d instead of %u bytes",
- readBytes,
- (uint32) sizeof(ReorderBufferDiskChange))));
-
- ondisk = (ReorderBufferDiskChange *) rb->outbuf;
-
- ReorderBufferSerializeReserve(rb,
- sizeof(ReorderBufferDiskChange) + ondisk->size);
- ondisk = (ReorderBufferDiskChange *) rb->outbuf;
-
- pgstat_report_wait_start(WAIT_EVENT_REORDER_BUFFER_READ);
- readBytes = read(*fd, rb->outbuf + sizeof(ReorderBufferDiskChange),
- ondisk->size - sizeof(ReorderBufferDiskChange));
- pgstat_report_wait_end();
-
- if (readBytes < 0)
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not read from reorderbuffer spill file: %m")));
- else if (readBytes != ondisk->size - sizeof(ReorderBufferDiskChange))
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not read from reorderbuffer spill file: read %d instead of %u bytes",
- readBytes,
- (uint32) (ondisk->size - sizeof(ReorderBufferDiskChange)))));
-
- /*
- * ok, read a full change from disk, now restore it into proper
- * in-memory format
- */
- ReorderBufferRestoreChange(rb, txn, rb->outbuf);
- restored++;
}
return restored;
@@ -2642,25 +2535,36 @@ ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn,
* Convert change from its on-disk format to in-memory format and queue it onto
* the TXN's ->changes list.
*
- * Note: although "data" is declared char*, at entry it points to a
- * maxalign'd buffer, making it safe in most of this function to assume
- * that the pointed-to data is suitably aligned for direct access.
+ * If no data was found in the file, close it and set *file to NULL.
*/
static void
ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
- char *data)
+ TransientBufFile **file)
{
- ReorderBufferDiskChange *ondisk;
+ ReorderBufferDiskChange ondisk;
+ bool no_data;
ReorderBufferChange *change;
- ondisk = (ReorderBufferDiskChange *) data;
+ /*
+ * Read the statically sized part of a change which has information about
+ * the total size. If we couldn't read a record, we're at the end of this
+ * file.
+ */
+ ReorderBufferReadData(*file, &ondisk, sizeof(ReorderBufferDiskChange),
+ &no_data);
+
+ /* eof */
+ if (no_data)
+ {
+ BufFileCloseTransient(*file);
+ *file = NULL;
+ return;
+ }
change = ReorderBufferGetChange(rb);
/* copy static part */
- memcpy(change, &ondisk->change, sizeof(ReorderBufferChange));
-
- data += sizeof(ReorderBufferDiskChange);
+ memcpy(change, &ondisk.change, sizeof(ReorderBufferChange));
/* restore individual stuff */
switch (change->action)
@@ -2671,50 +2575,10 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
case REORDER_BUFFER_CHANGE_DELETE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT:
if (change->data.tp.oldtuple)
- {
- uint32 tuplelen = ((HeapTuple) data)->t_len;
-
- change->data.tp.oldtuple =
- ReorderBufferGetTupleBuf(rb, tuplelen - SizeofHeapTupleHeader);
-
- /* restore ->tuple */
- memcpy(&change->data.tp.oldtuple->tuple, data,
- sizeof(HeapTupleData));
- data += sizeof(HeapTupleData);
-
- /* reset t_data pointer into the new tuplebuf */
- change->data.tp.oldtuple->tuple.t_data =
- ReorderBufferTupleBufData(change->data.tp.oldtuple);
-
- /* restore tuple data itself */
- memcpy(change->data.tp.oldtuple->tuple.t_data, data, tuplelen);
- data += tuplelen;
- }
+ change->data.tp.oldtuple = ReorderBufferRestoreTuple(rb, *file);
if (change->data.tp.newtuple)
- {
- /* here, data might not be suitably aligned! */
- uint32 tuplelen;
-
- memcpy(&tuplelen, data + offsetof(HeapTupleData, t_len),
- sizeof(uint32));
-
- change->data.tp.newtuple =
- ReorderBufferGetTupleBuf(rb, tuplelen - SizeofHeapTupleHeader);
-
- /* restore ->tuple */
- memcpy(&change->data.tp.newtuple->tuple, data,
- sizeof(HeapTupleData));
- data += sizeof(HeapTupleData);
-
- /* reset t_data pointer into the new tuplebuf */
- change->data.tp.newtuple->tuple.t_data =
- ReorderBufferTupleBufData(change->data.tp.newtuple);
-
- /* restore tuple data itself */
- memcpy(change->data.tp.newtuple->tuple.t_data, data, tuplelen);
- data += tuplelen;
- }
+ change->data.tp.newtuple = ReorderBufferRestoreTuple(rb, *file);
break;
case REORDER_BUFFER_CHANGE_MESSAGE:
@@ -2722,44 +2586,44 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
Size prefix_size;
/* read prefix */
- memcpy(&prefix_size, data, sizeof(Size));
- data += sizeof(Size);
+ ReorderBufferReadData(*file, &prefix_size, sizeof(Size), NULL);
change->data.msg.prefix = MemoryContextAlloc(rb->context,
prefix_size);
- memcpy(change->data.msg.prefix, data, prefix_size);
+ ReorderBufferReadData(*file, change->data.msg.prefix,
+ prefix_size, NULL);
Assert(change->data.msg.prefix[prefix_size - 1] == '\0');
- data += prefix_size;
/* read the message */
- memcpy(&change->data.msg.message_size, data, sizeof(Size));
- data += sizeof(Size);
+ ReorderBufferReadData(*file, &change->data.msg.message_size,
+ sizeof(Size), NULL);
change->data.msg.message = MemoryContextAlloc(rb->context,
change->data.msg.message_size);
- memcpy(change->data.msg.message, data,
- change->data.msg.message_size);
- data += change->data.msg.message_size;
+ ReorderBufferReadData(*file, change->data.msg.message,
+ change->data.msg.message_size, NULL);
break;
}
case REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT:
{
- Snapshot oldsnap;
+ SnapshotData oldsnap;
Snapshot newsnap;
Size size;
- oldsnap = (Snapshot) data;
+ ReorderBufferReadData(*file, &oldsnap, sizeof(SnapshotData), NULL);
size = sizeof(SnapshotData) +
- sizeof(TransactionId) * oldsnap->xcnt +
- sizeof(TransactionId) * (oldsnap->subxcnt + 0);
+ sizeof(TransactionId) * oldsnap.xcnt +
+ sizeof(TransactionId) * (oldsnap.subxcnt + 0);
change->data.snapshot = MemoryContextAllocZero(rb->context, size);
newsnap = change->data.snapshot;
- memcpy(newsnap, data, size);
+ memcpy(newsnap, &oldsnap, sizeof(SnapshotData));
newsnap->xip = (TransactionId *)
(((char *) newsnap) + sizeof(SnapshotData));
+ ReorderBufferReadData(*file, newsnap->xip,
+ size - sizeof(SnapshotData), NULL);
newsnap->subxip = newsnap->xip + newsnap->xcnt;
newsnap->copied = true;
break;
@@ -2771,7 +2635,9 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
relids = ReorderBufferGetRelids(rb,
change->data.truncate.nrelids);
- memcpy(relids, data, change->data.truncate.nrelids * sizeof(Oid));
+ ReorderBufferReadData(*file, relids,
+ change->data.truncate.nrelids * sizeof(Oid),
+ NULL);
change->data.truncate.relids = relids;
break;
@@ -2787,6 +2653,77 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
}
/*
+ * Convert heap tuple from its on-disk format to in-memory format.
+ */
+static ReorderBufferTupleBuf *
+ReorderBufferRestoreTuple(ReorderBuffer *rb, TransientBufFile *file)
+{
+ HeapTupleData tupdata;
+ uint32 tuplelen;
+ ReorderBufferTupleBuf *result;
+
+ ReorderBufferReadData(file, &tupdata, sizeof(HeapTupleData), NULL);
+ tuplelen = tupdata.t_len;
+
+ result = ReorderBufferGetTupleBuf(rb, tuplelen - SizeofHeapTupleHeader);
+
+ /* restore ->tuple */
+ memcpy(&result->tuple, &tupdata, sizeof(HeapTupleData));
+
+ /* reset t_data pointer into the new tuplebuf */
+ result->tuple.t_data = ReorderBufferTupleBufData(result);
+
+ /* restore tuple data itself */
+ ReorderBufferReadData(file, result->tuple.t_data, tuplelen, NULL);
+
+ return result;
+}
+
+/*
+ * Wrapper for BufFileReadTransient() that raises ERROR if the expected amount
+ * of bytes was not read.
+ *
+ * If valid pointer is passed for no_data_p, set *no_data_p to indicate
+ * whether zero bytes was read. If NULL is passed, do not tolerate missing
+ * data.
+ */
+static void
+ReorderBufferReadData(TransientBufFile *file, void *ptr, size_t size,
+ bool *no_data_p)
+{
+ int readBytes;
+
+ /*
+ * Caller should not request zero bytes. This assumption simplifies
+ * setting of *no_data_p below.
+ */
+ Assert(size > 0);
+
+ if ((readBytes = BufFileReadTransient(file, ptr, size)) != size)
+ {
+ if (no_data_p)
+ *no_data_p = readBytes == 0;
+
+ /*
+ * It is o.k. to receive exactly zero bytes if caller passed valid
+ * no_data_p.
+ */
+ if (no_data_p && *no_data_p)
+ return;
+
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from reorderbuffer spill file: read %d instead of %u bytes",
+ readBytes, (uint32) size)));
+ }
+ else if (no_data_p)
+ {
+ /* Given that size is non-zero, readBytes must be non-zero too. */
+ *no_data_p = false;
+ }
+}
+
+/*
* Remove all on-disk stored for the passed in transaction.
*/
static void
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index c2c445dbf4..e9fc11d162 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -41,6 +41,8 @@
#include "postgres.h"
+#include <unistd.h>
+
#include "executor/instrument.h"
#include "miscadmin.h"
#include "pgstat.h"
@@ -58,19 +60,44 @@
#define BUFFILE_SEG_SIZE (MAX_PHYSICAL_FILESIZE / BLCKSZ)
/*
+ * Fields that both BufFile and TransientBufFile structures need. It must be
+ * the first field of those structures.
+ */
+typedef struct BufFileCommon
+{
+ bool dirty; /* does buffer need to be written? */
+ int pos; /* next read/write position in buffer */
+ int nbytes; /* total # of valid bytes in buffer */
+
+ /*
+ * "current pos" is position of start of buffer within the logical file.
+ * Position as seen by user of BufFile is (curFile, curOffset + pos).
+ */
+ int curFile; /* file index (0..n) part of current pos,
+ * always zero for TransientBufFile */
+ off_t curOffset; /* offset part of current pos */
+
+ bool readOnly; /* has the file been set to read only? */
+
+ bool append; /* should new data be appended to the end? */
+
+ PGAlignedBlock buffer;
+} BufFileCommon;
+
+/*
* This data structure represents a buffered file that consists of one or
* more physical files (each accessed through a virtual file descriptor
* managed by fd.c).
*/
struct BufFile
{
+ BufFileCommon common; /* Common fields, see above. */
+
int numFiles; /* number of physical files in set */
/* all files except the last have length exactly MAX_PHYSICAL_FILESIZE */
File *files; /* palloc'd array with numFiles entries */
bool isInterXact; /* keep open over transactions? */
- bool dirty; /* does buffer need to be written? */
- bool readOnly; /* has the file been set to read only? */
SharedFileSet *fileset; /* space for segment files if shared */
const char *name; /* name of this BufFile if shared */
@@ -81,16 +108,20 @@ struct BufFile
* because after creation we only repalloc our arrays larger.)
*/
ResourceOwner resowner;
+};
- /*
- * "current pos" is position of start of buffer within the logical file.
- * Position as seen by user of BufFile is (curFile, curOffset + pos).
- */
- int curFile; /* file index (0..n) part of current pos */
- off_t curOffset; /* offset part of current pos */
- int pos; /* next read/write position in buffer */
- int nbytes; /* total # of valid bytes in buffer */
- PGAlignedBlock buffer;
+/*
+ * Buffered variant of a transient file. Unlike BufFile this is simpler in
+ * several ways: 1) it's not split into segments, 2) there's no need of seek,
+ * 3) there's no need to combine read and write access.
+ */
+struct TransientBufFile
+{
+ BufFileCommon common; /* Common fields, see above. */
+
+ /* The underlying file. */
+ char *path;
+ int fd;
};
static BufFile *makeBufFileCommon(int nfiles);
@@ -101,22 +132,32 @@ static void BufFileDumpBuffer(BufFile *file);
static int BufFileFlush(BufFile *file);
static File MakeNewSharedSegment(BufFile *file, int segment);
+static void BufFileLoadBufferTransient(TransientBufFile *file);
+static void BufFileDumpBufferTransient(TransientBufFile *file);
+
+static size_t BufFileReadCommon(BufFileCommon *file, void *ptr, size_t size,
+ bool is_transient);
+static size_t BufFileWriteCommon(BufFileCommon *file, void *ptr, size_t size,
+ bool is_transient);
+
/*
* Create BufFile and perform the common initialization.
*/
static BufFile *
makeBufFileCommon(int nfiles)
{
- BufFile *file = (BufFile *) palloc(sizeof(BufFile));
+ BufFile *file = (BufFile *) palloc0(sizeof(BufFile));
+ BufFileCommon *fcommon = &file->common;
+
+ fcommon->dirty = false;
+ fcommon->curFile = 0;
+ fcommon->curOffset = 0L;
+ fcommon->pos = 0;
+ fcommon->nbytes = 0;
file->numFiles = nfiles;
file->isInterXact = false;
- file->dirty = false;
file->resowner = CurrentResourceOwner;
- file->curFile = 0;
- file->curOffset = 0L;
- file->pos = 0;
- file->nbytes = 0;
return file;
}
@@ -132,7 +173,7 @@ makeBufFile(File firstfile)
file->files = (File *) palloc(sizeof(File));
file->files[0] = firstfile;
- file->readOnly = false;
+ file->common.readOnly = false;
file->fileset = NULL;
file->name = NULL;
@@ -252,7 +293,7 @@ BufFileCreateShared(SharedFileSet *fileset, const char *name)
file->name = pstrdup(name);
file->files = (File *) palloc(sizeof(File));
file->files[0] = MakeNewSharedSegment(file, 0);
- file->readOnly = false;
+ file->common.readOnly = false;
return file;
}
@@ -309,7 +350,7 @@ BufFileOpenShared(SharedFileSet *fileset, const char *name)
file = makeBufFileCommon(nfiles);
file->files = files;
- file->readOnly = true; /* Can't write to files opened this way */
+ file->common.readOnly = true; /* Can't write to files opened this way */
file->fileset = fileset;
file->name = pstrdup(name);
@@ -364,10 +405,10 @@ BufFileExportShared(BufFile *file)
Assert(file->fileset != NULL);
/* It's probably a bug if someone calls this twice. */
- Assert(!file->readOnly);
+ Assert(!file->common.readOnly);
BufFileFlush(file);
- file->readOnly = true;
+ file->common.readOnly = true;
}
/*
@@ -394,7 +435,7 @@ BufFileClose(BufFile *file)
* BufFileLoadBuffer
*
* Load some data into buffer, if possible, starting from curOffset.
- * At call, must have dirty = false, pos and nbytes = 0.
+ * At call, must have dirty = false, nbytes = 0.
* On exit, nbytes is number of bytes loaded.
*/
static void
@@ -405,27 +446,27 @@ BufFileLoadBuffer(BufFile *file)
/*
* Advance to next component file if necessary and possible.
*/
- if (file->curOffset >= MAX_PHYSICAL_FILESIZE &&
- file->curFile + 1 < file->numFiles)
+ if (file->common.curOffset >= MAX_PHYSICAL_FILESIZE &&
+ file->common.curFile + 1 < file->numFiles)
{
- file->curFile++;
- file->curOffset = 0L;
+ file->common.curFile++;
+ file->common.curOffset = 0L;
}
/*
* Read whatever we can get, up to a full bufferload.
*/
- thisfile = file->files[file->curFile];
- file->nbytes = FileRead(thisfile,
- file->buffer.data,
- sizeof(file->buffer),
- file->curOffset,
- WAIT_EVENT_BUFFILE_READ);
- if (file->nbytes < 0)
- file->nbytes = 0;
+ thisfile = file->files[file->common.curFile];
+ file->common.nbytes = FileRead(thisfile,
+ file->common.buffer.data,
+ sizeof(file->common.buffer),
+ file->common.curOffset,
+ WAIT_EVENT_BUFFILE_READ);
+ if (file->common.nbytes < 0)
+ file->common.nbytes = 0;
/* we choose not to advance curOffset here */
- if (file->nbytes > 0)
+ if (file->common.nbytes > 0)
pgBufferUsage.temp_blks_read++;
}
@@ -447,44 +488,44 @@ BufFileDumpBuffer(BufFile *file)
* Unlike BufFileLoadBuffer, we must dump the whole buffer even if it
* crosses a component-file boundary; so we need a loop.
*/
- while (wpos < file->nbytes)
+ while (wpos < file->common.nbytes)
{
off_t availbytes;
/*
* Advance to next component file if necessary and possible.
*/
- if (file->curOffset >= MAX_PHYSICAL_FILESIZE)
+ if (file->common.curOffset >= MAX_PHYSICAL_FILESIZE)
{
- while (file->curFile + 1 >= file->numFiles)
+ while (file->common.curFile + 1 >= file->numFiles)
extendBufFile(file);
- file->curFile++;
- file->curOffset = 0L;
+ file->common.curFile++;
+ file->common.curOffset = 0L;
}
/*
* Determine how much we need to write into this file.
*/
- bytestowrite = file->nbytes - wpos;
- availbytes = MAX_PHYSICAL_FILESIZE - file->curOffset;
+ bytestowrite = file->common.nbytes - wpos;
+ availbytes = MAX_PHYSICAL_FILESIZE - file->common.curOffset;
if ((off_t) bytestowrite > availbytes)
bytestowrite = (int) availbytes;
- thisfile = file->files[file->curFile];
+ thisfile = file->files[file->common.curFile];
bytestowrite = FileWrite(thisfile,
- file->buffer.data + wpos,
+ file->common.buffer.data + wpos,
bytestowrite,
- file->curOffset,
+ file->common.curOffset,
WAIT_EVENT_BUFFILE_WRITE);
if (bytestowrite <= 0)
return; /* failed to write */
- file->curOffset += bytestowrite;
+ file->common.curOffset += bytestowrite;
wpos += bytestowrite;
pgBufferUsage.temp_blks_written++;
}
- file->dirty = false;
+ file->common.dirty = false;
/*
* At this point, curOffset has been advanced to the end of the buffer,
@@ -492,19 +533,19 @@ BufFileDumpBuffer(BufFile *file)
* logical file position, ie, original value + pos, in case that is less
* (as could happen due to a small backwards seek in a dirty buffer!)
*/
- file->curOffset -= (file->nbytes - file->pos);
- if (file->curOffset < 0) /* handle possible segment crossing */
+ file->common.curOffset -= (file->common.nbytes - file->common.pos);
+ if (file->common.curOffset < 0) /* handle possible segment crossing */
{
- file->curFile--;
- Assert(file->curFile >= 0);
- file->curOffset += MAX_PHYSICAL_FILESIZE;
+ file->common.curFile--;
+ Assert(file->common.curFile >= 0);
+ file->common.curOffset += MAX_PHYSICAL_FILESIZE;
}
/*
* Now we can set the buffer empty without changing the logical position
*/
- file->pos = 0;
- file->nbytes = 0;
+ file->common.pos = 0;
+ file->common.nbytes = 0;
}
/*
@@ -515,43 +556,7 @@ BufFileDumpBuffer(BufFile *file)
size_t
BufFileRead(BufFile *file, void *ptr, size_t size)
{
- size_t nread = 0;
- size_t nthistime;
-
- if (file->dirty)
- {
- if (BufFileFlush(file) != 0)
- return 0; /* could not flush... */
- Assert(!file->dirty);
- }
-
- while (size > 0)
- {
- if (file->pos >= file->nbytes)
- {
- /* Try to load more data into buffer. */
- file->curOffset += file->pos;
- file->pos = 0;
- file->nbytes = 0;
- BufFileLoadBuffer(file);
- if (file->nbytes <= 0)
- break; /* no more data available */
- }
-
- nthistime = file->nbytes - file->pos;
- if (nthistime > size)
- nthistime = size;
- Assert(nthistime > 0);
-
- memcpy(ptr, file->buffer.data + file->pos, nthistime);
-
- file->pos += nthistime;
- ptr = (void *) ((char *) ptr + nthistime);
- size -= nthistime;
- nread += nthistime;
- }
-
- return nread;
+ return BufFileReadCommon(&file->common, ptr, size, false);
}
/*
@@ -562,48 +567,7 @@ BufFileRead(BufFile *file, void *ptr, size_t size)
size_t
BufFileWrite(BufFile *file, void *ptr, size_t size)
{
- size_t nwritten = 0;
- size_t nthistime;
-
- Assert(!file->readOnly);
-
- while (size > 0)
- {
- if (file->pos >= BLCKSZ)
- {
- /* Buffer full, dump it out */
- if (file->dirty)
- {
- BufFileDumpBuffer(file);
- if (file->dirty)
- break; /* I/O error */
- }
- else
- {
- /* Hmm, went directly from reading to writing? */
- file->curOffset += file->pos;
- file->pos = 0;
- file->nbytes = 0;
- }
- }
-
- nthistime = BLCKSZ - file->pos;
- if (nthistime > size)
- nthistime = size;
- Assert(nthistime > 0);
-
- memcpy(file->buffer.data + file->pos, ptr, nthistime);
-
- file->dirty = true;
- file->pos += nthistime;
- if (file->nbytes < file->pos)
- file->nbytes = file->pos;
- ptr = (void *) ((char *) ptr + nthistime);
- size -= nthistime;
- nwritten += nthistime;
- }
-
- return nwritten;
+ return BufFileWriteCommon(&file->common, ptr, size, false);
}
/*
@@ -614,10 +578,10 @@ BufFileWrite(BufFile *file, void *ptr, size_t size)
static int
BufFileFlush(BufFile *file)
{
- if (file->dirty)
+ if (file->common.dirty)
{
BufFileDumpBuffer(file);
- if (file->dirty)
+ if (file->common.dirty)
return EOF;
}
@@ -655,8 +619,8 @@ BufFileSeek(BufFile *file, int fileno, off_t offset, int whence)
* fileno. Note that large offsets (> 1 gig) risk overflow in this
* add, unless we have 64-bit off_t.
*/
- newFile = file->curFile;
- newOffset = (file->curOffset + file->pos) + offset;
+ newFile = file->common.curFile;
+ newOffset = (file->common.curOffset + file->common.pos) + offset;
break;
#ifdef NOT_USED
case SEEK_END:
@@ -673,9 +637,9 @@ BufFileSeek(BufFile *file, int fileno, off_t offset, int whence)
return EOF;
newOffset += MAX_PHYSICAL_FILESIZE;
}
- if (newFile == file->curFile &&
- newOffset >= file->curOffset &&
- newOffset <= file->curOffset + file->nbytes)
+ if (newFile == file->common.curFile &&
+ newOffset >= file->common.curOffset &&
+ newOffset <= file->common.curOffset + file->common.nbytes)
{
/*
* Seek is to a point within existing buffer; we can just adjust
@@ -683,7 +647,7 @@ BufFileSeek(BufFile *file, int fileno, off_t offset, int whence)
* whether reading or writing, but buffer remains dirty if we were
* writing.
*/
- file->pos = (int) (newOffset - file->curOffset);
+ file->common.pos = (int) (newOffset - file->common.curOffset);
return 0;
}
/* Otherwise, must reposition buffer, so flush any dirty data */
@@ -711,18 +675,18 @@ BufFileSeek(BufFile *file, int fileno, off_t offset, int whence)
if (newFile >= file->numFiles)
return EOF;
/* Seek is OK! */
- file->curFile = newFile;
- file->curOffset = newOffset;
- file->pos = 0;
- file->nbytes = 0;
+ file->common.curFile = newFile;
+ file->common.curOffset = newOffset;
+ file->common.pos = 0;
+ file->common.nbytes = 0;
return 0;
}
void
BufFileTell(BufFile *file, int *fileno, off_t *offset)
{
- *fileno = file->curFile;
- *offset = file->curOffset + file->pos;
+ *fileno = file->common.curFile;
+ *offset = file->common.curOffset + file->common.pos;
}
/*
@@ -756,8 +720,8 @@ BufFileTellBlock(BufFile *file)
{
long blknum;
- blknum = (file->curOffset + file->pos) / BLCKSZ;
- blknum += file->curFile * BUFFILE_SEG_SIZE;
+ blknum = (file->common.curOffset + file->common.pos) / BLCKSZ;
+ blknum += file->common.curFile * BUFFILE_SEG_SIZE;
return blknum;
}
@@ -816,8 +780,8 @@ BufFileAppend(BufFile *target, BufFile *source)
int i;
Assert(target->fileset != NULL);
- Assert(source->readOnly);
- Assert(!source->dirty);
+ Assert(source->common.readOnly);
+ Assert(!source->common.dirty);
Assert(source->fileset != NULL);
if (target->resowner != source->resowner)
@@ -831,3 +795,330 @@ BufFileAppend(BufFile *target, BufFile *source)
return startBlock;
}
+
+/*
+ * Open TransientBufFile at given path or create one if it does not
+ * exist. User will be allowed either to write to the file or to read from it,
+ * according to fileFlags, but not both.
+ */
+TransientBufFile *
+BufFileOpenTransient(const char *path, int fileFlags)
+{
+ bool readOnly;
+ bool append = false;
+ TransientBufFile *file;
+ BufFileCommon *fcommon;
+ int fd;
+ off_t size;
+
+ /* Either read or write mode, but not both. */
+ Assert((fileFlags & O_RDWR) == 0);
+
+ /* Check whether user wants read or write access. */
+ readOnly = (fileFlags & O_WRONLY) == 0;
+
+ /*
+ * Append mode for read access is not useful, so don't bother implementing
+ * it.
+ */
+ Assert(!(readOnly && append));
+
+ errno = 0;
+ fd = OpenTransientFile(path, fileFlags);
+ if (fd < 0)
+ {
+ /*
+ * If caller wants to read from file and the file is not there, he
+ * should be able to handle the condition on his own.
+ *
+ * XXX Shouldn't we always let caller evaluate errno?
+ */
+ if (errno == ENOENT && (fileFlags & O_RDONLY))
+ return NULL;
+
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not open file \"%s\": %m", path)));
+ }
+
+ file = (TransientBufFile *) palloc(sizeof(TransientBufFile));
+ fcommon = &file->common;
+ fcommon->dirty = false;
+ fcommon->pos = 0;
+ fcommon->nbytes = 0;
+ fcommon->readOnly = readOnly;
+ fcommon->append = append;
+ fcommon->curFile = 0;
+
+ file->path = pstrdup(path);
+ file->fd = fd;
+
+ errno = 0;
+ size = lseek(file->fd, 0, SEEK_END);
+ if (errno > 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not initialize TransientBufFile for file \"%s\": %m",
+ path)));
+
+ if (fcommon->append)
+ {
+ /* Position the buffer at the end of the file. */
+ fcommon->curOffset = size;
+ }
+ else
+ fcommon->curOffset = 0L;
+
+ return file;
+}
+
+/*
+ * Close a TransientBufFile.
+ */
+void
+BufFileCloseTransient(TransientBufFile *file)
+{
+ /* Flush any unwritten data. */
+ if (!file->common.readOnly &&
+ file->common.dirty && file->common.nbytes > 0)
+ {
+ BufFileDumpBufferTransient(file);
+
+ /*
+ * Caller of BufFileWriteTransient() recognizes the failure to flush
+ * buffer by the returned value, however this function has no return
+ * code.
+ */
+ if (file->common.dirty)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not flush file \"%s\": %m", file->path)));
+ }
+
+ if (CloseTransientFile(file->fd))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not close file \"%s\": %m", file->path)));
+
+ pfree(file->path);
+ pfree(file);
+}
+
+/*
+ * Load some data into buffer, if possible, starting from file->offset. At
+ * call, must have dirty = false, pos and nbytes = 0. On exit, nbytes is
+ * number of bytes loaded.
+ */
+static void
+BufFileLoadBufferTransient(TransientBufFile *file)
+{
+ Assert(file->common.readOnly);
+ Assert(!file->common.dirty);
+ Assert(file->common.pos == 0 && file->common.nbytes == 0);
+
+retry:
+
+ /*
+ * Read whatever we can get, up to a full bufferload.
+ */
+ errno = 0;
+ pgstat_report_wait_start(WAIT_EVENT_BUFFILE_READ);
+ file->common.nbytes = pg_pread(file->fd,
+ file->common.buffer.data,
+ sizeof(file->common.buffer),
+ file->common.curOffset);
+ pgstat_report_wait_end();
+
+ if (file->common.nbytes < 0)
+ {
+ /* TODO The W32 specific code, see FileWrite. */
+
+ /* OK to retry if interrupted */
+ if (errno == EINTR)
+ goto retry;
+
+ return;
+ }
+ /* we choose not to advance offset here */
+}
+
+/*
+ * Write contents of a transient file buffer to disk.
+ */
+static void
+BufFileDumpBufferTransient(TransientBufFile *file)
+{
+ int nwritten;
+
+ /* This function should only be needed during write access ... */
+ Assert(!file->common.readOnly);
+
+ /* ... and if there's some work to do. */
+ Assert(file->common.dirty);
+ Assert(file->common.nbytes > 0);
+
+retry:
+ errno = 0;
+ pgstat_report_wait_start(WAIT_EVENT_BUFFILE_WRITE);
+ nwritten = pg_pwrite(file->fd,
+ file->common.buffer.data,
+ file->common.nbytes,
+ file->common.curOffset);
+ pgstat_report_wait_end();
+
+ /* if write didn't set errno, assume problem is no disk space */
+ if (nwritten != file->common.nbytes && errno == 0)
+ errno = ENOSPC;
+
+ if (nwritten < 0)
+ {
+ /* TODO The W32 specific code, see FileWrite. */
+
+ /* OK to retry if interrupted */
+ if (errno == EINTR)
+ goto retry;
+
+ return; /* failed to write */
+ }
+
+ file->common.dirty = false;
+
+ file->common.pos = 0;
+ file->common.nbytes = 0;
+}
+
+/*
+ * Like BufFileRead() except it receives pointer to TransientBufFile.
+ */
+size_t
+BufFileReadTransient(TransientBufFile *file, void *ptr, size_t size)
+{
+ return BufFileReadCommon(&file->common, ptr, size, true);
+}
+
+/*
+ * Like BufFileWrite() except it receives pointer to TransientBufFile.
+ */
+size_t
+BufFileWriteTransient(TransientBufFile *file, void *ptr, size_t size)
+{
+ return BufFileWriteCommon(&file->common, ptr, size, true);
+}
+
+/*
+ * BufFileWriteCommon
+ *
+ * Functionality needed by both BufFileRead() and BufFileReadTransient().
+ */
+static size_t
+BufFileReadCommon(BufFileCommon *file, void *ptr, size_t size,
+ bool is_transient)
+{
+ size_t nread = 0;
+ size_t nthistime;
+
+ if (file->dirty)
+ {
+ /*
+ * Transient file currently does not allow both read and write access,
+ * so this function should not see dirty buffer.
+ */
+ Assert(!is_transient);
+
+ if (BufFileFlush((BufFile *) file) != 0)
+ return 0; /* could not flush... */
+ Assert(!file->dirty);
+ }
+
+ while (size > 0)
+ {
+ if (file->pos >= file->nbytes)
+ {
+ /* Try to load more data into buffer. */
+ file->curOffset += file->pos;
+ file->pos = 0;
+ file->nbytes = 0;
+
+ if (!is_transient)
+ BufFileLoadBuffer((BufFile *) file);
+ else
+ BufFileLoadBufferTransient((TransientBufFile *) file);
+
+ if (file->nbytes <= 0)
+ break; /* no more data available */
+ }
+
+ nthistime = file->nbytes - file->pos;
+ if (nthistime > size)
+ nthistime = size;
+ Assert(nthistime > 0);
+
+ memcpy(ptr, file->buffer.data + file->pos, nthistime);
+
+ file->pos += nthistime;
+ ptr = (void *) ((char *) ptr + nthistime);
+ size -= nthistime;
+ nread += nthistime;
+ }
+
+ return nread;
+}
+
+/*
+ * BufFileWriteCommon
+ *
+ * Functionality needed by both BufFileWrite() and BufFileWriteTransient().
+ */
+static size_t
+BufFileWriteCommon(BufFileCommon *file, void *ptr, size_t size,
+ bool is_transient)
+{
+ size_t nwritten = 0;
+ size_t nthistime;
+
+ Assert(!file->readOnly);
+
+ while (size > 0)
+ {
+ if (file->pos >= BLCKSZ)
+ {
+ /* Buffer full, dump it out */
+ if (file->dirty)
+ {
+ if (!is_transient)
+ BufFileDumpBuffer((BufFile *) file);
+ else
+ BufFileDumpBufferTransient((TransientBufFile *) file);
+
+ if (file->dirty)
+ break; /* I/O error */
+ }
+ else
+ {
+ Assert(!is_transient);
+
+ /* Hmm, went directly from reading to writing? */
+ file->curOffset += file->pos;
+ file->pos = 0;
+ file->nbytes = 0;
+ }
+ }
+
+ nthistime = BLCKSZ - file->pos;
+ if (nthistime > size)
+ nthistime = size;
+ Assert(nthistime > 0);
+
+ memcpy(file->buffer.data + file->pos, ptr, nthistime);
+
+ file->dirty = true;
+ file->pos += nthistime;
+ if (file->nbytes < file->pos)
+ file->nbytes = file->pos;
+ ptr = (void *) ((char *) ptr + nthistime);
+ size -= nthistime;
+ nwritten += nthistime;
+ }
+
+ return nwritten;
+}
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index fa5dca3b87..c615bac6e1 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -910,8 +910,6 @@ typedef enum
WAIT_EVENT_RELATION_MAP_READ,
WAIT_EVENT_RELATION_MAP_SYNC,
WAIT_EVENT_RELATION_MAP_WRITE,
- WAIT_EVENT_REORDER_BUFFER_READ,
- WAIT_EVENT_REORDER_BUFFER_WRITE,
WAIT_EVENT_REORDER_LOGICAL_MAPPING_READ,
WAIT_EVENT_REPLICATION_SLOT_READ,
WAIT_EVENT_REPLICATION_SLOT_RESTORE_SYNC,
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index 7646b60f94..31c50665ed 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -387,10 +387,6 @@ struct ReorderBuffer
MemoryContext tup_context;
XLogRecPtr current_restart_decoding_lsn;
-
- /* buffer for disk<->memory conversions */
- char *outbuf;
- Size outbufsize;
};
diff --git a/src/include/storage/buffile.h b/src/include/storage/buffile.h
index 1fba404fe2..625fa0014c 100644
--- a/src/include/storage/buffile.h
+++ b/src/include/storage/buffile.h
@@ -28,9 +28,13 @@
#include "storage/sharedfileset.h"
-/* BufFile is an opaque type whose details are not known outside buffile.c. */
+/*
+ * BufFile and TransientBufFile are opaque types whose details are not known
+ * outside buffile.c.
+ */
typedef struct BufFile BufFile;
+typedef struct TransientBufFile TransientBufFile;
/*
* prototypes for functions in buffile.c
@@ -51,4 +55,11 @@ extern void BufFileExportShared(BufFile *file);
extern BufFile *BufFileOpenShared(SharedFileSet *fileset, const char *name);
extern void BufFileDeleteShared(SharedFileSet *fileset, const char *name);
+extern TransientBufFile *BufFileOpenTransient(const char *path, int fileFlags);
+extern void BufFileCloseTransient(TransientBufFile *file);
+extern size_t BufFileReadTransient(TransientBufFile *file, void *ptr,
+ size_t size);
+extern size_t BufFileWriteTransient(TransientBufFile *file, void *ptr,
+ size_t size);
+
#endif /* BUFFILE_H */
v03-002-XLogWritePages.patch (text/x-diff, 2.7K) download
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 19b32e21df..6fabc5e27f 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -874,6 +874,7 @@ static void LocalSetXLogInsertAllowed(void);
static void CreateEndOfRecoveryRecord(void);
static void CheckPointGuts(XLogRecPtr checkPointRedo, int flags);
static void KeepLogSeg(XLogRecPtr recptr, XLogSegNo *logSegNo);
+static Size XLogWritePages(char *from, int npages, uint32 startoffset);
static XLogRecPtr XLogGetReplicationSlotMinimumLSN(void);
static void AdvanceXLInsertBuffer(XLogRecPtr upto, bool opportunistic);
@@ -2484,35 +2485,10 @@ XLogWrite(XLogwrtRqst WriteRqst, bool flexible)
finishing_seg)
{
char *from;
- Size nbytes;
- Size nleft;
- int written;
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
- nbytes = npages * (Size) XLOG_BLCKSZ;
- nleft = nbytes;
- do
- {
- errno = 0;
- pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
- written = pg_pwrite(openLogFile, from, nleft, startoffset);
- pgstat_report_wait_end();
- if (written <= 0)
- {
- if (errno == EINTR)
- continue;
- ereport(PANIC,
- (errcode_for_file_access(),
- errmsg("could not write to log file %s "
- "at offset %u, length %zu: %m",
- XLogFileNameP(ThisTimeLineID, openLogSegNo),
- startoffset, nleft)));
- }
- nleft -= written;
- from += written;
- startoffset += written;
- } while (nleft > 0);
+ startoffset += XLogWritePages(from, npages, startoffset);
npages = 0;
@@ -2629,6 +2605,45 @@ XLogWrite(XLogwrtRqst WriteRqst, bool flexible)
}
/*
+ * Write page(s) to the XLOG file.
+ *
+ * Returns the number of bytes written.
+ */
+static Size
+XLogWritePages(char *from, int npages, uint32 startoffset)
+{
+ Size nbytes,
+ nleft;
+ Size written;
+
+ nbytes = npages * (Size) XLOG_BLCKSZ;
+ nleft = nbytes;
+ do
+ {
+ errno = 0;
+ pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
+ written = pg_pwrite(openLogFile, from, nleft, startoffset);
+ pgstat_report_wait_end();
+ if (written <= 0)
+ {
+ if (errno == EINTR)
+ continue;
+ ereport(PANIC,
+ (errcode_for_file_access(),
+ errmsg("could not write to log file %s "
+ "at offset %u, length %zu: %m",
+ XLogFileNameP(ThisTimeLineID, openLogSegNo),
+ startoffset, nbytes)));
+ }
+ nleft -= written;
+ from += written;
+ startoffset += written;
+ } while (nleft > 0);
+
+ return written;
+}
+
+/*
* Record the LSN for an asynchronous transaction commit/abort
* and nudge the WALWriter if there is work for it to do.
* (This should not be called for synchronous commits.)
v03-003-ClusterEncryption.patch (text/x-diff, 249.8K) download
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 189219ad88..239181d3bb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -8463,6 +8463,62 @@ dynamic_library_path = 'C:\tools\postgresql;H:\my_project\lib;$libdir'
</sect2>
</sect1>
+ <sect1 id="runtime-config-encryption">
+ <title>Encryption</title>
+
+ <variablelist>
+
+ <varlistentry id="guc-encryption-key-cmd" xreflabel="encryption_key_command">
+ <term><varname>encryption_key_command</varname> (<type>string</type>)
+ <indexterm>
+ <primary>encryption</primary>
+ </indexterm>
+ <indexterm>
+ <primary>key command</primary>
+ </indexterm>
+ <indexterm>
+ <primary><varname>encryption_key_command</varname> configuration
+ parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ This setting specifies path to executable file that returns
+ either <literal>encryption_key=</literal> for key
+ and <literal>encryption_password=</literal> for password.
+ </para>
+
+ <para>
+ The encryption key is expected in hexadecimal format, two characters
+ (hexadecimal digits) per
+ byte. Since <productname>PostgreSQL</productname> currently uses key
+ of length 32 bytes (256 bits), the expected length of the key string
+ is 64 characters. For example:
+<computeroutput>
+encryption_key=882fb7c12e80280fd664c69d2d636913e86c381ba487c82f77653c0fac8ffc69
+</computeroutput>
+ </para>
+
+ <para>
+ An example of encryption password:
+<computeroutput>
+encryption_password=mysecurepwd
+</computeroutput>
+ Minimum password length is 8 characters, the maximum length is 16
+ characters.
+ </para>
+
+ <para>
+ <varname>encryption_key_command</varname> can only be set in
+ the <filename>postgresql.conf</filename> file or on the server command
+ line.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </sect1>
+
<sect1 id="runtime-config-locks">
<title>Lock Management</title>
@@ -8998,6 +9054,20 @@ dynamic_library_path = 'C:\tools\postgresql;H:\my_project\lib;$libdir'
</listitem>
</varlistentry>
+ <varlistentry id="guc-data-encryption" xreflabel="data_encryption">
+ <term><varname>data_encryption</varname> (<type>boolean</type>)
+ <indexterm>
+ <primary><varname>data_encryption</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Reports whether data encryption is enabled for this cluster.
+ See <xref linkend="encryption"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-data-directory-mode" xreflabel="data_directory_mode">
<term><varname>data_directory_mode</varname> (<type>integer</type>)
<indexterm>
@@ -9812,6 +9882,10 @@ LOG: CleanUpLock: deleting: lock(0xb7acd844) id(24688,24696,0,0,0,1)
<entry><literal>listen_addresses = '*'</literal></entry>
</row>
<row>
+ <entry><option>-K <replaceable>x</replaceable></option></entry>
+ <entry><literal>encryption_key_command = <replaceable>x</replaceable></literal></entry>
+ </row>
+ <row>
<entry><option>-k <replaceable>x</replaceable></option></entry>
<entry><literal>unix_socket_directories = <replaceable>x</replaceable></literal></entry>
</row>
diff --git a/doc/src/sgml/encryption.sgml b/doc/src/sgml/encryption.sgml
new file mode 100644
index 0000000000..20de042d1a
--- /dev/null
+++ b/doc/src/sgml/encryption.sgml
@@ -0,0 +1,74 @@
+<!-- doc/src/sgml/encryption.sgml -->
+
+<chapter id="encryption">
+ <title>Transparent Cluster Encryption</title>
+
+ <para>
+ Cluster encryption can be used if <acronym>DBA</acronym> cannot or does not
+ want to rely on the filesystem in terms of data confidentiality. If this
+ feature is enabled, <productname>PostgreSQL</productname> encrypts data
+ (both relations and write-ahead-log) when writing it to disk, and decrypts
+ it when reading. The encryption is transparent, so applications see no
+ difference between encrypted and unencrypted cluster.
+ </para>
+
+ <para>
+ To create encrypted cluster, use the <option>-K</option> option to pass the
+ <xref linkend="app-initdb"/> utility path to the command to retrieve the
+ encryption key or password. For example:
+<screen>
+<prompt>$</prompt> <userinput>initdb -D /usr/local/pgsql/data -K /usr/local/pgsql/fetch_key_cmd</userinput>
+</screen>
+ Here <filename>/usr/local/pgsql/fetch_key_cmd</filename> is an executable
+ file that returns either encryption key or encryption password with the
+ appropriate prefix, see <xref linkend="guc-encryption-key-cmd"/> for more
+ information.
+ </para>
+
+ <note>
+ <para>
+ Internally, <productname>PostgreSQL</productname> always uses the
+ encryption key. If the encryption key command returns password, the
+ encryption key is derived from it during startup. If user needs the key, he
+ can always use the <xref linkend="app-pgkeysetup"/> utility to derive it
+ himself.
+ </para>
+ </note>
+
+ <para>
+ On completion, <command>initdb</command> stores the encryption key command
+ to <filename>postgresql.conf</filename>. Thus user can control the cluster
+ using <xref linkend="app-pg-ctl"/> without passing it the encryption command
+ again and again.
+ </para>
+
+ <para>
+ If encryption is enabled, <xref linkend="guc-full-page-writes"/> must be
+ turned on, otherwise the server refuses to start. This is because the
+ encryption introduces dependencies between data within a page, and thus
+ server crash during disk write can result in more serious damage of the page
+ than it would do without encryption. The whole page needs to be retrieved
+ from WAL in such a case to ensure reliable recovery.
+ </para>
+
+ <para>
+ Once the <productname>PostgreSQL</productname> server is running, client
+ applications should recognize no difference from an unencrypted cluster,
+ except that <xref linkend="guc-data-encryption"/> configuration variable is
+ set.
+ </para>
+
+ <para>
+ Unlike <xref linkend="app-pg-ctl"/>, some of the server applications (for
+ example <xref linkend="pgwaldump"/>) do need the <option>-K</option> because
+ they are not able to process <filename>postgresql.conf</filename> file.
+ </para>
+
+ <para>
+ Since WAL is encrypted, any replication solution based on log shipping
+ (<xref linkend="warm-standby"/>) assume that all standby servers are
+ encrypted using the same key as their standby server. On the other hand,
+ <xref linkend="logical-replication"/> allows replication between encrypted
+ and unencrypted clusters, or between clusters encrypted with different keys.
+ </para>
+</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 7e37042a55..0e22070504 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -48,6 +48,7 @@
<!ENTITY wal SYSTEM "wal.sgml">
<!ENTITY logical-replication SYSTEM "logical-replication.sgml">
<!ENTITY jit SYSTEM "jit.sgml">
+<!ENTITY encryption SYSTEM "encryption.sgml">
<!-- programmer's guide -->
<!ENTITY bgworker SYSTEM "bgworker.sgml">
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index 449386243b..a2daf33e6f 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -245,8 +245,9 @@ su - postgres
<listitem>
<para>
You need <productname>OpenSSL</productname>, if you want to support
- encrypted client connections. <productname>OpenSSL</productname> is
- also required for random number generation on platforms that do not
+ on-disk data encryption or encrypted client
+ connections. <productname>OpenSSL</productname> is also required for
+ random number generation on platforms that do not
have <filename>/dev/urandom</filename> (except Windows). The minimum
version required is 0.9.8.
</para>
@@ -835,10 +836,11 @@ su - postgres
</term>
<listitem>
<para>
- Build with support for <acronym>SSL</acronym> (encrypted)
- connections. This requires the <productname>OpenSSL</productname>
- package to be installed. <filename>configure</filename> will check
- for the required header files and libraries to make sure that
+ Build with support for on-disk data encryption
+ or <acronym>SSL</acronym> (encrypted) connections. This requires
+ the <productname>OpenSSL</productname> package to be
+ installed. <filename>configure</filename> will check for the
+ required header files and libraries to make sure that
your <productname>OpenSSL</productname> installation is sufficient
before proceeding.
</para>
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 3e115f1c76..b0d35600b2 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -163,6 +163,7 @@
&wal;
&logical-replication;
&jit;
+ &encryption;
®ress;
</part>
diff --git a/doc/src/sgml/ref/allfiles.sgml b/doc/src/sgml/ref/allfiles.sgml
index 8d91f3529e..4799fe61c7 100644
--- a/doc/src/sgml/ref/allfiles.sgml
+++ b/doc/src/sgml/ref/allfiles.sgml
@@ -206,6 +206,7 @@ Complete list of usable sgml source files in this directory.
<!ENTITY pgDump SYSTEM "pg_dump.sgml">
<!ENTITY pgDumpall SYSTEM "pg_dumpall.sgml">
<!ENTITY pgIsready SYSTEM "pg_isready.sgml">
+<!ENTITY pgKeysetup SYSTEM "pg_keysetup.sgml">
<!ENTITY pgReceivewal SYSTEM "pg_receivewal.sgml">
<!ENTITY pgRecvlogical SYSTEM "pg_recvlogical.sgml">
<!ENTITY pgResetwal SYSTEM "pg_resetwal.sgml">
diff --git a/doc/src/sgml/ref/initdb.sgml b/doc/src/sgml/ref/initdb.sgml
index 7fc3152c6d..8b9b7ab25b 100644
--- a/doc/src/sgml/ref/initdb.sgml
+++ b/doc/src/sgml/ref/initdb.sgml
@@ -209,6 +209,18 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry id="app-initdb-data-encr-cmd" xreflabel="data encryption">
+ <term><option>-K</option></term>
+ <term><option>--encryption-key-command=<replaceable class="parameter">command</replaceable></option></term>
+ <listitem>
+ <para>
+ Encrypt the cluster data using a key or password retrieved from the
+ command specified here. This option can only be set during
+ initialization, and cannot be changed later.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="app-initdb-data-checksums" xreflabel="data checksums">
<term><option>-k</option></term>
<term><option>--data-checksums</option></term>
diff --git a/doc/src/sgml/ref/pg_checksums.sgml b/doc/src/sgml/ref/pg_checksums.sgml
index a0ffeb0ab0..bc42bb6168 100644
--- a/doc/src/sgml/ref/pg_checksums.sgml
+++ b/doc/src/sgml/ref/pg_checksums.sgml
@@ -101,6 +101,17 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-K</option></term>
+ <term><option>--encryption-key-command=<replaceable class="parameter">command</replaceable></option></term>
+ <listitem>
+ <para>
+ If the cluster is encrypted, use key or password retrieved from the
+ command specified here to decrypt and encrypt the data.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-N</option></term>
<term><option>--no-sync</option></term>
<listitem>
diff --git a/doc/src/sgml/ref/pg_keysetup.sgml b/doc/src/sgml/ref/pg_keysetup.sgml
new file mode 100644
index 0000000000..89428fe6bd
--- /dev/null
+++ b/doc/src/sgml/ref/pg_keysetup.sgml
@@ -0,0 +1,74 @@
+<!--
+doc/src/sgml/ref/pg_keysetup.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="app-pgkeysetup">
+ <indexterm zone="app-pgkeysetup">
+ <primary>pg_keysetup</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle><application>pg_keysetup</application></refentrytitle>
+ <manvolnum>1</manvolnum>
+ <refmiscinfo>Application</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>pg_keysetup</refname>
+ <refpurpose>derive encryption key from encryption password</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+ <cmdsynopsis>
+ <command>pg_keysetup</command>
+ <group choice="plain">
+ <group choice="opt">
+ <arg choice="plain"><option>-D</option></arg>
+ </group>
+ <replaceable class="parameter"> datadir</replaceable>
+ </group>
+ </cmdsynopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>Description</title>
+ <para>
+ <application>pg_keysetup</application> reads encryption password from the
+ standard input, runs <firstterm>key derivation function</firstterm> on it
+ and writes encryption key to the standard output. This is useful if
+ <xref linkend="guc-encryption-key-cmd"/> returns encryption password, but user
+ needs the encryption key. The typical use case is that user needs to run
+ some server application (for example <xref linkend="pgwaldump"/>) which
+ only accepts the key.
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>Options</title>
+
+ <para>
+ The application currently accepts only the following command-line option:
+
+ <variablelist>
+ <varlistentry>
+ <term><option>-D <replaceable>directory</replaceable></option></term>
+ <listitem>
+ <para>
+ Specifies the directory where the database cluster is stored.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>Notes</title>
+ <para>
+ When run against different data directories, the function is likely to turn
+ the same password into different keys.
+ </para>
+ </refsect1>
+</refentry>
diff --git a/doc/src/sgml/ref/pg_resetwal.sgml b/doc/src/sgml/ref/pg_resetwal.sgml
index 8a9e22d050..dbdc234666 100644
--- a/doc/src/sgml/ref/pg_resetwal.sgml
+++ b/doc/src/sgml/ref/pg_resetwal.sgml
@@ -100,6 +100,17 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-K</option></term>
+ <term><option>--encryption-key-command=<replaceable class="parameter">command</replaceable></option></term>
+ <listitem>
+ <para>
+ If the cluster is encrypted, use key or password retrieved from the
+ command specified here to decrypt and encrypt the data.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-n</option></term>
<term><option>--dry-run</option></term>
<listitem>
diff --git a/doc/src/sgml/ref/pg_rewind.sgml b/doc/src/sgml/ref/pg_rewind.sgml
index 4d91eeb0ff..79ed2347c0 100644
--- a/doc/src/sgml/ref/pg_rewind.sgml
+++ b/doc/src/sgml/ref/pg_rewind.sgml
@@ -140,6 +140,17 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-K</option></term>
+ <term><option>--encryption-key-command=<replaceable class="parameter">command</replaceable></option></term>
+ <listitem>
+ <para>
+ If the cluster is encrypted, use key or password retrieved from the
+ command specified here to decrypt and encrypt the data.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>--source-pgdata=<replaceable class="parameter">directory</replaceable></option></term>
<listitem>
<para>
diff --git a/doc/src/sgml/ref/pg_waldump.sgml b/doc/src/sgml/ref/pg_waldump.sgml
index 329c10e430..d012b303fc 100644
--- a/doc/src/sgml/ref/pg_waldump.sgml
+++ b/doc/src/sgml/ref/pg_waldump.sgml
@@ -101,6 +101,17 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-K</option></term>
+ <term><option>--encryption-key-command=<replaceable class="parameter">command</replaceable></option></term>
+ <listitem>
+ <para>
+ If the cluster is encrypted, use key or password retrieved from the
+ command specified here to decrypt and encrypt the data.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-n <replaceable>limit</replaceable></option></term>
<term><option>--limit=<replaceable>limit</replaceable></option></term>
<listitem>
diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml
index cef09dd38b..7db13f716a 100644
--- a/doc/src/sgml/reference.sgml
+++ b/doc/src/sgml/reference.sgml
@@ -279,6 +279,7 @@
&pgChecksums;
&pgControldata;
&pgCtl;
+ &pgKeysetup;
&pgResetwal;
&pgRewind;
&pgtestfsync;
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 6fabc5e27f..7918f0b0cf 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -55,6 +55,7 @@
#include "replication/walreceiver.h"
#include "replication/walsender.h"
#include "storage/bufmgr.h"
+#include "storage/encryption.h"
#include "storage/fd.h"
#include "storage/ipc.h"
#include "storage/large_object.h"
@@ -77,6 +78,7 @@
#include "pg_trace.h"
extern uint32 bootstrap_data_checksum_version;
+extern char *bootstrap_encryption_sample;
/* Unsupported old recovery command file names (relative to $PGDATA) */
#define RECOVERY_COMMAND_FILE "recovery.conf"
@@ -2488,7 +2490,61 @@ XLogWrite(XLogwrtRqst WriteRqst, bool flexible)
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
- startoffset += XLogWritePages(from, npages, startoffset);
+ if (data_encrypted)
+ {
+ int i,
+ nencrypted;
+ char *to;
+ uint32 encr_offset;
+
+ /*
+ * Encrypt and write multiple pages at a time, in order to
+ * reduce the number of syscalls.
+ */
+ nencrypted = 0;
+ to = encrypt_buf_xlog;
+ encr_offset = startoffset;
+ for (i = 1; i <= npages; i++)
+ {
+ char tweak[TWEAK_SIZE];
+ Size nbytes;
+
+ XLogEncryptionTweak(tweak, ThisTimeLineID, openLogSegNo, encr_offset);
+
+ /*
+ * We should not encrypt the unused space, in order to
+ * avoid "reused key attack".
+ */
+ if (i == npages && ispartialpage)
+ nbytes = WriteRqst.Write % XLOG_BLCKSZ;
+ else
+ nbytes = XLOG_BLCKSZ;
+
+ encrypt_block(from, to, nbytes, tweak, true);
+ nencrypted++;
+ from += XLOG_BLCKSZ;
+ to += XLOG_BLCKSZ;
+ encr_offset += XLOG_BLCKSZ;
+
+ /*
+ * Write the encrypted data if the encryption buffer is
+ * full or if the last page has been encrypted.
+ */
+ if (nencrypted >= XLOG_ENCRYPT_BUF_PAGES || i >= npages)
+ {
+ startoffset += XLogWritePages(encrypt_buf_xlog,
+ nencrypted,
+ startoffset);
+
+ /* Prepare for the next round of page encryptions. */
+ nencrypted = 0;
+ to = encrypt_buf_xlog;
+ encr_offset = startoffset;
+ }
+ }
+ }
+ else
+ startoffset += XLogWritePages(from, npages, startoffset);
npages = 0;
@@ -3478,6 +3534,24 @@ XLogFileCopy(XLogSegNo destsegno, TimeLineID srcTLI, XLogSegNo srcsegno,
}
pgstat_report_wait_end();
}
+
+ /*
+ * Since timeline is being changed and since encryption tweak contains
+ * the timeline, we need to decrypt the buffer and encrypt it with the
+ * new tweak. Do not encrypt the unused space, in order to avoid
+ * "reused key attack".
+ */
+ if (data_encrypted && nread > 0)
+ {
+ char tweak[TWEAK_SIZE];
+
+ XLogEncryptionTweak(tweak, srcTLI, srcsegno, nbytes);
+ decrypt_block(buffer.data, buffer.data, nread, tweak, true);
+
+ XLogEncryptionTweak(tweak, ThisTimeLineID, destsegno, nbytes);
+ encrypt_block(buffer.data, buffer.data, nread, tweak, true);
+ }
+
errno = 0;
pgstat_report_wait_start(WAIT_EVENT_WAL_COPY_WRITE);
if ((int) write(fd, buffer.data, sizeof(buffer)) != (int) sizeof(buffer))
@@ -4790,6 +4864,54 @@ ReadControlFile(void)
ereport(ERROR, (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("\"max_wal_size\" must be at least twice \"wal_segment_size\"")));
+ /*
+ * Initialize encryption, but not if the current backend has already done
+ * that.
+ */
+ if (ControlFile->data_cipher > PG_CIPHER_NONE && !data_encrypted)
+ {
+ char sample[ENCRYPTION_SAMPLE_SIZE];
+
+ setup_encryption(false, NULL);
+ SetConfigOption("data_encryption", "true", PGC_INTERNAL,
+ PGC_S_OVERRIDE);
+
+ memset(sample, 0, ENCRYPTION_SAMPLE_SIZE);
+ sample_encryption(sample);
+
+ if (memcmp(ControlFile->encryption_verification, sample, ENCRYPTION_SAMPLE_SIZE))
+ ereport(FATAL,
+ (errmsg("invalid encryption key"),
+ errdetail("The passed encryption key does not match"
+ " database encryption key.")));
+
+ /*
+ * full_page_writes must be set because torn page write of an
+ * encrypted page implies that decryption of the page will produce
+ * garbage. This damage can affect even those parts of the page that
+ * haven't been modified by any access method. And since no access
+ * method modified those parts, there might be no XLOG records to
+ * repair them during crash recovery. So full page image is the only
+ * way to fix such a page.
+ *
+ * XXX It would be nice to have guc.c check so that we don't have to
+ * copy and paste the error message. However it's unclear how to
+ * ensure that either ERROR is raised or nothing happens at all. It
+ * seems that set_config_option() can change
+ * config_generic.reset_source if the check succeeded, but that's too
+ * invasive.
+ */
+ if (!fullPageWrites)
+ ereport(FATAL,
+ (errmsg("invalid value for parameter \"full_page_writes\": %d",
+ fullPageWrites),
+ errdetail("Cannot disable parameter when the cluster is encrypted.")));
+ }
+
+ /*
+ * This calculation relies on data_encryption (in particular the header
+ * sizes do), so we could not do it earlier.
+ */
UsableBytesInSegment =
(wal_segment_size / XLOG_BLCKSZ * UsableBytesInPage) -
(SizeOfXLogLongPHD - SizeOfXLogShortPHD);
@@ -5220,6 +5342,14 @@ BootStrapXLOG(void)
use_existent = false;
openLogFile = XLogFileInit(1, &use_existent, false);
+ if (data_encrypted)
+ {
+ char tweak[TWEAK_SIZE];
+
+ XLogEncryptionTweak(tweak, ThisTimeLineID, 1, 0);
+ encrypt_block((char *) page, (char *) page, XLOG_BLCKSZ, tweak, true);
+ }
+
/* Write the first page with the initial record */
errno = 0;
pgstat_report_wait_start(WAIT_EVENT_WAL_BOOTSTRAP_WRITE);
@@ -5270,6 +5400,22 @@ BootStrapXLOG(void)
ControlFile->wal_log_hints = wal_log_hints;
ControlFile->track_commit_timestamp = track_commit_timestamp;
ControlFile->data_checksum_version = bootstrap_data_checksum_version;
+ ControlFile->data_cipher = PG_CIPHER_AES_BLOCK_CBC_256_STREAM_CTR_256;
+
+ if (data_encrypted)
+ {
+ char *sample;
+
+ sample = palloc0(ENCRYPTION_SAMPLE_SIZE);
+ sample_encryption(sample);
+
+ memcpy(ControlFile->encryption_verification, sample,
+ ENCRYPTION_SAMPLE_SIZE);
+ pfree(sample);
+ }
+ else
+ memset(ControlFile->encryption_verification, 0,
+ ENCRYPTION_SAMPLE_SIZE);
/* some additional ControlFile fields are set in WriteControlFile() */
@@ -11641,6 +11787,14 @@ retry:
Assert(targetPageOff == readOff);
Assert(reqLen <= readLen);
+ if (data_encrypted)
+ {
+ char tweak[TWEAK_SIZE];
+
+ XLogEncryptionTweak(tweak, curFileTLI, readSegNo, readOff);
+ decrypt_block(readBuf, readBuf, XLOG_BLCKSZ, tweak, true);
+ }
+
*readTLI = curFileTLI;
/*
diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c
index 1c76dcfa0d..6fdba0eff4 100644
--- a/src/backend/access/transam/xloginsert.c
+++ b/src/backend/access/transam/xloginsert.c
@@ -28,6 +28,7 @@
#include "miscadmin.h"
#include "replication/origin.h"
#include "storage/bufmgr.h"
+#include "storage/encryption.h"
#include "storage/proc.h"
#include "utils/memutils.h"
#include "pg_trace.h"
diff --git a/src/backend/access/transam/xlogutils.c b/src/backend/access/transam/xlogutils.c
index 10a663bae6..0ebed22895 100644
--- a/src/backend/access/transam/xlogutils.c
+++ b/src/backend/access/transam/xlogutils.c
@@ -25,6 +25,7 @@
#include "access/xlogutils.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "storage/encryption.h"
#include "storage/smgr.h"
#include "utils/guc.h"
#include "utils/hsearch.h"
@@ -653,12 +654,13 @@ XLogTruncateRelation(RelFileNode rnode, ForkNumber forkNum,
* frontend). Probably these should be merged at some point.
*/
static void
-XLogRead(char *buf, int segsize, TimeLineID tli, XLogRecPtr startptr,
- Size count)
+XLogRead(char *buf, int segsize, TimeLineID tli, XLogRecPtr startptr, Size count)
{
- char *p;
XLogRecPtr recptr;
Size nbytes;
+ char *decrypt_p;
+ uint32 decryptOff;
+ char *p;
/* state maintained across calls */
static int sendFile = -1;
@@ -668,17 +670,42 @@ XLogRead(char *buf, int segsize, TimeLineID tli, XLogRecPtr startptr,
Assert(segsize == wal_segment_size);
+ /*
+ * XXX Currently the function is only called with startptr at page
+ * boundary. If it should change, encryption needs to reflect this fact,
+ * i.e. read data from the beginning of the page. Actually this is a
+ * reason to adjust and use walsender.c:XLogRead().
+ */
+ Assert(startptr % XLOG_BLCKSZ == 0);
+
p = buf;
recptr = startptr;
nbytes = count;
+ decrypt_p = p;
+ decryptOff = XLogSegmentOffset(recptr, segsize);
+
while (nbytes > 0)
{
uint32 startoff;
int segbytes;
int readbytes;
- startoff = XLogSegmentOffset(recptr, segsize);
+ if (recptr == startptr)
+ startoff = decryptOff;
+ else
+ {
+ startoff = XLogSegmentOffset(recptr, segsize);
+ if (startoff == 0)
+ {
+ /*
+ * If segment boundary was reached, decryptOff should have
+ * caught up, so we can (and should) sync it with startoff.
+ */
+ Assert(decryptOff == segsize);
+ decryptOff = startoff;
+ }
+ }
/* Do we need to switch to a different xlog segment? */
if (sendFile < 0 || !XLByteInSeg(recptr, sendSegNo, segsize) ||
@@ -758,7 +785,30 @@ XLogRead(char *buf, int segsize, TimeLineID tli, XLogRecPtr startptr,
sendOff += readbytes;
nbytes -= readbytes;
p += readbytes;
+
+ /* Decrypt completed blocks */
+ if (data_encrypted)
+ {
+ while (decrypt_p + XLOG_BLCKSZ <= p)
+ {
+ char tweak[TWEAK_SIZE];
+
+ XLogEncryptionTweak(tweak, tli, sendSegNo, decryptOff);
+ decrypt_block(decrypt_p, decrypt_p, XLOG_BLCKSZ, tweak,
+ true);
+
+ decrypt_p += XLOG_BLCKSZ;
+ decryptOff += XLOG_BLCKSZ;
+ }
+ }
}
+
+ /*
+ * XXX Currently the function is only called with count==XLOG_BLCKSZ. If
+ * that should change, the rest of the data needs to be decrypted. Another
+ * reason to reuse walsender.c:XLogRead().
+ */
+ Assert(decrypt_p == p);
}
/*
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index d8776e192e..e68b8019f0 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -26,6 +26,7 @@
#include "bootstrap/bootstrap.h"
#include "catalog/index.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_control.h"
#include "catalog/pg_type.h"
#include "common/link-canary.h"
#include "libpq/pqsignal.h"
@@ -40,6 +41,7 @@
#include "storage/bufmgr.h"
#include "storage/bufpage.h"
#include "storage/condition_variable.h"
+#include "storage/encryption.h"
#include "storage/ipc.h"
#include "storage/proc.h"
#include "tcop/tcopprot.h"
@@ -52,7 +54,6 @@
uint32 bootstrap_data_checksum_version = 0; /* No checksum */
-
#define ALLOC(t, c) \
((t *) MemoryContextAllocZero(TopMemoryContext, (unsigned)(c) * sizeof(t)))
@@ -226,7 +227,7 @@ AuxiliaryProcessMain(int argc, char *argv[])
/* If no -x argument, we are a CheckerProcess */
MyAuxProcType = CheckerProcess;
- while ((flag = getopt(argc, argv, "B:c:d:D:Fkr:x:X:-:")) != -1)
+ while ((flag = getopt(argc, argv, "B:c:d:D:FkK:r:x:X:-:")) != -1)
{
switch (flag)
{
@@ -252,6 +253,25 @@ AuxiliaryProcessMain(int argc, char *argv[])
case 'F':
SetConfigOption("fsync", "false", PGC_POSTMASTER, PGC_S_ARGV);
break;
+#ifdef USE_ENCRYPTION
+ case 'K':
+ encryption_key_command = strdup(optarg);
+
+ /*
+ * When auxiliary process (typically bootstrap) starts, the
+ * control file might not exist yet. In this case we also use
+ * encryption_key_command to indicate that the encryption is
+ * enabled.
+ *
+ * Postmaster should not set this variable. Instead, it just
+ * sets data_encrypted according to the control file and child
+ * processes inherit that.
+ */
+ Assert(!IsUnderPostmaster);
+ data_encrypted = true;
+
+ break;
+#endif /* USE_ENCRYPTION */
case 'k':
bootstrap_data_checksum_version = PG_DATA_CHECKSUM_VERSION;
break;
@@ -373,6 +393,20 @@ AuxiliaryProcessMain(int argc, char *argv[])
if (!IsUnderPostmaster)
InitializeMaxBackends();
+ /*
+ * If data_encryption is set because of command line argument, do the
+ * setup now. (If set by postmaster, postmaster should have performed the
+ * setup.)
+ *
+ * This should only be useful for the bootstrap process. Anyone else
+ * initializes the encryption via ReadControlFile().
+ */
+ if (data_encrypted && MyAuxProcType == BootstrapProcess)
+ {
+ Assert(!IsUnderPostmaster);
+ setup_encryption(true, NULL);
+ }
+
BaseInit();
/*
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index 5015e5b3b6..4589a2c2f4 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -627,7 +627,12 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
*
* We don't need to copy subdirectories
*/
- copydir(srcpath, dstpath, false);
+ {
+ RelFileNode fromNode = {srctablespace, src_dboid, InvalidOid};
+ RelFileNode toNode = {dsttablespace, dboid, InvalidOid};
+
+ copydir(srcpath, dstpath, &fromNode, &toNode);
+ }
/* Record the filesystem change in XLOG */
{
@@ -1256,7 +1261,12 @@ movedb(const char *dbname, const char *tblspcname)
/*
* Copy files from the old tablespace to the new one
*/
- copydir(src_dbpath, dst_dbpath, false);
+ {
+ RelFileNode fromNode = {src_tblspcoid, db_id, InvalidOid};
+ RelFileNode toNode = {dst_tblspcoid, db_id, InvalidOid};
+
+ copydir(src_dbpath, dst_dbpath, &fromNode, &toNode);
+ }
/*
* Record the filesystem change in XLOG
@@ -2119,7 +2129,12 @@ dbase_redo(XLogReaderState *record)
*
* We don't need to copy subdirectories
*/
- copydir(src_path, dst_path, false);
+ {
+ RelFileNode fromNode = {xlrec->src_tablespace_id, xlrec->src_db_id, InvalidOid};
+ RelFileNode toNode = {xlrec->tablespace_id, xlrec->db_id, InvalidOid};
+
+ copydir(src_path, dst_path, &fromNode, &toNode);
+ }
}
else if (info == XLOG_DBASE_DROP)
{
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 51ff2cfa65..baedb4e95e 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3878,6 +3878,15 @@ pgstat_get_wait_io(WaitEventIO w)
case WAIT_EVENT_DSM_FILL_ZERO_WRITE:
event_name = "DSMFillZeroWrite";
break;
+ case WAIT_EVENT_KDF_FILE_READ:
+ event_name = "KDFFileRead";
+ break;
+ case WAIT_EVENT_KDF_FILE_SYNC:
+ event_name = "KDFFileSync";
+ break;
+ case WAIT_EVENT_KDF_FILE_WRITE:
+ event_name = "KDFFileWrite";
+ break;
case WAIT_EVENT_LOCK_FILE_ADDTODATADIR_READ:
event_name = "LockFileAddToDataDirRead";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 60d29a2089..fff0c67411 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -117,6 +117,7 @@
#include "postmaster/syslogger.h"
#include "replication/logicallauncher.h"
#include "replication/walsender.h"
+#include "storage/encryption.h"
#include "storage/fd.h"
#include "storage/ipc.h"
#include "storage/pg_shmem.h"
@@ -669,7 +670,7 @@ PostmasterMain(int argc, char *argv[])
* tcop/postgres.c (the option sets should not conflict) and with the
* common help() function in main/main.c.
*/
- while ((opt = getopt(argc, argv, "B:bc:C:D:d:EeFf:h:ijk:lN:nOo:Pp:r:S:sTt:W:-:")) != -1)
+ while ((opt = getopt(argc, argv, "B:bc:C:D:d:EeFf:h:ijK:k:lN:nOo:Pp:r:S:sTt:W:-:")) != -1)
{
switch (opt)
{
@@ -727,6 +728,12 @@ PostmasterMain(int argc, char *argv[])
/* only used by interactive backend */
break;
+#ifdef USE_ENCRYPTION
+ case 'K':
+ encryption_key_command = strdup(optarg);
+ break;
+#endif /* USE_ENCRYPTION */
+
case 'k':
SetConfigOption("unix_socket_directories", optarg, PGC_POSTMASTER, PGC_S_ARGV);
break;
@@ -1234,6 +1241,11 @@ PostmasterMain(int argc, char *argv[])
/*
* Set up shared memory and semaphores.
+ *
+ * This includes call of setup_encryption() as soon as we realize that
+ * "data_encrypted" field of the control file is set. The encryption must
+ * be initialized at the point so that "encryption_verification" field of
+ * the control file can be checked.
*/
reset_shared(PostPortNumber);
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index 36dcb28754..ba047f43b1 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -1109,7 +1109,7 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
/* Exclude all forks for unlogged tables except the init fork */
if (isDbDir &&
parse_filename_for_nontemp_relation(de->d_name, &relOidChars,
- &relForkNum))
+ &relForkNum, NULL))
{
/* Never exclude init forks */
if (relForkNum != INIT_FORKNUM)
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 09c8b5a5b3..59743976c0 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -77,6 +77,7 @@
#include "replication/walsender.h"
#include "replication/walsender_private.h"
#include "storage/condition_variable.h"
+#include "storage/encryption.h"
#include "storage/fd.h"
#include "storage/ipc.h"
#include "storage/pmsignal.h"
@@ -256,7 +257,9 @@ static void LagTrackerWrite(XLogRecPtr lsn, TimestampTz local_flush_time);
static TimeOffset LagTrackerRead(int head, XLogRecPtr lsn, TimestampTz now);
static bool TransactionIdInRecentPast(TransactionId xid, uint32 epoch);
-static void XLogRead(char *buf, XLogRecPtr startptr, Size count);
+static int XLogReadBuffer(char *buf, int nbytes, int startoff);
+static void XLogRead(char *buf, XLogRecPtr startptr, Size count,
+ bool decrypt);
/* Initialize walsender process before entering the main command loop */
@@ -787,7 +790,7 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req
count = flushptr - targetPagePtr; /* part of the page available */
/* now actually read the data, we know it's there */
- XLogRead(cur_page, targetPagePtr, XLOG_BLCKSZ);
+ XLogRead(cur_page, targetPagePtr, XLOG_BLCKSZ, true);
return count;
}
@@ -2346,19 +2349,53 @@ WalSndKill(int code, Datum arg)
SpinLockRelease(&walsnd->mutex);
}
+static int
+XLogReadBuffer(char *buf, int nbytes, int startoff)
+{
+ int readbytes;
+
+ /* Need to seek in the file? */
+ if (sendOff != startoff)
+ {
+ if (lseek(sendFile, (off_t) startoff, SEEK_SET) < 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not seek in log segment %s to offset %u: %m",
+ XLogFileNameP(curFileTimeLine, sendSegNo),
+ startoff)));
+ sendOff = startoff;
+ }
+
+ pgstat_report_wait_start(WAIT_EVENT_WAL_READ);
+ readbytes = read(sendFile, buf, nbytes);
+ pgstat_report_wait_end();
+ if (readbytes <= 0)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from log segment %s, offset %u, length %lu: %m",
+ XLogFileNameP(curFileTimeLine, sendSegNo),
+ sendOff, (unsigned long) nbytes)));
+ }
+
+ return readbytes;
+}
+
/*
- * Read 'count' bytes from WAL into 'buf', starting at location 'startptr'
+ * Read 'count' bytes from WAL into 'buf', starting at location
+ * 'startptr'. Decrypt the data if it's encrypted and if caller wants it
+ * decrypted.
*
- * XXX probably this should be improved to suck data directly from the
- * WAL buffers when possible.
+ * XXX probably this should be improved to fetch data directly from the WAL
+ * buffers when possible.
*
* Will open, and keep open, one WAL segment stored in the global file
- * descriptor sendFile. This means if XLogRead is used once, there will
- * always be one descriptor left open until the process ends, but never
- * more than one.
+ * descriptor sendFile. This means if XLogRead is used once, there will always
+ * be one descriptor left open until the process ends, but never more than
+ * one.
*/
static void
-XLogRead(char *buf, XLogRecPtr startptr, Size count)
+XLogRead(char *buf, XLogRecPtr startptr, Size count, bool decrypt)
{
char *p;
XLogRecPtr recptr;
@@ -2448,42 +2485,87 @@ retry:
sendOff = 0;
}
- /* Need to seek in the file? */
- if (sendOff != startoff)
+ /* Caller should not request decryption of unencrypted data. */
+ Assert(!(decrypt && !data_encrypted));
+
+ if (data_encrypted && decrypt)
{
- if (lseek(sendFile, (off_t) startoff, SEEK_SET) < 0)
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not seek in log segment %s to offset %u: %m",
- XLogFileNameP(curFileTimeLine, sendSegNo),
- startoff)));
- sendOff = startoff;
- }
+ int pageoff = startoff % XLOG_BLCKSZ;
+ uint32 pagebase = startoff - pageoff;
+ int bufbytes,
+ bufend,
+ i;
+ char tweak[TWEAK_SIZE];
- /* How many bytes are within this segment? */
- if (nbytes > (wal_segment_size - startoff))
- segbytes = wal_segment_size - startoff;
- else
- segbytes = nbytes;
+ /*
+ * Only accept as much data as what can fit into the buffer.
+ */
+ if (nbytes > (ENCRYPT_BUF_XLOG_SIZE - pageoff))
+ bufbytes = ENCRYPT_BUF_XLOG_SIZE - pageoff;
+ else
+ bufbytes = nbytes;
+ bufend = pageoff + bufbytes;
- pgstat_report_wait_start(WAIT_EVENT_WAL_READ);
- readbytes = read(sendFile, p, segbytes);
- pgstat_report_wait_end();
- if (readbytes < 0)
- {
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not read from log segment %s, offset %u, length %zu: %m",
- XLogFileNameP(curFileTimeLine, sendSegNo),
- sendOff, (Size) segbytes)));
+ /*
+ * Read the data, including the leading part of the page which
+ * caller is not interested in. The tweak we passed to
+ * encrypt_block() for encryption was for the beginning of the
+ * block, so it'd be hard to start decryption anywhere else.
+ */
+ readbytes = 0;
+ while (readbytes < bufend)
+ readbytes += XLogReadBuffer(encrypt_buf_xlog + readbytes,
+ bufend - readbytes,
+ pagebase + readbytes);
+
+ /*
+ * Decrypt the data one page at a time (the tweak is only valid
+ * for particular page).
+ */
+ for (i = 0; i < readbytes; i += XLOG_BLCKSZ)
+ {
+ Size nencrypt;
+
+ XLogEncryptionTweak(tweak,
+ curFileTimeLine,
+ sendSegNo,
+ pagebase + i);
+
+ /*
+ * If the last page is not complete, only decrypt the used
+ * part.
+ */
+ if ((bufend - i) < XLOG_BLCKSZ)
+ nencrypt = bufend - i;
+ else
+ nencrypt = XLOG_BLCKSZ;
+
+ decrypt_block(encrypt_buf_xlog + i,
+ encrypt_buf_xlog + i,
+ nencrypt,
+ tweak,
+ true);
+ }
+
+ /*
+ * Caller does not care that we possibly had to read pageoff bytes
+ * in addition (because we cannot decrypt trailing part of the
+ * page alone). This overhead must not affect the accounting.
+ */
+ readbytes = bufbytes;
+
+ /* Copy the data to the output buffer. */
+ memcpy(p, encrypt_buf_xlog + pageoff, bufbytes);
}
- else if (readbytes == 0)
+ else
{
- ereport(ERROR,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("could not read from log segment %s, offset %u: read %d of %zu",
- XLogFileNameP(curFileTimeLine, sendSegNo),
- sendOff, readbytes, (Size) segbytes)));
+ /* How many bytes are within this segment? */
+ if (nbytes > (wal_segment_size - startoff))
+ segbytes = wal_segment_size - startoff;
+ else
+ segbytes = nbytes;
+
+ readbytes = XLogReadBuffer(p, segbytes, startoff);
}
/* Update state for read */
@@ -2762,7 +2844,7 @@ XLogSendPhysical(void)
* calls.
*/
enlargeStringInfo(&output_message, nbytes);
- XLogRead(&output_message.data[output_message.len], startptr, nbytes);
+ XLogRead(&output_message.data[output_message.len], startptr, nbytes, false);
output_message.len += nbytes;
output_message.data[output_message.len] = '\0';
diff --git a/src/backend/storage/file/Makefile b/src/backend/storage/file/Makefile
index ca6a0e4f7d..9f277755fc 100644
--- a/src/backend/storage/file/Makefile
+++ b/src/backend/storage/file/Makefile
@@ -12,6 +12,6 @@ subdir = src/backend/storage/file
top_builddir = ../../../..
include $(top_builddir)/src/Makefile.global
-OBJS = fd.o buffile.o copydir.o reinit.o sharedfileset.o
+OBJS = fd.o buffile.o copydir.o reinit.o sharedfileset.o encryption.o
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/file/README.encryption b/src/backend/storage/file/README.encryption
new file mode 100644
index 0000000000..b98cb0debd
--- /dev/null
+++ b/src/backend/storage/file/README.encryption
@@ -0,0 +1,198 @@
+src/backend/storage/file/README.encryption
+
+Transparent Cluster Encryption
+==============================
+
+When creating a new cluster (instance), user can choose to have his data
+encrypted on disk. If this feature is active, data is encrypted before it's
+written to disk and decrypted after it has been read, however the data is
+unencrypted in memory (data-at-rest encryption). The following characteristics
+should be considered by anyone who is interested to enable this feature:
+
+ 1. The encryption is transparent from application's point of view.
+
+ 2. A single key is used to encrypt the whole cluster.
+
+The full instance encryption feature helps to ensure data confidentiality,
+especially when user cannot rely on confidentiality on filesystem level. On
+the other hand, it does not ensure data integrity, i.e. does not help to
+detect whether an adversary wrote his cipher data to the disk. The block
+cipher methods generally do not protect data integrity, and it'd probably be
+hard anyway because we encrypt Postgres data pages (typically 8 kB) separate
+from each other. If the attacker only changes part of the page, this can be
+detected if data checksums feature is enabled. And specifically for XLOG, each
+XLOG record has its own checksum.
+
+Since the data is stored on a disk, we naturally base our approach on "Disk
+encryption theory" [1]. For each kind of file we use the AES cipher in the
+appropriate mode of operation. The AES cipher itself encrypts / decrypts
+individual blocks ("encryption blocks") of 16 bytes (128 bits), while the mode
+of operation defines the rules how to apply the cipher to data which spans
+multiple encryption blocks.
+
+Relations
+---------
+
+AES cipher in CBC mode [2] is used to encrypt relation files, one relation
+block (page) at a time. The important characteristic of this mode is that, if
+a single encryption block changes, the next run of the encryption produces an
+encrypted page where not only that block, but also all the following blocks
+are encrypted differently. Furthermore, as the Postgres page starts with LSN,
+even if only the last encryption block of the page changes, the whole cipher
+text of the page will be different after the next encryption.
+
+This propagation of changes across encryption blocks makes it harder for
+adversary to deduce what happens in the database. Without the propagation he
+might be able to see data changes with higher granularity. For example, if
+only the beginning of index page changes, it gives some information on the
+keys inserted.
+
+On the other hand, the same change propagation can cause problems if only a
+part of a new version of encrypted page is written to disk (torn page
+write). If such a page is decrypted during crash recovery, part of the plain
+text can become garbage. This can affect even those parts of the page that
+haven't been modified recently, and therefore XLOG records to fix those parts
+are not guaranteed to be available. Therefore, if the cluster encryption is
+enabled, full_page_writes configuration variable must be set too, otherwise
+the server refuses to start. With this setting, the damage described here can
+be fixed by retrieving the full-page-image (FPI) from XLOG and applying it.
+
+However, even if full_page_writes is set, Postgres can still avoid writing FPI
+XLOG record if only hint bit(s) changed on a page, and if wal_log_hints
+configuration variable is not set. The problem with encryption is that even a
+single bit change is propagated to all the following encryption blocks during
+the next encryption of the page. Again, if only part of the changed cipher
+text is written, the remaining portion of the page becomes garbage after
+decryption, and we might not be able to fix it during recovery unless we have
+the FPI in the XLOG. So if encryption is enabled, Postgres behaves as if
+wal_log_hints was always set.
+
+XLOG
+----
+
+The specific problem of XLOG is that record must not be changed once it has
+been flushed to disk. However if we used a block cipher, and if a new XLOG
+record started in an encryption block in which the previous record ends, that
+encryption block would become completely different after the next run of
+encryption. Torn write of such a block (e.g. if memory page boundary crosses
+the encryption block) is likely to make decryption produce garbage, which will
+also appear in the already-flushed record.
+
+Therefore we encrypt XLOG using a stream cipher, or rather block cipher in
+stream mode of operation. Stream cipher uses XOR operation to combine a "key
+stream" with the input stream, and it does not matter if the length of the
+input stream is aligned to any value. In particular, the CTR mode [3] was
+chosen because it allows for both read and write operations to be
+parallelized.
+
+The XLOG is encrypted / decrypted one XLOG page (typically 8 kB) at a time, so
+that we can access the pages independently from each other. Thus, instead of
+being a single long stream, the XLOG is actually a sequence of short streams,
+each of which spans one page.
+
+Since stream ciphers are susceptible to "reused key attack" [4], we must
+ensure that the unused part of the last XLOG page (filled with zeroes) is
+never encrypted.
+
+Temporary files
+---------------
+
+BufFileWrite() and BufFileRead() functions (see buffile.c) hide the encryption
+/ decryption from caller. The encryption / decryption processes one buffer at
+a time so that the buffers can be retrieved independent from each other.
+
+If the encryption is enabled, the following requirements need to be taken into
+account:
+
+1. The file buffer cannot be positioned at arbitrary offset of the file. If
+the encryption routine starts at some position of the file, decryption must
+not start elsewhere because there's no way to determine which initialization
+vector was used internally for the corresponding encryption blocks (of 16
+bytes) during encryption. It makes sense to position the buffer at file offset
+that is whole multiple of buffer size.
+
+2. In general, the useful (written) data does not fill whole multiple of
+encryption blocks, but we must write the whole blocks for decryption to
+succeed. This implies that we need to fill the unused part of the last block
+with zeroes and also remember the amount of useful bytes in the segment file,
+which excludes the padding. (In fact we align the segment file size to file
+buffers instead of encryption blocks, which probably makes the implementation
+a bit simpler.)
+
+Stream cipher might seem like a solution of the padding problem, but we cannot
+use it here because parts of the temporary file can be rewritten. That would
+expose the temporary file to "reused key attack" [4].
+
+Auxiliary files
+---------------
+
+To store other kinds of data encrypted than the ones above, developers are
+advised to use BufFileWriteTransient() and BufFileReadTransient() functions
+(also located in buffile.c). These are especially useful if some data
+structure should be written to an encrypted file and user does not want to
+care whether the next write position is at encryption block boundary. In case
+it is not, BufFileWriteTransient() will read the encryption block the
+structure falls into, decrypt it, add the new data to it and decrypt and write
+it back to the file. Likewise, BufFileReadTransient() ensures that only the
+whole encryption blocks are read from disk and decrypted, whether the next
+read position is at block boundary or not.
+
+Serialization of data changes during logical decoding (reorderbuffer.c) is the
+typical use case for this API.
+
+Initialization vector (IV), encryption tweak
+--------------------------------------------
+
+Besides the input data and the key, both block and stream cipher used for the
+Postgres cluster encryption require an initialization vector. It should make
+analysis of the encrypted data more difficult. Also according to [1], the same
+data should be encrypted differently if located elsewhere on the disk. Term
+"encryption tweak" is more common in the context of disk encryption.
+
+When encrypting relations, each encryption unit (i.e. page) has an unique
+tweak, which consists of relfilenode, buffer number and fork number. As a
+consequence, we need to decrypt page using the existing tweak and encrypt it
+with a new tweak if copying it from one relation to another. Typically this
+happens when a new database is being created from template database, or during
+pg_upgrade. (Therefore the encrypted cluster does not support pg_upgrade with
+the --link option.)
+
+XLOG encryption tweak consists of timeline, segment number and offset at which
+the XLOG page starts in the segment. The "re-encryption" takes place when XLOG
+page is copied from one timeline to another, typicially at the end of
+recovery.
+
+As for temporary files, PID of the owning backend, file / fileset number and
+block number (where block is of the same size as a relation page) within the
+file provide sufficient uniqueness, so we use these to generate the tweak.
+
+Auxiliary files can be closed and reopened by another backend, so there's no
+PID strictly associated with them. Therefore we generate the tweak by hashing
+the file path, and appending block number to the hash.
+
+Replication
+-----------
+
+During streaming replication, the walsender process sends the XLOG encrypted
+and walreceiver just writes it. Decryption is performed before the slave
+cluster tries to apply the changes. That implies that the same encryption key
+must be used both master and slave. If it should be possible someday to stream
+the XLOG unencrypted, one implication is that pg_basebackup should also
+receive the relation data unencrypted.
+
+As for logical replication, the only change introduced by this feature is that
+the XLOG has to be decrypted before the contained data changes can be
+decoded. The data changes are transferred to the subscribing database /
+cluster unencrypted, so both master and slave can use different encryption
+keys.
+
+References
+----------
+
+[1] https://en.wikipedia.org/wiki/Disk_encryption_theory
+
+[2] https://en.wikipedia.org/wiki/Disk_encryption_theory#Cipher-block_chaining_(CBC)
+
+[3] https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Counter_(CTR)
+
+[4] https://en.wikipedia.org/wiki/Stream_cipher_attacks#Reused_key_attack
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index e9fc11d162..1a5fdaf9e5 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -43,18 +43,24 @@
#include <unistd.h>
+#include "common/sha2.h"
#include "executor/instrument.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "storage/fd.h"
#include "storage/buffile.h"
#include "storage/buf_internals.h"
+#include "storage/encryption.h"
+#include "utils/datetime.h"
#include "utils/resowner.h"
/*
* We break BufFiles into gigabyte-sized segments, regardless of RELSEG_SIZE.
* The reason is that we'd like large BufFiles to be spread across multiple
* tablespaces when available.
+ *
+ * The number of useful bytes is appended to each segment of shared encrypted
+ * file, see BufFile.useful.
*/
#define MAX_PHYSICAL_FILESIZE 0x40000000
#define BUFFILE_SEG_SIZE (MAX_PHYSICAL_FILESIZE / BLCKSZ)
@@ -81,6 +87,27 @@ typedef struct BufFileCommon
bool append; /* should new data be appended to the end? */
+ /*
+ * If the file is encrypted, only the whole buffer can be loaded / dumped
+ * --- see BufFileLoadBuffer() for more info --- whether it's space is
+ * used up or not. Therefore we need to keep track of the actual on-disk
+ * size buffer of each component file, as it would be if there was no
+ * encryption.
+ *
+ * List would make coding simpler, however it would not be good for
+ * performance. Random access is important here.
+ */
+ off_t *useful;
+
+ /*
+ * The "useful" array may need to be expanded independent from
+ * extendBufFile() (i.e. earlier than the buffer gets dumped), so store
+ * the number of elements separate from numFiles.
+ *
+ * Always 1 for TransientBufFile.
+ */
+ int nuseful;
+
PGAlignedBlock buffer;
} BufFileCommon;
@@ -97,12 +124,26 @@ struct BufFile
/* all files except the last have length exactly MAX_PHYSICAL_FILESIZE */
File *files; /* palloc'd array with numFiles entries */
+ /*
+ * Segment number is used to compute encryption tweak so we must remember
+ * the original numbers of segments if the file is encrypted and if it was
+ * passed as target to BufFileAppend() at least once. If this field is
+ * NULL, ->curFile is used to compute the tweak.
+ */
+ off_t *segnos;
+
bool isInterXact; /* keep open over transactions? */
SharedFileSet *fileset; /* space for segment files if shared */
const char *name; /* name of this BufFile if shared */
/*
+ * Per-PID identifier if the file is encrypted and not shared. Used for
+ * tweak computation.
+ */
+ uint32 number;
+
+ /*
* resowner is the ResourceOwner to use for underlying temp files. (We
* don't need to remember the memory context we're using explicitly,
* because after creation we only repalloc our arrays larger.)
@@ -132,6 +173,9 @@ static void BufFileDumpBuffer(BufFile *file);
static int BufFileFlush(BufFile *file);
static File MakeNewSharedSegment(BufFile *file, int segment);
+static void BufFileTweak(char *tweak, BufFileCommon *file, bool is_transient);
+static void ensureUsefulArraySize(BufFileCommon *file, int required);
+
static void BufFileLoadBufferTransient(TransientBufFile *file);
static void BufFileDumpBufferTransient(TransientBufFile *file);
@@ -159,6 +203,25 @@ makeBufFileCommon(int nfiles)
file->isInterXact = false;
file->resowner = CurrentResourceOwner;
+ if (data_encrypted)
+ {
+ fcommon->useful = (off_t *) palloc0(sizeof(off_t) * nfiles);
+ fcommon->nuseful = nfiles;
+
+ file->segnos = NULL;
+
+ /*
+ * The unused (trailing) part of the buffer should not contain
+ * undefined data: if we encrypt such a buffer and flush it to disk,
+ * the encrypted form of that "undefined part" can get zeroed due to
+ * seek and write beyond EOF. If such a buffer gets loaded and
+ * decrypted, the change of the undefined part to zeroes can affect
+ * the valid part if it does not end at block boundary. By setting the
+ * whole buffer to zeroes we ensure that the unused part of the buffer
+ * always contains zeroes.
+ */
+ MemSet(fcommon->buffer.data, 0, BLCKSZ);
+ }
return file;
}
@@ -204,6 +267,19 @@ extendBufFile(BufFile *file)
file->files = (File *) repalloc(file->files,
(file->numFiles + 1) * sizeof(File));
+
+ if (data_encrypted)
+ {
+ ensureUsefulArraySize(&file->common, file->numFiles + 1);
+
+ if (file->segnos)
+ {
+ file->segnos = (off_t *) repalloc(file->segnos,
+ (file->numFiles + 1) * sizeof(off_t));
+ file->segnos[file->numFiles] = file->numFiles;
+ }
+ }
+
file->files[file->numFiles] = pfile;
file->numFiles++;
}
@@ -226,10 +302,16 @@ BufFileCreateTemp(bool interXact)
BufFile *file;
File pfile;
+ static uint32 counter_temp = 0;
+
pfile = OpenTemporaryFile(interXact);
Assert(pfile >= 0);
file = makeBufFile(pfile);
+
+ file->number = counter_temp;
+ counter_temp = (counter_temp + 1) % INT_MAX;
+
file->isInterXact = interXact;
return file;
@@ -333,6 +415,7 @@ BufFileOpenShared(SharedFileSet *fileset, const char *name)
files[nfiles] = SharedFileSetOpen(fileset, segment_name);
if (files[nfiles] <= 0)
break;
+
++nfiles;
CHECK_FOR_INTERRUPTS();
@@ -349,7 +432,51 @@ BufFileOpenShared(SharedFileSet *fileset, const char *name)
segment_name, name)));
file = makeBufFileCommon(nfiles);
+
+ /*
+ * Shared encrypted segment should, at its end, contain information on the
+ * number of useful bytes in the last buffer.
+ */
+ if (data_encrypted)
+ {
+ off_t pos;
+ int i;
+
+ for (i = 0; i < nfiles; i++)
+ {
+ int nbytes;
+ File segment = files[i];
+
+ pos = FileSize(segment) - sizeof(off_t);
+
+ /*
+ * The word must immediately follow the last buffer of the
+ * segment.
+ */
+ if (pos <= 0 || pos % BLCKSZ != 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not find padding info in BufFile \"%s\": %m",
+ name)));
+
+ nbytes = FileRead(segment, (char *) &file->common.useful[i],
+ sizeof(off_t), pos, WAIT_EVENT_BUFFILE_READ);
+ if (nbytes != sizeof(off_t))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read padding info from BufFile \"%s\": %m",
+ name)));
+ Assert(file->common.useful[i] > 0);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+ }
+
file->files = files;
+
+ if (data_encrypted)
+ file->common.nuseful = nfiles;
+
file->common.readOnly = true; /* Can't write to files opened this way */
file->fileset = fileset;
file->name = pstrdup(name);
@@ -428,6 +555,14 @@ BufFileClose(BufFile *file)
FileClose(file->files[i]);
/* release the buffer space */
pfree(file->files);
+
+ if (data_encrypted)
+ {
+ if (file->segnos)
+ pfree(file->segnos);
+ pfree(file->common.useful);
+ }
+
pfree(file);
}
@@ -444,6 +579,14 @@ BufFileLoadBuffer(BufFile *file)
File thisfile;
/*
+ * Only whole multiple of ENCRYPTION_BLOCK can be encrypted / decrypted,
+ * see comments in BufFileDumpBuffer().
+ */
+ Assert((file->common.curOffset % BLCKSZ == 0 &&
+ file->common.curOffset % ENCRYPTION_BLOCK == 0) ||
+ !data_encrypted);
+
+ /*
* Advance to next component file if necessary and possible.
*/
if (file->common.curOffset >= MAX_PHYSICAL_FILESIZE &&
@@ -454,6 +597,20 @@ BufFileLoadBuffer(BufFile *file)
}
/*
+ * See makeBufFileCommon().
+ *
+ * Actually here we only handle the case of FileRead() returning zero
+ * bytes below. In contrast, if the buffer contains any data but it's not
+ * full, it should already have the trailing zeroes (encrypted) on disk.
+ * And as the encrypted buffer is always loaded in its entirety (i.e. EOF
+ * should only appear at buffer boundary if the data is encrypted), all
+ * unused bytes of the buffer should eventually be zeroes after
+ * decryption.
+ */
+ if (data_encrypted)
+ MemSet(file->common.buffer.data, 0, BLCKSZ);
+
+ /*
* Read whatever we can get, up to a full bufferload.
*/
thisfile = file->files[file->common.curFile];
@@ -466,6 +623,54 @@ BufFileLoadBuffer(BufFile *file)
file->common.nbytes = 0;
/* we choose not to advance curOffset here */
+ if (data_encrypted && file->common.nbytes > 0)
+ {
+ char tweak[TWEAK_SIZE];
+ int nbytes = file->common.nbytes;
+
+ /*
+ * The encrypted component file can only consist of whole number of
+ * our encryption units. (Only the whole buffers are dumped / loaded.)
+ * The only exception is that we're at the end of segment file and
+ * found the word indicating the number of useful bytes in the
+ * segment. This can only happen for shared file.
+ */
+ if (nbytes % BLCKSZ != 0)
+ {
+ Assert(nbytes == sizeof(off_t) && file->fileset != NULL);
+
+ /*
+ * This metadata his hidden to caller, so all he needs to know
+ * that there's no real data at the end of the file.
+ */
+ file->common.nbytes = 0;
+ return;
+ }
+
+ /* Decrypt the whole block at once. */
+ BufFileTweak(tweak, &file->common, false);
+ decrypt_block(file->common.buffer.data,
+ file->common.buffer.data,
+ BLCKSZ,
+ tweak,
+ false);
+
+#ifdef USE_ASSERT_CHECKING
+
+ /*
+ * The unused part of the buffer which we've read from disk and
+ * decrypted should only contain zeroes, as explained in front of the
+ * MemSet() call.
+ */
+ {
+ int i;
+
+ for (i = file->common.nbytes; i < BLCKSZ; i++)
+ Assert(file->common.buffer.data[i] == 0);
+ }
+#endif /* USE_ASSERT_CHECKING */
+ }
+
if (file->common.nbytes > 0)
pgBufferUsage.temp_blks_read++;
}
@@ -476,6 +681,9 @@ BufFileLoadBuffer(BufFile *file)
* Dump buffer contents starting at curOffset.
* At call, should have dirty = true, nbytes > 0.
* On exit, dirty is cleared if successful write, and curOffset is advanced.
+ *
+ * XXX Consider separate function for encrypted buffer. (No loop is needed to
+ * dump the encrypted buffer.)
*/
static void
BufFileDumpBuffer(BufFile *file)
@@ -485,12 +693,24 @@ BufFileDumpBuffer(BufFile *file)
File thisfile;
/*
+ * See comments in BufFileLoadBuffer();
+ */
+ Assert((file->common.curOffset % BLCKSZ == 0 &&
+ file->common.curOffset % ENCRYPTION_BLOCK == 0) ||
+ !data_encrypted);
+
+ /*
+ * Caller's responsibility.
+ */
+ Assert(file->common.pos <= file->common.nbytes);
+
+ /*
* Unlike BufFileLoadBuffer, we must dump the whole buffer even if it
* crosses a component-file boundary; so we need a loop.
*/
while (wpos < file->common.nbytes)
{
- off_t availbytes;
+ char *write_ptr;
/*
* Advance to next component file if necessary and possible.
@@ -503,23 +723,71 @@ BufFileDumpBuffer(BufFile *file)
file->common.curOffset = 0L;
}
- /*
- * Determine how much we need to write into this file.
- */
- bytestowrite = file->common.nbytes - wpos;
- availbytes = MAX_PHYSICAL_FILESIZE - file->common.curOffset;
+ if (!data_encrypted)
+ {
+ off_t availbytes;
+
+ bytestowrite = file->common.nbytes - wpos;
+ availbytes = MAX_PHYSICAL_FILESIZE - file->common.curOffset;
- if ((off_t) bytestowrite > availbytes)
- bytestowrite = (int) availbytes;
+ if ((off_t) bytestowrite > availbytes)
+ bytestowrite = (int) availbytes;
+
+ write_ptr = file->common.buffer.data;
+ }
+ else
+ {
+ char tweak[TWEAK_SIZE];
+
+ /*
+ * This condition plus the alignment of curOffset to BLCKSZ
+ * (checked above) ensure that the encrypted buffer never crosses
+ * component file boundary.
+ */
+ StaticAssertStmt((MAX_PHYSICAL_FILESIZE % BLCKSZ) == 0,
+ "BLCKSZ is not whole multiple of MAX_PHYSICAL_FILESIZE");
+
+ /*
+ * Encrypted data is dumped all at once.
+ *
+ * Here we don't have to check availbytes because --- according to
+ * the assertions above --- currOffset should be lower than
+ * MAX_PHYSICAL_FILESIZE by non-zero multiple of BLCKSZ.
+ */
+ bytestowrite = BLCKSZ;
+
+ /*
+ * The amount of data encrypted must be a multiple of
+ * ENCRYPTION_BLOCK. We meet this condition simply by encrypting
+ * the whole buffer.
+ *
+ * XXX Alternatively we could encrypt only as few encryption
+ * blocks that encompass file->common.nbyte bytes, but then we'd
+ * have to care how many blocks should be decrypted: decryption of
+ * the unencrypted trailing zeroes produces garbage, which can be
+ * a problem if lseek() created "holes" in the file. Such a hole
+ * should be read as a sequence of zeroes.
+ */
+ BufFileTweak(tweak, &file->common, false);
+
+ encrypt_block(file->common.buffer.data,
+ encrypt_buf.data,
+ BLCKSZ,
+ tweak,
+ false);
+ write_ptr = encrypt_buf.data;
+ }
thisfile = file->files[file->common.curFile];
bytestowrite = FileWrite(thisfile,
- file->common.buffer.data + wpos,
+ write_ptr + wpos,
bytestowrite,
file->common.curOffset,
WAIT_EVENT_BUFFILE_WRITE);
- if (bytestowrite <= 0)
+ if (bytestowrite <= 0 ||
+ (data_encrypted && bytestowrite != BLCKSZ))
return; /* failed to write */
+
file->common.curOffset += bytestowrite;
wpos += bytestowrite;
@@ -527,25 +795,115 @@ BufFileDumpBuffer(BufFile *file)
}
file->common.dirty = false;
- /*
- * At this point, curOffset has been advanced to the end of the buffer,
- * ie, its original value + nbytes. We need to make it point to the
- * logical file position, ie, original value + pos, in case that is less
- * (as could happen due to a small backwards seek in a dirty buffer!)
- */
- file->common.curOffset -= (file->common.nbytes - file->common.pos);
- if (file->common.curOffset < 0) /* handle possible segment crossing */
+ if (!data_encrypted)
{
- file->common.curFile--;
- Assert(file->common.curFile >= 0);
- file->common.curOffset += MAX_PHYSICAL_FILESIZE;
+ /*
+ * At this point, curOffset has been advanced to the end of the
+ * buffer, ie, its original value + nbytes. We need to make it point
+ * to the logical file position, ie, original value + pos, in case
+ * that is less (as could happen due to a small backwards seek in a
+ * dirty buffer!)
+ */
+ file->common.curOffset -= (file->common.nbytes - file->common.pos);
+ if (file->common.curOffset < 0) /* handle possible segment crossing */
+ {
+ file->common.curFile--;
+ Assert(file->common.curFile >= 0);
+ file->common.curOffset += MAX_PHYSICAL_FILESIZE;
+ }
+
+ /*
+ * Now we can set the buffer empty without changing the logical
+ * position
+ */
+ file->common.pos = 0;
+ file->common.nbytes = 0;
}
+ else
+ {
+ /*
+ * curOffset should be at buffer boundary and buffer is the smallest
+ * I/O unit for encrypted data.
+ */
+ Assert(file->common.curOffset % BLCKSZ == 0);
- /*
- * Now we can set the buffer empty without changing the logical position
- */
- file->common.pos = 0;
- file->common.nbytes = 0;
+ /*
+ * The number of useful bytes needs to be written at the end of each
+ * encrypted segment of a shared file so that the other backends know
+ * how many bytes of the last buffer are useful.
+ */
+ if (file->fileset != NULL)
+ {
+ off_t useful;
+
+ /*
+ * nuseful may be increased earlier than numFiles but not later,
+ * so the corresponding entry should always exist in ->useful.
+ */
+ Assert(file->common.curFile < file->common.nuseful);
+
+ /*
+ * The number of useful bytes in the current segment file.
+ */
+ useful = file->common.useful[file->common.curFile];
+
+ /*
+ * Have we dumped the last buffer of the segment, i.e. the one
+ * that can contain padding?
+ */
+ if (file->common.curOffset >= useful)
+ {
+ int bytes_extra;
+
+ /*
+ * Write the number of useful bytes in the segment.
+ *
+ * Do not increase curOffset afterwards. Thus we ensure that
+ * the next buffer appended will overwrite the "useful" value
+ * just written, instead of being appended to it.
+ */
+ bytes_extra = FileWrite(file->files[file->common.curFile],
+ (char *) &useful,
+ sizeof(useful),
+ file->common.curOffset,
+ WAIT_EVENT_BUFFILE_WRITE);
+ if (bytes_extra != sizeof(useful))
+ return; /* failed to write */
+ }
+ }
+
+ if (file->common.pos >= BLCKSZ)
+ {
+ Assert(file->common.pos == BLCKSZ);
+
+ /*
+ * curOffset points to the beginning of the next buffer, so just
+ * reset pos and nbytes.
+ */
+ file->common.pos = 0;
+ file->common.nbytes = 0;
+
+ /* See makeBufFile() */
+ if (data_encrypted)
+ MemSet(file->common.buffer.data, 0, BLCKSZ);
+ }
+ else
+ {
+ /*
+ * Move curOffset to the beginning of the just-written buffer and
+ * preserve pos.
+ */
+ file->common.curOffset -= BLCKSZ;
+
+ /*
+ * At least pos bytes should be written even if the first change
+ * since now appears at pos == nbytes, but in fact the whole
+ * buffer will be written regardless pos. This is the price we pay
+ * for the choosing BLCKSZ as the I/O unit for encrypted data.
+ */
+ file->common.nbytes = BLCKSZ;
+ }
+ }
}
/*
@@ -676,9 +1034,43 @@ BufFileSeek(BufFile *file, int fileno, off_t offset, int whence)
return EOF;
/* Seek is OK! */
file->common.curFile = newFile;
- file->common.curOffset = newOffset;
- file->common.pos = 0;
- file->common.nbytes = 0;
+ if (!data_encrypted)
+ {
+ file->common.curOffset = newOffset;
+ file->common.pos = 0;
+ file->common.nbytes = 0;
+ }
+ else
+ {
+ /*
+ * Offset of an encrypted buffer must be a multiple of BLCKSZ.
+ */
+ file->common.pos = newOffset % BLCKSZ;
+ file->common.curOffset = newOffset - file->common.pos;
+
+ /*
+ * BufFileLoadBuffer() will set nbytes iff it can read something.
+ */
+ file->common.nbytes = 0;
+
+ /*
+ * Load and decrypt the existing part of the buffer.
+ */
+ BufFileLoadBuffer(file);
+ if (file->common.nbytes == 0)
+ {
+ /*
+ * The data requested is not in the file, but this is not an
+ * error.
+ */
+ return 0;
+ }
+
+ /*
+ * The whole buffer should have been loaded.
+ */
+ Assert(file->common.nbytes == BLCKSZ);
+ }
return 0;
}
@@ -709,6 +1101,123 @@ BufFileSeekBlock(BufFile *file, long blknum)
SEEK_SET);
}
+static void
+BufFileTweak(char *tweak, BufFileCommon *file, bool is_transient)
+{
+ off_t block;
+
+ /*
+ * The unused bytes should always be defined.
+ */
+ memset(tweak, 0, TWEAK_SIZE);
+
+ if (!is_transient)
+ {
+ BufFile *tmpfile = (BufFile *) file;
+ pid_t pid;
+ uint32 number;
+ int curFile;
+
+ if (tmpfile->fileset)
+ {
+ pid = tmpfile->fileset->creator_pid;
+ number = tmpfile->fileset->number;
+ }
+ else
+ {
+ pid = MyProcPid;
+ number = tmpfile->number;
+ }
+
+ curFile = file->curFile;
+
+ /*
+ * If the file was produced by BufFileAppend(), we need the original
+ * curFile, as it was used originally for encryption.
+ */
+ if (tmpfile->segnos)
+ curFile = tmpfile->segnos[curFile];
+
+ block = curFile * BUFFILE_SEG_SIZE + file->curOffset / BLCKSZ;
+
+ StaticAssertStmt(sizeof(pid) + sizeof(number) + sizeof(block) <=
+ TWEAK_SIZE,
+ "tweak components do not fit into TWEAK_SIZE");
+
+ /*
+ * The tweak consists of PID of the owning backend (the leader backend
+ * in the case of parallel query processing), number within the PID
+ * and block number.
+ *
+ * XXX Additional flag would be handy to distinguish local file from
+ * shared one. Since there's no more room within TWEAK_SIZE, should we
+ * use the highest bit in one of the existing components (preferably
+ * other than pid so that tweaks of different processes do not become
+ * identical)?
+ */
+ *((pid_t *) tweak) = pid;
+ tweak += sizeof(pid_t);
+ *((uint32 *) tweak) = number;
+ tweak += sizeof(number);
+ *((off_t *) tweak) = block;
+ }
+ else
+ {
+ TransientBufFile *transfile = (TransientBufFile *) file;
+ pg_sha256_ctx sha_ctx;
+ unsigned char sha[PG_SHA256_DIGEST_LENGTH];
+#define BUF_FILE_PATH_HASH_LEN 8
+
+ /*
+ * For transient file we can't use any field of TransientBufFile
+ * because this info gets lost if the file is closed and reopened.
+ * Hash of the file path string is an easy way to get "persistent"
+ * tweak value, however usual hashes do not fit into TWEAK_SIZE. The
+ * hash portion that we can store is actually even smaller because
+ * block number needs to be stored too. Even though we only use part
+ * of the hash, the tweak we finally use for data encryption /
+ * decryption should not be predictable because the tweak we compute
+ * here is further processed, see cbc_essi_preprocess_tweak().
+ */
+ pg_sha256_init(&sha_ctx);
+ pg_sha256_update(&sha_ctx,
+ (uint8 *) transfile->path,
+ strlen(transfile->path));
+ pg_sha256_final(&sha_ctx, sha);
+
+ StaticAssertStmt(BUF_FILE_PATH_HASH_LEN + sizeof(block) <= TWEAK_SIZE,
+ "tweak components do not fit into TWEAK_SIZE");
+ memcpy(tweak, sha, BUF_FILE_PATH_HASH_LEN);
+ tweak += BUF_FILE_PATH_HASH_LEN;
+ block = file->curOffset / BLCKSZ;
+ *((off_t *) tweak) = block;
+ }
+}
+
+/*
+ * Make sure that BufFile.useful array has the required size.
+ */
+static void
+ensureUsefulArraySize(BufFileCommon *file, int required)
+{
+ /*
+ * Does the array already have enough space?
+ */
+ if (required <= file->nuseful)
+ return;
+
+ /*
+ * It shouldn't be possible to jump beyond the end of the last segment,
+ * i.e. skip more than 1 segment.
+ */
+ Assert(file->nuseful + 1 == required);
+
+ file->useful = (off_t *)
+ repalloc(file->useful, required * sizeof(off_t));
+ file->useful[file->nuseful] = 0L;
+ file->nuseful++;
+}
+
#ifdef NOT_USED
/*
* BufFileTellBlock --- block-oriented tell
@@ -740,14 +1249,32 @@ BufFileSize(BufFile *file)
Assert(file->fileset != NULL);
- /* Get the size of the last physical file. */
- lastFileSize = FileSize(file->files[file->numFiles - 1]);
- if (lastFileSize < 0)
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not determine size of temporary file \"%s\" from BufFile \"%s\": %m",
- FilePathName(file->files[file->numFiles - 1]),
- file->name)));
+ if (data_encrypted)
+ {
+ /*
+ * "useful" should be initialized even for shared file, see
+ * BufFileOpenShared().
+ */
+ Assert(file->common.useful != NULL &&
+ file->common.nuseful >= file->numFiles);
+
+ /*
+ * The number of useful bytes in the segment is what caller is
+ * interested in.
+ */
+ lastFileSize = file->common.useful[file->common.nuseful - 1];
+ }
+ else
+ {
+ /* Get the size of the last physical file. */
+ lastFileSize = FileSize(file->files[file->numFiles - 1]);
+ if (lastFileSize < 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not determine size of temporary file \"%s\" from BufFile \"%s\": %m",
+ FilePathName(file->files[file->numFiles - 1]),
+ file->name)));
+ }
return ((file->numFiles - 1) * (int64) MAX_PHYSICAL_FILESIZE) +
lastFileSize;
@@ -777,6 +1304,7 @@ BufFileAppend(BufFile *target, BufFile *source)
{
long startBlock = target->numFiles * BUFFILE_SEG_SIZE;
int newNumFiles = target->numFiles + source->numFiles;
+ int newNUseful = target->common.nuseful + source->common.nuseful;
int i;
Assert(target->fileset != NULL);
@@ -791,6 +1319,63 @@ BufFileAppend(BufFile *target, BufFile *source)
repalloc(target->files, sizeof(File) * newNumFiles);
for (i = target->numFiles; i < newNumFiles; i++)
target->files[i] = source->files[i - target->numFiles];
+
+ if (data_encrypted)
+ {
+ /*
+ * XXX As the typical use case is that parallel workers expose file to
+ * the leader, can we expect both target and source to have been
+ * exported, i.e. flushed? In such a case "nuseful" would have to be
+ * equal to "numFiles" for both input files and the code could get a
+ * bit simpler. It seems that at least source should be flushed, as
+ * source->readOnly is expected to be true above.
+ */
+ target->common.useful = (off_t *)
+ repalloc(target->common.useful, sizeof(off_t) * newNUseful);
+
+ for (i = target->common.nuseful; i < newNUseful; i++)
+ target->common.useful[i] = source->common.useful[i - target->common.nuseful];
+ target->common.nuseful = newNUseful;
+
+ /*
+ * File segments can appear at different position due to
+ * concatenation, so make sure we remember the original positions for
+ * the sake of encryption tweak.
+ */
+ if (target->segnos == NULL)
+ {
+ /*
+ * If the target does not have the array yet, allocate it for both
+ * target and source and initialize the target part.
+ */
+ target->segnos = (off_t *) palloc(newNumFiles * sizeof(off_t));
+ for (i = 0; i < target->numFiles; i++)
+ target->segnos[i] = i;
+ }
+ else
+ {
+ /*
+ * Use the existing target part and add space for the source part.
+ */
+ target->segnos = (off_t *) repalloc(target->segnos,
+ newNumFiles * sizeof(off_t));
+ }
+
+ /*
+ * The source segment number either equals to (0-based) index of the
+ * segment, or to an element of an already existing array.
+ */
+ for (i = target->numFiles; i < newNumFiles; i++)
+ {
+ off_t segno = i - target->numFiles;
+
+ if (source->segnos == NULL)
+ target->segnos[i] = segno;
+ else
+ target->segnos[i] = source->segnos[segno];
+ }
+ }
+
target->numFiles = newNumFiles;
return startBlock;
@@ -817,6 +1402,28 @@ BufFileOpenTransient(const char *path, int fileFlags)
/* Check whether user wants read or write access. */
readOnly = (fileFlags & O_WRONLY) == 0;
+ if (data_encrypted)
+ {
+ /*
+ * In the encryption case, even if user will only be allowed to write,
+ * internally we also need to read, see below.
+ */
+ fileFlags &= ~O_WRONLY;
+ fileFlags |= O_RDWR;
+
+ /*
+ * We can only emulate the append behavior by setting curOffset to
+ * file size because if the underlying file was opened in append mode,
+ * we could not rewrite the old value of file->common.useful[0] with
+ * data.
+ */
+ if (fileFlags & O_APPEND)
+ {
+ append = true;
+ fileFlags &= ~O_APPEND;
+ }
+ }
+
/*
* Append mode for read access is not useful, so don't bother implementing
* it.
@@ -849,6 +1456,8 @@ BufFileOpenTransient(const char *path, int fileFlags)
fcommon->readOnly = readOnly;
fcommon->append = append;
fcommon->curFile = 0;
+ fcommon->useful = (off_t *) palloc0(sizeof(off_t));
+ fcommon->nuseful = 1;
file->path = pstrdup(path);
file->fd = fd;
@@ -869,6 +1478,56 @@ BufFileOpenTransient(const char *path, int fileFlags)
else
fcommon->curOffset = 0L;
+ /*
+ * Encrypted transient file should, at its end, contain information on the
+ * number of useful bytes in the last buffer.
+ */
+ if (data_encrypted)
+ {
+ off_t pos = size;
+ int nbytes;
+
+ /* No metadata in an empty file. */
+ if (pos == 0)
+ return file;
+
+ pos -= sizeof(off_t);
+
+ /*
+ * The word must immediately follow the last buffer of the segment.
+ */
+ if (pos < 0 || pos % BLCKSZ != 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not find padding info in TransientBufFile \"%s\": %m",
+ path)));
+
+ errno = 0;
+ nbytes = pg_pread(file->fd,
+ (char *) &fcommon->useful[0],
+ sizeof(off_t),
+ pos);
+ if (nbytes != sizeof(off_t))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read padding info from TransientBufFile \"%s\": %m",
+ path)));
+ Assert(fcommon->useful[0] > 0);
+
+ if (fcommon->append)
+ {
+ off_t useful = fcommon->useful[0];
+
+ /*
+ * If new buffer should be added, make sure it will end up
+ * immediately after the last complete one, and also that the next
+ * write position follows the last valid byte.
+ */
+ fcommon->pos = useful % BLCKSZ;
+ fcommon->curOffset = useful - fcommon->pos;
+ }
+ }
+
return file;
}
@@ -900,6 +1559,8 @@ BufFileCloseTransient(TransientBufFile *file)
(errcode_for_file_access(),
errmsg("could not close file \"%s\": %m", file->path)));
+ if (data_encrypted)
+ pfree(file->common.useful);
pfree(file->path);
pfree(file);
}
@@ -916,6 +1577,9 @@ BufFileLoadBufferTransient(TransientBufFile *file)
Assert(!file->common.dirty);
Assert(file->common.pos == 0 && file->common.nbytes == 0);
+ /* See comments in BufFileLoadBuffer(). */
+ if (data_encrypted)
+ MemSet(file->common.buffer.data, 0, BLCKSZ);
retry:
/*
@@ -940,6 +1604,54 @@ retry:
return;
}
/* we choose not to advance offset here */
+
+ if (data_encrypted && file->common.nbytes > 0)
+ {
+ char tweak[TWEAK_SIZE];
+ int nbytes = file->common.nbytes;
+
+ /*
+ * The encrypted file can only consist of whole number of our
+ * encryption units. (Only the whole buffers are dumped / loaded.) The
+ * only exception is that we're at the end of segment file and found
+ * the word indicating the number of useful bytes in the segment. This
+ * can only happen for shared file.
+ */
+ if (nbytes % BLCKSZ != 0)
+ {
+ Assert(nbytes == sizeof(off_t));
+
+ /*
+ * This metadata his hidden to caller, so all he needs to know
+ * that there's no real data at the end of the file.
+ */
+ file->common.nbytes = 0;
+ return;
+ }
+
+ /* Decrypt the whole block at once. */
+ BufFileTweak(tweak, &file->common, true);
+ decrypt_block(file->common.buffer.data,
+ file->common.buffer.data,
+ BLCKSZ,
+ tweak,
+ false);
+
+#ifdef USE_ASSERT_CHECKING
+
+ /*
+ * The unused part of the buffer which we've read from disk and
+ * decrypted should only contain zeroes, as explained in front of the
+ * MemSet() call.
+ */
+ {
+ int i;
+
+ for (i = file->common.nbytes; i < BLCKSZ; i++)
+ Assert(file->common.buffer.data[i] == 0);
+ }
+#endif /* USE_ASSERT_CHECKING */
+ }
}
/*
@@ -948,7 +1660,9 @@ retry:
static void
BufFileDumpBufferTransient(TransientBufFile *file)
{
- int nwritten;
+ int bytestowrite,
+ nwritten;
+ char *write_ptr;
/* This function should only be needed during write access ... */
Assert(!file->common.readOnly);
@@ -957,12 +1671,33 @@ BufFileDumpBufferTransient(TransientBufFile *file)
Assert(file->common.dirty);
Assert(file->common.nbytes > 0);
+ if (!data_encrypted)
+ {
+ write_ptr = file->common.buffer.data;
+ bytestowrite = file->common.nbytes;
+ }
+ else
+ {
+ char tweak[TWEAK_SIZE];
+
+ /*
+ * Encrypt the whole buffer, see comments in BufFileDumpBuffer().
+ */
+ BufFileTweak(tweak, &file->common, true);
+ encrypt_block(file->common.buffer.data,
+ encrypt_buf.data,
+ BLCKSZ,
+ tweak,
+ false);
+ write_ptr = encrypt_buf.data;
+ bytestowrite = BLCKSZ;
+ }
retry:
errno = 0;
pgstat_report_wait_start(WAIT_EVENT_BUFFILE_WRITE);
nwritten = pg_pwrite(file->fd,
- file->common.buffer.data,
- file->common.nbytes,
+ write_ptr,
+ bytestowrite,
file->common.curOffset);
pgstat_report_wait_end();
@@ -981,6 +1716,43 @@ retry:
return; /* failed to write */
}
+ file->common.curOffset += nwritten;
+
+ if (data_encrypted)
+ {
+ off_t useful;
+
+ /*
+ * The number of useful bytes in file.
+ */
+ useful = file->common.useful[0];
+
+ /*
+ * Have we dumped the last buffer of the segment, i.e. the one that
+ * can contain padding?
+ */
+ if (file->common.curOffset >= useful)
+ {
+ int bytes_extra;
+
+ /*
+ * Write the number of useful bytes in the file
+ *
+ * Do not increase curOffset afterwards. Thus we ensure that the
+ * next buffer appended will overwrite the "useful" value just
+ * written, instead of being appended to it.
+ */
+ pgstat_report_wait_start(WAIT_EVENT_BUFFILE_WRITE);
+ bytes_extra = pg_pwrite(file->fd,
+ (char *) &useful,
+ sizeof(useful),
+ file->common.curOffset);
+ pgstat_report_wait_end();
+ if (bytes_extra != sizeof(useful))
+ return; /* failed to write */
+ }
+ }
+
file->common.dirty = false;
file->common.pos = 0;
@@ -1034,21 +1806,110 @@ BufFileReadCommon(BufFileCommon *file, void *ptr, size_t size,
{
if (file->pos >= file->nbytes)
{
- /* Try to load more data into buffer. */
- file->curOffset += file->pos;
- file->pos = 0;
+ /*
+ * Neither read nor write nor seek should leave pos greater than
+ * nbytes, regardless the data is encrypted or not. pos can only
+ * be greater if nbytes is zero --- this situation can be caused
+ * by BufFileSeek().
+ */
+ Assert(file->pos == file->nbytes || file->nbytes == 0);
+
+ /*
+ * The Assert() above implies that pos is a whole multiple of
+ * BLCKSZ, so curOffset has meet the same encryption-specific
+ * requirement too.
+ */
+ Assert(file->curOffset % BLCKSZ == 0 || !data_encrypted);
+
file->nbytes = 0;
+ /* Try to load more data into buffer. */
+ if (!data_encrypted || file->pos % BLCKSZ == 0)
+ {
+ file->curOffset += file->pos;
+ file->pos = 0;
- if (!is_transient)
- BufFileLoadBuffer((BufFile *) file);
+ if (!is_transient)
+ BufFileLoadBuffer((BufFile *) file);
+ else
+ BufFileLoadBufferTransient((TransientBufFile *) file);
+
+ if (file->nbytes <= 0)
+ break; /* no more data available */
+ }
else
- BufFileLoadBufferTransient((TransientBufFile *) file);
+ {
+ int nbytes_orig = file->nbytes;
+
+ /*
+ * Given that BLCKSZ is the I/O unit for encrypted data (see
+ * comments in BufFileDumpBuffer), we cannot add pos to
+ * curOffset because that would make it point outside block
+ * boundary. The only thing we can do is to reload the whole
+ * buffer and see if more data is eventually there than the
+ * previous load has fetched.
+ */
+ if (!is_transient)
+ BufFileLoadBuffer((BufFile *) file);
+ else
+ BufFileLoadBufferTransient((TransientBufFile *) file);
- if (file->nbytes <= 0)
- break; /* no more data available */
+ Assert(file->nbytes >= nbytes_orig);
+ if (file->nbytes == nbytes_orig)
+ break; /* no more data available */
+ }
}
nthistime = file->nbytes - file->pos;
+
+ /*
+ * The buffer can contain trailing zeroes because BLCKSZ is the I/O
+ * unit for encrypted data. These are not available for reading.
+ */
+ if (data_encrypted)
+ {
+ off_t useful = file->useful[file->curFile];
+
+ /*
+ * The criterion is whether the useful data ends within the
+ * currently loaded buffer.
+ */
+ if (useful < file->curOffset + BLCKSZ)
+ {
+ int avail;
+
+ /*
+ * Compute the number of bytes available in the current
+ * buffer.
+ */
+ avail = useful - file->curOffset;
+ Assert(avail >= 0);
+
+ /*
+ * An empty buffer can exist, e.g. after a seek to the end of
+ * the last component file.
+ */
+ if (avail == 0)
+ break;
+
+ /*
+ * Seek beyond the current EOF, which was not followed by
+ * write, could have resulted in position outside the useful
+ * data
+ */
+ if (file->pos > avail)
+ break;
+
+ nthistime = avail - file->pos;
+ Assert(nthistime >= 0);
+
+ /*
+ * Have we reached the end of the valid data?
+ */
+ if (nthistime == 0)
+ break;
+ }
+ }
+
if (nthistime > size)
nthistime = size;
Assert(nthistime > 0);
@@ -1070,8 +1931,7 @@ BufFileReadCommon(BufFileCommon *file, void *ptr, size_t size,
* Functionality needed by both BufFileWrite() and BufFileWriteTransient().
*/
static size_t
-BufFileWriteCommon(BufFileCommon *file, void *ptr, size_t size,
- bool is_transient)
+BufFileWriteCommon(BufFileCommon *file, void *ptr, size_t size, bool is_transient)
{
size_t nwritten = 0;
size_t nthistime;
@@ -1097,11 +1957,28 @@ BufFileWriteCommon(BufFileCommon *file, void *ptr, size_t size,
{
Assert(!is_transient);
- /* Hmm, went directly from reading to writing? */
+ /*
+ * Hmm, went directly from reading to writing?
+ *
+ * As pos should be exactly BLCKSZ, there is nothing special
+ * to do about data_encrypted, except for zeroing the buffer.
+ */
+ Assert(file->pos == BLCKSZ);
+
file->curOffset += file->pos;
file->pos = 0;
file->nbytes = 0;
+
+ /* See makeBufFile() */
+ if (data_encrypted)
+ MemSet(file->buffer.data, 0, BLCKSZ);
}
+
+ /*
+ * If curOffset changed above, it should still meet the assumption
+ * that buffer is the I/O unit for encrypted data.
+ */
+ Assert(file->curOffset % BLCKSZ == 0 || !data_encrypted);
}
nthistime = BLCKSZ - file->pos;
@@ -1115,6 +1992,82 @@ BufFileWriteCommon(BufFileCommon *file, void *ptr, size_t size,
file->pos += nthistime;
if (file->nbytes < file->pos)
file->nbytes = file->pos;
+
+ if (data_encrypted)
+ {
+ off_t new_offset;
+ int fileno;
+
+ if (!is_transient)
+ {
+ fileno = file->curFile;
+
+ /*
+ * curFile does not necessarily correspond to the offset: it
+ * can still have the initial value if BufFileSeek() skipped
+ * the previous file w/o dumping anything of it. While curFile
+ * will be fixed during the next dump, we need valid fileno
+ * now.
+ */
+ if (file->curOffset >= MAX_PHYSICAL_FILESIZE)
+ {
+ /*
+ * Even BufFileSeek() should not allow curOffset to become
+ * more than MAX_PHYSICAL_FILESIZE (if caller passes
+ * higher offset, curFile gets increased instead).
+ */
+ Assert(file->curOffset == MAX_PHYSICAL_FILESIZE);
+
+ fileno++;
+ }
+
+ /*
+ * fileno can now point to a segment that does not exist on
+ * disk yet.
+ */
+ ensureUsefulArraySize(file, fileno + 1);
+
+ /*
+ * Update the "useful offset" of the underlying component file
+ * if we've added any useful data.
+ */
+ new_offset = file->curOffset + file->pos;
+
+ /*
+ * Make sure the offset is relative to the correct component
+ * file. It should have been adjusted during sequential
+ * write, but if the buffer hasn't been dumped since
+ * BufFileSeek() had set curOffset to the end of the last
+ * segment, new_offset is now relative to the start of that
+ * *previously last* segment. Make sure it's relative to the
+ * new last (fileno) segment.
+ */
+ if (file->curOffset % MAX_PHYSICAL_FILESIZE == 0)
+ new_offset %= MAX_PHYSICAL_FILESIZE;
+
+ /*
+ * Adjust the number of useful bytes in the file if needed.
+ * This has to happen immediately, independent from
+ * BufFileDumpBuffer(), so that BufFileRead() works correctly
+ * anytime.
+ */
+ if (new_offset > file->useful[fileno])
+ file->useful[fileno] = new_offset;
+ }
+ else
+ {
+ /*
+ * Transient file is a single file on the disk, so the whole
+ * thing is much simpler.
+ */
+ fileno = 0;
+
+ new_offset = file->curOffset + file->pos;
+ if (new_offset > file->useful[fileno])
+ file->useful[fileno] = new_offset;
+ }
+ }
+
ptr = (void *) ((char *) ptr + nthistime);
size -= nthistime;
nwritten += nthistime;
diff --git a/src/backend/storage/file/copydir.c b/src/backend/storage/file/copydir.c
index 30f6200a86..b98e9c10d8 100644
--- a/src/backend/storage/file/copydir.c
+++ b/src/backend/storage/file/copydir.c
@@ -23,24 +23,29 @@
#include <sys/stat.h>
#include "storage/copydir.h"
+#include "storage/encryption.h"
#include "storage/fd.h"
+#include "storage/reinit.h"
#include "miscadmin.h"
#include "pgstat.h"
/*
* copydir: copy a directory
*
- * If recurse is false, subdirectories are ignored. Anything that's not
- * a directory or a regular file is ignored.
+ * RelFileNode values must specify tablespace and database oids for source
+ * and target to support re-encryption if necessary. relNode value in provided
+ * structs will be clobbered.
*/
void
-copydir(char *fromdir, char *todir, bool recurse)
+copydir(char *fromdir, char *todir, RelFileNode *fromNode, RelFileNode *toNode)
{
DIR *xldir;
struct dirent *xlde;
char fromfile[MAXPGPATH * 2];
char tofile[MAXPGPATH * 2];
+ Assert(!data_encrypted || (fromNode != NULL && toNode != NULL));
+
if (MakePGDirectory(todir) != 0)
ereport(ERROR,
(errcode_for_file_access(),
@@ -67,14 +72,32 @@ copydir(char *fromdir, char *todir, bool recurse)
(errcode_for_file_access(),
errmsg("could not stat file \"%s\": %m", fromfile)));
- if (S_ISDIR(fst.st_mode))
+ if (S_ISREG(fst.st_mode))
{
- /* recurse to handle subdirectories */
- if (recurse)
- copydir(fromfile, tofile, true);
+ int oidchars;
+ ForkNumber forkNum;
+ int segment;
+
+ /*
+ * For encrypted databases we need to reencrypt files with new
+ * tweaks.
+ */
+ if (data_encrypted &&
+ parse_filename_for_nontemp_relation(xlde->d_name,
+ &oidchars, &forkNum, &segment))
+ {
+ char oidbuf[OIDCHARS + 1];
+
+ memcpy(oidbuf, xlde->d_name, oidchars);
+ oidbuf[oidchars] = '\0';
+
+ /* We scribble over the provided RelFileNodes here */
+ fromNode->relNode = toNode->relNode = atol(oidbuf);
+ copy_file(fromfile, tofile, fromNode, toNode, forkNum, forkNum, segment);
+ }
+ else
+ copy_file(fromfile, tofile, NULL, NULL, 0, 0, 0);
}
- else if (S_ISREG(fst.st_mode))
- copy_file(fromfile, tofile);
}
FreeDir(xldir);
@@ -121,17 +144,22 @@ copydir(char *fromdir, char *todir, bool recurse)
}
/*
- * copy one file
+ * copy one file. If decryption and reencryption may be needed specify
+ * relfilenodes for source and target.
*/
void
-copy_file(char *fromfile, char *tofile)
+copy_file(char *fromfile, char *tofile, RelFileNode *fromNode,
+ RelFileNode *toNode, ForkNumber fromForkNum, ForkNumber toForkNum,
+ int segment)
{
char *buffer;
int srcfd;
int dstfd;
int nbytes;
+ int bytesread;
off_t offset;
off_t flush_offset;
+ BlockNumber blockNum = segment * RELSEG_SIZE;
/* Size of copy buffer (read and write requests) */
#define COPY_BUF_SIZE (8 * BLCKSZ)
@@ -186,15 +214,50 @@ copy_file(char *fromfile, char *tofile)
flush_offset = offset;
}
- pgstat_report_wait_start(WAIT_EVENT_COPY_FILE_READ);
- nbytes = read(srcfd, buffer, COPY_BUF_SIZE);
- pgstat_report_wait_end();
- if (nbytes < 0)
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not read file \"%s\": %m", fromfile)));
+ /*
+ * Try to read as much as we fit in the buffer so we can deal with
+ * complete blocks if we need to reencrypt.
+ */
+ nbytes = 0;
+ while (nbytes < COPY_BUF_SIZE)
+ {
+ pgstat_report_wait_start(WAIT_EVENT_COPY_FILE_READ);
+ bytesread = read(srcfd, buffer + nbytes, COPY_BUF_SIZE - nbytes);
+ pgstat_report_wait_end();
+ if (bytesread < 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read file \"%s\": %m", fromfile)));
+ nbytes += bytesread;
+ if (bytesread == 0)
+ break;
+ }
if (nbytes == 0)
break;
+
+ /*
+ * If the database is encrypted we need to decrypt the data here and
+ * reencrypt it to adjust the tweak values of blocks.
+ */
+ if (data_encrypted)
+ {
+ if (fromNode != NULL)
+ {
+ Assert(toNode != NULL);
+
+ /*
+ * There's no reason not to have whole number of pages read
+ * Computation of the number of blocks below relies on this
+ * fact.
+ */
+ Assert(nbytes % BLCKSZ == 0);
+
+ blockNum = ReencryptBlock(buffer, nbytes / BLCKSZ,
+ fromNode, toNode, fromForkNum,
+ toForkNum, blockNum);
+ }
+ }
+
errno = 0;
pgstat_report_wait_start(WAIT_EVENT_COPY_FILE_WRITE);
if ((int) write(dstfd, buffer, nbytes) != nbytes)
diff --git a/src/backend/storage/file/encryption.c b/src/backend/storage/file/encryption.c
new file mode 100644
index 0000000000..d776c2667f
--- /dev/null
+++ b/src/backend/storage/file/encryption.c
@@ -0,0 +1,874 @@
+/*-------------------------------------------------------------------------
+ *
+ * encryption.c
+ * This code handles encryption and decryption of data.
+ *
+ * Copyright (c) 20119, PostgreSQL Global Development Group
+ *
+ * See src/backend/storage/file/README.encryption for explanation of the
+ * design.
+ *
+ * IDENTIFICATION
+ * src/backend/storage/file/encryption.c
+ *
+ * NOTES
+ * This file is compiled as both front-end and backend code, so the
+ * FRONTEND macro must be used to distinguish the case if we need to
+ * report error or if server-defined variable / function seems useful.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include <unistd.h>
+
+#include "common/fe_memutils.h"
+#include "common/sha2.h"
+#include "common/string.h"
+#include "catalog/pg_control.h"
+#include "storage/encryption.h"
+
+#ifndef FRONTEND
+#include "pgstat.h"
+#include "storage/fd.h"
+#include "utils/memutils.h"
+#include "miscadmin.h"
+#include "port.h"
+#endif /* FRONTEND */
+
+#ifdef USE_ENCRYPTION
+#include <openssl/conf.h>
+#include <openssl/evp.h>
+#include <openssl/err.h>
+
+EVP_CIPHER_CTX *ctx_encrypt,
+ *ctx_decrypt,
+ *ctx_encrypt_stream,
+ *ctx_decrypt_stream;
+#endif /* USE_ENCRYPTION */
+
+const char *encryption_key_prefix = "encryption_key=";
+const char *encryption_pwd_prefix = "encryption_password=";
+
+/* Key to encrypt / decrypt data. */
+unsigned char encryption_key[ENCRYPTION_KEY_LENGTH];
+
+bool data_encrypted = false;
+
+char *encryption_key_command = NULL;
+
+PGAlignedBlock encrypt_buf;
+char *encrypt_buf_xlog = NULL;
+
+static void setup_encryption_internal(void);
+
+#ifdef USE_ENCRYPTION
+static void init_encryption_context(EVP_CIPHER_CTX **ctx_p, bool stream);
+static void evp_error(void);
+#endif /* USE_ENCRYPTION */
+
+/*
+ * Pointer to the KDF parameters.
+ *
+ * XXX Rename this and the write / read functions so they contain the
+ * 'keysetup' string?
+ */
+KDFParamsData *KDFParams = NULL;
+
+/*
+ * Initialize encryption subsystem for use. Must be called before any
+ * encryptable data is read from or written to data directory.
+ *
+ * data_dir should be passed if the current directory is different. It's
+ * needed to locate KDF file.
+ */
+void
+setup_encryption(bool bootstrap, char *data_dir)
+{
+ char *credentials;
+ bool is_key;
+ size_t len;
+
+ credentials = run_encryption_key_command(encryption_key_command,
+ &is_key,
+ &len);
+
+ /*
+ * Setup KDF if we need to derive the key from a password.
+ */
+ if (!is_key)
+ {
+ if (bootstrap)
+ {
+#ifndef FRONTEND
+ write_kdf_file();
+#endif /* FRONTEND */
+ }
+ else
+ read_kdf_file(data_dir);
+ }
+
+ setup_encryption_key(credentials, is_key, len);
+ pfree(credentials);
+ setup_encryption_internal();
+}
+
+/*
+ * Initialize KDFParamsData and write it to a file.
+ *
+ * This is very similar to WriteControlFile().
+ */
+#ifndef FRONTEND
+void
+write_kdf_file(void)
+{
+ KDFParamsPBKDF2 *params;
+ int i,
+ fd;
+
+ StaticAssertStmt(sizeof(KDFParamsData) <= KDF_PARAMS_FILE_SIZE,
+ "kdf file is too large for atomic disk writes");
+
+ /*
+ * The initialization should not be repeated.
+ */
+ Assert(KDFParams == NULL);
+
+ KDFParams = MemoryContextAllocZero(TopMemoryContext,
+ KDF_PARAMS_FILE_SIZE);
+ KDFParams->function = KDF_OPENSSL_PKCS5_PBKDF2_HMAC_SHA;
+ params = &KDFParams->data.pbkdf2;
+ params->niter = ENCRYPTION_KDF_NITER;
+ for (i = 0; i < ENCRYPTION_KDF_SALT_LEN; i++)
+ params->salt[i] = (unsigned char) random();
+
+ /* Contents are protected with a CRC */
+ INIT_CRC32C(KDFParams->crc);
+ COMP_CRC32C(KDFParams->crc,
+ (char *) KDFParams,
+ offsetof(KDFParamsData, crc));
+ FIN_CRC32C(KDFParams->crc);
+
+ fd = BasicOpenFile(KDF_PARAMS_FILE,
+ O_RDWR | O_CREAT | O_EXCL | PG_BINARY);
+ if (fd < 0)
+ ereport(PANIC,
+ (errcode_for_file_access(),
+ errmsg("could not create key setup file \"%s\": %m",
+ KDF_PARAMS_FILE)));
+
+ pgstat_report_wait_start(WAIT_EVENT_KDF_FILE_WRITE);
+ if (write(fd, KDFParams, KDF_PARAMS_FILE_SIZE) != KDF_PARAMS_FILE_SIZE)
+ {
+ /* if write didn't set errno, assume problem is no disk space */
+ if (errno == 0)
+ errno = ENOSPC;
+ ereport(PANIC,
+ (errcode_for_file_access(),
+ errmsg("could not write to key setup file: %m")));
+ }
+ pgstat_report_wait_end();
+
+ pgstat_report_wait_start(WAIT_EVENT_KDF_FILE_SYNC);
+ if (pg_fsync(fd) != 0)
+ ereport(PANIC,
+ (errcode_for_file_access(),
+ errmsg("could not fsync key setup file: %m")));
+ pgstat_report_wait_end();
+
+ if (close(fd))
+ ereport(PANIC,
+ (errcode_for_file_access(),
+ errmsg("could not close key setup file: %m")));
+}
+#endif /* FRONTEND */
+
+/*
+ * Read KDFParamsData from file and store it in local memory.
+ *
+ * If dir is NULL, assume we're in the data directory.
+ *
+ * postmaster should call the function early enough for any other process to
+ * inherit valid pointer to the data.
+ */
+void
+read_kdf_file(char *dir)
+{
+ pg_crc32c crc;
+ int fd;
+ char path[MAXPGPATH];
+ int written = 0;
+ int remains;
+
+ if (dir)
+ {
+ written = snprintf(path, MAXPGPATH, "%s/", dir);
+ if (written >= MAXPGPATH)
+ encryption_error(true, "KDF directory too long");
+ }
+
+ remains = MAXPGPATH - written;
+ if (remains < (strlen(KDF_PARAMS_FILE) + 1))
+ {
+ /*
+ * KDF_PARAMS_FILE should always fit in MAXPGPATH, so directory length
+ * must be the problem.
+ */
+ encryption_error(true, "KDF directory too long");
+ }
+
+ snprintf(path + written, remains, "%s", KDF_PARAMS_FILE);
+
+#ifndef FRONTEND
+ KDFParams = MemoryContextAllocZero(TopMemoryContext,
+ KDF_PARAMS_FILE_SIZE);
+ fd = BasicOpenFile(path, O_RDONLY | PG_BINARY);
+#else
+ KDFParams = palloc(KDF_PARAMS_FILE_SIZE);
+ fd = open(path, O_RDONLY | PG_BINARY, S_IRUSR);
+#endif
+
+ if (fd < 0)
+ encryption_error(true, "could not open key setup file");
+
+#ifndef FRONTEND
+ pgstat_report_wait_start(WAIT_EVENT_KDF_FILE_READ);
+#endif
+
+ if (read(fd, KDFParams, sizeof(KDFParamsData)) != sizeof(KDFParamsData))
+ encryption_error(true, "could not read from key setup file");
+
+#ifndef FRONTEND
+ pgstat_report_wait_end();
+#endif
+
+ close(fd);
+
+ /* Now check the CRC. */
+ INIT_CRC32C(crc);
+ COMP_CRC32C(crc,
+ (char *) KDFParams,
+ offsetof(KDFParamsData, crc));
+ FIN_CRC32C(crc);
+
+ if (!EQ_CRC32C(crc, KDFParams->crc))
+ encryption_error(true, "incorrect checksum in key setup file");
+
+
+ if (KDFParams->function != KDF_OPENSSL_PKCS5_PBKDF2_HMAC_SHA)
+ encryption_error(true, "unsupported KDF function");
+}
+
+/*
+ * Encrypts a fixed value into *buf to verify that encryption key is correct.
+ * Caller provided buf needs to be able to hold at least ENCRYPTION_SAMPLE_SIZE
+ * bytes.
+ */
+void
+sample_encryption(char *buf)
+{
+ char tweak[TWEAK_SIZE];
+ int i;
+
+ for (i = 0; i < TWEAK_SIZE; i++)
+ tweak[i] = i;
+
+ encrypt_block("postgresqlcrypt", buf, ENCRYPTION_SAMPLE_SIZE, tweak,
+ false);
+}
+
+/*
+ * Encrypts one block of data with a specified tweak value. May only be called
+ * when encryption_enabled is true.
+ *
+ * Input and output buffer may point to the same location.
+ *
+ * "size" must be a (non-zero) multiple of ENCRYPTION_BLOCK.
+ *
+ * "tweak" value must be TWEAK_SIZE bytes long.
+ *
+ * If "stream" is set, stream cipher is used instead of block one.
+ *
+ * All-zero blocks are not encrypted to correctly handle relation extension,
+ * and also to simplify handling of holes created by seek past EOF and
+ * consequent write (see buffile.c).
+ */
+void
+encrypt_block(const char *input, char *output, Size size, char *tweak,
+ bool stream)
+{
+#ifdef USE_ENCRYPTION
+ int out_size;
+ EVP_CIPHER_CTX *ctx;
+
+ Assert(data_encrypted);
+
+ /*
+ * Block cipher should only be used if the size is whole multiple of
+ * encryption block size.
+ */
+ Assert((size >= ENCRYPTION_BLOCK && size % ENCRYPTION_BLOCK == 0) ||
+ stream);
+
+ /*
+ * Empty page is not worth encryption. Do not waste cycles checking for
+ * stream cipher as this is currently used only for XLOG pages, and empty
+ * XLOG page should not be written to disk.
+ */
+ if (!stream && IsAllZero(input, size))
+ {
+ memset(output, 0, size);
+ return;
+ }
+
+ ctx = !stream ? ctx_encrypt : ctx_encrypt_stream;
+
+ /* The remaining initialization. */
+ if (EVP_EncryptInit_ex(ctx, NULL, NULL, encryption_key,
+ (unsigned char *) tweak) != 1)
+ evp_error();
+
+ /* Do the actual encryption. */
+ if (EVP_EncryptUpdate(ctx, (unsigned char *) output,
+ &out_size, (unsigned char *) input, size) != 1)
+ evp_error();
+
+ Assert(out_size == size);
+#else
+ /* data_encrypted should not be set */
+ Assert(false);
+#endif /* USE_ENCRYPTION */
+}
+
+/*
+ * Decrypts one block of data with a specified tweak value. May only be called
+ * when encryption_enabled is true.
+ *
+ * Input and output buffer may point to the same location.
+ *
+ * For detailed comments see encrypt_block().
+ */
+void
+decrypt_block(const char *input, char *output, Size size, char *tweak,
+ bool stream)
+{
+#ifdef USE_ENCRYPTION
+ int out_size;
+ EVP_CIPHER_CTX *ctx;
+
+ Assert(data_encrypted);
+ Assert((size >= ENCRYPTION_BLOCK && size % ENCRYPTION_BLOCK == 0) ||
+ stream);
+
+ if (!stream && IsAllZero(input, size))
+ {
+ memset(output, 0, size);
+ return;
+ }
+
+ ctx = !stream ? ctx_decrypt : ctx_decrypt_stream;
+
+ /* The remaining initialization. */
+ if (EVP_DecryptInit_ex(ctx, NULL, NULL, encryption_key,
+ (unsigned char *) tweak) != 1)
+ evp_error();
+
+ /* Do the actual encryption. */
+ if (EVP_DecryptUpdate(ctx, (unsigned char *) output,
+ &out_size, (unsigned char *) input, size) != 1)
+ evp_error();
+
+ Assert(out_size == size);
+#else
+ /* data_encrypted should not be set */
+ Assert(false);
+#endif /* USE_ENCRYPTION */
+}
+
+/*
+ * Report an error in an universal way so that caller does not have to care
+ * whether it executes in backend or front-end.
+ */
+void
+encryption_error(bool fatal, char *message)
+{
+#ifndef FRONTEND
+ /*
+ * Fatal is actually PANIC so that the encryption code never uses lower
+ * elevel than the code around, e.g. that in ReadControlFile().
+ */
+ elog(fatal ? PANIC : INFO, "%s", message);
+#else
+ fprintf(stderr, "%s\n", message);
+ if (fatal)
+ exit(EXIT_FAILURE);
+#endif
+}
+
+/*
+ * If credentials is a key, just copy it to encryption_key. If it's a
+ * password, derive the key from it.
+ */
+void
+setup_encryption_key(char *credentials, bool is_key, size_t len)
+{
+#ifdef USE_ENCRYPTION
+ Assert(credentials != NULL);
+
+ if (is_key)
+ {
+ Assert(len == ENCRYPTION_KEY_LENGTH);
+ memcpy(encryption_key, credentials, len);
+ }
+ else
+ {
+ KDFParamsPBKDF2 *params;
+ int rc;
+
+ /*
+ * The file contains password so we need the KDF parameters to turn it
+ * to key.
+ */
+ if (KDFParams == NULL)
+ {
+#ifndef FRONTEND
+ ereport(FATAL,
+ (errmsg("this instance does not accept encryption password"),
+ errdetail("Encryption key was probably used to initialize the instance.")));
+#else
+ encryption_error(true,
+ "this instance does not accept encryption password.\n"
+ "Encryption key was probably used to initialize the instance.\n");
+#endif /* FRONTEND */
+ }
+
+ /*
+ * Turn the password into the encryption key.
+ */
+ params = &KDFParams->data.pbkdf2;
+ rc = PKCS5_PBKDF2_HMAC(credentials,
+ len,
+ params->salt,
+ ENCRYPTION_KDF_SALT_LEN,
+ params->niter,
+ EVP_sha1(),
+ ENCRYPTION_KEY_LENGTH,
+ encryption_key);
+
+ if (rc != 1)
+ {
+#ifndef FRONTEND
+ ereport(FATAL,
+ (errmsg("failed to derive key from password")));
+#else
+ encryption_error(true, "failed to derive key from password");
+#endif /* FRONTEND */
+ }
+ }
+#else
+ /*
+ * If no encryption implementation is linked and caller requests
+ * encryption, we should error out here and thus cause the calling process
+ * to fail (preferably postmaster, so the child processes don't make the
+ * same mistake).
+ *
+ * We could actually #ifdef only the call of the KDF above, but that would
+ * be inconsistent, and the following ERROR would have to be raised
+ * elsewhere in addition.
+ */
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+}
+
+static void
+setup_encryption_internal(void)
+{
+#ifdef USE_ENCRYPTION
+ /*
+ * Setup OpenSSL.
+ *
+ * None of these functions should return a value or raise error.
+ */
+ ERR_load_crypto_strings();
+ OpenSSL_add_all_algorithms();
+
+ /*
+ * TODO Find out if this needs to be called for OpenSSL < 1.1.0.
+ */
+ /* OPENSSL_config(NULL); */
+
+ init_encryption_context(&ctx_encrypt, false);
+ init_encryption_context(&ctx_decrypt, false);
+
+ init_encryption_context(&ctx_encrypt_stream, true);
+ init_encryption_context(&ctx_decrypt_stream, true);
+
+ /*
+ * Unlike encrypt_buf we need multiple pages here, so allocate the memory
+ * dynamically. That also ensures it'll be MAXALIGNed, which is useful
+ * because the buffer will be used for I/O.
+ *
+ * Use TopMemoryContext because on server side this code is run by
+ * postmaster and postmaster context gets freed after fork().
+ */
+
+ /*
+ * The same for encrypt_buf_xlog, except that this one spans multiple
+ * pages.
+ */
+#ifndef FRONTEND
+ encrypt_buf_xlog = (char *) MemoryContextAlloc(TopMemoryContext,
+ ENCRYPT_BUF_XLOG_SIZE);
+#else
+ encrypt_buf_xlog = (char *) palloc(ENCRYPT_BUF_XLOG_SIZE);
+#endif
+
+#else
+ /* setup_encryption_key() should have been called (and ERROR) by now. */
+ Assert(false);
+#endif /* USE_ENCRYPTION */
+}
+
+#ifdef USE_ENCRYPTION
+/*
+ * Initialize the OpenSSL context for passed cipher.
+ *
+ * On server side this happens during postmaster startup, so other processes
+ * inherit the initialized context via fork(). There's no reason to this again
+ * and again in encrypt_block() / decrypt_block(), also because we cannot
+ * handle out-of-memory conditions encountered by OpenSSL in another way than
+ * ereport(FATAL). The OOM is much less likely to happen during postmaster
+ * startup, and even if it happens, troubleshooting should be easier than if
+ * it happened during normal operation.
+ *
+ * XXX Do we need to call EVP_CIPHER_CTX_cleanup() (via on_proc_exit callback
+ * for server processes and other way for front-ends)? Not sure it's
+ * necessary, as the initialization does not involve any shared resources
+ * (e.g. files).
+ */
+static void
+init_encryption_context(EVP_CIPHER_CTX **ctx_p, bool stream)
+{
+ EVP_CIPHER_CTX *ctx;
+ const EVP_CIPHER *cipher;
+#ifdef USE_ASSERT_CHECKING
+ int block_size;
+#endif /* USE_ASSERT_CHECKING */
+
+ cipher = !stream ? EVP_aes_256_cbc() : EVP_aes_256_ctr();
+
+ if ((*ctx_p = EVP_CIPHER_CTX_new()) == NULL)
+ evp_error();
+ ctx = *ctx_p;
+ if (EVP_EncryptInit_ex(ctx, cipher, NULL, NULL, NULL) != 1)
+ evp_error();
+
+ /*
+ * No padding is needed. For a block cipher, the input block size should
+ * already be a multiple of ENCRYPTION_BLOCK. For stream cipher, we don't
+ * need padding anyway. This might save some cycles at the OpenSSL end.
+ * XXX Is it setting worth when we don't call EVP_DecryptFinal_ex()
+ * anyway?
+ */
+ EVP_CIPHER_CTX_set_padding(ctx, 0);
+
+ Assert(EVP_CIPHER_CTX_iv_length(ctx) == TWEAK_SIZE);
+ Assert(EVP_CIPHER_CTX_key_length(ctx) == ENCRYPTION_KEY_LENGTH);
+ block_size = EVP_CIPHER_CTX_block_size(ctx);
+#ifdef USE_ASSERT_CHECKING
+ if (!stream)
+ Assert(block_size == ENCRYPTION_BLOCK);
+ else
+ Assert(block_size == 1);
+#endif /* USE_ASSERT_CHECKING */
+}
+
+#endif /* USE_ENCRYPTION */
+
+/*
+ * Run the command stored in encryption_key_command and return the key or
+ * password.
+ *
+ * *is_key_p receives true if the command returns key, false if it's
+ * password. *len_p receives length of the data.
+ */
+char *
+run_encryption_key_command(const char *cmd, bool *is_key_p, size_t *len_p)
+{
+ FILE *fp;
+ char *buf,
+ *result;
+ bool is_key = false;
+ size_t key_pref_len,
+ pwd_pref_len,
+ key_chars,
+ read_len,
+ bytes_read;
+ size_t buf_size,
+ result_size;
+
+ if (cmd == NULL || !strlen(cmd))
+ {
+ /*
+ * encryption_key_command should have been set by initdb. It's weird
+ * if it was not, but there's no better recommendation we can give the
+ * user.
+ *
+ * TODO Move this check outside the function since the function does
+ * not care whether the "cmd" argument comes from
+ * "encryption_key_command".
+ */
+#ifndef FRONTEND
+ ereport(FATAL,
+ (errmsg("encryption key not provided"),
+ errdetail("The database cluster was initialized with encryption"
+ " but the application was started without an encryption key."),
+ errhint("Set the encryption_key_command configuration variable.")));
+#else /* FRONTEND */
+ encryption_error(true,
+ "The database cluster was initialized with encryption"
+ " but the server was started without an encryption key. "
+ "Set the encryption_key_command configuration variable.\n");
+#endif /* FRONTEND */
+ }
+
+ encryption_error(false,
+ psprintf("Executing \"%s\" to set up encryption key",
+ cmd));
+
+ fp = popen(cmd, "r");
+ if (fp == NULL)
+ encryption_error(true,
+ psprintf("Failed to execute \"%s\"", cmd));
+
+ /*
+ * Check which prefix the file starts with.
+ *
+ * The prefixes probably won't change after the release but they might
+ * change during development. The reading logic should be generic so that
+ * change of prefix length requires no additional coding.
+ */
+ key_pref_len = strlen(encryption_key_prefix);
+ pwd_pref_len = strlen(encryption_pwd_prefix);
+
+ /*
+ * The buffer must accommodate either prefix.
+ */
+ buf_size = Max(key_pref_len, pwd_pref_len);
+ buf = (char *) palloc(buf_size);
+
+ /*
+ * Read as few bytes as necessary so that we don't have to move back in
+ * the buffer if the first comparison does not match.
+ */
+ read_len = Min(key_pref_len, pwd_pref_len);
+
+ if (fread(buf, 1, read_len, fp) != read_len)
+ encryption_error(true, "Not enough data received from encryption key command");
+
+ if (read_len == key_pref_len &&
+ strncmp(buf, encryption_key_prefix, key_pref_len) == 0)
+ is_key = true;
+ else if (read_len == pwd_pref_len &&
+ strncmp(buf, encryption_pwd_prefix, pwd_pref_len) == 0)
+ is_key = false;
+ else if (buf_size > read_len)
+ {
+ size_t len_diff;
+
+ /*
+ * Read enough data so that one of the prefixes must match.
+ */
+ len_diff = buf_size - read_len;
+ if (fread(buf + read_len, 1, len_diff, fp) != len_diff)
+ encryption_error(true,
+ "Not enough data received from encryption key command");
+ read_len += len_diff;
+
+ /*
+ * Try to match the prefixes again.
+ */
+ if (read_len == key_pref_len &&
+ strncmp(buf, encryption_key_prefix, key_pref_len) == 0)
+ is_key = true;
+ else if (read_len == pwd_pref_len &&
+ strncmp(buf, encryption_pwd_prefix, pwd_pref_len) == 0)
+ is_key = false;
+ else
+ encryption_error(true,
+ "Unknown data received from encryption key command");
+ }
+
+ *is_key_p = is_key;
+
+ /* Key length in characters (two characters per hexadecimal digit) */
+ key_chars = ENCRYPTION_KEY_LENGTH * 2;
+
+ /*
+ * Read the actual credentials.
+ */
+ read_len = is_key ? key_chars : ENCRYPTION_PWD_MAX_LENGTH;
+
+ /*
+ * Accept one extra character so that we can detect key / password too
+ * long.
+ */
+ read_len++;
+
+ if (read_len > buf_size)
+ {
+ buf = (char *) repalloc(buf, read_len);
+ buf_size = read_len;
+ }
+
+ bytes_read = fread(buf, 1, read_len, fp);
+ if (bytes_read == 0)
+ encryption_error(true,
+ "Not enough data provided by encryption key command");
+
+ /*
+ * Since read_len accommodates one extra byte, equality means that too
+ * much data was received, unless the extra byte is line delimiter.
+ */
+ if (bytes_read == read_len && buf[bytes_read - 1] != '\n')
+ encryption_error(true, "Too much data returned by encryption key command");
+
+ /* The line delimiter is not valid data. */
+ if (buf[bytes_read - 1] == '\n')
+ bytes_read--;
+
+ if ((is_key && bytes_read < key_chars) ||
+ (!is_key && bytes_read < ENCRYPTION_PWD_MIN_LENGTH))
+ {
+ if (feof(fp))
+ encryption_error(true,
+ "Not enough data provided by encryption key command");
+ else
+ encryption_error(true,
+ psprintf("encryption key command returned error code %d",
+ ferror(fp)));
+ }
+
+ /*
+ * For a key the result size is different from the amount of data read.
+ */
+ result_size = is_key ? ENCRYPTION_KEY_LENGTH : ENCRYPTION_PWD_MAX_LENGTH;
+
+ result = (char *) palloc(result_size);
+
+ if (is_key)
+ {
+ int i;
+
+ for (i = 0; i < ENCRYPTION_KEY_LENGTH; i++)
+ {
+ if (sscanf(buf + 2 * i, "%2hhx", result + i) == 0)
+ encryption_error(true,
+ psprintf("Invalid character in encryption key at position %d",
+ 2 * i));
+ }
+ *len_p = ENCRYPTION_KEY_LENGTH;
+ }
+ else
+ {
+ *len_p = bytes_read;
+ memcpy(result, buf, *len_p);
+ }
+
+ /*
+ * No extra data is allowed.
+ */
+ if (fread(buf, 1, 1, fp) > 0)
+ encryption_error(true,
+ "Credentials are followed by useless data");
+
+
+ pfree(buf);
+ pclose(fp);
+
+ return result;
+}
+
+#ifdef USE_ENCRYPTION
+/*
+ * Error callback for openssl.
+ */
+static void
+evp_error(void)
+{
+ ERR_print_errors_fp(stderr);
+#ifndef FRONTEND
+
+ /*
+ * FATAL is the appropriate level because backend can hardly fix anything
+ * if encryption / decryption has failed.
+ *
+ * XXX Do we yet need EVP_CIPHER_CTX_cleanup() here?
+ */
+ elog(FATAL, "OpenSSL encountered error during encryption or decryption.");
+#else
+ fprintf(stderr,
+ "OpenSSL encountered error during encryption or decryption.");
+ exit(EXIT_FAILURE);
+#endif /* FRONTEND */
+}
+#endif /* USE_ENCRYPTION */
+
+/*
+ * Xlog is encrypted page at a time. Each xlog page gets a unique tweak via
+ * timeline, segment and offset.
+ *
+ * The function is located here rather than some of the xlog*.c modules so
+ * that front-end applications can easily use it too.
+ */
+void
+XLogEncryptionTweak(char *tweak, TimeLineID timeline, XLogSegNo segment,
+ uint32 offset)
+{
+ memset(tweak, 0, TWEAK_SIZE);
+ memcpy(tweak, &timeline, sizeof(timeline));
+ tweak += sizeof(timeline);
+ memcpy(tweak, &segment, sizeof(XLogSegNo));
+ tweak += sizeof(XLogSegNo);
+ memcpy(tweak, &offset, sizeof(offset));
+}
+
+/*
+ * Copying relations between tablespaces/databases means that the tweak values
+ * of each block will change. This function transcodes a series of blocks with
+ * new tweak values. Returns the new block number for convenience.
+ */
+BlockNumber
+ReencryptBlock(char *buffer, int blocks,
+ RelFileNode *srcNode, RelFileNode *dstNode,
+ ForkNumber srcForkNum, ForkNumber dstForkNum,
+ BlockNumber blockNum)
+{
+ char *cur;
+ char srcTweak[TWEAK_SIZE];
+ char dstTweak[TWEAK_SIZE];
+
+ for (cur = buffer; cur < buffer + blocks * BLCKSZ; cur += BLCKSZ)
+ {
+ mdtweak(srcTweak, srcNode, srcForkNum, blockNum);
+ mdtweak(dstTweak, dstNode, dstForkNum, blockNum);
+ decrypt_block(cur, cur, BLCKSZ, srcTweak, false);
+ encrypt_block(cur, cur, BLCKSZ, dstTweak, false);
+ blockNum++;
+ }
+ return blockNum;
+}
+
+/*
+ * md files are encrypted block at a time. Tweak will alias higher numbered
+ * forks for huge tables.
+ */
+void
+mdtweak(char *tweak, RelFileNode *relnode, ForkNumber forknum, BlockNumber blocknum)
+{
+ uint32 fork_and_block = (forknum << 24) ^ blocknum;
+
+ memcpy(tweak, relnode, sizeof(RelFileNode));
+ memcpy(tweak + sizeof(RelFileNode), &fork_and_block, 4);
+}
diff --git a/src/backend/storage/file/reinit.c b/src/backend/storage/file/reinit.c
index 1ead778cc9..6af20cfe64 100644
--- a/src/backend/storage/file/reinit.c
+++ b/src/backend/storage/file/reinit.c
@@ -16,6 +16,7 @@
#include <unistd.h>
+#include "catalog/pg_tablespace.h"
#include "common/relpath.h"
#include "storage/copydir.h"
#include "storage/fd.h"
@@ -24,9 +25,9 @@
#include "utils/memutils.h"
static void ResetUnloggedRelationsInTablespaceDir(const char *tsdirname,
- int op);
+ int op, Oid spcOid);
static void ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname,
- int op);
+ int op, Oid spcOid, Oid dbOid);
typedef struct
{
@@ -68,7 +69,7 @@ ResetUnloggedRelations(int op)
/*
* First process unlogged files in pg_default ($PGDATA/base)
*/
- ResetUnloggedRelationsInTablespaceDir("base", op);
+ ResetUnloggedRelationsInTablespaceDir("base", op, DEFAULTTABLESPACE_OID);
/*
* Cycle through directories for all non-default tablespaces.
@@ -77,13 +78,16 @@ ResetUnloggedRelations(int op)
while ((spc_de = ReadDir(spc_dir, "pg_tblspc")) != NULL)
{
+ Oid spcOid;
+
if (strcmp(spc_de->d_name, ".") == 0 ||
strcmp(spc_de->d_name, "..") == 0)
continue;
snprintf(temp_path, sizeof(temp_path), "pg_tblspc/%s/%s",
spc_de->d_name, TABLESPACE_VERSION_DIRECTORY);
- ResetUnloggedRelationsInTablespaceDir(temp_path, op);
+ spcOid = atoi(spc_de->d_name);
+ ResetUnloggedRelationsInTablespaceDir(temp_path, op, spcOid);
}
FreeDir(spc_dir);
@@ -99,7 +103,8 @@ ResetUnloggedRelations(int op)
* Process one tablespace directory for ResetUnloggedRelations
*/
static void
-ResetUnloggedRelationsInTablespaceDir(const char *tsdirname, int op)
+ResetUnloggedRelationsInTablespaceDir(const char *tsdirname, int op,
+ Oid spcOid)
{
DIR *ts_dir;
struct dirent *de;
@@ -126,6 +131,8 @@ ResetUnloggedRelationsInTablespaceDir(const char *tsdirname, int op)
while ((de = ReadDir(ts_dir, tsdirname)) != NULL)
{
+ Oid dbOid;
+
/*
* We're only interested in the per-database directories, which have
* numeric names. Note that this code will also (properly) ignore "."
@@ -134,9 +141,10 @@ ResetUnloggedRelationsInTablespaceDir(const char *tsdirname, int op)
if (strspn(de->d_name, "0123456789") != strlen(de->d_name))
continue;
+ dbOid = atoi(de->d_name);
snprintf(dbspace_path, sizeof(dbspace_path), "%s/%s",
tsdirname, de->d_name);
- ResetUnloggedRelationsInDbspaceDir(dbspace_path, op);
+ ResetUnloggedRelationsInDbspaceDir(dbspace_path, op, spcOid, dbOid);
}
FreeDir(ts_dir);
@@ -146,7 +154,8 @@ ResetUnloggedRelationsInTablespaceDir(const char *tsdirname, int op)
* Process one per-dbspace directory for ResetUnloggedRelations
*/
static void
-ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op)
+ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op,
+ Oid spcOid, Oid dbOid)
{
DIR *dbspace_dir;
struct dirent *de;
@@ -187,7 +196,7 @@ ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op)
/* Skip anything that doesn't look like a relation data file. */
if (!parse_filename_for_nontemp_relation(de->d_name, &oidchars,
- &forkNum))
+ &forkNum, NULL))
continue;
/* Also skip it unless this is the init fork. */
@@ -229,7 +238,7 @@ ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op)
/* Skip anything that doesn't look like a relation data file. */
if (!parse_filename_for_nontemp_relation(de->d_name, &oidchars,
- &forkNum))
+ &forkNum, NULL))
continue;
/* We never remove the init fork. */
@@ -279,13 +288,14 @@ ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op)
{
ForkNumber forkNum;
int oidchars;
+ int segment;
char oidbuf[OIDCHARS + 1];
char srcpath[MAXPGPATH * 2];
char dstpath[MAXPGPATH];
/* Skip anything that doesn't look like a relation data file. */
if (!parse_filename_for_nontemp_relation(de->d_name, &oidchars,
- &forkNum))
+ &forkNum, &segment))
continue;
/* Also skip it unless this is the init fork. */
@@ -305,7 +315,13 @@ ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op)
/* OK, we're ready to perform the actual copy. */
elog(DEBUG2, "copying %s to %s", srcpath, dstpath);
- copy_file(srcpath, dstpath);
+ {
+ RelFileNode srcNode = {spcOid, dbOid, atol(oidbuf)};
+ RelFileNode dstNode = srcNode;
+
+ copy_file(srcpath, dstpath, &srcNode, &dstNode,
+ INIT_FORKNUM, MAIN_FORKNUM, segment);
+ }
}
FreeDir(dbspace_dir);
@@ -327,7 +343,7 @@ ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op)
/* Skip anything that doesn't look like a relation data file. */
if (!parse_filename_for_nontemp_relation(de->d_name, &oidchars,
- &forkNum))
+ &forkNum, NULL))
continue;
/* Also skip it unless this is the init fork. */
@@ -372,9 +388,10 @@ ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op)
*/
bool
parse_filename_for_nontemp_relation(const char *name, int *oidchars,
- ForkNumber *fork)
+ ForkNumber *fork, int *segment)
{
int pos;
+ int segstart = 0;
/* Look for a non-empty string of digits (that isn't too long). */
for (pos = 0; isdigit((unsigned char) name[pos]); ++pos)
@@ -401,6 +418,7 @@ parse_filename_for_nontemp_relation(const char *name, int *oidchars,
{
int segchar;
+ segstart = pos + 1;
for (segchar = 1; isdigit((unsigned char) name[pos + segchar]); ++segchar)
;
if (segchar <= 1)
@@ -411,5 +429,14 @@ parse_filename_for_nontemp_relation(const char *name, int *oidchars,
/* Now we should be at the end. */
if (name[pos] != '\0')
return false;
+
+ if (segment != NULL)
+ {
+ if (segstart == 0)
+ *segment = 0;
+ else
+ *segment = atoi(name + segstart);
+ }
+
return true;
}
diff --git a/src/backend/storage/page/bufpage.c b/src/backend/storage/page/bufpage.c
index 14bc61b8ad..ebc50320e1 100644
--- a/src/backend/storage/page/bufpage.c
+++ b/src/backend/storage/page/bufpage.c
@@ -17,6 +17,7 @@
#include "access/htup_details.h"
#include "access/itup.h"
#include "access/xlog.h"
+#include "common/string.h"
#include "pgstat.h"
#include "storage/checksum.h"
#include "utils/memdebug.h"
@@ -82,11 +83,8 @@ bool
PageIsVerified(Page page, BlockNumber blkno)
{
PageHeader p = (PageHeader) page;
- size_t *pagebytes;
- int i;
bool checksum_failure = false;
bool header_sane = false;
- bool all_zeroes = false;
uint16 checksum = 0;
/*
@@ -119,26 +117,8 @@ PageIsVerified(Page page, BlockNumber blkno)
return true;
}
- /*
- * Check all-zeroes case. Luckily BLCKSZ is guaranteed to always be a
- * multiple of size_t - and it's much faster to compare memory using the
- * native word size.
- */
- StaticAssertStmt(BLCKSZ == (BLCKSZ / sizeof(size_t)) * sizeof(size_t),
- "BLCKSZ has to be a multiple of sizeof(size_t)");
-
- all_zeroes = true;
- pagebytes = (size_t *) page;
- for (i = 0; i < (BLCKSZ / sizeof(size_t)); i++)
- {
- if (pagebytes[i] != 0)
- {
- all_zeroes = false;
- break;
- }
- }
-
- if (all_zeroes)
+ /* Check all-zeroes case */
+ if (IsAllZero((char *) page, BLCKSZ))
return true;
/*
diff --git a/src/backend/storage/smgr/md.c b/src/backend/storage/smgr/md.c
index 61a8f11469..82265da74f 100644
--- a/src/backend/storage/smgr/md.c
+++ b/src/backend/storage/smgr/md.c
@@ -32,6 +32,7 @@
#include "postmaster/bgwriter.h"
#include "storage/fd.h"
#include "storage/bufmgr.h"
+#include "storage/encryption.h"
#include "storage/md.h"
#include "storage/relfilenode.h"
#include "storage/smgr.h"
@@ -86,6 +87,7 @@ typedef struct _MdfdVec
static MemoryContext MdCxt; /* context for all MdfdVec objects */
+static char *md_encryption_tweak;
/* Populate a file tag describing an md.c segment file. */
#define INIT_MD_FILETAG(a,xx_rnode,xx_forknum,xx_segno) \
@@ -139,6 +141,8 @@ static MdfdVec *_mdfd_getseg(SMgrRelation reln, ForkNumber forkno,
static BlockNumber _mdnblocks(SMgrRelation reln, ForkNumber forknum,
MdfdVec *seg);
+static void mdencrypt(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum, char *buffer);
+static void mddecrypt(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum, char *buffer);
/*
* mdinit() -- Initialize private state for magnetic disk storage manager.
@@ -149,6 +153,8 @@ mdinit(void)
MdCxt = AllocSetContextCreate(TopMemoryContext,
"MdSmgr",
ALLOCSET_DEFAULT_SIZES);
+
+ md_encryption_tweak = MemoryContextAllocZero(MdCxt, TWEAK_SIZE);
}
/*
@@ -401,6 +407,12 @@ mdextend(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
Assert(seekpos < (off_t) BLCKSZ * RELSEG_SIZE);
+ if (data_encrypted)
+ {
+ mdencrypt(reln, forknum, blocknum, buffer);
+ buffer = encrypt_buf.data;
+ }
+
if ((nbytes = FileWrite(v->mdfd_vfd, buffer, BLCKSZ, seekpos, WAIT_EVENT_DATA_FILE_EXTEND)) != BLCKSZ)
{
if (nbytes < 0)
@@ -587,6 +599,7 @@ mdread(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
off_t seekpos;
int nbytes;
MdfdVec *v;
+ char *buffer_read = buffer;
TRACE_POSTGRESQL_SMGR_MD_READ_START(forknum, blocknum,
reln->smgr_rnode.node.spcNode,
@@ -601,7 +614,10 @@ mdread(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
Assert(seekpos < (off_t) BLCKSZ * RELSEG_SIZE);
- nbytes = FileRead(v->mdfd_vfd, buffer, BLCKSZ, seekpos, WAIT_EVENT_DATA_FILE_READ);
+ if (data_encrypted)
+ buffer_read = encrypt_buf.data;
+
+ nbytes = FileRead(v->mdfd_vfd, buffer_read, BLCKSZ, seekpos, WAIT_EVENT_DATA_FILE_READ);
TRACE_POSTGRESQL_SMGR_MD_READ_DONE(forknum, blocknum,
reln->smgr_rnode.node.spcNode,
@@ -636,6 +652,8 @@ mdread(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
blocknum, FilePathName(v->mdfd_vfd),
nbytes, BLCKSZ)));
}
+ else if (data_encrypted)
+ mddecrypt(reln, forknum, blocknum, buffer);
}
/*
@@ -671,6 +689,11 @@ mdwrite(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
Assert(seekpos < (off_t) BLCKSZ * RELSEG_SIZE);
+ if (data_encrypted)
+ {
+ mdencrypt(reln, forknum, blocknum, buffer);
+ buffer = encrypt_buf.data;
+ }
nbytes = FileWrite(v->mdfd_vfd, buffer, BLCKSZ, seekpos, WAIT_EVENT_DATA_FILE_WRITE);
TRACE_POSTGRESQL_SMGR_MD_WRITE_DONE(forknum, blocknum,
@@ -1248,6 +1271,22 @@ _mdnblocks(SMgrRelation reln, ForkNumber forknum, MdfdVec *seg)
return (BlockNumber) (len / BLCKSZ);
}
+
+static void
+mdencrypt(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum, char *buffer)
+{
+ mdtweak(md_encryption_tweak, &(reln->smgr_rnode.node), forknum, blocknum);
+ encrypt_block(buffer, encrypt_buf.data, BLCKSZ, md_encryption_tweak,
+ false);
+}
+
+static void
+mddecrypt(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum, char *dest)
+{
+ mdtweak(md_encryption_tweak, &(reln->smgr_rnode.node), forknum, blocknum);
+ decrypt_block(encrypt_buf.data, dest, BLCKSZ, md_encryption_tweak, false);
+}
+
/*
* Sync a file to disk, given a file tag. Write the path into an output
* buffer so the caller can use it in error messages.
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 44a59e1d4f..40459168a4 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -63,6 +63,7 @@
#include "replication/walsender.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/encryption.h"
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
@@ -3497,7 +3498,7 @@ process_postgres_switches(int argc, char *argv[], GucContext ctx,
* postmaster/postmaster.c (the option sets should not conflict) and with
* the common help() function in main/main.c.
*/
- while ((flag = getopt(argc, argv, "B:bc:C:D:d:EeFf:h:ijk:lN:nOo:Pp:r:S:sTt:v:W:-:")) != -1)
+ while ((flag = getopt(argc, argv, "B:bc:C:D:d:EeFf:h:ijK:k:lN:nOo:Pp:r:S:sTt:v:W:-:")) != -1)
{
switch (flag)
{
@@ -3555,6 +3556,12 @@ process_postgres_switches(int argc, char *argv[], GucContext ctx,
UseSemiNewlineNewline = true;
break;
+#ifdef USE_ENCRYPTION
+ case 'K':
+ encryption_key_command = strdup(optarg);
+ break;
+#endif /* USE_OPENSSL */
+
case 'k':
SetConfigOption("unix_socket_directories", optarg, ctx, gucsource);
break;
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index 83c9514856..053e2a14cd 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -1514,6 +1514,7 @@ ValidatePgVersion(const char *path)
* GUC variables: lists of library names to be preloaded at postmaster
* start and at backend start
*/
+char *encryption_library_string = NULL;
char *session_preload_libraries_string = NULL;
char *shared_preload_libraries_string = NULL;
char *local_preload_libraries_string = NULL;
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index e24f003983..31b87c7097 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -72,6 +72,7 @@
#include "replication/walsender.h"
#include "storage/bufmgr.h"
#include "storage/dsm_impl.h"
+#include "storage/encryption.h"
#include "storage/standby.h"
#include "storage/fd.h"
#include "storage/large_object.h"
@@ -172,6 +173,7 @@ static void assign_session_replication_role(int newval, void *extra);
static bool check_temp_buffers(int *newval, void **extra, GucSource source);
static bool check_bonjour(bool *newval, void **extra, GucSource source);
static bool check_ssl(bool *newval, void **extra, GucSource source);
+static bool check_full_page_writes(bool *newval, void **extra, GucSource source);
static bool check_stage_log_stats(bool *newval, void **extra, GucSource source);
static bool check_log_stats(bool *newval, void **extra, GucSource source);
static bool check_canonical_path(char **newval, void **extra, GucSource source);
@@ -216,6 +218,10 @@ static void assign_recovery_target_lsn(const char *newval, void *extra);
static bool check_primary_slot_name(char **newval, void **extra, GucSource source);
static bool check_default_with_oids(bool *newval, void **extra, GucSource source);
+#ifdef USE_ENCRYPTION
+static const char *show_encryption_key_command(void);
+#endif /* USE_ENCRYPTION */
+
/* Private functions in guc-file.l that need to be called from guc.c */
static ConfigVariable *ProcessConfigFileInternal(GucContext context,
bool applySettings, int elevel);
@@ -1173,7 +1179,7 @@ static struct config_bool ConfigureNamesBool[] =
},
&fullPageWrites,
true,
- NULL, NULL, NULL
+ check_full_page_writes, NULL, NULL
},
{
@@ -1837,6 +1843,17 @@ static struct config_bool ConfigureNamesBool[] =
},
{
+ {"data_encryption", PGC_INTERNAL, PRESET_OPTIONS,
+ gettext_noop("Shows whether data encryption is turned on for this cluster."),
+ NULL,
+ GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE
+ },
+ &data_encrypted,
+ false,
+ NULL, NULL, NULL
+ },
+
+ {
{"syslog_sequence_numbers", PGC_SIGHUP, LOGGING_WHERE,
gettext_noop("Add sequence number to syslog messages to avoid duplicate suppression."),
NULL
@@ -4213,6 +4230,19 @@ static struct config_string ConfigureNamesString[] =
NULL, NULL, NULL
},
+#ifdef USE_ENCRYPTION
+ {
+ {"encryption_key_command", PGC_POSTMASTER, 0,
+ gettext_noop("Sets the shell command that will be called to fetch database encryption key."),
+ NULL,
+ GUC_NOT_IN_SAMPLE | GUC_SUPERUSER_ONLY | GUC_IS_NAME
+ },
+ &encryption_key_command,
+ NULL,
+ NULL, NULL, show_encryption_key_command
+ },
+#endif /* USE_ENCRYPTION */
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, NULL, NULL, NULL, NULL
@@ -11094,6 +11124,19 @@ check_ssl(bool *newval, void **extra, GucSource source)
}
static bool
+check_full_page_writes(bool *newval, void **extra, GucSource source)
+{
+ if (!(*newval) && data_encrypted)
+ {
+ GUC_check_errdetail("Cannot disable parameter when the cluster is encrypted.");
+
+ return false;
+ }
+
+ return true;
+}
+
+static bool
check_stage_log_stats(bool *newval, void **extra, GucSource source)
{
if (*newval && log_statement_stats)
@@ -11751,4 +11794,15 @@ check_default_with_oids(bool *newval, void **extra, GucSource source)
return true;
}
+#ifdef USE_ENCRYPTION
+static const char *
+show_encryption_key_command(void)
+{
+ if (encryption_key_command)
+ return encryption_key_command;
+ else
+ return "(disabled)";
+}
+#endif /* USE_ENCRYPTION */
+
#include "guc-file.c"
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 0c905bb55d..e0580ea69f 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -90,6 +90,7 @@
#authentication_timeout = 1min # 1s-600s
#password_encryption = md5 # md5 or scram-sha-256
#db_user_namespace = off
+#encryption_key_command = ''
# GSSAPI using Kerberos
#krb_server_keyfile = ''
diff --git a/src/bin/Makefile b/src/bin/Makefile
index 903e58121f..d23dbeda55 100644
--- a/src/bin/Makefile
+++ b/src/bin/Makefile
@@ -22,6 +22,7 @@ SUBDIRS = \
pg_controldata \
pg_ctl \
pg_dump \
+ pg_keysetup \
pg_resetwal \
pg_rewind \
pg_test_fsync \
diff --git a/src/bin/initdb/Makefile b/src/bin/initdb/Makefile
index 7c404430a9..b72de55d62 100644
--- a/src/bin/initdb/Makefile
+++ b/src/bin/initdb/Makefile
@@ -26,7 +26,7 @@ ifneq (,$(with_system_tzdata))
override CPPFLAGS += '-DSYSTEMTZDIR="$(with_system_tzdata)"'
endif
-OBJS= initdb.o findtimezone.o localtime.o encnames.o $(WIN32RES)
+OBJS= initdb.o findtimezone.o localtime.o encnames.o encryption.o $(WIN32RES)
all: initdb
@@ -45,6 +45,9 @@ encnames.c: % : $(top_srcdir)/src/backend/utils/mb/%
localtime.c: % : $(top_srcdir)/src/timezone/%
rm -f $@ && $(LN_S) $< .
+encryption.c: % : $(top_srcdir)/src/backend/storage/file/%
+ rm -f $@ && $(LN_S) $< .
+
install: all installdirs
$(INSTALL_PROGRAM) initdb$(X) '$(DESTDIR)$(bindir)/initdb$(X)'
@@ -55,7 +58,7 @@ uninstall:
rm -f '$(DESTDIR)$(bindir)/initdb$(X)'
clean distclean maintainer-clean:
- rm -f initdb$(X) $(OBJS) encnames.c localtime.c
+ rm -f initdb$(X) $(OBJS) encnames.c localtime.c encryption.c
rm -rf tmp_check
# ensure that changes in datadir propagate into object file
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index 176bd50075..ad9950e334 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -69,6 +69,9 @@
#include "common/username.h"
#include "fe_utils/logging.h"
#include "fe_utils/string_utils.h"
+#include "storage/encryption.h"
+#include "lib/ilist.h"
+#include "mb/pg_wchar.h"
#include "getaddrinfo.h"
#include "getopt_long.h"
#include "mb/pg_wchar.h"
@@ -78,6 +81,12 @@
/* Ideally this would be in a .h file, but it hardly seems worth the trouble */
extern const char *select_default_timezone(const char *share_path);
+typedef struct
+{
+ dlist_node list_node;
+ char *value;
+} extra_option;
+
static const char *const auth_methods_host[] = {
"trust", "reject", "scram-sha-256", "md5", "password", "ident", "radius",
#ifdef ENABLE_GSS
@@ -142,9 +151,11 @@ static bool do_sync = true;
static bool sync_only = false;
static bool show_setting = false;
static bool data_checksums = false;
+static char *encr_key_cmd_str = NULL;
static char *xlog_dir = NULL;
static char *str_wal_segment_size_mb = NULL;
static int wal_segment_size_mb;
+static dlist_head extra_options = DLIST_STATIC_INIT(extra_options);
/* internal vars */
@@ -528,6 +539,7 @@ readfile(const char *path)
fclose(infile);
free(buffer);
+
result[n] = NULL;
return result;
@@ -1072,6 +1084,39 @@ pretty_wal_size(int segment_count)
return result;
}
+static void
+append_extra_options(char ***conflines)
+{
+ dlist_iter iter;
+ int n_extra = 0;
+ int n_current = 0;
+ int i = 0;
+ char **new_conflines;
+
+ dlist_foreach(iter, &extra_options)
+ {
+ n_extra++;
+ }
+ while ((*conflines)[i++] != NULL)
+ n_current++;
+
+ new_conflines = (char **) pg_malloc((n_current + n_extra + 1) * sizeof(char *));
+ for (i = 0; i < n_current; i++)
+ new_conflines[i] = (*conflines)[i];
+
+ dlist_foreach(iter, &extra_options)
+ {
+ extra_option *opt = dlist_container(extra_option, list_node, iter.cur);
+
+ new_conflines[i++] = opt->value;
+ }
+
+ new_conflines[i] = NULL;
+ pg_free(*conflines);
+ *conflines = new_conflines;
+}
+
+
/*
* set up all the config files
*/
@@ -1090,6 +1135,8 @@ setup_config(void)
conflines = readfile(conf_file);
+ append_extra_options(&conflines);
+
snprintf(repltok, sizeof(repltok), "max_connections = %d", n_connections);
conflines = replace_token(conflines, "#max_connections = 100", repltok);
@@ -1231,6 +1278,14 @@ setup_config(void)
"log_file_mode = 0640");
}
+ if (encryption_key_command)
+ {
+ snprintf(repltok, sizeof(repltok), "encryption_key_command = '%s'",
+ encryption_key_command);
+ conflines = replace_token(conflines,
+ "#encryption_key_command = ''", repltok);
+ }
+
snprintf(path, sizeof(path), "%s/postgresql.conf", pg_data);
writefile(path, conflines);
@@ -1444,11 +1499,27 @@ bootstrap_template1(void)
/* Also ensure backend isn't confused by this environment var: */
unsetenv("PGCLIENTENCODING");
+ /* Prepare the -K option for the backend. */
+ if (encryption_key_command)
+ {
+ size_t len;
+
+ len = 3 + strlen(encryption_key_command) + 1;
+ encr_key_cmd_str = (char *) pg_malloc(len);
+ snprintf(encr_key_cmd_str, len, "-K %s", encryption_key_command);
+ }
+ else
+ {
+ encr_key_cmd_str = (char *) pg_malloc(1);
+ encr_key_cmd_str[0] = '\0';
+ }
+
snprintf(cmd, sizeof(cmd),
- "\"%s\" --boot -x1 -X %u %s %s %s",
+ "\"%s\" --boot -x1 -X %u %s %s %s %s",
backend_exec,
wal_segment_size_mb * (1024 * 1024),
data_checksums ? "-k" : "",
+ encr_key_cmd_str,
boot_options,
debug ? "-d 5" : "");
@@ -2376,6 +2447,10 @@ usage(const char *progname)
printf(_("\nLess commonly used options:\n"));
printf(_(" -d, --debug generate lots of debugging output\n"));
printf(_(" -k, --data-checksums use data page checksums\n"));
+#ifdef USE_ENCRYPTION
+ printf(_(" -K, --encryption-key-command\n"
+ " command that returns encryption key\n"));
+#endif /* USE_OPENSSL */
printf(_(" -L DIRECTORY where to find the input files\n"));
printf(_(" -n, --no-clean do not clean up after errors\n"));
printf(_(" -N, --no-sync do not wait for changes to be written safely to disk\n"));
@@ -2479,7 +2554,6 @@ setup_pgdata(void)
putenv(pgdata_set_env);
}
-
void
setup_bin_paths(const char *argv0)
{
@@ -2978,8 +3052,8 @@ initialize_data_directory(void)
fflush(stdout);
snprintf(cmd, sizeof(cmd),
- "\"%s\" %s template1 >%s",
- backend_exec, backend_options,
+ "\"%s\" %s %s template1 >%s",
+ backend_exec, backend_options, encr_key_cmd_str,
DEVNULL);
PG_CMD_OPEN;
@@ -3018,6 +3092,23 @@ initialize_data_directory(void)
check_ok();
}
+static void
+parse_extra_option_arg(char *optarg)
+{
+ extra_option *opt;
+
+ if (!strchr(optarg, '='))
+ {
+ fprintf(stderr, _("Option value is not in key=value format"));
+ exit(1);
+ }
+
+ opt = malloc(sizeof(extra_option));
+ if (asprintf(&opt->value, "%s\n", optarg) < 0)
+ exit(1);
+
+ dlist_push_tail(&extra_options, &opt->list_node);
+}
int
main(int argc, char *argv[])
@@ -3052,6 +3143,9 @@ main(int argc, char *argv[])
{"waldir", required_argument, NULL, 'X'},
{"wal-segsize", required_argument, NULL, 12},
{"data-checksums", no_argument, NULL, 'k'},
+#ifdef USE_ENCRYPTION
+ {"encryption-key-command", required_argument, NULL, 'K'},
+#endif /* USE_OPENSSL */
{"allow-group-access", no_argument, NULL, 'g'},
{NULL, 0, NULL, 0}
};
@@ -3094,7 +3188,7 @@ main(int argc, char *argv[])
/* process command-line options */
- while ((c = getopt_long(argc, argv, "dD:E:kL:nNU:WA:sST:X:g", long_options, &option_index)) != -1)
+ while ((c = getopt_long(argc, argv, "dD:E:kK:L:nNU:WA:sST:X:g", long_options, &option_index)) != -1)
{
switch (c)
{
@@ -3146,6 +3240,14 @@ main(int argc, char *argv[])
case 'k':
data_checksums = true;
break;
+#ifdef USE_ENCRYPTION
+ case 'K':
+ encryption_key_command = pg_strdup(optarg);
+ break;
+#endif /* USE_ENCRYPTION */
+ case 'c':
+ parse_extra_option_arg(optarg);
+ break;
case 'L':
share_path = pg_strdup(optarg);
break;
@@ -3316,7 +3418,10 @@ main(int argc, char *argv[])
if (pwprompt || pwfilename)
get_su_pwd();
- printf("\n");
+ if (encryption_key_command)
+ printf(_("Data encryption is enabled.\n"));
+ else
+ printf(_("Data encryption is disabled.\n"));
initialize_data_directory();
diff --git a/src/bin/pg_checksums/Makefile b/src/bin/pg_checksums/Makefile
index 13a25f5e33..90f12d233f 100644
--- a/src/bin/pg_checksums/Makefile
+++ b/src/bin/pg_checksums/Makefile
@@ -17,13 +17,18 @@ include $(top_builddir)/src/Makefile.global
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils
-OBJS= pg_checksums.o $(WIN32RES)
+OBJS= encryption.o pg_checksums.o $(WIN32RES)
+
+override CPPFLAGS := -DFRONTEND $(CPPFLAGS)
all: pg_checksums
pg_checksums: $(OBJS) | submake-libpgport submake-libpgfeutils
$(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+encryption.c: % : $(top_srcdir)/src/backend/storage/file/%
+ rm -f $@ && $(LN_S) $< .
+
install: all installdirs
$(INSTALL_PROGRAM) pg_checksums$(X) '$(DESTDIR)$(bindir)/pg_checksums$(X)'
@@ -36,6 +41,7 @@ uninstall:
clean distclean maintainer-clean:
rm -f pg_checksums$(X) $(OBJS)
rm -rf tmp_check
+ rm -f encryption.c
check:
$(prove_check)
diff --git a/src/bin/pg_checksums/pg_checksums.c b/src/bin/pg_checksums/pg_checksums.c
index bc89982658..0cf49eadcd 100644
--- a/src/bin/pg_checksums/pg_checksums.c
+++ b/src/bin/pg_checksums/pg_checksums.c
@@ -20,6 +20,7 @@
#include <unistd.h>
#include "access/xlog_internal.h"
+#include "catalog/pg_tablespace_d.h"
#include "common/controldata_utils.h"
#include "common/file_perm.h"
#include "common/file_utils.h"
@@ -29,6 +30,7 @@
#include "storage/bufpage.h"
#include "storage/checksum.h"
#include "storage/checksum_impl.h"
+#include "storage/encryption.h"
static int64 files = 0;
@@ -80,6 +82,10 @@ usage(void)
printf(_(" -c, --check check data checksums (default)\n"));
printf(_(" -d, --disable disable data checksums\n"));
printf(_(" -e, --enable enable data checksums\n"));
+#ifdef USE_ENCRYPTION
+ printf(_(" -K, --encryption-key-command=COMMAND\n"
+ " command that returns encryption key\n"));
+#endif /* USE_ENCRYPTION */
printf(_(" -N, --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" -P, --progress show progress information\n"));
printf(_(" -v, --verbose output verbose messages\n"));
@@ -167,7 +173,8 @@ skipfile(const char *fn)
}
static void
-scan_file(const char *fn, BlockNumber segmentno)
+scan_file(const char *fn, Oid relnode, BlockNumber segmentno,
+ ForkNumber forkno, Oid tbspace, Oid db)
{
PGAlignedBlock buf;
PageHeader header = (PageHeader) buf.data;
@@ -193,6 +200,7 @@ scan_file(const char *fn, BlockNumber segmentno)
{
uint16 csum;
int r = read(f, buf.data, BLCKSZ);
+ char tweak[TWEAK_SIZE];
if (r == 0)
break;
@@ -204,6 +212,18 @@ scan_file(const char *fn, BlockNumber segmentno)
}
blocks++;
+ if (data_encrypted)
+ {
+ RelFileNode node;
+
+ node.spcNode = tbspace;
+ node.dbNode = db;
+ node.relNode = relnode;
+
+ mdtweak(tweak, &node, forkno, blockno);
+ decrypt_block(buf.data, buf.data, BLCKSZ, tweak, false);
+ }
+
/* New pages have no checksum yet */
if (PageIsNew(header))
continue;
@@ -232,6 +252,9 @@ scan_file(const char *fn, BlockNumber segmentno)
exit(1);
}
+ if (data_encrypted)
+ encrypt_block(buf.data, buf.data, BLCKSZ, tweak, false);
+
/* Write block with checksum */
if (write(f, buf.data, BLCKSZ) != BLCKSZ)
{
@@ -262,9 +285,13 @@ scan_file(const char *fn, BlockNumber segmentno)
* all the items which have checksums is computed and returned back
* to the caller without operating on the files. This is used to compile
* the total size of the data directory for progress reports.
+ *
+ * If db is a valid pointer, *db contains database OID. If it's NULL, the
+ * database OID needs to be recognized, possibly by recursive call.
*/
static int64
-scan_directory(const char *basedir, const char *subdir, bool sizeonly)
+scan_directory(const char *basedir, const char *subdir, bool sizeonly,
+ Oid tbspace, Oid *db)
{
int64 dirsize = 0;
char path[MAXPGPATH];
@@ -282,6 +309,9 @@ scan_directory(const char *basedir, const char *subdir, bool sizeonly)
{
char fn[MAXPGPATH];
struct stat st;
+ bool tbsp_identified = false;
+ Oid database;
+ bool db_identified = false;
if (strcmp(de->d_name, ".") == 0 ||
strcmp(de->d_name, "..") == 0)
@@ -305,12 +335,58 @@ scan_directory(const char *basedir, const char *subdir, bool sizeonly)
pg_log_error("could not stat file \"%s\": %m", fn);
exit(1);
}
+
+ if (tbspace != InvalidOid && db == NULL)
+ {
+ /*
+ * If tablespace is passed by caller or identified by upper call
+ * of this function, and if caller could not identify the database
+ * OID, try to do it now.
+ */
+ if (strcmp(de->d_name, TABLESPACE_VERSION_DIRECTORY) == 0)
+ {
+ /*
+ * Major-version-specific tablespace subdirectory needs no
+ * special attention, we'll recurse into it below.
+ */
+ }
+ else
+ {
+ errno = 0;
+ database = strtol(de->d_name, NULL, 10);
+ if (errno != 0 || database == InvalidOid)
+ {
+ fprintf(stderr, _("%s: invalid database oid \"%s\"\n"),
+ progname, de->d_name);
+ exit(1);
+ }
+ db_identified = true;
+ }
+ }
+ else if (tbspace == InvalidOid)
+ {
+ /*
+ * This entry should be a direct subdirectory of pg_tblspc, so the
+ * name is supposedly tablespace oid.
+ */
+ errno = 0;
+ tbspace = strtol(de->d_name, NULL, 10);
+ if (errno != 0 || tbspace == InvalidOid)
+ {
+ fprintf(stderr, _("%s: invalid tablespace oid \"%s\"\n"),
+ progname, de->d_name);
+ exit(1);
+ }
+ tbsp_identified = true;
+ }
+
if (S_ISREG(st.st_mode))
{
char fnonly[MAXPGPATH];
char *forkpath,
*segmentpath;
BlockNumber segmentno = 0;
+ ForkNumber forkno;
if (skipfile(de->d_name))
continue;
@@ -337,8 +413,14 @@ scan_directory(const char *basedir, const char *subdir, bool sizeonly)
forkpath = strchr(fnonly, '_');
if (forkpath != NULL)
+ {
*forkpath++ = '\0';
+ forkno = forkname_to_number(forkpath);
+ }
+ else
+ forkno = MAIN_FORKNUM;
+
if (only_relfilenode && strcmp(only_relfilenode, fnonly) != 0)
/* Relfilenode not to be included */
continue;
@@ -350,14 +432,31 @@ scan_directory(const char *basedir, const char *subdir, bool sizeonly)
* the items in the data folder.
*/
if (!sizeonly)
- scan_file(fn, segmentno);
+ {
+ Oid relnode = atoi(fnonly);
+
+ scan_file(fn, relnode, segmentno, forkno, tbspace, *db);
+ }
}
#ifndef WIN32
else if (S_ISDIR(st.st_mode) || S_ISLNK(st.st_mode))
#else
else if (S_ISDIR(st.st_mode) || pgwin32_is_junction(fn))
#endif
- dirsize += scan_directory(path, de->d_name, sizeonly);
+
+ /*
+ * If database OID is not passed by caller, use the one we
+ * identified ourselves.
+ */
+ dirsize = scan_directory(path, de->d_name, sizeonly, tbspace,
+ db_identified ? &database : db);
+
+ /*
+ * If tablespace is not passed by caller, forget what we found out so
+ * that next directory entry is examined in the same way.
+ */
+ if (tbsp_identified)
+ tbspace = InvalidOid;
}
closedir(dir);
return dirsize;
@@ -371,6 +470,7 @@ main(int argc, char *argv[])
{"pgdata", required_argument, NULL, 'D'},
{"disable", no_argument, NULL, 'd'},
{"enable", no_argument, NULL, 'e'},
+ {"encryption-key-command", required_argument, NULL, 'K'},
{"no-sync", no_argument, NULL, 'N'},
{"progress", no_argument, NULL, 'P'},
{"verbose", no_argument, NULL, 'v'},
@@ -400,7 +500,7 @@ main(int argc, char *argv[])
}
}
- while ((c = getopt_long(argc, argv, "cD:deNPr:v", long_options, &option_index)) != -1)
+ while ((c = getopt_long(argc, argv, "cD:deNK:Pr:v", long_options, &option_index)) != -1)
{
switch (c)
{
@@ -413,6 +513,12 @@ main(int argc, char *argv[])
case 'e':
mode = PG_MODE_ENABLE;
break;
+#ifdef USE_ENCRYPTION
+ case 'K':
+ encryption_key_command = pg_strdup(optarg);
+ data_encrypted = true;
+ break;
+#endif /* USE_ENCRYPTION */
case 'N':
do_sync = false;
break;
@@ -482,6 +588,23 @@ main(int argc, char *argv[])
exit(1);
}
+ if (ControlFile->data_cipher > PG_CIPHER_NONE)
+ {
+ if (encryption_key_command == NULL)
+ {
+#ifdef USE_ENCRYPTION
+ fprintf(stderr, _("%s: please specify command to retrieve encryption key\n"),
+ progname);
+#else
+ fprintf(stderr, _("%s: compile postgres with --with-openssl to use encryption\n"),
+ progname);
+#endif /* USE_ENCRYPTION */
+ exit(1);
+ }
+
+ setup_encryption(false, DataDir);
+ }
+
if (ControlFile->pg_control_version != PG_CONTROL_VERSION)
{
pg_log_error("cluster is not compatible with this version of pg_checksums");
@@ -527,6 +650,8 @@ main(int argc, char *argv[])
/* Operate on all files if checking or enabling checksums */
if (mode == PG_MODE_CHECK || mode == PG_MODE_ENABLE)
{
+ Oid db_shared = 0;
+
/*
* If progress status information is requested, we need to scan the
* directory tree twice: once to know how much total data needs to be
@@ -534,14 +659,19 @@ main(int argc, char *argv[])
*/
if (showprogress)
{
- total_size = scan_directory(DataDir, "global", true);
- total_size += scan_directory(DataDir, "base", true);
- total_size += scan_directory(DataDir, "pg_tblspc", true);
+ total_size = scan_directory(DataDir, "global", true,
+ GLOBALTABLESPACE_OID, &db_shared);
+ total_size += scan_directory(DataDir, "base", true,
+ DEFAULTTABLESPACE_OID, NULL);
+ total_size += scan_directory(DataDir, "pg_tblspc", true,
+ InvalidOid, NULL);
}
- (void) scan_directory(DataDir, "global", false);
- (void) scan_directory(DataDir, "base", false);
- (void) scan_directory(DataDir, "pg_tblspc", false);
+ (void) scan_directory(DataDir, "global", false, GLOBALTABLESPACE_OID,
+ &db_shared);
+ (void) scan_directory(DataDir, "base", false, DEFAULTTABLESPACE_OID,
+ NULL);
+ (void) scan_directory(DataDir, "pg_tblspc", false, InvalidOid, NULL);
if (showprogress)
{
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index a674f52f0b..f39dfb50e8 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -344,5 +344,14 @@ main(int argc, char *argv[])
ControlFile->data_checksum_version);
printf(_("Mock authentication nonce: %s\n"),
mock_auth_nonce_str);
+ printf(_("Data encryption: %s\n"),
+ ControlFile->data_cipher > PG_CIPHER_NONE ? _("on") : _("off"));
+ if (ControlFile->data_cipher > PG_CIPHER_NONE)
+ printf(_("Data encryption fingerprint: %08X%08X%08X%08X\n"),
+ htonl(((uint32 *) ControlFile->encryption_verification)[0]),
+ htonl(((uint32 *) ControlFile->encryption_verification)[1]),
+ htonl(((uint32 *) ControlFile->encryption_verification)[2]),
+ htonl(((uint32 *) ControlFile->encryption_verification)[3])
+ );
return 0;
}
diff --git a/src/bin/pg_keysetup/Makefile b/src/bin/pg_keysetup/Makefile
new file mode 100644
index 0000000000..92ef5034ee
--- /dev/null
+++ b/src/bin/pg_keysetup/Makefile
@@ -0,0 +1,32 @@
+# src/bin/pg_keysetup/Makefile
+
+PGFILEDESC = "pg_keysetup - generate encryption key from a password"
+PGAPPICON=win32
+
+subdir = src/bin/pg_keysetup
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = pg_keysetup.o $(RMGRDESCOBJS) encryption.o $(WIN32RES)
+
+override CPPFLAGS := -DFRONTEND $(CPPFLAGS)
+
+all: pg_keysetup
+
+pg_keysetup: $(OBJS) | submake-libpgport
+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+
+encryption.c: % : $(top_srcdir)/src/backend/storage/file/%
+ rm -f $@ && $(LN_S) $< .
+
+install: all installdirs
+ $(INSTALL_PROGRAM) pg_keysetup$(X) '$(DESTDIR)$(bindir)/pg_keysetup$(X)'
+
+installdirs:
+ $(MKDIR_P) '$(DESTDIR)$(bindir)'
+
+uninstall:
+ rm -f '$(DESTDIR)$(bindir)/pg_keysetup$(X)'
+
+clean distclean maintainer-clean:
+ rm -f pg_keysetup$(X) $(OBJS) encryption.c
diff --git a/src/bin/pg_keysetup/pg_keysetup.c b/src/bin/pg_keysetup/pg_keysetup.c
new file mode 100644
index 0000000000..99891e6a69
--- /dev/null
+++ b/src/bin/pg_keysetup/pg_keysetup.c
@@ -0,0 +1,166 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_keysetup.c - Turn password into encryption key.
+ *
+ * Copyright (c) 2013-2017, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/bin/pg_keysetup/pg_keysetup.c
+ *-------------------------------------------------------------------------
+ */
+
+#define FRONTEND 1
+#include "postgres.h"
+
+#include <dirent.h>
+#include <unistd.h>
+
+#include "port/pg_crc32c.h"
+#include "storage/encryption.h"
+#include "getopt_long.h"
+
+#ifdef USE_ENCRYPTION
+static const char *progname;
+
+static void
+usage(const char *progname)
+{
+ printf(_("%s derives encryption key from a password.\n\n"),
+ progname);
+ printf(_("Usage:\n"));
+ printf(_(" %s [OPTION]...\n"), progname);
+ printf(_("\nOptions:\n"));
+ printf(_(" [-D] DATADIR data directory\n"));
+ printf(_(" -?, --help show this help, then exit\n\n"));
+ printf(_("Password is read from stdin and the key is sent to stdout\n"));
+}
+
+/*
+ * Big red button to push when things go horribly wrong.
+ */
+static void
+fatal_error(const char *fmt,...)
+{
+ va_list args;
+
+ fflush(stdout);
+
+ fprintf(stderr, _("%s: FATAL: "), progname);
+ va_start(args, fmt);
+ vfprintf(stderr, _(fmt), args);
+ va_end(args);
+ fputc('\n', stderr);
+
+
+ exit(EXIT_FAILURE);
+}
+#endif /* USE_ENCRYPTION */
+
+int
+main(int argc, char **argv)
+{
+/*
+ * If no encryption library is linked, let the utility fail immediately. It'd
+ * be weird if we reported incorrect usage just to say later that no useful
+ * work can be done anyway.
+ */
+#ifdef USE_ENCRYPTION
+ int c;
+ char *DataDir = NULL;
+ char password[ENCRYPTION_PWD_MAX_LENGTH];
+ size_t pwd_len,
+ i;
+
+ progname = get_progname(argv[0]);
+
+ if (argc > 1)
+ {
+ if (strcmp(argv[1], "--help") == 0 || strcmp(argv[1], "-?") == 0)
+ {
+ usage(progname);
+ exit(0);
+ }
+ if (strcmp(argv[1], "--version") == 0 || strcmp(argv[1], "-V") == 0)
+ {
+ puts("pg_controldata (PostgreSQL) " PG_VERSION);
+ exit(0);
+ }
+ }
+
+ while ((c = getopt(argc, argv, "D:")) != -1)
+ {
+ switch (c)
+ {
+ case 'D':
+ DataDir = optarg;
+ break;
+
+ default:
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
+ exit(1);
+ }
+ }
+
+ if (DataDir == NULL)
+ {
+ if (optind < argc)
+ DataDir = argv[optind++];
+ else
+ DataDir = getenv("PGDATA");
+ }
+
+ /* Complain if any arguments remain */
+ if (optind < argc)
+ {
+ fprintf(stderr, _("%s: too many command-line arguments (first is \"%s\")\n"),
+ progname, argv[optind]);
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+ if (DataDir == NULL)
+ {
+ fprintf(stderr, _("%s: no data directory specified\n"), progname);
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
+ exit(1);
+ }
+
+ read_kdf_file(DataDir);
+
+ /*
+ * Read the password.
+ */
+ pwd_len = 0;
+ while (true)
+ {
+ int c = getchar();
+
+ if (c == EOF || c == '\n')
+ break;
+
+ if (pwd_len >= ENCRYPTION_PWD_MAX_LENGTH)
+ fatal_error("The password is too long");
+
+ password[pwd_len++] = c;
+ }
+
+ if (pwd_len < ENCRYPTION_PWD_MIN_LENGTH)
+ fatal_error("The password is too short");
+
+ /*
+ * Run the key derivation function.
+ */
+ setup_encryption_key(password, false, pwd_len);
+
+ /*
+ * Finally print the encryption key.
+ */
+ for (i = 0; i < ENCRYPTION_KEY_LENGTH; i++)
+ printf("%.2x", encryption_key[i]);
+ printf("\n");
+#else
+ ENCRYPTION_NOT_SUPPORTED_MSG;
+#endif /* USE_ENCRYPTION */
+ return 0;
+}
diff --git a/src/bin/pg_resetwal/Makefile b/src/bin/pg_resetwal/Makefile
index 09ede1efd2..5d31ac5baa 100644
--- a/src/bin/pg_resetwal/Makefile
+++ b/src/bin/pg_resetwal/Makefile
@@ -15,15 +15,20 @@ subdir = src/bin/pg_resetwal
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+override CPPFLAGS := -DFRONTEND $(CPPFLAGS)
+
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils
-OBJS= pg_resetwal.o $(WIN32RES)
+OBJS= pg_resetwal.o encryption.o $(WIN32RES)
all: pg_resetwal
pg_resetwal: $(OBJS) | submake-libpgport submake-libpgfeutils
$(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+encryption.c: % : $(top_srcdir)/src/backend/storage/file/%
+ rm -f $@ && $(LN_S) $< .
+
install: all installdirs
$(INSTALL_PROGRAM) pg_resetwal$(X) '$(DESTDIR)$(bindir)/pg_resetwal$(X)'
@@ -34,7 +39,7 @@ uninstall:
rm -f '$(DESTDIR)$(bindir)/pg_resetwal$(X)'
clean distclean maintainer-clean:
- rm -f pg_resetwal$(X) $(OBJS)
+ rm -f pg_resetwal$(X) $(OBJS) encryption.c
rm -rf tmp_check
check:
diff --git a/src/bin/pg_resetwal/pg_resetwal.c b/src/bin/pg_resetwal/pg_resetwal.c
index 82a8ec993d..90bb3b63d5 100644
--- a/src/bin/pg_resetwal/pg_resetwal.c
+++ b/src/bin/pg_resetwal/pg_resetwal.c
@@ -54,11 +54,11 @@
#include "common/file_perm.h"
#include "common/restricted_token.h"
#include "fe_utils/logging.h"
+#include "storage/encryption.h"
#include "storage/large_object.h"
#include "pg_getopt.h"
#include "getopt_long.h"
-
static ControlFileData ControlFile; /* pg_control values */
static XLogSegNo newXlogSegNo; /* new XLOG segment # */
static bool guessed = false; /* T if we had to guess at any values */
@@ -135,7 +135,7 @@ main(int argc, char *argv[])
}
- while ((c = getopt_long(argc, argv, "c:D:e:fl:m:no:O:x:", long_options, NULL)) != -1)
+ while ((c = getopt_long(argc, argv, "c:D:e:fK:l:m:no:O:x:", long_options, NULL)) != -1)
{
switch (c)
{
@@ -277,6 +277,12 @@ main(int argc, char *argv[])
}
break;
+#ifdef USE_ENCRYPTION
+ case 'K':
+ encryption_key_command = strdup(optarg);
+ break;
+#endif /* USE_ENCRYPTION */
+
case 'l':
if (strspn(optarg, "01234567890ABCDEFabcdef") != XLOG_FNAME_LEN)
{
@@ -408,6 +414,16 @@ main(int argc, char *argv[])
XLogFromFileName(log_fname, &minXlogTli, &minXlogSegNo, WalSegSz);
/*
+ * If the data is encrypted, we also might need to encrypt the XLOG record
+ * below.
+ */
+ if (ControlFile.data_cipher > PG_CIPHER_NONE && !noupdate)
+ {
+ setup_encryption(false, NULL);
+ data_encrypted = true;
+ }
+
+ /*
* Also look at existing segment files to set up newXlogSegNo
*/
FindEndOfXLOG();
@@ -823,6 +839,12 @@ PrintControlValues(bool guessed)
(ControlFile.float8ByVal ? _("by value") : _("by reference")));
printf(_("Data page checksum version: %u\n"),
ControlFile.data_checksum_version);
+ if (ControlFile.data_cipher > PG_CIPHER_NONE)
+ printf(_("Data encryption fingerprint: %08X%08X%08X%08X\n"),
+ htonl(((uint32 *) ControlFile.encryption_verification)[0]),
+ htonl(((uint32 *) ControlFile.encryption_verification)[1]),
+ htonl(((uint32 *) ControlFile.encryption_verification)[2]),
+ htonl(((uint32 *) ControlFile.encryption_verification)[3]));
}
@@ -1173,6 +1195,14 @@ WriteEmptyXLOG(void)
FIN_CRC32C(crc);
record->xl_crc = crc;
+ if (data_encrypted)
+ {
+ char tweak[TWEAK_SIZE];
+
+ XLogEncryptionTweak(tweak, page->xlp_tli, newXlogSegNo, 0);
+ encrypt_block(buffer.data, buffer.data, XLOG_BLCKSZ, tweak, true);
+ }
+
/* Write the first page */
XLogFilePath(path, ControlFile.checkPointCopy.ThisTimeLineID,
newXlogSegNo, WalSegSz);
@@ -1233,6 +1263,10 @@ usage(void)
printf(_(" [-D, --pgdata=]DATADIR data directory\n"));
printf(_(" -e, --epoch=XIDEPOCH set next transaction ID epoch\n"));
printf(_(" -f, --force force update to be done\n"));
+#ifdef USE_ENCRYPTION
+ printf(_(" -K, --encryption-key-command\n"
+ " command that returns encryption key\n"));
+#endif /* USE_ENCRYPTION */
printf(_(" -l, --next-wal-file=WALFILE set minimum starting location for new WAL\n"));
printf(_(" -m, --multixact-ids=MXID,MXID set next and oldest multitransaction ID\n"));
printf(_(" -n, --dry-run no update, just show what would be done\n"));
diff --git a/src/bin/pg_rewind/Makefile b/src/bin/pg_rewind/Makefile
index 019e19986e..8b7b3fe750 100644
--- a/src/bin/pg_rewind/Makefile
+++ b/src/bin/pg_rewind/Makefile
@@ -20,7 +20,7 @@ LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
OBJS = pg_rewind.o parsexlog.o xlogreader.o datapagemap.o timeline.o \
fetch.o file_ops.o copy_fetch.o libpq_fetch.o filemap.o logging.o \
- $(WIN32RES)
+ encryption.o $(WIN32RES)
EXTRA_CLEAN = xlogreader.c
@@ -32,6 +32,9 @@ pg_rewind: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
xlogreader.c: % : $(top_srcdir)/src/backend/access/transam/%
rm -f $@ && $(LN_S) $< .
+encryption.c: % : $(top_srcdir)/src/backend/storage/file/%
+ rm -f $@ && $(LN_S) $< .
+
install: all installdirs
$(INSTALL_PROGRAM) pg_rewind$(X) '$(DESTDIR)$(bindir)/pg_rewind$(X)'
@@ -42,7 +45,7 @@ uninstall:
rm -f '$(DESTDIR)$(bindir)/pg_rewind$(X)'
clean distclean maintainer-clean:
- rm -f pg_rewind$(X) $(OBJS) xlogreader.c
+ rm -f pg_rewind$(X) $(OBJS) xlogreader.c encryption.c
rm -rf tmp_check
check:
diff --git a/src/bin/pg_rewind/parsexlog.c b/src/bin/pg_rewind/parsexlog.c
index 04a3535dfb..b185895368 100644
--- a/src/bin/pg_rewind/parsexlog.c
+++ b/src/bin/pg_rewind/parsexlog.c
@@ -23,7 +23,7 @@
#include "catalog/pg_control.h"
#include "catalog/storage_xlog.h"
#include "commands/dbcommands_xlog.h"
-
+#include "storage/encryption.h"
/*
* RmgrNames is an array of resource manager names, to make error messages
@@ -321,6 +321,15 @@ SimpleXLogPageRead(XLogReaderState *xlogreader, XLogRecPtr targetPagePtr,
return -1;
}
+ if (data_encrypted)
+ {
+ char tweak[TWEAK_SIZE];
+
+ XLogEncryptionTweak(tweak, targetHistory[private->tliIndex].tli,
+ xlogreadsegno, targetPageOff);
+ decrypt_block(readBuf, readBuf, XLOG_BLCKSZ, tweak, true);
+ }
+
Assert(targetSegNo == xlogreadsegno);
*pageTLI = targetHistory[private->tliIndex].tli;
diff --git a/src/bin/pg_rewind/pg_rewind.c b/src/bin/pg_rewind/pg_rewind.c
index e552c84964..8528c933c0 100644
--- a/src/bin/pg_rewind/pg_rewind.c
+++ b/src/bin/pg_rewind/pg_rewind.c
@@ -31,6 +31,7 @@
#include "fe_utils/logging.h"
#include "getopt_long.h"
#include "storage/bufpage.h"
+#include "storage/encryption.h"
static void usage(const char *progname);
@@ -72,6 +73,10 @@ usage(const char *progname)
printf(_(" -D, --target-pgdata=DIRECTORY existing data directory to modify\n"));
printf(_(" --source-pgdata=DIRECTORY source data directory to synchronize with\n"));
printf(_(" --source-server=CONNSTR source server to synchronize with\n"));
+#ifdef USE_ENCRYPTION
+ printf(_(" -K, --encryption-key-command=COMMAND\n"
+ " command that returns encryption key\n"));
+#endif /* USE_OPENSSL */
printf(_(" -n, --dry-run stop before modifying anything\n"));
printf(_(" -N, --no-sync do not wait for changes to be written\n"));
printf(_(" safely to disk\n"));
@@ -96,6 +101,9 @@ main(int argc, char **argv)
{"no-sync", no_argument, NULL, 'N'},
{"progress", no_argument, NULL, 'P'},
{"debug", no_argument, NULL, 3},
+#ifdef USE_ENCRYPTION
+ {"encryption-key-command", required_argument, NULL, 'K'},
+#endif /* USE_OPENSSL */
{NULL, 0, NULL, 0}
};
int option_index;
@@ -131,7 +139,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "D:nNP", long_options, &option_index)) != -1)
+ while ((c = getopt_long(argc, argv, "D:K:nNP", long_options, &option_index)) != -1)
{
switch (c)
{
@@ -166,6 +174,12 @@ main(int argc, char **argv)
case 2: /* --source-server */
connstr_source = pg_strdup(optarg);
break;
+#ifdef USE_ENCRYPTION
+ case 4: /* --encryption-key-command */
+ case 'K':
+ encryption_key_command = strdup(optarg);
+ break;
+#endif /* USE_OPENSSL */
}
}
@@ -245,6 +259,21 @@ main(int argc, char **argv)
sanityChecks();
/*
+ * Setup encryption if it's obvious that we'll have to deal with encrypted
+ * XLOG.
+ */
+ if (ControlFile_target.data_cipher > PG_CIPHER_NONE)
+ {
+ /*
+ * It should not matter whether we pass the source or target data
+ * directory. It should have been checked earlier that both clusters
+ * are encrypted using the same key.
+ */
+ setup_encryption(false, datadir_source);
+ data_encrypted = true;
+ }
+
+ /*
* If both clusters are already on the same timeline, there's nothing to
* do.
*/
@@ -443,6 +472,24 @@ sanityChecks(void)
ControlFile_source.state != DB_SHUTDOWNED &&
ControlFile_source.state != DB_SHUTDOWNED_IN_RECOVERY)
pg_fatal("source data directory must be shut down cleanly");
+
+ /*
+ * Since slave receives XLOG stream encrypted by master, handling
+ * differently encrypted clusters is not the typical use case for
+ * pg_rewind. Yet we should check the encryption.
+ */
+ if (ControlFile_source.data_cipher > PG_CIPHER_NONE ||
+ ControlFile_target.data_cipher > PG_CIPHER_NONE)
+ {
+ if (ControlFile_source.data_cipher !=
+ ControlFile_target.data_cipher)
+ pg_fatal("source and target server must be both unencrypted or both encrypted\n");
+
+ if (memcmp(ControlFile_source.encryption_verification,
+ ControlFile_target.encryption_verification,
+ ENCRYPTION_SAMPLE_SIZE))
+ pg_fatal("both source and target server must use the same encryption key");
+ }
}
/*
diff --git a/src/bin/pg_upgrade/Makefile b/src/bin/pg_upgrade/Makefile
index 5a18948425..99e7846b4d 100644
--- a/src/bin/pg_upgrade/Makefile
+++ b/src/bin/pg_upgrade/Makefile
@@ -9,9 +9,9 @@ include $(top_builddir)/src/Makefile.global
OBJS = check.o controldata.o dump.o exec.o file.o function.o info.o \
option.o parallel.o pg_upgrade.o relfilenode.o server.o \
- tablespace.o util.o version.o $(WIN32RES)
+ tablespace.o util.o version.o encryption.o $(WIN32RES)
-override CPPFLAGS := -DDLSUFFIX=\"$(DLSUFFIX)\" -I$(srcdir) -I$(libpq_srcdir) $(CPPFLAGS)
+override CPPFLAGS := -DDLSUFFIX=\"$(DLSUFFIX)\" -I$(srcdir) -I$(libpq_srcdir) -DFRONTEND $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
ifdef NO_TEMP_INSTALL
@@ -29,6 +29,9 @@ all: pg_upgrade
pg_upgrade: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
$(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+encryption.c: % : $(top_srcdir)/src/backend/storage/file/%
+ rm -f $@ && $(LN_S) $< .
+
install: all installdirs
$(INSTALL_PROGRAM) pg_upgrade$(X) '$(DESTDIR)$(bindir)/pg_upgrade$(X)'
@@ -43,7 +46,8 @@ clean distclean maintainer-clean:
rm -rf analyze_new_cluster.sh delete_old_cluster.sh log/ tmp_check/ \
loadable_libraries.txt reindex_hash.sql \
pg_upgrade_dump_globals.sql \
- pg_upgrade_dump_*.custom pg_upgrade_*.log
+ pg_upgrade_dump_*.custom pg_upgrade_*.log \
+ encryption.c
check: test.sh all
MAKE=$(MAKE) bindir="$(tbindir)" libdir="$(tlibdir)" EXTRA_REGRESS_OPTS="$(EXTRA_REGRESS_OPTS)" $(SHELL) $< $(DOINST)
diff --git a/src/bin/pg_upgrade/check.c b/src/bin/pg_upgrade/check.c
index fc5aa7010f..11f2d036f9 100644
--- a/src/bin/pg_upgrade/check.c
+++ b/src/bin/pg_upgrade/check.c
@@ -12,11 +12,13 @@
#include "catalog/pg_authid_d.h"
#include "fe_utils/string_utils.h"
#include "mb/pg_wchar.h"
+#include "storage/encryption.h"
#include "pg_upgrade.h"
static void check_new_cluster_is_empty(void);
static void check_databases_are_compatible(void);
+static void get_encryption_key_command(ClusterInfo *cluster);
static void check_locale_and_encoding(DbInfo *olddb, DbInfo *newdb);
static bool equivalent_locale(int category, const char *loca, const char *locb);
static void check_is_install_user(ClusterInfo *cluster);
@@ -307,6 +309,92 @@ check_cluster_compatibility(bool live_check)
"the old and new port numbers must be different.\n");
}
+/*
+ * Check if both clusters are encrypted using the same key.
+ *
+ * One extra start / stop cycle of each cluster is needed to retrieve the
+ * encryption_key_command, which we'll be needed by
+ * check_cluster_compatibility(). XXX Should we get the value from
+ * postgresql.conf (note that it can be located outside the data directory)?
+ */
+void
+check_encryption_keys(void)
+{
+ char *old_key,
+ *new_key;
+ bool is_key;
+ size_t old_key_len,
+ new_key_len;
+
+ start_postmaster(&old_cluster, true);
+ get_encryption_key_command(&old_cluster);
+ stop_postmaster(false);
+
+ old_key = run_encryption_key_command(old_cluster.encryption_key_command,
+ &is_key, &old_key_len);
+
+ /*
+ * pg_upgrade only accepts encryption key since different keys should be
+ * derived from the same password on different systems anyway.
+ */
+ if (!is_key)
+ encryption_error(true,
+ "The old cluster uses encryption password instead of encryption key");
+
+ /*
+ * The same for the new cluster.
+ */
+ start_postmaster(&new_cluster, true);
+ get_encryption_key_command(&new_cluster);
+ stop_postmaster(false);
+ new_key = run_encryption_key_command(new_cluster.encryption_key_command,
+ &is_key, &new_key_len);
+ if (!is_key)
+ encryption_error(true,
+ "The new cluster uses encryption password instead of encryption key");
+
+ /*
+ * Naturally, both clusters must use the same key, else the files we don't
+ * re-encrypt will not be usable by the new cluster.
+ */
+ if (old_key_len != new_key_len ||
+ memcmp(old_key, new_key, new_key_len) != 0)
+ encryption_error(true, "The clusters use different encryption key");
+}
+
+/*
+ * get_encryption_key_command()
+ *
+ * Retrieve the value of encryption_key_command configuration variable.
+ */
+static void
+get_encryption_key_command(ClusterInfo *cluster)
+{
+ PGresult *res;
+ PGconn *conn = connectToServer(cluster, "template1");
+
+ prep_status("Retrieving encryption_key_command");
+
+ res = executeQueryOrDie(conn,
+ "SELECT setting "
+ "FROM pg_catalog.pg_settings "
+ "WHERE name = 'encryption_key_command'");
+
+ if (PQntuples(res) != 0)
+ {
+ int i_setting = PQfnumber(res, "setting");
+ char *setting = PQgetvalue(res, 0, i_setting);
+
+ cluster->encryption_key_command = setting != NULL ?
+ pg_strdup(setting) : NULL;
+ }
+
+ PQclear(res);
+
+ PQfinish(conn);
+
+ check_ok();
+}
/*
* check_locale_and_encoding()
diff --git a/src/bin/pg_upgrade/controldata.c b/src/bin/pg_upgrade/controldata.c
index 163a3f8c9e..b04d7b9b26 100644
--- a/src/bin/pg_upgrade/controldata.c
+++ b/src/bin/pg_upgrade/controldata.c
@@ -8,6 +8,7 @@
*/
#include "postgres_fe.h"
+#include "catalog/pg_control.h"
#include "pg_upgrade.h"
@@ -486,6 +487,33 @@ get_control_data(ClusterInfo *cluster, bool live_check)
cluster->controldata.data_checksum_version = str2uint(p);
got_data_checksum_version = true;
}
+ else if ((p = strstr(bufin, "encryption fingerprint")) != NULL)
+ {
+ int i;
+
+ p = strchr(p, ':');
+
+ if (p == NULL || strlen(p) <= 1)
+ pg_fatal("%d: controldata retrieval problem\n", __LINE__);
+
+ cluster->controldata.data_encrypted = true;
+
+ /* Skip the colon and any whitespace after it */
+ p = strchr(p, ':');
+ if (p == NULL || strlen(p) <= 1)
+ pg_fatal("%d: controldata retrieval problem\n", __LINE__);
+ p = strpbrk(p, "01234567890ABCDEF");
+ if (p == NULL || strlen(p) <= 1)
+ pg_fatal("%d: controldata retrieval problem\n", __LINE__);
+
+ /* Make sure it looks like a valid finerprint */
+ if (strspn(p, "0123456789ABCDEF") != 32)
+ pg_fatal("%d: controldata retrieval problem\n", __LINE__);
+
+ for (i = 0; i < ENCRYPTION_SAMPLE_SIZE; i++)
+ sscanf(p + 2 * i, "%2hhx",
+ cluster->controldata.encryption_verification + i);
+ }
}
pclose(output);
@@ -670,6 +698,18 @@ check_control_data(ControlData *oldctrl,
pg_fatal("old cluster uses data checksums but the new one does not\n");
else if (oldctrl->data_checksum_version != newctrl->data_checksum_version)
pg_fatal("old and new cluster pg_controldata checksum versions do not match\n");
+
+ if (oldctrl->data_encrypted && !newctrl->data_encrypted)
+ pg_fatal("old cluster is encrypted, but the new one is not\n");
+ else if (!oldctrl->data_encrypted && newctrl->data_encrypted)
+ pg_fatal("old cluster is not encrypted, but the new one is\n");
+ else if (oldctrl->data_encrypted)
+ {
+ if (memcmp(oldctrl->encryption_verification,
+ newctrl->encryption_verification,
+ ENCRYPTION_SAMPLE_SIZE) != 0)
+ pg_fatal("encryption of the new cluster is not compatible with encryption of the old one\n");
+ }
}
diff --git a/src/bin/pg_upgrade/file.c b/src/bin/pg_upgrade/file.c
index c7fed24df9..6443b65261 100644
--- a/src/bin/pg_upgrade/file.c
+++ b/src/bin/pg_upgrade/file.c
@@ -13,6 +13,7 @@
#include "common/file_perm.h"
#include "pg_upgrade.h"
#include "storage/bufpage.h"
+#include "storage/encryption.h"
#include "storage/checksum.h"
#include "storage/checksum_impl.h"
@@ -78,15 +79,20 @@ cloneFile(const char *src, const char *dst,
*
* Copies a relation file from src to dst.
* schemaName/relName are relation's SQL name (used for error messages only).
+ *
+ * Re-encrypt each block in order to handle change of relfilenode.
*/
void
-copyFile(const char *src, const char *dst,
- const char *schemaName, const char *relName)
+copyFile(const char *src, RelFileNode *src_relnode,
+ const char *dst, RelFileNode *dst_relnode,
+ ForkNumber forknum, int segno, const char *schemaName,
+ const char *relName)
{
#ifndef WIN32
int src_fd;
int dest_fd;
char *buffer;
+ BlockNumber block_num = segno * RELSEG_SIZE;
if ((src_fd = open(src, O_RDONLY | PG_BINARY, 0)) < 0)
pg_fatal("error while copying relation \"%s.%s\": could not open file \"%s\": %s\n",
@@ -102,20 +108,55 @@ copyFile(const char *src, const char *dst,
buffer = (char *) pg_malloc(COPY_BUF_SIZE);
- /* perform data copying i.e read src source, write to destination */
+ /*
+ * Perform data copying i.e read source, write to destination. As the file
+ * can be encrypted, we only handle whole pages.
+ */
while (true)
{
- ssize_t nbytes = read(src_fd, buffer, COPY_BUF_SIZE);
+ ssize_t nbytes_total = 0;
+
+ while (nbytes_total < COPY_BUF_SIZE)
+ {
+ ssize_t nbytes;
+
+ nbytes = read(src_fd, buffer + nbytes_total, COPY_BUF_SIZE - nbytes_total);
+ nbytes_total += nbytes;
+
+ if (nbytes < 0)
+ pg_fatal("error while copying relation \"%s.%s\": could not read file \"%s\": %s\n",
+ schemaName, relName, src, strerror(errno));
+
+ if (nbytes == 0)
+ break;
+ }
- if (nbytes < 0)
+ /*
+ * The buffer is not necessarily full but the data must end at page
+ * boundary.
+ */
+ if ((nbytes_total % BLCKSZ) != 0)
pg_fatal("error while copying relation \"%s.%s\": could not read file \"%s\": %s\n",
schemaName, relName, src, strerror(errno));
- if (nbytes == 0)
+ if (nbytes_total == 0)
break;
+ /* Re-encrypt the block(s) if copying changes encryption tweak. */
+ if (encryption_enabled &&
+ (src_relnode->spcNode != dst_relnode->spcNode ||
+ src_relnode->dbNode != dst_relnode->dbNode ||
+ src_relnode->relNode != dst_relnode->relNode))
+ block_num = ReencryptBlock(buffer,
+ nbytes_total / BLCKSZ,
+ src_relnode,
+ dst_relnode,
+ forknum,
+ forknum,
+ block_num);
+
errno = 0;
- if (write(dest_fd, buffer, nbytes) != nbytes)
+ if (write(dest_fd, buffer, nbytes_total) != nbytes_total)
{
/* if write didn't set errno, assume problem is no disk space */
if (errno == 0)
@@ -130,7 +171,7 @@ copyFile(const char *src, const char *dst,
close(dest_fd);
#else /* WIN32 */
-
+ /* TODO Re-encrypt if needed. */
if (CopyFile(src, dst, true) == 0)
{
_dosmaperr(GetLastError());
diff --git a/src/bin/pg_upgrade/info.c b/src/bin/pg_upgrade/info.c
index 902bfc647e..4500a14a26 100644
--- a/src/bin/pg_upgrade/info.c
+++ b/src/bin/pg_upgrade/info.c
@@ -185,6 +185,10 @@ create_rel_filename_map(const char *old_data, const char *new_data,
map->old_tablespace = old_rel->tablespace;
map->old_tablespace_suffix = old_cluster.tablespace_suffix;
}
+ if (old_rel->tablespace_oid != InvalidOid)
+ map->old_tablespace_oid = old_rel->tablespace_oid;
+ else
+ map->old_tablespace_oid = old_db->db_tablespace_oid;
/* Do the same for new tablespaces */
if (strlen(new_rel->tablespace) == 0)
@@ -197,6 +201,10 @@ create_rel_filename_map(const char *old_data, const char *new_data,
map->new_tablespace = new_rel->tablespace;
map->new_tablespace_suffix = new_cluster.tablespace_suffix;
}
+ if (new_rel->tablespace_oid != InvalidOid)
+ map->new_tablespace_oid = new_rel->tablespace_oid;
+ else
+ map->new_tablespace_oid = new_db->db_tablespace_oid;
map->old_db_oid = old_db->db_oid;
map->new_db_oid = new_db->db_oid;
@@ -351,12 +359,13 @@ get_db_infos(ClusterInfo *cluster)
i_encoding,
i_datcollate,
i_datctype,
- i_spclocation;
+ i_spclocation,
+ i_tablespace;
char query[QUERY_ALLOC];
snprintf(query, sizeof(query),
"SELECT d.oid, d.datname, d.encoding, d.datcollate, d.datctype, "
- "%s AS spclocation "
+ "%s AS spclocation, d.dattablespace "
"FROM pg_catalog.pg_database d "
" LEFT OUTER JOIN pg_catalog.pg_tablespace t "
" ON d.dattablespace = t.oid "
@@ -375,6 +384,7 @@ get_db_infos(ClusterInfo *cluster)
i_datcollate = PQfnumber(res, "datcollate");
i_datctype = PQfnumber(res, "datctype");
i_spclocation = PQfnumber(res, "spclocation");
+ i_tablespace = PQfnumber(res, "dattablespace");
ntups = PQntuples(res);
dbinfos = (DbInfo *) pg_malloc(sizeof(DbInfo) * ntups);
@@ -388,6 +398,7 @@ get_db_infos(ClusterInfo *cluster)
dbinfos[tupnum].db_ctype = pg_strdup(PQgetvalue(res, tupnum, i_datctype));
snprintf(dbinfos[tupnum].db_tablespace, sizeof(dbinfos[tupnum].db_tablespace), "%s",
PQgetvalue(res, tupnum, i_spclocation));
+ dbinfos[tupnum].db_tablespace_oid = atooid(PQgetvalue(res, tupnum, i_tablespace));
}
PQclear(res);
@@ -571,7 +582,8 @@ get_rel_infos(ClusterInfo *cluster, DbInfo *dbinfo)
curr->tblsp_alloc = false;
/* Is the tablespace oid non-default? */
- if (atooid(PQgetvalue(res, relnum, i_reltablespace)) != 0)
+ curr->tablespace_oid = atooid(PQgetvalue(res, relnum, i_reltablespace));
+ if (curr->tablespace_oid != 0)
{
/*
* The tablespace location might be "", meaning the cluster
@@ -655,9 +667,10 @@ print_rel_infos(RelInfoArr *rel_arr)
int relnum;
for (relnum = 0; relnum < rel_arr->nrels; relnum++)
- pg_log(PG_VERBOSE, "relname: %s.%s: reloid: %u reltblspace: %s\n",
+ pg_log(PG_VERBOSE, "relname: %s.%s: reloid: %u reltblspace: %s reltblspaceoid: %u\n",
rel_arr->rels[relnum].nspname,
rel_arr->rels[relnum].relname,
rel_arr->rels[relnum].reloid,
- rel_arr->rels[relnum].tablespace);
+ rel_arr->rels[relnum].tablespace,
+ rel_arr->rels[relnum].tablespace_oid);
}
diff --git a/src/bin/pg_upgrade/pg_upgrade.c b/src/bin/pg_upgrade/pg_upgrade.c
index 0b304bbd56..402e4dff3c 100644
--- a/src/bin/pg_upgrade/pg_upgrade.c
+++ b/src/bin/pg_upgrade/pg_upgrade.c
@@ -36,13 +36,14 @@
#include "postgres_fe.h"
-
#include "pg_upgrade.h"
+
#include "catalog/pg_class_d.h"
#include "common/file_perm.h"
#include "common/restricted_token.h"
#include "fe_utils/logging.h"
#include "fe_utils/string_utils.h"
+#include "storage/encryption.h"
#ifdef HAVE_LANGINFO_H
#include <langinfo.h>
@@ -71,6 +72,7 @@ char *output_files[] = {
NULL
};
+bool encryption_enabled = false;
int
main(int argc, char **argv)
@@ -78,6 +80,7 @@ main(int argc, char **argv)
char *analyze_script_file_name = NULL;
char *deletion_script_file_name = NULL;
bool live_check = false;
+ char keycmd_opt_str[MAX_STRING];
pg_logging_init(argv[0]);
set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_upgrade"));
@@ -103,6 +106,28 @@ main(int argc, char **argv)
check_cluster_compatibility(live_check);
+ /*
+ * If encryption is in place, we expect that some files will need
+ * re-encryption due to change of their RelFileNode, so neither link nor
+ * clone helps. (We might copy files that need re-encryption and link the
+ * others, but not sure it's worth the effort.)
+ *
+ * By now we've checked that the clusters are both encrypted or both
+ * unencrypted, so check data_encrypted of any one of them.
+ */
+ if (old_cluster.controldata.data_encrypted)
+ {
+ if (user_opts.transfer_mode == TRANSFER_MODE_LINK)
+ pg_fatal("link mode cannot be used for encrypted instance\n");
+ else if (user_opts.transfer_mode == TRANSFER_MODE_CLONE)
+ pg_fatal("clone mode cannot be used for encrypted instance\n");
+
+ /*
+ * Encryption key will be needed for file re-encryption.
+ */
+ check_encryption_keys();
+ }
+
/* Set mask based on PGDATA permissions */
if (!GetDataDirectoryCreatePerm(new_cluster.pgdata))
{
@@ -133,7 +158,15 @@ main(int argc, char **argv)
/*
* Destructive Changes to New Cluster
+ *
+ * We'll deal with encrypted files, so prepare for it.
*/
+ if (new_cluster.encryption_key_command)
+ {
+ encryption_key_command = pg_strdup(new_cluster.encryption_key_command);
+ setup_encryption(false, NULL);
+ encryption_enabled = true;
+ }
copy_xact_xlog_xid();
@@ -160,6 +193,12 @@ main(int argc, char **argv)
transfer_all_new_tablespaces(&old_cluster.dbarr, &new_cluster.dbarr,
old_cluster.pgdata, new_cluster.pgdata);
+ if (new_cluster.encryption_key_command)
+ snprintf(keycmd_opt_str, sizeof(keycmd_opt_str),
+ " -K %s", new_cluster.encryption_key_command);
+ else
+ keycmd_opt_str[0] = '\0';
+
/*
* Assuming OIDs are only used in system tables, there is no need to
* restore the OID counter because we have not transferred any OIDs from
@@ -168,8 +207,10 @@ main(int argc, char **argv)
*/
prep_status("Setting next OID for new cluster");
exec_prog(UTILITY_LOG_FILE, NULL, true, true,
- "\"%s/pg_resetwal\" -o %u \"%s\"",
- new_cluster.bindir, old_cluster.controldata.chkpnt_nxtoid,
+ "\"%s/pg_resetwal\"%s -o %u \"%s\"",
+ new_cluster.bindir,
+ keycmd_opt_str,
+ old_cluster.controldata.chkpnt_nxtoid,
new_cluster.pgdata);
check_ok();
@@ -461,6 +502,8 @@ copy_subdir_files(const char *old_subdir, const char *new_subdir)
static void
copy_xact_xlog_xid(void)
{
+ char keycmd_opt_str[MAX_STRING];
+
/*
* Copy old commit logs to new data dir. pg_clog has been renamed to
* pg_xact in post-10 clusters.
@@ -470,20 +513,31 @@ copy_xact_xlog_xid(void)
GET_MAJOR_VERSION(new_cluster.major_version) < 1000 ?
"pg_clog" : "pg_xact");
+ if (new_cluster.encryption_key_command)
+ snprintf(keycmd_opt_str, sizeof(keycmd_opt_str),
+ " -K %s", new_cluster.encryption_key_command);
+ else
+ keycmd_opt_str[0] = '\0';
+
/* set the next transaction id and epoch of the new cluster */
prep_status("Setting next transaction ID and epoch for new cluster");
exec_prog(UTILITY_LOG_FILE, NULL, true, true,
- "\"%s/pg_resetwal\" -f -x %u \"%s\"",
- new_cluster.bindir, old_cluster.controldata.chkpnt_nxtxid,
+ "\"%s/pg_resetwal\"%s -f -x %u \"%s\"",
+ new_cluster.bindir,
+ keycmd_opt_str,
+ old_cluster.controldata.chkpnt_nxtxid,
new_cluster.pgdata);
exec_prog(UTILITY_LOG_FILE, NULL, true, true,
- "\"%s/pg_resetwal\" -f -e %u \"%s\"",
- new_cluster.bindir, old_cluster.controldata.chkpnt_nxtepoch,
+ "\"%s/pg_resetwal\"%s -f -e %u \"%s\"",
+ new_cluster.bindir,
+ keycmd_opt_str,
+ old_cluster.controldata.chkpnt_nxtepoch,
new_cluster.pgdata);
/* must reset commit timestamp limits also */
exec_prog(UTILITY_LOG_FILE, NULL, true, true,
- "\"%s/pg_resetwal\" -f -c %u,%u \"%s\"",
+ "\"%s/pg_resetwal\"%s -f -c %u,%u \"%s\"",
new_cluster.bindir,
+ keycmd_opt_str,
old_cluster.controldata.chkpnt_nxtxid,
old_cluster.controldata.chkpnt_nxtxid,
new_cluster.pgdata);
@@ -508,8 +562,9 @@ copy_xact_xlog_xid(void)
* counters here and the oldest multi present on system.
*/
exec_prog(UTILITY_LOG_FILE, NULL, true, true,
- "\"%s/pg_resetwal\" -O %u -m %u,%u \"%s\"",
+ "\"%s/pg_resetwal\"%s -O %u -m %u,%u \"%s\"",
new_cluster.bindir,
+ keycmd_opt_str,
old_cluster.controldata.chkpnt_nxtmxoff,
old_cluster.controldata.chkpnt_nxtmulti,
old_cluster.controldata.chkpnt_oldstMulti,
@@ -536,8 +591,9 @@ copy_xact_xlog_xid(void)
* next=MaxMultiXactId, but multixact.c can cope with that just fine.
*/
exec_prog(UTILITY_LOG_FILE, NULL, true, true,
- "\"%s/pg_resetwal\" -m %u,%u \"%s\"",
+ "\"%s/pg_resetwal\"%s -m %u,%u \"%s\"",
new_cluster.bindir,
+ keycmd_opt_str,
old_cluster.controldata.chkpnt_nxtmulti + 1,
old_cluster.controldata.chkpnt_nxtmulti,
new_cluster.pgdata);
@@ -548,7 +604,8 @@ copy_xact_xlog_xid(void)
prep_status("Resetting WAL archives");
exec_prog(UTILITY_LOG_FILE, NULL, true, true,
/* use timeline 1 to match controldata and no WAL history file */
- "\"%s/pg_resetwal\" -l 00000001%s \"%s\"", new_cluster.bindir,
+ "\"%s/pg_resetwal\"%s -l 00000001%s \"%s\"", new_cluster.bindir,
+ keycmd_opt_str,
old_cluster.controldata.nextxlogfile + 8,
new_cluster.pgdata);
check_ok();
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index baeb8ff0f8..757047360b 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -11,6 +11,8 @@
#include <sys/time.h>
#include "libpq-fe.h"
+#include "catalog/pg_control.h"
+#include "storage/relfilenode.h"
/* Use port in the private/dynamic port number range */
#define DEF_PGUPORT 50432
@@ -145,6 +147,7 @@ typedef struct
Oid indtable; /* if index, OID of its table, else 0 */
Oid toastheap; /* if toast table, OID of base table, else 0 */
char *tablespace; /* tablespace path; "" for cluster default */
+ Oid tablespace_oid; /* tablespace OID */
bool nsp_alloc; /* should nspname be freed? */
bool tblsp_alloc; /* should tablespace be freed? */
int32 relpages; /* # of pages -- see pg_class.h */
@@ -164,6 +167,8 @@ typedef struct
{
const char *old_tablespace;
const char *new_tablespace;
+ Oid old_tablespace_oid;
+ Oid new_tablespace_oid;
const char *old_tablespace_suffix;
const char *new_tablespace_suffix;
Oid old_db_oid;
@@ -193,6 +198,7 @@ typedef struct
char *db_name; /* database name */
char db_tablespace[MAXPGPATH]; /* database default tablespace
* path */
+ Oid db_tablespace_oid;
char *db_collate;
char *db_ctype;
int db_encoding;
@@ -233,6 +239,8 @@ typedef struct
bool date_is_int;
bool float8_pass_by_value;
bool data_checksum_version;
+ bool data_encrypted;
+ uint8 encryption_verification[ENCRYPTION_SAMPLE_SIZE];
} ControlData;
/*
@@ -282,6 +290,8 @@ typedef struct
char major_version_str[64]; /* string PG_VERSION of cluster */
uint32 bin_version; /* version returned from pg_ctl */
const char *tablespace_suffix; /* directory specification */
+ const char *encryption_key_command; /* command to retrieve encryption key
+ * or passphrase */
} ClusterInfo;
@@ -339,6 +349,7 @@ extern UserOpts user_opts;
extern ClusterInfo old_cluster,
new_cluster;
extern OSInfo os_info;
+extern bool encryption_enabled;
/* check.c */
@@ -352,6 +363,7 @@ void output_completion_banner(char *analyze_script_file_name,
char *deletion_script_file_name);
void check_cluster_versions(void);
void check_cluster_compatibility(bool live_check);
+void check_encryption_keys(void);
void create_script_for_old_cluster_deletion(char **deletion_script_file_name);
void create_script_for_cluster_analyze(char **analyze_script_file_name);
@@ -382,8 +394,10 @@ bool pid_lock_file_exists(const char *datadir);
void cloneFile(const char *src, const char *dst,
const char *schemaName, const char *relName);
-void copyFile(const char *src, const char *dst,
- const char *schemaName, const char *relName);
+void copyFile(const char *src, RelFileNode *src_relnode,
+ const char *dst, RelFileNode *dst_relnode,
+ ForkNumber forknum, int segno, const char *schemaName,
+ const char *relName);
void linkFile(const char *src, const char *dst,
const char *schemaName, const char *relName);
void rewriteVisibilityMap(const char *fromfile, const char *tofile,
diff --git a/src/bin/pg_upgrade/relfilenode.c b/src/bin/pg_upgrade/relfilenode.c
index dd3c8cefe4..7c4c9046d3 100644
--- a/src/bin/pg_upgrade/relfilenode.c
+++ b/src/bin/pg_upgrade/relfilenode.c
@@ -14,11 +14,13 @@
#include <sys/stat.h>
#include "catalog/pg_class_d.h"
#include "access/transam.h"
+#include "storage/encryption.h"
#include "storage/freespace.h"
static void transfer_single_new_db(FileNameMap *maps, int size, char *old_tablespace);
-static void transfer_relfile(FileNameMap *map, const char *suffix, bool vm_must_add_frozenbit);
+static void transfer_relfile(FileNameMap *map, const char *suffix, ForkNumber forknum,
+ bool vm_must_add_frozenbit);
static bool new_cluster_needs_fsm(FileNameMap *map);
@@ -168,7 +170,8 @@ transfer_single_new_db(FileNameMap *maps, int size, char *old_tablespace)
strcmp(maps[mapnum].old_tablespace, old_tablespace) == 0)
{
/* transfer primary file */
- transfer_relfile(&maps[mapnum], "", vm_must_add_frozenbit);
+ transfer_relfile(&maps[mapnum], "", MAIN_FORKNUM,
+ vm_must_add_frozenbit);
/* fsm/vm files added in PG 8.4 */
if (GET_MAJOR_VERSION(old_cluster.major_version) >= 804)
@@ -177,9 +180,10 @@ transfer_single_new_db(FileNameMap *maps, int size, char *old_tablespace)
* Copy/link any fsm and vm files, if they exist
*/
if (new_cluster_needs_fsm(&maps[mapnum]))
- transfer_relfile(&maps[mapnum], "_fsm", vm_must_add_frozenbit);
+ transfer_relfile(&maps[mapnum], "_fsm", FSM_FORKNUM, vm_must_add_frozenbit);
if (vm_crashsafe_match)
- transfer_relfile(&maps[mapnum], "_vm", vm_must_add_frozenbit);
+ transfer_relfile(&maps[mapnum], "_vm", VISIBILITYMAP_FORKNUM,
+ vm_must_add_frozenbit);
}
}
}
@@ -194,7 +198,8 @@ transfer_single_new_db(FileNameMap *maps, int size, char *old_tablespace)
* mode.
*/
static void
-transfer_relfile(FileNameMap *map, const char *type_suffix, bool vm_must_add_frozenbit)
+transfer_relfile(FileNameMap *map, const char *type_suffix, ForkNumber forknum,
+ bool vm_must_add_frozenbit)
{
char old_file[MAXPGPATH];
char new_file[MAXPGPATH];
@@ -256,6 +261,14 @@ transfer_relfile(FileNameMap *map, const char *type_suffix, bool vm_must_add_fro
if (vm_must_add_frozenbit && strcmp(type_suffix, "_vm") == 0)
{
+ /*
+ * Encryption is not supported for the old versions that need to
+ * rewrite the visibility map. Something bad must have happened if
+ * this condition is met.
+ */
+ if (encryption_enabled)
+ pg_fatal("the old cluster is too old to be encrypted\n");
+
/* Need to rewrite visibility map format */
pg_log(PG_VERBOSE, "rewriting \"%s\" to \"%s\"\n",
old_file, new_file);
@@ -270,9 +283,24 @@ transfer_relfile(FileNameMap *map, const char *type_suffix, bool vm_must_add_fro
cloneFile(old_file, new_file, map->nspname, map->relname);
break;
case TRANSFER_MODE_COPY:
- pg_log(PG_VERBOSE, "copying \"%s\" to \"%s\"\n",
- old_file, new_file);
- copyFile(old_file, new_file, map->nspname, map->relname);
+ {
+ RelFileNode old_relnode,
+ new_relnode;
+
+ pg_log(PG_VERBOSE, "copying \"%s\" to \"%s\"\n",
+ old_file, new_file);
+
+ old_relnode.spcNode = map->old_tablespace_oid;
+ old_relnode.dbNode = map->old_db_oid;
+ old_relnode.relNode = map->old_relfilenode;
+ new_relnode.spcNode = map->new_tablespace_oid;
+ new_relnode.dbNode = map->new_db_oid;
+ new_relnode.relNode = map->new_relfilenode;
+
+ copyFile(old_file, &old_relnode,
+ new_file, &new_relnode, forknum,
+ segno, map->nspname, map->relname);
+ }
break;
case TRANSFER_MODE_LINK:
pg_log(PG_VERBOSE, "linking \"%s\" to \"%s\"\n",
diff --git a/src/bin/pg_waldump/Makefile b/src/bin/pg_waldump/Makefile
index bfe9498b80..e673d6fdd6 100644
--- a/src/bin/pg_waldump/Makefile
+++ b/src/bin/pg_waldump/Makefile
@@ -7,7 +7,7 @@ subdir = src/bin/pg_waldump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-OBJS = pg_waldump.o compat.o xlogreader.o rmgrdesc.o \
+OBJS = pg_waldump.o compat.o xlogreader.o rmgrdesc.o encryption.o \
$(RMGRDESCOBJS) $(WIN32RES)
override CPPFLAGS := -DFRONTEND $(CPPFLAGS)
@@ -28,6 +28,9 @@ xlogreader.c: % : $(top_srcdir)/src/backend/access/transam/%
$(RMGRDESCSOURCES): % : $(top_srcdir)/src/backend/access/rmgrdesc/%
rm -f $@ && $(LN_S) $< .
+encryption.c: % : $(top_srcdir)/src/backend/storage/file/%
+ rm -f $@ && $(LN_S) $< .
+
install: all installdirs
$(INSTALL_PROGRAM) pg_waldump$(X) '$(DESTDIR)$(bindir)/pg_waldump$(X)'
@@ -38,4 +41,5 @@ uninstall:
rm -f '$(DESTDIR)$(bindir)/pg_waldump$(X)'
clean distclean maintainer-clean:
- rm -f pg_waldump$(X) $(OBJS) $(RMGRDESCSOURCES) xlogreader.c
+ rm -f pg_waldump$(X) $(OBJS) $(RMGRDESCSOURCES) xlogreader.c \
+encryption.c
diff --git a/src/bin/pg_waldump/pg_waldump.c b/src/bin/pg_waldump/pg_waldump.c
index d37e9f0817..792ac9fed3 100644
--- a/src/bin/pg_waldump/pg_waldump.c
+++ b/src/bin/pg_waldump/pg_waldump.c
@@ -22,6 +22,7 @@
#include "access/transam.h"
#include "common/fe_memutils.h"
#include "fe_utils/logging.h"
+#include "storage/encryption.h"
#include "getopt_long.h"
#include "rmgrdesc.h"
@@ -194,27 +195,59 @@ search_directory(const char *directory, const char *fname)
PGAlignedXLogBlock buf;
int r;
- r = read(fd, buf.data, XLOG_BLCKSZ);
- if (r == XLOG_BLCKSZ)
+ if (data_encrypted)
{
- XLogLongPageHeader longhdr = (XLogLongPageHeader) buf.data;
-
- WalSegSz = longhdr->xlp_seg_size;
+ /*
+ * Segment size affects calculation of segNo and thus also the
+ * encryption tweak, so we cannot get the size from the header
+ * until the page is decrypted. Here we need to take more
+ * expensive approach and really check the file size.
+ */
+ WalSegSz = (int) lseek(fd, 0, SEEK_END);
+ if (WalSegSz <= 0)
+ fatal_error("Could not determine size of WAL segment \"%s\"", fname);
+ /*
+ * Verification of the file size is the only useful thing we can
+ * do. If anything else is wrong, the XLOG reader should find out
+ * after decryption.
+ */
if (!IsValidWalSegSize(WalSegSz))
- fatal_error(ngettext("WAL segment size must be a power of two between 1 MB and 1 GB, but the WAL file \"%s\" header specifies %d byte",
- "WAL segment size must be a power of two between 1 MB and 1 GB, but the WAL file \"%s\" header specifies %d bytes",
+ fatal_error(ngettext("WAL segment size must be a power of two between 1 MB and 1 GB, but size of the WAL file \"%s\" is %d byte",
+ "WAL segment size must be a power of two between 1 MB and 1 GB, but size of the WAL file \"%s\" is %d bytes",
WalSegSz),
fname, WalSegSz);
}
else
{
- if (errno != 0)
- fatal_error("could not read file \"%s\": %s",
- fname, strerror(errno));
+ r = read(fd, buf.data, XLOG_BLCKSZ);
+ if (r == XLOG_BLCKSZ)
+ {
+ XLogLongPageHeader longhdr;
+
+ longhdr = (XLogLongPageHeader) buf.data;
+
+ if (longhdr->std.xlp_magic != XLOG_PAGE_MAGIC)
+ fatal_error(gettext("WAL segment \"%s\" has an incorrect magic number. If it's encrypted, use the -K option to pass encryption credentials"),
+ fname);
+
+ WalSegSz = longhdr->xlp_seg_size;
+
+ if (!IsValidWalSegSize(WalSegSz))
+ fatal_error(ngettext("WAL segment size must be a power of two between 1 MB and 1 GB, but the WAL file \"%s\" header specifies %d byte",
+ "WAL segment size must be a power of two between 1 MB and 1 GB, but the WAL file \"%s\" header specifies %d bytes",
+ WalSegSz),
+ fname, WalSegSz);
+ }
else
- fatal_error("could not read file \"%s\": read %d of %zu",
- fname, r, (Size) XLOG_BLCKSZ);
+ {
+ if (errno != 0)
+ fatal_error("could not read file \"%s\": %s",
+ fname, strerror(errno));
+ else
+ fatal_error("could not read file \"%s\": read %d of %zu",
+ fname, r, (Size) XLOG_BLCKSZ);
+ }
}
close(fd);
return true;
@@ -444,6 +477,19 @@ XLogDumpReadPage(XLogReaderState *state, XLogRecPtr targetPagePtr, int reqLen,
XLogDumpXLogRead(private->inpath, private->timeline, targetPagePtr,
readBuff, count);
+ if (data_encrypted)
+ {
+ char tweak[TWEAK_SIZE];
+ XLogSegNo readSegNo;
+ uint32 readSegOff;
+
+ XLByteToSeg(targetPagePtr, readSegNo, WalSegSz);
+ readSegOff = targetPagePtr % WalSegSz;
+
+ XLogEncryptionTweak(tweak, private->timeline, readSegNo, readSegOff);
+ decrypt_block(readBuff, readBuff, count, tweak, true);
+ }
+
return count;
}
@@ -791,7 +837,15 @@ usage(void)
printf(_("\nOptions:\n"));
printf(_(" -b, --bkp-details output detailed information about backup blocks\n"));
printf(_(" -e, --end=RECPTR stop reading at WAL location RECPTR\n"));
+#ifdef USE_ENCRYPTION
+ /* Data directory may be needed to locate KDF parameters file. */
+ printf(_(" -D DATADIR data directory\n"));
+#endif /* USE_ENCRYPTION */
printf(_(" -f, --follow keep retrying after reaching end of WAL\n"));
+#ifdef USE_ENCRYPTION
+ printf(_(" -K, --encryption-key-command=COMMAND\n"
+ " command that returns encryption key\n"));
+#endif /* USE_ENCRYPTION */
printf(_(" -n, --limit=N number of records to display\n"));
printf(_(" -p, --path=PATH directory in which to find log segment files or a\n"
" directory with a ./pg_wal that contains such files\n"
@@ -816,6 +870,7 @@ main(int argc, char **argv)
XLogReaderState *xlogreader_state;
XLogDumpPrivate private;
XLogDumpConfig config;
+ char *DataDir = NULL;
XLogDumpStats stats;
XLogRecord *record;
XLogRecPtr first_record;
@@ -826,6 +881,7 @@ main(int argc, char **argv)
{"end", required_argument, NULL, 'e'},
{"follow", no_argument, NULL, 'f'},
{"help", no_argument, NULL, '?'},
+ {"encryption-key-command", required_argument, NULL, 'K'},
{"limit", required_argument, NULL, 'n'},
{"path", required_argument, NULL, 'p'},
{"rmgr", required_argument, NULL, 'r'},
@@ -869,7 +925,7 @@ main(int argc, char **argv)
goto bad_argument;
}
- while ((option = getopt_long(argc, argv, "be:?fn:p:r:s:t:Vx:z",
+ while ((option = getopt_long(argc, argv, "bD:e:?fK:n:p:r:s:t:Vx:z",
long_options, &optindex)) != -1)
{
switch (option)
@@ -877,6 +933,9 @@ main(int argc, char **argv)
case 'b':
config.bkp_details = true;
break;
+ case 'D':
+ DataDir = optarg;
+ break;
case 'e':
if (sscanf(optarg, "%X/%X", &xlogid, &xrecoff) != 2)
{
@@ -893,6 +952,12 @@ main(int argc, char **argv)
usage();
exit(EXIT_SUCCESS);
break;
+#ifdef USE_ENCRYPTION
+ case 'K':
+ encryption_key_command = pg_strdup(optarg);
+ data_encrypted = true;
+ break;
+#endif /* USE_ENCRYPTION */
case 'n':
if (sscanf(optarg, "%d", &config.stop_after_records) != 1)
{
@@ -1092,6 +1157,9 @@ main(int argc, char **argv)
/* done with argument parsing, do the actual work */
+ if (data_encrypted)
+ setup_encryption(false, DataDir);
+
/* we have everything we need, start reading */
xlogreader_state = XLogReaderAllocate(WalSegSz, XLogDumpReadPage,
&private);
diff --git a/src/common/string.c b/src/common/string.c
index b01a56ceaa..5ba890d37d 100644
--- a/src/common/string.c
+++ b/src/common/string.c
@@ -42,6 +42,45 @@ pg_str_endswith(const char *str, const char *end)
return strcmp(str, end) == 0;
}
+/*
+ * Helper function to check if a page is completely empty.
+ *
+ * TODO Invent name that is more consistent with that of the other function(s)
+ * in this module.
+ */
+bool
+IsAllZero(const char *input, Size size)
+{
+ const char *pos = input;
+ const char *aligned_start = (char *) MAXALIGN64(input);
+ const char *end = input + size;
+
+ /* Check 1 byte at a time until pos is 8 byte aligned */
+ while (pos < aligned_start)
+ if (*pos++ != 0)
+ return false;
+
+ /*
+ * Run 8 parallel 8 byte checks in one iteration. On 2016 hardware
+ * slightly faster than 4 parallel checks.
+ */
+ while (pos + 8 * sizeof(uint64) <= end)
+ {
+ uint64 *p = (uint64 *) pos;
+
+ if ((p[0] | p[1] | p[2] | p[3] | p[4] | p[5] | p[6] | p[7]) != 0)
+ return false;
+ pos += 8 * sizeof(uint64);
+ }
+
+ /* Handle unaligned tail. */
+ while (pos < end)
+ if (*pos++ != 0)
+ return false;
+
+ return true;
+}
+
/*
* strtoint --- just like strtol, but returns int not long
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 1ead0256ad..35a6264d19 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -19,6 +19,7 @@
#include "lib/stringinfo.h"
#include "nodes/pg_list.h"
#include "storage/fd.h"
+#include "storage/encryption.h"
/* Sync methods */
@@ -186,10 +187,19 @@ extern PGDLLIMPORT int wal_level;
* Normally, we don't WAL-log hint bit updates, but if checksums are enabled,
* we have to protect them against torn page writes. When you only set
* individual bits on a page, it's still consistent no matter what combination
- * of the bits make it to disk, but the checksum wouldn't match. Also WAL-log
- * them if forced by wal_log_hints=on.
+ * of the bits make it to disk, but the checksum wouldn't match.
+ *
+ * Regardless checksums, if encryption is enabled, hint bit change followed by
+ * a torn page write can result in such a situation that decryption produces
+ * page whose contents (following the hint bit) is garbage. This is because
+ * the block cipher we use propagates changes in the lower addresses of plain
+ * data to higher addresses of the cipher data. We need full-page image also
+ * to recover from this state.
+ *
+ * Also WAL-log the hint bits if forced by wal_log_hints=on.
*/
-#define XLogHintBitIsNeeded() (DataChecksumsEnabled() || wal_log_hints)
+#define XLogHintBitIsNeeded() (DataChecksumsEnabled() || data_encrypted || \
+ wal_log_hints)
/* Do we need to WAL-log information required only for Hot Standby and logical replication? */
#define XLogStandbyInfoActive() (wal_level >= WAL_LEVEL_REPLICA)
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index ff98d9e91a..927aa2ff61 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -94,6 +94,11 @@ typedef enum DBState
} DBState;
/*
+ * Number of bytes reserved to store encryption sample in ControlFileData.
+ */
+#define ENCRYPTION_SAMPLE_SIZE 16
+
+/*
* Contents of pg_control.
*/
@@ -228,6 +233,16 @@ typedef struct ControlFileData
*/
char mock_authentication_nonce[MOCK_AUTH_NONCE_LEN];
+ /*
+ * Cipher used to encrypt data. Zero if unencrypted.
+ *
+ * The data type is actually CipherKind, but we don't want to include
+ * encryption.h just because of this field.
+ */
+ uint8 data_cipher;
+ /* Sample value for encryption key verification */
+ uint8 encryption_verification[ENCRYPTION_SAMPLE_SIZE];
+
/* CRC of all above ... MUST BE LAST! */
pg_crc32c crc;
} ControlFileData;
diff --git a/src/include/common/string.h b/src/include/common/string.h
index f3467f0ee4..bcdeffba2b 100644
--- a/src/include/common/string.h
+++ b/src/include/common/string.h
@@ -14,5 +14,6 @@ extern bool pg_str_endswith(const char *str, const char *end);
extern int strtoint(const char *pg_restrict str, char **pg_restrict endptr,
int base);
extern void pg_clean_ascii(char *str);
+extern bool IsAllZero(const char *input, Size size);
#endif /* COMMON_STRING_H */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 61283806b6..d01f2c960c 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -431,6 +431,7 @@ extern void BaseInit(void);
/* in utils/init/miscinit.c */
extern bool IgnoreSystemIndexes;
extern PGDLLIMPORT bool process_shared_preload_libraries_in_progress;
+extern char *encryption_library_string;
extern char *session_preload_libraries_string;
extern char *shared_preload_libraries_string;
extern char *local_preload_libraries_string;
diff --git a/src/include/pg_config_manual.h b/src/include/pg_config_manual.h
index 743401cb96..b716fb4d13 100644
--- a/src/include/pg_config_manual.h
+++ b/src/include/pg_config_manual.h
@@ -172,6 +172,13 @@
#endif
/*
+ * OpenSSL is currently the only implementation of encryption we use.
+ */
+#ifdef USE_OPENSSL
+#define USE_ENCRYPTION
+#endif
+
+/*
* This is the default directory in which AF_UNIX socket files are
* placed. Caution: changing this risks breaking your existing client
* applications, which are likely to continue to look in the old
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 3bedd268a9..4e3058784b 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -894,6 +894,9 @@ typedef enum
WAIT_EVENT_DATA_FILE_TRUNCATE,
WAIT_EVENT_DATA_FILE_WRITE,
WAIT_EVENT_DSM_FILL_ZERO_WRITE,
+ WAIT_EVENT_KDF_FILE_READ,
+ WAIT_EVENT_KDF_FILE_SYNC,
+ WAIT_EVENT_KDF_FILE_WRITE,
WAIT_EVENT_LOCK_FILE_ADDTODATADIR_READ,
WAIT_EVENT_LOCK_FILE_ADDTODATADIR_SYNC,
WAIT_EVENT_LOCK_FILE_ADDTODATADIR_WRITE,
diff --git a/src/include/storage/copydir.h b/src/include/storage/copydir.h
index 525cc6203e..21c18bb6c3 100644
--- a/src/include/storage/copydir.h
+++ b/src/include/storage/copydir.h
@@ -13,7 +13,9 @@
#ifndef COPYDIR_H
#define COPYDIR_H
-extern void copydir(char *fromdir, char *todir, bool recurse);
-extern void copy_file(char *fromfile, char *tofile);
+#include "storage/relfilenode.h"
+
+extern void copydir(char *fromdir, char *todir, RelFileNode *fromNode, RelFileNode *toNode);
+extern void copy_file(char *fromfile, char *tofile, RelFileNode *fromNode, RelFileNode *toNode, ForkNumber fromForkNum, ForkNumber toForkNum, int segment);
#endif /* COPYDIR_H */
diff --git a/src/include/storage/encryption.h b/src/include/storage/encryption.h
new file mode 100644
index 0000000000..93c951488a
--- /dev/null
+++ b/src/include/storage/encryption.h
@@ -0,0 +1,167 @@
+/*-------------------------------------------------------------------------
+ *
+ * encryption.h
+ * Full database encryption support
+ *
+ *
+ * Portions Copyright (c) 1996-2015, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/storage/encryption.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef ENCRYPTION_H
+#define ENCRYPTION_H
+
+#include "access/xlogdefs.h"
+#include "storage/block.h"
+#include "storage/relfilenode.h"
+#include "port/pg_crc32c.h"
+
+extern void encryption_error(bool fatal, char *message);
+
+/*
+ * Common error message issued when particular code path cannot be executed
+ * due to absence of the OpenSSL library.
+ */
+#define ENCRYPTION_NOT_SUPPORTED_MSG \
+ encryption_error(true, "compile postgres with --with-openssl to use encryption.")
+
+/*
+ * Full database encryption key.
+ *
+ * The key of EVP_aes_256_cbc() cipher is 256 bits long.
+ */
+#define ENCRYPTION_KEY_LENGTH 32
+
+/*
+ * Cipher used to encrypt data.
+ *
+ * Due to very specific requirements, the ciphers are not likely to change,
+ * but we should be somewhat flexible.
+ *
+ * XXX If we have more than one cipher someday, have pg_controldata report the
+ * cipher kind (in textual form) instead of merely saying "on".
+ */
+typedef enum CipherKind
+{
+ /* The cluster is not encrypted. */
+ PG_CIPHER_NONE = 0,
+
+ /*
+ * AES (Rijndael) in CBC mode of operation as block cipher, and in CTR
+ * mode as stream cipher. Key length is always 256 bits.
+ */
+ PG_CIPHER_AES_BLOCK_CBC_256_STREAM_CTR_256
+} CipherKind;
+
+/*
+ * TODO Tune these values.
+ */
+#define ENCRYPTION_PWD_MIN_LENGTH 8
+#define ENCRYPTION_PWD_MAX_LENGTH 16
+#define ENCRYPTION_KDF_NITER 1048576
+#define ENCRYPTION_KDF_SALT_LEN 8
+
+/* Key to encrypt / decrypt data. */
+extern unsigned char encryption_key[];
+
+/*
+ * The encrypted data is a series of blocks of size
+ * ENCRYPTION_BLOCK. Currently we use the EVP_aes_256_xts implementation. Make
+ * sure the following constants match if adopting another algorithm.
+ */
+#define ENCRYPTION_BLOCK 16
+
+#define TWEAK_SIZE 16
+
+#define KDF_PARAMS_FILE "global/pg_keysetup"
+#define KDF_PARAMS_FILE_SIZE 512
+
+/*
+ * Key derivation function.
+ */
+typedef enum KDFKind
+{
+ KDF_OPENSSL_PKCS5_PBKDF2_HMAC_SHA = 0
+} KFDKind;
+
+typedef struct KDFParamsPBKDF2
+{
+ unsigned long int niter;
+ unsigned char salt[ENCRYPTION_KDF_SALT_LEN];
+} KDFParamsPBKDF2;
+
+/*
+ * Parameters of the key derivation function.
+ *
+ * The parameters are generated by initdb and stored into a file, which is
+ * then read during PG startup. This is similar to storing various settings in
+ * pg_control. However an existing KDF file is read only, so it does not have
+ * to be stored in shared memory.
+ */
+typedef struct KDFParamsData
+{
+ KFDKind function;
+
+ /*
+ * Function-specific parameters.
+ */
+ union
+ {
+ KDFParamsPBKDF2 pbkdf2;
+ } data;
+
+ /* CRC of all above ... MUST BE LAST! */
+ pg_crc32c crc;
+} KDFParamsData;
+
+extern KDFParamsData *KDFParams;
+
+extern PGDLLIMPORT bool data_encrypted;
+
+extern void read_kdf_file(char *dir);
+#ifndef FRONTEND
+extern void write_kdf_file(void);
+#endif /* FRONTEND */
+
+extern PGDLLIMPORT char *encryption_key_command;
+
+/*
+ * In some cases we need a separate copy of the data because encryption
+ * in-place (typically in the shared buffers) would make the data unusable for
+ * backends.
+ */
+extern PGAlignedBlock encrypt_buf;
+
+/*
+ * The same for XLOG. This buffer spans multiple pages, in order to reduce the
+ * number of syscalls when doing I/O.
+ *
+ * XXX Fine tune the buffer size.
+ */
+extern char *encrypt_buf_xlog;
+#define XLOG_ENCRYPT_BUF_PAGES 8
+#define ENCRYPT_BUF_XLOG_SIZE (XLOG_ENCRYPT_BUF_PAGES * XLOG_BLCKSZ)
+
+extern char *run_encryption_key_command(const char *cmd, bool *is_key_p, size_t *len_p);
+
+extern void setup_encryption(bool bootstrap, char *data_dir);
+extern void setup_encryption_key(char *credentials, bool is_key, size_t len);
+extern void sample_encryption(char *buf);
+extern void encrypt_block(const char *input, char *output, Size size,
+ char *tweak, bool stream);
+extern void decrypt_block(const char *input, char *output, Size size,
+ char *tweak, bool stream);
+extern void encryption_error(bool fatal, char *message);
+
+extern void XLogEncryptionTweak(char *tweak, TimeLineID timeline,
+ XLogSegNo segment, uint32 offset);
+extern BlockNumber ReencryptBlock(char *buffer, int blocks,
+ RelFileNode *srcNode, RelFileNode *dstNode,
+ ForkNumber srcForkNum, ForkNumber dstForkNum,
+ BlockNumber blockNum);
+extern void mdtweak(char *tweak, RelFileNode *relnode, ForkNumber forknum,
+ BlockNumber blocknum);
+#endif /* ENCRYPTION_H */
diff --git a/src/include/storage/reinit.h b/src/include/storage/reinit.h
index 36eeef4aec..0e028cac4d 100644
--- a/src/include/storage/reinit.h
+++ b/src/include/storage/reinit.h
@@ -20,7 +20,8 @@
extern void ResetUnloggedRelations(int op);
extern bool parse_filename_for_nontemp_relation(const char *name,
- int *oidchars, ForkNumber *fork);
+ int *oidchars, ForkNumber *fork,
+ int *segment);
#define UNLOGGED_RELATION_CLEANUP 0x0001
#define UNLOGGED_RELATION_INIT 0x0002
diff --git a/src/test/modules/buffile/Makefile b/src/test/modules/buffile/Makefile
new file mode 100644
index 0000000000..d0329e082f
--- /dev/null
+++ b/src/test/modules/buffile/Makefile
@@ -0,0 +1,21 @@
+PG_CONFIG ?= pg_config
+MODULE_big = buffile_test
+OBJS = buffile.o $(WIN32RES)
+PGFILEDESC = "buffile_test"
+
+EXTENSION = buffile
+DATA = buffile--1.0.sql
+
+REGRESS = test_00 test_01 test_02 test_03 test_04 test_05 test_06 test_07 \
+test_08 test_09 test_10 test_11 test_12 test_13
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/buffile
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/buffile/README b/src/test/modules/buffile/README
new file mode 100644
index 0000000000..9c80787797
--- /dev/null
+++ b/src/test/modules/buffile/README
@@ -0,0 +1,11 @@
+This extension was written to check if changes introduced due to cluster
+encryption do not break buffile.c.
+
+Caution: To make the test cheaper, it was decided to adjust the segment sizeq,
+see
+
+#define MAX_PHYSICAL_FILESIZE (4 * BLCKSZ)
+
+in buffile.c. BLCKSZ is 8192 (the default). All the tests rely on this
+value. So if you haven't compiled Postgres with this value, the tests will
+create 1 GB files and they will fail.
diff --git a/src/test/modules/buffile/buffile--1.0.sql b/src/test/modules/buffile/buffile--1.0.sql
new file mode 100644
index 0000000000..7c168f3514
--- /dev/null
+++ b/src/test/modules/buffile/buffile--1.0.sql
@@ -0,0 +1,54 @@
+CREATE FUNCTION buffile_create()
+RETURNS void
+AS 'MODULE_PATHNAME', 'buffile_create'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_close()
+RETURNS void
+AS 'MODULE_PATHNAME', 'buffile_close'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_write(text)
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'buffile_write'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_read(bigint)
+RETURNS bytea
+AS 'MODULE_PATHNAME', 'buffile_read'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_seek(int, bigint)
+RETURNS int
+AS 'MODULE_PATHNAME', 'buffile_seek'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_assert_fileno(int)
+RETURNS void
+AS 'MODULE_PATHNAME', 'buffile_assert_fileno'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_test_shared()
+RETURNS void
+AS 'MODULE_PATHNAME', 'buffile_test_shared'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_test_shared_append()
+RETURNS void
+AS 'MODULE_PATHNAME', 'buffile_test_shared_append'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_open_transient(text, bool, bool)
+RETURNS void
+AS 'MODULE_PATHNAME', 'buffile_open_transient'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_close_transient()
+RETURNS void
+AS 'MODULE_PATHNAME', 'buffile_close_transient'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_delete_file(text)
+RETURNS void
+AS 'MODULE_PATHNAME', 'buffile_delete_file'
+LANGUAGE C;
diff --git a/src/test/modules/buffile/buffile.c b/src/test/modules/buffile/buffile.c
new file mode 100644
index 0000000000..7fbb7939ff
--- /dev/null
+++ b/src/test/modules/buffile/buffile.c
@@ -0,0 +1,428 @@
+#include <fcntl.h>
+
+#include "postgres.h"
+#include "fmgr.h"
+#include "lib/stringinfo.h"
+#include "storage/buffile.h"
+#include "utils/array.h"
+#include "utils/builtins.h"
+#include "utils/lsyscache.h"
+#include "utils/memutils.h"
+#include "utils/resowner.h"
+
+PG_MODULE_MAGIC;
+
+/*
+ * To cover various corner cases, the tests assume MAX_PHYSICAL_FILESIZE to be
+ * exactly MAX_PHYSICAL_FILESIZE_TEST.
+ */
+#define MAX_PHYSICAL_FILESIZE_TEST (4 * BLCKSZ)
+
+static BufFile *bf = NULL;
+static TransientBufFile *bft = NULL;
+
+static void check_file(void);
+
+extern Datum buffile_create_transient(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_create);
+Datum
+buffile_create(PG_FUNCTION_ARGS)
+{
+ MemoryContext old_cxt;
+ ResourceOwner old_ro;
+
+ if (bf != NULL)
+ elog(ERROR, "file already exists");
+
+ old_cxt = MemoryContextSwitchTo(TopMemoryContext);
+
+ /*
+ * Make sure the file is not deleted across function calls.
+ */
+ old_ro = CurrentResourceOwner;
+ CurrentResourceOwner = TopTransactionResourceOwner;
+
+ bf = BufFileCreateTemp(false);
+
+ CurrentResourceOwner = old_ro;
+ MemoryContextSwitchTo(old_cxt);
+
+ PG_RETURN_VOID();
+}
+
+extern Datum buffile_close(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_close);
+Datum
+buffile_close(PG_FUNCTION_ARGS)
+{
+ if (bf == NULL)
+ elog(ERROR, "there's no file to close");
+
+ BufFileClose(bf);
+ bf = NULL;
+
+ PG_RETURN_VOID();
+}
+
+extern Datum buffile_write(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_write);
+Datum
+buffile_write(PG_FUNCTION_ARGS)
+{
+ Datum d = PG_GETARG_DATUM(0);
+ char *s = TextDatumGetCString(d);
+ size_t res;
+
+ if (bf)
+ res = BufFileWrite(bf, s, strlen(s));
+ else if (bft)
+ res = BufFileWriteTransient(bft, s, strlen(s));
+ else
+ elog(ERROR, "No file is open");
+
+ PG_RETURN_INT64(res);
+}
+
+extern Datum buffile_read(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_read);
+Datum
+buffile_read(PG_FUNCTION_ARGS)
+{
+ int64 size = PG_GETARG_INT64(0);
+ StringInfo buf = makeStringInfo();
+ size_t res_size;
+ bytea *result;
+
+ enlargeStringInfo(buf, size);
+
+ if (bf)
+ res_size = BufFileRead(bf, buf->data, size);
+ else if (bft)
+ res_size = BufFileReadTransient(bft, buf->data, size);
+ else
+ elog(ERROR, "No file is open");
+
+ buf->len = res_size;
+
+ result = DatumGetByteaPP(DirectFunctionCall1(bytearecv,
+ PointerGetDatum(buf)));
+ PG_RETURN_BYTEA_P(result);
+}
+
+extern Datum buffile_seek(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_seek);
+Datum
+buffile_seek(PG_FUNCTION_ARGS)
+{
+ int32 fileno = PG_GETARG_INT32(0);
+ int64 offset = PG_GETARG_INT64(1);
+ int32 res;
+
+ check_file();
+ res = BufFileSeek(bf, fileno, offset, SEEK_SET);
+
+ PG_RETURN_INT32(res);
+}
+
+extern Datum buffile_assert_fileno(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_assert_fileno);
+Datum
+buffile_assert_fileno(PG_FUNCTION_ARGS)
+{
+ int32 fileno_expected = PG_GETARG_INT32(0);
+ int32 fileno;
+ off_t offset;
+
+ check_file();
+ BufFileTell(bf, &fileno, &offset);
+
+ if (fileno != fileno_expected)
+ {
+ /*
+ * Bring the backend down so that the following tests have no chance
+ * to create the 1GB files.
+ */
+ elog(FATAL, "file number does not match");
+ }
+
+ PG_RETURN_VOID();
+}
+
+static void
+check_file(void)
+{
+ if (bf == NULL)
+ elog(ERROR, "the file is not opened");
+}
+
+/*
+ * This test is especially important for shared encrypted files, see the
+ * comments below.
+ */
+extern Datum buffile_test_shared(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_test_shared);
+Datum
+buffile_test_shared(PG_FUNCTION_ARGS)
+{
+ dsm_segment *seg;
+ SharedFileSet *fileset;
+ BufFile *bf_1,
+ *bf_2;
+ char *data_1,
+ *data_2,
+ *data;
+ Size chunk_size_1,
+ chunk_size_2;
+ int fileno,
+ i;
+ off_t offset,
+ res,
+ total_size;
+
+ /*
+ * The size is not important, we actually do not need the shared memory.
+ * The segment is only needed to initialize the fileset.
+ */
+ seg = dsm_create(1024, 0);
+
+ /*
+ * The fileset must survive error handling, so that dsm_detach works fine.
+ * (The typical use case is that the fileset is in shared memory.)
+ */
+ fileset = (SharedFileSet *) MemoryContextAlloc(TopTransactionContext,
+ sizeof(SharedFileSet));
+ SharedFileSetInit(fileset, seg);
+
+ bf_1 = BufFileCreateShared(fileset, "file_1");
+
+ /*
+ * Write more data than the buffer size, so that we can check that the
+ * number of "useful bytes" word is only appended at the end of the
+ * segment, not after each buffer.
+ */
+ chunk_size_1 = BLCKSZ + 256;
+ data_1 = (char *) palloc(chunk_size_1);
+ memset(data_1, 1, chunk_size_1);
+ if (BufFileWrite(bf_1, data_1, chunk_size_1) != chunk_size_1)
+ elog(ERROR, "Failed to write data");
+ pfree(data_1);
+
+ /*
+ * Enforce buffer flush (The BufFileFlush() function is not exported).
+ * Thus the "useful bytes" metadata should appear at the current end the
+ * first file segment. The next write will have to seek back to overwrite
+ * the metadata.
+ */
+ BufFileTell(bf_1, &fileno, &offset);
+ if (BufFileSeek(bf_1, 0, 0, SEEK_SET) != 0)
+ elog(ERROR, "seek failed");
+ if (BufFileSeek(bf_1, fileno, offset, SEEK_SET) != 0)
+ elog(ERROR, "seek failed");
+
+ /*
+ * Write another chunk that does not fit into the first segment file. Thus
+ * the "useful bytes" metadata should appear at the end of both segments.
+ */
+ chunk_size_2 = 3 * BLCKSZ;
+ data_2 = (char *) palloc(chunk_size_2);
+ memset(data_2, 1, chunk_size_2);
+ if (BufFileWrite(bf_1, data_2, chunk_size_2) != chunk_size_2)
+ elog(ERROR, "Failed to write data");
+ pfree(data_2);
+ BufFileClose(bf_1);
+
+ /*
+ * The word indicating the number of "useful bytes" (i.e. the actual data
+ * w/o padding to buffer size) is stored at the end of each segment file.
+ * Check that this metadata is read correctly.
+ */
+ bf_2 = BufFileOpenShared(fileset, "file_1");
+ total_size = BufFileSize(bf_2);
+ if (total_size != (chunk_size_1 + chunk_size_2))
+ elog(ERROR, "Incorrect file size: %zu", total_size);
+
+ data = (char *) palloc(total_size);
+ res = BufFileRead(bf_2, data, total_size);
+ if (res != total_size)
+ elog(ERROR, "Incorrect chunk size read: %zu", res);
+ for (i = 0; i < total_size; i++)
+ if (data[i] != 1)
+ elog(ERROR, "Unexpected data read from the file");
+ pfree(data);
+ BufFileClose(bf_2);
+
+ dsm_detach(seg);
+
+ PG_RETURN_VOID();
+}
+
+
+/*
+ * Test BufFileAppend().
+ */
+extern Datum buffile_test_shared_append(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_test_shared_append);
+Datum
+buffile_test_shared_append(PG_FUNCTION_ARGS)
+{
+ dsm_segment *seg;
+ SharedFileSet *fileset;
+ BufFile *bf_1,
+ *bf_2,
+ *bf_3;
+ char *data;
+ Size chunk_size;
+ int i;
+ off_t res,
+ total_size;
+
+ seg = dsm_create(1024, 0);
+
+ fileset = (SharedFileSet *) MemoryContextAlloc(TopTransactionContext,
+ sizeof(SharedFileSet));
+ SharedFileSetInit(fileset, seg);
+
+ /*
+ * XXX Does the chunk size matter much?
+ */
+ chunk_size = 8;
+ data = (char *) palloc(chunk_size);
+ memset(data, 1, chunk_size);
+
+ bf_1 = BufFileCreateShared(fileset, "file_1");
+ if (BufFileWrite(bf_1, data, chunk_size) != chunk_size)
+ elog(ERROR, "Failed to write data");
+
+ bf_2 = BufFileCreateShared(fileset, "file_2");
+ if (BufFileWrite(bf_2, data, chunk_size) != chunk_size)
+ elog(ERROR, "Failed to write data");
+
+ /*
+ * Make sure it's read-only so that BufFileAppend() can accept it as
+ * source.
+ */
+ BufFileClose(bf_2);
+ bf_2 = BufFileOpenShared(fileset, "file_2");
+
+ bf_3 = BufFileCreateShared(fileset, "file_3");
+ if (BufFileWrite(bf_3, data, chunk_size) != chunk_size)
+ elog(ERROR, "Failed to write data");
+ BufFileClose(bf_3);
+ bf_3 = BufFileOpenShared(fileset, "file_3");
+
+ BufFileAppend(bf_1, bf_2);
+ BufFileAppend(bf_1, bf_3);
+
+ total_size = BufFileSize(bf_1);
+
+ /*
+ * The result should contain complete segments of bf_1 and bf_2 and the
+ * valid part of bf_3.
+ */
+ if (total_size != (2 * MAX_PHYSICAL_FILESIZE_TEST + chunk_size))
+ elog(ERROR, "Incorrect total size of the appended data: %zu",
+ total_size);
+
+ /*
+ * Check that data of the 2nd segment was decrypted correctly.
+ */
+ if (BufFileSeek(bf_1, 1, 0, SEEK_SET) != 0)
+ elog(ERROR, "seek failed");
+ res = BufFileRead(bf_1, data, chunk_size);
+ if (res != chunk_size)
+ elog(ERROR, "Incorrect chunk size read: %zu", res);
+ for (i = 0; i < chunk_size; i++)
+ if (data[i] != 1)
+ elog(ERROR, "Unexpected data read from the file");
+
+ /*
+ * And the same for the 3rd segment.
+ *
+ * TODO Reuse the code above by putting it into a function.
+ */
+ if (BufFileSeek(bf_1, 2, 0, SEEK_SET) != 0)
+ elog(ERROR, "seek failed");
+ res = BufFileRead(bf_1, data, chunk_size);
+ if (res != chunk_size)
+ elog(ERROR, "Incorrect chunk size read: %zu", res);
+ for (i = 0; i < chunk_size; i++)
+ if (data[i] != 1)
+ elog(ERROR, "Unexpected data read from the file");
+
+ BufFileClose(bf_1);
+ dsm_detach(seg);
+ PG_RETURN_VOID();
+}
+
+extern Datum buffile_open_transient(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_open_transient);
+Datum
+buffile_open_transient(PG_FUNCTION_ARGS)
+{
+ MemoryContext old_cxt;
+ Datum d = PG_GETARG_DATUM(0);
+ char *path = TextDatumGetCString(d);
+ bool write_only = PG_GETARG_BOOL(1);
+ bool append = PG_GETARG_BOOL(2);
+ int flags = O_CREAT | PG_BINARY;
+
+ if (bft != NULL)
+ elog(ERROR, "file already exists");
+
+ if (write_only)
+ flags |= O_WRONLY;
+ if (append)
+ flags |= O_APPEND;
+
+ old_cxt = MemoryContextSwitchTo(TopMemoryContext);
+
+ /*
+ * Make sure the file is not deleted across function calls.
+ */
+ bft = BufFileOpenTransient(path, flags);
+
+ MemoryContextSwitchTo(old_cxt);
+
+ PG_RETURN_VOID();
+}
+
+extern Datum buffile_close_transient(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_close_transient);
+Datum
+buffile_close_transient(PG_FUNCTION_ARGS)
+{
+ if (bft == NULL)
+ elog(ERROR, "there's no file to close");
+
+ BufFileCloseTransient(bft);
+ bft = NULL;
+
+ PG_RETURN_VOID();
+}
+
+extern Datum buffile_delete_file(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_delete_file);
+Datum
+buffile_delete_file(PG_FUNCTION_ARGS)
+{
+ Datum d = PG_GETARG_DATUM(0);
+ char *path = TextDatumGetCString(d);
+
+ if (bft != NULL)
+ elog(ERROR, "the file is still open");
+
+ PathNameDeleteTemporaryFile(path, true);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/test/modules/buffile/buffile.control b/src/test/modules/buffile/buffile.control
new file mode 100644
index 0000000000..8472c5a348
--- /dev/null
+++ b/src/test/modules/buffile/buffile.control
@@ -0,0 +1,5 @@
+# buffile_test extension
+comment = 'buffile_test'
+default_version = '1.0'
+module_pathname = '$libdir/buffile_test'
+relocatable = true
diff --git a/src/test/modules/buffile/expected/test_00.out b/src/test/modules/buffile/expected/test_00.out
new file mode 100644
index 0000000000..637c33689f
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_00.out
@@ -0,0 +1,47 @@
+CREATE EXTENSION IF NOT EXISTS buffile;
+-- This test only verifies that PG is compiled with a component file size of
+-- 32 kB (i.e. 4 buffers of 8 kB) instead of 1 GB. That seems appropriate for
+-- testing. Some other tests may rely on it.
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+-- Skip the first component file.
+SELECT buffile_seek(0, 32768);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+-- Write the first byte of the second component file. We can't simply
+-- buffile_seek() beyond the position 32768 as it would return EOF.
+SELECT buffile_write('a');
+ buffile_write
+---------------
+ 1
+(1 row)
+
+-- Enforce BufFileFlush(), which actually adds the component file.
+SELECT buffile_read(1);
+ buffile_read
+--------------
+ \x
+(1 row)
+
+-- Check that we're in the 2nd file, i.e. the file size is as expected.
+SELECT buffile_assert_fileno(1);
+ buffile_assert_fileno
+-----------------------
+
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_01.out b/src/test/modules/buffile/expected/test_01.out
new file mode 100644
index 0000000000..448bac171c
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_01.out
@@ -0,0 +1,59 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+SELECT buffile_seek(0, 1);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_write('abc');
+ buffile_write
+---------------
+ 3
+(1 row)
+
+SELECT buffile_seek(0, 0);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+-- Check that the trailing zeroes are not fetched.
+SELECT buffile_read(16);
+ buffile_read
+--------------
+ \x00616263
+(1 row)
+
+-- Adjust the number of useful bytes.
+SELECT buffile_write('abc');
+ buffile_write
+---------------
+ 3
+(1 row)
+
+-- ... and check again.
+SELECT buffile_seek(0, 0);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(16);
+ buffile_read
+------------------
+ \x00616263616263
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_02.out b/src/test/modules/buffile/expected/test_02.out
new file mode 100644
index 0000000000..f783d0cb24
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_02.out
@@ -0,0 +1,48 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+SELECT buffile_seek(0, 8189);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+-- Initialize the last 3 positions of the first buffer and the initial 3
+-- positions of the 2nd buffer.
+SELECT buffile_write('abcdef');
+ buffile_write
+---------------
+ 6
+(1 row)
+
+SELECT buffile_seek(0, 0);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+-- Read the first buffer.
+SELECT length(buffile_read(8192));
+ length
+--------
+ 8192
+(1 row)
+
+-- Only 3 bytes of the 2nd buffer should be fetched.
+SELECT length(buffile_read(8192));
+ length
+--------
+ 3
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_03.out b/src/test/modules/buffile/expected/test_03.out
new file mode 100644
index 0000000000..e899fa3b38
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_03.out
@@ -0,0 +1,27 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+-- Read from an empty file.
+SELECT buffile_seek(0, 8);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(16);
+ buffile_read
+--------------
+ \x
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_04.out b/src/test/modules/buffile/expected/test_04.out
new file mode 100644
index 0000000000..1f8eeabe48
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_04.out
@@ -0,0 +1,84 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+-- Write something near the end of the first buffer, but leave some trailing
+-- space.
+SELECT buffile_seek(0, 8184);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_write('abcd');
+ buffile_write
+---------------
+ 4
+(1 row)
+
+-- Leave the 2nd buffer empty, as well as a few leading bytes. Thus we should
+-- get a hole that spans the whole 2nd buffer as well as a few adjacent bytes
+-- on each side.
+SELECT buffile_seek(0, 2 * 8192 + 4);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_write('efgh');
+ buffile_write
+---------------
+ 4
+(1 row)
+
+-- Check the initial part of the hole, which crosses the boundary of the 1st
+-- and the 2nd buffer.
+SELECT buffile_seek(0, 8184);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(16);
+ buffile_read
+------------------------------------
+ \x61626364000000000000000000000000
+(1 row)
+
+-- Check the trailing part of the whole, which crosses the boundary of the 2nd
+-- and the 3rd buffer.
+SELECT buffile_seek(0, 2 * 8192 - 8);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(16);
+ buffile_read
+------------------------------------
+ \x00000000000000000000000065666768
+(1 row)
+
+-- Check that the hole contains nothing but zeroes.
+SELECT buffile_seek(0, 8192 - 4);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT btrim(buffile_read(8192 + 8), '\x00');
+ btrim
+-------
+ \x
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_05.out b/src/test/modules/buffile/expected/test_05.out
new file mode 100644
index 0000000000..6a73147711
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_05.out
@@ -0,0 +1,33 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+-- Seek does not extend the file if it's not followed by write.
+SELECT buffile_seek(0, 1);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_seek(0, 0);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(2);
+ buffile_read
+--------------
+ \x
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_06.out b/src/test/modules/buffile/expected/test_06.out
new file mode 100644
index 0000000000..def9af5112
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_06.out
@@ -0,0 +1,41 @@
+-- This test shows that the first component file (segment) stays empty, read
+-- stops prematurely even if it starts on that segment, even though it'd
+-- otherwise receive some data from the following one.
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+SELECT buffile_seek(0, 32768);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_write('a');
+ buffile_write
+---------------
+ 1
+(1 row)
+
+SELECT buffile_seek(0, 32767);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(2);
+ buffile_read
+--------------
+ \x
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_07.out b/src/test/modules/buffile/expected/test_07.out
new file mode 100644
index 0000000000..215280f0b0
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_07.out
@@ -0,0 +1,39 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+-- Write data at component file boundary and try to read it.
+SELECT buffile_seek(0, 32768);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_write('abcd');
+ buffile_write
+---------------
+ 4
+(1 row)
+
+SELECT buffile_seek(0, 32768);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(8);
+ buffile_read
+--------------
+ \x61626364
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_08.out b/src/test/modules/buffile/expected/test_08.out
new file mode 100644
index 0000000000..7c1069930c
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_08.out
@@ -0,0 +1,39 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+-- Write data across component file boundary and try to read it.
+SELECT buffile_seek(0, 32766);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_write('abcd');
+ buffile_write
+---------------
+ 4
+(1 row)
+
+SELECT buffile_seek(0, 32766);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(8);
+ buffile_read
+--------------
+ \x61626364
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_09.out b/src/test/modules/buffile/expected/test_09.out
new file mode 100644
index 0000000000..b9d325b676
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_09.out
@@ -0,0 +1,39 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+-- Write data across buffer boundary and try to read it.
+SELECT buffile_seek(0, 8190);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_write('abcd');
+ buffile_write
+---------------
+ 4
+(1 row)
+
+SELECT buffile_seek(0, 8190);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(8);
+ buffile_read
+--------------
+ \x61626364
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_10.out b/src/test/modules/buffile/expected/test_10.out
new file mode 100644
index 0000000000..8e457fdcda
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_10.out
@@ -0,0 +1,76 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+-- Write some data at the end of the buffer.
+SELECT buffile_seek(0, 8188);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_write('abcd');
+ buffile_write
+---------------
+ 4
+(1 row)
+
+SELECT buffile_seek(0, 8189);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+-- Enforce flush with the write position not at the end of the buffer. This is
+-- special by not moving curOffset to the next buffer.
+SELECT buffile_read(1);
+ buffile_read
+--------------
+ \x62
+(1 row)
+
+-- Therefore the next writes should eventually affect the original data. (Here
+-- we also test going directly from read to write and vice versa.)
+SELECT buffile_write('x');
+ buffile_write
+---------------
+ 1
+(1 row)
+
+SELECT buffile_read(1);
+ buffile_read
+--------------
+ \x64
+(1 row)
+
+-- Start a new buffer, i.e. force flushing of the previous one.
+SELECT buffile_write('z');
+ buffile_write
+---------------
+ 1
+(1 row)
+
+-- Check that the 'x' and 'y' letters are in the first buffer, not in the
+-- 2nd. (We read enough data to find any non-zero bytes in the 2nd buffer.)
+SELECT buffile_seek(0, 8188);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(4 + 8192);
+ buffile_read
+--------------
+ \x616278647a
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_11.out b/src/test/modules/buffile/expected/test_11.out
new file mode 100644
index 0000000000..c6804d08af
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_11.out
@@ -0,0 +1,34 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+SELECT buffile_write('abcd');
+ buffile_write
+---------------
+ 4
+(1 row)
+
+-- Seek beyond EOF not followed by write.
+SELECT buffile_seek(0, 5);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+-- Nothing should be fetched.
+SELECT buffile_read(8);
+ buffile_read
+--------------
+ \x
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_12.out b/src/test/modules/buffile/expected/test_12.out
new file mode 100644
index 0000000000..3dce7eaf0f
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_12.out
@@ -0,0 +1,12 @@
+SELECT buffile_test_shared();
+ buffile_test_shared
+---------------------
+
+(1 row)
+
+SELECT buffile_test_shared_append();
+ buffile_test_shared_append
+----------------------------
+
+(1 row)
+
diff --git a/src/test/modules/buffile/expected/test_13.out b/src/test/modules/buffile/expected/test_13.out
new file mode 100644
index 0000000000..7ba50a9c9d
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_13.out
@@ -0,0 +1,86 @@
+-- Use transaction block so that the file does not closed automatically at
+-- command boundary.
+BEGIN;
+SELECT buffile_open_transient('trans1', true, false);
+ buffile_open_transient
+------------------------
+
+(1 row)
+
+SELECT buffile_write('01234567');
+ buffile_write
+---------------
+ 8
+(1 row)
+
+SELECT buffile_close_transient();
+ buffile_close_transient
+-------------------------
+
+(1 row)
+
+-- Open for reading.
+SELECT buffile_open_transient('trans1', false, false);
+ buffile_open_transient
+------------------------
+
+(1 row)
+
+SELECT length(buffile_read(65536));
+ length
+--------
+ 8
+(1 row)
+
+SELECT buffile_close_transient();
+ buffile_close_transient
+-------------------------
+
+(1 row)
+
+-- Open for writing in append mode.
+SELECT buffile_open_transient('trans1', true, true);
+ buffile_open_transient
+------------------------
+
+(1 row)
+
+-- Add BLCKSZ bytes, so that buffer boundary is crossed.
+SELECT buffile_write(repeat('x', 8192));
+ buffile_write
+---------------
+ 8192
+(1 row)
+
+SELECT buffile_close_transient();
+ buffile_close_transient
+-------------------------
+
+(1 row)
+
+-- Open for reading and verify the valid part.
+SELECT buffile_open_transient('trans1', false, false);
+ buffile_open_transient
+------------------------
+
+(1 row)
+
+SELECT length(buffile_read(65536));
+ length
+--------
+ 8200
+(1 row)
+
+SELECT buffile_close_transient();
+ buffile_close_transient
+-------------------------
+
+(1 row)
+
+SELECT buffile_delete_file('trans1');
+ buffile_delete_file
+---------------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_00.sql b/src/test/modules/buffile/sql/test_00.sql
new file mode 100644
index 0000000000..c907a5e195
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_00.sql
@@ -0,0 +1,18 @@
+CREATE EXTENSION IF NOT EXISTS buffile;
+
+-- This test only verifies that PG is compiled with a component file size of
+-- 32 kB (i.e. 4 buffers of 8 kB) instead of 1 GB. That seems appropriate for
+-- testing. Some other tests may rely on it.
+BEGIN;
+SELECT buffile_create();
+-- Skip the first component file.
+SELECT buffile_seek(0, 32768);
+-- Write the first byte of the second component file. We can't simply
+-- buffile_seek() beyond the position 32768 as it would return EOF.
+SELECT buffile_write('a');
+-- Enforce BufFileFlush(), which actually adds the component file.
+SELECT buffile_read(1);
+-- Check that we're in the 2nd file, i.e. the file size is as expected.
+SELECT buffile_assert_fileno(1);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_01.sql b/src/test/modules/buffile/sql/test_01.sql
new file mode 100644
index 0000000000..8cc87de841
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_01.sql
@@ -0,0 +1,14 @@
+BEGIN;
+SELECT buffile_create();
+SELECT buffile_seek(0, 1);
+SELECT buffile_write('abc');
+SELECT buffile_seek(0, 0);
+-- Check that the trailing zeroes are not fetched.
+SELECT buffile_read(16);
+-- Adjust the number of useful bytes.
+SELECT buffile_write('abc');
+-- ... and check again.
+SELECT buffile_seek(0, 0);
+SELECT buffile_read(16);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_02.sql b/src/test/modules/buffile/sql/test_02.sql
new file mode 100644
index 0000000000..35d34722d7
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_02.sql
@@ -0,0 +1,13 @@
+BEGIN;
+SELECT buffile_create();
+SELECT buffile_seek(0, 8189);
+-- Initialize the last 3 positions of the first buffer and the initial 3
+-- positions of the 2nd buffer.
+SELECT buffile_write('abcdef');
+SELECT buffile_seek(0, 0);
+-- Read the first buffer.
+SELECT length(buffile_read(8192));
+-- Only 3 bytes of the 2nd buffer should be fetched.
+SELECT length(buffile_read(8192));
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_03.sql b/src/test/modules/buffile/sql/test_03.sql
new file mode 100644
index 0000000000..a95391f7c3
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_03.sql
@@ -0,0 +1,7 @@
+BEGIN;
+SELECT buffile_create();
+-- Read from an empty file.
+SELECT buffile_seek(0, 8);
+SELECT buffile_read(16);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_04.sql b/src/test/modules/buffile/sql/test_04.sql
new file mode 100644
index 0000000000..64e8d39f94
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_04.sql
@@ -0,0 +1,25 @@
+BEGIN;
+SELECT buffile_create();
+-- Write something near the end of the first buffer, but leave some trailing
+-- space.
+SELECT buffile_seek(0, 8184);
+SELECT buffile_write('abcd');
+-- Leave the 2nd buffer empty, as well as a few leading bytes. Thus we should
+-- get a hole that spans the whole 2nd buffer as well as a few adjacent bytes
+-- on each side.
+SELECT buffile_seek(0, 2 * 8192 + 4);
+SELECT buffile_write('efgh');
+-- Check the initial part of the hole, which crosses the boundary of the 1st
+-- and the 2nd buffer.
+SELECT buffile_seek(0, 8184);
+SELECT buffile_read(16);
+-- Check the trailing part of the whole, which crosses the boundary of the 2nd
+-- and the 3rd buffer.
+SELECT buffile_seek(0, 2 * 8192 - 8);
+SELECT buffile_read(16);
+-- Check that the hole contains nothing but zeroes.
+SELECT buffile_seek(0, 8192 - 4);
+SELECT btrim(buffile_read(8192 + 8), '\x00');
+
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_05.sql b/src/test/modules/buffile/sql/test_05.sql
new file mode 100644
index 0000000000..5fd642e558
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_05.sql
@@ -0,0 +1,8 @@
+BEGIN;
+SELECT buffile_create();
+-- Seek does not extend the file if it's not followed by write.
+SELECT buffile_seek(0, 1);
+SELECT buffile_seek(0, 0);
+SELECT buffile_read(2);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_06.sql b/src/test/modules/buffile/sql/test_06.sql
new file mode 100644
index 0000000000..bbf506ec0e
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_06.sql
@@ -0,0 +1,11 @@
+-- This test shows that the first component file (segment) stays empty, read
+-- stops prematurely even if it starts on that segment, even though it'd
+-- otherwise receive some data from the following one.
+BEGIN;
+SELECT buffile_create();
+SELECT buffile_seek(0, 32768);
+SELECT buffile_write('a');
+SELECT buffile_seek(0, 32767);
+SELECT buffile_read(2);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_07.sql b/src/test/modules/buffile/sql/test_07.sql
new file mode 100644
index 0000000000..ca78506ecb
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_07.sql
@@ -0,0 +1,9 @@
+BEGIN;
+SELECT buffile_create();
+-- Write data at component file boundary and try to read it.
+SELECT buffile_seek(0, 32768);
+SELECT buffile_write('abcd');
+SELECT buffile_seek(0, 32768);
+SELECT buffile_read(8);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_08.sql b/src/test/modules/buffile/sql/test_08.sql
new file mode 100644
index 0000000000..0e5b1de383
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_08.sql
@@ -0,0 +1,9 @@
+BEGIN;
+SELECT buffile_create();
+-- Write data across component file boundary and try to read it.
+SELECT buffile_seek(0, 32766);
+SELECT buffile_write('abcd');
+SELECT buffile_seek(0, 32766);
+SELECT buffile_read(8);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_09.sql b/src/test/modules/buffile/sql/test_09.sql
new file mode 100644
index 0000000000..cc7060932e
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_09.sql
@@ -0,0 +1,9 @@
+BEGIN;
+SELECT buffile_create();
+-- Write data across buffer boundary and try to read it.
+SELECT buffile_seek(0, 8190);
+SELECT buffile_write('abcd');
+SELECT buffile_seek(0, 8190);
+SELECT buffile_read(8);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_10.sql b/src/test/modules/buffile/sql/test_10.sql
new file mode 100644
index 0000000000..63af760d9f
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_10.sql
@@ -0,0 +1,25 @@
+BEGIN;
+SELECT buffile_create();
+-- Write some data at the end of the buffer.
+SELECT buffile_seek(0, 8188);
+SELECT buffile_write('abcd');
+SELECT buffile_seek(0, 8189);
+-- Enforce flush with the write position not at the end of the buffer. This is
+-- special by not moving curOffset to the next buffer.
+SELECT buffile_read(1);
+
+-- Therefore the next writes should eventually affect the original data. (Here
+-- we also test going directly from read to write and vice versa.)
+SELECT buffile_write('x');
+SELECT buffile_read(1);
+
+-- Start a new buffer, i.e. force flushing of the previous one.
+SELECT buffile_write('z');
+
+-- Check that the 'x' and 'y' letters are in the first buffer, not in the
+-- 2nd. (We read enough data to find any non-zero bytes in the 2nd buffer.)
+SELECT buffile_seek(0, 8188);
+SELECT buffile_read(4 + 8192);
+
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_11.sql b/src/test/modules/buffile/sql/test_11.sql
new file mode 100644
index 0000000000..94300d253f
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_11.sql
@@ -0,0 +1,9 @@
+BEGIN;
+SELECT buffile_create();
+SELECT buffile_write('abcd');
+-- Seek beyond EOF not followed by write.
+SELECT buffile_seek(0, 5);
+-- Nothing should be fetched.
+SELECT buffile_read(8);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_12.sql b/src/test/modules/buffile/sql/test_12.sql
new file mode 100644
index 0000000000..a28ae395ef
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_12.sql
@@ -0,0 +1,2 @@
+SELECT buffile_test_shared();
+SELECT buffile_test_shared_append();
diff --git a/src/test/modules/buffile/sql/test_13.sql b/src/test/modules/buffile/sql/test_13.sql
new file mode 100644
index 0000000000..bcabf7bfe6
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_13.sql
@@ -0,0 +1,25 @@
+-- Use transaction block so that the file does not closed automatically at
+-- command boundary.
+BEGIN;
+SELECT buffile_open_transient('trans1', true, false);
+SELECT buffile_write('01234567');
+SELECT buffile_close_transient();
+
+-- Open for reading.
+SELECT buffile_open_transient('trans1', false, false);
+SELECT length(buffile_read(65536));
+SELECT buffile_close_transient();
+
+-- Open for writing in append mode.
+SELECT buffile_open_transient('trans1', true, true);
+-- Add BLCKSZ bytes, so that buffer boundary is crossed.
+SELECT buffile_write(repeat('x', 8192));
+SELECT buffile_close_transient();
+
+-- Open for reading and verify the valid part.
+SELECT buffile_open_transient('trans1', false, false);
+SELECT length(buffile_read(65536));
+SELECT buffile_close_transient();
+
+SELECT buffile_delete_file('trans1');
+COMMIT;
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-05-22 15:16 Robert Haas <robertmhaas@gmail.com>
parent: Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 37+ messages in thread
From: Robert Haas @ 2019-05-22 15:16 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
On Wed, Apr 24, 2019 at 10:48 AM Antonin Houska <ah@cybertec.at> wrote:
> Attached is the next version. It tries to address various problems pointed out
> upthread, including documentation.
It's not immediately evident to me, perhaps because I haven't looked
at this stuff in a while, what the purpose of 0001 and 0002 is. I
think it'd be helpful if you would generate these patches with 'git
format-patch' and include a meaningful commit message in each patch of
the series.
+ <para>
+ This setting specifies path to executable file that returns
+ either <literal>encryption_key=</literal> for key
+ and <literal>encryption_password=</literal> for password.
+ </para>
This is incomprehensible. Executables don't return strings. Also,
you can return "either a or b" or you can return "both a and b," but
you can't return "either a and b".
- Build with support for <acronym>SSL</acronym> (encrypted)
- connections. This requires the <productname>OpenSSL</productname>
- package to be installed. <filename>configure</filename> will check
- for the required header files and libraries to make sure that
+ Build with support for on-disk data encryption
+ or <acronym>SSL</acronym> (encrypted) connections. This requires
+ the <productname>OpenSSL</productname> package to be
+ installed. <filename>configure</filename> will check for the
+ required header files and libraries to make sure that
I think we should instead just remove the word 'connections'. We
don't want to have multiple places in the documentation listing all
the things for which OpenSSL is required.
Generally, the documentation changes here have a lot of stuff about
key-or-password, which is quite confusing. It seems like the idea is
that you accept either a key or a password from which a key is
derived, but I don't think that's explained super-clearly, and I
wonder whether it's unnecessary complexity. Why not just require a
key always?
Another general point is that the documentation, comments, and README
need some work on the English. They contain lots of information, but
they read somewhat awkwardly and are hard to understand in some
places.
Wherever you are adding a large hunk of code into the middle of an
existing function (e.g. XLogWrite), please consider putting the logic
in a new function and calling it from the original function, instead
of just inlining the entire thing.
xloginsert.c adds a new #include even though there are no other
changes to that file. That should not be necessary. If it is, you've
broken the principle the header files are compilable independently of
other header files (except postgres.h, which is a prerequisite for
every header file).
-XLogRead(char *buf, int segsize, TimeLineID tli, XLogRecPtr startptr,
- Size count)
+XLogRead(char *buf, int segsize, TimeLineID tli, XLogRecPtr startptr,
Size count)
Spurious hunk. Please try to avoid unnecessary changes.
+ * XXX Currently the function is only called with startptr at page
+ * boundary. If it should change, encryption needs to reflect this fact,
+ * i.e. read data from the beginning of the page. Actually this is a
+ * reason to adjust and use walsender.c:XLogRead().
This comment and the similar comment at the end of the function are
not very clear. It sorta sounds, though, like maybe the decryption
logic should move from here to the caller or something. You seem to
be saying that, after your changes, this function depends on the
caller using it in a certain way, which is often a sign that you
haven't abstracted things in the right way. And if walsender.c's
version of the logic is better, why not also do that stuff here?
@@ -52,7 +54,6 @@
uint32 bootstrap_data_checksum_version = 0; /* No checksum */
-
#define ALLOC(t, c) \
((t *) MemoryContextAllocZero(TopMemoryContext, (unsigned)(c) * sizeof(t)))
Another useless hunk.
+ {
+ RelFileNode fromNode = {srctablespace, src_dboid, InvalidOid};
+ RelFileNode toNode = {dsttablespace, dboid, InvalidOid};
+
+ copydir(srcpath, dstpath, &fromNode, &toNode);
+ }
I suggest instead declaring these variables in the surrounding scope
and initializing them here. This style is unlike most PostgreSQL
code.
+ case WAIT_EVENT_KDF_FILE_READ:
+ event_name = "KDFFileRead";
+ break;
New wait events need documentation.
+are encrypted differently. Furthermore, as the Postgres page starts with LSN,
+even if only the last encryption block of the page changes, the whole cipher
+text of the page will be different after the next encryption.
But for a hint-bit-only change, this would not be true, unless we
force wal_log_hints. Sounds like we're doing that, but it might be
worth mentioning it here also.
There's also the question of unlogged tables, which do not have an
LSN. This logic wouldn't apply there, which might be a problem.
+If the encryption is enabled, the following requirements need to be taken into
+account:
+
+1. The file buffer cannot be positioned at arbitrary offset of the file. If
I think it's no good at all to think about enforcing requirements like
this only when data_encryption is used. That's just going to lead to
bugs. I think we need to either find a way to mask this restriction
from the users of the buffile facility, or else revise all those users
so that they respect this new rule and enforce it with an
unconditional assert.
That brings me to another point, which is that a complex patch like
this really needs to be divided up into smaller patches for review. I
see you've started that a little bit (but see comments on commit
messages above) but you should try to go further with it. For
example, changing the buffile contract so that there is some new rule
about positioning should be done as a separate patch (or not done at
all). That patch should be written so that it applies before the main
encryption patch, have comments and asserts explaining the new rules,
etc.
+To store other kinds of data encrypted than the ones above, developers are
+advised to use BufFileWriteTransient() and BufFileReadTransient() functions
The previous paragraph is about BufFileWrite() and BufFileRead(), and
that section is titled "temporary files." Now here you say that for
other kinds of data (not relations, xlog, or temporary files) we
should use BufFileWriteTransient(). That seems pretty weird, because
data that it is not temporary files would presumably be, uh, permanent
files? So then why is Transient in the name? I imagine there are
good reasons here but it needs to be made clearer.
+pg_upgrade. (Therefore the encrypted cluster does not support pg_upgrade with
+the --link option.)
That seems pretty unfortunate. Any way to do better?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-05-28 15:26 Antonin Houska <ah@cybertec.at>
parent: Robert Haas <robertmhaas@gmail.com>
0 siblings, 2 replies; 37+ messages in thread
From: Antonin Houska @ 2019-05-28 15:26 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
Thanks of another round of review.
Robert Haas <robertmhaas@gmail.com> wrote:
> On Wed, Apr 24, 2019 at 10:48 AM Antonin Houska <ah@cybertec.at> wrote:
> > Attached is the next version. It tries to address various problems pointed out
> > upthread, including documentation.
>
> It's not immediately evident to me, perhaps because I haven't looked
> at this stuff in a while, what the purpose of 0001 and 0002 is. I
> think it'd be helpful if you would generate these patches with 'git
> format-patch' and include a meaningful commit message in each patch of
> the series.
ok, will do.
> + <para>
> + This setting specifies path to executable file that returns
> + either <literal>encryption_key=</literal> for key
> + and <literal>encryption_password=</literal> for password.
> + </para>
>
> This is incomprehensible. Executables don't return strings.
That's right, the key should be written to the standard output.
> Also, you can return "either a or b" or you can return "both a and b," but
> you can't return "either a and b".
Sure, this is a thinko.
> - Build with support for <acronym>SSL</acronym> (encrypted)
> - connections. This requires the <productname>OpenSSL</productname>
> - package to be installed. <filename>configure</filename> will check
> - for the required header files and libraries to make sure that
> + Build with support for on-disk data encryption
> + or <acronym>SSL</acronym> (encrypted) connections. This requires
> + the <productname>OpenSSL</productname> package to be
> + installed. <filename>configure</filename> will check for the
> + required header files and libraries to make sure that
>
> I think we should instead just remove the word 'connections'. We
> don't want to have multiple places in the documentation listing all
> the things for which OpenSSL is required.
ok
> Generally, the documentation changes here have a lot of stuff about
> key-or-password, which is quite confusing. It seems like the idea is
> that you accept either a key or a password from which a key is
> derived, but I don't think that's explained super-clearly, and I
> wonder whether it's unnecessary complexity. Why not just require a
> key always?
We thought that it's more convenient for administrator to enter password. To
achieve that, we can instead tell the admin how to use the key derivation tool
(pg_keysetup) and send its output to postgres. So yes, it's possible to always
receive the key from the "encryption key command". I'll change this.
> Another general point is that the documentation, comments, and README
> need some work on the English. They contain lots of information, but
> they read somewhat awkwardly and are hard to understand in some
> places.
ok, I'll try to explain the tricky things in simple senteces.
> Wherever you are adding a large hunk of code into the middle of an
> existing function (e.g. XLogWrite), please consider putting the logic
> in a new function and calling it from the original function, instead
> of just inlining the entire thing.
ok
> xloginsert.c adds a new #include even though there are no other
> changes to that file. That should not be necessary. If it is, you've
> broken the principle the header files are compilable independently of
> other header files (except postgres.h, which is a prerequisite for
> every header file).
It's not intentional, I failed to remove it when moving some encryption
related function calls elsewhere.
> -XLogRead(char *buf, int segsize, TimeLineID tli, XLogRecPtr startptr,
> - Size count)
> +XLogRead(char *buf, int segsize, TimeLineID tli, XLogRecPtr startptr,
> Size count)
> Spurious hunk. Please try to avoid unnecessary changes.
Maybe I added and later removed some argument and forgot to enforce the
maximum line length. I expected pg_indent to fix things like this but it
apparently does not. I'll pay more attention to such things next time.
> + * XXX Currently the function is only called with startptr at page
> + * boundary. If it should change, encryption needs to reflect this fact,
> + * i.e. read data from the beginning of the page. Actually this is a
> + * reason to adjust and use walsender.c:XLogRead().
>
> This comment and the similar comment at the end of the function are
> not very clear. It sorta sounds, though, like maybe the decryption
> logic should move from here to the caller or something. You seem to
> be saying that, after your changes, this function depends on the
> caller using it in a certain way, which is often a sign that you
> haven't abstracted things in the right way. And if walsender.c's
> version of the logic is better, why not also do that stuff here?
Please consider this an incomplete implementation of the XLOG
decryption. Eventually these restrictions will be relaxed and the decryption
will be hidden from caller. Currently there are two implementations of
XLogRead() in the backend (plus one in the frontend) and I didn't want to
spend too much effort on merging the decryption into core until this another
patch gets committed:
https://commitfest.postgresql.org/23/2098/
The current XLogRead() functions are not identical so the decryption specific
code can't be simply copy & pasted.
So far I'll turn this comment into a TODO comment and mention the refactoring
patch there.
> + {
> + RelFileNode fromNode = {srctablespace, src_dboid, InvalidOid};
> + RelFileNode toNode = {dsttablespace, dboid, InvalidOid};
> +
> + copydir(srcpath, dstpath, &fromNode, &toNode);
> + }
>
> I suggest instead declaring these variables in the surrounding scope
> and initializing them here. This style is unlike most PostgreSQL
> code.
ok
> + case WAIT_EVENT_KDF_FILE_READ:
> + event_name = "KDFFileRead";
> + break;
>
> New wait events need documentation.
ok
> +are encrypted differently. Furthermore, as the Postgres page starts with LSN,
> +even if only the last encryption block of the page changes, the whole cipher
> +text of the page will be different after the next encryption.
>
> But for a hint-bit-only change, this would not be true, unless we
> force wal_log_hints. Sounds like we're doing that, but it might be
> worth mentioning it here also.
ok
> There's also the question of unlogged tables, which do not have an
> LSN. This logic wouldn't apply there, which might be a problem.
I missed that. The only idea I have now is that a "fake LSN" is set even for
unlogged table if encryption is enabled. Actually that LSN would only be a
counter that gets incremented each time the page contents has changed.
> +If the encryption is enabled, the following requirements need to be taken into
> +account:
> +
> +1. The file buffer cannot be positioned at arbitrary offset of the file. If
>
> I think it's no good at all to think about enforcing requirements like
> this only when data_encryption is used. That's just going to lead to
> bugs. I think we need to either find a way to mask this restriction
> from the users of the buffile facility, or else revise all those users
> so that they respect this new rule and enforce it with an
> unconditional assert.
Sorry, this is misunderstanding. The requirements mentioned in the "Temporary
files" section of the README apply to BufFileWrite() and BufFileRead()
internally. These functions take care of the correct buffer positioning. The
caller does not have to care. I'll try to make this part clearer.
> That brings me to another point, which is that a complex patch like
> this really needs to be divided up into smaller patches for review. I
> see you've started that a little bit (but see comments on commit
> messages above) but you should try to go further with it.
Maintenance of a long patch series requires additional effort and I was busy
enough with major code rearrangements so far. I'll continue splitting the
stuff into more diffs.
> +To store other kinds of data encrypted than the ones above, developers are
> +advised to use BufFileWriteTransient() and BufFileReadTransient() functions
>
> The previous paragraph is about BufFileWrite() and BufFileRead(), and
> that section is titled "temporary files." Now here you say that for
> other kinds of data (not relations, xlog, or temporary files) we
> should use BufFileWriteTransient(). That seems pretty weird, because
> data that it is not temporary files would presumably be, uh, permanent
> files? So then why is Transient in the name? I imagine there are
> good reasons here but it needs to be made clearer.
"Transient" here means that the file descriptor is kept open for rather short
time, typically until transaction boundary (unlike relation files). However
the file remains on disk even if the descriptor is closed. I believe this is
consistent with the use of OpenTransientFile(). The phrase "other kinds of
data" is probably too generic and more explanation is needed.
> +pg_upgrade. (Therefore the encrypted cluster does not support pg_upgrade with
> +the --link option.)
>
> That seems pretty unfortunate. Any way to do better?
The reason is that the initialization vector (IV) for relation page encryption
is derived from RelFileNode, and both dbNode and relNode can be different in
the new cluster. Therefore the data of the old cluster need to be decrypted
and encrypted using the new IV before it's copied to the new cluster. That's
why we can't use symlink.
As an alternative, I thought of deriving the IV by hashing the
<database>.<schema>.<object name> string, but that would mean that the
"reencryption" is triggered by commands like "ALTER TABLE ... RENAME TO ...",
"ALTER TABLE ... SET SCHEMA ...", etc.
Currently I'm thinking of making the IV less dependent on RelFileNode
(e.g. generate a random number for which RelFileNode serves as the seed) and
storing it in pg_class as a storage parameter. Therefore we wouldn't have to
pay any extra attention to transfer of the IV to the new cluster. However, the
IV storage parameter would need special treatment:
1. DBA should not be able to remove or change it using ALTER TABLE command
because inability to decrypt the data can be the only outcome.
2. The \d+ command of psql client should not display the IV. Displaying it
probably wouldn't be a security risk, but as we encrypt the whole instance,
the IV would appear for every single table and that might be annoying.
What do you think about this approach?
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-05-29 21:51 Robert Haas <robertmhaas@gmail.com>
parent: Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 37+ messages in thread
From: Robert Haas @ 2019-05-29 21:51 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
On Tue, May 28, 2019 at 11:27 AM Antonin Houska <ah@cybertec.at> wrote:
> We thought that it's more convenient for administrator to enter password. To
> achieve that, we can instead tell the admin how to use the key derivation tool
> (pg_keysetup) and send its output to postgres. So yes, it's possible to always
> receive the key from the "encryption key command". I'll change this.
Sounds good. I'm not quite sure how this is going to work. Ideally
you'd like the encryption key command to fetch the key from something
like ssh-agent, or maybe pop up a window on the user's terminal with a
key prompt. Just reading from stdin and writing to stdout is not
going to be very convenient...
> Maintenance of a long patch series requires additional effort and I was busy
> enough with major code rearrangements so far. I'll continue splitting the
> stuff into more diffs.
Right, I understand. Whatever can be split off can be reviewed and
committed separately, which will start to ease the burden of
maintaining the patch set. I hope. But getting over the initial hump
can be a lot of work.
> > +pg_upgrade. (Therefore the encrypted cluster does not support pg_upgrade with
> > +the --link option.)
> >
> > That seems pretty unfortunate. Any way to do better?
>
> The reason is that the initialization vector (IV) for relation page encryption
> is derived from RelFileNode, and both dbNode and relNode can be different in
> the new cluster. Therefore the data of the old cluster need to be decrypted
> and encrypted using the new IV before it's copied to the new cluster. That's
> why we can't use symlink.
>
> As an alternative, I thought of deriving the IV by hashing the
> <database>.<schema>.<object name> string, but that would mean that the
> "reencryption" is triggered by commands like "ALTER TABLE ... RENAME TO ...",
> "ALTER TABLE ... SET SCHEMA ...", etc.
>
> Currently I'm thinking of making the IV less dependent on RelFileNode
> (e.g. generate a random number for which RelFileNode serves as the seed) and
> storing it in pg_class as a storage parameter. Therefore we wouldn't have to
> pay any extra attention to transfer of the IV to the new cluster. However, the
> IV storage parameter would need special treatment:
>
> 1. DBA should not be able to remove or change it using ALTER TABLE command
> because inability to decrypt the data can be the only outcome.
>
> 2. The \d+ command of psql client should not display the IV. Displaying it
> probably wouldn't be a security risk, but as we encrypt the whole instance,
> the IV would appear for every single table and that might be annoying.
>
> What do you think about this approach?
I don't think it's a very good idea. For one thing, you can't store
the IV for pg_class in pg_class; that has a circularity problem. For
another thing, I'm not sure we can or should try to do catalog access
from every place where an IV might be needed. In fact, I'd go so far
as to say that sounds like a recipe for a lot of pain and fragility.
One potential approach is to modify pg_upgrade to preserve the dbnode
and relfilenode. I don't know of any fundamental reason why such a
change shouldn't be possible, although it is probably a bunch of work,
and there may be problems with which I am not acquainted.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-05-31 06:59 Antonin Houska <ah@cybertec.at>
parent: Robert Haas <robertmhaas@gmail.com>
0 siblings, 1 reply; 37+ messages in thread
From: Antonin Houska @ 2019-05-31 06:59 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
Robert Haas <robertmhaas@gmail.com> wrote:
> On Tue, May 28, 2019 at 11:27 AM Antonin Houska <ah@cybertec.at> wrote:
> > We thought that it's more convenient for administrator to enter password. To
> > achieve that, we can instead tell the admin how to use the key derivation tool
> > (pg_keysetup) and send its output to postgres. So yes, it's possible to always
> > receive the key from the "encryption key command". I'll change this.
>
> Sounds good. I'm not quite sure how this is going to work. Ideally
> you'd like the encryption key command to fetch the key from something
> like ssh-agent, or maybe pop up a window on the user's terminal with a
> key prompt. Just reading from stdin and writing to stdout is not
> going to be very convenient...
When I mentioned writing to stdout in my previous email, I viewed it from the
perspective of postmaster: whichever way the command gets the password from
the DBA, it'll always write it to stdout and postmaster will read it from
there. This was to address your objection that the executables do not actually
"return" strings.
> > > +pg_upgrade. (Therefore the encrypted cluster does not support pg_upgrade with
> > > +the --link option.)
> > >
> > > That seems pretty unfortunate. Any way to do better?
> >
> >
> > Currently I'm thinking of making the IV less dependent on RelFileNode
> > (e.g. generate a random number for which RelFileNode serves as the seed) and
> > storing it in pg_class as a storage parameter.
>
> I don't think it's a very good idea. For one thing, you can't store
> the IV for pg_class in pg_class; that has a circularity problem. For
> another thing, I'm not sure we can or should try to do catalog access
> from every place where an IV might be needed. In fact, I'd go so far
> as to say that sounds like a recipe for a lot of pain and fragility.
>
> One potential approach is to modify pg_upgrade to preserve the dbnode
> and relfilenode. I don't know of any fundamental reason why such a
> change shouldn't be possible, although it is probably a bunch of work,
> and there may be problems with which I am not acquainted.
Actually it doesn't seem to be that tricky. In the --binary-upgrade mode,
pg_dump does record pg_class.oid:
-- For binary upgrade, must preserve pg_class oids
SELECT pg_catalog.binary_upgrade_set_next_heap_pg_class_oid('57465'::pg_catalog.oid);
The header comment in pg_upgrade.c indicates that this is because of TOAST. So
I think that dbnode and relfilenode are not preserved just because there was
no strong reason to do so by now.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-06-03 16:04 Robert Haas <robertmhaas@gmail.com>
parent: Antonin Houska <ah@cybertec.at>
0 siblings, 2 replies; 37+ messages in thread
From: Robert Haas @ 2019-06-03 16:04 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
On Fri, May 31, 2019 at 2:59 AM Antonin Houska <ah@cybertec.at> wrote:
> > Sounds good. I'm not quite sure how this is going to work. Ideally
> > you'd like the encryption key command to fetch the key from something
> > like ssh-agent, or maybe pop up a window on the user's terminal with a
> > key prompt. Just reading from stdin and writing to stdout is not
> > going to be very convenient...
>
> When I mentioned writing to stdout in my previous email, I viewed it from the
> perspective of postmaster: whichever way the command gets the password from
> the DBA, it'll always write it to stdout and postmaster will read it from
> there. This was to address your objection that the executables do not actually
> "return" strings.
So the part about returning strings is really just a wording issue:
the documentation needs to talk about data sent to stdout or wherever,
because that's what really happens, and somebody writing such a
command from scratch needs to know what it must do.
What I'm talking about here is that it also has to be reasonably
possible to write an encryption key command that does something
useful. I don't have a really clear vision for how that's going to
work. Nobody wants the server, which is probably being launched by
pg_ctl or systemd or both, to prompt using its own stdin/stderr, but
the we need to think about how the prompting is actually going to
work. One idea is to do it via the FEBE protocol: connect to the
server using libpq, issue a new command that causes the server to
enter COPY mode, and then send the encryption key as COPY data.
However, that idea doesn't really work, because we've got to be able
to get the key before we run recovery or reach consistency, so the
server won't be listening for connections at that point. Also, we've
got to have a way for this to work in single-user mode, which also
can't listen for connections. It's probably all fine if your
encryption key command is something like 'curl
https://my.secret.keyserver.me/sekcret.key'; because then there's an
external server which you can just go access - but I don't quite
understand how you'd do interactive prompting from here. Sorry to get
hung up on what may seem like a minor point, but I think it's actually
fairly fundamental: we've got to have a clear vision of how end-users
will really be able to make use of this.
> The header comment in pg_upgrade.c indicates that this is because of TOAST. So
> I think that dbnode and relfilenode are not preserved just because there was
> no strong reason to do so by now.
Maybe you want to propose an independent patch making that change?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-06-05 12:32 Antonin Houska <ah@cybertec.at>
parent: Robert Haas <robertmhaas@gmail.com>
1 sibling, 1 reply; 37+ messages in thread
From: Antonin Houska @ 2019-06-05 12:32 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
Robert Haas <robertmhaas@gmail.com> wrote:
> On Fri, May 31, 2019 at 2:59 AM Antonin Houska <ah@cybertec.at> wrote:
> > > Sounds good. I'm not quite sure how this is going to work. Ideally
> > > you'd like the encryption key command to fetch the key from something
> > > like ssh-agent, or maybe pop up a window on the user's terminal with a
> > > key prompt. Just reading from stdin and writing to stdout is not
> > > going to be very convenient...
> >
> > When I mentioned writing to stdout in my previous email, I viewed it from the
> > perspective of postmaster: whichever way the command gets the password from
> > the DBA, it'll always write it to stdout and postmaster will read it from
> > there. This was to address your objection that the executables do not actually
> > "return" strings.
>
> So the part about returning strings is really just a wording issue:
> the documentation needs to talk about data sent to stdout or wherever,
> because that's what really happens, and somebody writing such a
> command from scratch needs to know what it must do.
>
> What I'm talking about here is that it also has to be reasonably
> possible to write an encryption key command that does something
> useful. I don't have a really clear vision for how that's going to
> work. Nobody wants the server, which is probably being launched by
> pg_ctl or systemd or both, to prompt using its own stdin/stderr, but
> the we need to think about how the prompting is actually going to
> work.
Since you mentioned ssh-agent above, I think that postmaster can, instead of
running a command, create an unix socket and read the key from there. (I refer
only to the socket as a communication channel, not to the protocol - ssh-agent
does not seem to actually send key over the socket.) Unlike the socket for
backend connections, this one would only be readable by the user that runs
postmaster, and would only exist during the encryption initialization.
The simplest approach from the perspective of the DBA is that pg_ctl can write
the key to the socket. Besides that we can also implement a separate utility
to send the key, to be used in other special cases such as starting postgres
via systemd.
(If the unix socket is a problem on windows, we might need to use named pipe
instead.)
> > The header comment in pg_upgrade.c indicates that this is because of TOAST. So
> > I think that dbnode and relfilenode are not preserved just because there was
> > no strong reason to do so by now.
>
> Maybe you want to propose an independent patch making that change?
I think of a separate diff in the encryption patch series. As the encryption
is the only reason for this enhancement, I'm not sure if it deserves a
separate CF entry. (Although it might be considered refactoring because
eventually pg_upgrade won't need to handle the different relnodes, and thus it
might become a little bit simpler.)
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-06-06 10:58 Pavlo Golub <pavlo.golub@cybertec.at>
parent: Antonin Houska <ah@cybertec.at>
0 siblings, 0 replies; 37+ messages in thread
From: Pavlo Golub @ 2019-06-06 10:58 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; Robert Haas <robertmhaas@gmail.com>; +Cc: Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
Greetings, Antonin.
You wrote 2019-06-05, 15:32:
> Robert Haas <robertmhaas@gmail.com> wrote:
>> On Fri, May 31, 2019 at 2:59 AM Antonin Houska <ah@cybertec.at> wrote:
>> > > Sounds good. I'm not quite sure how this is going to work. Ideally
>> > > you'd like the encryption key command to fetch the key from something
>> > > like ssh-agent, or maybe pop up a window on the user's terminal with a
>> > > key prompt. Just reading from stdin and writing to stdout is not
>> > > going to be very convenient...
>> >
>> > When I mentioned writing to stdout in my previous email, I viewed it from the
>> > perspective of postmaster: whichever way the command gets the password from
>> > the DBA, it'll always write it to stdout and postmaster will read it from
>> > there. This was to address your objection that the executables do not actually
>> > "return" strings.
>>
>> So the part about returning strings is really just a wording issue:
>> the documentation needs to talk about data sent to stdout or wherever,
>> because that's what really happens, and somebody writing such a
>> command from scratch needs to know what it must do.
>>
>> What I'm talking about here is that it also has to be reasonably
>> possible to write an encryption key command that does something
>> useful. I don't have a really clear vision for how that's going to
>> work. Nobody wants the server, which is probably being launched by
>> pg_ctl or systemd or both, to prompt using its own stdin/stderr, but
>> the we need to think about how the prompting is actually going to
>> work.
> Since you mentioned ssh-agent above, I think that postmaster can, instead of
> running a command, create an unix socket and read the key from there. (I refer
> only to the socket as a communication channel, not to the protocol - ssh-agent
> does not seem to actually send key over the socket.) Unlike the socket for
> backend connections, this one would only be readable by the user that runs
> postmaster, and would only exist during the encryption initialization.
> The simplest approach from the perspective of the DBA is that pg_ctl can write
> the key to the socket. Besides that we can also implement a separate utility
> to send the key, to be used in other special cases such as starting postgres
> via systemd.
> (If the unix socket is a problem on windows, we might need to use named pipe
> instead.)
Yes, that definitely a problem on Windows. Pipes seems reasonable. Are
mapped files are appropriate from the security point of view?
>> > The header comment in pg_upgrade.c indicates that this is because of TOAST. So
>> > I think that dbnode and relfilenode are not preserved just because there was
>> > no strong reason to do so by now.
>>
>> Maybe you want to propose an independent patch making that change?
> I think of a separate diff in the encryption patch series. As the encryption
> is the only reason for this enhancement, I'm not sure if it deserves a
> separate CF entry. (Although it might be considered refactoring because
> eventually pg_upgrade won't need to handle the different relnodes, and thus it
> might become a little bit simpler.)
--
Kind regards,
Pavlo mailto:pavlo.golub@cybertec.at
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-06-14 01:14 Bruce Momjian <bruce@momjian.us>
parent: Robert Haas <robertmhaas@gmail.com>
1 sibling, 2 replies; 37+ messages in thread
From: Bruce Momjian @ 2019-06-14 01:14 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
On Mon, Jun 3, 2019 at 12:04:54PM -0400, Robert Haas wrote:
>
> What I'm talking about here is that it also has to be reasonably
> possible to write an encryption key command that does something
> useful. I don't have a really clear vision for how that's going to
> work. Nobody wants the server, which is probably being launched by
> pg_ctl or systemd or both, to prompt using its own stdin/stderr, but
> the we need to think about how the prompting is actually going to
> work. One idea is to do it via the FEBE protocol: connect to the
> server using libpq, issue a new command that causes the server to
> enter COPY mode, and then send the encryption key as COPY data.
> However, that idea doesn't really work, because we've got to be able
> to get the key before we run recovery or reach consistency, so the
> server won't be listening for connections at that point. Also, we've
> got to have a way for this to work in single-user mode, which also
> can't listen for connections. It's probably all fine if your
> encryption key command is something like 'curl
> https://my.secret.keyserver.me/sekcret.key'; because then there's an
> external server which you can just go access - but I don't quite
> understand how you'd do interactive prompting from here. Sorry to get
> hung up on what may seem like a minor point, but I think it's actually
> fairly fundamental: we've got to have a clear vision of how end-users
> will really be able to make use of this.
pgcryptoey has an example of prompting from /dev/tty:
http://momjian.us/download/pgcryptokey/
Look at pgcryptokey_default.sample.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-06-14 13:02 Bruce Momjian <bruce@momjian.us>
parent: Bruce Momjian <bruce@momjian.us>
1 sibling, 0 replies; 37+ messages in thread
From: Bruce Momjian @ 2019-06-14 13:02 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
On Thu, Jun 13, 2019 at 09:14:13PM -0400, Bruce Momjian wrote:
> On Mon, Jun 3, 2019 at 12:04:54PM -0400, Robert Haas wrote:
> >
> > What I'm talking about here is that it also has to be reasonably
> > possible to write an encryption key command that does something
> > useful. I don't have a really clear vision for how that's going to
> > work. Nobody wants the server, which is probably being launched by
> > pg_ctl or systemd or both, to prompt using its own stdin/stderr, but
> > the we need to think about how the prompting is actually going to
> > work. One idea is to do it via the FEBE protocol: connect to the
> > server using libpq, issue a new command that causes the server to
> > enter COPY mode, and then send the encryption key as COPY data.
> > However, that idea doesn't really work, because we've got to be able
> > to get the key before we run recovery or reach consistency, so the
> > server won't be listening for connections at that point. Also, we've
> > got to have a way for this to work in single-user mode, which also
> > can't listen for connections. It's probably all fine if your
> > encryption key command is something like 'curl
> > https://my.secret.keyserver.me/sekcret.key'; because then there's an
> > external server which you can just go access - but I don't quite
> > understand how you'd do interactive prompting from here. Sorry to get
> > hung up on what may seem like a minor point, but I think it's actually
> > fairly fundamental: we've got to have a clear vision of how end-users
> > will really be able to make use of this.
>
> pgcryptoey has an example of prompting from /dev/tty:
>
> http://momjian.us/download/pgcryptokey/
>
> Look at pgcryptokey_default.sample.
Also, look at pgcryptokey_default.c and its use with
shared_preload_libraries for a full example of prompting on server boot.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-06-17 09:23 Antonin Houska <ah@cybertec.at>
parent: Bruce Momjian <bruce@momjian.us>
1 sibling, 1 reply; 37+ messages in thread
From: Antonin Houska @ 2019-06-17 09:23 UTC (permalink / raw)
To: Bruce Momjian <bruce@momjian.us>; +Cc: Robert Haas <robertmhaas@gmail.com>; Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
Bruce Momjian <bruce@momjian.us> wrote:
> On Mon, Jun 3, 2019 at 12:04:54PM -0400, Robert Haas wrote:
> >
> > What I'm talking about here is that it also has to be reasonably
> > possible to write an encryption key command that does something
> > useful. I don't have a really clear vision for how that's going to
> > work. Nobody wants the server, which is probably being launched by
> > pg_ctl or systemd or both, to prompt using its own stdin/stderr, but
> > the we need to think about how the prompting is actually going to
> > work. One idea is to do it via the FEBE protocol: connect to the
> > server using libpq, issue a new command that causes the server to
> > enter COPY mode, and then send the encryption key as COPY data.
> > However, that idea doesn't really work, because we've got to be able
> > to get the key before we run recovery or reach consistency, so the
> > server won't be listening for connections at that point. Also, we've
> > got to have a way for this to work in single-user mode, which also
> > can't listen for connections. It's probably all fine if your
> > encryption key command is something like 'curl
> > https://my.secret.keyserver.me/sekcret.key'; because then there's an
> > external server which you can just go access - but I don't quite
> > understand how you'd do interactive prompting from here. Sorry to get
> > hung up on what may seem like a minor point, but I think it's actually
> > fairly fundamental: we've got to have a clear vision of how end-users
> > will really be able to make use of this.
>
> pgcryptoey has an example of prompting from /dev/tty:
>
> http://momjian.us/download/pgcryptokey/
>
> Look at pgcryptokey_default.sample.
It's a nice exercise of shell features but does not seem to be easily
portable, and it has some other restrictions. One is mentioned in a comment:
# Use 'pg_ctl -s' so starting dots do not interfere with password entry
I think that besides the dots, log messages can also be disturbing if logging
collector is not enabled. Another corner case might be there when user runs
pg_ctl with --no-wait option and wants to do run some other command
immediately.
I'm thinking how to teach postmaster to accept FEBE protocol connections
temporarily, just to receive the key. The user applications like pg_ctl,
initdb or pg_upgrade would retrieve the key / password from the DBA, then
start postmaster and send it the key.
Perhaps the message format should be a bit generic so that extensions like
this can use it to receive their keys too.
(The idea of an unix socket or named pipe I proposed upthread is not good
because it's harder to implement in a portable way.)
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-06-25 12:28 Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
parent: Antonin Houska <ah@cybertec.at>
0 siblings, 2 replies; 37+ messages in thread
From: Peter Eisentraut @ 2019-06-25 12:28 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; Bruce Momjian <bruce@momjian.us>; +Cc: Robert Haas <robertmhaas@gmail.com>; Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
On 2019-06-17 11:23, Antonin Houska wrote:
> I'm thinking how to teach postmaster to accept FEBE protocol connections
> temporarily, just to receive the key. The user applications like pg_ctl,
> initdb or pg_upgrade would retrieve the key / password from the DBA, then
> start postmaster and send it the key.
>
> Perhaps the message format should be a bit generic so that extensions like
> this can use it to receive their keys too.
>
> (The idea of an unix socket or named pipe I proposed upthread is not good
> because it's harder to implement in a portable way.)
How are the requirements here different from ssl_passphrase_command?
Why do we need a new mechanism?
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-06-25 13:39 Robert Haas <robertmhaas@gmail.com>
parent: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
1 sibling, 1 reply; 37+ messages in thread
From: Robert Haas @ 2019-06-25 13:39 UTC (permalink / raw)
To: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>; +Cc: Antonin Houska <ah@cybertec.at>; Bruce Momjian <bruce@momjian.us>; Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
On Tue, Jun 25, 2019 at 8:28 AM Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
> How are the requirements here different from ssl_passphrase_command?
> Why do we need a new mechanism?
I don't think that the requirements are different, and I don't think
we need a new mechanism.
I am curious exactly how you would set up ssl_passphrase_command to
prompt interactively.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-06-26 13:05 Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
parent: Robert Haas <robertmhaas@gmail.com>
0 siblings, 0 replies; 37+ messages in thread
From: Peter Eisentraut @ 2019-06-26 13:05 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Bruce Momjian <bruce@momjian.us>; Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
On 2019-06-25 15:39, Robert Haas wrote:
> On Tue, Jun 25, 2019 at 8:28 AM Peter Eisentraut
> <peter.eisentraut@2ndquadrant.com> wrote:
>> How are the requirements here different from ssl_passphrase_command?
>> Why do we need a new mechanism?
>
> I don't think that the requirements are different, and I don't think
> we need a new mechanism.
>
> I am curious exactly how you would set up ssl_passphrase_command to
> prompt interactively.
For example like this:
https://www.2ndquadrant.com/en/blog/postgresql-passphrase-protected-ssl-keys-systemd/
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-07-05 14:41 Antonin Houska <ah@cybertec.at>
parent: Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 37+ messages in thread
From: Antonin Houska @ 2019-07-05 14:41 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
Antonin Houska <ah@cybertec.at> wrote:
> Thanks of another round of review.
>
> Robert Haas <robertmhaas@gmail.com> wrote:
>
> > On Wed, Apr 24, 2019 at 10:48 AM Antonin Houska <ah@cybertec.at> wrote:
> > > Attached is the next version. It tries to address various problems pointed out
> > > upthread, including documentation.
This is the next version.
I've reworked the way key is passed to postmaster (v04-0003-...) so it's
easier to call the encryption key command interactively, adjusted pg_upgrade
so that it preserves database, tablespace and relfilenode OIDs (v04-0014-...),
reorganized the code a bit and split the code into more diffs.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
Attachments: download all (17 files, .tar.gz)
v04-0001-Make-postgres-aware-of-the-encryption.patch (text/x-diff, 38.6K) download
From a6cdae26675a74d102dfe38a3a8f77430ea5480d Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Fri, 5 Jul 2019 16:24:01 +0200
Subject: [PATCH 01/17] Make postgres aware of the encryption
This patch makes postgres aware of the encryption (i.e. sets up encryption key
and initializes the encryption library) but does not actually encrypt any
data.
If initdb is passed the "encryption key command" (the -K option), it executes
it and reads the encryption key from the command's stdout:
initdb -D data -K "echo 71916b8b93063c280e4e0ee646395f450bda4f1bac85c57bc3a6afd168f3ab47"
-K option (without any argument) is then passed to bootstrap process just so
it knows that the new cluster should be encrypted. The first thing the
bootstrap process does in such a case is that it reads the encryption key from
stdin. The point is that a flag is eventually sets a control file that
indidicates that encryption is enabled for the cluster. A sample string is
also encrypted and stored in the control file. It'll be used on each startup
to verify that user passed the correct key.
pg_controldata utility shows two new fields now: "Data encryption" and "Data
encryption fingerprint".
Once a standalone backend it started, it reads the control file and recognizes
the cluster is encrypted. Therefore it expects the encryption key to be the
first user's input:
postgres --single -D data postgres
key>71916b8b93063c280e4e0ee646395f450bda4f1bac85c57bc3a6afd168f3ab47
Once the correct key is passed, the session can continue as if there was no
encryption. The only difference user can see is that "data_encryption"
variable is set:
PostgreSQL stand-alone backend 13devel
backend> SHOW data_encryption;
1: data_encryption (typeid = 25, len = -1, typmod = -1, byval = f)
----
1: data_encryption = "on" (typeid = 25, len = -1, typmod = -1, byval = f)
----
---
src/backend/access/transam/xlog.c | 72 ++++++++
src/backend/bootstrap/bootstrap.c | 48 ++++-
src/backend/storage/file/Makefile | 2 +-
src/backend/storage/file/encryption.c | 313 ++++++++++++++++++++++++++++++++
src/backend/tcop/postgres.c | 30 +++
src/backend/utils/misc/guc.c | 12 ++
src/bin/initdb/.gitignore | 1 +
src/bin/initdb/Makefile | 7 +-
src/bin/initdb/initdb.c | 74 +++++++-
src/bin/pg_controldata/pg_controldata.c | 9 +
src/common/string.c | 39 ++++
src/fe_utils/Makefile | 3 +-
src/fe_utils/encryption.c | 90 +++++++++
src/include/catalog/pg_control.h | 11 ++
src/include/common/string.h | 1 +
src/include/fe_utils/encryption.h | 18 ++
src/include/pg_config_manual.h | 7 +
src/include/storage/encryption.h | 99 ++++++++++
18 files changed, 829 insertions(+), 7 deletions(-)
create mode 100644 src/backend/storage/file/encryption.c
create mode 100644 src/fe_utils/encryption.c
create mode 100644 src/include/fe_utils/encryption.h
create mode 100644 src/include/storage/encryption.h
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 13e0d2366f..6b58e0ac76 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -55,6 +55,7 @@
#include "replication/walreceiver.h"
#include "replication/walsender.h"
#include "storage/bufmgr.h"
+#include "storage/encryption.h"
#include "storage/fd.h"
#include "storage/ipc.h"
#include "storage/large_object.h"
@@ -77,6 +78,7 @@
#include "pg_trace.h"
extern uint32 bootstrap_data_checksum_version;
+extern char *bootstrap_encryption_sample;
/* Unsupported old recovery command file names (relative to $PGDATA) */
#define RECOVERY_COMMAND_FILE "recovery.conf"
@@ -4774,6 +4776,58 @@ ReadControlFile(void)
ereport(ERROR, (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("\"max_wal_size\" must be at least twice \"wal_segment_size\"")));
+ /*
+ * Initialize encryption, but not if the current backend has already done
+ * that.
+ */
+ if (ControlFile->data_cipher > PG_CIPHER_NONE && !data_encrypted)
+ {
+ /*
+ * Set data_encryption for caller to know that he needs to retrieve
+ * the key and initialize the encryption library.
+ */
+ SetConfigOption("data_encryption", "true", PGC_INTERNAL,
+ PGC_S_OVERRIDE);
+
+ /*
+ * Save the verification string. We'll perform the actual verification
+ * as soon as the encryption setup is done.
+ */
+ memcpy(encryption_verification,
+ ControlFile->encryption_verification,
+ ENCRYPTION_SAMPLE_SIZE);
+
+ /*
+ * full_page_writes must be set because torn page write of an
+ * encrypted page implies that decryption of the page will produce
+ * garbage. This damage can affect even those parts of the page that
+ * haven't been modified by any access method. And since no access
+ * method modified those parts, there might be no XLOG records to
+ * repair them during crash recovery. So full page image is the only
+ * way to fix such a page.
+ *
+ * Do not enforce this setting in binary upgrade mode, since
+ * pg_upgrade sets it to off for performance reasons. (If the upgrade
+ * crashes, the new cluster must be created from scratch anyway.)
+ *
+ * XXX It would be nice to have guc.c check so that we don't have to
+ * copy and paste the error message. However it's unclear how to
+ * ensure that either ERROR is raised or nothing happens at all. It
+ * seems that set_config_option() can change
+ * config_generic.reset_source if the check succeeded, but that's too
+ * invasive.
+ */
+ if (!fullPageWrites && !IsBinaryUpgrade)
+ ereport(FATAL,
+ (errmsg("invalid value for parameter \"full_page_writes\": %d",
+ fullPageWrites),
+ errdetail("Cannot disable parameter when the cluster is encrypted.")));
+ }
+
+ /*
+ * This calculation relies on data_encryption (in particular the header
+ * sizes do), so we could not do it earlier.
+ */
UsableBytesInSegment =
(wal_segment_size / XLOG_BLCKSZ * UsableBytesInPage) -
(SizeOfXLogLongPHD - SizeOfXLogShortPHD);
@@ -5255,6 +5309,24 @@ BootStrapXLOG(void)
ControlFile->track_commit_timestamp = track_commit_timestamp;
ControlFile->data_checksum_version = bootstrap_data_checksum_version;
+ if (data_encrypted)
+ {
+ char sample[ENCRYPTION_SAMPLE_SIZE];
+
+ ControlFile->data_cipher = PG_CIPHER_AES_BLOCK_CBC_256_STREAM_CTR_256;
+
+ sample_encryption(sample);
+
+ memcpy(ControlFile->encryption_verification, sample,
+ ENCRYPTION_SAMPLE_SIZE);
+ }
+ else
+ {
+ ControlFile->data_cipher = PG_CIPHER_NONE;
+ memset(ControlFile->encryption_verification, 0,
+ ENCRYPTION_SAMPLE_SIZE);
+ }
+
/* some additional ControlFile fields are set in WriteControlFile() */
WriteControlFile();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 43627ab8f4..406fff51a9 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/bufmgr.h"
#include "storage/bufpage.h"
#include "storage/condition_variable.h"
+#include "storage/encryption.h"
#include "storage/ipc.h"
#include "storage/proc.h"
#include "tcop/tcopprot.h"
@@ -58,6 +59,7 @@ uint32 bootstrap_data_checksum_version = 0; /* No checksum */
static void CheckerModeMain(void);
static void BootstrapModeMain(void);
+static int bootstrap_getc(void);
static void bootstrap_signals(void);
static void ShutdownAuxiliaryProcess(int code, Datum arg);
static Form_pg_attribute AllocateAttribute(void);
@@ -226,7 +228,7 @@ AuxiliaryProcessMain(int argc, char *argv[])
/* If no -x argument, we are a CheckerProcess */
MyAuxProcType = CheckerProcess;
- while ((flag = getopt(argc, argv, "B:c:d:D:Fkr:x:X:-:")) != -1)
+ while ((flag = getopt(argc, argv, "B:c:d:D:FkKr:x:X:-:")) != -1)
{
switch (flag)
{
@@ -252,6 +254,22 @@ AuxiliaryProcessMain(int argc, char *argv[])
case 'F':
SetConfigOption("fsync", "false", PGC_POSTMASTER, PGC_S_ARGV);
break;
+#ifdef USE_ENCRYPTION
+ case 'K':
+ /*
+ * When auxiliary process (bootstrap) starts, the control file
+ * does not exist yet, so command line option needs to be used
+ * to indicate that the encryption is enabled.
+ *
+ * Postmaster should not pass this option. Instead, it just
+ * sets data_encrypted according to the control file and child
+ * processes inherit that.
+ */
+ Assert(!IsUnderPostmaster);
+ data_encrypted = true;
+
+ break;
+#endif /* USE_ENCRYPTION */
case 'k':
bootstrap_data_checksum_version = PG_DATA_CHECKSUM_VERSION;
break;
@@ -373,6 +391,24 @@ AuxiliaryProcessMain(int argc, char *argv[])
if (!IsUnderPostmaster)
InitializeMaxBackends();
+ /*
+ * If data_encryption is set because of command line option, do the setup
+ * now. (If set by postmaster, postmaster should have performed the
+ * setup.)
+ *
+ * This should only be useful for the bootstrap process. Anyone else
+ * detects the encryption via ReadControlFile().
+ */
+ if (data_encrypted && MyAuxProcType == BootstrapProcess)
+ {
+ Assert(!IsUnderPostmaster);
+
+ /* Read the key from stdin. */
+ read_encryption_key(bootstrap_getc);
+
+ setup_encryption();
+ }
+
BaseInit();
/*
@@ -551,6 +587,16 @@ BootstrapModeMain(void)
*/
/*
+ * Read a single character from stdin. This is a callback for
+ * read_encryption_key().
+ */
+static int
+bootstrap_getc(void)
+{
+ return getc(stdin);
+}
+
+/*
* Set up signal handling for a bootstrap process
*/
static void
diff --git a/src/backend/storage/file/Makefile b/src/backend/storage/file/Makefile
index ca6a0e4f7d..9f277755fc 100644
--- a/src/backend/storage/file/Makefile
+++ b/src/backend/storage/file/Makefile
@@ -12,6 +12,6 @@ subdir = src/backend/storage/file
top_builddir = ../../../..
include $(top_builddir)/src/Makefile.global
-OBJS = fd.o buffile.o copydir.o reinit.o sharedfileset.o
+OBJS = fd.o buffile.o copydir.o reinit.o sharedfileset.o encryption.o
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/storage/file/encryption.c b/src/backend/storage/file/encryption.c
new file mode 100644
index 0000000000..629138fae7
--- /dev/null
+++ b/src/backend/storage/file/encryption.c
@@ -0,0 +1,313 @@
+/*-------------------------------------------------------------------------
+ *
+ * encryption.c
+ * This code handles encryption and decryption of data.
+ *
+ * Portions Copyright (c) 1996-2019, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * See src/backend/storage/file/README.encryption for explanation of the
+ * design.
+ *
+ * IDENTIFICATION
+ * src/backend/storage/file/encryption.c
+ *
+ * NOTES
+ * This file is compiled as both front-end and backend code, so the
+ * FRONTEND macro must be used to distinguish the case if we need to
+ * report error or if server-defined variable / function seems useful.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include <sys/stat.h>
+
+#include "access/xlog.h"
+#include "access/xlogdefs.h"
+#include "common/fe_memutils.h"
+#include "common/sha2.h"
+#include "common/string.h"
+#include "catalog/pg_control.h"
+#include "storage/bufpage.h"
+#include "storage/encryption.h"
+
+#ifndef FRONTEND
+#include "port.h"
+#include "storage/shmem.h"
+#include "storage/fd.h"
+#include "utils/memutils.h"
+#endif /* FRONTEND */
+
+#ifdef USE_ENCRYPTION
+#include <openssl/conf.h>
+#include <openssl/evp.h>
+#include <openssl/err.h>
+
+EVP_CIPHER_CTX *ctx_encrypt,
+ *ctx_decrypt,
+ *ctx_encrypt_stream,
+ *ctx_decrypt_stream;
+#endif /* USE_ENCRYPTION */
+
+unsigned char encryption_key[ENCRYPTION_KEY_LENGTH];
+
+bool data_encrypted = false;
+
+char encryption_verification[ENCRYPTION_SAMPLE_SIZE];
+
+bool encryption_setup_done = false;
+
+#ifdef USE_ENCRYPTION
+static void init_encryption_context(EVP_CIPHER_CTX **ctx_p, bool stream);
+static void evp_error(void);
+#endif /* USE_ENCRYPTION */
+
+#ifndef FRONTEND
+/*
+ * Read encryption key in hexadecimal form from stdin and store it in
+ * encryption_key variable.
+ */
+void
+read_encryption_key(read_encryption_key_cb read_char)
+{
+ char *buf;
+ int read_len, i, c;
+
+ buf = (char *) palloc(ENCRYPTION_KEY_CHARS);
+
+ read_len = 0;
+ while ((c = (*read_char)()) != EOF && c != '\n')
+ {
+ if (read_len >= ENCRYPTION_KEY_CHARS)
+ ereport(FATAL, (errmsg("Encryption key is too long")));
+
+ buf[read_len++] = c;
+ }
+
+ if (read_len < ENCRYPTION_KEY_CHARS)
+ ereport(FATAL, (errmsg("Encryption key is too short")));
+
+ /* Turn the hexadecimal representation into an array of bytes. */
+ for (i = 0; i < ENCRYPTION_KEY_LENGTH; i++)
+ {
+ if (sscanf(buf + 2 * i, "%2hhx", encryption_key + i) == 0)
+ {
+ ereport(FATAL,
+ (errmsg("Invalid character in encryption key at position %d",
+ 2 * i)));
+ }
+ }
+
+ pfree(buf);
+}
+#endif /* FRONTEND */
+
+/*
+ * Initialize encryption subsystem for use. Must be called before any
+ * encryptable data is read from or written to data directory.
+ */
+void
+setup_encryption(void)
+{
+#ifdef USE_ENCRYPTION
+ /*
+ * Setup OpenSSL.
+ *
+ * None of these functions should return a value or raise error.
+ */
+ ERR_load_crypto_strings();
+ OpenSSL_add_all_algorithms();
+
+ /*
+ * TODO Find out if this needs to be called for OpenSSL < 1.1.0.
+ */
+ /* OPENSSL_config(NULL); */
+
+ init_encryption_context(&ctx_encrypt, false);
+ init_encryption_context(&ctx_decrypt, false);
+
+ init_encryption_context(&ctx_encrypt_stream, true);
+ init_encryption_context(&ctx_decrypt_stream, true);
+
+ encryption_setup_done = true;
+#else /* !USE_ENCRYPTION */
+#ifndef FRONTEND
+ /*
+ * If no encryption implementation is linked and caller requests
+ * encryption, we should error out here and thus cause the calling process
+ * to fail (preferably postmaster, so the child processes don't make the
+ * same mistake).
+ */
+ ereport(FATAL, (errmsg(ENCRYPTION_NOT_SUPPORTED_MSG)));
+#else
+ /* Front-end shouldn't actually get here, but be careful. */
+ fprintf(stderr, "%s\n", ENCRYPTION_NOT_SUPPORTED_MSG);
+ exit(EXIT_FAILURE);
+#endif /* FRONTEND */
+#endif /* USE_ENCRYPTION */
+}
+
+/*
+ * Encrypts a fixed value into *buf to verify that encryption key is correct.
+ * Caller provided buf needs to be able to hold at least ENCRYPTION_SAMPLE_SIZE
+ * bytes.
+ */
+void
+sample_encryption(char *buf)
+{
+ char tweak[TWEAK_SIZE];
+ int i;
+
+ for (i = 0; i < TWEAK_SIZE; i++)
+ tweak[i] = i;
+
+ encrypt_block("postgresqlcrypt", buf, ENCRYPTION_SAMPLE_SIZE, tweak,
+ false);
+}
+
+/*
+ * Encrypts one block of data with a specified tweak value. May only be called
+ * when encryption_enabled is true.
+ *
+ * Input and output buffer may point to the same location.
+ *
+ * "size" must be a (non-zero) multiple of ENCRYPTION_BLOCK.
+ *
+ * "tweak" value must be TWEAK_SIZE bytes long.
+ *
+ * If "stream" is set, stream cipher is used instead of block one.
+ *
+ * All-zero blocks are not encrypted to correctly handle relation extension,
+ * and also to simplify handling of holes created by seek past EOF and
+ * consequent write (see buffile.c).
+ */
+void
+encrypt_block(const char *input, char *output, Size size, char *tweak,
+ bool stream)
+{
+#ifdef USE_ENCRYPTION
+ int out_size;
+ EVP_CIPHER_CTX *ctx;
+
+ Assert(data_encrypted);
+
+ /*
+ * Block cipher should only be used if the size is whole multiple of
+ * encryption block size.
+ */
+ Assert((size >= ENCRYPTION_BLOCK && size % ENCRYPTION_BLOCK == 0) ||
+ stream);
+
+ /*
+ * Empty page is not worth encryption. Do not waste cycles checking for
+ * stream cipher as this is currently used only for XLOG pages, and empty
+ * XLOG page should not be written to disk.
+ */
+ if (!stream && IsAllZero(input, size))
+ {
+ memset(output, 0, size);
+ return;
+ }
+
+ ctx = !stream ? ctx_encrypt : ctx_encrypt_stream;
+
+ /* The remaining initialization. */
+ if (EVP_EncryptInit_ex(ctx, NULL, NULL, encryption_key,
+ (unsigned char *) tweak) != 1)
+ evp_error();
+
+ /* Do the actual encryption. */
+ if (EVP_EncryptUpdate(ctx, (unsigned char *) output,
+ &out_size, (unsigned char *) input, size) != 1)
+ evp_error();
+
+ Assert(out_size == size);
+#else
+ /* data_encrypted should not be set */
+ Assert(false);
+#endif /* USE_ENCRYPTION */
+}
+
+
+#ifdef USE_ENCRYPTION
+/*
+ * Initialize the OpenSSL context for passed cipher.
+ *
+ * On server side this happens during postmaster startup, so other processes
+ * inherit the initialized context via fork(). There's no reason to this again
+ * and again in encrypt_block() / decrypt_block(), also because we cannot
+ * handle out-of-memory conditions encountered by OpenSSL in another way than
+ * ereport(FATAL). The OOM is much less likely to happen during postmaster
+ * startup, and even if it happens, troubleshooting should be easier than if
+ * it happened during normal operation.
+ *
+ * XXX Do we need to call EVP_CIPHER_CTX_cleanup() (via on_proc_exit callback
+ * for server processes and other way for front-ends)? Not sure it's
+ * necessary, as the initialization does not involve any shared resources
+ * (e.g. files).
+ */
+static void
+init_encryption_context(EVP_CIPHER_CTX **ctx_p, bool stream)
+{
+ EVP_CIPHER_CTX *ctx;
+ const EVP_CIPHER *cipher;
+#ifdef USE_ASSERT_CHECKING
+ int block_size;
+#endif /* USE_ASSERT_CHECKING */
+
+ cipher = !stream ? EVP_aes_256_cbc() : EVP_aes_256_ctr();
+
+ if ((*ctx_p = EVP_CIPHER_CTX_new()) == NULL)
+ evp_error();
+ ctx = *ctx_p;
+ if (EVP_EncryptInit_ex(ctx, cipher, NULL, NULL, NULL) != 1)
+ evp_error();
+
+ /*
+ * No padding is needed. For a block cipher, the input block size should
+ * already be a multiple of ENCRYPTION_BLOCK. For stream cipher, we don't
+ * need padding anyway. This might save some cycles at the OpenSSL end.
+ * XXX Is it setting worth when we don't call EVP_DecryptFinal_ex()
+ * anyway?
+ */
+ EVP_CIPHER_CTX_set_padding(ctx, 0);
+
+ Assert(EVP_CIPHER_CTX_iv_length(ctx) == TWEAK_SIZE);
+ Assert(EVP_CIPHER_CTX_key_length(ctx) == ENCRYPTION_KEY_LENGTH);
+ block_size = EVP_CIPHER_CTX_block_size(ctx);
+#ifdef USE_ASSERT_CHECKING
+ if (!stream)
+ Assert(block_size == ENCRYPTION_BLOCK);
+ else
+ Assert(block_size == 1);
+#endif /* USE_ASSERT_CHECKING */
+}
+
+#endif /* USE_ENCRYPTION */
+
+#ifdef USE_ENCRYPTION
+/*
+ * Error callback for openssl.
+ */
+static void
+evp_error(void)
+{
+ ERR_print_errors_fp(stderr);
+#ifndef FRONTEND
+
+ /*
+ * FATAL is the appropriate level because backend can hardly fix anything
+ * if encryption / decryption has failed.
+ *
+ * XXX Do we yet need EVP_CIPHER_CTX_cleanup() here?
+ */
+ elog(FATAL, "OpenSSL encountered error during encryption or decryption.");
+#else
+ fprintf(stderr,
+ "OpenSSL encountered error during encryption or decryption.");
+ exit(EXIT_FAILURE);
+#endif /* FRONTEND */
+}
+#endif /* USE_ENCRYPTION */
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 44a59e1d4f..38dad9311f 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -63,6 +63,7 @@
#include "replication/walsender.h"
#include "rewrite/rewriteHandler.h"
#include "storage/bufmgr.h"
+#include "storage/encryption.h"
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/procsignal.h"
@@ -3868,6 +3869,35 @@ PostgresMain(int argc, char *argv[],
PG_SETMASK(&UnBlockSig);
/*
+ * Standalone backend operating on an encrypted cluster needs encryption
+ * key.
+ */
+ if (!IsUnderPostmaster && data_encrypted)
+ {
+ char sample[ENCRYPTION_SAMPLE_SIZE];
+
+ /* Display a prompt for user to enter the encryption key. */
+ printf("key> ");
+ fflush(stdout);
+
+ /*
+ * Read the key from stdin. Pass interactive_getc() as the callback so
+ * that the reading is interruptible.
+ */
+ read_encryption_key(interactive_getc);
+
+ setup_encryption();
+
+ /* Verify the key. */
+ sample_encryption(sample);
+ if (memcmp(encryption_verification, sample, ENCRYPTION_SAMPLE_SIZE))
+ ereport(FATAL,
+ (errmsg("invalid encryption key"),
+ errdetail("The passed encryption key does not match"
+ " database encryption key.")));
+ }
+
+ /*
* General initialization.
*
* NOTE: if you are tempted to add code in this vicinity, consider putting
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 631f16f5fe..2b7fc2fb0f 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -71,6 +71,7 @@
#include "replication/walreceiver.h"
#include "replication/walsender.h"
#include "storage/bufmgr.h"
+#include "storage/encryption.h"
#include "storage/dsm_impl.h"
#include "storage/standby.h"
#include "storage/fd.h"
@@ -1836,6 +1837,17 @@ static struct config_bool ConfigureNamesBool[] =
},
{
+ {"data_encryption", PGC_INTERNAL, PRESET_OPTIONS,
+ gettext_noop("Shows whether data encryption is turned on for this cluster."),
+ NULL,
+ GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE
+ },
+ &data_encrypted,
+ false,
+ NULL, NULL, NULL
+ },
+
+ {
{"syslog_sequence_numbers", PGC_SIGHUP, LOGGING_WHERE,
gettext_noop("Add sequence number to syslog messages to avoid duplicate suppression."),
NULL
diff --git a/src/bin/initdb/.gitignore b/src/bin/initdb/.gitignore
index 71a899ffb8..0f33b5c67d 100644
--- a/src/bin/initdb/.gitignore
+++ b/src/bin/initdb/.gitignore
@@ -1,5 +1,6 @@
/encnames.c
/localtime.c
+/encryption.c
/initdb
diff --git a/src/bin/initdb/Makefile b/src/bin/initdb/Makefile
index 7c404430a9..b72de55d62 100644
--- a/src/bin/initdb/Makefile
+++ b/src/bin/initdb/Makefile
@@ -26,7 +26,7 @@ ifneq (,$(with_system_tzdata))
override CPPFLAGS += '-DSYSTEMTZDIR="$(with_system_tzdata)"'
endif
-OBJS= initdb.o findtimezone.o localtime.o encnames.o $(WIN32RES)
+OBJS= initdb.o findtimezone.o localtime.o encnames.o encryption.o $(WIN32RES)
all: initdb
@@ -45,6 +45,9 @@ encnames.c: % : $(top_srcdir)/src/backend/utils/mb/%
localtime.c: % : $(top_srcdir)/src/timezone/%
rm -f $@ && $(LN_S) $< .
+encryption.c: % : $(top_srcdir)/src/backend/storage/file/%
+ rm -f $@ && $(LN_S) $< .
+
install: all installdirs
$(INSTALL_PROGRAM) initdb$(X) '$(DESTDIR)$(bindir)/initdb$(X)'
@@ -55,7 +58,7 @@ uninstall:
rm -f '$(DESTDIR)$(bindir)/initdb$(X)'
clean distclean maintainer-clean:
- rm -f initdb$(X) $(OBJS) encnames.c localtime.c
+ rm -f initdb$(X) $(OBJS) encnames.c localtime.c encryption.c
rm -rf tmp_check
# ensure that changes in datadir propagate into object file
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index f1acbdfcf2..f2a582e975 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -68,11 +68,13 @@
#include "common/logging.h"
#include "common/restricted_token.h"
#include "common/username.h"
+#include "fe_utils/encryption.h"
#include "fe_utils/string_utils.h"
#include "getaddrinfo.h"
#include "getopt_long.h"
#include "mb/pg_wchar.h"
#include "miscadmin.h"
+#include "storage/encryption.h"
/* Ideally this would be in a .h file, but it hardly seems worth the trouble */
@@ -171,6 +173,8 @@ static bool output_failed = false;
static int output_errno = 0;
static char *pgdata_native;
+unsigned char encryption_key[ENCRYPTION_KEY_LENGTH];
+
/* defaults */
static int n_connections = 10;
static int n_buffers = 50;
@@ -239,6 +243,7 @@ static char **filter_lines_with_token(char **lines, const char *token);
static char **readfile(const char *path);
static void writefile(char *path, char **lines);
static FILE *popen_check(const char *command, const char *mode);
+static void send_encryption_key(FILE *f);
static char *get_id(void);
static int get_encoding_id(const char *encoding_name);
static void set_input(char **dest, const char *filename);
@@ -584,6 +589,22 @@ popen_check(const char *command, const char *mode)
}
/*
+ * Send encryption key in hexadecimal format to the file stream passed.
+ *
+ * The backend processes could actually receive binary data but that would
+ * make startup of postgres in single-user mode less convenient.
+ */
+static void
+send_encryption_key(FILE *f)
+{
+ int i;
+
+ for (i = 0; i < ENCRYPTION_KEY_LENGTH; i++)
+ fprintf(f, "%.2x", encryption_key[i]);
+ fputc('\n', f);
+}
+
+/*
* clean up any files we created on failure
* if we created the data directory remove it too
*/
@@ -1379,6 +1400,7 @@ bootstrap_template1(void)
char **bki_lines;
char headerline[MAXPGPATH];
char buf[64];
+ static char *encr_opt_str = NULL;
printf(_("running bootstrap script ... "));
fflush(stdout);
@@ -1446,17 +1468,37 @@ bootstrap_template1(void)
/* Also ensure backend isn't confused by this environment var: */
unsetenv("PGCLIENTENCODING");
+ /* Prepare the -K option for the backend. */
+ if (encryption_key_command)
+ {
+ size_t len;
+
+ len = 3;
+ encr_opt_str = (char *) pg_malloc(len);
+ snprintf(encr_opt_str, len, "-K");
+ }
+ else
+ {
+ encr_opt_str = (char *) pg_malloc(1);
+ encr_opt_str[0] = '\0';
+ }
+
snprintf(cmd, sizeof(cmd),
- "\"%s\" --boot -x1 -X %u %s %s %s",
+ "\"%s\" --boot -x1 -X %u %s %s %s %s",
backend_exec,
wal_segment_size_mb * (1024 * 1024),
data_checksums ? "-k" : "",
+ encr_opt_str,
boot_options,
debug ? "-d 5" : "");
PG_CMD_OPEN;
+ /* If the cluster is encrypted, first send the encryption key. */
+ if (encryption_key_command)
+ send_encryption_key(cmdfd);
+
for (line = bki_lines; *line != NULL; line++)
{
PG_CMD_PUTS(*line);
@@ -2378,6 +2420,10 @@ usage(const char *progname)
printf(_("\nLess commonly used options:\n"));
printf(_(" -d, --debug generate lots of debugging output\n"));
printf(_(" -k, --data-checksums use data page checksums\n"));
+#ifdef USE_ENCRYPTION
+ printf(_(" -K, --encryption-key-command\n"
+ " command that returns encryption key\n"));
+#endif /* USE_ENCRYPTION */
printf(_(" -L DIRECTORY where to find the input files\n"));
printf(_(" -n, --no-clean do not clean up after errors\n"));
printf(_(" -N, --no-sync do not wait for changes to be written safely to disk\n"));
@@ -2957,6 +3003,13 @@ initialize_data_directory(void)
/* Top level PG_VERSION is checked by bootstrapper, so make it first */
write_version_file(NULL);
+ /*
+ * If the cluster will be encrypted, run the command to generate the
+ * encryption key.
+ */
+ if (encryption_key_command)
+ run_encryption_key_command(encryption_key);
+
/* Select suitable configuration settings */
set_null_conf();
test_config_settings();
@@ -2986,6 +3039,10 @@ initialize_data_directory(void)
PG_CMD_OPEN;
+ /* If the cluster is encrypted, first send the encryption key. */
+ if (encryption_key_command)
+ send_encryption_key(cmdfd);
+
setup_auth(cmdfd);
setup_depend(cmdfd);
@@ -3054,6 +3111,9 @@ main(int argc, char *argv[])
{"waldir", required_argument, NULL, 'X'},
{"wal-segsize", required_argument, NULL, 12},
{"data-checksums", no_argument, NULL, 'k'},
+#ifdef USE_ENCRYPTION
+ {"encryption-key-command", required_argument, NULL, 'K'},
+#endif /* USE_ENCRYPTION */
{"allow-group-access", no_argument, NULL, 'g'},
{NULL, 0, NULL, 0}
};
@@ -3096,7 +3156,7 @@ main(int argc, char *argv[])
/* process command-line options */
- while ((c = getopt_long(argc, argv, "dD:E:kL:nNU:WA:sST:X:g", long_options, &option_index)) != -1)
+ while ((c = getopt_long(argc, argv, "dD:E:kK:L:nNU:WA:sST:X:g", long_options, &option_index)) != -1)
{
switch (c)
{
@@ -3148,6 +3208,11 @@ main(int argc, char *argv[])
case 'k':
data_checksums = true;
break;
+#ifdef USE_ENCRYPTION
+ case 'K':
+ encryption_key_command = pg_strdup(optarg);
+ break;
+#endif /* USE_ENCRYPTION */
case 'L':
share_path = pg_strdup(optarg);
break;
@@ -3318,6 +3383,11 @@ main(int argc, char *argv[])
if (pwprompt || pwfilename)
get_su_pwd();
+ if (encryption_key_command)
+ printf(_("Data encryption is enabled.\n"));
+ else
+ printf(_("Data encryption is disabled.\n"));
+
printf("\n");
initialize_data_directory();
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index 390ea0a939..3eef925f97 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -336,5 +336,14 @@ main(int argc, char *argv[])
ControlFile->data_checksum_version);
printf(_("Mock authentication nonce: %s\n"),
mock_auth_nonce_str);
+ printf(_("Data encryption: %s\n"),
+ ControlFile->data_cipher > PG_CIPHER_NONE ? _("on") : _("off"));
+ if (ControlFile->data_cipher > PG_CIPHER_NONE)
+ printf(_("Data encryption fingerprint: %08X%08X%08X%08X\n"),
+ htonl(((uint32 *) ControlFile->encryption_verification)[0]),
+ htonl(((uint32 *) ControlFile->encryption_verification)[1]),
+ htonl(((uint32 *) ControlFile->encryption_verification)[2]),
+ htonl(((uint32 *) ControlFile->encryption_verification)[3])
+ );
return 0;
}
diff --git a/src/common/string.c b/src/common/string.c
index b01a56ceaa..5ba890d37d 100644
--- a/src/common/string.c
+++ b/src/common/string.c
@@ -42,6 +42,45 @@ pg_str_endswith(const char *str, const char *end)
return strcmp(str, end) == 0;
}
+/*
+ * Helper function to check if a page is completely empty.
+ *
+ * TODO Invent name that is more consistent with that of the other function(s)
+ * in this module.
+ */
+bool
+IsAllZero(const char *input, Size size)
+{
+ const char *pos = input;
+ const char *aligned_start = (char *) MAXALIGN64(input);
+ const char *end = input + size;
+
+ /* Check 1 byte at a time until pos is 8 byte aligned */
+ while (pos < aligned_start)
+ if (*pos++ != 0)
+ return false;
+
+ /*
+ * Run 8 parallel 8 byte checks in one iteration. On 2016 hardware
+ * slightly faster than 4 parallel checks.
+ */
+ while (pos + 8 * sizeof(uint64) <= end)
+ {
+ uint64 *p = (uint64 *) pos;
+
+ if ((p[0] | p[1] | p[2] | p[3] | p[4] | p[5] | p[6] | p[7]) != 0)
+ return false;
+ pos += 8 * sizeof(uint64);
+ }
+
+ /* Handle unaligned tail. */
+ while (pos < end)
+ if (*pos++ != 0)
+ return false;
+
+ return true;
+}
+
/*
* strtoint --- just like strtol, but returns int not long
diff --git a/src/fe_utils/Makefile b/src/fe_utils/Makefile
index 7d73800323..a508ec150e 100644
--- a/src/fe_utils/Makefile
+++ b/src/fe_utils/Makefile
@@ -19,7 +19,8 @@ include $(top_builddir)/src/Makefile.global
override CPPFLAGS := -DFRONTEND -I$(libpq_srcdir) $(CPPFLAGS)
-OBJS = mbprint.o print.o psqlscan.o simple_list.o string_utils.o conditional.o
+OBJS = mbprint.o print.o psqlscan.o simple_list.o string_utils.o conditional.o \
+ encryption.o
all: libpgfeutils.a
diff --git a/src/fe_utils/encryption.c b/src/fe_utils/encryption.c
new file mode 100644
index 0000000000..dab1435014
--- /dev/null
+++ b/src/fe_utils/encryption.c
@@ -0,0 +1,90 @@
+/*-------------------------------------------------------------------------
+ *
+ * encryption.c
+ * Front-end code to handle keys for full cluster encryption. The actual
+ * encryption is not performed here.
+ *
+ * Portions Copyright (c) 1996-2019, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/fe_utils/encryption.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include <unistd.h>
+
+#include "postgres_fe.h"
+
+#include "common/fe_memutils.h"
+#include "common/file_perm.h"
+#include "common/logging.h"
+#include "fe_utils/encryption.h"
+#include "storage/encryption.h"
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "libpq/pqcomm.h"
+
+char *encryption_key_command = NULL;
+
+/*
+ * Run the command that is supposed to generate encryption key and store it
+ * where encryption_key points to.
+ */
+void
+run_encryption_key_command(unsigned char *encryption_key)
+{
+ FILE *fp;
+ char *buf;
+ int read_len, i, c;
+
+ Assert(encryption_key_command != NULL &&
+ strlen(encryption_key_command) > 0);
+
+ fp = popen(encryption_key_command, "r");
+ if (fp == NULL)
+ {
+ pg_log_fatal("Failed to execute \"%s\"", encryption_key_command);
+ exit(EXIT_FAILURE);
+ }
+
+ buf = (char *) palloc(ENCRYPTION_KEY_CHARS);
+
+ /*
+ * Read the key. This is very similar to backend's read_encryption_key()
+ * but there seems to be no straightforward way to call the function from
+ * here.
+ */
+ read_len = 0;
+ while ((c = fgetc(fp)) != EOF && c != '\n')
+ {
+ if (read_len >= ENCRYPTION_KEY_CHARS)
+ {
+ pg_log_fatal("Encryption key is too long");
+ exit(EXIT_FAILURE);
+ }
+
+ buf[read_len++] = c;
+ }
+
+ if (read_len < ENCRYPTION_KEY_CHARS)
+ {
+ pg_log_fatal("Encryption key is too short");
+ exit(EXIT_FAILURE);
+ }
+
+ /* Turn the hexadecimal representation into an array of bytes. */
+ for (i = 0; i < ENCRYPTION_KEY_LENGTH; i++)
+ {
+ if (sscanf(buf + 2 * i, "%2hhx", encryption_key + i) == 0)
+ {
+ pg_log_fatal("Invalid character in encryption key at position %d",
+ 2 * i);
+ exit(EXIT_FAILURE);
+ }
+ }
+
+ pfree(buf);
+ pclose(fp);
+}
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index ff98d9e91a..c1b53e1ebf 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -19,6 +19,7 @@
#include "access/xlogdefs.h"
#include "pgtime.h" /* for pg_time_t */
#include "port/pg_crc32c.h"
+#include "storage/encryption.h"
/* Version identifier for this pg_control format */
@@ -228,6 +229,16 @@ typedef struct ControlFileData
*/
char mock_authentication_nonce[MOCK_AUTH_NONCE_LEN];
+ /*
+ * Cipher used to encrypt data. Zero if unencrypted.
+ *
+ * The data type is actually CipherKind, but we don't want to include
+ * encryption.h just because of this field.
+ */
+ uint8 data_cipher;
+ /* Sample value for encryption key verification */
+ uint8 encryption_verification[ENCRYPTION_SAMPLE_SIZE];
+
/* CRC of all above ... MUST BE LAST! */
pg_crc32c crc;
} ControlFileData;
diff --git a/src/include/common/string.h b/src/include/common/string.h
index 77f31337ca..6128d81b8c 100644
--- a/src/include/common/string.h
+++ b/src/include/common/string.h
@@ -14,5 +14,6 @@ extern bool pg_str_endswith(const char *str, const char *end);
extern int strtoint(const char *pg_restrict str, char **pg_restrict endptr,
int base);
extern void pg_clean_ascii(char *str);
+extern bool IsAllZero(const char *input, Size size);
#endif /* COMMON_STRING_H */
diff --git a/src/include/fe_utils/encryption.h b/src/include/fe_utils/encryption.h
new file mode 100644
index 0000000000..69e3b2f756
--- /dev/null
+++ b/src/include/fe_utils/encryption.h
@@ -0,0 +1,18 @@
+/*-------------------------------------------------------------------------
+ *
+ * encryption.h
+ * Client code to support full cluster encryption.
+ *
+ * Portions Copyright (c) 1996-2019, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/include/fe_utils/encryption.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+/* Executable to retrieve the encryption key. */
+extern char *encryption_key_command;
+
+extern void run_encryption_key_command(unsigned char *encryption_key);
diff --git a/src/include/pg_config_manual.h b/src/include/pg_config_manual.h
index 743401cb96..b716fb4d13 100644
--- a/src/include/pg_config_manual.h
+++ b/src/include/pg_config_manual.h
@@ -172,6 +172,13 @@
#endif
/*
+ * OpenSSL is currently the only implementation of encryption we use.
+ */
+#ifdef USE_OPENSSL
+#define USE_ENCRYPTION
+#endif
+
+/*
* This is the default directory in which AF_UNIX socket files are
* placed. Caution: changing this risks breaking your existing client
* applications, which are likely to continue to look in the old
diff --git a/src/include/storage/encryption.h b/src/include/storage/encryption.h
new file mode 100644
index 0000000000..8479b76a34
--- /dev/null
+++ b/src/include/storage/encryption.h
@@ -0,0 +1,99 @@
+/*-------------------------------------------------------------------------
+ *
+ * encryption.h
+ * Full database encryption support
+ *
+ *
+ * Portions Copyright (c) 1996-2019, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/storage/encryption.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef ENCRYPTION_H
+#define ENCRYPTION_H
+
+#include "access/xlogdefs.h"
+#include "miscadmin.h"
+#include "storage/block.h"
+#include "storage/relfilenode.h"
+#include "port/pg_crc32c.h"
+
+/*
+ * Common error message issued when particular code path cannot be executed
+ * due to absence of the OpenSSL library.
+ */
+#define ENCRYPTION_NOT_SUPPORTED_MSG \
+ "compile postgres with --with-openssl to use encryption."
+
+/*
+ * Full database encryption key.
+ *
+ * The key of EVP_aes_256_cbc() cipher is 256 bits long.
+ */
+#define ENCRYPTION_KEY_LENGTH 32
+/* Key length in characters (two characters per hexadecimal digit) */
+#define ENCRYPTION_KEY_CHARS (ENCRYPTION_KEY_LENGTH * 2)
+
+/*
+ * Cipher used to encrypt data.
+ *
+ * Due to very specific requirements, the ciphers are not likely to change,
+ * but we should be somewhat flexible.
+ *
+ * XXX If we have more than one cipher someday, have pg_controldata report the
+ * cipher kind (in textual form) instead of merely saying "on".
+ */
+typedef enum CipherKind
+{
+ /* The cluster is not encrypted. */
+ PG_CIPHER_NONE = 0,
+
+ /*
+ * AES (Rijndael) in CBC mode of operation as block cipher, and in CTR
+ * mode as stream cipher. Key length is always 256 bits.
+ */
+ PG_CIPHER_AES_BLOCK_CBC_256_STREAM_CTR_256
+} CipherKind;
+
+/* Key to encrypt / decrypt data. */
+extern unsigned char encryption_key[];
+
+/*
+ * The encrypted data is a series of blocks of size
+ * ENCRYPTION_BLOCK. Currently we use the EVP_aes_256_xts implementation. Make
+ * sure the following constants match if adopting another algorithm.
+ */
+#define ENCRYPTION_BLOCK 16
+
+#define TWEAK_SIZE 16
+
+/* Is the cluster encrypted? */
+extern PGDLLIMPORT bool data_encrypted;
+
+/*
+ * Number of bytes reserved to store encryption sample in ControlFileData.
+ */
+#define ENCRYPTION_SAMPLE_SIZE 16
+
+#ifndef FRONTEND
+/* Copy of the same field of ControlFileData. */
+extern char encryption_verification[];
+#endif /* FRONTEND */
+
+/* Do we have encryption_key and the encryption library initialized? */
+extern bool encryption_setup_done;
+
+#ifndef FRONTEND
+typedef int (*read_encryption_key_cb) (void);
+extern void read_encryption_key(read_encryption_key_cb read_char);
+#endif /* FRONTEND */
+
+extern void setup_encryption(void);
+extern void sample_encryption(char *buf);
+extern void encrypt_block(const char *input, char *output, Size size,
+ char *tweak, bool stream);
+extern void encryption_error(bool fatal, char *message);
+
+#endif /* ENCRYPTION_H */
--
2.13.7
v04-0002-Prepare-ServerLoop-for-the-next-patch.patch (text/x-diff, 5.8K) download
From 9246acbc3fd7ed00de831687c8e504f4d5c46f9e Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Fri, 5 Jul 2019 16:24:01 +0200
Subject: [PATCH 02/17] Prepare ServerLoop for the next patch.
This is only refactoring patch that tries to make the following one easier to
read. It simply moves some code from ServerLoop() to
ServerLoopCheckTimeouts().
---
src/backend/postmaster/postmaster.c | 110 ++++++++++++++++++++----------------
1 file changed, 60 insertions(+), 50 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 688ad439ed..e8e3f8a4d3 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -403,6 +403,7 @@ static void BackendInitialize(Port *port);
static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
+static void ServerLoopCheckTimeouts(void);
static int BackendStartup(Port *port);
static int ProcessStartupPacket(Port *port, bool secure_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
@@ -1611,6 +1612,8 @@ DetermineSleepTime(struct timeval *timeout)
}
}
+static time_t last_lockfile_recheck_time, last_touch_time;
+
/*
* Main idle loop of postmaster
*
@@ -1621,8 +1624,6 @@ ServerLoop(void)
{
fd_set readmask;
int nSockets;
- time_t last_lockfile_recheck_time,
- last_touch_time;
last_lockfile_recheck_time = last_touch_time = time(NULL);
@@ -1632,7 +1633,6 @@ ServerLoop(void)
{
fd_set rmask;
int selres;
- time_t now;
/*
* Wait for a connection request to arrive.
@@ -1797,58 +1797,68 @@ ServerLoop(void)
* us sleep at most that long; except for SIGKILL timeout which has
* special-case logic there.
*/
- now = time(NULL);
+ ServerLoopCheckTimeouts();
+ }
+}
- /*
- * If we already sent SIGQUIT to children and they are slow to shut
- * down, it's time to send them SIGKILL. This doesn't happen
- * normally, but under certain conditions backends can get stuck while
- * shutting down. This is a last measure to get them unwedged.
- *
- * Note we also do this during recovery from a process crash.
- */
- if ((Shutdown >= ImmediateShutdown || (FatalError && !SendStop)) &&
- AbortStartTime != 0 &&
- (now - AbortStartTime) >= SIGKILL_CHILDREN_AFTER_SECS)
- {
- /* We were gentle with them before. Not anymore */
- TerminateChildren(SIGKILL);
- /* reset flag so we don't SIGKILL again */
- AbortStartTime = 0;
- }
+/*
+ * Subroutine of ServerLoop() that checks if various timeouts elapsed, and if
+ * so, takes the appropriate action.
+ */
+static void
+ServerLoopCheckTimeouts(void)
+{
+ time_t now = time(NULL);
- /*
- * Once a minute, verify that postmaster.pid hasn't been removed or
- * overwritten. If it has, we force a shutdown. This avoids having
- * postmasters and child processes hanging around after their database
- * is gone, and maybe causing problems if a new database cluster is
- * created in the same place. It also provides some protection
- * against a DBA foolishly removing postmaster.pid and manually
- * starting a new postmaster. Data corruption is likely to ensue from
- * that anyway, but we can minimize the damage by aborting ASAP.
- */
- if (now - last_lockfile_recheck_time >= 1 * SECS_PER_MINUTE)
- {
- if (!RecheckDataDirLockFile())
- {
- ereport(LOG,
- (errmsg("performing immediate shutdown because data directory lock file is invalid")));
- kill(MyProcPid, SIGQUIT);
- }
- last_lockfile_recheck_time = now;
- }
+ /*
+ * If we already sent SIGQUIT to children and they are slow to shut down,
+ * it's time to send them SIGKILL. This doesn't happen normally, but
+ * under certain conditions backends can get stuck while shutting down.
+ * This is a last measure to get them unwedged.
+ *
+ * Note we also do this during recovery from a process crash.
+ */
+ if ((Shutdown >= ImmediateShutdown || (FatalError && !SendStop)) &&
+ AbortStartTime != 0 &&
+ (now - AbortStartTime) >= SIGKILL_CHILDREN_AFTER_SECS)
+ {
+ /* We were gentle with them before. Not anymore */
+ TerminateChildren(SIGKILL);
+ /* reset flag so we don't SIGKILL again */
+ AbortStartTime = 0;
+ }
- /*
- * Touch Unix socket and lock files every 58 minutes, to ensure that
- * they are not removed by overzealous /tmp-cleaning tasks. We assume
- * no one runs cleaners with cutoff times of less than an hour ...
- */
- if (now - last_touch_time >= 58 * SECS_PER_MINUTE)
+ /*
+ * Once a minute, verify that postmaster.pid hasn't been removed or
+ * overwritten. If it has, we force a shutdown. This avoids having
+ * postmasters and child processes hanging around after their database is
+ * gone, and maybe causing problems if a new database cluster is created
+ * in the same place. It also provides some protection against a DBA
+ * foolishly removing postmaster.pid and manually starting a new
+ * postmaster. Data corruption is likely to ensue from that anyway, but
+ * we can minimize the damage by aborting ASAP.
+ */
+ if (now - last_lockfile_recheck_time >= 1 * SECS_PER_MINUTE)
+ {
+ if (!RecheckDataDirLockFile())
{
- TouchSocketFiles();
- TouchSocketLockFiles();
- last_touch_time = now;
+ ereport(LOG,
+ (errmsg("performing immediate shutdown because data directory lock file is invalid")));
+ kill(MyProcPid, SIGQUIT);
}
+ last_lockfile_recheck_time = now;
+ }
+
+ /*
+ * Touch Unix socket and lock files every 58 minutes, to ensure that they
+ * are not removed by overzealous /tmp-cleaning tasks. We assume no one
+ * runs cleaners with cutoff times of less than an hour ...
+ */
+ if (now - last_touch_time >= 58 * SECS_PER_MINUTE)
+ {
+ TouchSocketFiles();
+ TouchSocketLockFiles();
+ last_touch_time = now;
}
}
--
2.13.7
v04-0003-Teach-postmaster-to-accept-encryption-key.patch (text/x-diff, 26.2K) download
From af9da6990b09276e65054d0d341f758f928e0fc6 Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Fri, 5 Jul 2019 16:24:01 +0200
Subject: [PATCH 03/17] Teach postmaster to accept encryption key
This patch teaches postmaster to accept encryption key (as a special kind of
startup packet) before recovery starts. In fact the key is received by a
backend that postmaster launches if a client tries to initiate connection
while the postmaster is in PM_INIT state. At this stage, only the key is
received, whereas regular connection attempt ends up with
FATAL: the database system is starting up
If postmaster receives no key for some time, it raises this error and exits:
FATAL: Encryption key not received.
With this patch, user can start postmaster as usual:
postgres -D data
and use pg_keytool utility in another session to send the key:
echo 71916b8b93063c280e4e0ee646395f450bda4f1bac85c57bc3a6afd168f3ab47 | pg_keytool -s
---
src/backend/postmaster/postmaster.c | 211 +++++++++++++++++++++++++++++++++-
src/backend/storage/file/encryption.c | 34 ++++++
src/backend/storage/ipc/ipci.c | 2 +
src/bin/Makefile | 1 +
src/bin/pg_keytool/.gitignore | 1 +
src/bin/pg_keytool/Makefile | 31 +++++
src/bin/pg_keytool/pg_keytool.c | 199 ++++++++++++++++++++++++++++++++
src/fe_utils/encryption.c | 101 ++++++++++++++++
src/include/fe_utils/encryption.h | 2 +
src/include/libpq/pqcomm.h | 24 ++++
src/include/storage/encryption.h | 26 +++++
11 files changed, 628 insertions(+), 4 deletions(-)
create mode 100644 src/bin/pg_keytool/.gitignore
create mode 100644 src/bin/pg_keytool/Makefile
create mode 100644 src/bin/pg_keytool/pg_keytool.c
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index e8e3f8a4d3..47f7bc93f7 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -117,6 +117,7 @@
#include "postmaster/syslogger.h"
#include "replication/logicallauncher.h"
#include "replication/walsender.h"
+#include "storage/encryption.h"
#include "storage/fd.h"
#include "storage/ipc.h"
#include "storage/pg_shmem.h"
@@ -402,12 +403,13 @@ static void PostmasterStateMachine(void);
static void BackendInitialize(Port *port);
static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
-static int ServerLoop(void);
+static int ServerLoop(bool receive_encryption_key);
static void ServerLoopCheckTimeouts(void);
static int BackendStartup(Port *port);
static int ProcessStartupPacket(Port *port, bool secure_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
+static void processEncryptionKey(void *pkt);
static int initMasks(fd_set *rmask);
static void report_fork_failure_to_client(Port *port, int errnum);
static CAC_state canAcceptConnections(void);
@@ -1365,6 +1367,41 @@ PostmasterMain(int argc, char *argv[])
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STARTING);
/*
+ * If encryption key is needed and we don't have it yet, call ServerLoop()
+ * in a restricted mode which allows a client application to send us the
+ * key. Regular client connections are not allowed yet.
+ */
+ if (data_encrypted && !encryption_setup_done)
+ {
+ char sample[ENCRYPTION_SAMPLE_SIZE];
+
+ status = ServerLoop(true);
+
+ /* No other return code should be seen here. */
+ Assert(status == STATUS_OK);
+
+ /* ServerLoop() shouldn't have exited otherwise. */
+ Assert(encryption_key_shmem->initialized);
+
+ /*
+ * Take a local copy of the key so that we don't have to receive it
+ * again if restarting after a crash.
+ */
+ memcpy(encryption_key, encryption_key_shmem, ENCRYPTION_KEY_LENGTH);
+
+ /* Finalize the setup. */
+ setup_encryption();
+
+ /* Verify the key. */
+ sample_encryption(sample);
+ if (memcmp(encryption_verification, sample, ENCRYPTION_SAMPLE_SIZE))
+ ereport(FATAL,
+ (errmsg("invalid encryption key"),
+ errdetail("The passed encryption key does not match"
+ " database encryption key.")));
+ }
+
+ /*
* We're ready to rock and roll...
*/
StartupPID = StartupDataBase();
@@ -1375,7 +1412,7 @@ PostmasterMain(int argc, char *argv[])
/* Some workers may be scheduled to start now */
maybe_start_bgworkers();
- status = ServerLoop();
+ status = ServerLoop(false);
/*
* ServerLoop probably shouldn't ever return, but if it does, close down.
@@ -1615,18 +1652,37 @@ DetermineSleepTime(struct timeval *timeout)
static time_t last_lockfile_recheck_time, last_touch_time;
/*
+ * The maximum amount of time postmaster can wait for encryption key if the
+ * cluster is encrypted. It should be shorter than the time pg_ctl waits for
+ * postgres to start (60 seconds) and should not be too long so that failure
+ * to specify encryption key command is recognized soon.
+ */
+#define MAX_WAIT_FOR_KEY_SECS 20
+
+/* When should waiting for encryption key end. */
+static TimestampTz wait_for_keys_until = 0;
+
+/*
* Main idle loop of postmaster
*
+ * If receive_encryption_key is true, only launch backend(s) to receive
+ * cluster encryption key and stop as soon as the key is in shared memory.
+ *
* NB: Needs to be called with signals blocked
*/
static int
-ServerLoop(void)
+ServerLoop(bool receive_encryption_key)
{
fd_set readmask;
int nSockets;
last_lockfile_recheck_time = last_touch_time = time(NULL);
+ /* Compute wait_for_keys_until if needed and not done yet. */
+ if (receive_encryption_key && wait_for_keys_until == 0)
+ wait_for_keys_until = PgStartTime +
+ MAX_WAIT_FOR_KEY_SECS * USECS_PER_SEC;
+
nSockets = initMasks(&readmask);
for (;;)
@@ -1663,6 +1719,27 @@ ServerLoop(void)
/* Needs to run with blocked signals! */
DetermineSleepTime(&timeout);
+ /*
+ * If waiting for the encryption key, make sure
+ * MAX_WAIT_FOR_KEY_SECS is not exceeded.
+ */
+ if (receive_encryption_key)
+ {
+ TimestampTz timeout_tz, timeout_end;;
+
+ timeout_tz = timeout.tv_sec * USECS_PER_SEC +
+ timeout.tv_usec;
+ timeout_end = GetCurrentTimestamp() + timeout_tz;
+
+ if (timeout_end > wait_for_keys_until)
+ {
+ timeout_tz -= (timeout_end - wait_for_keys_until);
+
+ timeout.tv_sec = timeout_tz / USECS_PER_SEC;
+ timeout.tv_usec = timeout_tz % USECS_PER_SEC;
+ }
+ }
+
PG_SETMASK(&UnBlockSig);
selres = select(nSockets, &rmask, NULL, NULL, &timeout);
@@ -1714,6 +1791,33 @@ ServerLoop(void)
}
}
+ /*
+ * If only waiting for the encryption key, it's too early to launch
+ * the other processes.
+ */
+ if (receive_encryption_key)
+ {
+ /* The regular checks should take place though. */
+ ServerLoopCheckTimeouts();
+
+ /* Done if the key has been delivered. */
+ if (encryption_key_shmem->initialized)
+ return STATUS_OK;
+
+ /*
+ * Do not wait for the key forever. One problem we solve here is
+ * that pg_ctl (or custom script that starts postgres) does not
+ * have to check whether the cluster is encrypted. If DBA forgets
+ * to pass the encryption key command, he'll simply see the
+ * startup to fail and the appropriate message to appear in the
+ * server log.
+ */
+ if (GetCurrentTimestamp() >= wait_for_keys_until)
+ ereport(FATAL, (errmsg("Encryption key not received.")));
+
+ continue;
+ }
+
/* If we have lost the log collector, try to start a new one */
if (SysLoggerPID == 0 && Logging_collector)
SysLoggerPID = SysLogger_Start();
@@ -1990,6 +2094,13 @@ ProcessStartupPacket(Port *port, bool secure_done)
return STATUS_ERROR;
}
+ if (proto == ENCRYPTION_KEY_MSG_CODE && data_encrypted)
+ {
+ processEncryptionKey(buf);
+ /* Not really an error, but we don't want to proceed further */
+ return STATUS_ERROR;
+ }
+
if (proto == NEGOTIATE_SSL_CODE && !secure_done)
{
char SSLok;
@@ -2394,6 +2505,77 @@ processCancelRequest(Port *port, void *pkt)
}
/*
+ * Process a message from backend that contains the encryption key.
+ *
+ * Currently there's no guarantee that only a single backend tries to deliver
+ * the key, but it's not worth trying to synchronize the access. If multiple
+ * callers try to process the same message, nothing should get broken. If some
+ * caller is delivering a wrong key (e.g. due to misconfiguration of another
+ * cluster), it does not help if it waits until processing of the correct key
+ * is finished.
+ *
+ * If the key gets messed up, the worst case is that the server fails to start
+ * up. (No data corruption is expected.)
+ *
+ * Client should not expect any response to this request: failure on client
+ * side indicates either broken connection, wrong key, bug in
+ * send_key_to_postmaster() or repeated call of the client - nothing to be
+ * handled easily by client code. Most of these failures need attention of the
+ * DBA.
+ */
+void
+processEncryptionKey(void *pkt)
+{
+ EncryptionKeyMsg *msg = (EncryptionKeyMsg *) pkt;
+
+ /* Backend to receive the key should not be launched otherwise. */
+ Assert(data_encrypted);
+
+ /*
+ * The checks below would fire in this case too, but this error message is
+ * clearer for the case like this.
+ */
+ if (pmState != PM_INIT)
+ {
+ ereport(COMMERROR,
+ (errmsg("encryption key can only be setup during startup"),
+ errhint("Check if the cluster is encrypted.")));
+ return;
+ }
+
+ /*
+ * Someone else already initialized the key. This is not fatal but seems
+ * to indicate misconfiguration which DBA should be aware of. Check also
+ * encryption_setup_done because shared memory could have been reset. As
+ * postmaster has a local copy of the key, the new key should not be used,
+ * but it'd be inconsistent if we didn't complain in this special case.
+ */
+ if (encryption_key_shmem->initialized || encryption_setup_done)
+ {
+ ereport(COMMERROR,
+ (errmsg("received encryption key more than once")));
+ return;
+ }
+
+ if (msg->version != 1)
+ {
+ ereport(COMMERROR,
+ (errmsg("unexpected version of encryption key message %d",
+ msg->version)));
+ return;
+ }
+
+ memcpy(encryption_key_shmem->data, msg->data, ENCRYPTION_KEY_LENGTH);
+ /*
+ * XXX Is memory barrier needed here to make sure that the copying is
+ * done?
+ */
+ encryption_key_shmem->initialized = true;
+
+ ereport(DEBUG1, (errmsg_internal("encryption key received")));
+}
+
+/*
* canAcceptConnections --- check to see if database state allows connections.
*/
static CAC_state
@@ -2418,7 +2600,8 @@ canAcceptConnections(void)
else if (Shutdown > NoShutdown)
return CAC_SHUTDOWN; /* shutdown is pending */
else if (!FatalError &&
- (pmState == PM_STARTUP ||
+ ((data_encrypted && pmState == PM_INIT) ||
+ pmState == PM_STARTUP ||
pmState == PM_RECOVERY))
return CAC_STARTUP; /* normal startup */
else if (!FatalError &&
@@ -2761,6 +2944,16 @@ pmdie(SIGNAL_ARGS)
pmState = (pmState == PM_RUN) ?
PM_WAIT_BACKUP : PM_WAIT_READONLY;
}
+ else if (pmState == PM_INIT && data_encrypted &&
+ !encryption_setup_done)
+ {
+ /*
+ * Only backends to receive encryption key may be active now,
+ * and these should not be involved in any transactions.
+ */
+ SignalSomeChildren(SIGTERM, BACKEND_TYPE_NORMAL);
+ pmState = PM_WAIT_BACKENDS;
+ }
/*
* Now wait for online backup mode to end and backends to exit. If
@@ -2828,6 +3021,16 @@ pmdie(SIGNAL_ARGS)
signal_child(WalWriterPID, SIGTERM);
pmState = PM_WAIT_BACKENDS;
}
+ else if (pmState == PM_INIT && data_encrypted &&
+ !encryption_setup_done)
+ {
+ /*
+ * Only backends to receive encryption key may be active now,
+ * and these should not be involved in any transactions.
+ */
+ SignalSomeChildren(SIGTERM, BACKEND_TYPE_NORMAL);
+ pmState = PM_WAIT_BACKENDS;
+ }
/*
* Now wait for backends to exit. If there are none,
diff --git a/src/backend/storage/file/encryption.c b/src/backend/storage/file/encryption.c
index 629138fae7..1eda334762 100644
--- a/src/backend/storage/file/encryption.c
+++ b/src/backend/storage/file/encryption.c
@@ -53,6 +53,10 @@ EVP_CIPHER_CTX *ctx_encrypt,
unsigned char encryption_key[ENCRYPTION_KEY_LENGTH];
+#ifndef FRONTEND
+ShmemEncryptionKey *encryption_key_shmem = NULL;
+#endif /* FRONTEND */
+
bool data_encrypted = false;
char encryption_verification[ENCRYPTION_SAMPLE_SIZE];
@@ -66,6 +70,36 @@ static void evp_error(void);
#ifndef FRONTEND
/*
+ * Report space needed for our shared memory area
+ */
+Size
+EncryptionShmemSize(void)
+{
+ return sizeof(ShmemEncryptionKey);
+}
+
+/*
+ * Initialize our shared memory area
+ */
+void
+EncryptionShmemInit(void)
+{
+ bool found;
+
+ encryption_key_shmem = ShmemInitStruct("Cluster Encryption Key",
+ EncryptionShmemSize(),
+ &found);
+ if (!IsUnderPostmaster)
+ {
+ Assert(!found);
+
+ encryption_key_shmem->initialized = false;
+ }
+ else
+ Assert(found);
+}
+
+/*
* Read encryption key in hexadecimal form from stdin and store it in
* encryption_key variable.
*/
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index d7d733530f..771bf30c63 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -147,6 +147,7 @@ CreateSharedMemoryAndSemaphores(int port)
size = add_size(size, BTreeShmemSize());
size = add_size(size, SyncScanShmemSize());
size = add_size(size, AsyncShmemSize());
+ size = add_size(size, EncryptionShmemSize());
#ifdef EXEC_BACKEND
size = add_size(size, ShmemBackendArraySize());
#endif
@@ -263,6 +264,7 @@ CreateSharedMemoryAndSemaphores(int port)
BTreeShmemInit();
SyncScanShmemInit();
AsyncShmemInit();
+ EncryptionShmemInit();
#ifdef EXEC_BACKEND
diff --git a/src/bin/Makefile b/src/bin/Makefile
index 903e58121f..b05a8bbb4e 100644
--- a/src/bin/Makefile
+++ b/src/bin/Makefile
@@ -22,6 +22,7 @@ SUBDIRS = \
pg_controldata \
pg_ctl \
pg_dump \
+ pg_keytool \
pg_resetwal \
pg_rewind \
pg_test_fsync \
diff --git a/src/bin/pg_keytool/.gitignore b/src/bin/pg_keytool/.gitignore
new file mode 100644
index 0000000000..249876a6ed
--- /dev/null
+++ b/src/bin/pg_keytool/.gitignore
@@ -0,0 +1 @@
+/pg_keytool
diff --git a/src/bin/pg_keytool/Makefile b/src/bin/pg_keytool/Makefile
new file mode 100644
index 0000000000..6da0631f32
--- /dev/null
+++ b/src/bin/pg_keytool/Makefile
@@ -0,0 +1,31 @@
+# src/bin/pg_keysetup/Makefile
+
+PGFILEDESC = "pg_keytool - handle cluster encryption key"
+PGAPPICON=win32
+
+subdir = src/bin/pg_keytool
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = pg_keytool.o $(RMGRDESCOBJS) $(WIN32RES)
+
+override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
+override CPPFLAGS := -DFRONTEND $(CPPFLAGS)
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
+
+all: pg_keytool
+
+pg_keytool: $(OBJS) | submake-libpgport
+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+
+install: all installdirs
+ $(INSTALL_PROGRAM) pg_keytool$(X) '$(DESTDIR)$(bindir)/pg_keytool$(X)'
+
+installdirs:
+ $(MKDIR_P) '$(DESTDIR)$(bindir)'
+
+uninstall:
+ rm -f '$(DESTDIR)$(bindir)/pg_keytool$(X)'
+
+clean distclean maintainer-clean:
+ rm -f pg_keytool$(X) $(OBJS) encryption.c
diff --git a/src/bin/pg_keytool/pg_keytool.c b/src/bin/pg_keytool/pg_keytool.c
new file mode 100644
index 0000000000..322625da41
--- /dev/null
+++ b/src/bin/pg_keytool/pg_keytool.c
@@ -0,0 +1,199 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_keytool.c - Handle cluster encryption key.
+ *
+ * Copyright (c) 2013-2019, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/bin/pg_keytool/pg_keytool.c
+ *-------------------------------------------------------------------------
+ */
+/*
+ * TODO Adopt the new frontend logging API, after some things are clarified:
+ * https://www.postgresql.org/message-id/1939.1560773970%40localhost
+ */
+#define FRONTEND 1
+#include "postgres.h"
+
+#include <dirent.h>
+#include <unistd.h>
+
+#include "common/fe_memutils.h"
+#include "common/logging.h"
+#include "fe_utils/encryption.h"
+#include "libpq-fe.h"
+#include "libpq-int.h"
+#include "libpq/pqcomm.h"
+#include "port/pg_crc32c.h"
+#include "storage/encryption.h"
+#include "getopt_long.h"
+
+#ifdef USE_ENCRYPTION
+static const char *progname;
+
+unsigned char encryption_key[ENCRYPTION_KEY_LENGTH];
+
+static void
+usage(const char *progname)
+{
+ const char *env;
+
+ printf(_("%s is a tool to handle cluster encryption key.\n\n"),
+ progname);
+ printf(_("Usage:\n"));
+ printf(_(" %s [OPTION]...\n"), progname);
+ printf(_("\nOptions:\n"));
+ /* Display default host */
+ env = getenv("PGHOST");
+ printf(_(" -h, --host=HOSTNAME database server host or socket directory (default: \"%s\")\n"),
+ env ? env : _("local socket"));
+ /* Display default port */
+ env = getenv("PGPORT");
+ printf(_(" -p, --port=PORT database server port (default: \"%s\")\n"),
+ env ? env : DEF_PGPORT_STR);
+ printf(_(" -s, send output to database server\n"));
+ printf(_(" -?, --help show this help, then exit\n\n"));
+ printf(_("Key is read from stdin and sent either to stdout or to PostgreSQL server being started\n"));
+}
+#endif /* USE_ENCRYPTION */
+
+int
+main(int argc, char **argv)
+{
+/*
+ * If no encryption library is linked, let the utility fail immediately. It'd
+ * be weird if we reported incorrect usage just to say later that no useful
+ * work can be done anyway.
+ */
+#ifdef USE_ENCRYPTION
+ int c;
+ char *host = NULL;
+ char *port_str = NULL;
+ bool to_server = false;
+ int i, n;
+ int optindex;
+ char key_chars[ENCRYPTION_KEY_CHARS];
+
+ static struct option long_options[] =
+ {
+ {"pgdata", required_argument, NULL, 'D'},
+ {"host", required_argument, NULL, 'h'},
+ {"port", required_argument, NULL, 'p'},
+ {NULL, 0, NULL, 0}
+ };
+
+ pg_logging_init(argv[0]);
+ progname = get_progname(argv[0]);
+
+ if (argc > 1)
+ {
+ if (strcmp(argv[1], "--help") == 0 || strcmp(argv[1], "-?") == 0)
+ {
+ usage(progname);
+ exit(0);
+ }
+ if (strcmp(argv[1], "--version") == 0 || strcmp(argv[1], "-V") == 0)
+ {
+ puts("pg_keytool (PostgreSQL) " PG_VERSION);
+ exit(0);
+ }
+ }
+
+ while ((c = getopt_long(argc, argv, "h:p:s",
+ long_options, &optindex)) != -1)
+ {
+ switch (c)
+ {
+ case 'h':
+ host = pg_strdup(optarg);
+ break;
+
+ case 'p':
+ port_str = pg_strdup(optarg);
+ break;
+
+ case 's':
+ to_server = true;
+ break;
+
+ case '?':
+ /* Actual help option given */
+ if (strcmp(argv[optind - 1], "-?") == 0)
+ {
+ usage(progname);
+ exit(EXIT_SUCCESS);
+ }
+
+ default:
+ pg_log_error("Try \"%s --help\" for more information.", progname);
+ exit(1);
+ }
+ }
+
+ /* Complain if any arguments remain */
+ if (optind < argc)
+ {
+ fprintf(stderr, _("%s: too many command-line arguments (first is \"%s\")\n"),
+ progname, argv[optind]);
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+ if ((host || port_str) && !to_server)
+ {
+ pg_log_error("host and port can only be passed along with the -s option");
+ exit(1);
+ }
+
+ /* Read the key. */
+ n = 0;
+ /* Key length in characters (two characters per hexadecimal digit) */
+ while ((c = getchar()) != EOF && c != '\n')
+ {
+ if (n >= ENCRYPTION_KEY_CHARS)
+ {
+ pg_log_error("The key is too long");
+ exit(EXIT_FAILURE);
+ }
+
+ key_chars[n++] = c;
+ }
+
+ if (n < ENCRYPTION_KEY_CHARS)
+ {
+ pg_log_error("The key is too short");
+ exit(EXIT_FAILURE);
+ }
+
+ for (i = 0; i < ENCRYPTION_KEY_LENGTH; i++)
+ {
+ if (sscanf(key_chars + 2 * i, "%2hhx", encryption_key + i) == 0)
+ {
+ pg_log_error("Invalid character in encryption key at position %d",
+ 2 * i);
+ exit(EXIT_FAILURE);
+ }
+ }
+
+ /*
+ * Send the encryption key either to stdout or to server.
+ */
+ if (!to_server)
+ {
+ for (i = 0; i < ENCRYPTION_KEY_LENGTH; i++)
+ printf("%.2x", encryption_key[i]);
+ printf("\n");
+ }
+ else
+ {
+ if (!send_key_to_postmaster(host, port_str, encryption_key))
+ pg_log_error("could not send encryption key to server");
+ }
+
+#else
+ pg_log_fatal(ENCRYPTION_NOT_SUPPORTED_MSG);
+ exit(EXIT_FAILURE);
+#endif /* USE_ENCRYPTION */
+ return 0;
+}
diff --git a/src/fe_utils/encryption.c b/src/fe_utils/encryption.c
index dab1435014..ca37c9f373 100644
--- a/src/fe_utils/encryption.c
+++ b/src/fe_utils/encryption.c
@@ -88,3 +88,104 @@ run_encryption_key_command(unsigned char *encryption_key)
pfree(buf);
pclose(fp);
}
+
+/*
+ * Send the contents of encryption_key in the form of special startup packet
+ * to a server that is being started.
+ *
+ * Returns true if we could send the message and false if not, however even
+ * success does not guarantee that server started up - caller should
+ * eventually test server connection himself.
+ *
+ * TODO Windows stuff?
+ */
+bool
+send_key_to_postmaster(const char *host, const char *port,
+ const unsigned char *encryption_Key)
+{
+ const char **keywords = pg_malloc0(3 * sizeof(*keywords));
+ const char **values = pg_malloc0(3 * sizeof(*values));
+ int i;
+ PGconn *conn = NULL;
+ int sock = -1;
+ EncryptionKeyMsg message;
+ int msg_size, packet_size;
+ char *packet = NULL;
+ bool res = true;
+
+/* How many seconds we can wait for the postmaster to receive the key. */
+#define SEND_ENCRYPT_KEY_TIMEOUT 60
+
+ if (host)
+ {
+ keywords[0] = "host";
+ values[0] = host;
+ }
+ keywords[1] = "port";
+ values[1] = port;
+
+ /* Compose the message. */
+ message.encryptionKeyCode = pg_hton32(ENCRYPTION_KEY_MSG_CODE);
+ message.version = 1;
+ memcpy(message.data, encryption_key, ENCRYPTION_KEY_LENGTH);
+ msg_size = offsetof(EncryptionKeyMsg, data) + ENCRYPTION_KEY_LENGTH;
+
+ packet_size = msg_size + 4;
+ packet = (char *) palloc(packet_size);
+ *((int32 *) packet) = pg_hton32(packet_size);
+ memcpy(packet + 4, &message, msg_size);
+
+ /*
+ * Although we don't expect the server to accept regular libpq messages,
+ * we try to get at least a valid socket.
+ */
+ for (i = 0; i < SEND_ENCRYPT_KEY_TIMEOUT + 1; i++)
+ {
+ if (i > 0)
+ /* Sleep for 1 second. */
+ pg_usleep(1000000L);
+
+ if (conn)
+ {
+ PQfinish(conn);
+ conn = NULL;
+ }
+
+ conn = PQconnectStartParams(keywords, values, false);
+ if (conn == NULL)
+ continue;
+
+ sock = PQsocket(conn);
+ /* Cannot send the key if there's no valid socket. */
+ if (sock == -1)
+ continue;
+
+ /* Non-blocking write would only make this simple case tricky. */
+ if (!pg_set_block(sock))
+ continue;
+
+ retry:
+ /*
+ * Send the packet. Here we need to use low level API because the server
+ * does is not fully up so libpq cannot be used properly.
+ */
+ if (send(sock, (char *) packet, packet_size, 0) != packet_size)
+ {
+ if (SOCK_ERRNO == EINTR)
+ /* Interrupted system call - we'll just try again */
+ goto retry;
+ continue;
+ }
+ else
+ /* Success */
+ break;
+ }
+
+ pg_free(keywords);
+ pg_free(values);
+ if (conn)
+ PQfinish(conn);
+ pfree(packet);
+
+ return res;
+}
diff --git a/src/include/fe_utils/encryption.h b/src/include/fe_utils/encryption.h
index 69e3b2f756..7307557ed6 100644
--- a/src/include/fe_utils/encryption.h
+++ b/src/include/fe_utils/encryption.h
@@ -16,3 +16,5 @@
extern char *encryption_key_command;
extern void run_encryption_key_command(unsigned char *encryption_key);
+extern bool send_key_to_postmaster(const char *host, const char *port,
+ const unsigned char *encryption_Key);
diff --git a/src/include/libpq/pqcomm.h b/src/include/libpq/pqcomm.h
index baf6a4b6c0..09822801c7 100644
--- a/src/include/libpq/pqcomm.h
+++ b/src/include/libpq/pqcomm.h
@@ -23,6 +23,8 @@
#endif
#include <netinet/in.h>
+#include "storage/encryption.h"
+
#ifdef HAVE_STRUCT_SOCKADDR_STORAGE
#ifndef HAVE_STRUCT_SOCKADDR_STORAGE_SS_FAMILY
@@ -205,4 +207,26 @@ typedef struct CancelRequestPacket
#define NEGOTIATE_SSL_CODE PG_PROTOCOL(1234,5679)
#define NEGOTIATE_GSS_CODE PG_PROTOCOL(1234,5680)
+/* Client can also send cluster encryption key to the postmaster. */
+#define ENCRYPTION_KEY_MSG_CODE PG_PROTOCOL(1234,5681)
+
+/*
+ * Message containing the encryption key, received by postmaster during
+ * startup.
+ *
+ * TODO Consider adding extension identifier and key_length field so that
+ * postmaster can also receove extension-specific keys. Extension that needs a
+ * key would call a function from _PG_init() that tells postmaster where to
+ * store the key and how long the key is. (Should also postgres in standalone
+ * mode accept keys for extensions?)
+ */
+typedef struct EncryptionKeyMsg
+{
+ /* Note this integer field is stored in network byte order! */
+ MsgType encryptionKeyCode; /* code to identify a key message */
+
+ unsigned char version; /* message format version. */
+ char data[ENCRYPTION_KEY_LENGTH]; /* the key itself. */
+} EncryptionKeyMsg;
+
#endif /* PQCOMM_H */
diff --git a/src/include/storage/encryption.h b/src/include/storage/encryption.h
index 8479b76a34..4f7b96d4f3 100644
--- a/src/include/storage/encryption.h
+++ b/src/include/storage/encryption.h
@@ -60,6 +60,29 @@ typedef enum CipherKind
/* Key to encrypt / decrypt data. */
extern unsigned char encryption_key[];
+#ifndef FRONTEND
+/*
+ * Space for the encryption key in shared memory. Backend that receives the
+ * key during startup stores it here so postmaster can eventually take a local
+ * copy.
+ *
+ * We cannot protect the "initialized" field with a spinlock because the data
+ * will be accessed by postmaster, but it should not be necessary for bool
+ * type. Furthermore, synchronization is not really critical here, see
+ * processEncryptionKey().
+ */
+typedef struct ShmemEncryptionKey
+{
+ char data[ENCRYPTION_KEY_LENGTH]; /* the key */
+ bool initialized; /* received the key? */
+} ShmemEncryptionKey;
+
+/*
+ * Encryption key in the shared memory.
+ */
+extern ShmemEncryptionKey *encryption_key_shmem;
+#endif /* FRONTEND */
+
/*
* The encrypted data is a series of blocks of size
* ENCRYPTION_BLOCK. Currently we use the EVP_aes_256_xts implementation. Make
@@ -86,6 +109,9 @@ extern char encryption_verification[];
extern bool encryption_setup_done;
#ifndef FRONTEND
+extern Size EncryptionShmemSize(void);
+extern void EncryptionShmemInit(void);
+
typedef int (*read_encryption_key_cb) (void);
extern void read_encryption_key(read_encryption_key_cb read_char);
#endif /* FRONTEND */
--
2.13.7
v04-0004-Minor-refactoring-of-pg_ctl.c.patch (text/x-diff, 4.3K) download
From bdc832086b1d98aec69ab6d3b6c3ae6c09560d05 Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Fri, 5 Jul 2019 16:24:01 +0200
Subject: [PATCH 04/17] Minor refactoring of pg_ctl.c.
Added get_config_variable() function to pg_ctl.c because the next patch of the
series will need to retrieve some more configuration variables from
postgresql.conf.
---
src/bin/pg_ctl/pg_ctl.c | 69 ++++++++++++++++++++++++++++++++---------------
src/fe_utils/encryption.c | 4 +++
2 files changed, 52 insertions(+), 21 deletions(-)
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index a10bc8d545..71741ffc81 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -133,6 +133,7 @@ static void do_logrotate(void);
static void do_kill(pgpid_t pid);
static void print_msg(const char *msg);
static void adjust_data_dir(void);
+static char *get_config_variable(const char *var_name, size_t res_size);
#ifdef WIN32
#if (_MSC_VER >= 1800)
@@ -2172,28 +2173,46 @@ set_starttype(char *starttypeopt)
static void
adjust_data_dir(void)
{
- char cmd[MAXPGPATH],
- filename[MAXPGPATH],
- *my_exec_path;
- FILE *fd;
+ char conf_path[MAXPGPATH];
+ char *filename;
+ FILE *fd;
/* do nothing if we're working without knowledge of data dir */
if (pg_config == NULL)
return;
- /* If there is no postgresql.conf, it can't be a config-only dir */
- snprintf(filename, sizeof(filename), "%s/postgresql.conf", pg_config);
- if ((fd = fopen(filename, "r")) == NULL)
+ /* If PG_VERSION exists, it can't be a config-only dir */
+ snprintf(conf_path, sizeof(conf_path), "%s/PG_VERSION", pg_config);
+ if ((fd = fopen(conf_path, "r")) != NULL)
return;
- fclose(fd);
- /* If PG_VERSION exists, it can't be a config-only dir */
- snprintf(filename, sizeof(filename), "%s/PG_VERSION", pg_config);
- if ((fd = fopen(filename, "r")) != NULL)
+ filename = get_config_variable("data_directory", MAXPGPATH);
+
+ if (filename)
{
- fclose(fd);
- return;
+ free(pg_data);
+ pg_data = filename;
+ canonicalize_path(pg_data);
}
+}
+
+/*
+ * Retrieve value of configuration variable from configuration file.
+ */
+static char *
+get_config_variable(const char *var_name, size_t res_size)
+{
+ char cmd[MAXPGPATH],
+ filename[MAXPGPATH],
+ *my_exec_path;
+ char *result;
+ FILE *fd;
+
+ /* If there is no postgresql.conf, the data dir is not useful. */
+ snprintf(filename, sizeof(filename), "%s/postgresql.conf", pg_config);
+ if ((fd = fopen(filename, "r")) == NULL)
+ return NULL;
+ fclose(fd);
/* Must be a configuration directory, so find the data directory */
@@ -2204,27 +2223,35 @@ adjust_data_dir(void)
my_exec_path = pg_strdup(exec_path);
/* it's important for -C to be the first option, see main.c */
- snprintf(cmd, MAXPGPATH, "\"%s\" -C data_directory %s%s",
+ snprintf(cmd, MAXPGPATH, "\"%s\" -C %s %s%s",
my_exec_path,
+ var_name,
pgdata_opt ? pgdata_opt : "",
post_opts ? post_opts : "");
+ result = pg_malloc(res_size);
fd = popen(cmd, "r");
- if (fd == NULL || fgets(filename, sizeof(filename), fd) == NULL)
+ if (fd == NULL || fgets(result, res_size, fd) == NULL)
{
- write_stderr(_("%s: could not determine the data directory using command \"%s\"\n"), progname, cmd);
+ write_stderr(_("%s: could not determine the value of \"%s\" \"%s\"\n"),
+ progname, var_name, cmd);
+ free(result);
exit(1);
}
pclose(fd);
free(my_exec_path);
/* Remove trailing newline */
- if (strchr(filename, '\n') != NULL)
- *strchr(filename, '\n') = '\0';
+ if (strchr(result, '\n') != NULL)
+ *strchr(result, '\n') = '\0';
- free(pg_data);
- pg_data = pg_strdup(filename);
- canonicalize_path(pg_data);
+ if (strlen(result) == 0)
+ {
+ pg_free(result);
+ result = NULL;
+ }
+
+ return result;
}
diff --git a/src/fe_utils/encryption.c b/src/fe_utils/encryption.c
index ca37c9f373..e888d4ae34 100644
--- a/src/fe_utils/encryption.c
+++ b/src/fe_utils/encryption.c
@@ -31,6 +31,10 @@ char *encryption_key_command = NULL;
/*
* Run the command that is supposed to generate encryption key and store it
* where encryption_key points to.
+ *
+ * TODO Replace pattern (e.g. %D) in the command with data directory so that
+ * DBA knows for which cluster he enters the password. That should also make
+ * the use of pg_keytool in the command easier.
*/
void
run_encryption_key_command(unsigned char *encryption_key)
--
2.13.7
v04-0005-Teach-pg_ctl-to-send-the-encryption-key-to-postmaste.patch (text/x-diff, 8.0K) download
From aa5e08b1f545dd65bffe1ff473e6b890e4831eda Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Fri, 5 Jul 2019 16:24:01 +0200
Subject: [PATCH 05/17] Teach pg_ctl to send the encryption key to postmaster.
With this patch the server can be started this way:
pg_ctl -D data -K "echo 71916b8b93063c280e4e0ee646395f450bda4f1bac85c57bc3a6afd168f3ab47" start
---
src/bin/pg_ctl/Makefile | 2 +-
src/bin/pg_ctl/pg_ctl.c | 117 ++++++++++++++++++++++++++++++++++++++++++++--
src/fe_utils/encryption.c | 4 --
3 files changed, 115 insertions(+), 8 deletions(-)
diff --git a/src/bin/pg_ctl/Makefile b/src/bin/pg_ctl/Makefile
index 83cbf97ed8..b9f755d93a 100644
--- a/src/bin/pg_ctl/Makefile
+++ b/src/bin/pg_ctl/Makefile
@@ -20,9 +20,9 @@ include $(top_builddir)/src/Makefile.global
# but let's not pull that in on platforms where we don't need it.
ifeq ($(PORTNAME), win32)
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
-LDFLAGS_INTERNAL += $(libpq_pgport)
SUBMAKE_LIBPQ := submake-libpq
endif
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
OBJS= pg_ctl.o $(WIN32RES)
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 71741ffc81..7e414ac048 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -27,6 +27,7 @@
#include "common/controldata_utils.h"
#include "common/file_perm.h"
#include "common/logging.h"
+#include "fe_utils/encryption.h"
#include "getopt_long.h"
#include "utils/pidfile.h"
@@ -106,6 +107,12 @@ static char logrotate_file[MAXPGPATH];
static volatile pgpid_t postmasterPID = -1;
+/*
+ * Define encryption_key locally rather that linking to storage/encryption.c
+ * only because of this one variable.
+ */
+unsigned char encryption_key[ENCRYPTION_KEY_LENGTH];
+
#ifdef WIN32
static DWORD pgctl_start_type = SERVICE_AUTO_START;
static SERVICE_STATUS status;
@@ -134,6 +141,8 @@ static void do_kill(pgpid_t pid);
static void print_msg(const char *msg);
static void adjust_data_dir(void);
static char *get_config_variable(const char *var_name, size_t res_size);
+static char *get_first_csv_item(char *csv_list);
+static void get_postmaster_address(char **host_p, char **port_str_p);
#ifdef WIN32
#if (_MSC_VER >= 1800)
@@ -825,6 +834,8 @@ do_start(void)
{
pgpid_t old_pid = 0;
pgpid_t pm_pid;
+ char *host = NULL;
+ char *port_str = NULL;
if (ctl_command != RESTART_COMMAND)
{
@@ -864,8 +875,36 @@ do_start(void)
}
#endif
+ if (encryption_key_command)
+ {
+ /*
+ * If encryption key is needed, retrieve it before trying to start
+ * postmaster.
+ */
+ run_encryption_key_command(encryption_key);
+
+ /*
+ * Where should the key be sent?
+ *
+ * Retrieve the info now because the time the retrieval will take
+ * should not overlap with waiting for the startup completion below.
+ */
+ get_postmaster_address(&host, &port_str);
+ }
+
pm_pid = start_postmaster();
+ if (encryption_key_command)
+ {
+ /* Send the key to the postmaster */
+ if (!send_key_to_postmaster(host, port_str, encryption_key))
+ {
+ write_stderr(_("%s: could not send encryption key to postmaster\n"),
+ progname);
+ exit(1);
+ }
+ }
+
if (do_wait)
{
/*
@@ -2025,10 +2064,10 @@ do_help(void)
printf(_("%s is a utility to initialize, start, stop, or control a PostgreSQL server.\n\n"), progname);
printf(_("Usage:\n"));
printf(_(" %s init[db] [-D DATADIR] [-s] [-o OPTIONS]\n"), progname);
- printf(_(" %s start [-D DATADIR] [-l FILENAME] [-W] [-t SECS] [-s]\n"
+ printf(_(" %s start [-D DATADIR] [-l FILENAME] [-W] [-t SECS] [-K KEY-CMD] [-s]\n"
" [-o OPTIONS] [-p PATH] [-c]\n"), progname);
printf(_(" %s stop [-D DATADIR] [-m SHUTDOWN-MODE] [-W] [-t SECS] [-s]\n"), progname);
- printf(_(" %s restart [-D DATADIR] [-m SHUTDOWN-MODE] [-W] [-t SECS] [-s]\n"
+ printf(_(" %s restart [-D DATADIR] [-m SHUTDOWN-MODE] [-W] [-t SECS] [-K KEY-CMD] [-s]\n"
" [-o OPTIONS] [-c]\n"), progname);
printf(_(" %s reload [-D DATADIR] [-s]\n"), progname);
printf(_(" %s status [-D DATADIR]\n"), progname);
@@ -2046,6 +2085,10 @@ do_help(void)
#ifdef WIN32
printf(_(" -e SOURCE event source for logging when running as a service\n"));
#endif
+#ifdef USE_ENCRYPTION
+ printf(_(" -K, --encryption-key-command\n"
+ " command that returns encryption key\n\n"));
+#endif /* USE_ENCRYPTION */
printf(_(" -s, --silent only print errors, no informational messages\n"));
printf(_(" -t, --timeout=SECS seconds to wait when using -w option\n"));
printf(_(" -V, --version output version information, then exit\n"));
@@ -2254,6 +2297,66 @@ get_config_variable(const char *var_name, size_t res_size)
return result;
}
+/*
+ * Get the first item of comma-separated list or NULL if there's no valid
+ * item.
+ */
+static char *
+get_first_csv_item(char *csv_list)
+{
+ char *start, *end, *result;
+
+ start = csv_list;
+ /* First, skip the leading space. */
+ while (isspace(*start))
+ start++;
+ if (*start == '\0')
+ return NULL;
+
+ end = start;
+ while (*end != '\0' && *end != ',' && !isspace(*end))
+ end++;
+ if (end == start)
+ return NULL;
+
+ result = pg_strdup(start);
+
+ /* Trim the string if needed. */
+ if (*end != '\0')
+ result[end - start] = '\0';
+
+ return result;
+}
+
+/*
+ * Retrieve host name and port to which the encryption key should be
+ * sent. host can remain NULL, which will make libpq assume "localhost".
+ */
+static void
+get_postmaster_address(char **host_p, char **port_str_p)
+{
+ char *socket_dirs;
+
+ /* First, try to connect via the unix socket. */
+ socket_dirs = get_config_variable("unix_socket_directories",
+ MAXPGPATH);
+ if (socket_dirs)
+ {
+ /* If there are multiple sockets, use the first one. */
+ *host_p = get_first_csv_item(socket_dirs);
+ }
+ else
+ {
+ /* Let libpq assume "localhost".*/
+ *host_p = NULL;
+ }
+
+ /*
+ * The maximum length of the port number is 5 characters for the port
+ * number (65535 at maximum) + '\n' + terminating '\0'.
+ */
+ *port_str_p = get_config_variable("port", 7);
+}
static DBState
get_control_dbstate(void)
@@ -2286,6 +2389,7 @@ main(int argc, char **argv)
{"options", required_argument, NULL, 'o'},
{"silent", no_argument, NULL, 's'},
{"timeout", required_argument, NULL, 't'},
+ {"encryption-key-command", required_argument, NULL, 'K'},
{"core-files", no_argument, NULL, 'c'},
{"wait", no_argument, NULL, 'w'},
{"no-wait", no_argument, NULL, 'W'},
@@ -2356,7 +2460,7 @@ main(int argc, char **argv)
/* process command-line options */
while (optind < argc)
{
- while ((c = getopt_long(argc, argv, "cD:e:l:m:N:o:p:P:sS:t:U:wW",
+ while ((c = getopt_long(argc, argv, "cD:e:K:l:m:N:o:p:P:sS:t:U:wW",
long_options, &option_index)) != -1)
{
switch (c)
@@ -2382,6 +2486,9 @@ main(int argc, char **argv)
case 'e':
event_source = pg_strdup(optarg);
break;
+ case 'K':
+ encryption_key_command = pg_strdup(optarg);
+ break;
case 'l':
log_file = pg_strdup(optarg);
break;
@@ -2558,6 +2665,10 @@ main(int argc, char **argv)
umask(pg_mode_mask);
}
+ if (encryption_key_command && ctl_command !=
+ START_COMMAND && ctl_command != RESTART_COMMAND)
+ write_stderr(_("%s: ignoring the -K option, it's only useful for start or restart commands\n"), progname);
+
switch (ctl_command)
{
case INIT_COMMAND:
diff --git a/src/fe_utils/encryption.c b/src/fe_utils/encryption.c
index e888d4ae34..ca37c9f373 100644
--- a/src/fe_utils/encryption.c
+++ b/src/fe_utils/encryption.c
@@ -31,10 +31,6 @@ char *encryption_key_command = NULL;
/*
* Run the command that is supposed to generate encryption key and store it
* where encryption_key points to.
- *
- * TODO Replace pattern (e.g. %D) in the command with data directory so that
- * DBA knows for which cluster he enters the password. That should also make
- * the use of pg_keytool in the command easier.
*/
void
run_encryption_key_command(unsigned char *encryption_key)
--
2.13.7
v04-0006-Allow-user-to-use-password-instead-of-encryption-key.patch (text/x-diff, 15.7K) download
From 0e853e3074afd029c665d14cab750cacdba149b9 Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Fri, 5 Jul 2019 16:24:01 +0200
Subject: [PATCH 06/17] Allow user to use password instead of encryption key.
pg_keytool reads the password from stdin, uses "key derivation function (KDF)
parameters" stored in the data directory to derive the key and writes the key
to standard output.
The cluster can be created this way
initdb -D data -K "echo securepwd | pg_keytool -D data -w"
and started either this way
pg_ctl -D data -K "echo securepwd | pg_keytool -D data -w" start
or this way
postgres -D data
and (in another session)
echo securepwd | pg_keytool -D data -ws
Both initdb and pg_ctl can substitute the data directory for %D pattern in the
key encryption command. Thus user uses pg_keytool to derive the password, he
can in fact say
initdb -D data -K "echo securepwd | pg_keytool -D %D -w"
and
pg_ctl -D data -K "echo securepwd | pg_keytool -D %D -w" start
respectively.
---
src/bin/initdb/initdb.c | 26 +++-
src/bin/pg_ctl/pg_ctl.c | 2 +-
src/bin/pg_keytool/pg_keytool.c | 87 +++++++++---
src/fe_utils/encryption.c | 273 +++++++++++++++++++++++++++++++++++++-
src/include/fe_utils/encryption.h | 8 +-
src/include/storage/encryption.h | 8 ++
6 files changed, 379 insertions(+), 25 deletions(-)
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index f2a582e975..bfa8f5fad4 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -3004,11 +3004,31 @@ initialize_data_directory(void)
write_version_file(NULL);
/*
- * If the cluster will be encrypted, run the command to generate the
- * encryption key.
+ * If the cluster will be encrypted, write the KDF file so that encryption
+ * key can be derived from password.
*/
if (encryption_key_command)
- run_encryption_key_command(encryption_key);
+ {
+ /*
+ * XXX Since execution of encryption_key_command produce the key (as
+ * opposed to password), we don't know if the command received the key
+ * itself or a password. If DBA provided initdb with a key, he will
+ * never use password in the future (there was no KDF so far so the
+ * key could not be derived from password, and the password can hardly
+ * be derived from the key), so the KDF file may be useless. We don't
+ * have enough information to recognize this special case, so just
+ * initialize and write the KDF unconditionally.
+ */
+ init_kdf();
+ write_kdf_file(pg_data);
+
+ /*
+ * The key command is allowed to use pg_keytool, which in turn needs
+ * the KDF parameters. The KDF parameters are now available so we can
+ * run the command.
+ */
+ run_encryption_key_command(encryption_key, pg_data);
+ }
/* Select suitable configuration settings */
set_null_conf();
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
index 7e414ac048..e8e4eee162 100644
--- a/src/bin/pg_ctl/pg_ctl.c
+++ b/src/bin/pg_ctl/pg_ctl.c
@@ -881,7 +881,7 @@ do_start(void)
* If encryption key is needed, retrieve it before trying to start
* postmaster.
*/
- run_encryption_key_command(encryption_key);
+ run_encryption_key_command(encryption_key, pg_data);
/*
* Where should the key be sent?
diff --git a/src/bin/pg_keytool/pg_keytool.c b/src/bin/pg_keytool/pg_keytool.c
index 322625da41..c5151ad57c 100644
--- a/src/bin/pg_keytool/pg_keytool.c
+++ b/src/bin/pg_keytool/pg_keytool.c
@@ -43,6 +43,7 @@ usage(const char *progname)
printf(_("Usage:\n"));
printf(_(" %s [OPTION]...\n"), progname);
printf(_("\nOptions:\n"));
+ printf(_(" -D, --pgdata=DATADIR data directory\n"));
/* Display default host */
env = getenv("PGHOST");
printf(_(" -h, --host=HOSTNAME database server host or socket directory (default: \"%s\")\n"),
@@ -52,8 +53,9 @@ usage(const char *progname)
printf(_(" -p, --port=PORT database server port (default: \"%s\")\n"),
env ? env : DEF_PGPORT_STR);
printf(_(" -s, send output to database server\n"));
+ printf(_(" -w expect password on input, not a key\n"));
printf(_(" -?, --help show this help, then exit\n\n"));
- printf(_("Key is read from stdin and sent either to stdout or to PostgreSQL server being started\n"));
+ printf(_("Password or key is read from stdin. Key is sent to PostgreSQL server being started\n"));
}
#endif /* USE_ENCRYPTION */
@@ -69,9 +71,12 @@ main(int argc, char **argv)
int c;
char *host = NULL;
char *port_str = NULL;
+ char *DataDir = NULL;
bool to_server = false;
+ bool expect_password = false;
int i, n;
int optindex;
+ char password[ENCRYPTION_PWD_MAX_LENGTH];
char key_chars[ENCRYPTION_KEY_CHARS];
static struct option long_options[] =
@@ -99,11 +104,15 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "h:p:s",
+ while ((c = getopt_long(argc, argv, "h:D:p:sw",
long_options, &optindex)) != -1)
{
switch (c)
{
+ case 'D':
+ DataDir = optarg;
+ break;
+
case 'h':
host = pg_strdup(optarg);
break;
@@ -116,6 +125,10 @@ main(int argc, char **argv)
to_server = true;
break;
+ case 'w':
+ expect_password = true;
+ break;
+
case '?':
/* Actual help option given */
if (strcmp(argv[optind - 1], "-?") == 0)
@@ -146,34 +159,78 @@ main(int argc, char **argv)
exit(1);
}
- /* Read the key. */
+ /*
+ * The KDF file is needed to derive the key from password, and this file
+ * is located in the data directory.
+ */
+ if (expect_password && DataDir == NULL)
+ {
+ pg_log_error("%s: no data directory specified", progname);
+ pg_log_error("Try \"%s --help\" for more information.", progname);
+ exit(EXIT_FAILURE);
+ }
+
+ /*
+ * Read the credentials (key or password).
+ */
n = 0;
/* Key length in characters (two characters per hexadecimal digit) */
while ((c = getchar()) != EOF && c != '\n')
{
- if (n >= ENCRYPTION_KEY_CHARS)
+ if (!expect_password)
{
- pg_log_error("The key is too long");
- exit(EXIT_FAILURE);
+ if (n >= ENCRYPTION_KEY_CHARS)
+ {
+ pg_log_error("The key is too long");
+ exit(EXIT_FAILURE);
+ }
+
+ key_chars[n++] = c;
}
+ else
+ {
+ if (n >= ENCRYPTION_PWD_MAX_LENGTH)
+ {
+ pg_log_error("The password is too long");
+ exit(EXIT_FAILURE);
+ }
- key_chars[n++] = c;
+ password[n++] = c;
+ }
}
- if (n < ENCRYPTION_KEY_CHARS)
+ /* If password was received, turn it into encryption key. */
+ if (!expect_password)
{
- pg_log_error("The key is too short");
- exit(EXIT_FAILURE);
- }
+ if (n < ENCRYPTION_KEY_CHARS)
+ {
+ pg_log_error("The key is too short");
+ exit(EXIT_FAILURE);
+ }
- for (i = 0; i < ENCRYPTION_KEY_LENGTH; i++)
+ for (i = 0; i < ENCRYPTION_KEY_LENGTH; i++)
+ {
+ if (sscanf(key_chars + 2 * i, "%2hhx", encryption_key + i) == 0)
+ {
+ pg_log_error("Invalid character in encryption key at position %d",
+ 2 * i);
+ exit(EXIT_FAILURE);
+ }
+ }
+ }
+ else
{
- if (sscanf(key_chars + 2 * i, "%2hhx", encryption_key + i) == 0)
+ if (n < ENCRYPTION_PWD_MIN_LENGTH)
{
- pg_log_error("Invalid character in encryption key at position %d",
- 2 * i);
+ pg_log_error("The password is too short");
exit(EXIT_FAILURE);
}
+
+ /* Read the KDF parameters. */
+ read_kdf_file(DataDir);
+
+ /* Run the KDF. */
+ derive_key_from_password(encryption_key, password, n);
}
/*
diff --git a/src/fe_utils/encryption.c b/src/fe_utils/encryption.c
index ca37c9f373..134b3bde9b 100644
--- a/src/fe_utils/encryption.c
+++ b/src/fe_utils/encryption.c
@@ -21,31 +21,294 @@
#include "common/file_perm.h"
#include "common/logging.h"
#include "fe_utils/encryption.h"
-#include "storage/encryption.h"
#include "libpq-fe.h"
#include "libpq-int.h"
#include "libpq/pqcomm.h"
char *encryption_key_command = NULL;
+#define KDF_PARAMS_FILE "global/kdf_params"
+#define KDF_PARAMS_FILE_SIZE 512
+
+/*
+ * Key derivation function.
+ */
+typedef enum KDFKind
+{
+ KDF_OPENSSL_PKCS5_PBKDF2_HMAC_SHA = 0
+} KFDKind;
+
+typedef struct KDFParamsPBKDF2
+{
+ unsigned long int niter;
+ unsigned char salt[ENCRYPTION_KDF_SALT_LEN];
+} KDFParamsPBKDF2;
+
+/*
+ * Parameters of the key derivation function.
+ *
+ * The parameters are generated by initdb and stored into a file, which is
+ * then read during PG startup. This is similar to storing various settings in
+ * pg_control. However an existing KDF file is read only, so it does not have
+ * to be stored in shared memory.
+ */
+typedef struct KDFParamsData
+{
+ KFDKind function;
+
+ /*
+ * Function-specific parameters.
+ */
+ union
+ {
+ KDFParamsPBKDF2 pbkdf2;
+ } data;
+
+ /* CRC of all above ... MUST BE LAST! */
+ pg_crc32c crc;
+} KDFParamsData;
+
+extern KDFParamsData *KDFParams;
+
+/*
+ * Pointer to the KDF parameters.
+ */
+KDFParamsData *KDFParams = NULL;
+
+/* Initialize KDF file. */
+void
+init_kdf(void)
+{
+ KDFParamsPBKDF2 *params;
+ struct timeval tv;
+ uint64 salt;
+
+ /*
+ * The initialization should not be repeated.
+ */
+ Assert(KDFParams == NULL);
+
+ KDFParams = palloc0(KDF_PARAMS_FILE_SIZE);
+ KDFParams->function = KDF_OPENSSL_PKCS5_PBKDF2_HMAC_SHA;
+ params = &KDFParams->data.pbkdf2;
+
+ /*
+ * Currently we derive the salt in the same way as system identifier,
+ * however these two values are not supposed to match. XXX Is it worth the
+ * effort if initdb derives the system identifier, passes it to this
+ * function and also sends it to the bootstrap process? Not sure.
+ */
+ gettimeofday(&tv, NULL);
+ salt = ((uint64) tv.tv_sec) << 32;
+ salt |= ((uint64) tv.tv_usec) << 12;
+ salt |= getpid() & 0xFFF;
+
+ memcpy(params->salt, &salt, sizeof(uint64));
+ params->niter = ENCRYPTION_KDF_NITER;
+}
+
+/*
+ * Write KDFParamsData to file.
+ */
+void
+write_kdf_file(char *dir)
+{
+ char path[MAXPGPATH];
+ int fd;
+
+ Assert(KDFParams != NULL);
+
+ /* Account for both file separator and terminating NULL character. */
+ if ((strlen(dir) + 1 + strlen(KDF_PARAMS_FILE) + 1) > MAXPGPATH)
+ {
+ pg_log_fatal("KDF directory is too long");
+ exit(EXIT_FAILURE);
+ }
+
+ snprintf(path, MAXPGPATH, "%s/%s", dir, KDF_PARAMS_FILE);
+
+ /* Contents are protected with a CRC */
+ INIT_CRC32C(KDFParams->crc);
+ COMP_CRC32C(KDFParams->crc,
+ (char *) KDFParams,
+ offsetof(KDFParamsData, crc));
+ FIN_CRC32C(KDFParams->crc);
+
+ fd = open(path, O_WRONLY | O_CREAT | PG_BINARY,
+ pg_file_create_mode);
+ if (fd < 0)
+ {
+ pg_log_fatal("could not create key derivation file \"%s\": %m", path);
+ exit(EXIT_FAILURE);
+ }
+
+ if (write(fd, KDFParams, KDF_PARAMS_FILE_SIZE) != KDF_PARAMS_FILE_SIZE)
+ {
+ /* if write didn't set errno, assume problem is no disk space */
+ if (errno == 0)
+ errno = ENOSPC;
+ pg_log_fatal("could not write to key derivation file \"%s\": %m",
+ path);
+ exit(EXIT_FAILURE);
+ }
+
+ if (close(fd))
+ {
+ pg_log_fatal("could not close key setup file: %m");
+ exit(EXIT_FAILURE);
+ }
+}
+
+/*
+ * Read KDFParamsData from file and store it in local memory.
+ *
+ * If dir is NULL, assume we're in the data directory.
+ *
+ * postmaster should call the function early enough for any other process to
+ * inherit valid pointer to the data.
+ */
+void
+read_kdf_file(char *dir)
+{
+ pg_crc32c crc;
+ char path[MAXPGPATH];
+ int fd;
+
+ /* Account for both file separator and terminating NULL character. */
+ if ((strlen(dir) + 1 + strlen(KDF_PARAMS_FILE) + 1) > MAXPGPATH)
+ {
+ pg_log_fatal("KDF directory is too long");
+ exit(EXIT_FAILURE);
+ }
+
+ snprintf(path, MAXPGPATH, "%s/%s", dir, KDF_PARAMS_FILE);
+
+ KDFParams = palloc0(KDF_PARAMS_FILE_SIZE);
+ fd = open(path, O_RDONLY | PG_BINARY, S_IRUSR);
+
+ if (fd < 0)
+ {
+ pg_log_fatal("could not open key setup file \"%s\": %m", path);
+ exit(EXIT_FAILURE);
+ }
+
+ if (read(fd, KDFParams, sizeof(KDFParamsData)) != sizeof(KDFParamsData))
+ {
+ pg_log_fatal("could not read from key setup file \"%s\": %m", path);
+ exit(EXIT_FAILURE);
+ }
+
+ close(fd);
+
+ /* Now check the CRC. */
+ INIT_CRC32C(crc);
+ COMP_CRC32C(crc,
+ (char *) KDFParams,
+ offsetof(KDFParamsData, crc));
+ FIN_CRC32C(crc);
+
+ if (!EQ_CRC32C(crc, KDFParams->crc))
+ {
+ pg_log_fatal("incorrect checksum in key setup file \"%s\"", path);
+ exit(EXIT_FAILURE);
+ }
+
+
+ if (KDFParams->function != KDF_OPENSSL_PKCS5_PBKDF2_HMAC_SHA)
+ {
+ pg_log_fatal("unsupported KDF function");
+ exit(EXIT_FAILURE);
+ }
+}
+
+/*
+ * Run the key derivation function and initialize encryption_key variable.
+ */
+void
+derive_key_from_password(unsigned char *encryption_key, const char *password,
+ int len)
+{
+ KDFParamsPBKDF2 *params;
+ int rc;
+
+ params = &KDFParams->data.pbkdf2;
+ rc = PKCS5_PBKDF2_HMAC(password,
+ len,
+ params->salt,
+ ENCRYPTION_KDF_SALT_LEN,
+ params->niter,
+ EVP_sha1(),
+ ENCRYPTION_KEY_LENGTH,
+ encryption_key);
+
+ if (rc != 1)
+ {
+ pg_log_fatal("failed to derive key from password");
+ exit(EXIT_FAILURE);
+ }
+}
+
/*
* Run the command that is supposed to generate encryption key and store it
- * where encryption_key points to.
+ * where encryption_key points to. If valid string is passed for data_dir,
+ * it's used to replace '%D' pattern in the command.
*/
void
-run_encryption_key_command(unsigned char *encryption_key)
+run_encryption_key_command(unsigned char *encryption_key, char *data_dir)
{
FILE *fp;
+ char cmd[MAXPGPATH];
+ char *sp, *dp, *endp;
char *buf;
int read_len, i, c;
Assert(encryption_key_command != NULL &&
strlen(encryption_key_command) > 0);
- fp = popen(encryption_key_command, "r");
+ /*
+ * Replace %D pattern in the command with the actual data directory path.
+ */
+ dp = cmd;
+ endp = cmd + MAXPGPATH - 1;
+ *endp = '\0';
+ for (sp = encryption_key_command; *sp; sp++)
+ {
+ if (*sp == '%')
+ {
+ if (sp[1] == 'D')
+ {
+ if (data_dir == NULL)
+ {
+ pg_log_fatal("data directory is not known, %%D pattern cannot be replaced");
+ exit(EXIT_FAILURE);
+ }
+
+ sp++;
+ strlcpy(dp, data_dir, endp - dp);
+ make_native_path(dp);
+ dp += strlen(dp);
+ }
+ else if (dp < endp)
+ *dp++ = *sp;
+ else
+ break;
+ }
+ else
+ {
+ if (dp < endp)
+ *dp++ = *sp;
+ else
+ break;
+ }
+ }
+ *dp = '\0';
+
+ pg_log_debug("executing encryption key command \"%s\"", cmd);
+
+ fp = popen(cmd, "r");
if (fp == NULL)
{
- pg_log_fatal("Failed to execute \"%s\"", encryption_key_command);
+ pg_log_fatal("Failed to execute \"%s\"", cmd);
exit(EXIT_FAILURE);
}
diff --git a/src/include/fe_utils/encryption.h b/src/include/fe_utils/encryption.h
index 7307557ed6..da302fe494 100644
--- a/src/include/fe_utils/encryption.h
+++ b/src/include/fe_utils/encryption.h
@@ -15,6 +15,12 @@
/* Executable to retrieve the encryption key. */
extern char *encryption_key_command;
-extern void run_encryption_key_command(unsigned char *encryption_key);
+extern void init_kdf(void);
+extern void write_kdf_file(char *dir);
+extern void read_kdf_file(char *dir);
+extern void derive_key_from_password(unsigned char *encryption_key,
+ const char *password, int len);
+extern void run_encryption_key_command(unsigned char *encryption_key,
+ char *data_dir);
extern bool send_key_to_postmaster(const char *host, const char *port,
const unsigned char *encryption_Key);
diff --git a/src/include/storage/encryption.h b/src/include/storage/encryption.h
index 4f7b96d4f3..72bcae0972 100644
--- a/src/include/storage/encryption.h
+++ b/src/include/storage/encryption.h
@@ -57,6 +57,14 @@ typedef enum CipherKind
PG_CIPHER_AES_BLOCK_CBC_256_STREAM_CTR_256
} CipherKind;
+/*
+ * TODO Tune these values.
+ */
+#define ENCRYPTION_PWD_MIN_LENGTH 8
+#define ENCRYPTION_PWD_MAX_LENGTH 16
+#define ENCRYPTION_KDF_NITER 1048576
+#define ENCRYPTION_KDF_SALT_LEN sizeof(uint64)
+
/* Key to encrypt / decrypt data. */
extern unsigned char encryption_key[];
--
2.13.7
v04-0007-Refactoring-patch.-This-only-moves-some-code-to-XLog.patch (text/x-diff, 3.2K) download
From 2ec11d17adb6f19216a50998728d6a39c38f53a9 Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Fri, 5 Jul 2019 16:24:01 +0200
Subject: [PATCH 07/17] Refactoring patch. This only moves some code to
XLogWritePages() function because it needs to be used in a little bit
different way for encrypted XLOG. Done in a separate diff so that the
following diff is a bit easier to read.
---
src/backend/access/transam/xlog.c | 67 ++++++++++++++++++++++++---------------
1 file changed, 41 insertions(+), 26 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 6b58e0ac76..cce7eca641 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -875,6 +875,7 @@ static void LocalSetXLogInsertAllowed(void);
static void CreateEndOfRecoveryRecord(void);
static void CheckPointGuts(XLogRecPtr checkPointRedo, int flags);
static void KeepLogSeg(XLogRecPtr recptr, XLogSegNo *logSegNo);
+static Size XLogWritePages(char *from, int npages, uint32 startoffset);
static XLogRecPtr XLogGetReplicationSlotMinimumLSN(void);
static void AdvanceXLInsertBuffer(XLogRecPtr upto, bool opportunistic);
@@ -2485,35 +2486,10 @@ XLogWrite(XLogwrtRqst WriteRqst, bool flexible)
finishing_seg)
{
char *from;
- Size nbytes;
- Size nleft;
- int written;
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
- nbytes = npages * (Size) XLOG_BLCKSZ;
- nleft = nbytes;
- do
- {
- errno = 0;
- pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
- written = pg_pwrite(openLogFile, from, nleft, startoffset);
- pgstat_report_wait_end();
- if (written <= 0)
- {
- if (errno == EINTR)
- continue;
- ereport(PANIC,
- (errcode_for_file_access(),
- errmsg("could not write to log file %s "
- "at offset %u, length %zu: %m",
- XLogFileNameP(ThisTimeLineID, openLogSegNo),
- startoffset, nleft)));
- }
- nleft -= written;
- from += written;
- startoffset += written;
- } while (nleft > 0);
+ startoffset += XLogWritePages(from, npages, startoffset);
npages = 0;
@@ -2630,6 +2606,45 @@ XLogWrite(XLogwrtRqst WriteRqst, bool flexible)
}
/*
+ * Write page(s) to the XLOG file.
+ *
+ * Returns the number of bytes written.
+ */
+static Size
+XLogWritePages(char *from, int npages, uint32 startoffset)
+{
+ Size nbytes,
+ nleft;
+ Size written;
+
+ nbytes = npages * (Size) XLOG_BLCKSZ;
+ nleft = nbytes;
+ do
+ {
+ errno = 0;
+ pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
+ written = pg_pwrite(openLogFile, from, nleft, startoffset);
+ pgstat_report_wait_end();
+ if (written <= 0)
+ {
+ if (errno == EINTR)
+ continue;
+ ereport(PANIC,
+ (errcode_for_file_access(),
+ errmsg("could not write to log file %s "
+ "at offset %u, length %zu: %m",
+ XLogFileNameP(ThisTimeLineID, openLogSegNo),
+ startoffset, nbytes)));
+ }
+ nleft -= written;
+ from += written;
+ startoffset += written;
+ } while (nleft > 0);
+
+ return written;
+}
+
+/*
* Record the LSN for an asynchronous transaction commit/abort
* and nudge the WALWriter if there is work for it to do.
* (This should not be called for synchronous commits.)
--
2.13.7
v04-0008-Enable-XLOG-encryption.patch (text/x-diff, 21.5K) download
From 7947b5452c58a4743586180bbde489c35887b253 Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Fri, 5 Jul 2019 16:24:01 +0200
Subject: [PATCH 08/17] Enable XLOG encryption.
---
src/backend/access/transam/xlog.c | 90 +++++++++++++++++-
src/backend/access/transam/xlogutils.c | 58 +++++++++++-
src/backend/replication/walsender.c | 164 ++++++++++++++++++++++++---------
src/backend/storage/file/encryption.c | 80 ++++++++++++++++
src/backend/utils/misc/guc.c | 16 +++-
src/include/access/xlog.h | 16 +++-
src/include/storage/encryption.h | 17 +++-
7 files changed, 392 insertions(+), 49 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index cce7eca641..2f026f274e 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -2489,7 +2489,61 @@ XLogWrite(XLogwrtRqst WriteRqst, bool flexible)
/* OK to write the page(s) */
from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
- startoffset += XLogWritePages(from, npages, startoffset);
+ if (data_encrypted)
+ {
+ int i,
+ nencrypted;
+ char *to;
+ uint32 encr_offset;
+
+ /*
+ * Encrypt and write multiple pages at a time, in order to
+ * reduce the number of syscalls.
+ */
+ nencrypted = 0;
+ to = encrypt_buf_xlog;
+ encr_offset = startoffset;
+ for (i = 1; i <= npages; i++)
+ {
+ char tweak[TWEAK_SIZE];
+ Size nbytes;
+
+ XLogEncryptionTweak(tweak, ThisTimeLineID, openLogSegNo, encr_offset);
+
+ /*
+ * We should not encrypt the unused space, in order to
+ * avoid "reused key attack".
+ */
+ if (i == npages && ispartialpage)
+ nbytes = WriteRqst.Write % XLOG_BLCKSZ;
+ else
+ nbytes = XLOG_BLCKSZ;
+
+ encrypt_block(from, to, nbytes, tweak, true);
+ nencrypted++;
+ from += XLOG_BLCKSZ;
+ to += XLOG_BLCKSZ;
+ encr_offset += XLOG_BLCKSZ;
+
+ /*
+ * Write the encrypted data if the encryption buffer is
+ * full or if the last page has been encrypted.
+ */
+ if (nencrypted >= XLOG_ENCRYPT_BUF_PAGES || i >= npages)
+ {
+ startoffset += XLogWritePages(encrypt_buf_xlog,
+ nencrypted,
+ startoffset);
+
+ /* Prepare for the next round of page encryptions. */
+ nencrypted = 0;
+ to = encrypt_buf_xlog;
+ encr_offset = startoffset;
+ }
+ }
+ }
+ else
+ startoffset += XLogWritePages(from, npages, startoffset);
npages = 0;
@@ -3479,6 +3533,24 @@ XLogFileCopy(XLogSegNo destsegno, TimeLineID srcTLI, XLogSegNo srcsegno,
}
pgstat_report_wait_end();
}
+
+ /*
+ * Since timeline is being changed and since encryption tweak contains
+ * the timeline, we need to decrypt the buffer and encrypt it with the
+ * new tweak. Do not encrypt the unused space, in order to avoid
+ * "reused key attack".
+ */
+ if (data_encrypted && nread > 0)
+ {
+ char tweak[TWEAK_SIZE];
+
+ XLogEncryptionTweak(tweak, srcTLI, srcsegno, nbytes);
+ decrypt_block(buffer.data, buffer.data, nread, tweak, true);
+
+ XLogEncryptionTweak(tweak, ThisTimeLineID, destsegno, nbytes);
+ encrypt_block(buffer.data, buffer.data, nread, tweak, true);
+ }
+
errno = 0;
pgstat_report_wait_start(WAIT_EVENT_WAL_COPY_WRITE);
if ((int) write(fd, buffer.data, sizeof(buffer)) != (int) sizeof(buffer))
@@ -5273,6 +5345,14 @@ BootStrapXLOG(void)
use_existent = false;
openLogFile = XLogFileInit(1, &use_existent, false);
+ if (data_encrypted)
+ {
+ char tweak[TWEAK_SIZE];
+
+ XLogEncryptionTweak(tweak, ThisTimeLineID, 1, 0);
+ encrypt_block((char *) page, (char *) page, XLOG_BLCKSZ, tweak, true);
+ }
+
/* Write the first page with the initial record */
errno = 0;
pgstat_report_wait_start(WAIT_EVENT_WAL_BOOTSTRAP_WRITE);
@@ -11724,6 +11804,14 @@ retry:
Assert(targetPageOff == readOff);
Assert(reqLen <= readLen);
+ if (data_encrypted)
+ {
+ char tweak[TWEAK_SIZE];
+
+ XLogEncryptionTweak(tweak, curFileTLI, readSegNo, readOff);
+ decrypt_block(readBuf, readBuf, XLOG_BLCKSZ, tweak, true);
+ }
+
*readTLI = curFileTLI;
/*
diff --git a/src/backend/access/transam/xlogutils.c b/src/backend/access/transam/xlogutils.c
index 10a663bae6..25139e7a86 100644
--- a/src/backend/access/transam/xlogutils.c
+++ b/src/backend/access/transam/xlogutils.c
@@ -25,6 +25,7 @@
#include "access/xlogutils.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "storage/encryption.h"
#include "storage/smgr.h"
#include "utils/guc.h"
#include "utils/hsearch.h"
@@ -656,9 +657,11 @@ static void
XLogRead(char *buf, int segsize, TimeLineID tli, XLogRecPtr startptr,
Size count)
{
- char *p;
XLogRecPtr recptr;
Size nbytes;
+ char *decrypt_p;
+ uint32 decryptOff;
+ char *p;
/* state maintained across calls */
static int sendFile = -1;
@@ -668,17 +671,46 @@ XLogRead(char *buf, int segsize, TimeLineID tli, XLogRecPtr startptr,
Assert(segsize == wal_segment_size);
+ /*
+ * TODO Currently the function is only called with startptr at page
+ * boundary. walsender.c:XLogRead() is more generic but it's a bit
+ * different, so decryption also needs to be adopted in a little different
+ * way (i.e. the decryption cannot be merged here mechanically). Manual
+ * merge is not worth the effort right now because a separate patch to
+ * consolidate XLogRead() functions is in the queue
+ * (https://commitfest.postgresql.org/23/2098/). Let's implement the
+ * decryption in a generic way when that patch is committed.
+ */
+ Assert(startptr % XLOG_BLCKSZ == 0);
+
p = buf;
recptr = startptr;
nbytes = count;
+ decrypt_p = p;
+ decryptOff = XLogSegmentOffset(recptr, segsize);
+
while (nbytes > 0)
{
uint32 startoff;
int segbytes;
int readbytes;
- startoff = XLogSegmentOffset(recptr, segsize);
+ if (recptr == startptr)
+ startoff = decryptOff;
+ else
+ {
+ startoff = XLogSegmentOffset(recptr, segsize);
+ if (startoff == 0)
+ {
+ /*
+ * If segment boundary was reached, decryptOff should have
+ * caught up, so we can (and should) sync it with startoff.
+ */
+ Assert(decryptOff == segsize);
+ decryptOff = startoff;
+ }
+ }
/* Do we need to switch to a different xlog segment? */
if (sendFile < 0 || !XLByteInSeg(recptr, sendSegNo, segsize) ||
@@ -758,7 +790,29 @@ XLogRead(char *buf, int segsize, TimeLineID tli, XLogRecPtr startptr,
sendOff += readbytes;
nbytes -= readbytes;
p += readbytes;
+
+ /* Decrypt completed blocks */
+ if (data_encrypted)
+ {
+ while (decrypt_p + XLOG_BLCKSZ <= p)
+ {
+ char tweak[TWEAK_SIZE];
+
+ XLogEncryptionTweak(tweak, tli, sendSegNo, decryptOff);
+ decrypt_block(decrypt_p, decrypt_p, XLOG_BLCKSZ, tweak,
+ true);
+
+ decrypt_p += XLOG_BLCKSZ;
+ decryptOff += XLOG_BLCKSZ;
+ }
+ }
}
+
+ /*
+ * TODO Currently the function is only called with count==XLOG_BLCKSZ. See
+ * the comment on consolidation of XLogRead() above.
+ */
+ Assert(decrypt_p == p);
}
/*
diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c
index 92fa86fc9d..ae9c1418d1 100644
--- a/src/backend/replication/walsender.c
+++ b/src/backend/replication/walsender.c
@@ -77,6 +77,7 @@
#include "replication/walsender.h"
#include "replication/walsender_private.h"
#include "storage/condition_variable.h"
+#include "storage/encryption.h"
#include "storage/fd.h"
#include "storage/ipc.h"
#include "storage/pmsignal.h"
@@ -256,7 +257,9 @@ static void LagTrackerWrite(XLogRecPtr lsn, TimestampTz local_flush_time);
static TimeOffset LagTrackerRead(int head, XLogRecPtr lsn, TimestampTz now);
static bool TransactionIdInRecentPast(TransactionId xid, uint32 epoch);
-static void XLogRead(char *buf, XLogRecPtr startptr, Size count);
+static int XLogReadBuffer(char *buf, int nbytes, int startoff);
+static void XLogRead(char *buf, XLogRecPtr startptr, Size count,
+ bool decrypt);
/* Initialize walsender process before entering the main command loop */
@@ -787,7 +790,7 @@ logical_read_xlog_page(XLogReaderState *state, XLogRecPtr targetPagePtr, int req
count = flushptr - targetPagePtr; /* part of the page available */
/* now actually read the data, we know it's there */
- XLogRead(cur_page, targetPagePtr, XLOG_BLCKSZ);
+ XLogRead(cur_page, targetPagePtr, XLOG_BLCKSZ, true);
return count;
}
@@ -2352,19 +2355,53 @@ WalSndKill(int code, Datum arg)
SpinLockRelease(&walsnd->mutex);
}
+static int
+XLogReadBuffer(char *buf, int nbytes, int startoff)
+{
+ int readbytes;
+
+ /* Need to seek in the file? */
+ if (sendOff != startoff)
+ {
+ if (lseek(sendFile, (off_t) startoff, SEEK_SET) < 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not seek in log segment %s to offset %u: %m",
+ XLogFileNameP(curFileTimeLine, sendSegNo),
+ startoff)));
+ sendOff = startoff;
+ }
+
+ pgstat_report_wait_start(WAIT_EVENT_WAL_READ);
+ readbytes = read(sendFile, buf, nbytes);
+ pgstat_report_wait_end();
+ if (readbytes <= 0)
+ {
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from log segment %s, offset %u, length %lu: %m",
+ XLogFileNameP(curFileTimeLine, sendSegNo),
+ sendOff, (unsigned long) nbytes)));
+ }
+
+ return readbytes;
+}
+
/*
- * Read 'count' bytes from WAL into 'buf', starting at location 'startptr'
+ * Read 'count' bytes from WAL into 'buf', starting at location
+ * 'startptr'. Decrypt the data if it's encrypted and if caller wants it
+ * decrypted.
*
- * XXX probably this should be improved to suck data directly from the
- * WAL buffers when possible.
+ * XXX probably this should be improved to fetch data directly from the WAL
+ * buffers when possible.
*
* Will open, and keep open, one WAL segment stored in the global file
- * descriptor sendFile. This means if XLogRead is used once, there will
- * always be one descriptor left open until the process ends, but never
- * more than one.
+ * descriptor sendFile. This means if XLogRead is used once, there will always
+ * be one descriptor left open until the process ends, but never more than
+ * one.
*/
static void
-XLogRead(char *buf, XLogRecPtr startptr, Size count)
+XLogRead(char *buf, XLogRecPtr startptr, Size count, bool decrypt)
{
char *p;
XLogRecPtr recptr;
@@ -2454,42 +2491,87 @@ retry:
sendOff = 0;
}
- /* Need to seek in the file? */
- if (sendOff != startoff)
+ /* Caller should not request decryption of unencrypted data. */
+ Assert(!(decrypt && !data_encrypted));
+
+ if (data_encrypted && decrypt)
{
- if (lseek(sendFile, (off_t) startoff, SEEK_SET) < 0)
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not seek in log segment %s to offset %u: %m",
- XLogFileNameP(curFileTimeLine, sendSegNo),
- startoff)));
- sendOff = startoff;
- }
+ int pageoff = startoff % XLOG_BLCKSZ;
+ uint32 pagebase = startoff - pageoff;
+ int bufbytes,
+ bufend,
+ i;
+ char tweak[TWEAK_SIZE];
- /* How many bytes are within this segment? */
- if (nbytes > (wal_segment_size - startoff))
- segbytes = wal_segment_size - startoff;
- else
- segbytes = nbytes;
+ /*
+ * Only accept as much data as what can fit into the buffer.
+ */
+ if (nbytes > (ENCRYPT_BUF_XLOG_SIZE - pageoff))
+ bufbytes = ENCRYPT_BUF_XLOG_SIZE - pageoff;
+ else
+ bufbytes = nbytes;
+ bufend = pageoff + bufbytes;
- pgstat_report_wait_start(WAIT_EVENT_WAL_READ);
- readbytes = read(sendFile, p, segbytes);
- pgstat_report_wait_end();
- if (readbytes < 0)
- {
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not read from log segment %s, offset %u, length %zu: %m",
- XLogFileNameP(curFileTimeLine, sendSegNo),
- sendOff, (Size) segbytes)));
+ /*
+ * Read the data, including the leading part of the page which
+ * caller is not interested in. The tweak we passed to
+ * encrypt_block() for encryption was for the beginning of the
+ * block, so it'd be hard to start decryption anywhere else.
+ */
+ readbytes = 0;
+ while (readbytes < bufend)
+ readbytes += XLogReadBuffer(encrypt_buf_xlog + readbytes,
+ bufend - readbytes,
+ pagebase + readbytes);
+
+ /*
+ * Decrypt the data one page at a time (the tweak is only valid
+ * for particular page).
+ */
+ for (i = 0; i < readbytes; i += XLOG_BLCKSZ)
+ {
+ Size nencrypt;
+
+ XLogEncryptionTweak(tweak,
+ curFileTimeLine,
+ sendSegNo,
+ pagebase + i);
+
+ /*
+ * If the last page is not complete, only decrypt the used
+ * part.
+ */
+ if ((bufend - i) < XLOG_BLCKSZ)
+ nencrypt = bufend - i;
+ else
+ nencrypt = XLOG_BLCKSZ;
+
+ decrypt_block(encrypt_buf_xlog + i,
+ encrypt_buf_xlog + i,
+ nencrypt,
+ tweak,
+ true);
+ }
+
+ /*
+ * Caller does not care that we possibly had to read pageoff bytes
+ * in addition (because we cannot decrypt trailing part of the
+ * page alone). This overhead must not affect the accounting.
+ */
+ readbytes = bufbytes;
+
+ /* Copy the data to the output buffer. */
+ memcpy(p, encrypt_buf_xlog + pageoff, bufbytes);
}
- else if (readbytes == 0)
+ else
{
- ereport(ERROR,
- (errcode(ERRCODE_DATA_CORRUPTED),
- errmsg("could not read from log segment %s, offset %u: read %d of %zu",
- XLogFileNameP(curFileTimeLine, sendSegNo),
- sendOff, readbytes, (Size) segbytes)));
+ /* How many bytes are within this segment? */
+ if (nbytes > (wal_segment_size - startoff))
+ segbytes = wal_segment_size - startoff;
+ else
+ segbytes = nbytes;
+
+ readbytes = XLogReadBuffer(p, segbytes, startoff);
}
/* Update state for read */
@@ -2768,7 +2850,7 @@ XLogSendPhysical(void)
* calls.
*/
enlargeStringInfo(&output_message, nbytes);
- XLogRead(&output_message.data[output_message.len], startptr, nbytes);
+ XLogRead(&output_message.data[output_message.len], startptr, nbytes, false);
output_message.len += nbytes;
output_message.data[output_message.len] = '\0';
diff --git a/src/backend/storage/file/encryption.c b/src/backend/storage/file/encryption.c
index 1eda334762..971b6928e4 100644
--- a/src/backend/storage/file/encryption.c
+++ b/src/backend/storage/file/encryption.c
@@ -63,6 +63,8 @@ char encryption_verification[ENCRYPTION_SAMPLE_SIZE];
bool encryption_setup_done = false;
+char *encrypt_buf_xlog = NULL;
+
#ifdef USE_ENCRYPTION
static void init_encryption_context(EVP_CIPHER_CTX **ctx_p, bool stream);
static void evp_error(void);
@@ -165,6 +167,21 @@ setup_encryption(void)
init_encryption_context(&ctx_encrypt_stream, true);
init_encryption_context(&ctx_decrypt_stream, true);
+ /*
+ * We need multiple pages here, so allocate the memory dynamically instead
+ * of using PGAlignedBlock. That also ensures it'll be MAXALIGNed, which
+ * is useful because the buffer will be used for I/O.
+ *
+ * Use TopMemoryContext because on server side this code is run by
+ * postmaster and postmaster context gets freed after fork().
+ */
+#ifndef FRONTEND
+ encrypt_buf_xlog = (char *) MemoryContextAlloc(TopMemoryContext,
+ ENCRYPT_BUF_XLOG_SIZE);
+#else
+ encrypt_buf_xlog = (char *) palloc(ENCRYPT_BUF_XLOG_SIZE);
+#endif
+
encryption_setup_done = true;
#else /* !USE_ENCRYPTION */
#ifndef FRONTEND
@@ -264,6 +281,50 @@ encrypt_block(const char *input, char *output, Size size, char *tweak,
#endif /* USE_ENCRYPTION */
}
+/*
+ * Decrypts one block of data with a specified tweak value. May only be called
+ * when encryption_enabled is true.
+ *
+ * Input and output buffer may point to the same location.
+ *
+ * For detailed comments see encrypt_block().
+ */
+void
+decrypt_block(const char *input, char *output, Size size, char *tweak,
+ bool stream)
+{
+#ifdef USE_ENCRYPTION
+ int out_size;
+ EVP_CIPHER_CTX *ctx;
+
+ Assert(data_encrypted);
+ Assert((size >= ENCRYPTION_BLOCK && size % ENCRYPTION_BLOCK == 0) ||
+ stream);
+
+ if (!stream && IsAllZero(input, size))
+ {
+ memset(output, 0, size);
+ return;
+ }
+
+ ctx = !stream ? ctx_decrypt : ctx_decrypt_stream;
+
+ /* The remaining initialization. */
+ if (EVP_DecryptInit_ex(ctx, NULL, NULL, encryption_key,
+ (unsigned char *) tweak) != 1)
+ evp_error();
+
+ /* Do the actual encryption. */
+ if (EVP_DecryptUpdate(ctx, (unsigned char *) output,
+ &out_size, (unsigned char *) input, size) != 1)
+ evp_error();
+
+ Assert(out_size == size);
+#else
+ /* data_encrypted should not be set */
+ Assert(false);
+#endif /* USE_ENCRYPTION */
+}
#ifdef USE_ENCRYPTION
/*
@@ -345,3 +406,22 @@ evp_error(void)
#endif /* FRONTEND */
}
#endif /* USE_ENCRYPTION */
+
+/*
+ * Xlog is encrypted page at a time. Each xlog page gets a unique tweak via
+ * timeline, segment and offset.
+ *
+ * The function is located here rather than some of the xlog*.c modules so
+ * that front-end applications can easily use it too.
+ */
+void
+XLogEncryptionTweak(char *tweak, TimeLineID timeline, XLogSegNo segment,
+ uint32 offset)
+{
+ memset(tweak, 0, TWEAK_SIZE);
+ memcpy(tweak, &timeline, sizeof(timeline));
+ tweak += sizeof(timeline);
+ memcpy(tweak, &segment, sizeof(XLogSegNo));
+ tweak += sizeof(XLogSegNo);
+ memcpy(tweak, &offset, sizeof(offset));
+}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 2b7fc2fb0f..440120635e 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -173,6 +173,7 @@ static void assign_session_replication_role(int newval, void *extra);
static bool check_temp_buffers(int *newval, void **extra, GucSource source);
static bool check_bonjour(bool *newval, void **extra, GucSource source);
static bool check_ssl(bool *newval, void **extra, GucSource source);
+static bool check_full_page_writes(bool *newval, void **extra, GucSource source);
static bool check_stage_log_stats(bool *newval, void **extra, GucSource source);
static bool check_log_stats(bool *newval, void **extra, GucSource source);
static bool check_canonical_path(char **newval, void **extra, GucSource source);
@@ -1173,7 +1174,7 @@ static struct config_bool ConfigureNamesBool[] =
},
&fullPageWrites,
true,
- NULL, NULL, NULL
+ check_full_page_writes, NULL, NULL
},
{
@@ -11105,6 +11106,19 @@ check_ssl(bool *newval, void **extra, GucSource source)
}
static bool
+check_full_page_writes(bool *newval, void **extra, GucSource source)
+{
+ if (!(*newval) && data_encrypted)
+ {
+ GUC_check_errdetail("Cannot disable parameter when the cluster is encrypted.");
+
+ return false;
+ }
+
+ return true;
+}
+
+static bool
check_stage_log_stats(bool *newval, void **extra, GucSource source)
{
if (*newval && log_statement_stats)
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index d519252aad..2ad441623d 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -19,6 +19,7 @@
#include "lib/stringinfo.h"
#include "nodes/pg_list.h"
#include "storage/fd.h"
+#include "storage/encryption.h"
/* Sync methods */
@@ -186,10 +187,19 @@ extern PGDLLIMPORT int wal_level;
* Normally, we don't WAL-log hint bit updates, but if checksums are enabled,
* we have to protect them against torn page writes. When you only set
* individual bits on a page, it's still consistent no matter what combination
- * of the bits make it to disk, but the checksum wouldn't match. Also WAL-log
- * them if forced by wal_log_hints=on.
+ * of the bits make it to disk, but the checksum wouldn't match.
+ *
+ * Regardless checksums, if encryption is enabled, hint bit change followed by
+ * a torn page write can result in such a situation that decryption produces
+ * page whose contents (following the hint bit) is garbage. This is because
+ * the block cipher we use propagates changes in the lower addresses of plain
+ * data to higher addresses of the cipher data. We need full-page image also
+ * to recover from this state.
+ *
+ * Also WAL-log the hint bits if forced by wal_log_hints=on.
*/
-#define XLogHintBitIsNeeded() (DataChecksumsEnabled() || wal_log_hints)
+#define XLogHintBitIsNeeded() (DataChecksumsEnabled() || data_encrypted || \
+ wal_log_hints)
/* Do we need to WAL-log information required only for Hot Standby and logical replication? */
#define XLogStandbyInfoActive() (wal_level >= WAL_LEVEL_REPLICA)
diff --git a/src/include/storage/encryption.h b/src/include/storage/encryption.h
index 72bcae0972..c6ad9b43bd 100644
--- a/src/include/storage/encryption.h
+++ b/src/include/storage/encryption.h
@@ -116,6 +116,16 @@ extern char encryption_verification[];
/* Do we have encryption_key and the encryption library initialized? */
extern bool encryption_setup_done;
+/*
+ * XLOG encryption/decryption buffer. This buffer spans multiple pages, in
+ * order to reduce the number of syscalls when doing I/O.
+ *
+ * XXX Fine tune the buffer size.
+ */
+extern char *encrypt_buf_xlog;
+#define XLOG_ENCRYPT_BUF_PAGES 8
+#define ENCRYPT_BUF_XLOG_SIZE (XLOG_ENCRYPT_BUF_PAGES * XLOG_BLCKSZ)
+
#ifndef FRONTEND
extern Size EncryptionShmemSize(void);
extern void EncryptionShmemInit(void);
@@ -127,7 +137,12 @@ extern void read_encryption_key(read_encryption_key_cb read_char);
extern void setup_encryption(void);
extern void sample_encryption(char *buf);
extern void encrypt_block(const char *input, char *output, Size size,
- char *tweak, bool stream);
+ char *tweak, bool stream);
+extern void decrypt_block(const char *input, char *output, Size size,
+ char *tweak, bool stream);
extern void encryption_error(bool fatal, char *message);
+extern void XLogEncryptionTweak(char *tweak, TimeLineID timeline,
+ XLogSegNo segment, uint32 offset);
+
#endif /* ENCRYPTION_H */
--
2.13.7
v04-0009-Enable-encryption-of-relations.patch (text/x-diff, 31.3K) download
From 5868d43af039b86a9b0cc6303646b7782a20f77b Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Fri, 5 Jul 2019 16:24:01 +0200
Subject: [PATCH 09/17] Enable encryption of relations.
---
src/backend/access/heap/rewriteheap.c | 16 ++++++
src/backend/access/nbtree/nbtsort.c | 2 +
src/backend/commands/dbcommands.c | 17 +++++-
src/backend/replication/basebackup.c | 2 +-
src/backend/storage/buffer/bufmgr.c | 15 +++++
src/backend/storage/buffer/localbuf.c | 8 +++
src/backend/storage/file/copydir.c | 99 ++++++++++++++++++++++++++------
src/backend/storage/file/encryption.c | 104 ++++++++++++++++++++++++++++++++++
src/backend/storage/file/reinit.c | 53 ++++++++++++-----
src/backend/storage/page/bufpage.c | 26 +--------
src/backend/storage/smgr/md.c | 41 +++++++++++++-
src/include/storage/copydir.h | 6 +-
src/include/storage/encryption.h | 22 ++++++-
src/include/storage/reinit.h | 3 +-
14 files changed, 350 insertions(+), 64 deletions(-)
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index 369694fa2e..37aba1eabe 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -330,18 +330,27 @@ end_heap_rewrite(RewriteState state)
/* Write the last page, if any */
if (state->rs_buffer_valid)
{
+ bool lsn_is_fake = false;
+
if (state->rs_use_wal)
log_newpage(&state->rs_new_rel->rd_node,
MAIN_FORKNUM,
state->rs_blockno,
state->rs_buffer,
true);
+ else if (data_encrypted)
+ lsn_is_fake = EnforceLSNUpdateForEncryption((char *)
+ state->rs_buffer);
+
RelationOpenSmgr(state->rs_new_rel);
PageSetChecksumInplace(state->rs_buffer, state->rs_blockno);
smgrextend(state->rs_new_rel->rd_smgr, MAIN_FORKNUM, state->rs_blockno,
(char *) state->rs_buffer, true);
+
+ if (lsn_is_fake)
+ RestoreInvalidLSN((char *) state->rs_buffer);
}
/*
@@ -692,6 +701,8 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
if (len + saveFreeSpace > pageFreeSpace)
{
+ bool lsn_is_fake = false;
+
/* Doesn't fit, so write out the existing page */
/* XLOG stuff */
@@ -701,6 +712,8 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
state->rs_blockno,
page,
true);
+ else if (data_encrypted)
+ lsn_is_fake = EnforceLSNUpdateForEncryption((char *) page);
/*
* Now write the page. We say isTemp = true even if it's not a
@@ -715,6 +728,9 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
smgrextend(state->rs_new_rel->rd_smgr, MAIN_FORKNUM,
state->rs_blockno, (char *) page, true);
+ if (lsn_is_fake)
+ RestoreInvalidLSN((char *) page);
+
state->rs_blockno++;
state->rs_buffer_valid = false;
}
diff --git a/src/backend/access/nbtree/nbtsort.c b/src/backend/access/nbtree/nbtsort.c
index d0b9013caf..e451d6054f 100644
--- a/src/backend/access/nbtree/nbtsort.c
+++ b/src/backend/access/nbtree/nbtsort.c
@@ -660,6 +660,8 @@ _bt_blwritepage(BTWriteState *wstate, Page page, BlockNumber blkno)
/* We use the heap NEWPAGE record type for this */
log_newpage(&wstate->index->rd_node, MAIN_FORKNUM, blkno, page, true);
}
+ else if (data_encrypted)
+ EnforceLSNUpdateForEncryption((char *) page);
/*
* If we have to write pages nonsequentially, fill in the space with
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index 863f89f19d..7ea783d096 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -609,6 +609,8 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
Oid dsttablespace;
char *srcpath;
char *dstpath;
+ RelFileNode src_node = {srctablespace, src_dboid, InvalidOid};
+ RelFileNode dst_node;
struct stat st;
/* No need to copy global tablespace */
@@ -630,6 +632,10 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
else
dsttablespace = srctablespace;
+ dst_node.spcNode = dsttablespace;
+ dst_node.dbNode = dboid;
+ dst_node.relNode= InvalidOid;
+
dstpath = GetDatabasePath(dboid, dsttablespace);
/*
@@ -637,7 +643,7 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
*
* We don't need to copy subdirectories
*/
- copydir(srcpath, dstpath, false);
+ copydir(srcpath, dstpath, &src_node, &dst_node);
/* Record the filesystem change in XLOG */
{
@@ -1272,10 +1278,13 @@ movedb(const char *dbname, const char *tblspcname)
PG_ENSURE_ERROR_CLEANUP(movedb_failure_callback,
PointerGetDatum(&fparms));
{
+ RelFileNode src_node = {src_tblspcoid, db_id, InvalidOid};
+ RelFileNode dst_node = {dst_tblspcoid, db_id, InvalidOid};
+
/*
* Copy files from the old tablespace to the new one
*/
- copydir(src_dbpath, dst_dbpath, false);
+ copydir(src_dbpath, dst_dbpath, &src_node, &dst_node);
/*
* Record the filesystem change in XLOG
@@ -2109,6 +2118,8 @@ dbase_redo(XLogReaderState *record)
char *src_path;
char *dst_path;
struct stat st;
+ RelFileNode src_node = {xlrec->src_tablespace_id, xlrec->src_db_id, InvalidOid};
+ RelFileNode dst_node = {xlrec->tablespace_id, xlrec->db_id, InvalidOid};
src_path = GetDatabasePath(xlrec->src_db_id, xlrec->src_tablespace_id);
dst_path = GetDatabasePath(xlrec->db_id, xlrec->tablespace_id);
@@ -2138,7 +2149,7 @@ dbase_redo(XLogReaderState *record)
*
* We don't need to copy subdirectories
*/
- copydir(src_path, dst_path, false);
+ copydir(src_path, dst_path, &src_node, &dst_node);
}
else if (info == XLOG_DBASE_DROP)
{
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index 7a1b38466b..678fdfa5a2 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -1104,7 +1104,7 @@ sendDir(const char *path, int basepathlen, bool sizeonly, List *tablespaces,
/* Exclude all forks for unlogged tables except the init fork */
if (isDbDir &&
parse_filename_for_nontemp_relation(de->d_name, &relOidChars,
- &relForkNum))
+ &relForkNum, NULL))
{
/* Never exclude init forks */
if (relForkNum != INIT_FORKNUM)
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 7332e6b590..a36fb7bea7 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -2678,6 +2678,7 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
Block bufBlock;
char *bufToWrite;
uint32 buf_state;
+ bool lsn_is_fake = false;
/*
* Acquire the buffer's io_in_progress lock. If StartBufferIO returns
@@ -2742,6 +2743,9 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
*/
bufBlock = BufHdrGetBlock(buf);
+ if (data_encrypted && !(buf_state & BM_PERMANENT))
+ lsn_is_fake = EnforceLSNUpdateForEncryption(bufBlock);
+
/*
* Update page checksum if desired. Since we have only shared lock on the
* buffer, other processes might be updating hint bits in it, so we must
@@ -2771,6 +2775,9 @@ FlushBuffer(BufferDesc *buf, SMgrRelation reln)
pgBufferUsage.shared_blks_written++;
+ if (lsn_is_fake)
+ RestoreInvalidLSN(bufBlock);
+
/*
* Mark the buffer as clean (unless BM_JUST_DIRTIED has become set) and
* end the io_in_progress state.
@@ -3209,6 +3216,7 @@ FlushRelationBuffers(Relation rel)
{
ErrorContextCallback errcallback;
Page localpage;
+ bool lsn_is_fake = false;
localpage = (char *) LocalBufHdrGetBlock(bufHdr);
@@ -3218,6 +3226,10 @@ FlushRelationBuffers(Relation rel)
errcallback.previous = error_context_stack;
error_context_stack = &errcallback;
+ if (data_encrypted)
+ lsn_is_fake = EnforceLSNUpdateForEncryption((char *)
+ localpage);
+
PageSetChecksumInplace(localpage, bufHdr->tag.blockNum);
smgrwrite(rel->rd_smgr,
@@ -3226,6 +3238,9 @@ FlushRelationBuffers(Relation rel)
localpage,
false);
+ if (lsn_is_fake)
+ RestoreInvalidLSN((char *) localpage);
+
buf_state &= ~(BM_DIRTY | BM_JUST_DIRTIED);
pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
diff --git a/src/backend/storage/buffer/localbuf.c b/src/backend/storage/buffer/localbuf.c
index 391b6d6e16..cd36ec78a7 100644
--- a/src/backend/storage/buffer/localbuf.c
+++ b/src/backend/storage/buffer/localbuf.c
@@ -20,6 +20,7 @@
#include "executor/instrument.h"
#include "storage/buf_internals.h"
#include "storage/bufmgr.h"
+#include "storage/encryption.h"
#include "utils/guc.h"
#include "utils/memutils.h"
#include "utils/resowner_private.h"
@@ -207,10 +208,14 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum,
{
SMgrRelation oreln;
Page localpage = (char *) LocalBufHdrGetBlock(bufHdr);
+ bool lsn_is_fake = false;
/* Find smgr relation for buffer */
oreln = smgropen(bufHdr->tag.rnode, MyBackendId);
+ if (data_encrypted)
+ lsn_is_fake = EnforceLSNUpdateForEncryption((char *) localpage);
+
PageSetChecksumInplace(localpage, bufHdr->tag.blockNum);
/* And write... */
@@ -220,6 +225,9 @@ LocalBufferAlloc(SMgrRelation smgr, ForkNumber forkNum, BlockNumber blockNum,
localpage,
false);
+ if (lsn_is_fake)
+ RestoreInvalidLSN(localpage);
+
/* Mark not-dirty now in case we error out below */
buf_state &= ~BM_DIRTY;
pg_atomic_unlocked_write_u32(&bufHdr->state, buf_state);
diff --git a/src/backend/storage/file/copydir.c b/src/backend/storage/file/copydir.c
index 30f6200a86..b98e9c10d8 100644
--- a/src/backend/storage/file/copydir.c
+++ b/src/backend/storage/file/copydir.c
@@ -23,24 +23,29 @@
#include <sys/stat.h>
#include "storage/copydir.h"
+#include "storage/encryption.h"
#include "storage/fd.h"
+#include "storage/reinit.h"
#include "miscadmin.h"
#include "pgstat.h"
/*
* copydir: copy a directory
*
- * If recurse is false, subdirectories are ignored. Anything that's not
- * a directory or a regular file is ignored.
+ * RelFileNode values must specify tablespace and database oids for source
+ * and target to support re-encryption if necessary. relNode value in provided
+ * structs will be clobbered.
*/
void
-copydir(char *fromdir, char *todir, bool recurse)
+copydir(char *fromdir, char *todir, RelFileNode *fromNode, RelFileNode *toNode)
{
DIR *xldir;
struct dirent *xlde;
char fromfile[MAXPGPATH * 2];
char tofile[MAXPGPATH * 2];
+ Assert(!data_encrypted || (fromNode != NULL && toNode != NULL));
+
if (MakePGDirectory(todir) != 0)
ereport(ERROR,
(errcode_for_file_access(),
@@ -67,14 +72,32 @@ copydir(char *fromdir, char *todir, bool recurse)
(errcode_for_file_access(),
errmsg("could not stat file \"%s\": %m", fromfile)));
- if (S_ISDIR(fst.st_mode))
+ if (S_ISREG(fst.st_mode))
{
- /* recurse to handle subdirectories */
- if (recurse)
- copydir(fromfile, tofile, true);
+ int oidchars;
+ ForkNumber forkNum;
+ int segment;
+
+ /*
+ * For encrypted databases we need to reencrypt files with new
+ * tweaks.
+ */
+ if (data_encrypted &&
+ parse_filename_for_nontemp_relation(xlde->d_name,
+ &oidchars, &forkNum, &segment))
+ {
+ char oidbuf[OIDCHARS + 1];
+
+ memcpy(oidbuf, xlde->d_name, oidchars);
+ oidbuf[oidchars] = '\0';
+
+ /* We scribble over the provided RelFileNodes here */
+ fromNode->relNode = toNode->relNode = atol(oidbuf);
+ copy_file(fromfile, tofile, fromNode, toNode, forkNum, forkNum, segment);
+ }
+ else
+ copy_file(fromfile, tofile, NULL, NULL, 0, 0, 0);
}
- else if (S_ISREG(fst.st_mode))
- copy_file(fromfile, tofile);
}
FreeDir(xldir);
@@ -121,17 +144,22 @@ copydir(char *fromdir, char *todir, bool recurse)
}
/*
- * copy one file
+ * copy one file. If decryption and reencryption may be needed specify
+ * relfilenodes for source and target.
*/
void
-copy_file(char *fromfile, char *tofile)
+copy_file(char *fromfile, char *tofile, RelFileNode *fromNode,
+ RelFileNode *toNode, ForkNumber fromForkNum, ForkNumber toForkNum,
+ int segment)
{
char *buffer;
int srcfd;
int dstfd;
int nbytes;
+ int bytesread;
off_t offset;
off_t flush_offset;
+ BlockNumber blockNum = segment * RELSEG_SIZE;
/* Size of copy buffer (read and write requests) */
#define COPY_BUF_SIZE (8 * BLCKSZ)
@@ -186,15 +214,50 @@ copy_file(char *fromfile, char *tofile)
flush_offset = offset;
}
- pgstat_report_wait_start(WAIT_EVENT_COPY_FILE_READ);
- nbytes = read(srcfd, buffer, COPY_BUF_SIZE);
- pgstat_report_wait_end();
- if (nbytes < 0)
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not read file \"%s\": %m", fromfile)));
+ /*
+ * Try to read as much as we fit in the buffer so we can deal with
+ * complete blocks if we need to reencrypt.
+ */
+ nbytes = 0;
+ while (nbytes < COPY_BUF_SIZE)
+ {
+ pgstat_report_wait_start(WAIT_EVENT_COPY_FILE_READ);
+ bytesread = read(srcfd, buffer + nbytes, COPY_BUF_SIZE - nbytes);
+ pgstat_report_wait_end();
+ if (bytesread < 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read file \"%s\": %m", fromfile)));
+ nbytes += bytesread;
+ if (bytesread == 0)
+ break;
+ }
if (nbytes == 0)
break;
+
+ /*
+ * If the database is encrypted we need to decrypt the data here and
+ * reencrypt it to adjust the tweak values of blocks.
+ */
+ if (data_encrypted)
+ {
+ if (fromNode != NULL)
+ {
+ Assert(toNode != NULL);
+
+ /*
+ * There's no reason not to have whole number of pages read
+ * Computation of the number of blocks below relies on this
+ * fact.
+ */
+ Assert(nbytes % BLCKSZ == 0);
+
+ blockNum = ReencryptBlock(buffer, nbytes / BLCKSZ,
+ fromNode, toNode, fromForkNum,
+ toForkNum, blockNum);
+ }
+ }
+
errno = 0;
pgstat_report_wait_start(WAIT_EVENT_COPY_FILE_WRITE);
if ((int) write(dstfd, buffer, nbytes) != nbytes)
diff --git a/src/backend/storage/file/encryption.c b/src/backend/storage/file/encryption.c
index 971b6928e4..e79f967fbb 100644
--- a/src/backend/storage/file/encryption.c
+++ b/src/backend/storage/file/encryption.c
@@ -63,6 +63,7 @@ char encryption_verification[ENCRYPTION_SAMPLE_SIZE];
bool encryption_setup_done = false;
+PGAlignedBlock encrypt_buf;
char *encrypt_buf_xlog = NULL;
#ifdef USE_ENCRYPTION
@@ -425,3 +426,106 @@ XLogEncryptionTweak(char *tweak, TimeLineID timeline, XLogSegNo segment,
tweak += sizeof(XLogSegNo);
memcpy(tweak, &offset, sizeof(offset));
}
+
+/*
+ * Copying relations between tablespaces/databases means that the tweak values
+ * of each block will change. This function transcodes a series of blocks with
+ * new tweak values. Returns the new block number for convenience.
+ */
+BlockNumber
+ReencryptBlock(char *buffer, int blocks,
+ RelFileNode *srcNode, RelFileNode *dstNode,
+ ForkNumber srcForkNum, ForkNumber dstForkNum,
+ BlockNumber blockNum)
+{
+ char *cur;
+ char srcTweak[TWEAK_SIZE];
+ char dstTweak[TWEAK_SIZE];
+
+ for (cur = buffer; cur < buffer + blocks * BLCKSZ; cur += BLCKSZ)
+ {
+ mdtweak(srcTweak, srcNode, srcForkNum, blockNum);
+ mdtweak(dstTweak, dstNode, dstForkNum, blockNum);
+ decrypt_block(cur, cur, BLCKSZ, srcTweak, false);
+ encrypt_block(cur, cur, BLCKSZ, dstTweak, false);
+ blockNum++;
+ }
+ return blockNum;
+}
+
+/*
+ * md files are encrypted block at a time. Tweak will alias higher numbered
+ * forks for huge tables.
+ */
+void
+mdtweak(char *tweak, RelFileNode *relnode, ForkNumber forknum, BlockNumber blocknum)
+{
+ uint32 fork_and_block = (forknum << 24) ^ blocknum;
+
+ memcpy(tweak, relnode, sizeof(RelFileNode));
+ memcpy(tweak + sizeof(RelFileNode), &fork_and_block, 4);
+}
+
+#ifndef FRONTEND
+/*
+ * When page is encrypted using a cipher in the cipher-block chaining (CBC)
+ * mode, the fact that the page starts with LSN makes it harder for adversary
+ * to see which part of the plain (unencrypted) page changed: the LSN changes
+ * even if only the plain data at the end of the page changed, and, due to the
+ * chaining, the encrypted page becomes completely different. Although no LSN
+ * is needed for unlogged tables, we still want to keep the advantage
+ * described here. So we set pd_lsn to "fake LSN" before each write.
+ *
+ * Note that caller needs to restore InvalidXLogRecPtr after the write so that
+ * it can be recognized later that the page needs to be treated specially.
+ *
+ * LW_SHARED on the buffer contents is sufficient because pd_lsn is not
+ * expected to be used for other purposes.
+ *
+ * Returns true iff the LSN was updated.
+ */
+bool
+EnforceLSNUpdateForEncryption(char *buf_contents)
+{
+ PageHeader hdr = (PageHeader) buf_contents;
+ XLogRecPtr recptr;
+
+ /* Failure indicates incorrect user of the function. */
+ Assert(data_encrypted);
+
+ /*
+ * All-zero page shouldn't be encrypted, so don't care about the fake LSN.
+ */
+ if (PageIsNew(hdr))
+ return false;
+
+ recptr = PageXLogRecPtrGet(hdr->pd_lsn);
+
+ /*
+ * Valid LSN indicates that the relation is either logged or it's a page
+ * of unlogged GIST index. In either case we must not touch the LSN.
+ */
+ if (!XLogRecPtrIsInvalid(recptr))
+ return false;
+
+ PageXLogRecPtrSet(hdr->pd_lsn, GetFakeLSNForUnloggedRel());
+
+ return true;
+}
+
+/*
+ * If EnforceLSNUpdateForEncryption() returned true and if the function can be
+ * passed that buffer again, use this function to restore the
+ * InvalidXLogRecPtr value.
+ */
+void
+RestoreInvalidLSN(char *buf_contents)
+{
+ PageHeader hdr = (PageHeader) buf_contents;
+
+ /* Failure indicates incorrect user of the function. */
+ Assert(data_encrypted);
+
+ PageXLogRecPtrSet(hdr->pd_lsn, (XLogRecPtr) InvalidXLogRecPtr);
+}
+#endif /* FRONTEND */
diff --git a/src/backend/storage/file/reinit.c b/src/backend/storage/file/reinit.c
index ccdf80d614..6af20cfe64 100644
--- a/src/backend/storage/file/reinit.c
+++ b/src/backend/storage/file/reinit.c
@@ -16,6 +16,7 @@
#include <unistd.h>
+#include "catalog/pg_tablespace.h"
#include "common/relpath.h"
#include "storage/copydir.h"
#include "storage/fd.h"
@@ -24,9 +25,9 @@
#include "utils/memutils.h"
static void ResetUnloggedRelationsInTablespaceDir(const char *tsdirname,
- int op);
+ int op, Oid spcOid);
static void ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname,
- int op);
+ int op, Oid spcOid, Oid dbOid);
typedef struct
{
@@ -68,7 +69,7 @@ ResetUnloggedRelations(int op)
/*
* First process unlogged files in pg_default ($PGDATA/base)
*/
- ResetUnloggedRelationsInTablespaceDir("base", op);
+ ResetUnloggedRelationsInTablespaceDir("base", op, DEFAULTTABLESPACE_OID);
/*
* Cycle through directories for all non-default tablespaces.
@@ -77,13 +78,16 @@ ResetUnloggedRelations(int op)
while ((spc_de = ReadDir(spc_dir, "pg_tblspc")) != NULL)
{
+ Oid spcOid;
+
if (strcmp(spc_de->d_name, ".") == 0 ||
strcmp(spc_de->d_name, "..") == 0)
continue;
snprintf(temp_path, sizeof(temp_path), "pg_tblspc/%s/%s",
spc_de->d_name, TABLESPACE_VERSION_DIRECTORY);
- ResetUnloggedRelationsInTablespaceDir(temp_path, op);
+ spcOid = atoi(spc_de->d_name);
+ ResetUnloggedRelationsInTablespaceDir(temp_path, op, spcOid);
}
FreeDir(spc_dir);
@@ -99,7 +103,8 @@ ResetUnloggedRelations(int op)
* Process one tablespace directory for ResetUnloggedRelations
*/
static void
-ResetUnloggedRelationsInTablespaceDir(const char *tsdirname, int op)
+ResetUnloggedRelationsInTablespaceDir(const char *tsdirname, int op,
+ Oid spcOid)
{
DIR *ts_dir;
struct dirent *de;
@@ -126,6 +131,8 @@ ResetUnloggedRelationsInTablespaceDir(const char *tsdirname, int op)
while ((de = ReadDir(ts_dir, tsdirname)) != NULL)
{
+ Oid dbOid;
+
/*
* We're only interested in the per-database directories, which have
* numeric names. Note that this code will also (properly) ignore "."
@@ -134,9 +141,10 @@ ResetUnloggedRelationsInTablespaceDir(const char *tsdirname, int op)
if (strspn(de->d_name, "0123456789") != strlen(de->d_name))
continue;
+ dbOid = atoi(de->d_name);
snprintf(dbspace_path, sizeof(dbspace_path), "%s/%s",
tsdirname, de->d_name);
- ResetUnloggedRelationsInDbspaceDir(dbspace_path, op);
+ ResetUnloggedRelationsInDbspaceDir(dbspace_path, op, spcOid, dbOid);
}
FreeDir(ts_dir);
@@ -146,7 +154,8 @@ ResetUnloggedRelationsInTablespaceDir(const char *tsdirname, int op)
* Process one per-dbspace directory for ResetUnloggedRelations
*/
static void
-ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op)
+ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op,
+ Oid spcOid, Oid dbOid)
{
DIR *dbspace_dir;
struct dirent *de;
@@ -187,7 +196,7 @@ ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op)
/* Skip anything that doesn't look like a relation data file. */
if (!parse_filename_for_nontemp_relation(de->d_name, &oidchars,
- &forkNum))
+ &forkNum, NULL))
continue;
/* Also skip it unless this is the init fork. */
@@ -229,7 +238,7 @@ ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op)
/* Skip anything that doesn't look like a relation data file. */
if (!parse_filename_for_nontemp_relation(de->d_name, &oidchars,
- &forkNum))
+ &forkNum, NULL))
continue;
/* We never remove the init fork. */
@@ -279,13 +288,14 @@ ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op)
{
ForkNumber forkNum;
int oidchars;
+ int segment;
char oidbuf[OIDCHARS + 1];
char srcpath[MAXPGPATH * 2];
char dstpath[MAXPGPATH];
/* Skip anything that doesn't look like a relation data file. */
if (!parse_filename_for_nontemp_relation(de->d_name, &oidchars,
- &forkNum))
+ &forkNum, &segment))
continue;
/* Also skip it unless this is the init fork. */
@@ -305,7 +315,13 @@ ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op)
/* OK, we're ready to perform the actual copy. */
elog(DEBUG2, "copying %s to %s", srcpath, dstpath);
- copy_file(srcpath, dstpath);
+ {
+ RelFileNode srcNode = {spcOid, dbOid, atol(oidbuf)};
+ RelFileNode dstNode = srcNode;
+
+ copy_file(srcpath, dstpath, &srcNode, &dstNode,
+ INIT_FORKNUM, MAIN_FORKNUM, segment);
+ }
}
FreeDir(dbspace_dir);
@@ -327,7 +343,7 @@ ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op)
/* Skip anything that doesn't look like a relation data file. */
if (!parse_filename_for_nontemp_relation(de->d_name, &oidchars,
- &forkNum))
+ &forkNum, NULL))
continue;
/* Also skip it unless this is the init fork. */
@@ -372,9 +388,10 @@ ResetUnloggedRelationsInDbspaceDir(const char *dbspacedirname, int op)
*/
bool
parse_filename_for_nontemp_relation(const char *name, int *oidchars,
- ForkNumber *fork)
+ ForkNumber *fork, int *segment)
{
int pos;
+ int segstart = 0;
/* Look for a non-empty string of digits (that isn't too long). */
for (pos = 0; isdigit((unsigned char) name[pos]); ++pos)
@@ -401,6 +418,7 @@ parse_filename_for_nontemp_relation(const char *name, int *oidchars,
{
int segchar;
+ segstart = pos + 1;
for (segchar = 1; isdigit((unsigned char) name[pos + segchar]); ++segchar)
;
if (segchar <= 1)
@@ -411,5 +429,14 @@ parse_filename_for_nontemp_relation(const char *name, int *oidchars,
/* Now we should be at the end. */
if (name[pos] != '\0')
return false;
+
+ if (segment != NULL)
+ {
+ if (segstart == 0)
+ *segment = 0;
+ else
+ *segment = atoi(name + segstart);
+ }
+
return true;
}
diff --git a/src/backend/storage/page/bufpage.c b/src/backend/storage/page/bufpage.c
index 6b49810e37..b6c358a642 100644
--- a/src/backend/storage/page/bufpage.c
+++ b/src/backend/storage/page/bufpage.c
@@ -17,6 +17,7 @@
#include "access/htup_details.h"
#include "access/itup.h"
#include "access/xlog.h"
+#include "common/string.h"
#include "pgstat.h"
#include "storage/checksum.h"
#include "utils/memdebug.h"
@@ -82,11 +83,8 @@ bool
PageIsVerified(Page page, BlockNumber blkno)
{
PageHeader p = (PageHeader) page;
- size_t *pagebytes;
- int i;
bool checksum_failure = false;
bool header_sane = false;
- bool all_zeroes = false;
uint16 checksum = 0;
/*
@@ -119,26 +117,8 @@ PageIsVerified(Page page, BlockNumber blkno)
return true;
}
- /*
- * Check all-zeroes case. Luckily BLCKSZ is guaranteed to always be a
- * multiple of size_t - and it's much faster to compare memory using the
- * native word size.
- */
- StaticAssertStmt(BLCKSZ == (BLCKSZ / sizeof(size_t)) * sizeof(size_t),
- "BLCKSZ has to be a multiple of sizeof(size_t)");
-
- all_zeroes = true;
- pagebytes = (size_t *) page;
- for (i = 0; i < (BLCKSZ / sizeof(size_t)); i++)
- {
- if (pagebytes[i] != 0)
- {
- all_zeroes = false;
- break;
- }
- }
-
- if (all_zeroes)
+ /* Check all-zeroes case */
+ if (IsAllZero((char *) page, BLCKSZ))
return true;
/*
diff --git a/src/backend/storage/smgr/md.c b/src/backend/storage/smgr/md.c
index 64acc3fa43..1b986eac01 100644
--- a/src/backend/storage/smgr/md.c
+++ b/src/backend/storage/smgr/md.c
@@ -32,6 +32,7 @@
#include "postmaster/bgwriter.h"
#include "storage/fd.h"
#include "storage/bufmgr.h"
+#include "storage/encryption.h"
#include "storage/md.h"
#include "storage/relfilenode.h"
#include "storage/smgr.h"
@@ -86,6 +87,7 @@ typedef struct _MdfdVec
static MemoryContext MdCxt; /* context for all MdfdVec objects */
+static char *md_encryption_tweak;
/* Populate a file tag describing an md.c segment file. */
#define INIT_MD_FILETAG(a,xx_rnode,xx_forknum,xx_segno) \
@@ -139,6 +141,8 @@ static MdfdVec *_mdfd_getseg(SMgrRelation reln, ForkNumber forkno,
static BlockNumber _mdnblocks(SMgrRelation reln, ForkNumber forknum,
MdfdVec *seg);
+static void mdencrypt(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum, char *buffer);
+static void mddecrypt(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum, char *buffer);
/*
* mdinit() -- Initialize private state for magnetic disk storage manager.
@@ -149,6 +153,8 @@ mdinit(void)
MdCxt = AllocSetContextCreate(TopMemoryContext,
"MdSmgr",
ALLOCSET_DEFAULT_SIZES);
+
+ md_encryption_tweak = MemoryContextAllocZero(MdCxt, TWEAK_SIZE);
}
/*
@@ -401,6 +407,12 @@ mdextend(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
Assert(seekpos < (off_t) BLCKSZ * RELSEG_SIZE);
+ if (data_encrypted)
+ {
+ mdencrypt(reln, forknum, blocknum, buffer);
+ buffer = encrypt_buf.data;
+ }
+
if ((nbytes = FileWrite(v->mdfd_vfd, buffer, BLCKSZ, seekpos, WAIT_EVENT_DATA_FILE_EXTEND)) != BLCKSZ)
{
if (nbytes < 0)
@@ -587,6 +599,7 @@ mdread(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
off_t seekpos;
int nbytes;
MdfdVec *v;
+ char *buffer_read = buffer;
TRACE_POSTGRESQL_SMGR_MD_READ_START(forknum, blocknum,
reln->smgr_rnode.node.spcNode,
@@ -601,7 +614,10 @@ mdread(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
Assert(seekpos < (off_t) BLCKSZ * RELSEG_SIZE);
- nbytes = FileRead(v->mdfd_vfd, buffer, BLCKSZ, seekpos, WAIT_EVENT_DATA_FILE_READ);
+ if (data_encrypted)
+ buffer_read = encrypt_buf.data;
+
+ nbytes = FileRead(v->mdfd_vfd, buffer_read, BLCKSZ, seekpos, WAIT_EVENT_DATA_FILE_READ);
TRACE_POSTGRESQL_SMGR_MD_READ_DONE(forknum, blocknum,
reln->smgr_rnode.node.spcNode,
@@ -636,6 +652,8 @@ mdread(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
blocknum, FilePathName(v->mdfd_vfd),
nbytes, BLCKSZ)));
}
+ else if (data_encrypted)
+ mddecrypt(reln, forknum, blocknum, buffer);
}
/*
@@ -671,6 +689,11 @@ mdwrite(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
Assert(seekpos < (off_t) BLCKSZ * RELSEG_SIZE);
+ if (data_encrypted)
+ {
+ mdencrypt(reln, forknum, blocknum, buffer);
+ buffer = encrypt_buf.data;
+ }
nbytes = FileWrite(v->mdfd_vfd, buffer, BLCKSZ, seekpos, WAIT_EVENT_DATA_FILE_WRITE);
TRACE_POSTGRESQL_SMGR_MD_WRITE_DONE(forknum, blocknum,
@@ -1248,6 +1271,22 @@ _mdnblocks(SMgrRelation reln, ForkNumber forknum, MdfdVec *seg)
return (BlockNumber) (len / BLCKSZ);
}
+
+static void
+mdencrypt(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum, char *buffer)
+{
+ mdtweak(md_encryption_tweak, &(reln->smgr_rnode.node), forknum, blocknum);
+ encrypt_block(buffer, encrypt_buf.data, BLCKSZ, md_encryption_tweak,
+ false);
+}
+
+static void
+mddecrypt(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum, char *dest)
+{
+ mdtweak(md_encryption_tweak, &(reln->smgr_rnode.node), forknum, blocknum);
+ decrypt_block(encrypt_buf.data, dest, BLCKSZ, md_encryption_tweak, false);
+}
+
/*
* Sync a file to disk, given a file tag. Write the path into an output
* buffer so the caller can use it in error messages.
diff --git a/src/include/storage/copydir.h b/src/include/storage/copydir.h
index 525cc6203e..21c18bb6c3 100644
--- a/src/include/storage/copydir.h
+++ b/src/include/storage/copydir.h
@@ -13,7 +13,9 @@
#ifndef COPYDIR_H
#define COPYDIR_H
-extern void copydir(char *fromdir, char *todir, bool recurse);
-extern void copy_file(char *fromfile, char *tofile);
+#include "storage/relfilenode.h"
+
+extern void copydir(char *fromdir, char *todir, RelFileNode *fromNode, RelFileNode *toNode);
+extern void copy_file(char *fromfile, char *tofile, RelFileNode *fromNode, RelFileNode *toNode, ForkNumber fromForkNum, ForkNumber toForkNum, int segment);
#endif /* COPYDIR_H */
diff --git a/src/include/storage/encryption.h b/src/include/storage/encryption.h
index c6ad9b43bd..eab8b50e98 100644
--- a/src/include/storage/encryption.h
+++ b/src/include/storage/encryption.h
@@ -117,8 +117,15 @@ extern char encryption_verification[];
extern bool encryption_setup_done;
/*
- * XLOG encryption/decryption buffer. This buffer spans multiple pages, in
- * order to reduce the number of syscalls when doing I/O.
+ * In some cases we need a separate copy of the data because encryption
+ * in-place (typically in the shared buffers) would make the data unusable for
+ * backends.
+ */
+extern PGAlignedBlock encrypt_buf;
+
+/*
+ * The same for XLOG. This buffer spans multiple pages, in order to reduce the
+ * number of syscalls when doing I/O.
*
* XXX Fine tune the buffer size.
*/
@@ -144,5 +151,16 @@ extern void encryption_error(bool fatal, char *message);
extern void XLogEncryptionTweak(char *tweak, TimeLineID timeline,
XLogSegNo segment, uint32 offset);
+extern BlockNumber ReencryptBlock(char *buffer, int blocks,
+ RelFileNode *srcNode, RelFileNode *dstNode,
+ ForkNumber srcForkNum, ForkNumber dstForkNum,
+ BlockNumber blockNum);
+extern void mdtweak(char *tweak, RelFileNode *relnode, ForkNumber forknum,
+ BlockNumber blocknum);
+
+#ifndef FRONTEND
+extern bool EnforceLSNUpdateForEncryption(char *buf_contents);
+extern void RestoreInvalidLSN(char *buf_contents);
+#endif /* FRONTEND */
#endif /* ENCRYPTION_H */
diff --git a/src/include/storage/reinit.h b/src/include/storage/reinit.h
index 63d0b56d48..0e028cac4d 100644
--- a/src/include/storage/reinit.h
+++ b/src/include/storage/reinit.h
@@ -20,7 +20,8 @@
extern void ResetUnloggedRelations(int op);
extern bool parse_filename_for_nontemp_relation(const char *name,
- int *oidchars, ForkNumber *fork);
+ int *oidchars, ForkNumber *fork,
+ int *segment);
#define UNLOGGED_RELATION_CLEANUP 0x0001
#define UNLOGGED_RELATION_INIT 0x0002
--
2.13.7
v04-0010-Introduce-transient-buffered-file.patch (text/x-diff, 49.0K) download
From b280f32f789a919d87978ea945de9521da837b9a Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Fri, 5 Jul 2019 16:24:01 +0200
Subject: [PATCH 10/17] Introduce "transient buffered file".
Currently we need it only to encrypt transaction data which gets spilled to
disk during logical decoding. This way we hide the encryption specific details
from reorderbuffer.c, as well as any other (future) user of transient files
that need to be encrypted.
The next patch of the series implements the actual encryption of those files.
---
src/backend/postmaster/pgstat.c | 6 -
src/backend/replication/logical/reorderbuffer.c | 457 ++++++++----------
src/backend/storage/file/buffile.c | 595 ++++++++++++++++++------
src/include/pgstat.h | 2 -
src/include/replication/reorderbuffer.h | 4 -
src/include/storage/buffile.h | 13 +-
6 files changed, 652 insertions(+), 425 deletions(-)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index b4f2b28b51..54786899fd 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -4004,12 +4004,6 @@ pgstat_get_wait_io(WaitEventIO w)
case WAIT_EVENT_RELATION_MAP_WRITE:
event_name = "RelationMapWrite";
break;
- case WAIT_EVENT_REORDER_BUFFER_READ:
- event_name = "ReorderBufferRead";
- break;
- case WAIT_EVENT_REORDER_BUFFER_WRITE:
- event_name = "ReorderBufferWrite";
- break;
case WAIT_EVENT_REORDER_LOGICAL_MAPPING_READ:
event_name = "ReorderLogicalMappingRead";
break;
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index e7c32f2a13..844b67d44b 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -71,6 +71,7 @@
#include "replication/slot.h"
#include "replication/snapbuild.h" /* just for SnapBuildSnapDecRefcount */
#include "storage/bufmgr.h"
+#include "storage/buffile.h"
#include "storage/fd.h"
#include "storage/sinval.h"
#include "utils/builtins.h"
@@ -109,7 +110,7 @@ typedef struct ReorderBufferIterTXNEntry
XLogRecPtr lsn;
ReorderBufferChange *change;
ReorderBufferTXN *txn;
- int fd;
+ TransientBufFile *file;
XLogSegNo segno;
} ReorderBufferIterTXNEntry;
@@ -192,11 +193,17 @@ static void ReorderBufferExecuteInvalidations(ReorderBuffer *rb, ReorderBufferTX
static void ReorderBufferCheckSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn);
static void ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn);
static void ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
- int fd, ReorderBufferChange *change);
+ TransientBufFile *file, ReorderBufferChange *change);
+static void ReorderBufferWriteData(TransientBufFile *file, void *ptr, size_t size,
+ ReorderBufferTXN *txn);
static Size ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn,
- int *fd, XLogSegNo *segno);
+ TransientBufFile **file, XLogSegNo *segno);
static void ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
- char *change);
+ TransientBufFile **file);
+static ReorderBufferTupleBuf *ReorderBufferRestoreTuple(ReorderBuffer *rb,
+ TransientBufFile *file);
+static void ReorderBufferReadData(TransientBufFile *file, void *ptr, size_t size,
+ bool *no_data_p);
static void ReorderBufferRestoreCleanup(ReorderBuffer *rb, ReorderBufferTXN *txn);
static void ReorderBufferCleanupSerializedTXNs(const char *slotname);
static void ReorderBufferSerializedPath(char *path, ReplicationSlot *slot,
@@ -267,9 +274,6 @@ ReorderBufferAllocate(void)
buffer->by_txn_last_xid = InvalidTransactionId;
buffer->by_txn_last_txn = NULL;
- buffer->outbuf = NULL;
- buffer->outbufsize = 0;
-
buffer->current_restart_decoding_lsn = InvalidXLogRecPtr;
dlist_init(&buffer->toplevel_by_lsn);
@@ -988,7 +992,7 @@ ReorderBufferIterTXNInit(ReorderBuffer *rb, ReorderBufferTXN *txn)
for (off = 0; off < state->nr_txns; off++)
{
- state->entries[off].fd = -1;
+ state->entries[off].file = NULL;
state->entries[off].segno = 0;
}
@@ -1013,7 +1017,7 @@ ReorderBufferIterTXNInit(ReorderBuffer *rb, ReorderBufferTXN *txn)
{
/* serialize remaining changes */
ReorderBufferSerializeTXN(rb, txn);
- ReorderBufferRestoreChanges(rb, txn, &state->entries[off].fd,
+ ReorderBufferRestoreChanges(rb, txn, &state->entries[off].file,
&state->entries[off].segno);
}
@@ -1043,7 +1047,7 @@ ReorderBufferIterTXNInit(ReorderBuffer *rb, ReorderBufferTXN *txn)
/* serialize remaining changes */
ReorderBufferSerializeTXN(rb, cur_txn);
ReorderBufferRestoreChanges(rb, cur_txn,
- &state->entries[off].fd,
+ &state->entries[off].file,
&state->entries[off].segno);
}
cur_change = dlist_head_element(ReorderBufferChange, node,
@@ -1124,7 +1128,7 @@ ReorderBufferIterTXNNext(ReorderBuffer *rb, ReorderBufferIterTXNState *state)
dlist_delete(&change->node);
dlist_push_tail(&state->old_change, &change->node);
- if (ReorderBufferRestoreChanges(rb, entry->txn, &entry->fd,
+ if (ReorderBufferRestoreChanges(rb, entry->txn, &entry->file,
&state->entries[off].segno))
{
/* successfully restored changes from disk */
@@ -1163,8 +1167,8 @@ ReorderBufferIterTXNFinish(ReorderBuffer *rb,
for (off = 0; off < state->nr_txns; off++)
{
- if (state->entries[off].fd != -1)
- CloseTransientFile(state->entries[off].fd);
+ if (state->entries[off].file)
+ BufFileCloseTransient(state->entries[off].file);
}
/* free memory we might have "leaked" in the last *Next call */
@@ -2212,24 +2216,6 @@ ReorderBufferXidHasBaseSnapshot(ReorderBuffer *rb, TransactionId xid)
*/
/*
- * Ensure the IO buffer is >= sz.
- */
-static void
-ReorderBufferSerializeReserve(ReorderBuffer *rb, Size sz)
-{
- if (!rb->outbufsize)
- {
- rb->outbuf = MemoryContextAlloc(rb->context, sz);
- rb->outbufsize = sz;
- }
- else if (rb->outbufsize < sz)
- {
- rb->outbuf = repalloc(rb->outbuf, sz);
- rb->outbufsize = sz;
- }
-}
-
-/*
* Check whether the transaction tx should spill its data to disk.
*/
static void
@@ -2254,7 +2240,7 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
{
dlist_iter subtxn_i;
dlist_mutable_iter change_i;
- int fd = -1;
+ TransientBufFile *file = NULL;
XLogSegNo curOpenSegNo = 0;
Size spilled = 0;
@@ -2281,13 +2267,13 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
* store in segment in which it belongs by start lsn, don't split over
* multiple segments tho
*/
- if (fd == -1 ||
+ if (file == NULL ||
!XLByteInSeg(change->lsn, curOpenSegNo, wal_segment_size))
{
char path[MAXPGPATH];
- if (fd != -1)
- CloseTransientFile(fd);
+ if (file)
+ BufFileCloseTransient(file);
XLByteToSeg(change->lsn, curOpenSegNo, wal_segment_size);
@@ -2299,16 +2285,11 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
curOpenSegNo);
/* open segment, create it if necessary */
- fd = OpenTransientFile(path,
- O_CREAT | O_WRONLY | O_APPEND | PG_BINARY);
-
- if (fd < 0)
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not open file \"%s\": %m", path)));
+ file = BufFileOpenTransient(path,
+ O_CREAT | O_WRONLY | O_APPEND | PG_BINARY);
}
- ReorderBufferSerializeChange(rb, txn, fd, change);
+ ReorderBufferSerializeChange(rb, txn, file, change);
dlist_delete(&change->node);
ReorderBufferReturnChange(rb, change);
@@ -2320,8 +2301,8 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
txn->nentries_mem = 0;
txn->serialized = true;
- if (fd != -1)
- CloseTransientFile(fd);
+ if (file)
+ BufFileCloseTransient(file);
}
/*
@@ -2329,15 +2310,13 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
*/
static void
ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
- int fd, ReorderBufferChange *change)
+ TransientBufFile *file, ReorderBufferChange *change)
{
- ReorderBufferDiskChange *ondisk;
+ ReorderBufferDiskChange hdr;
Size sz = sizeof(ReorderBufferDiskChange);
- ReorderBufferSerializeReserve(rb, sz);
-
- ondisk = (ReorderBufferDiskChange *) rb->outbuf;
- memcpy(&ondisk->change, change, sizeof(ReorderBufferChange));
+ memcpy((char *) &hdr + offsetof(ReorderBufferDiskChange, change),
+ change, sizeof(ReorderBufferChange));
switch (change->action)
{
@@ -2347,7 +2326,6 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
case REORDER_BUFFER_CHANGE_DELETE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT:
{
- char *data;
ReorderBufferTupleBuf *oldtup,
*newtup;
Size oldlen = 0;
@@ -2370,66 +2348,55 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
sz += newlen;
}
- /* make sure we have enough space */
- ReorderBufferSerializeReserve(rb, sz);
-
- data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
- /* might have been reallocated above */
- ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+ hdr.size = sz;
+ ReorderBufferWriteData(file, &hdr, sizeof(ReorderBufferDiskChange),
+ txn);
if (oldlen)
{
- memcpy(data, &oldtup->tuple, sizeof(HeapTupleData));
- data += sizeof(HeapTupleData);
-
- memcpy(data, oldtup->tuple.t_data, oldlen);
- data += oldlen;
+ ReorderBufferWriteData(file, &oldtup->tuple,
+ sizeof(HeapTupleData), txn);
+ ReorderBufferWriteData(file, oldtup->tuple.t_data, oldlen,
+ txn);
}
if (newlen)
{
- memcpy(data, &newtup->tuple, sizeof(HeapTupleData));
- data += sizeof(HeapTupleData);
-
- memcpy(data, newtup->tuple.t_data, newlen);
- data += newlen;
+ ReorderBufferWriteData(file, &newtup->tuple,
+ sizeof(HeapTupleData), txn);
+ ReorderBufferWriteData(file, newtup->tuple.t_data, newlen,
+ txn);
}
break;
}
case REORDER_BUFFER_CHANGE_MESSAGE:
{
- char *data;
Size prefix_size = strlen(change->data.msg.prefix) + 1;
sz += prefix_size + change->data.msg.message_size +
sizeof(Size) + sizeof(Size);
- ReorderBufferSerializeReserve(rb, sz);
- data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
-
- /* might have been reallocated above */
- ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+ hdr.size = sz;
+ ReorderBufferWriteData(file, &hdr,
+ sizeof(ReorderBufferDiskChange),
+ txn);
/* write the prefix including the size */
- memcpy(data, &prefix_size, sizeof(Size));
- data += sizeof(Size);
- memcpy(data, change->data.msg.prefix,
- prefix_size);
- data += prefix_size;
+ ReorderBufferWriteData(file, &prefix_size, sizeof(Size), txn);
+ ReorderBufferWriteData(file, change->data.msg.prefix,
+ prefix_size, txn);
/* write the message including the size */
- memcpy(data, &change->data.msg.message_size, sizeof(Size));
- data += sizeof(Size);
- memcpy(data, change->data.msg.message,
- change->data.msg.message_size);
- data += change->data.msg.message_size;
+ ReorderBufferWriteData(file, &change->data.msg.message_size,
+ sizeof(Size), txn);
+ ReorderBufferWriteData(file, change->data.msg.message,
+ change->data.msg.message_size, txn);
break;
}
case REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT:
{
Snapshot snap;
- char *data;
snap = change->data.snapshot;
@@ -2438,49 +2405,37 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
sizeof(TransactionId) * snap->subxcnt
;
- /* make sure we have enough space */
- ReorderBufferSerializeReserve(rb, sz);
- data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
- /* might have been reallocated above */
- ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+ hdr.size = sz;
+ ReorderBufferWriteData(file, &hdr,
+ sizeof(ReorderBufferDiskChange), txn);
- memcpy(data, snap, sizeof(SnapshotData));
- data += sizeof(SnapshotData);
+ ReorderBufferWriteData(file, snap, sizeof(SnapshotData), txn);
if (snap->xcnt)
- {
- memcpy(data, snap->xip,
- sizeof(TransactionId) * snap->xcnt);
- data += sizeof(TransactionId) * snap->xcnt;
- }
+ ReorderBufferWriteData(file, snap->xip,
+ sizeof(TransactionId) * snap->xcnt,
+ txn);
if (snap->subxcnt)
- {
- memcpy(data, snap->subxip,
- sizeof(TransactionId) * snap->subxcnt);
- data += sizeof(TransactionId) * snap->subxcnt;
- }
+ ReorderBufferWriteData(file, snap->subxip,
+ sizeof(TransactionId) * snap->subxcnt,
+ txn);
break;
}
case REORDER_BUFFER_CHANGE_TRUNCATE:
{
Size size;
- char *data;
/* account for the OIDs of truncated relations */
size = sizeof(Oid) * change->data.truncate.nrelids;
sz += size;
- /* make sure we have enough space */
- ReorderBufferSerializeReserve(rb, sz);
-
- data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
- /* might have been reallocated above */
- ondisk = (ReorderBufferDiskChange *) rb->outbuf;
-
- memcpy(data, change->data.truncate.relids, size);
- data += size;
+ hdr.size = sz;
+ ReorderBufferWriteData(file, &hdr, sizeof(ReorderBufferDiskChange),
+ txn);
+ ReorderBufferWriteData(file, change->data.truncate.relids, size,
+ txn);
break;
}
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM:
@@ -2489,27 +2444,21 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
/* ReorderBufferChange contains everything important */
break;
}
+}
- ondisk->size = sz;
-
- errno = 0;
- pgstat_report_wait_start(WAIT_EVENT_REORDER_BUFFER_WRITE);
- if (write(fd, rb->outbuf, ondisk->size) != ondisk->size)
- {
- int save_errno = errno;
-
- CloseTransientFile(fd);
-
- /* if write didn't set errno, assume problem is no disk space */
- errno = save_errno ? save_errno : ENOSPC;
+/*
+ * Wrapper for BufFileWriteTransient() that raises ERROR if the whole chunk
+ * was not written. XXX Should this be a macro?
+ */
+static void
+ReorderBufferWriteData(TransientBufFile *file, void *ptr, size_t size,
+ ReorderBufferTXN *txn)
+{
+ if (BufFileWriteTransient(file, ptr, size) != size)
ereport(ERROR,
(errcode_for_file_access(),
errmsg("could not write to data file for XID %u: %m",
txn->xid)));
- }
- pgstat_report_wait_end();
-
- Assert(ondisk->change.action == change->action);
}
/*
@@ -2517,7 +2466,7 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
*/
static Size
ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn,
- int *fd, XLogSegNo *segno)
+ TransientBufFile **file, XLogSegNo *segno)
{
Size restored = 0;
XLogSegNo last_segno;
@@ -2542,10 +2491,7 @@ ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn,
while (restored < max_changes_in_memory && *segno <= last_segno)
{
- int readBytes;
- ReorderBufferDiskChange *ondisk;
-
- if (*fd == -1)
+ if (*file == NULL)
{
char path[MAXPGPATH];
@@ -2562,77 +2508,24 @@ ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferSerializedPath(path, MyReplicationSlot, txn->xid,
*segno);
- *fd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
- if (*fd < 0 && errno == ENOENT)
+ *file = BufFileOpenTransient(path, O_RDONLY | PG_BINARY);
+ if (*file == NULL)
{
- *fd = -1;
+ Assert(errno == ENOENT);
(*segno)++;
continue;
}
- else if (*fd < 0)
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not open file \"%s\": %m",
- path)));
}
- /*
- * Read the statically sized part of a change which has information
- * about the total size. If we couldn't read a record, we're at the
- * end of this file.
- */
- ReorderBufferSerializeReserve(rb, sizeof(ReorderBufferDiskChange));
- pgstat_report_wait_start(WAIT_EVENT_REORDER_BUFFER_READ);
- readBytes = read(*fd, rb->outbuf, sizeof(ReorderBufferDiskChange));
- pgstat_report_wait_end();
-
- /* eof */
- if (readBytes == 0)
+ ReorderBufferRestoreChange(rb, txn, file);
+ if (*file)
+ restored++;
+ else
{
- CloseTransientFile(*fd);
- *fd = -1;
+ /* No data could be restored. */
(*segno)++;
continue;
}
- else if (readBytes < 0)
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not read from reorderbuffer spill file: %m")));
- else if (readBytes != sizeof(ReorderBufferDiskChange))
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not read from reorderbuffer spill file: read %d instead of %u bytes",
- readBytes,
- (uint32) sizeof(ReorderBufferDiskChange))));
-
- ondisk = (ReorderBufferDiskChange *) rb->outbuf;
-
- ReorderBufferSerializeReserve(rb,
- sizeof(ReorderBufferDiskChange) + ondisk->size);
- ondisk = (ReorderBufferDiskChange *) rb->outbuf;
-
- pgstat_report_wait_start(WAIT_EVENT_REORDER_BUFFER_READ);
- readBytes = read(*fd, rb->outbuf + sizeof(ReorderBufferDiskChange),
- ondisk->size - sizeof(ReorderBufferDiskChange));
- pgstat_report_wait_end();
-
- if (readBytes < 0)
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not read from reorderbuffer spill file: %m")));
- else if (readBytes != ondisk->size - sizeof(ReorderBufferDiskChange))
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not read from reorderbuffer spill file: read %d instead of %u bytes",
- readBytes,
- (uint32) (ondisk->size - sizeof(ReorderBufferDiskChange)))));
-
- /*
- * ok, read a full change from disk, now restore it into proper
- * in-memory format
- */
- ReorderBufferRestoreChange(rb, txn, rb->outbuf);
- restored++;
}
return restored;
@@ -2642,25 +2535,36 @@ ReorderBufferRestoreChanges(ReorderBuffer *rb, ReorderBufferTXN *txn,
* Convert change from its on-disk format to in-memory format and queue it onto
* the TXN's ->changes list.
*
- * Note: although "data" is declared char*, at entry it points to a
- * maxalign'd buffer, making it safe in most of this function to assume
- * that the pointed-to data is suitably aligned for direct access.
+ * If no data was found in the file, close it and set *file to NULL.
*/
static void
ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
- char *data)
+ TransientBufFile **file)
{
- ReorderBufferDiskChange *ondisk;
+ ReorderBufferDiskChange ondisk;
+ bool no_data;
ReorderBufferChange *change;
- ondisk = (ReorderBufferDiskChange *) data;
+ /*
+ * Read the statically sized part of a change which has information about
+ * the total size. If we couldn't read a record, we're at the end of this
+ * file.
+ */
+ ReorderBufferReadData(*file, &ondisk, sizeof(ReorderBufferDiskChange),
+ &no_data);
+
+ /* eof */
+ if (no_data)
+ {
+ BufFileCloseTransient(*file);
+ *file = NULL;
+ return;
+ }
change = ReorderBufferGetChange(rb);
/* copy static part */
- memcpy(change, &ondisk->change, sizeof(ReorderBufferChange));
-
- data += sizeof(ReorderBufferDiskChange);
+ memcpy(change, &ondisk.change, sizeof(ReorderBufferChange));
/* restore individual stuff */
switch (change->action)
@@ -2671,50 +2575,10 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
case REORDER_BUFFER_CHANGE_DELETE:
case REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT:
if (change->data.tp.oldtuple)
- {
- uint32 tuplelen = ((HeapTuple) data)->t_len;
-
- change->data.tp.oldtuple =
- ReorderBufferGetTupleBuf(rb, tuplelen - SizeofHeapTupleHeader);
-
- /* restore ->tuple */
- memcpy(&change->data.tp.oldtuple->tuple, data,
- sizeof(HeapTupleData));
- data += sizeof(HeapTupleData);
-
- /* reset t_data pointer into the new tuplebuf */
- change->data.tp.oldtuple->tuple.t_data =
- ReorderBufferTupleBufData(change->data.tp.oldtuple);
-
- /* restore tuple data itself */
- memcpy(change->data.tp.oldtuple->tuple.t_data, data, tuplelen);
- data += tuplelen;
- }
+ change->data.tp.oldtuple = ReorderBufferRestoreTuple(rb, *file);
if (change->data.tp.newtuple)
- {
- /* here, data might not be suitably aligned! */
- uint32 tuplelen;
-
- memcpy(&tuplelen, data + offsetof(HeapTupleData, t_len),
- sizeof(uint32));
-
- change->data.tp.newtuple =
- ReorderBufferGetTupleBuf(rb, tuplelen - SizeofHeapTupleHeader);
-
- /* restore ->tuple */
- memcpy(&change->data.tp.newtuple->tuple, data,
- sizeof(HeapTupleData));
- data += sizeof(HeapTupleData);
-
- /* reset t_data pointer into the new tuplebuf */
- change->data.tp.newtuple->tuple.t_data =
- ReorderBufferTupleBufData(change->data.tp.newtuple);
-
- /* restore tuple data itself */
- memcpy(change->data.tp.newtuple->tuple.t_data, data, tuplelen);
- data += tuplelen;
- }
+ change->data.tp.newtuple = ReorderBufferRestoreTuple(rb, *file);
break;
case REORDER_BUFFER_CHANGE_MESSAGE:
@@ -2722,44 +2586,44 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
Size prefix_size;
/* read prefix */
- memcpy(&prefix_size, data, sizeof(Size));
- data += sizeof(Size);
+ ReorderBufferReadData(*file, &prefix_size, sizeof(Size), NULL);
change->data.msg.prefix = MemoryContextAlloc(rb->context,
prefix_size);
- memcpy(change->data.msg.prefix, data, prefix_size);
+ ReorderBufferReadData(*file, change->data.msg.prefix,
+ prefix_size, NULL);
Assert(change->data.msg.prefix[prefix_size - 1] == '\0');
- data += prefix_size;
/* read the message */
- memcpy(&change->data.msg.message_size, data, sizeof(Size));
- data += sizeof(Size);
+ ReorderBufferReadData(*file, &change->data.msg.message_size,
+ sizeof(Size), NULL);
change->data.msg.message = MemoryContextAlloc(rb->context,
change->data.msg.message_size);
- memcpy(change->data.msg.message, data,
- change->data.msg.message_size);
- data += change->data.msg.message_size;
+ ReorderBufferReadData(*file, change->data.msg.message,
+ change->data.msg.message_size, NULL);
break;
}
case REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT:
{
- Snapshot oldsnap;
+ SnapshotData oldsnap;
Snapshot newsnap;
Size size;
- oldsnap = (Snapshot) data;
+ ReorderBufferReadData(*file, &oldsnap, sizeof(SnapshotData), NULL);
size = sizeof(SnapshotData) +
- sizeof(TransactionId) * oldsnap->xcnt +
- sizeof(TransactionId) * (oldsnap->subxcnt + 0);
+ sizeof(TransactionId) * oldsnap.xcnt +
+ sizeof(TransactionId) * (oldsnap.subxcnt + 0);
change->data.snapshot = MemoryContextAllocZero(rb->context, size);
newsnap = change->data.snapshot;
- memcpy(newsnap, data, size);
+ memcpy(newsnap, &oldsnap, sizeof(SnapshotData));
newsnap->xip = (TransactionId *)
(((char *) newsnap) + sizeof(SnapshotData));
+ ReorderBufferReadData(*file, newsnap->xip,
+ size - sizeof(SnapshotData), NULL);
newsnap->subxip = newsnap->xip + newsnap->xcnt;
newsnap->copied = true;
break;
@@ -2771,7 +2635,9 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
relids = ReorderBufferGetRelids(rb,
change->data.truncate.nrelids);
- memcpy(relids, data, change->data.truncate.nrelids * sizeof(Oid));
+ ReorderBufferReadData(*file, relids,
+ change->data.truncate.nrelids * sizeof(Oid),
+ NULL);
change->data.truncate.relids = relids;
break;
@@ -2787,6 +2653,77 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
}
/*
+ * Convert heap tuple from its on-disk format to in-memory format.
+ */
+static ReorderBufferTupleBuf *
+ReorderBufferRestoreTuple(ReorderBuffer *rb, TransientBufFile *file)
+{
+ HeapTupleData tupdata;
+ uint32 tuplelen;
+ ReorderBufferTupleBuf *result;
+
+ ReorderBufferReadData(file, &tupdata, sizeof(HeapTupleData), NULL);
+ tuplelen = tupdata.t_len;
+
+ result = ReorderBufferGetTupleBuf(rb, tuplelen - SizeofHeapTupleHeader);
+
+ /* restore ->tuple */
+ memcpy(&result->tuple, &tupdata, sizeof(HeapTupleData));
+
+ /* reset t_data pointer into the new tuplebuf */
+ result->tuple.t_data = ReorderBufferTupleBufData(result);
+
+ /* restore tuple data itself */
+ ReorderBufferReadData(file, result->tuple.t_data, tuplelen, NULL);
+
+ return result;
+}
+
+/*
+ * Wrapper for BufFileReadTransient() that raises ERROR if the expected amount
+ * of bytes was not read.
+ *
+ * If valid pointer is passed for no_data_p, set *no_data_p to indicate
+ * whether zero bytes was read. If NULL is passed, do not tolerate missing
+ * data.
+ */
+static void
+ReorderBufferReadData(TransientBufFile *file, void *ptr, size_t size,
+ bool *no_data_p)
+{
+ int readBytes;
+
+ /*
+ * Caller should not request zero bytes. This assumption simplifies
+ * setting of *no_data_p below.
+ */
+ Assert(size > 0);
+
+ if ((readBytes = BufFileReadTransient(file, ptr, size)) != size)
+ {
+ if (no_data_p)
+ *no_data_p = readBytes == 0;
+
+ /*
+ * It is o.k. to receive exactly zero bytes if caller passed valid
+ * no_data_p.
+ */
+ if (no_data_p && *no_data_p)
+ return;
+
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from reorderbuffer spill file: read %d instead of %u bytes",
+ readBytes, (uint32) size)));
+ }
+ else if (no_data_p)
+ {
+ /* Given that size is non-zero, readBytes must be non-zero too. */
+ *no_data_p = false;
+ }
+}
+
+/*
* Remove all on-disk stored for the passed in transaction.
*/
static void
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index b773a76049..b31324627a 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -41,6 +41,8 @@
#include "postgres.h"
+#include <unistd.h>
+
#include "commands/tablespace.h"
#include "executor/instrument.h"
#include "miscadmin.h"
@@ -59,19 +61,44 @@
#define BUFFILE_SEG_SIZE (MAX_PHYSICAL_FILESIZE / BLCKSZ)
/*
+ * Fields that both BufFile and TransientBufFile structures need. It must be
+ * the first field of those structures.
+ */
+typedef struct BufFileCommon
+{
+ bool dirty; /* does buffer need to be written? */
+ int pos; /* next read/write position in buffer */
+ int nbytes; /* total # of valid bytes in buffer */
+
+ /*
+ * "current pos" is position of start of buffer within the logical file.
+ * Position as seen by user of BufFile is (curFile, curOffset + pos).
+ */
+ int curFile; /* file index (0..n) part of current pos,
+ * always zero for TransientBufFile */
+ off_t curOffset; /* offset part of current pos */
+
+ bool readOnly; /* has the file been set to read only? */
+
+ bool append; /* should new data be appended to the end? */
+
+ PGAlignedBlock buffer;
+} BufFileCommon;
+
+/*
* This data structure represents a buffered file that consists of one or
* more physical files (each accessed through a virtual file descriptor
* managed by fd.c).
*/
struct BufFile
{
+ BufFileCommon common; /* Common fields, see above. */
+
int numFiles; /* number of physical files in set */
/* all files except the last have length exactly MAX_PHYSICAL_FILESIZE */
File *files; /* palloc'd array with numFiles entries */
bool isInterXact; /* keep open over transactions? */
- bool dirty; /* does buffer need to be written? */
- bool readOnly; /* has the file been set to read only? */
SharedFileSet *fileset; /* space for segment files if shared */
const char *name; /* name of this BufFile if shared */
@@ -82,16 +109,20 @@ struct BufFile
* because after creation we only repalloc our arrays larger.)
*/
ResourceOwner resowner;
+};
- /*
- * "current pos" is position of start of buffer within the logical file.
- * Position as seen by user of BufFile is (curFile, curOffset + pos).
- */
- int curFile; /* file index (0..n) part of current pos */
- off_t curOffset; /* offset part of current pos */
- int pos; /* next read/write position in buffer */
- int nbytes; /* total # of valid bytes in buffer */
- PGAlignedBlock buffer;
+/*
+ * Buffered variant of a transient file. Unlike BufFile this is simpler in
+ * several ways: 1) it's not split into segments, 2) there's no need of seek,
+ * 3) there's no need to combine read and write access.
+ */
+struct TransientBufFile
+{
+ BufFileCommon common; /* Common fields, see above. */
+
+ /* The underlying file. */
+ char *path;
+ int fd;
};
static BufFile *makeBufFileCommon(int nfiles);
@@ -102,22 +133,32 @@ static void BufFileDumpBuffer(BufFile *file);
static int BufFileFlush(BufFile *file);
static File MakeNewSharedSegment(BufFile *file, int segment);
+static void BufFileLoadBufferTransient(TransientBufFile *file);
+static void BufFileDumpBufferTransient(TransientBufFile *file);
+
+static size_t BufFileReadCommon(BufFileCommon *file, void *ptr, size_t size,
+ bool is_transient);
+static size_t BufFileWriteCommon(BufFileCommon *file, void *ptr, size_t size,
+ bool is_transient);
+
/*
* Create BufFile and perform the common initialization.
*/
static BufFile *
makeBufFileCommon(int nfiles)
{
- BufFile *file = (BufFile *) palloc(sizeof(BufFile));
+ BufFile *file = (BufFile *) palloc0(sizeof(BufFile));
+ BufFileCommon *fcommon = &file->common;
+
+ fcommon->dirty = false;
+ fcommon->curFile = 0;
+ fcommon->curOffset = 0L;
+ fcommon->pos = 0;
+ fcommon->nbytes = 0;
file->numFiles = nfiles;
file->isInterXact = false;
- file->dirty = false;
file->resowner = CurrentResourceOwner;
- file->curFile = 0;
- file->curOffset = 0L;
- file->pos = 0;
- file->nbytes = 0;
return file;
}
@@ -133,7 +174,7 @@ makeBufFile(File firstfile)
file->files = (File *) palloc(sizeof(File));
file->files[0] = firstfile;
- file->readOnly = false;
+ file->common.readOnly = false;
file->fileset = NULL;
file->name = NULL;
@@ -264,7 +305,7 @@ BufFileCreateShared(SharedFileSet *fileset, const char *name)
file->name = pstrdup(name);
file->files = (File *) palloc(sizeof(File));
file->files[0] = MakeNewSharedSegment(file, 0);
- file->readOnly = false;
+ file->common.readOnly = false;
return file;
}
@@ -321,7 +362,7 @@ BufFileOpenShared(SharedFileSet *fileset, const char *name)
file = makeBufFileCommon(nfiles);
file->files = files;
- file->readOnly = true; /* Can't write to files opened this way */
+ file->common.readOnly = true; /* Can't write to files opened this way */
file->fileset = fileset;
file->name = pstrdup(name);
@@ -376,10 +417,10 @@ BufFileExportShared(BufFile *file)
Assert(file->fileset != NULL);
/* It's probably a bug if someone calls this twice. */
- Assert(!file->readOnly);
+ Assert(!file->common.readOnly);
BufFileFlush(file);
- file->readOnly = true;
+ file->common.readOnly = true;
}
/*
@@ -406,7 +447,7 @@ BufFileClose(BufFile *file)
* BufFileLoadBuffer
*
* Load some data into buffer, if possible, starting from curOffset.
- * At call, must have dirty = false, pos and nbytes = 0.
+ * At call, must have dirty = false, nbytes = 0.
* On exit, nbytes is number of bytes loaded.
*/
static void
@@ -417,27 +458,27 @@ BufFileLoadBuffer(BufFile *file)
/*
* Advance to next component file if necessary and possible.
*/
- if (file->curOffset >= MAX_PHYSICAL_FILESIZE &&
- file->curFile + 1 < file->numFiles)
+ if (file->common.curOffset >= MAX_PHYSICAL_FILESIZE &&
+ file->common.curFile + 1 < file->numFiles)
{
- file->curFile++;
- file->curOffset = 0L;
+ file->common.curFile++;
+ file->common.curOffset = 0L;
}
/*
* Read whatever we can get, up to a full bufferload.
*/
- thisfile = file->files[file->curFile];
- file->nbytes = FileRead(thisfile,
- file->buffer.data,
- sizeof(file->buffer),
- file->curOffset,
- WAIT_EVENT_BUFFILE_READ);
- if (file->nbytes < 0)
- file->nbytes = 0;
+ thisfile = file->files[file->common.curFile];
+ file->common.nbytes = FileRead(thisfile,
+ file->common.buffer.data,
+ sizeof(file->common.buffer),
+ file->common.curOffset,
+ WAIT_EVENT_BUFFILE_READ);
+ if (file->common.nbytes < 0)
+ file->common.nbytes = 0;
/* we choose not to advance curOffset here */
- if (file->nbytes > 0)
+ if (file->common.nbytes > 0)
pgBufferUsage.temp_blks_read++;
}
@@ -459,44 +500,44 @@ BufFileDumpBuffer(BufFile *file)
* Unlike BufFileLoadBuffer, we must dump the whole buffer even if it
* crosses a component-file boundary; so we need a loop.
*/
- while (wpos < file->nbytes)
+ while (wpos < file->common.nbytes)
{
off_t availbytes;
/*
* Advance to next component file if necessary and possible.
*/
- if (file->curOffset >= MAX_PHYSICAL_FILESIZE)
+ if (file->common.curOffset >= MAX_PHYSICAL_FILESIZE)
{
- while (file->curFile + 1 >= file->numFiles)
+ while (file->common.curFile + 1 >= file->numFiles)
extendBufFile(file);
- file->curFile++;
- file->curOffset = 0L;
+ file->common.curFile++;
+ file->common.curOffset = 0L;
}
/*
* Determine how much we need to write into this file.
*/
- bytestowrite = file->nbytes - wpos;
- availbytes = MAX_PHYSICAL_FILESIZE - file->curOffset;
+ bytestowrite = file->common.nbytes - wpos;
+ availbytes = MAX_PHYSICAL_FILESIZE - file->common.curOffset;
if ((off_t) bytestowrite > availbytes)
bytestowrite = (int) availbytes;
- thisfile = file->files[file->curFile];
+ thisfile = file->files[file->common.curFile];
bytestowrite = FileWrite(thisfile,
- file->buffer.data + wpos,
+ file->common.buffer.data + wpos,
bytestowrite,
- file->curOffset,
+ file->common.curOffset,
WAIT_EVENT_BUFFILE_WRITE);
if (bytestowrite <= 0)
return; /* failed to write */
- file->curOffset += bytestowrite;
+ file->common.curOffset += bytestowrite;
wpos += bytestowrite;
pgBufferUsage.temp_blks_written++;
}
- file->dirty = false;
+ file->common.dirty = false;
/*
* At this point, curOffset has been advanced to the end of the buffer,
@@ -504,19 +545,19 @@ BufFileDumpBuffer(BufFile *file)
* logical file position, ie, original value + pos, in case that is less
* (as could happen due to a small backwards seek in a dirty buffer!)
*/
- file->curOffset -= (file->nbytes - file->pos);
- if (file->curOffset < 0) /* handle possible segment crossing */
+ file->common.curOffset -= (file->common.nbytes - file->common.pos);
+ if (file->common.curOffset < 0) /* handle possible segment crossing */
{
- file->curFile--;
- Assert(file->curFile >= 0);
- file->curOffset += MAX_PHYSICAL_FILESIZE;
+ file->common.curFile--;
+ Assert(file->common.curFile >= 0);
+ file->common.curOffset += MAX_PHYSICAL_FILESIZE;
}
/*
* Now we can set the buffer empty without changing the logical position
*/
- file->pos = 0;
- file->nbytes = 0;
+ file->common.pos = 0;
+ file->common.nbytes = 0;
}
/*
@@ -527,43 +568,7 @@ BufFileDumpBuffer(BufFile *file)
size_t
BufFileRead(BufFile *file, void *ptr, size_t size)
{
- size_t nread = 0;
- size_t nthistime;
-
- if (file->dirty)
- {
- if (BufFileFlush(file) != 0)
- return 0; /* could not flush... */
- Assert(!file->dirty);
- }
-
- while (size > 0)
- {
- if (file->pos >= file->nbytes)
- {
- /* Try to load more data into buffer. */
- file->curOffset += file->pos;
- file->pos = 0;
- file->nbytes = 0;
- BufFileLoadBuffer(file);
- if (file->nbytes <= 0)
- break; /* no more data available */
- }
-
- nthistime = file->nbytes - file->pos;
- if (nthistime > size)
- nthistime = size;
- Assert(nthistime > 0);
-
- memcpy(ptr, file->buffer.data + file->pos, nthistime);
-
- file->pos += nthistime;
- ptr = (void *) ((char *) ptr + nthistime);
- size -= nthistime;
- nread += nthistime;
- }
-
- return nread;
+ return BufFileReadCommon(&file->common, ptr, size, false);
}
/*
@@ -574,48 +579,7 @@ BufFileRead(BufFile *file, void *ptr, size_t size)
size_t
BufFileWrite(BufFile *file, void *ptr, size_t size)
{
- size_t nwritten = 0;
- size_t nthistime;
-
- Assert(!file->readOnly);
-
- while (size > 0)
- {
- if (file->pos >= BLCKSZ)
- {
- /* Buffer full, dump it out */
- if (file->dirty)
- {
- BufFileDumpBuffer(file);
- if (file->dirty)
- break; /* I/O error */
- }
- else
- {
- /* Hmm, went directly from reading to writing? */
- file->curOffset += file->pos;
- file->pos = 0;
- file->nbytes = 0;
- }
- }
-
- nthistime = BLCKSZ - file->pos;
- if (nthistime > size)
- nthistime = size;
- Assert(nthistime > 0);
-
- memcpy(file->buffer.data + file->pos, ptr, nthistime);
-
- file->dirty = true;
- file->pos += nthistime;
- if (file->nbytes < file->pos)
- file->nbytes = file->pos;
- ptr = (void *) ((char *) ptr + nthistime);
- size -= nthistime;
- nwritten += nthistime;
- }
-
- return nwritten;
+ return BufFileWriteCommon(&file->common, ptr, size, false);
}
/*
@@ -626,10 +590,10 @@ BufFileWrite(BufFile *file, void *ptr, size_t size)
static int
BufFileFlush(BufFile *file)
{
- if (file->dirty)
+ if (file->common.dirty)
{
BufFileDumpBuffer(file);
- if (file->dirty)
+ if (file->common.dirty)
return EOF;
}
@@ -667,8 +631,8 @@ BufFileSeek(BufFile *file, int fileno, off_t offset, int whence)
* fileno. Note that large offsets (> 1 gig) risk overflow in this
* add, unless we have 64-bit off_t.
*/
- newFile = file->curFile;
- newOffset = (file->curOffset + file->pos) + offset;
+ newFile = file->common.curFile;
+ newOffset = (file->common.curOffset + file->common.pos) + offset;
break;
#ifdef NOT_USED
case SEEK_END:
@@ -685,9 +649,9 @@ BufFileSeek(BufFile *file, int fileno, off_t offset, int whence)
return EOF;
newOffset += MAX_PHYSICAL_FILESIZE;
}
- if (newFile == file->curFile &&
- newOffset >= file->curOffset &&
- newOffset <= file->curOffset + file->nbytes)
+ if (newFile == file->common.curFile &&
+ newOffset >= file->common.curOffset &&
+ newOffset <= file->common.curOffset + file->common.nbytes)
{
/*
* Seek is to a point within existing buffer; we can just adjust
@@ -695,7 +659,7 @@ BufFileSeek(BufFile *file, int fileno, off_t offset, int whence)
* whether reading or writing, but buffer remains dirty if we were
* writing.
*/
- file->pos = (int) (newOffset - file->curOffset);
+ file->common.pos = (int) (newOffset - file->common.curOffset);
return 0;
}
/* Otherwise, must reposition buffer, so flush any dirty data */
@@ -723,18 +687,18 @@ BufFileSeek(BufFile *file, int fileno, off_t offset, int whence)
if (newFile >= file->numFiles)
return EOF;
/* Seek is OK! */
- file->curFile = newFile;
- file->curOffset = newOffset;
- file->pos = 0;
- file->nbytes = 0;
+ file->common.curFile = newFile;
+ file->common.curOffset = newOffset;
+ file->common.pos = 0;
+ file->common.nbytes = 0;
return 0;
}
void
BufFileTell(BufFile *file, int *fileno, off_t *offset)
{
- *fileno = file->curFile;
- *offset = file->curOffset + file->pos;
+ *fileno = file->common.curFile;
+ *offset = file->common.curOffset + file->common.pos;
}
/*
@@ -768,8 +732,8 @@ BufFileTellBlock(BufFile *file)
{
long blknum;
- blknum = (file->curOffset + file->pos) / BLCKSZ;
- blknum += file->curFile * BUFFILE_SEG_SIZE;
+ blknum = (file->common.curOffset + file->common.pos) / BLCKSZ;
+ blknum += file->common.curFile * BUFFILE_SEG_SIZE;
return blknum;
}
@@ -828,8 +792,8 @@ BufFileAppend(BufFile *target, BufFile *source)
int i;
Assert(target->fileset != NULL);
- Assert(source->readOnly);
- Assert(!source->dirty);
+ Assert(source->common.readOnly);
+ Assert(!source->common.dirty);
Assert(source->fileset != NULL);
if (target->resowner != source->resowner)
@@ -843,3 +807,330 @@ BufFileAppend(BufFile *target, BufFile *source)
return startBlock;
}
+
+/*
+ * Open TransientBufFile at given path or create one if it does not
+ * exist. User will be allowed either to write to the file or to read from it,
+ * according to fileFlags, but not both.
+ */
+TransientBufFile *
+BufFileOpenTransient(const char *path, int fileFlags)
+{
+ bool readOnly;
+ bool append = false;
+ TransientBufFile *file;
+ BufFileCommon *fcommon;
+ int fd;
+ off_t size;
+
+ /* Either read or write mode, but not both. */
+ Assert((fileFlags & O_RDWR) == 0);
+
+ /* Check whether user wants read or write access. */
+ readOnly = (fileFlags & O_WRONLY) == 0;
+
+ /*
+ * Append mode for read access is not useful, so don't bother implementing
+ * it.
+ */
+ Assert(!(readOnly && append));
+
+ errno = 0;
+ fd = OpenTransientFile(path, fileFlags);
+ if (fd < 0)
+ {
+ /*
+ * If caller wants to read from file and the file is not there, he
+ * should be able to handle the condition on his own.
+ *
+ * XXX Shouldn't we always let caller evaluate errno?
+ */
+ if (errno == ENOENT && (fileFlags & O_RDONLY))
+ return NULL;
+
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not open file \"%s\": %m", path)));
+ }
+
+ file = (TransientBufFile *) palloc(sizeof(TransientBufFile));
+ fcommon = &file->common;
+ fcommon->dirty = false;
+ fcommon->pos = 0;
+ fcommon->nbytes = 0;
+ fcommon->readOnly = readOnly;
+ fcommon->append = append;
+ fcommon->curFile = 0;
+
+ file->path = pstrdup(path);
+ file->fd = fd;
+
+ errno = 0;
+ size = lseek(file->fd, 0, SEEK_END);
+ if (errno > 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not initialize TransientBufFile for file \"%s\": %m",
+ path)));
+
+ if (fcommon->append)
+ {
+ /* Position the buffer at the end of the file. */
+ fcommon->curOffset = size;
+ }
+ else
+ fcommon->curOffset = 0L;
+
+ return file;
+}
+
+/*
+ * Close a TransientBufFile.
+ */
+void
+BufFileCloseTransient(TransientBufFile *file)
+{
+ /* Flush any unwritten data. */
+ if (!file->common.readOnly &&
+ file->common.dirty && file->common.nbytes > 0)
+ {
+ BufFileDumpBufferTransient(file);
+
+ /*
+ * Caller of BufFileWriteTransient() recognizes the failure to flush
+ * buffer by the returned value, however this function has no return
+ * code.
+ */
+ if (file->common.dirty)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not flush file \"%s\": %m", file->path)));
+ }
+
+ if (CloseTransientFile(file->fd))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not close file \"%s\": %m", file->path)));
+
+ pfree(file->path);
+ pfree(file);
+}
+
+/*
+ * Load some data into buffer, if possible, starting from file->offset. At
+ * call, must have dirty = false, pos and nbytes = 0. On exit, nbytes is
+ * number of bytes loaded.
+ */
+static void
+BufFileLoadBufferTransient(TransientBufFile *file)
+{
+ Assert(file->common.readOnly);
+ Assert(!file->common.dirty);
+ Assert(file->common.pos == 0 && file->common.nbytes == 0);
+
+retry:
+
+ /*
+ * Read whatever we can get, up to a full bufferload.
+ */
+ errno = 0;
+ pgstat_report_wait_start(WAIT_EVENT_BUFFILE_READ);
+ file->common.nbytes = pg_pread(file->fd,
+ file->common.buffer.data,
+ sizeof(file->common.buffer),
+ file->common.curOffset);
+ pgstat_report_wait_end();
+
+ if (file->common.nbytes < 0)
+ {
+ /* TODO The W32 specific code, see FileWrite. */
+
+ /* OK to retry if interrupted */
+ if (errno == EINTR)
+ goto retry;
+
+ return;
+ }
+ /* we choose not to advance offset here */
+}
+
+/*
+ * Write contents of a transient file buffer to disk.
+ */
+static void
+BufFileDumpBufferTransient(TransientBufFile *file)
+{
+ int nwritten;
+
+ /* This function should only be needed during write access ... */
+ Assert(!file->common.readOnly);
+
+ /* ... and if there's some work to do. */
+ Assert(file->common.dirty);
+ Assert(file->common.nbytes > 0);
+
+retry:
+ errno = 0;
+ pgstat_report_wait_start(WAIT_EVENT_BUFFILE_WRITE);
+ nwritten = pg_pwrite(file->fd,
+ file->common.buffer.data,
+ file->common.nbytes,
+ file->common.curOffset);
+ pgstat_report_wait_end();
+
+ /* if write didn't set errno, assume problem is no disk space */
+ if (nwritten != file->common.nbytes && errno == 0)
+ errno = ENOSPC;
+
+ if (nwritten < 0)
+ {
+ /* TODO The W32 specific code, see FileWrite. */
+
+ /* OK to retry if interrupted */
+ if (errno == EINTR)
+ goto retry;
+
+ return; /* failed to write */
+ }
+
+ file->common.dirty = false;
+
+ file->common.pos = 0;
+ file->common.nbytes = 0;
+}
+
+/*
+ * Like BufFileRead() except it receives pointer to TransientBufFile.
+ */
+size_t
+BufFileReadTransient(TransientBufFile *file, void *ptr, size_t size)
+{
+ return BufFileReadCommon(&file->common, ptr, size, true);
+}
+
+/*
+ * Like BufFileWrite() except it receives pointer to TransientBufFile.
+ */
+size_t
+BufFileWriteTransient(TransientBufFile *file, void *ptr, size_t size)
+{
+ return BufFileWriteCommon(&file->common, ptr, size, true);
+}
+
+/*
+ * BufFileWriteCommon
+ *
+ * Functionality needed by both BufFileRead() and BufFileReadTransient().
+ */
+static size_t
+BufFileReadCommon(BufFileCommon *file, void *ptr, size_t size,
+ bool is_transient)
+{
+ size_t nread = 0;
+ size_t nthistime;
+
+ if (file->dirty)
+ {
+ /*
+ * Transient file currently does not allow both read and write access,
+ * so this function should not see dirty buffer.
+ */
+ Assert(!is_transient);
+
+ if (BufFileFlush((BufFile *) file) != 0)
+ return 0; /* could not flush... */
+ Assert(!file->dirty);
+ }
+
+ while (size > 0)
+ {
+ if (file->pos >= file->nbytes)
+ {
+ /* Try to load more data into buffer. */
+ file->curOffset += file->pos;
+ file->pos = 0;
+ file->nbytes = 0;
+
+ if (!is_transient)
+ BufFileLoadBuffer((BufFile *) file);
+ else
+ BufFileLoadBufferTransient((TransientBufFile *) file);
+
+ if (file->nbytes <= 0)
+ break; /* no more data available */
+ }
+
+ nthistime = file->nbytes - file->pos;
+ if (nthistime > size)
+ nthistime = size;
+ Assert(nthistime > 0);
+
+ memcpy(ptr, file->buffer.data + file->pos, nthistime);
+
+ file->pos += nthistime;
+ ptr = (void *) ((char *) ptr + nthistime);
+ size -= nthistime;
+ nread += nthistime;
+ }
+
+ return nread;
+}
+
+/*
+ * BufFileWriteCommon
+ *
+ * Functionality needed by both BufFileWrite() and BufFileWriteTransient().
+ */
+static size_t
+BufFileWriteCommon(BufFileCommon *file, void *ptr, size_t size,
+ bool is_transient)
+{
+ size_t nwritten = 0;
+ size_t nthistime;
+
+ Assert(!file->readOnly);
+
+ while (size > 0)
+ {
+ if (file->pos >= BLCKSZ)
+ {
+ /* Buffer full, dump it out */
+ if (file->dirty)
+ {
+ if (!is_transient)
+ BufFileDumpBuffer((BufFile *) file);
+ else
+ BufFileDumpBufferTransient((TransientBufFile *) file);
+
+ if (file->dirty)
+ break; /* I/O error */
+ }
+ else
+ {
+ Assert(!is_transient);
+
+ /* Hmm, went directly from reading to writing? */
+ file->curOffset += file->pos;
+ file->pos = 0;
+ file->nbytes = 0;
+ }
+ }
+
+ nthistime = BLCKSZ - file->pos;
+ if (nthistime > size)
+ nthistime = size;
+ Assert(nthistime > 0);
+
+ memcpy(file->buffer.data + file->pos, ptr, nthistime);
+
+ file->dirty = true;
+ file->pos += nthistime;
+ if (file->nbytes < file->pos)
+ file->nbytes = file->pos;
+ ptr = (void *) ((char *) ptr + nthistime);
+ size -= nthistime;
+ nwritten += nthistime;
+ }
+
+ return nwritten;
+}
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 0a3ad3a188..b923a182a4 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -912,8 +912,6 @@ typedef enum
WAIT_EVENT_RELATION_MAP_READ,
WAIT_EVENT_RELATION_MAP_SYNC,
WAIT_EVENT_RELATION_MAP_WRITE,
- WAIT_EVENT_REORDER_BUFFER_READ,
- WAIT_EVENT_REORDER_BUFFER_WRITE,
WAIT_EVENT_REORDER_LOGICAL_MAPPING_READ,
WAIT_EVENT_REPLICATION_SLOT_READ,
WAIT_EVENT_REPLICATION_SLOT_RESTORE_SYNC,
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index 735e6d32b8..01fc91a8d0 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -387,10 +387,6 @@ struct ReorderBuffer
MemoryContext tup_context;
XLogRecPtr current_restart_decoding_lsn;
-
- /* buffer for disk<->memory conversions */
- char *outbuf;
- Size outbufsize;
};
diff --git a/src/include/storage/buffile.h b/src/include/storage/buffile.h
index 1fba404fe2..625fa0014c 100644
--- a/src/include/storage/buffile.h
+++ b/src/include/storage/buffile.h
@@ -28,9 +28,13 @@
#include "storage/sharedfileset.h"
-/* BufFile is an opaque type whose details are not known outside buffile.c. */
+/*
+ * BufFile and TransientBufFile are opaque types whose details are not known
+ * outside buffile.c.
+ */
typedef struct BufFile BufFile;
+typedef struct TransientBufFile TransientBufFile;
/*
* prototypes for functions in buffile.c
@@ -51,4 +55,11 @@ extern void BufFileExportShared(BufFile *file);
extern BufFile *BufFileOpenShared(SharedFileSet *fileset, const char *name);
extern void BufFileDeleteShared(SharedFileSet *fileset, const char *name);
+extern TransientBufFile *BufFileOpenTransient(const char *path, int fileFlags);
+extern void BufFileCloseTransient(TransientBufFile *file);
+extern size_t BufFileReadTransient(TransientBufFile *file, void *ptr,
+ size_t size);
+extern size_t BufFileWriteTransient(TransientBufFile *file, void *ptr,
+ size_t size);
+
#endif /* BUFFILE_H */
--
2.13.7
v04-0011-Make-buffile.c-aware-of-encryption.patch (text/x-diff, 38.3K) download
From 98f42482f8e2b78dd8368a35d63944bb83c02293 Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Fri, 5 Jul 2019 16:24:01 +0200
Subject: [PATCH 11/17] Make buffile.c aware of encryption.
This includes temporary files as well as the transient files introduced by
this patch series.
One particular problem encountered here is that the whole multiple of
encryption block needs to be written, backend needs to know how many bytes at
the end of file segment is just padding. The backend can easily remember it
for normal temporary file, but for shared temporary file also this information
needs to be shared. Currently we append 4-byte word at the end of the segment
which informs on length of the useful data. Should this information be shared
via shared memory instead?
No padding would be needed if we used stream cipher, but that's not safe,
README.encryption)
---
src/backend/storage/file/buffile.c | 1047 +++++++++++++++++++++++++++++++++++-
1 file changed, 1021 insertions(+), 26 deletions(-)
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index b31324627a..308ceb01b7 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -44,18 +44,25 @@
#include <unistd.h>
#include "commands/tablespace.h"
+#include "common/sha2.h"
#include "executor/instrument.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "storage/fd.h"
#include "storage/buffile.h"
#include "storage/buf_internals.h"
+#include "storage/encryption.h"
+#include "utils/datetime.h"
#include "utils/resowner.h"
/*
* We break BufFiles into gigabyte-sized segments, regardless of RELSEG_SIZE.
* The reason is that we'd like large BufFiles to be spread across multiple
* tablespaces when available.
+ *
+ * An integer value indicating the number of useful bytes in the segment is
+ * appended to each segment of if the file is both shared and encrypted, see
+ * BufFile.useful.
*/
#define MAX_PHYSICAL_FILESIZE 0x40000000
#define BUFFILE_SEG_SIZE (MAX_PHYSICAL_FILESIZE / BLCKSZ)
@@ -82,6 +89,27 @@ typedef struct BufFileCommon
bool append; /* should new data be appended to the end? */
+ /*
+ * If the file is encrypted, only the whole buffer can be loaded / dumped
+ * --- see BufFileLoadBuffer() for more info --- whether it's space is
+ * used up or not. Therefore we need to keep track of the actual on-disk
+ * size buffer of each component file, as it would be if there was no
+ * encryption.
+ *
+ * List would make coding simpler, however it would not be good for
+ * performance. Random access is important here.
+ */
+ off_t *useful;
+
+ /*
+ * The "useful" array may need to be expanded independent from
+ * extendBufFile() (i.e. earlier than the buffer gets dumped), so store
+ * the number of elements separate from numFiles.
+ *
+ * Always 1 for TransientBufFile.
+ */
+ int nuseful;
+
PGAlignedBlock buffer;
} BufFileCommon;
@@ -98,12 +126,26 @@ struct BufFile
/* all files except the last have length exactly MAX_PHYSICAL_FILESIZE */
File *files; /* palloc'd array with numFiles entries */
+ /*
+ * Segment number is used to compute encryption tweak so we must remember
+ * the original numbers of segments if the file is encrypted and if it was
+ * passed as target to BufFileAppend() at least once. If this field is
+ * NULL, ->curFile is used to compute the tweak.
+ */
+ off_t *segnos;
+
bool isInterXact; /* keep open over transactions? */
SharedFileSet *fileset; /* space for segment files if shared */
const char *name; /* name of this BufFile if shared */
/*
+ * Per-PID identifier if the file is encrypted and not shared. Used for
+ * tweak computation.
+ */
+ uint32 number;
+
+ /*
* resowner is the ResourceOwner to use for underlying temp files. (We
* don't need to remember the memory context we're using explicitly,
* because after creation we only repalloc our arrays larger.)
@@ -130,16 +172,22 @@ static BufFile *makeBufFile(File firstfile);
static void extendBufFile(BufFile *file);
static void BufFileLoadBuffer(BufFile *file);
static void BufFileDumpBuffer(BufFile *file);
+static void BufFileDumpBufferEncrypted(BufFile *file);
static int BufFileFlush(BufFile *file);
static File MakeNewSharedSegment(BufFile *file, int segment);
+static void BufFileTweak(char *tweak, BufFileCommon *file, bool is_transient);
+static void ensureUsefulArraySize(BufFileCommon *file, int required);
+static void BufFileAppendMetadata(BufFile *target, BufFile *source);
+
static void BufFileLoadBufferTransient(TransientBufFile *file);
static void BufFileDumpBufferTransient(TransientBufFile *file);
static size_t BufFileReadCommon(BufFileCommon *file, void *ptr, size_t size,
bool is_transient);
static size_t BufFileWriteCommon(BufFileCommon *file, void *ptr, size_t size,
- bool is_transient);
+ bool is_transient);
+static void BufFileUpdateUsefulLength(BufFileCommon *file, bool is_transient);
/*
* Create BufFile and perform the common initialization.
@@ -160,6 +208,25 @@ makeBufFileCommon(int nfiles)
file->isInterXact = false;
file->resowner = CurrentResourceOwner;
+ if (data_encrypted)
+ {
+ fcommon->useful = (off_t *) palloc0(sizeof(off_t) * nfiles);
+ fcommon->nuseful = nfiles;
+
+ file->segnos = NULL;
+
+ /*
+ * The unused (trailing) part of the buffer should not contain
+ * undefined data: if we encrypt such a buffer and flush it to disk,
+ * the encrypted form of that "undefined part" can get zeroed due to
+ * seek and write beyond EOF. If such a buffer gets loaded and
+ * decrypted, the change of the undefined part to zeroes can affect
+ * the valid part if it does not end at block boundary. By setting the
+ * whole buffer to zeroes we ensure that the unused part of the buffer
+ * always contains zeroes.
+ */
+ MemSet(fcommon->buffer.data, 0, BLCKSZ);
+ }
return file;
}
@@ -205,6 +272,19 @@ extendBufFile(BufFile *file)
file->files = (File *) repalloc(file->files,
(file->numFiles + 1) * sizeof(File));
+
+ if (data_encrypted)
+ {
+ ensureUsefulArraySize(&file->common, file->numFiles + 1);
+
+ if (file->segnos)
+ {
+ file->segnos = (off_t *) repalloc(file->segnos,
+ (file->numFiles + 1) * sizeof(off_t));
+ file->segnos[file->numFiles] = file->numFiles;
+ }
+ }
+
file->files[file->numFiles] = pfile;
file->numFiles++;
}
@@ -227,6 +307,8 @@ BufFileCreateTemp(bool interXact)
BufFile *file;
File pfile;
+ static uint32 counter_temp = 0;
+
/*
* Ensure that temp tablespaces are set up for OpenTemporaryFile to use.
* Possibly the caller will have done this already, but it seems useful to
@@ -242,6 +324,10 @@ BufFileCreateTemp(bool interXact)
Assert(pfile >= 0);
file = makeBufFile(pfile);
+
+ file->number = counter_temp;
+ counter_temp = (counter_temp + 1) % INT_MAX;
+
file->isInterXact = interXact;
return file;
@@ -345,6 +431,7 @@ BufFileOpenShared(SharedFileSet *fileset, const char *name)
files[nfiles] = SharedFileSetOpen(fileset, segment_name);
if (files[nfiles] <= 0)
break;
+
++nfiles;
CHECK_FOR_INTERRUPTS();
@@ -361,7 +448,51 @@ BufFileOpenShared(SharedFileSet *fileset, const char *name)
segment_name, name)));
file = makeBufFileCommon(nfiles);
+
+ /*
+ * Shared encrypted segment should, at its end, contain information on the
+ * number of useful bytes in the last buffer.
+ */
+ if (data_encrypted)
+ {
+ off_t pos;
+ int i;
+
+ for (i = 0; i < nfiles; i++)
+ {
+ int nbytes;
+ File segment = files[i];
+
+ pos = FileSize(segment) - sizeof(off_t);
+
+ /*
+ * The word must immediately follow the last buffer of the
+ * segment.
+ */
+ if (pos <= 0 || pos % BLCKSZ != 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not find padding info in BufFile \"%s\": %m",
+ name)));
+
+ nbytes = FileRead(segment, (char *) &file->common.useful[i],
+ sizeof(off_t), pos, WAIT_EVENT_BUFFILE_READ);
+ if (nbytes != sizeof(off_t))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read padding info from BufFile \"%s\": %m",
+ name)));
+ Assert(file->common.useful[i] > 0);
+
+ CHECK_FOR_INTERRUPTS();
+ }
+ }
+
file->files = files;
+
+ if (data_encrypted)
+ file->common.nuseful = nfiles;
+
file->common.readOnly = true; /* Can't write to files opened this way */
file->fileset = fileset;
file->name = pstrdup(name);
@@ -440,6 +571,14 @@ BufFileClose(BufFile *file)
FileClose(file->files[i]);
/* release the buffer space */
pfree(file->files);
+
+ if (data_encrypted)
+ {
+ if (file->segnos)
+ pfree(file->segnos);
+ pfree(file->common.useful);
+ }
+
pfree(file);
}
@@ -456,6 +595,13 @@ BufFileLoadBuffer(BufFile *file)
File thisfile;
/*
+ * Only whole multiple of ENCRYPTION_BLOCK can be encrypted / decrypted,
+ */
+ Assert((file->common.curOffset % BLCKSZ == 0 &&
+ file->common.curOffset % ENCRYPTION_BLOCK == 0) ||
+ !data_encrypted);
+
+ /*
* Advance to next component file if necessary and possible.
*/
if (file->common.curOffset >= MAX_PHYSICAL_FILESIZE &&
@@ -466,6 +612,20 @@ BufFileLoadBuffer(BufFile *file)
}
/*
+ * See makeBufFileCommon().
+ *
+ * Actually here we only handle the case of FileRead() returning zero
+ * bytes below. In contrast, if the buffer contains any data but it's not
+ * full, it should already have the trailing zeroes (encrypted) on disk.
+ * And as the encrypted buffer is always loaded in its entirety (i.e. EOF
+ * should only appear at buffer boundary if the data is encrypted), all
+ * unused bytes of the buffer should eventually be zeroes after
+ * decryption.
+ */
+ if (data_encrypted)
+ MemSet(file->common.buffer.data, 0, BLCKSZ);
+
+ /*
* Read whatever we can get, up to a full bufferload.
*/
thisfile = file->files[file->common.curFile];
@@ -478,6 +638,54 @@ BufFileLoadBuffer(BufFile *file)
file->common.nbytes = 0;
/* we choose not to advance curOffset here */
+ if (data_encrypted && file->common.nbytes > 0)
+ {
+ char tweak[TWEAK_SIZE];
+ int nbytes = file->common.nbytes;
+
+ /*
+ * The encrypted component file can only consist of whole number of
+ * our encryption units. (Only the whole buffers are dumped / loaded.)
+ * The only exception is that we're at the end of segment file and
+ * found the word indicating the number of useful bytes in the
+ * segment. This can only happen for shared file.
+ */
+ if (nbytes % BLCKSZ != 0)
+ {
+ Assert(nbytes == sizeof(off_t) && file->fileset != NULL);
+
+ /*
+ * This metadata his hidden to caller, so all he needs to know
+ * that there's no real data at the end of the file.
+ */
+ file->common.nbytes = 0;
+ return;
+ }
+
+ /* Decrypt the whole block at once. */
+ BufFileTweak(tweak, &file->common, false);
+ decrypt_block(file->common.buffer.data,
+ file->common.buffer.data,
+ BLCKSZ,
+ tweak,
+ false);
+
+#ifdef USE_ASSERT_CHECKING
+
+ /*
+ * The unused part of the buffer which we've read from disk and
+ * decrypted should only contain zeroes, as explained in front of the
+ * MemSet() call.
+ */
+ {
+ int i;
+
+ for (i = file->common.nbytes; i < BLCKSZ; i++)
+ Assert(file->common.buffer.data[i] == 0);
+ }
+#endif /* USE_ASSERT_CHECKING */
+ }
+
if (file->common.nbytes > 0)
pgBufferUsage.temp_blks_read++;
}
@@ -561,6 +769,169 @@ BufFileDumpBuffer(BufFile *file)
}
/*
+ * BufFileDumpBufferEncrypted
+ *
+ * Encrypt buffer and dump it. The functionality is sufficiently different
+ * from BufFileDumpBuffer to be implemented as a separate function. The most
+ * notable difference is that no loop is needed here.
+ */
+static void
+BufFileDumpBufferEncrypted(BufFile *file)
+{
+ char tweak[TWEAK_SIZE];
+ int bytestowrite;
+ File thisfile;
+
+ /*
+ * Caller's responsibility.
+ */
+ Assert(file->common.pos <= file->common.nbytes);
+
+ /*
+ * See comments in BufFileLoadBuffer();
+ */
+ Assert((file->common.curOffset % BLCKSZ == 0 &&
+ file->common.curOffset % ENCRYPTION_BLOCK == 0));
+
+ /*
+ * Advance to next component file if necessary and possible.
+ */
+ if (file->common.curOffset >= MAX_PHYSICAL_FILESIZE)
+ {
+ while (file->common.curFile + 1 >= file->numFiles)
+ extendBufFile(file);
+ file->common.curFile++;
+ file->common.curOffset = 0L;
+ }
+
+ /*
+ * This condition plus the alignment of curOffset to BLCKSZ (checked
+ * above) ensure that the encrypted buffer never crosses component file
+ * boundary.
+ */
+ StaticAssertStmt((MAX_PHYSICAL_FILESIZE % BLCKSZ) == 0,
+ "BLCKSZ is not whole multiple of MAX_PHYSICAL_FILESIZE");
+
+ /*
+ * Encrypted data is dumped all at once.
+ *
+ * Unlike BufFileDumpBuffer(), we don't have to check here how much bytes
+ * is available in the segment. According to the assertions above,
+ * currOffset should be lower than MAX_PHYSICAL_FILESIZE by non-zero
+ * multiple of BLCKSZ.
+ */
+ bytestowrite = BLCKSZ;
+
+ /*
+ * The amount of data encrypted must be a multiple of ENCRYPTION_BLOCK. We
+ * meet this condition simply by encrypting the whole buffer.
+ *
+ * XXX Alternatively we could encrypt only as few encryption blocks that
+ * encompass file->common.nbyte bytes, but then we'd have to care how many
+ * blocks should be decrypted: decryption of the unencrypted trailing
+ * zeroes produces garbage, which can be a problem if lseek() created
+ * "holes" in the file. Such a hole should be read as a sequence of
+ * zeroes.
+ */
+ BufFileTweak(tweak, &file->common, false);
+
+ encrypt_block(file->common.buffer.data,
+ encrypt_buf.data,
+ BLCKSZ,
+ tweak,
+ false);
+
+ thisfile = file->files[file->common.curFile];
+ bytestowrite = FileWrite(thisfile,
+ encrypt_buf.data,
+ bytestowrite,
+ file->common.curOffset,
+ WAIT_EVENT_BUFFILE_WRITE);
+ if (bytestowrite <= 0 || bytestowrite != BLCKSZ)
+ return; /* failed to write */
+
+ file->common.curOffset += bytestowrite;
+ pgBufferUsage.temp_blks_written++;
+
+ /*
+ * The number of useful bytes needs to be written at the end of each
+ * encrypted segment of a shared file so that the other backends know
+ * how many bytes of the last buffer are useful.
+ */
+ if (file->fileset != NULL)
+ {
+ off_t useful;
+
+ /*
+ * nuseful may be increased earlier than numFiles but not later, so
+ * the corresponding entry should already exist in ->useful.
+ */
+ Assert(file->common.curFile < file->common.nuseful);
+
+ /*
+ * The number of useful bytes in the current segment file.
+ */
+ useful = file->common.useful[file->common.curFile];
+
+ /*
+ * Have we dumped the last buffer of the segment, i.e. the one that
+ * can contain padding?
+ */
+ if (file->common.curOffset >= useful)
+ {
+ int bytes_extra;
+
+ /*
+ * Write the number of useful bytes in the segment.
+ *
+ * Do not increase curOffset afterwards. Thus we ensure that the
+ * next buffer appended will overwrite the "useful" value just
+ * written, instead of being appended to it.
+ */
+ bytes_extra = FileWrite(file->files[file->common.curFile],
+ (char *) &useful,
+ sizeof(useful),
+ file->common.curOffset,
+ WAIT_EVENT_BUFFILE_WRITE);
+ if (bytes_extra != sizeof(useful))
+ return; /* failed to write */
+ }
+ }
+ file->common.dirty = false;
+
+ if (file->common.pos >= BLCKSZ)
+ {
+ Assert(file->common.pos == BLCKSZ);
+
+ /*
+ * curOffset points to the beginning of the next buffer, so just reset
+ * pos and nbytes.
+ */
+ file->common.pos = 0;
+ file->common.nbytes = 0;
+
+ /* See makeBufFile() */
+ MemSet(file->common.buffer.data, 0, BLCKSZ);
+ }
+ else
+ {
+ /*
+ * Move curOffset to the beginning of the just-written buffer and
+ * preserve pos.
+ */
+ file->common.curOffset -= BLCKSZ;
+
+ /*
+ * At least pos bytes should be written even if the first change since
+ * now appears at pos == nbytes, but in fact the whole buffer will be
+ * written regardless pos. This is the price we pay for the choosing
+ * BLCKSZ as the I/O unit for encrypted data.
+ */
+ file->common.nbytes = BLCKSZ;
+ }
+}
+
+/*
* BufFileRead
*
* Like fread() except we assume 1-byte element size.
@@ -592,7 +963,10 @@ BufFileFlush(BufFile *file)
{
if (file->common.dirty)
{
- BufFileDumpBuffer(file);
+ if (!data_encrypted)
+ BufFileDumpBuffer(file);
+ else
+ BufFileDumpBufferEncrypted(file);
if (file->common.dirty)
return EOF;
}
@@ -688,9 +1062,43 @@ BufFileSeek(BufFile *file, int fileno, off_t offset, int whence)
return EOF;
/* Seek is OK! */
file->common.curFile = newFile;
- file->common.curOffset = newOffset;
- file->common.pos = 0;
- file->common.nbytes = 0;
+ if (!data_encrypted)
+ {
+ file->common.curOffset = newOffset;
+ file->common.pos = 0;
+ file->common.nbytes = 0;
+ }
+ else
+ {
+ /*
+ * Offset of an encrypted buffer must be a multiple of BLCKSZ.
+ */
+ file->common.pos = newOffset % BLCKSZ;
+ file->common.curOffset = newOffset - file->common.pos;
+
+ /*
+ * BufFileLoadBuffer() will set nbytes iff it can read something.
+ */
+ file->common.nbytes = 0;
+
+ /*
+ * Load and decrypt the existing part of the buffer.
+ */
+ BufFileLoadBuffer(file);
+ if (file->common.nbytes == 0)
+ {
+ /*
+ * The data requested is not in the file, but this is not an
+ * error.
+ */
+ return 0;
+ }
+
+ /*
+ * The whole buffer should have been loaded.
+ */
+ Assert(file->common.nbytes == BLCKSZ);
+ }
return 0;
}
@@ -721,6 +1129,123 @@ BufFileSeekBlock(BufFile *file, long blknum)
SEEK_SET);
}
+static void
+BufFileTweak(char *tweak, BufFileCommon *file, bool is_transient)
+{
+ off_t block;
+
+ /*
+ * The unused bytes should always be defined.
+ */
+ memset(tweak, 0, TWEAK_SIZE);
+
+ if (!is_transient)
+ {
+ BufFile *tmpfile = (BufFile *) file;
+ pid_t pid;
+ uint32 number;
+ int curFile;
+
+ if (tmpfile->fileset)
+ {
+ pid = tmpfile->fileset->creator_pid;
+ number = tmpfile->fileset->number;
+ }
+ else
+ {
+ pid = MyProcPid;
+ number = tmpfile->number;
+ }
+
+ curFile = file->curFile;
+
+ /*
+ * If the file was produced by BufFileAppend(), we need the original
+ * curFile, as it was used originally for encryption.
+ */
+ if (tmpfile->segnos)
+ curFile = tmpfile->segnos[curFile];
+
+ block = curFile * BUFFILE_SEG_SIZE + file->curOffset / BLCKSZ;
+
+ StaticAssertStmt(sizeof(pid) + sizeof(number) + sizeof(block) <=
+ TWEAK_SIZE,
+ "tweak components do not fit into TWEAK_SIZE");
+
+ /*
+ * The tweak consists of PID of the owning backend (the leader backend
+ * in the case of parallel query processing), number within the PID
+ * and block number.
+ *
+ * XXX Additional flag would be handy to distinguish local file from
+ * shared one. Since there's no more room within TWEAK_SIZE, should we
+ * use the highest bit in one of the existing components (preferably
+ * other than pid so that tweaks of different processes do not become
+ * identical)?
+ */
+ *((pid_t *) tweak) = pid;
+ tweak += sizeof(pid_t);
+ *((uint32 *) tweak) = number;
+ tweak += sizeof(number);
+ *((off_t *) tweak) = block;
+ }
+ else
+ {
+ TransientBufFile *transfile = (TransientBufFile *) file;
+ pg_sha256_ctx sha_ctx;
+ unsigned char sha[PG_SHA256_DIGEST_LENGTH];
+#define BUF_FILE_PATH_HASH_LEN 8
+
+ /*
+ * For transient file we can't use any field of TransientBufFile
+ * because this info gets lost if the file is closed and reopened.
+ * Hash of the file path string is an easy way to get "persistent"
+ * tweak value, however usual hashes do not fit into TWEAK_SIZE. The
+ * hash portion that we can store is actually even smaller because
+ * block number needs to be stored too. Even though we only use part
+ * of the hash, the tweak we finally use for data encryption /
+ * decryption should not be predictable because the tweak we compute
+ * here is further processed, see cbc_essi_preprocess_tweak().
+ */
+ pg_sha256_init(&sha_ctx);
+ pg_sha256_update(&sha_ctx,
+ (uint8 *) transfile->path,
+ strlen(transfile->path));
+ pg_sha256_final(&sha_ctx, sha);
+
+ StaticAssertStmt(BUF_FILE_PATH_HASH_LEN + sizeof(block) <= TWEAK_SIZE,
+ "tweak components do not fit into TWEAK_SIZE");
+ memcpy(tweak, sha, BUF_FILE_PATH_HASH_LEN);
+ tweak += BUF_FILE_PATH_HASH_LEN;
+ block = file->curOffset / BLCKSZ;
+ *((off_t *) tweak) = block;
+ }
+}
+
+/*
+ * Make sure that BufFile.useful array has the required size.
+ */
+static void
+ensureUsefulArraySize(BufFileCommon *file, int required)
+{
+ /*
+ * Does the array already have enough space?
+ */
+ if (required <= file->nuseful)
+ return;
+
+ /*
+ * It shouldn't be possible to jump beyond the end of the last segment,
+ * i.e. skip more than 1 segment.
+ */
+ Assert(file->nuseful + 1 == required);
+
+ file->useful = (off_t *)
+ repalloc(file->useful, required * sizeof(off_t));
+ file->useful[file->nuseful] = 0L;
+ file->nuseful++;
+}
+
#ifdef NOT_USED
/*
* BufFileTellBlock --- block-oriented tell
@@ -752,14 +1277,32 @@ BufFileSize(BufFile *file)
Assert(file->fileset != NULL);
- /* Get the size of the last physical file. */
- lastFileSize = FileSize(file->files[file->numFiles - 1]);
- if (lastFileSize < 0)
- ereport(ERROR,
- (errcode_for_file_access(),
- errmsg("could not determine size of temporary file \"%s\" from BufFile \"%s\": %m",
- FilePathName(file->files[file->numFiles - 1]),
- file->name)));
+ if (!data_encrypted)
+ {
+ /* Get the size of the last physical file. */
+ lastFileSize = FileSize(file->files[file->numFiles - 1]);
+ if (lastFileSize < 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not determine size of temporary file \"%s\" from BufFile \"%s\": %m",
+ FilePathName(file->files[file->numFiles - 1]),
+ file->name)));
+ }
+ else
+ {
+ /*
+ * "useful" should be initialized even for shared file, see
+ * BufFileOpenShared().
+ */
+ Assert(file->common.useful != NULL &&
+ file->common.nuseful >= file->numFiles);
+
+ /*
+ * The number of useful bytes in the segment is what caller is
+ * interested in.
+ */
+ lastFileSize = file->common.useful[file->common.nuseful - 1];
+ }
return ((file->numFiles - 1) * (int64) MAX_PHYSICAL_FILESIZE) +
lastFileSize;
@@ -803,12 +1346,80 @@ BufFileAppend(BufFile *target, BufFile *source)
repalloc(target->files, sizeof(File) * newNumFiles);
for (i = target->numFiles; i < newNumFiles; i++)
target->files[i] = source->files[i - target->numFiles];
+
+ if (data_encrypted)
+ BufFileAppendMetadata(target, source);
+
target->numFiles = newNumFiles;
return startBlock;
}
/*
+ * Add encryption-specific metadata of the source file to the target.
+ */
+static void
+BufFileAppendMetadata(BufFile *target, BufFile *source)
+{
+ int newNumFiles = target->numFiles + source->numFiles;
+ int newNUseful = target->common.nuseful + source->common.nuseful;
+ int i;
+
+ /*
+ * XXX As the typical use case is that parallel workers expose file to the
+ * leader, can we expect both target and source to have been exported,
+ * i.e. flushed? In such a case "nuseful" would have to be equal to
+ * "numFiles" for both input files and the code could get a bit
+ * simpler. It seems that at least source should be flushed, as
+ * source->readOnly is expected to be true above.
+ */
+ target->common.useful = (off_t *)
+ repalloc(target->common.useful, sizeof(off_t) * newNUseful);
+
+ for (i = target->common.nuseful; i < newNUseful; i++)
+ target->common.useful[i] = source->common.useful[i - target->common.nuseful];
+ target->common.nuseful = newNUseful;
+
+ /*
+ * File segments can appear at different position due to concatenation, so
+ * make sure we remember the original positions for the sake of encryption
+ * tweak.
+ */
+ if (target->segnos == NULL)
+ {
+ /*
+ * If the target does not have the array yet, allocate it for both
+ * target and source and initialize the target part.
+ */
+ target->segnos = (off_t *) palloc(newNumFiles * sizeof(off_t));
+ for (i = 0; i < target->numFiles; i++)
+ target->segnos[i] = i;
+ }
+ else
+ {
+ /*
+ * Use the existing target part and add space for the source part.
+ */
+ target->segnos = (off_t *) repalloc(target->segnos,
+ newNumFiles * sizeof(off_t));
+ }
+
+ /*
+ * The source segment number either equals to (0-based) index of the
+ * segment, or to an element of an already existing array.
+ */
+ for (i = target->numFiles; i < newNumFiles; i++)
+ {
+ off_t segno = i - target->numFiles;
+
+ if (source->segnos == NULL)
+ target->segnos[i] = segno;
+ else
+ target->segnos[i] = source->segnos[segno];
+ }
+}
+
+/*
* Open TransientBufFile at given path or create one if it does not
* exist. User will be allowed either to write to the file or to read from it,
* according to fileFlags, but not both.
@@ -829,6 +1440,28 @@ BufFileOpenTransient(const char *path, int fileFlags)
/* Check whether user wants read or write access. */
readOnly = (fileFlags & O_WRONLY) == 0;
+ if (data_encrypted)
+ {
+ /*
+ * In the encryption case, even if user will only be allowed to write,
+ * internally we also need to read, see below.
+ */
+ fileFlags &= ~O_WRONLY;
+ fileFlags |= O_RDWR;
+
+ /*
+ * We can only emulate the append behavior by setting curOffset to
+ * file size because if the underlying file was opened in append mode,
+ * we could not rewrite the old value of file->common.useful[0] with
+ * data.
+ */
+ if (fileFlags & O_APPEND)
+ {
+ append = true;
+ fileFlags &= ~O_APPEND;
+ }
+ }
+
/*
* Append mode for read access is not useful, so don't bother implementing
* it.
@@ -861,6 +1494,8 @@ BufFileOpenTransient(const char *path, int fileFlags)
fcommon->readOnly = readOnly;
fcommon->append = append;
fcommon->curFile = 0;
+ fcommon->useful = (off_t *) palloc0(sizeof(off_t));
+ fcommon->nuseful = 1;
file->path = pstrdup(path);
file->fd = fd;
@@ -881,6 +1516,56 @@ BufFileOpenTransient(const char *path, int fileFlags)
else
fcommon->curOffset = 0L;
+ /*
+ * Encrypted transient file should, at its end, contain information on the
+ * number of useful bytes in the last buffer.
+ */
+ if (data_encrypted)
+ {
+ off_t pos = size;
+ int nbytes;
+
+ /* No metadata in an empty file. */
+ if (pos == 0)
+ return file;
+
+ pos -= sizeof(off_t);
+
+ /*
+ * The word must immediately follow the last buffer of the segment.
+ */
+ if (pos < 0 || pos % BLCKSZ != 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not find padding info in TransientBufFile \"%s\": %m",
+ path)));
+
+ errno = 0;
+ nbytes = pg_pread(file->fd,
+ (char *) &fcommon->useful[0],
+ sizeof(off_t),
+ pos);
+ if (nbytes != sizeof(off_t))
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read padding info from TransientBufFile \"%s\": %m",
+ path)));
+ Assert(fcommon->useful[0] > 0);
+
+ if (fcommon->append)
+ {
+ off_t useful = fcommon->useful[0];
+
+ /*
+ * If new buffer should be added, make sure it will end up
+ * immediately after the last complete one, and also that the next
+ * write position follows the last valid byte.
+ */
+ fcommon->pos = useful % BLCKSZ;
+ fcommon->curOffset = useful - fcommon->pos;
+ }
+ }
+
return file;
}
@@ -912,6 +1597,8 @@ BufFileCloseTransient(TransientBufFile *file)
(errcode_for_file_access(),
errmsg("could not close file \"%s\": %m", file->path)));
+ if (data_encrypted)
+ pfree(file->common.useful);
pfree(file->path);
pfree(file);
}
@@ -928,6 +1615,9 @@ BufFileLoadBufferTransient(TransientBufFile *file)
Assert(!file->common.dirty);
Assert(file->common.pos == 0 && file->common.nbytes == 0);
+ /* See comments in BufFileLoadBuffer(). */
+ if (data_encrypted)
+ MemSet(file->common.buffer.data, 0, BLCKSZ);
retry:
/*
@@ -952,6 +1642,54 @@ retry:
return;
}
/* we choose not to advance offset here */
+
+ if (data_encrypted && file->common.nbytes > 0)
+ {
+ char tweak[TWEAK_SIZE];
+ int nbytes = file->common.nbytes;
+
+ /*
+ * The encrypted file can only consist of whole number of our
+ * encryption units. (Only the whole buffers are dumped / loaded.) The
+ * only exception is that we're at the end of segment file and found
+ * the word indicating the number of useful bytes in the segment. This
+ * can only happen for shared file.
+ */
+ if (nbytes % BLCKSZ != 0)
+ {
+ Assert(nbytes == sizeof(off_t));
+
+ /*
+ * This metadata his hidden to caller, so all he needs to know
+ * that there's no real data at the end of the file.
+ */
+ file->common.nbytes = 0;
+ return;
+ }
+
+ /* Decrypt the whole block at once. */
+ BufFileTweak(tweak, &file->common, true);
+ decrypt_block(file->common.buffer.data,
+ file->common.buffer.data,
+ BLCKSZ,
+ tweak,
+ false);
+
+#ifdef USE_ASSERT_CHECKING
+
+ /*
+ * The unused part of the buffer which we've read from disk and
+ * decrypted should only contain zeroes, as explained in front of the
+ * MemSet() call.
+ */
+ {
+ int i;
+
+ for (i = file->common.nbytes; i < BLCKSZ; i++)
+ Assert(file->common.buffer.data[i] == 0);
+ }
+#endif /* USE_ASSERT_CHECKING */
+ }
}
/*
@@ -960,7 +1698,9 @@ retry:
static void
BufFileDumpBufferTransient(TransientBufFile *file)
{
- int nwritten;
+ int bytestowrite,
+ nwritten;
+ char *write_ptr;
/* This function should only be needed during write access ... */
Assert(!file->common.readOnly);
@@ -969,12 +1709,33 @@ BufFileDumpBufferTransient(TransientBufFile *file)
Assert(file->common.dirty);
Assert(file->common.nbytes > 0);
+ if (!data_encrypted)
+ {
+ write_ptr = file->common.buffer.data;
+ bytestowrite = file->common.nbytes;
+ }
+ else
+ {
+ char tweak[TWEAK_SIZE];
+
+ /*
+ * Encrypt the whole buffer, see comments in BufFileDumpBuffer().
+ */
+ BufFileTweak(tweak, &file->common, true);
+ encrypt_block(file->common.buffer.data,
+ encrypt_buf.data,
+ BLCKSZ,
+ tweak,
+ false);
+ write_ptr = encrypt_buf.data;
+ bytestowrite = BLCKSZ;
+ }
retry:
errno = 0;
pgstat_report_wait_start(WAIT_EVENT_BUFFILE_WRITE);
nwritten = pg_pwrite(file->fd,
- file->common.buffer.data,
- file->common.nbytes,
+ write_ptr,
+ bytestowrite,
file->common.curOffset);
pgstat_report_wait_end();
@@ -993,6 +1754,43 @@ retry:
return; /* failed to write */
}
+ file->common.curOffset += nwritten;
+
+ if (data_encrypted)
+ {
+ off_t useful;
+
+ /*
+ * The number of useful bytes in file.
+ */
+ useful = file->common.useful[0];
+
+ /*
+ * Have we dumped the last buffer of the segment, i.e. the one that
+ * can contain padding?
+ */
+ if (file->common.curOffset >= useful)
+ {
+ int bytes_extra;
+
+ /*
+ * Write the number of useful bytes in the file
+ *
+ * Do not increase curOffset afterwards. Thus we ensure that the
+ * next buffer appended will overwrite the "useful" value just
+ * written, instead of being appended to it.
+ */
+ pgstat_report_wait_start(WAIT_EVENT_BUFFILE_WRITE);
+ bytes_extra = pg_pwrite(file->fd,
+ (char *) &useful,
+ sizeof(useful),
+ file->common.curOffset);
+ pgstat_report_wait_end();
+ if (bytes_extra != sizeof(useful))
+ return; /* failed to write */
+ }
+ }
+
file->common.dirty = false;
file->common.pos = 0;
@@ -1046,21 +1844,110 @@ BufFileReadCommon(BufFileCommon *file, void *ptr, size_t size,
{
if (file->pos >= file->nbytes)
{
- /* Try to load more data into buffer. */
- file->curOffset += file->pos;
- file->pos = 0;
+ /*
+ * Neither read nor write nor seek should leave pos greater than
+ * nbytes, regardless the data is encrypted or not. pos can only
+ * be greater if nbytes is zero --- this situation can be caused
+ * by BufFileSeek().
+ */
+ Assert(file->pos == file->nbytes || file->nbytes == 0);
+
+ /*
+ * The Assert() above implies that pos is a whole multiple of
+ * BLCKSZ, so curOffset has meet the same encryption-specific
+ * requirement too.
+ */
+ Assert(file->curOffset % BLCKSZ == 0 || !data_encrypted);
+
file->nbytes = 0;
+ /* Try to load more data into buffer. */
+ if (!data_encrypted || file->pos % BLCKSZ == 0)
+ {
+ file->curOffset += file->pos;
+ file->pos = 0;
- if (!is_transient)
- BufFileLoadBuffer((BufFile *) file);
+ if (!is_transient)
+ BufFileLoadBuffer((BufFile *) file);
+ else
+ BufFileLoadBufferTransient((TransientBufFile *) file);
+
+ if (file->nbytes <= 0)
+ break; /* no more data available */
+ }
else
- BufFileLoadBufferTransient((TransientBufFile *) file);
+ {
+ int nbytes_orig = file->nbytes;
+
+ /*
+ * Given that BLCKSZ is the I/O unit for encrypted data (see
+ * comments in BufFileDumpBuffer), we cannot add pos to
+ * curOffset because that would make it point outside block
+ * boundary. The only thing we can do is to reload the whole
+ * buffer and see if more data is eventually there than the
+ * previous load has fetched.
+ */
+ if (!is_transient)
+ BufFileLoadBuffer((BufFile *) file);
+ else
+ BufFileLoadBufferTransient((TransientBufFile *) file);
- if (file->nbytes <= 0)
- break; /* no more data available */
+ Assert(file->nbytes >= nbytes_orig);
+ if (file->nbytes == nbytes_orig)
+ break; /* no more data available */
+ }
}
nthistime = file->nbytes - file->pos;
+
+ /*
+ * The buffer can contain trailing zeroes because BLCKSZ is the I/O
+ * unit for encrypted data. These are not available for reading.
+ */
+ if (data_encrypted)
+ {
+ off_t useful = file->useful[file->curFile];
+
+ /*
+ * The criterion is whether the useful data ends within the
+ * currently loaded buffer.
+ */
+ if (useful < file->curOffset + BLCKSZ)
+ {
+ int avail;
+
+ /*
+ * Compute the number of bytes available in the current
+ * buffer.
+ */
+ avail = useful - file->curOffset;
+ Assert(avail >= 0);
+
+ /*
+ * An empty buffer can exist, e.g. after a seek to the end of
+ * the last component file.
+ */
+ if (avail == 0)
+ break;
+
+ /*
+ * Seek beyond the current EOF, which was not followed by
+ * write, could have resulted in position outside the useful
+ * data
+ */
+ if (file->pos > avail)
+ break;
+
+ nthistime = avail - file->pos;
+ Assert(nthistime >= 0);
+
+ /*
+ * Have we reached the end of the valid data?
+ */
+ if (nthistime == 0)
+ break;
+ }
+ }
+
if (nthistime > size)
nthistime = size;
Assert(nthistime > 0);
@@ -1098,7 +1985,12 @@ BufFileWriteCommon(BufFileCommon *file, void *ptr, size_t size,
if (file->dirty)
{
if (!is_transient)
- BufFileDumpBuffer((BufFile *) file);
+ {
+ if (!data_encrypted)
+ BufFileDumpBuffer((BufFile *)file);
+ else
+ BufFileDumpBufferEncrypted((BufFile *) file);
+ }
else
BufFileDumpBufferTransient((TransientBufFile *) file);
@@ -1109,11 +2001,28 @@ BufFileWriteCommon(BufFileCommon *file, void *ptr, size_t size,
{
Assert(!is_transient);
- /* Hmm, went directly from reading to writing? */
+ /*
+ * Hmm, went directly from reading to writing?
+ *
+ * As pos should be exactly BLCKSZ, there is nothing special
+ * to do about data_encrypted, except for zeroing the buffer.
+ */
+ Assert(file->pos == BLCKSZ);
+
file->curOffset += file->pos;
file->pos = 0;
file->nbytes = 0;
+
+ /* See makeBufFile() */
+ if (data_encrypted)
+ MemSet(file->buffer.data, 0, BLCKSZ);
}
+
+ /*
+ * If curOffset changed above, it should still meet the assumption
+ * that buffer is the I/O unit for encrypted data.
+ */
+ Assert(file->curOffset % BLCKSZ == 0 || !data_encrypted);
}
nthistime = BLCKSZ - file->pos;
@@ -1127,6 +2036,14 @@ BufFileWriteCommon(BufFileCommon *file, void *ptr, size_t size,
file->pos += nthistime;
if (file->nbytes < file->pos)
file->nbytes = file->pos;
+
+ /*
+ * Remember how many bytes of the file are valid - the rest is
+ * padding.
+ */
+ if (data_encrypted)
+ BufFileUpdateUsefulLength(file, is_transient);
+
ptr = (void *) ((char *) ptr + nthistime);
size -= nthistime;
nwritten += nthistime;
@@ -1134,3 +2051,81 @@ BufFileWriteCommon(BufFileCommon *file, void *ptr, size_t size,
return nwritten;
}
+
+/*
+ * Update information about the effective length of the file.
+ */
+static void
+BufFileUpdateUsefulLength(BufFileCommon *file, bool is_transient)
+{
+ off_t new_useful;
+ int fileno;
+
+ if (is_transient)
+
+ {
+ /*
+ * Transient file is a single file on the disk, so the whole thing is
+ * pretty simple.
+ */
+ fileno = 0;
+
+ new_useful = file->curOffset + file->pos;
+ if (new_useful > file->useful[fileno])
+ file->useful[fileno] = new_useful;
+ }
+ else
+ {
+ fileno = file->curFile;
+
+ /*
+ * curFile does not necessarily correspond to the offset: it can still
+ * have the initial value if BufFileSeek() skipped the previous file
+ * w/o dumping anything of it. While curFile will be fixed during the
+ * next dump, we need valid fileno now.
+ */
+ if (file->curOffset >= MAX_PHYSICAL_FILESIZE)
+ {
+ /*
+ * Even BufFileSeek() should not allow curOffset to become more
+ * than MAX_PHYSICAL_FILESIZE (if caller passes higher offset,
+ * curFile gets increased instead).
+ */
+ Assert(file->curOffset == MAX_PHYSICAL_FILESIZE);
+
+ fileno++;
+ }
+
+ /*
+ * fileno can now point to a segment that does not exist on disk yet.
+ */
+ ensureUsefulArraySize(file, fileno + 1);
+
+ /*
+ * Update the number of useful bytes in the underlying component file
+ * if we've added any useful data.
+ */
+ new_useful = file->curOffset;
+
+ /*
+ * Make sure the offset is relative to the correct component file
+ * (segment). If the write just crossed segment boundary, curOffset
+ * can still point at the end of the previous segment, and so
+ * new_useful is also relative to the start of that previous
+ * segment. Make sure it's relative to the current (fileno) segment.
+ */
+ if (file->curOffset % MAX_PHYSICAL_FILESIZE == 0)
+ new_useful %= MAX_PHYSICAL_FILESIZE;
+
+ /* Finalize the offset. */
+ new_useful += file->pos;
+
+ /*
+ * Adjust the number of useful bytes in the file if needed. This has
+ * to happen immediately, independent from BufFileDumpBuffer(), so
+ * that BufFileRead() works correctly anytime.
+ */
+ if (new_useful > file->useful[fileno])
+ file->useful[fileno] = new_useful;
+ }
+}
--
2.13.7
v04-0012-Add-tests-for-buffile.c.patch (text/plain, 52.5K) download
From 4d1502a37c329c28977c84281cdec3a000ba30de Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Fri, 5 Jul 2019 16:24:02 +0200
Subject: [PATCH 12/17] Add tests for buffile.c.
Since the encryption adds complexity to the buffile.c, regression tests seem
like a good idea. In order to be able to test corner cases that deal with
segment boudaries, this patch makes the segment size configurable - the tests
would be too expensive with 1GB segments.
The new GUC is marked with DEVELOPER_OPTIONS, DBA is not supposed to use it
during normal operation.
---
src/backend/storage/file/buffile.c | 73 ++---
src/backend/utils/misc/guc.c | 34 +-
src/include/storage/buffile.h | 19 ++
src/test/modules/buffile/Makefile | 21 ++
src/test/modules/buffile/README | 11 +
src/test/modules/buffile/buffile--1.0.sql | 54 ++++
src/test/modules/buffile/buffile.c | 429 ++++++++++++++++++++++++++
src/test/modules/buffile/buffile.control | 5 +
src/test/modules/buffile/expected/test_01.out | 61 ++++
src/test/modules/buffile/expected/test_02.out | 48 +++
src/test/modules/buffile/expected/test_03.out | 27 ++
src/test/modules/buffile/expected/test_04.out | 84 +++++
src/test/modules/buffile/expected/test_05.out | 33 ++
src/test/modules/buffile/expected/test_06.out | 43 +++
src/test/modules/buffile/expected/test_07.out | 41 +++
src/test/modules/buffile/expected/test_08.out | 41 +++
src/test/modules/buffile/expected/test_09.out | 39 +++
src/test/modules/buffile/expected/test_10.out | 76 +++++
src/test/modules/buffile/expected/test_11.out | 34 ++
src/test/modules/buffile/expected/test_12.out | 16 +
src/test/modules/buffile/expected/test_13.out | 86 ++++++
src/test/modules/buffile/sql/test_01.sql | 17 +
src/test/modules/buffile/sql/test_02.sql | 13 +
src/test/modules/buffile/sql/test_03.sql | 7 +
src/test/modules/buffile/sql/test_04.sql | 25 ++
src/test/modules/buffile/sql/test_05.sql | 8 +
src/test/modules/buffile/sql/test_06.sql | 15 +
src/test/modules/buffile/sql/test_07.sql | 13 +
src/test/modules/buffile/sql/test_08.sql | 13 +
src/test/modules/buffile/sql/test_09.sql | 9 +
src/test/modules/buffile/sql/test_10.sql | 25 ++
src/test/modules/buffile/sql/test_11.sql | 9 +
src/test/modules/buffile/sql/test_12.sql | 7 +
src/test/modules/buffile/sql/test_13.sql | 25 ++
34 files changed, 1420 insertions(+), 41 deletions(-)
create mode 100644 src/test/modules/buffile/Makefile
create mode 100644 src/test/modules/buffile/README
create mode 100644 src/test/modules/buffile/buffile--1.0.sql
create mode 100644 src/test/modules/buffile/buffile.c
create mode 100644 src/test/modules/buffile/buffile.control
create mode 100644 src/test/modules/buffile/expected/test_01.out
create mode 100644 src/test/modules/buffile/expected/test_02.out
create mode 100644 src/test/modules/buffile/expected/test_03.out
create mode 100644 src/test/modules/buffile/expected/test_04.out
create mode 100644 src/test/modules/buffile/expected/test_05.out
create mode 100644 src/test/modules/buffile/expected/test_06.out
create mode 100644 src/test/modules/buffile/expected/test_07.out
create mode 100644 src/test/modules/buffile/expected/test_08.out
create mode 100644 src/test/modules/buffile/expected/test_09.out
create mode 100644 src/test/modules/buffile/expected/test_10.out
create mode 100644 src/test/modules/buffile/expected/test_11.out
create mode 100644 src/test/modules/buffile/expected/test_12.out
create mode 100644 src/test/modules/buffile/expected/test_13.out
create mode 100644 src/test/modules/buffile/sql/test_01.sql
create mode 100644 src/test/modules/buffile/sql/test_02.sql
create mode 100644 src/test/modules/buffile/sql/test_03.sql
create mode 100644 src/test/modules/buffile/sql/test_04.sql
create mode 100644 src/test/modules/buffile/sql/test_05.sql
create mode 100644 src/test/modules/buffile/sql/test_06.sql
create mode 100644 src/test/modules/buffile/sql/test_07.sql
create mode 100644 src/test/modules/buffile/sql/test_08.sql
create mode 100644 src/test/modules/buffile/sql/test_09.sql
create mode 100644 src/test/modules/buffile/sql/test_10.sql
create mode 100644 src/test/modules/buffile/sql/test_11.sql
create mode 100644 src/test/modules/buffile/sql/test_12.sql
create mode 100644 src/test/modules/buffile/sql/test_13.sql
diff --git a/src/backend/storage/file/buffile.c b/src/backend/storage/file/buffile.c
index 308ceb01b7..998dabd134 100644
--- a/src/backend/storage/file/buffile.c
+++ b/src/backend/storage/file/buffile.c
@@ -56,16 +56,11 @@
#include "utils/resowner.h"
/*
- * We break BufFiles into gigabyte-sized segments, regardless of RELSEG_SIZE.
- * The reason is that we'd like large BufFiles to be spread across multiple
- * tablespaces when available.
- *
- * An integer value indicating the number of useful bytes in the segment is
- * appended to each segment of if the file is both shared and encrypted, see
- * BufFile.useful.
+ * The functions bellow actually use integer constants so that the size can be
+ * controlled by GUC. This is useful for development and regression tests.
*/
-#define MAX_PHYSICAL_FILESIZE 0x40000000
-#define BUFFILE_SEG_SIZE (MAX_PHYSICAL_FILESIZE / BLCKSZ)
+int buffile_max_filesize = MAX_PHYSICAL_FILESIZE;
+int buffile_seg_blocks = BUFFILE_SEG_BLOCKS(MAX_PHYSICAL_FILESIZE);
/*
* Fields that both BufFile and TransientBufFile structures need. It must be
@@ -123,7 +118,7 @@ struct BufFile
BufFileCommon common; /* Common fields, see above. */
int numFiles; /* number of physical files in set */
- /* all files except the last have length exactly MAX_PHYSICAL_FILESIZE */
+ /* all files except the last have length exactly buffile_max_filesize */
File *files; /* palloc'd array with numFiles entries */
/*
@@ -291,7 +286,7 @@ extendBufFile(BufFile *file)
/*
* Create a BufFile for a new temporary file (which will expand to become
- * multiple temporary files if more than MAX_PHYSICAL_FILESIZE bytes are
+ * multiple temporary files if more than buffile_max_filesize bytes are
* written to it).
*
* If interXact is true, the temp file will not be automatically deleted
@@ -595,7 +590,7 @@ BufFileLoadBuffer(BufFile *file)
File thisfile;
/*
- * Only whole multiple of ENCRYPTION_BLOCK can be encrypted / decrypted,
+ * Only whole multiple of ENCRYPTION_BLOCK can be encrypted / decrypted.
*/
Assert((file->common.curOffset % BLCKSZ == 0 &&
file->common.curOffset % ENCRYPTION_BLOCK == 0) ||
@@ -604,7 +599,7 @@ BufFileLoadBuffer(BufFile *file)
/*
* Advance to next component file if necessary and possible.
*/
- if (file->common.curOffset >= MAX_PHYSICAL_FILESIZE &&
+ if (file->common.curOffset >= buffile_max_filesize &&
file->common.curFile + 1 < file->numFiles)
{
file->common.curFile++;
@@ -715,7 +710,7 @@ BufFileDumpBuffer(BufFile *file)
/*
* Advance to next component file if necessary and possible.
*/
- if (file->common.curOffset >= MAX_PHYSICAL_FILESIZE)
+ if (file->common.curOffset >= buffile_max_filesize)
{
while (file->common.curFile + 1 >= file->numFiles)
extendBufFile(file);
@@ -727,7 +722,7 @@ BufFileDumpBuffer(BufFile *file)
* Determine how much we need to write into this file.
*/
bytestowrite = file->common.nbytes - wpos;
- availbytes = MAX_PHYSICAL_FILESIZE - file->common.curOffset;
+ availbytes = buffile_max_filesize - file->common.curOffset;
if ((off_t) bytestowrite > availbytes)
bytestowrite = (int) availbytes;
@@ -758,7 +753,7 @@ BufFileDumpBuffer(BufFile *file)
{
file->common.curFile--;
Assert(file->common.curFile >= 0);
- file->common.curOffset += MAX_PHYSICAL_FILESIZE;
+ file->common.curOffset += buffile_max_filesize;
}
/*
@@ -796,7 +791,7 @@ BufFileDumpBufferEncrypted(BufFile *file)
/*
* Advance to next component file if necessary and possible.
*/
- if (file->common.curOffset >= MAX_PHYSICAL_FILESIZE)
+ if (file->common.curOffset >= buffile_max_filesize)
{
while (file->common.curFile + 1 >= file->numFiles)
extendBufFile(file);
@@ -809,15 +804,14 @@ BufFileDumpBufferEncrypted(BufFile *file)
* above) ensure that the encrypted buffer never crosses component file
* boundary.
*/
- StaticAssertStmt((MAX_PHYSICAL_FILESIZE % BLCKSZ) == 0,
- "BLCKSZ is not whole multiple of MAX_PHYSICAL_FILESIZE");
+ Assert((buffile_max_filesize % BLCKSZ) == 0);
/*
* Encrypted data is dumped all at once.
*
* Unlike BufFileDumpBuffer(), we don't have to check here how much bytes
* is available in the segment. According to the assertions above,
- * currOffset should be lower than MAX_PHYSICAL_FILESIZE by non-zero
+ * currOffset should be lower than buffile_max_filesize by non-zero
* multiple of BLCKSZ.
*/
bytestowrite = BLCKSZ;
@@ -1021,7 +1015,7 @@ BufFileSeek(BufFile *file, int fileno, off_t offset, int whence)
{
if (--newFile < 0)
return EOF;
- newOffset += MAX_PHYSICAL_FILESIZE;
+ newOffset += buffile_max_filesize;
}
if (newFile == file->common.curFile &&
newOffset >= file->common.curOffset &&
@@ -1050,13 +1044,13 @@ BufFileSeek(BufFile *file, int fileno, off_t offset, int whence)
if (newFile == file->numFiles && newOffset == 0)
{
newFile--;
- newOffset = MAX_PHYSICAL_FILESIZE;
+ newOffset = buffile_max_filesize;
}
- while (newOffset > MAX_PHYSICAL_FILESIZE)
+ while (newOffset > buffile_max_filesize)
{
if (++newFile >= file->numFiles)
return EOF;
- newOffset -= MAX_PHYSICAL_FILESIZE;
+ newOffset -= buffile_max_filesize;
}
if (newFile >= file->numFiles)
return EOF;
@@ -1124,8 +1118,8 @@ int
BufFileSeekBlock(BufFile *file, long blknum)
{
return BufFileSeek(file,
- (int) (blknum / BUFFILE_SEG_SIZE),
- (off_t) (blknum % BUFFILE_SEG_SIZE) * BLCKSZ,
+ (int) (blknum / buffile_seg_blocks),
+ (off_t) (blknum % buffile_seg_blocks) * BLCKSZ,
SEEK_SET);
}
@@ -1166,7 +1160,7 @@ BufFileTweak(char *tweak, BufFileCommon *file, bool is_transient)
if (tmpfile->segnos)
curFile = tmpfile->segnos[curFile];
- block = curFile * BUFFILE_SEG_SIZE + file->curOffset / BLCKSZ;
+ block = curFile * buffile_seg_blocks + file->curOffset / BLCKSZ;
StaticAssertStmt(sizeof(pid) + sizeof(number) + sizeof(block) <=
TWEAK_SIZE,
@@ -1258,7 +1252,7 @@ BufFileTellBlock(BufFile *file)
long blknum;
blknum = (file->common.curOffset + file->common.pos) / BLCKSZ;
- blknum += file->common.curFile * BUFFILE_SEG_SIZE;
+ blknum += file->common.curFile * buffile_seg_blocks;
return blknum;
}
@@ -1304,7 +1298,7 @@ BufFileSize(BufFile *file)
lastFileSize = file->common.useful[file->common.nuseful - 1];
}
- return ((file->numFiles - 1) * (int64) MAX_PHYSICAL_FILESIZE) +
+ return ((file->numFiles - 1) * (int64) buffile_max_filesize) +
lastFileSize;
}
@@ -1317,11 +1311,10 @@ BufFileSize(BufFile *file)
* called here first. Resource owners for source and target must match,
* too.
*
- * This operation works by manipulating lists of segment files, so the
- * file content is always appended at a MAX_PHYSICAL_FILESIZE-aligned
- * boundary, typically creating empty holes before the boundary. These
- * areas do not contain any interesting data, and cannot be read from by
- * caller.
+ * This operation works by manipulating lists of segment files, so the file
+ * content is always appended at a buffile_max_filesize-aligned boundary,
+ * typically creating empty holes before the boundary. These areas do not
+ * contain any interesting data, and cannot be read from by caller.
*
* Returns the block number within target where the contents of source
* begins. Caller should apply this as an offset when working off block
@@ -1330,7 +1323,7 @@ BufFileSize(BufFile *file)
long
BufFileAppend(BufFile *target, BufFile *source)
{
- long startBlock = target->numFiles * BUFFILE_SEG_SIZE;
+ long startBlock = target->numFiles * buffile_seg_blocks;
int newNumFiles = target->numFiles + source->numFiles;
int i;
@@ -2084,14 +2077,14 @@ BufFileUpdateUsefulLength(BufFileCommon *file, bool is_transient)
* w/o dumping anything of it. While curFile will be fixed during the
* next dump, we need valid fileno now.
*/
- if (file->curOffset >= MAX_PHYSICAL_FILESIZE)
+ if (file->curOffset >= buffile_max_filesize)
{
/*
* Even BufFileSeek() should not allow curOffset to become more
- * than MAX_PHYSICAL_FILESIZE (if caller passes higher offset,
+ * than buffile_max_filesize (if caller passes higher offset,
* curFile gets increased instead).
*/
- Assert(file->curOffset == MAX_PHYSICAL_FILESIZE);
+ Assert(file->curOffset == buffile_max_filesize);
fileno++;
}
@@ -2114,8 +2107,8 @@ BufFileUpdateUsefulLength(BufFileCommon *file, bool is_transient)
* new_useful is also relative to the start of that previous
* segment. Make sure it's relative to the current (fileno) segment.
*/
- if (file->curOffset % MAX_PHYSICAL_FILESIZE == 0)
- new_useful %= MAX_PHYSICAL_FILESIZE;
+ if (file->curOffset % buffile_max_filesize == 0)
+ new_useful %= buffile_max_filesize;
/* Finalize the offset. */
new_useful += file->pos;
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 440120635e..879c8fd816 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -70,9 +70,10 @@
#include "replication/syncrep.h"
#include "replication/walreceiver.h"
#include "replication/walsender.h"
+#include "storage/buffile.h"
#include "storage/bufmgr.h"
-#include "storage/encryption.h"
#include "storage/dsm_impl.h"
+#include "storage/encryption.h"
#include "storage/standby.h"
#include "storage/fd.h"
#include "storage/large_object.h"
@@ -189,6 +190,8 @@ static const char *show_tcp_keepalives_idle(void);
static const char *show_tcp_keepalives_interval(void);
static const char *show_tcp_keepalives_count(void);
static const char *show_tcp_user_timeout(void);
+static bool check_buffile_max_filesize(int *newval, void **extra, GucSource source);
+static void assign_buffile_max_filesize(int newval, void *extra);
static bool check_maxconnections(int *newval, void **extra, GucSource source);
static bool check_max_worker_processes(int *newval, void **extra, GucSource source);
static bool check_autovacuum_max_workers(int *newval, void **extra, GucSource source);
@@ -3209,6 +3212,18 @@ static struct config_int ConfigureNamesInt[] =
NULL, assign_tcp_user_timeout, show_tcp_user_timeout
},
+ {
+ /* Not for general use */
+ {"buffile_max_filesize", PGC_SUSET, DEVELOPER_OPTIONS,
+ gettext_noop("Maximum size of BufFile segment."),
+ gettext_noop("This makes testing of some corner cases easier, especially when read or write crosses segment boundary."),
+ GUC_NOT_IN_SAMPLE | GUC_UNIT_BYTE
+ },
+ &buffile_max_filesize,
+ MAX_PHYSICAL_FILESIZE, BLCKSZ, MAX_PHYSICAL_FILESIZE,
+ check_buffile_max_filesize, assign_buffile_max_filesize, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, 0, 0, 0, NULL, NULL, NULL
@@ -11301,6 +11316,23 @@ show_tcp_user_timeout(void)
}
static bool
+check_buffile_max_filesize(int *newval, void **extra, GucSource source)
+{
+ if (*newval % BLCKSZ != 0)
+ {
+ GUC_check_errdetail("The value must be whole multiple of BLCKSZ.");
+ return false;
+ }
+ return true;
+}
+
+static void
+assign_buffile_max_filesize(int newval, void *extra)
+{
+ buffile_seg_blocks = BUFFILE_SEG_BLOCKS(newval);
+}
+
+static bool
check_maxconnections(int *newval, void **extra, GucSource source)
{
if (*newval + autovacuum_max_workers + 1 +
diff --git a/src/include/storage/buffile.h b/src/include/storage/buffile.h
index 625fa0014c..977942f4b3 100644
--- a/src/include/storage/buffile.h
+++ b/src/include/storage/buffile.h
@@ -37,6 +37,25 @@ typedef struct BufFile BufFile;
typedef struct TransientBufFile TransientBufFile;
/*
+ * We break BufFiles into gigabyte-sized segments, regardless of RELSEG_SIZE.
+ * The reason is that we'd like large BufFiles to be spread across multiple
+ * tablespaces when available.
+ *
+ * An integer value indicating the number of useful bytes in the segment is
+ * appended to each segment of if the file is both shared and encrypted, see
+ * BufFile.useful.
+ */
+#define MAX_PHYSICAL_FILESIZE 0x40000000
+
+/* Express segment size in the number of blocks. */
+#define BUFFILE_SEG_BLOCKS(phys) ((phys) / BLCKSZ)
+
+/* GUC to control size of the file segment. */
+extern int buffile_max_filesize;
+/* Segment size in blocks, derived from the above. */
+extern int buffile_seg_blocks;
+
+/*
* prototypes for functions in buffile.c
*/
diff --git a/src/test/modules/buffile/Makefile b/src/test/modules/buffile/Makefile
new file mode 100644
index 0000000000..f0f163c45e
--- /dev/null
+++ b/src/test/modules/buffile/Makefile
@@ -0,0 +1,21 @@
+PG_CONFIG ?= pg_config
+MODULE_big = buffile_test
+OBJS = buffile.o $(WIN32RES)
+PGFILEDESC = "buffile_test"
+
+EXTENSION = buffile
+DATA = buffile--1.0.sql
+
+REGRESS = test_01 test_02 test_03 test_04 test_05 test_06 test_07 test_08 \
+test_09 test_10 test_11 test_12 test_13
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/buffile
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/buffile/README b/src/test/modules/buffile/README
new file mode 100644
index 0000000000..9c80787797
--- /dev/null
+++ b/src/test/modules/buffile/README
@@ -0,0 +1,11 @@
+This extension was written to check if changes introduced due to cluster
+encryption do not break buffile.c.
+
+Caution: To make the test cheaper, it was decided to adjust the segment sizeq,
+see
+
+#define MAX_PHYSICAL_FILESIZE (4 * BLCKSZ)
+
+in buffile.c. BLCKSZ is 8192 (the default). All the tests rely on this
+value. So if you haven't compiled Postgres with this value, the tests will
+create 1 GB files and they will fail.
diff --git a/src/test/modules/buffile/buffile--1.0.sql b/src/test/modules/buffile/buffile--1.0.sql
new file mode 100644
index 0000000000..7c168f3514
--- /dev/null
+++ b/src/test/modules/buffile/buffile--1.0.sql
@@ -0,0 +1,54 @@
+CREATE FUNCTION buffile_create()
+RETURNS void
+AS 'MODULE_PATHNAME', 'buffile_create'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_close()
+RETURNS void
+AS 'MODULE_PATHNAME', 'buffile_close'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_write(text)
+RETURNS bigint
+AS 'MODULE_PATHNAME', 'buffile_write'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_read(bigint)
+RETURNS bytea
+AS 'MODULE_PATHNAME', 'buffile_read'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_seek(int, bigint)
+RETURNS int
+AS 'MODULE_PATHNAME', 'buffile_seek'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_assert_fileno(int)
+RETURNS void
+AS 'MODULE_PATHNAME', 'buffile_assert_fileno'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_test_shared()
+RETURNS void
+AS 'MODULE_PATHNAME', 'buffile_test_shared'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_test_shared_append()
+RETURNS void
+AS 'MODULE_PATHNAME', 'buffile_test_shared_append'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_open_transient(text, bool, bool)
+RETURNS void
+AS 'MODULE_PATHNAME', 'buffile_open_transient'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_close_transient()
+RETURNS void
+AS 'MODULE_PATHNAME', 'buffile_close_transient'
+LANGUAGE C;
+
+CREATE FUNCTION buffile_delete_file(text)
+RETURNS void
+AS 'MODULE_PATHNAME', 'buffile_delete_file'
+LANGUAGE C;
diff --git a/src/test/modules/buffile/buffile.c b/src/test/modules/buffile/buffile.c
new file mode 100644
index 0000000000..fe0680771a
--- /dev/null
+++ b/src/test/modules/buffile/buffile.c
@@ -0,0 +1,429 @@
+#include <fcntl.h>
+
+#include "postgres.h"
+#include "fmgr.h"
+#include "lib/stringinfo.h"
+#include "storage/buffile.h"
+#include "utils/array.h"
+#include "utils/builtins.h"
+#include "utils/lsyscache.h"
+#include "utils/memutils.h"
+#include "utils/resowner.h"
+
+PG_MODULE_MAGIC;
+
+/*
+ * To cope with files that span multiple segments w/o wasting resources, use
+ * the smallest possible segment size. The test scripts need to set
+ * buffile_max_filesize (GUC) accordingly.
+ */
+#define MAX_PHYSICAL_FILESIZE_TEST (4 * BLCKSZ)
+
+static BufFile *bf = NULL;
+static TransientBufFile *bft = NULL;
+
+static void check_file(void);
+
+extern Datum buffile_create_transient(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_create);
+Datum
+buffile_create(PG_FUNCTION_ARGS)
+{
+ MemoryContext old_cxt;
+ ResourceOwner old_ro;
+
+ if (bf != NULL)
+ elog(ERROR, "file already exists");
+
+ old_cxt = MemoryContextSwitchTo(TopMemoryContext);
+
+ /*
+ * Make sure the file is not deleted across function calls.
+ */
+ old_ro = CurrentResourceOwner;
+ CurrentResourceOwner = TopTransactionResourceOwner;
+
+ bf = BufFileCreateTemp(false);
+
+ CurrentResourceOwner = old_ro;
+ MemoryContextSwitchTo(old_cxt);
+
+ PG_RETURN_VOID();
+}
+
+extern Datum buffile_close(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_close);
+Datum
+buffile_close(PG_FUNCTION_ARGS)
+{
+ if (bf == NULL)
+ elog(ERROR, "there's no file to close");
+
+ BufFileClose(bf);
+ bf = NULL;
+
+ PG_RETURN_VOID();
+}
+
+extern Datum buffile_write(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_write);
+Datum
+buffile_write(PG_FUNCTION_ARGS)
+{
+ Datum d = PG_GETARG_DATUM(0);
+ char *s = TextDatumGetCString(d);
+ size_t res;
+
+ if (bf)
+ res = BufFileWrite(bf, s, strlen(s));
+ else if (bft)
+ res = BufFileWriteTransient(bft, s, strlen(s));
+ else
+ elog(ERROR, "No file is open");
+
+ PG_RETURN_INT64(res);
+}
+
+extern Datum buffile_read(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_read);
+Datum
+buffile_read(PG_FUNCTION_ARGS)
+{
+ int64 size = PG_GETARG_INT64(0);
+ StringInfo buf = makeStringInfo();
+ size_t res_size;
+ bytea *result;
+
+ enlargeStringInfo(buf, size);
+
+ if (bf)
+ res_size = BufFileRead(bf, buf->data, size);
+ else if (bft)
+ res_size = BufFileReadTransient(bft, buf->data, size);
+ else
+ elog(ERROR, "No file is open");
+
+ buf->len = res_size;
+
+ result = DatumGetByteaPP(DirectFunctionCall1(bytearecv,
+ PointerGetDatum(buf)));
+ PG_RETURN_BYTEA_P(result);
+}
+
+extern Datum buffile_seek(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_seek);
+Datum
+buffile_seek(PG_FUNCTION_ARGS)
+{
+ int32 fileno = PG_GETARG_INT32(0);
+ int64 offset = PG_GETARG_INT64(1);
+ int32 res;
+
+ check_file();
+ res = BufFileSeek(bf, fileno, offset, SEEK_SET);
+
+ PG_RETURN_INT32(res);
+}
+
+extern Datum buffile_assert_fileno(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_assert_fileno);
+Datum
+buffile_assert_fileno(PG_FUNCTION_ARGS)
+{
+ int32 fileno_expected = PG_GETARG_INT32(0);
+ int32 fileno;
+ off_t offset;
+
+ check_file();
+ BufFileTell(bf, &fileno, &offset);
+
+ if (fileno != fileno_expected)
+ {
+ /*
+ * Bring the backend down so that the following tests have no chance
+ * to create the 1GB files.
+ */
+ elog(FATAL, "file number does not match");
+ }
+
+ PG_RETURN_VOID();
+}
+
+static void
+check_file(void)
+{
+ if (bf == NULL)
+ elog(ERROR, "the file is not opened");
+}
+
+/*
+ * This test is especially important for shared encrypted files, see the
+ * comments below.
+ */
+extern Datum buffile_test_shared(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_test_shared);
+Datum
+buffile_test_shared(PG_FUNCTION_ARGS)
+{
+ dsm_segment *seg;
+ SharedFileSet *fileset;
+ BufFile *bf_1,
+ *bf_2;
+ char *data_1,
+ *data_2,
+ *data;
+ Size chunk_size_1,
+ chunk_size_2;
+ int fileno,
+ i;
+ off_t offset,
+ res,
+ total_size;
+
+ /*
+ * The size is not important, we actually do not need the shared memory.
+ * The segment is only needed to initialize the fileset.
+ */
+ seg = dsm_create(1024, 0);
+
+ /*
+ * The fileset must survive error handling, so that dsm_detach works fine.
+ * (The typical use case is that the fileset is in shared memory.)
+ */
+ fileset = (SharedFileSet *) MemoryContextAlloc(TopTransactionContext,
+ sizeof(SharedFileSet));
+ SharedFileSetInit(fileset, seg);
+
+ bf_1 = BufFileCreateShared(fileset, "file_1");
+
+ /*
+ * Write more data than the buffer size, so that we can check that the
+ * number of "useful bytes" word is only appended at the end of the
+ * segment, not after each buffer.
+ */
+ chunk_size_1 = BLCKSZ + 256;
+ data_1 = (char *) palloc(chunk_size_1);
+ memset(data_1, 1, chunk_size_1);
+ if (BufFileWrite(bf_1, data_1, chunk_size_1) != chunk_size_1)
+ elog(ERROR, "Failed to write data");
+ pfree(data_1);
+
+ /*
+ * Enforce buffer flush (The BufFileFlush() function is not exported).
+ * Thus the "useful bytes" metadata should appear at the current end the
+ * first file segment. The next write will have to seek back to overwrite
+ * the metadata.
+ */
+ BufFileTell(bf_1, &fileno, &offset);
+ if (BufFileSeek(bf_1, 0, 0, SEEK_SET) != 0)
+ elog(ERROR, "seek failed");
+ if (BufFileSeek(bf_1, fileno, offset, SEEK_SET) != 0)
+ elog(ERROR, "seek failed");
+
+ /*
+ * Write another chunk that does not fit into the first segment file. Thus
+ * the "useful bytes" metadata should appear at the end of both segments.
+ */
+ chunk_size_2 = 3 * BLCKSZ;
+ data_2 = (char *) palloc(chunk_size_2);
+ memset(data_2, 1, chunk_size_2);
+ if (BufFileWrite(bf_1, data_2, chunk_size_2) != chunk_size_2)
+ elog(ERROR, "Failed to write data");
+ pfree(data_2);
+ BufFileClose(bf_1);
+
+ /*
+ * The word indicating the number of "useful bytes" (i.e. the actual data
+ * w/o padding to buffer size) is stored at the end of each segment file.
+ * Check that this metadata is read correctly.
+ */
+ bf_2 = BufFileOpenShared(fileset, "file_1");
+ total_size = BufFileSize(bf_2);
+ if (total_size != (chunk_size_1 + chunk_size_2))
+ elog(ERROR, "Incorrect file size: %zu", total_size);
+
+ data = (char *) palloc(total_size);
+ res = BufFileRead(bf_2, data, total_size);
+ if (res != total_size)
+ elog(ERROR, "Incorrect chunk size read: %zu", res);
+ for (i = 0; i < total_size; i++)
+ if (data[i] != 1)
+ elog(ERROR, "Unexpected data read from the file");
+ pfree(data);
+ BufFileClose(bf_2);
+
+ dsm_detach(seg);
+
+ PG_RETURN_VOID();
+}
+
+
+/*
+ * Test BufFileAppend().
+ */
+extern Datum buffile_test_shared_append(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_test_shared_append);
+Datum
+buffile_test_shared_append(PG_FUNCTION_ARGS)
+{
+ dsm_segment *seg;
+ SharedFileSet *fileset;
+ BufFile *bf_1,
+ *bf_2,
+ *bf_3;
+ char *data;
+ Size chunk_size;
+ int i;
+ off_t res,
+ total_size;
+
+ seg = dsm_create(1024, 0);
+
+ fileset = (SharedFileSet *) MemoryContextAlloc(TopTransactionContext,
+ sizeof(SharedFileSet));
+ SharedFileSetInit(fileset, seg);
+
+ /*
+ * XXX Does the chunk size matter much?
+ */
+ chunk_size = 8;
+ data = (char *) palloc(chunk_size);
+ memset(data, 1, chunk_size);
+
+ bf_1 = BufFileCreateShared(fileset, "file_1");
+ if (BufFileWrite(bf_1, data, chunk_size) != chunk_size)
+ elog(ERROR, "Failed to write data");
+
+ bf_2 = BufFileCreateShared(fileset, "file_2");
+ if (BufFileWrite(bf_2, data, chunk_size) != chunk_size)
+ elog(ERROR, "Failed to write data");
+
+ /*
+ * Make sure it's read-only so that BufFileAppend() can accept it as
+ * source.
+ */
+ BufFileClose(bf_2);
+ bf_2 = BufFileOpenShared(fileset, "file_2");
+
+ bf_3 = BufFileCreateShared(fileset, "file_3");
+ if (BufFileWrite(bf_3, data, chunk_size) != chunk_size)
+ elog(ERROR, "Failed to write data");
+ BufFileClose(bf_3);
+ bf_3 = BufFileOpenShared(fileset, "file_3");
+
+ BufFileAppend(bf_1, bf_2);
+ BufFileAppend(bf_1, bf_3);
+
+ total_size = BufFileSize(bf_1);
+
+ /*
+ * The result should contain complete segments of bf_1 and bf_2 and the
+ * valid part of bf_3.
+ */
+ if (total_size != (2 * MAX_PHYSICAL_FILESIZE_TEST + chunk_size))
+ elog(ERROR, "Incorrect total size of the appended data: %zu",
+ total_size);
+
+ /*
+ * Check that data of the 2nd segment was decrypted correctly.
+ */
+ if (BufFileSeek(bf_1, 1, 0, SEEK_SET) != 0)
+ elog(ERROR, "seek failed");
+ res = BufFileRead(bf_1, data, chunk_size);
+ if (res != chunk_size)
+ elog(ERROR, "Incorrect chunk size read: %zu", res);
+ for (i = 0; i < chunk_size; i++)
+ if (data[i] != 1)
+ elog(ERROR, "Unexpected data read from the file");
+
+ /*
+ * And the same for the 3rd segment.
+ *
+ * TODO Reuse the code above by putting it into a function.
+ */
+ if (BufFileSeek(bf_1, 2, 0, SEEK_SET) != 0)
+ elog(ERROR, "seek failed");
+ res = BufFileRead(bf_1, data, chunk_size);
+ if (res != chunk_size)
+ elog(ERROR, "Incorrect chunk size read: %zu", res);
+ for (i = 0; i < chunk_size; i++)
+ if (data[i] != 1)
+ elog(ERROR, "Unexpected data read from the file");
+
+ BufFileClose(bf_1);
+ dsm_detach(seg);
+ PG_RETURN_VOID();
+}
+
+extern Datum buffile_open_transient(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_open_transient);
+Datum
+buffile_open_transient(PG_FUNCTION_ARGS)
+{
+ MemoryContext old_cxt;
+ Datum d = PG_GETARG_DATUM(0);
+ char *path = TextDatumGetCString(d);
+ bool write_only = PG_GETARG_BOOL(1);
+ bool append = PG_GETARG_BOOL(2);
+ int flags = O_CREAT | PG_BINARY;
+
+ if (bft != NULL)
+ elog(ERROR, "file already exists");
+
+ if (write_only)
+ flags |= O_WRONLY;
+ if (append)
+ flags |= O_APPEND;
+
+ old_cxt = MemoryContextSwitchTo(TopMemoryContext);
+
+ /*
+ * Make sure the file is not deleted across function calls.
+ */
+ bft = BufFileOpenTransient(path, flags);
+
+ MemoryContextSwitchTo(old_cxt);
+
+ PG_RETURN_VOID();
+}
+
+extern Datum buffile_close_transient(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_close_transient);
+Datum
+buffile_close_transient(PG_FUNCTION_ARGS)
+{
+ if (bft == NULL)
+ elog(ERROR, "there's no file to close");
+
+ BufFileCloseTransient(bft);
+ bft = NULL;
+
+ PG_RETURN_VOID();
+}
+
+extern Datum buffile_delete_file(PG_FUNCTION_ARGS);
+
+PG_FUNCTION_INFO_V1(buffile_delete_file);
+Datum
+buffile_delete_file(PG_FUNCTION_ARGS)
+{
+ Datum d = PG_GETARG_DATUM(0);
+ char *path = TextDatumGetCString(d);
+
+ if (bft != NULL)
+ elog(ERROR, "the file is still open");
+
+ PathNameDeleteTemporaryFile(path, true);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/test/modules/buffile/buffile.control b/src/test/modules/buffile/buffile.control
new file mode 100644
index 0000000000..8472c5a348
--- /dev/null
+++ b/src/test/modules/buffile/buffile.control
@@ -0,0 +1,5 @@
+# buffile_test extension
+comment = 'buffile_test'
+default_version = '1.0'
+module_pathname = '$libdir/buffile_test'
+relocatable = true
diff --git a/src/test/modules/buffile/expected/test_01.out b/src/test/modules/buffile/expected/test_01.out
new file mode 100644
index 0000000000..c1a4f5ac7b
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_01.out
@@ -0,0 +1,61 @@
+-- This is the first test, so make sure the test extension is there.
+CREATE EXTENSION IF NOT EXISTS buffile;
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+SELECT buffile_seek(0, 1);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_write('abc');
+ buffile_write
+---------------
+ 3
+(1 row)
+
+SELECT buffile_seek(0, 0);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+-- Check that the trailing zeroes are not fetched.
+SELECT buffile_read(16);
+ buffile_read
+--------------
+ \x00616263
+(1 row)
+
+-- Adjust the number of useful bytes.
+SELECT buffile_write('abc');
+ buffile_write
+---------------
+ 3
+(1 row)
+
+-- ... and check again.
+SELECT buffile_seek(0, 0);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(16);
+ buffile_read
+------------------
+ \x00616263616263
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_02.out b/src/test/modules/buffile/expected/test_02.out
new file mode 100644
index 0000000000..f783d0cb24
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_02.out
@@ -0,0 +1,48 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+SELECT buffile_seek(0, 8189);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+-- Initialize the last 3 positions of the first buffer and the initial 3
+-- positions of the 2nd buffer.
+SELECT buffile_write('abcdef');
+ buffile_write
+---------------
+ 6
+(1 row)
+
+SELECT buffile_seek(0, 0);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+-- Read the first buffer.
+SELECT length(buffile_read(8192));
+ length
+--------
+ 8192
+(1 row)
+
+-- Only 3 bytes of the 2nd buffer should be fetched.
+SELECT length(buffile_read(8192));
+ length
+--------
+ 3
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_03.out b/src/test/modules/buffile/expected/test_03.out
new file mode 100644
index 0000000000..e899fa3b38
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_03.out
@@ -0,0 +1,27 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+-- Read from an empty file.
+SELECT buffile_seek(0, 8);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(16);
+ buffile_read
+--------------
+ \x
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_04.out b/src/test/modules/buffile/expected/test_04.out
new file mode 100644
index 0000000000..1f8eeabe48
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_04.out
@@ -0,0 +1,84 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+-- Write something near the end of the first buffer, but leave some trailing
+-- space.
+SELECT buffile_seek(0, 8184);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_write('abcd');
+ buffile_write
+---------------
+ 4
+(1 row)
+
+-- Leave the 2nd buffer empty, as well as a few leading bytes. Thus we should
+-- get a hole that spans the whole 2nd buffer as well as a few adjacent bytes
+-- on each side.
+SELECT buffile_seek(0, 2 * 8192 + 4);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_write('efgh');
+ buffile_write
+---------------
+ 4
+(1 row)
+
+-- Check the initial part of the hole, which crosses the boundary of the 1st
+-- and the 2nd buffer.
+SELECT buffile_seek(0, 8184);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(16);
+ buffile_read
+------------------------------------
+ \x61626364000000000000000000000000
+(1 row)
+
+-- Check the trailing part of the whole, which crosses the boundary of the 2nd
+-- and the 3rd buffer.
+SELECT buffile_seek(0, 2 * 8192 - 8);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(16);
+ buffile_read
+------------------------------------
+ \x00000000000000000000000065666768
+(1 row)
+
+-- Check that the hole contains nothing but zeroes.
+SELECT buffile_seek(0, 8192 - 4);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT btrim(buffile_read(8192 + 8), '\x00');
+ btrim
+-------
+ \x
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_05.out b/src/test/modules/buffile/expected/test_05.out
new file mode 100644
index 0000000000..6a73147711
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_05.out
@@ -0,0 +1,33 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+-- Seek does not extend the file if it's not followed by write.
+SELECT buffile_seek(0, 1);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_seek(0, 0);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(2);
+ buffile_read
+--------------
+ \x
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_06.out b/src/test/modules/buffile/expected/test_06.out
new file mode 100644
index 0000000000..d406bda9e6
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_06.out
@@ -0,0 +1,43 @@
+-- This test shows that the if first component file (segment) stays empty,
+-- read stops prematurely even if it starts within that segment, although it'd
+-- otherwise receive some data from the following one.
+BEGIN;
+-- Neither disk space nor time needs to be wasted.
+SET buffile_max_filesize TO 8192;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+SELECT buffile_seek(0, 8192);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_write('a');
+ buffile_write
+---------------
+ 1
+(1 row)
+
+SELECT buffile_seek(0, 8191);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(2);
+ buffile_read
+--------------
+ \x
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_07.out b/src/test/modules/buffile/expected/test_07.out
new file mode 100644
index 0000000000..6f2ee3d2d9
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_07.out
@@ -0,0 +1,41 @@
+BEGIN;
+-- Use a small segment, not to waste disk space and time.
+SET buffile_max_filesize TO 8192;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+-- Write data at component file boundary and try to read it.
+SELECT buffile_seek(0, 8192);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_write('abcd');
+ buffile_write
+---------------
+ 4
+(1 row)
+
+SELECT buffile_seek(0, 8192);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(8);
+ buffile_read
+--------------
+ \x61626364
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_08.out b/src/test/modules/buffile/expected/test_08.out
new file mode 100644
index 0000000000..6ffcb9dcd6
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_08.out
@@ -0,0 +1,41 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+-- Neither disk space nor time needs to be wasted.
+SET buffile_max_filesize TO 8192;
+-- Write data across component file boundary and try to read it.
+SELECT buffile_seek(0, 8190);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_write('abcd');
+ buffile_write
+---------------
+ 4
+(1 row)
+
+SELECT buffile_seek(0, 8190);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(8);
+ buffile_read
+--------------
+ \x61626364
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_09.out b/src/test/modules/buffile/expected/test_09.out
new file mode 100644
index 0000000000..b9d325b676
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_09.out
@@ -0,0 +1,39 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+-- Write data across buffer boundary and try to read it.
+SELECT buffile_seek(0, 8190);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_write('abcd');
+ buffile_write
+---------------
+ 4
+(1 row)
+
+SELECT buffile_seek(0, 8190);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(8);
+ buffile_read
+--------------
+ \x61626364
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_10.out b/src/test/modules/buffile/expected/test_10.out
new file mode 100644
index 0000000000..8e457fdcda
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_10.out
@@ -0,0 +1,76 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+-- Write some data at the end of the buffer.
+SELECT buffile_seek(0, 8188);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_write('abcd');
+ buffile_write
+---------------
+ 4
+(1 row)
+
+SELECT buffile_seek(0, 8189);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+-- Enforce flush with the write position not at the end of the buffer. This is
+-- special by not moving curOffset to the next buffer.
+SELECT buffile_read(1);
+ buffile_read
+--------------
+ \x62
+(1 row)
+
+-- Therefore the next writes should eventually affect the original data. (Here
+-- we also test going directly from read to write and vice versa.)
+SELECT buffile_write('x');
+ buffile_write
+---------------
+ 1
+(1 row)
+
+SELECT buffile_read(1);
+ buffile_read
+--------------
+ \x64
+(1 row)
+
+-- Start a new buffer, i.e. force flushing of the previous one.
+SELECT buffile_write('z');
+ buffile_write
+---------------
+ 1
+(1 row)
+
+-- Check that the 'x' and 'y' letters are in the first buffer, not in the
+-- 2nd. (We read enough data to find any non-zero bytes in the 2nd buffer.)
+SELECT buffile_seek(0, 8188);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+SELECT buffile_read(4 + 8192);
+ buffile_read
+--------------
+ \x616278647a
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_11.out b/src/test/modules/buffile/expected/test_11.out
new file mode 100644
index 0000000000..c6804d08af
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_11.out
@@ -0,0 +1,34 @@
+BEGIN;
+SELECT buffile_create();
+ buffile_create
+----------------
+
+(1 row)
+
+SELECT buffile_write('abcd');
+ buffile_write
+---------------
+ 4
+(1 row)
+
+-- Seek beyond EOF not followed by write.
+SELECT buffile_seek(0, 5);
+ buffile_seek
+--------------
+ 0
+(1 row)
+
+-- Nothing should be fetched.
+SELECT buffile_read(8);
+ buffile_read
+--------------
+ \x
+(1 row)
+
+SELECT buffile_close();
+ buffile_close
+---------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/expected/test_12.out b/src/test/modules/buffile/expected/test_12.out
new file mode 100644
index 0000000000..fe507a8e58
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_12.out
@@ -0,0 +1,16 @@
+-- Neither disk space nor time needs to be wasted.
+--
+-- Here the tests are designed for segments of multiple buffers.
+SET buffile_max_filesize TO 32768;
+SELECT buffile_test_shared();
+ buffile_test_shared
+---------------------
+
+(1 row)
+
+SELECT buffile_test_shared_append();
+ buffile_test_shared_append
+----------------------------
+
+(1 row)
+
diff --git a/src/test/modules/buffile/expected/test_13.out b/src/test/modules/buffile/expected/test_13.out
new file mode 100644
index 0000000000..7ba50a9c9d
--- /dev/null
+++ b/src/test/modules/buffile/expected/test_13.out
@@ -0,0 +1,86 @@
+-- Use transaction block so that the file does not closed automatically at
+-- command boundary.
+BEGIN;
+SELECT buffile_open_transient('trans1', true, false);
+ buffile_open_transient
+------------------------
+
+(1 row)
+
+SELECT buffile_write('01234567');
+ buffile_write
+---------------
+ 8
+(1 row)
+
+SELECT buffile_close_transient();
+ buffile_close_transient
+-------------------------
+
+(1 row)
+
+-- Open for reading.
+SELECT buffile_open_transient('trans1', false, false);
+ buffile_open_transient
+------------------------
+
+(1 row)
+
+SELECT length(buffile_read(65536));
+ length
+--------
+ 8
+(1 row)
+
+SELECT buffile_close_transient();
+ buffile_close_transient
+-------------------------
+
+(1 row)
+
+-- Open for writing in append mode.
+SELECT buffile_open_transient('trans1', true, true);
+ buffile_open_transient
+------------------------
+
+(1 row)
+
+-- Add BLCKSZ bytes, so that buffer boundary is crossed.
+SELECT buffile_write(repeat('x', 8192));
+ buffile_write
+---------------
+ 8192
+(1 row)
+
+SELECT buffile_close_transient();
+ buffile_close_transient
+-------------------------
+
+(1 row)
+
+-- Open for reading and verify the valid part.
+SELECT buffile_open_transient('trans1', false, false);
+ buffile_open_transient
+------------------------
+
+(1 row)
+
+SELECT length(buffile_read(65536));
+ length
+--------
+ 8200
+(1 row)
+
+SELECT buffile_close_transient();
+ buffile_close_transient
+-------------------------
+
+(1 row)
+
+SELECT buffile_delete_file('trans1');
+ buffile_delete_file
+---------------------
+
+(1 row)
+
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_01.sql b/src/test/modules/buffile/sql/test_01.sql
new file mode 100644
index 0000000000..92331dd0de
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_01.sql
@@ -0,0 +1,17 @@
+-- This is the first test, so make sure the test extension is there.
+CREATE EXTENSION IF NOT EXISTS buffile;
+
+BEGIN;
+SELECT buffile_create();
+SELECT buffile_seek(0, 1);
+SELECT buffile_write('abc');
+SELECT buffile_seek(0, 0);
+-- Check that the trailing zeroes are not fetched.
+SELECT buffile_read(16);
+-- Adjust the number of useful bytes.
+SELECT buffile_write('abc');
+-- ... and check again.
+SELECT buffile_seek(0, 0);
+SELECT buffile_read(16);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_02.sql b/src/test/modules/buffile/sql/test_02.sql
new file mode 100644
index 0000000000..35d34722d7
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_02.sql
@@ -0,0 +1,13 @@
+BEGIN;
+SELECT buffile_create();
+SELECT buffile_seek(0, 8189);
+-- Initialize the last 3 positions of the first buffer and the initial 3
+-- positions of the 2nd buffer.
+SELECT buffile_write('abcdef');
+SELECT buffile_seek(0, 0);
+-- Read the first buffer.
+SELECT length(buffile_read(8192));
+-- Only 3 bytes of the 2nd buffer should be fetched.
+SELECT length(buffile_read(8192));
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_03.sql b/src/test/modules/buffile/sql/test_03.sql
new file mode 100644
index 0000000000..a95391f7c3
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_03.sql
@@ -0,0 +1,7 @@
+BEGIN;
+SELECT buffile_create();
+-- Read from an empty file.
+SELECT buffile_seek(0, 8);
+SELECT buffile_read(16);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_04.sql b/src/test/modules/buffile/sql/test_04.sql
new file mode 100644
index 0000000000..64e8d39f94
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_04.sql
@@ -0,0 +1,25 @@
+BEGIN;
+SELECT buffile_create();
+-- Write something near the end of the first buffer, but leave some trailing
+-- space.
+SELECT buffile_seek(0, 8184);
+SELECT buffile_write('abcd');
+-- Leave the 2nd buffer empty, as well as a few leading bytes. Thus we should
+-- get a hole that spans the whole 2nd buffer as well as a few adjacent bytes
+-- on each side.
+SELECT buffile_seek(0, 2 * 8192 + 4);
+SELECT buffile_write('efgh');
+-- Check the initial part of the hole, which crosses the boundary of the 1st
+-- and the 2nd buffer.
+SELECT buffile_seek(0, 8184);
+SELECT buffile_read(16);
+-- Check the trailing part of the whole, which crosses the boundary of the 2nd
+-- and the 3rd buffer.
+SELECT buffile_seek(0, 2 * 8192 - 8);
+SELECT buffile_read(16);
+-- Check that the hole contains nothing but zeroes.
+SELECT buffile_seek(0, 8192 - 4);
+SELECT btrim(buffile_read(8192 + 8), '\x00');
+
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_05.sql b/src/test/modules/buffile/sql/test_05.sql
new file mode 100644
index 0000000000..5fd642e558
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_05.sql
@@ -0,0 +1,8 @@
+BEGIN;
+SELECT buffile_create();
+-- Seek does not extend the file if it's not followed by write.
+SELECT buffile_seek(0, 1);
+SELECT buffile_seek(0, 0);
+SELECT buffile_read(2);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_06.sql b/src/test/modules/buffile/sql/test_06.sql
new file mode 100644
index 0000000000..35aa48857c
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_06.sql
@@ -0,0 +1,15 @@
+-- This test shows that the if first component file (segment) stays empty,
+-- read stops prematurely even if it starts within that segment, although it'd
+-- otherwise receive some data from the following one.
+BEGIN;
+
+-- Neither disk space nor time needs to be wasted.
+SET buffile_max_filesize TO 8192;
+
+SELECT buffile_create();
+SELECT buffile_seek(0, 8192);
+SELECT buffile_write('a');
+SELECT buffile_seek(0, 8191);
+SELECT buffile_read(2);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_07.sql b/src/test/modules/buffile/sql/test_07.sql
new file mode 100644
index 0000000000..06057c5a1b
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_07.sql
@@ -0,0 +1,13 @@
+BEGIN;
+
+-- Use a small segment, not to waste disk space and time.
+SET buffile_max_filesize TO 8192;
+
+SELECT buffile_create();
+-- Write data at component file boundary and try to read it.
+SELECT buffile_seek(0, 8192);
+SELECT buffile_write('abcd');
+SELECT buffile_seek(0, 8192);
+SELECT buffile_read(8);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_08.sql b/src/test/modules/buffile/sql/test_08.sql
new file mode 100644
index 0000000000..2736dc22df
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_08.sql
@@ -0,0 +1,13 @@
+BEGIN;
+SELECT buffile_create();
+
+-- Neither disk space nor time needs to be wasted.
+SET buffile_max_filesize TO 8192;
+
+-- Write data across component file boundary and try to read it.
+SELECT buffile_seek(0, 8190);
+SELECT buffile_write('abcd');
+SELECT buffile_seek(0, 8190);
+SELECT buffile_read(8);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_09.sql b/src/test/modules/buffile/sql/test_09.sql
new file mode 100644
index 0000000000..cc7060932e
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_09.sql
@@ -0,0 +1,9 @@
+BEGIN;
+SELECT buffile_create();
+-- Write data across buffer boundary and try to read it.
+SELECT buffile_seek(0, 8190);
+SELECT buffile_write('abcd');
+SELECT buffile_seek(0, 8190);
+SELECT buffile_read(8);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_10.sql b/src/test/modules/buffile/sql/test_10.sql
new file mode 100644
index 0000000000..63af760d9f
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_10.sql
@@ -0,0 +1,25 @@
+BEGIN;
+SELECT buffile_create();
+-- Write some data at the end of the buffer.
+SELECT buffile_seek(0, 8188);
+SELECT buffile_write('abcd');
+SELECT buffile_seek(0, 8189);
+-- Enforce flush with the write position not at the end of the buffer. This is
+-- special by not moving curOffset to the next buffer.
+SELECT buffile_read(1);
+
+-- Therefore the next writes should eventually affect the original data. (Here
+-- we also test going directly from read to write and vice versa.)
+SELECT buffile_write('x');
+SELECT buffile_read(1);
+
+-- Start a new buffer, i.e. force flushing of the previous one.
+SELECT buffile_write('z');
+
+-- Check that the 'x' and 'y' letters are in the first buffer, not in the
+-- 2nd. (We read enough data to find any non-zero bytes in the 2nd buffer.)
+SELECT buffile_seek(0, 8188);
+SELECT buffile_read(4 + 8192);
+
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_11.sql b/src/test/modules/buffile/sql/test_11.sql
new file mode 100644
index 0000000000..94300d253f
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_11.sql
@@ -0,0 +1,9 @@
+BEGIN;
+SELECT buffile_create();
+SELECT buffile_write('abcd');
+-- Seek beyond EOF not followed by write.
+SELECT buffile_seek(0, 5);
+-- Nothing should be fetched.
+SELECT buffile_read(8);
+SELECT buffile_close();
+COMMIT;
diff --git a/src/test/modules/buffile/sql/test_12.sql b/src/test/modules/buffile/sql/test_12.sql
new file mode 100644
index 0000000000..017050ef5b
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_12.sql
@@ -0,0 +1,7 @@
+-- Neither disk space nor time needs to be wasted.
+--
+-- Here the tests are designed for segments of multiple buffers.
+SET buffile_max_filesize TO 32768;
+
+SELECT buffile_test_shared();
+SELECT buffile_test_shared_append();
diff --git a/src/test/modules/buffile/sql/test_13.sql b/src/test/modules/buffile/sql/test_13.sql
new file mode 100644
index 0000000000..bcabf7bfe6
--- /dev/null
+++ b/src/test/modules/buffile/sql/test_13.sql
@@ -0,0 +1,25 @@
+-- Use transaction block so that the file does not closed automatically at
+-- command boundary.
+BEGIN;
+SELECT buffile_open_transient('trans1', true, false);
+SELECT buffile_write('01234567');
+SELECT buffile_close_transient();
+
+-- Open for reading.
+SELECT buffile_open_transient('trans1', false, false);
+SELECT length(buffile_read(65536));
+SELECT buffile_close_transient();
+
+-- Open for writing in append mode.
+SELECT buffile_open_transient('trans1', true, true);
+-- Add BLCKSZ bytes, so that buffer boundary is crossed.
+SELECT buffile_write(repeat('x', 8192));
+SELECT buffile_close_transient();
+
+-- Open for reading and verify the valid part.
+SELECT buffile_open_transient('trans1', false, false);
+SELECT length(buffile_read(65536));
+SELECT buffile_close_transient();
+
+SELECT buffile_delete_file('trans1');
+COMMIT;
--
2.13.7
v04-0013-Make-database-OID-constant-for-postgres-database.patch (text/x-diff, 2.7K) download
From 02df324bdcbaf4ff475025b11c37afea9c86623b Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Fri, 5 Jul 2019 16:24:02 +0200
Subject: [PATCH 13/17] Make database OID constant for postgres database.
Later in the patch series we teach pg_upgrade to preserve OIDs of
relfilenodes, tablespaces and databases. To ensure the last, we need to make
the OID of the "postgres" database constant because pg_upgrade can drop it in
the new cluster and create it again.
---
src/backend/commands/dbcommands.c | 21 +++++++++++++++++----
src/include/catalog/pg_database.h | 14 ++++++++++++++
2 files changed, 31 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index 7ea783d096..4ffb6dccf4 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -513,11 +513,24 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
*/
pg_database_rel = table_open(DatabaseRelationId, RowExclusiveLock);
- do
+ /* The "postgres" database has fixed OID. */
+ if (strcmp(dbname, "postgres") == 0)
{
- dboid = GetNewOidWithIndex(pg_database_rel, DatabaseOidIndexId,
- Anum_pg_database_oid);
- } while (check_db_file_conflict(dboid));
+ dboid = PostgresDbOid;
+
+ if (check_db_file_conflict(dboid))
+ ereport(ERROR,
+ (errcode(ERRCODE_DUPLICATE_DATABASE),
+ errmsg("database \"%s\" already exists", dbname)));
+ }
+ else
+ {
+ do
+ {
+ dboid = GetNewOidWithIndex(pg_database_rel, DatabaseOidIndexId,
+ Anum_pg_database_oid);
+ } while (check_db_file_conflict(dboid));
+ }
/*
* Insert a new tuple into pg_database. This establishes our ownership of
diff --git a/src/include/catalog/pg_database.h b/src/include/catalog/pg_database.h
index 06fea45f53..ebb17e3051 100644
--- a/src/include/catalog/pg_database.h
+++ b/src/include/catalog/pg_database.h
@@ -80,4 +80,18 @@ CATALOG(pg_database,1262,DatabaseRelationId) BKI_SHARED_RELATION BKI_ROWTYPE_OID
*/
typedef FormData_pg_database *Form_pg_database;
+/*
+ * The "postgres" database has reserved pg_database(oid) because initially
+ * it's below FirstNormalObjectId, and thus pg_upgrade cannot guarantee that
+ * the same OID will be assigned in the new cluster (unlike TemplateDbOid,
+ * which has always OID=1). Since database OID is used to construct encryption
+ * initialization vector (tweak), different OID would break data decryption in
+ * the new cluster.
+ *
+ * The easiest way to avoid collision with any system OID is to use the
+ * maximum system OID. initdb won't reach this value and GetNewObjectId() will
+ * always skip that.
+ */
+#define PostgresDbOid (FirstNormalObjectId - 1)
+
#endif /* PG_DATABASE_H */
--
2.13.7
v04-0014-Teach-pg_upgrade-to-preserve-OIDs-of-relfilenode-dat.patch (text/x-diff, 23.8K) download
From c9a9d7f731a7e103f2346d52d3c6deae0c4d7045 Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Fri, 5 Jul 2019 16:24:02 +0200
Subject: [PATCH 14/17] Teach pg_upgrade to preserve OIDs of relfilenode,
database and tablespace.
Since these OIDs are used to construct encryption initialization vector, we
need to keep them the same in the new cluster, otherwise relation files need
to be "reencrypted" during the update and the --link mode must be disabled for
encrypted cluster.
---
src/backend/catalog/heap.c | 71 +++++++++++++++++++----
src/backend/catalog/index.c | 1 +
src/backend/commands/dbcommands.c | 11 ++++
src/backend/commands/tablespace.c | 14 ++++-
src/backend/utils/adt/pg_upgrade_support.c | 55 ++++++++++++++++++
src/bin/pg_dump/pg_dump.c | 90 ++++++++++++++++++++++++++----
src/bin/pg_dump/pg_dumpall.c | 3 +
src/bin/pg_upgrade/Makefile | 2 +-
src/bin/pg_upgrade/pg_upgrade.c | 14 +++--
src/include/catalog/binary_upgrade.h | 7 +++
src/include/catalog/pg_proc.dat | 20 +++++++
11 files changed, 257 insertions(+), 31 deletions(-)
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3b8c8b193a..0aef214cae 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -91,7 +91,9 @@
/* Potentially set by pg_upgrade_support functions */
Oid binary_upgrade_next_heap_pg_class_oid = InvalidOid;
+Oid binary_upgrade_next_heap_pg_class_relfilenode = InvalidOid;
Oid binary_upgrade_next_toast_pg_class_oid = InvalidOid;
+Oid binary_upgrade_next_toast_pg_class_relfilenode = InvalidOid;
static void AddNewRelationTuple(Relation pg_class_desc,
Relation new_rel_desc,
@@ -366,17 +368,60 @@ heap_create(const char *relname,
break;
}
- /*
- * Decide whether to create storage. If caller passed a valid relfilenode,
- * storage is already created, so don't do it here. Also don't create it
- * for relkinds without physical storage.
- */
- if (!RELKIND_HAS_STORAGE(relkind) || OidIsValid(relfilenode))
- create_storage = false;
+ /* Decide whether to create storage. */
+ if (!IsBinaryUpgrade)
+ {
+ /*
+ * If caller passed a valid relfilenode, storage is already created,
+ * so don't do it here. Also don't create it for relkinds without
+ * physical storage.
+ */
+ if (!RELKIND_HAS_STORAGE(relkind) || OidIsValid(relfilenode))
+ create_storage = false;
+ else
+ {
+ create_storage = true;
+ relfilenode = relid;
+ }
+ }
else
{
+ /* Override relfilenode? */
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_SEQUENCE ||
+ relkind == RELKIND_MATVIEW)
+ {
+ if (!OidIsValid(binary_upgrade_next_heap_pg_class_relfilenode))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("relfilenode value not set when in binary upgrade mode")));
+
+ relfilenode = binary_upgrade_next_heap_pg_class_relfilenode;
+ binary_upgrade_next_heap_pg_class_relfilenode = InvalidOid;
+ }
+ else if (relkind == RELKIND_TOASTVALUE)
+ {
+ if (!OidIsValid(binary_upgrade_next_toast_pg_class_relfilenode))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("toast relfilenode value not set when in binary upgrade mode")));
+ relfilenode = binary_upgrade_next_toast_pg_class_relfilenode;
+ binary_upgrade_next_toast_pg_class_relfilenode = InvalidOid;
+ }
+ else if (relkind == RELKIND_INDEX)
+ {
+ if (!OidIsValid(binary_upgrade_next_index_pg_class_relfilenode))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("pg_class index relfilenode value not set when in binary upgrade mode")));
+ relfilenode = binary_upgrade_next_index_pg_class_relfilenode;
+ binary_upgrade_next_index_pg_class_relfilenode = InvalidOid;
+ }
+
+ /*
+ * The storage is needed for the following DDLs, although it will
+ * eventually be overwritten with that of the old cluster.
+ */
create_storage = true;
- relfilenode = relid;
}
/*
@@ -1149,9 +1194,6 @@ heap_create_with_catalog(const char *relname,
/*
* Allocate an OID for the relation, unless we were told what to use.
- *
- * The OID will be the relfilenode as well, so make sure it doesn't
- * collide with either pg_class OIDs or existing physical files.
*/
if (!OidIsValid(relid))
{
@@ -1179,8 +1221,15 @@ heap_create_with_catalog(const char *relname,
binary_upgrade_next_toast_pg_class_oid = InvalidOid;
}
else
+ {
+ /*
+ * The OID will be the relfilenode as well, so make sure it
+ * doesn't collide with either pg_class OIDs or existing physical
+ * files.
+ */
relid = GetNewRelFileNode(reltablespace, pg_class_desc,
relpersistence);
+ }
}
/*
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index bb60b23093..1566d03dcc 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -83,6 +83,7 @@
/* Potentially set by pg_upgrade_support functions */
Oid binary_upgrade_next_index_pg_class_oid = InvalidOid;
+Oid binary_upgrade_next_index_pg_class_relfilenode = InvalidOid;
/*
* Pointer-free representation of variables used when reindexing system
diff --git a/src/backend/commands/dbcommands.c b/src/backend/commands/dbcommands.c
index 4ffb6dccf4..692bd40907 100644
--- a/src/backend/commands/dbcommands.c
+++ b/src/backend/commands/dbcommands.c
@@ -92,6 +92,7 @@ static void remove_dbtablespaces(Oid db_id);
static bool check_db_file_conflict(Oid db_id);
static int errdetail_busy_db(int notherbackends, int npreparedxacts);
+Oid binary_upgrade_next_pg_database_oid = InvalidOid;
/*
* CREATE DATABASE
@@ -523,6 +524,16 @@ createdb(ParseState *pstate, const CreatedbStmt *stmt)
(errcode(ERRCODE_DUPLICATE_DATABASE),
errmsg("database \"%s\" already exists", dbname)));
}
+ else if (IsBinaryUpgrade)
+ {
+ if (!OidIsValid(binary_upgrade_next_pg_database_oid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("pg_database OID value not set when in binary upgrade mode")));
+
+ dboid = binary_upgrade_next_pg_database_oid;
+ binary_upgrade_next_pg_database_oid = InvalidOid;
+ }
else
{
do
diff --git a/src/backend/commands/tablespace.c b/src/backend/commands/tablespace.c
index 84efb414d8..dcbecc7196 100644
--- a/src/backend/commands/tablespace.c
+++ b/src/backend/commands/tablespace.c
@@ -89,6 +89,7 @@
char *default_tablespace = NULL;
char *temp_tablespaces = NULL;
+Oid binary_upgrade_next_pg_tablespace_oid = InvalidOid;
static void create_tablespace_directories(const char *location,
const Oid tablespaceoid);
@@ -336,7 +337,18 @@ CreateTableSpace(CreateTableSpaceStmt *stmt)
MemSet(nulls, false, sizeof(nulls));
- tablespaceoid = GetNewOidWithIndex(rel, TablespaceOidIndexId,
+ if (IsBinaryUpgrade)
+ {
+ if (!OidIsValid(binary_upgrade_next_pg_tablespace_oid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("pg_tablespace OID value not set when in binary upgrade mode")));
+
+ tablespaceoid = binary_upgrade_next_pg_tablespace_oid;
+ binary_upgrade_next_pg_tablespace_oid = InvalidOid;
+ }
+ else
+ tablespaceoid = GetNewOidWithIndex(rel, TablespaceOidIndexId,
Anum_pg_tablespace_oid);
values[Anum_pg_tablespace_oid - 1] = ObjectIdGetDatum(tablespaceoid);
values[Anum_pg_tablespace_spcname - 1] =
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 99db5ba389..d2e7c0d7eb 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -30,6 +30,28 @@ do { \
} while (0)
Datum
+binary_upgrade_set_next_pg_database_oid(PG_FUNCTION_ARGS)
+{
+ Oid dboid = PG_GETARG_OID(0);
+
+ CHECK_IS_BINARY_UPGRADE;
+ binary_upgrade_next_pg_database_oid = dboid;
+
+ PG_RETURN_VOID();
+}
+
+Datum
+binary_upgrade_set_next_pg_tablespace_oid(PG_FUNCTION_ARGS)
+{
+ Oid tbspoid = PG_GETARG_OID(0);
+
+ CHECK_IS_BINARY_UPGRADE;
+ binary_upgrade_next_pg_tablespace_oid = tbspoid;
+
+ PG_RETURN_VOID();
+}
+
+Datum
binary_upgrade_set_next_pg_type_oid(PG_FUNCTION_ARGS)
{
Oid typoid = PG_GETARG_OID(0);
@@ -74,6 +96,17 @@ binary_upgrade_set_next_heap_pg_class_oid(PG_FUNCTION_ARGS)
}
Datum
+binary_upgrade_set_next_heap_pg_class_relfilenode(PG_FUNCTION_ARGS)
+{
+ Oid nodeoid = PG_GETARG_OID(0);
+
+ CHECK_IS_BINARY_UPGRADE;
+ binary_upgrade_next_heap_pg_class_relfilenode = nodeoid;
+
+ PG_RETURN_VOID();
+}
+
+Datum
binary_upgrade_set_next_index_pg_class_oid(PG_FUNCTION_ARGS)
{
Oid reloid = PG_GETARG_OID(0);
@@ -85,6 +118,17 @@ binary_upgrade_set_next_index_pg_class_oid(PG_FUNCTION_ARGS)
}
Datum
+binary_upgrade_set_next_index_pg_class_relfilenode(PG_FUNCTION_ARGS)
+{
+ Oid nodeoid = PG_GETARG_OID(0);
+
+ CHECK_IS_BINARY_UPGRADE;
+ binary_upgrade_next_index_pg_class_relfilenode = nodeoid;
+
+ PG_RETURN_VOID();
+}
+
+Datum
binary_upgrade_set_next_toast_pg_class_oid(PG_FUNCTION_ARGS)
{
Oid reloid = PG_GETARG_OID(0);
@@ -96,6 +140,17 @@ binary_upgrade_set_next_toast_pg_class_oid(PG_FUNCTION_ARGS)
}
Datum
+binary_upgrade_set_next_toast_pg_class_relfilenode(PG_FUNCTION_ARGS)
+{
+ Oid nodeoid = PG_GETARG_OID(0);
+
+ CHECK_IS_BINARY_UPGRADE;
+ binary_upgrade_next_toast_pg_class_relfilenode = nodeoid;
+
+ PG_RETURN_VOID();
+}
+
+Datum
binary_upgrade_set_next_pg_enum_oid(PG_FUNCTION_ARGS)
{
Oid enumoid = PG_GETARG_OID(0);
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 806fc78f04..912bfcb8f3 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -48,6 +48,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_database.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -280,7 +281,7 @@ static void binary_upgrade_set_type_oids_by_type_oid(Archive *fout,
bool force_array_type);
static bool binary_upgrade_set_type_oids_by_rel_oid(Archive *fout,
PQExpBuffer upgrade_buffer, Oid pg_rel_oid);
-static void binary_upgrade_set_pg_class_oids(Archive *fout,
+static void binary_upgrade_set_pg_class_oids_and_relfilenodes(Archive *fout,
PQExpBuffer upgrade_buffer,
Oid pg_class_oid, bool is_index);
static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
@@ -2630,6 +2631,7 @@ dumpDatabase(Archive *fout)
PQExpBuffer dbQry = createPQExpBuffer();
PQExpBuffer delQry = createPQExpBuffer();
PQExpBuffer creaQry = createPQExpBuffer();
+ PQExpBuffer setDBIdQry = createPQExpBuffer();
PQExpBuffer labelq = createPQExpBuffer();
PGconn *conn = GetConnection(fout);
PGresult *res;
@@ -2800,6 +2802,37 @@ dumpDatabase(Archive *fout)
qdatname = pg_strdup(fmtId(datname));
/*
+ * Make sure that pg_upgrade does not change database OID. Don't care
+ * about "postgres" database, backend will assign it fixed OID
+ * anyway. ("template1" has fixed OID too but the value 1 should not
+ * collide with any other OID so backend pays no attention to it.)
+ */
+ if (dopt->binary_upgrade && dbCatId.oid != PostgresDbOid)
+ {
+ appendPQExpBufferStr(setDBIdQry, "\n-- For binary upgrade, must preserve pg_database oid\n");
+ appendPQExpBuffer(setDBIdQry,
+ "SELECT pg_catalog.binary_upgrade_set_next_pg_database_oid('%u'::pg_catalog.oid);\n",
+ dbCatId.oid);
+
+ /*
+ * Need a separate entry, otherwise the command will be run in the
+ * same transaction as the CREATE DATABASE command, which is not
+ * allowed.
+ */
+ ArchiveEntry(fout,
+ dbCatId, /* catalog ID */
+ dbDumpId, /* dump ID */
+ ARCHIVE_OPTS(.tag = datname,
+ .owner = dba,
+ .description = "SET_DB_OID",
+ .section = SECTION_PRE_DATA,
+ .createStmt = setDBIdQry->data,
+ .dropStmt = NULL));
+
+ destroyPQExpBuffer(setDBIdQry);
+ }
+
+ /*
* Prepare the CREATE DATABASE command. We must specify encoding, locale,
* and tablespace since those can't be altered later. Other DB properties
* are left to the DATABASE PROPERTIES entry, so that they can be applied
@@ -4391,36 +4424,57 @@ binary_upgrade_set_type_oids_by_rel_oid(Archive *fout,
return toast_set;
}
+/*
+ * The function handles both oid and relfilenode since they are related to one
+ * another. It'd be inefficient if we ran two similar queries for each
+ * relation instead of just one.
+ */
static void
-binary_upgrade_set_pg_class_oids(Archive *fout,
- PQExpBuffer upgrade_buffer, Oid pg_class_oid,
- bool is_index)
+binary_upgrade_set_pg_class_oids_and_relfilenodes(Archive *fout,
+ PQExpBuffer upgrade_buffer,
+ Oid pg_class_oid,
+ bool is_index)
{
PQExpBuffer upgrade_query = createPQExpBuffer();
PGresult *upgrade_res;
+ Oid pg_class_relfilenode;
Oid pg_class_reltoastrelid;
+ Oid pg_class_relfilenode_toast;
Oid pg_index_indexrelid;
+ Oid pg_class_relfilenode_toast_idx;
appendPQExpBuffer(upgrade_query,
- "SELECT c.reltoastrelid, i.indexrelid "
+ "SELECT c.relfilenode, c.reltoastrelid, ct.relfilenode AS relfilenode_toast, i.indexrelid, cti.relfilenode AS relfilenode_toast_index "
"FROM pg_catalog.pg_class c LEFT JOIN "
"pg_catalog.pg_index i ON (c.reltoastrelid = i.indrelid AND i.indisvalid) "
+ "LEFT JOIN pg_catalog.pg_class ct ON (c.reltoastrelid = ct.oid) "
+ "LEFT JOIN pg_catalog.pg_class AS cti ON (i.indexrelid = cti.oid) "
"WHERE c.oid = '%u'::pg_catalog.oid;",
pg_class_oid);
upgrade_res = ExecuteSqlQueryForSingleRow(fout, upgrade_query->data);
+ pg_class_relfilenode = atooid(PQgetvalue(upgrade_res, 0, PQfnumber(upgrade_res, "relfilenode")));
pg_class_reltoastrelid = atooid(PQgetvalue(upgrade_res, 0, PQfnumber(upgrade_res, "reltoastrelid")));
+ pg_class_relfilenode_toast = atooid(PQgetvalue(upgrade_res, 0, PQfnumber(upgrade_res, "relfilenode_toast")));
pg_index_indexrelid = atooid(PQgetvalue(upgrade_res, 0, PQfnumber(upgrade_res, "indexrelid")));
+ pg_class_relfilenode_toast_idx = atooid(PQgetvalue(upgrade_res, 0, PQfnumber(upgrade_res, "relfilenode_toast_index")));
appendPQExpBufferStr(upgrade_buffer,
- "\n-- For binary upgrade, must preserve pg_class oids\n");
+ "\n-- For binary upgrade, must preserve pg_class oids and relfilenodes\n");
if (!is_index)
{
appendPQExpBuffer(upgrade_buffer,
"SELECT pg_catalog.binary_upgrade_set_next_heap_pg_class_oid('%u'::pg_catalog.oid);\n",
pg_class_oid);
+
+ /* Not every relation has storage. */
+ if (OidIsValid(pg_class_relfilenode))
+ appendPQExpBuffer(upgrade_buffer,
+ "SELECT pg_catalog.binary_upgrade_set_next_heap_pg_class_relfilenode('%u'::pg_catalog.oid);\n",
+ pg_class_relfilenode);
+
/* only tables have toast tables, not indexes */
if (OidIsValid(pg_class_reltoastrelid))
{
@@ -4436,17 +4490,28 @@ binary_upgrade_set_pg_class_oids(Archive *fout,
appendPQExpBuffer(upgrade_buffer,
"SELECT pg_catalog.binary_upgrade_set_next_toast_pg_class_oid('%u'::pg_catalog.oid);\n",
pg_class_reltoastrelid);
+ appendPQExpBuffer(upgrade_buffer,
+ "SELECT pg_catalog.binary_upgrade_set_next_toast_pg_class_relfilenode('%u'::pg_catalog.oid);\n",
+ pg_class_relfilenode_toast);
/* every toast table has an index */
appendPQExpBuffer(upgrade_buffer,
"SELECT pg_catalog.binary_upgrade_set_next_index_pg_class_oid('%u'::pg_catalog.oid);\n",
pg_index_indexrelid);
+ appendPQExpBuffer(upgrade_buffer,
+ "SELECT pg_catalog.binary_upgrade_set_next_index_pg_class_relfilenode('%u'::pg_catalog.oid);\n",
+ pg_class_relfilenode_toast_idx);
}
}
else
+ {
appendPQExpBuffer(upgrade_buffer,
"SELECT pg_catalog.binary_upgrade_set_next_index_pg_class_oid('%u'::pg_catalog.oid);\n",
pg_class_oid);
+ appendPQExpBuffer(upgrade_buffer,
+ "SELECT pg_catalog.binary_upgrade_set_next_index_pg_class_relfilenode('%u'::pg_catalog.oid);\n",
+ pg_class_relfilenode);
+ }
appendPQExpBufferChar(upgrade_buffer, '\n');
@@ -11008,7 +11073,7 @@ dumpCompositeType(Archive *fout, TypeInfo *tyinfo)
binary_upgrade_set_type_oids_by_type_oid(fout, q,
tyinfo->dobj.catId.oid,
false);
- binary_upgrade_set_pg_class_oids(fout, q, tyinfo->typrelid, false);
+ binary_upgrade_set_pg_class_oids_and_relfilenodes(fout, q, tyinfo->typrelid, false);
}
qtypname = pg_strdup(fmtId(tyinfo->dobj.name));
@@ -15507,7 +15572,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
appendPQExpBuffer(delq, "DROP VIEW %s;\n", qualrelname);
if (dopt->binary_upgrade)
- binary_upgrade_set_pg_class_oids(fout, q,
+ binary_upgrade_set_pg_class_oids_and_relfilenodes(fout, q,
tbinfo->dobj.catId.oid, false);
appendPQExpBuffer(q, "CREATE VIEW %s", qualrelname);
@@ -15583,7 +15648,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
appendPQExpBuffer(delq, "DROP %s %s;\n", reltypename, qualrelname);
if (dopt->binary_upgrade)
- binary_upgrade_set_pg_class_oids(fout, q,
+ binary_upgrade_set_pg_class_oids_and_relfilenodes(fout, q,
tbinfo->dobj.catId.oid, false);
appendPQExpBuffer(q, "CREATE %s%s %s",
@@ -16337,7 +16402,7 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
int nstatvals;
if (dopt->binary_upgrade)
- binary_upgrade_set_pg_class_oids(fout, q,
+ binary_upgrade_set_pg_class_oids_and_relfilenodes(fout, q,
indxinfo->dobj.catId.oid, true);
/* Plain secondary index */
@@ -16559,7 +16624,7 @@ dumpConstraint(Archive *fout, ConstraintInfo *coninfo)
coninfo->dobj.name);
if (dopt->binary_upgrade)
- binary_upgrade_set_pg_class_oids(fout, q,
+ binary_upgrade_set_pg_class_oids_and_relfilenodes(fout, q,
indxinfo->dobj.catId.oid, true);
appendPQExpBuffer(q, "ALTER TABLE ONLY %s\n",
@@ -16971,12 +17036,13 @@ dumpSequence(Archive *fout, TableInfo *tbinfo)
if (dopt->binary_upgrade)
{
- binary_upgrade_set_pg_class_oids(fout, query,
+ binary_upgrade_set_pg_class_oids_and_relfilenodes(fout, query,
tbinfo->dobj.catId.oid, false);
binary_upgrade_set_type_oids_by_rel_oid(fout, query,
tbinfo->dobj.catId.oid);
}
+
if (tbinfo->is_identity_sequence)
{
TableInfo *owning_tab = findTableByOid(tbinfo->owning_tab);
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 158c0c74b2..d072f28956 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -1259,6 +1259,9 @@ dumpTablespaces(PGconn *conn)
/* needed for buildACLCommands() */
fspcname = pg_strdup(fmtId(spcname));
+ appendPQExpBufferStr(buf, "\n-- For binary upgrade, must preserve pg_tablespace oid\n");
+ appendPQExpBuffer(buf, "SELECT pg_catalog.binary_upgrade_set_next_pg_tablespace_oid('%u'::pg_catalog.oid);\n", spcoid);
+
appendPQExpBuffer(buf, "CREATE TABLESPACE %s", fspcname);
appendPQExpBuffer(buf, " OWNER %s", fmtId(spcowner));
diff --git a/src/bin/pg_upgrade/Makefile b/src/bin/pg_upgrade/Makefile
index 2cb829acd5..0eaf869119 100644
--- a/src/bin/pg_upgrade/Makefile
+++ b/src/bin/pg_upgrade/Makefile
@@ -11,7 +11,7 @@ OBJS = check.o controldata.o dump.o exec.o file.o function.o info.o \
option.o parallel.o pg_upgrade.o relfilenode.o server.o \
tablespace.o util.o version.o $(WIN32RES)
-override CPPFLAGS := -DDLSUFFIX=\"$(DLSUFFIX)\" -I$(srcdir) -I$(libpq_srcdir) $(CPPFLAGS)
+override CPPFLAGS := -DDLSUFFIX=\"$(DLSUFFIX)\" -I$(srcdir) -I$(libpq_srcdir) -DFRONTEND $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
all: pg_upgrade
diff --git a/src/bin/pg_upgrade/pg_upgrade.c b/src/bin/pg_upgrade/pg_upgrade.c
index ff78929707..336df7378f 100644
--- a/src/bin/pg_upgrade/pg_upgrade.c
+++ b/src/bin/pg_upgrade/pg_upgrade.c
@@ -15,12 +15,14 @@
* oids are the same between old and new clusters. This is important
* because toast oids are stored as toast pointers in user tables.
*
- * While pg_class.oid and pg_class.relfilenode are initially the same
- * in a cluster, they can diverge due to CLUSTER, REINDEX, or VACUUM
- * FULL. In the new cluster, pg_class.oid and pg_class.relfilenode will
- * be the same and will match the old pg_class.oid value. Because of
- * this, old/new pg_class.relfilenode values will not match if CLUSTER,
- * REINDEX, or VACUUM FULL have been performed in the old cluster.
+ * While pg_class.oid and pg_class.relfilenode are initially the same in a
+ * cluster, they can diverge due to CLUSTER, REINDEX, or VACUUM FULL. We
+ * control assignments of pg_class.relfilenode because relfilenode is used to
+ * derive initialization vector for data encryption. If relfilenode changed
+ * during upgrade, we'd be unable to decrypt the data in the new cluster.
+ *
+ * pg_database.oid and pg_tablespace.oid are also preserved because they
+ * participate in the encryption initialization vector.
*
* We control all assignments of pg_type.oid because these oids are stored
* in user composite type values.
diff --git a/src/include/catalog/binary_upgrade.h b/src/include/catalog/binary_upgrade.h
index 2927b7a4d3..cffafbc219 100644
--- a/src/include/catalog/binary_upgrade.h
+++ b/src/include/catalog/binary_upgrade.h
@@ -14,14 +14,21 @@
#ifndef BINARY_UPGRADE_H
#define BINARY_UPGRADE_H
+extern PGDLLIMPORT Oid binary_upgrade_next_pg_database_oid;
+extern PGDLLIMPORT Oid binary_upgrade_next_pg_tablespace_oid;
+
extern PGDLLIMPORT Oid binary_upgrade_next_pg_type_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_array_pg_type_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_toast_pg_type_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_heap_pg_class_oid;
+extern PGDLLIMPORT Oid binary_upgrade_next_heap_pg_class_relfilenode;
extern PGDLLIMPORT Oid binary_upgrade_next_index_pg_class_oid;
+extern PGDLLIMPORT Oid binary_upgrade_next_index_pg_class_relfilenode;
extern PGDLLIMPORT Oid binary_upgrade_next_toast_pg_class_oid;
+extern PGDLLIMPORT Oid binary_upgrade_next_toast_pg_class_relfilenode;
+extern PGDLLIMPORT Oid binary_upgrade_next_heap_pg_class_relfilenode;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_enum_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_authid_oid;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 29f0944774..bcb78168d3 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10080,6 +10080,26 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '4142', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_next_heap_pg_class_relfilenode', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid',
+ prosrc => 'binary_upgrade_set_next_heap_pg_class_relfilenode' },
+{ oid => '4191', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_next_index_pg_class_relfilenode', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid',
+ prosrc => 'binary_upgrade_set_next_index_pg_class_relfilenode' },
+{ oid => '4192', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_next_toast_pg_class_relfilenode', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid',
+ prosrc => 'binary_upgrade_set_next_toast_pg_class_relfilenode' },
+{ oid => '4193', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_next_pg_database_oid', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid',
+ prosrc => 'binary_upgrade_set_next_pg_database_oid' },
+{ oid => '4194', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_next_pg_tablespace_oid', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid',
+ prosrc => 'binary_upgrade_set_next_pg_tablespace_oid' }
# conversion functions
{ oid => '4300',
--
2.13.7
v04-0015-Make-pg_upgrade-aware-of-encryption.patch (text/x-diff, 23.5K) download
From 33b4699fd753965689e0d0d63881ea689b68c2f5 Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Fri, 5 Jul 2019 16:24:02 +0200
Subject: [PATCH 15/17] Make pg_upgrade aware of encryption.
This includes pg_resetwal utility becaue pg_upgrade calls it.
---
src/bin/pg_resetwal/Makefile | 10 +++++--
src/bin/pg_resetwal/pg_resetwal.c | 51 +++++++++++++++++++++++++++++++++--
src/bin/pg_upgrade/controldata.c | 39 +++++++++++++++++++++++++++
src/bin/pg_upgrade/dump.c | 2 +-
src/bin/pg_upgrade/exec.c | 30 +++++++++++++++++++--
src/bin/pg_upgrade/option.c | 10 ++++++-
src/bin/pg_upgrade/parallel.c | 7 ++---
src/bin/pg_upgrade/pg_upgrade.c | 57 +++++++++++++++++++++++++++++++++------
src/bin/pg_upgrade/pg_upgrade.h | 9 ++++++-
src/bin/pg_upgrade/server.c | 39 +++++++++++++++++++++++++--
src/fe_utils/encryption.c | 25 ++++++++++-------
src/include/fe_utils/encryption.h | 1 +
12 files changed, 249 insertions(+), 31 deletions(-)
diff --git a/src/bin/pg_resetwal/Makefile b/src/bin/pg_resetwal/Makefile
index 2a3835691f..faebfda27e 100644
--- a/src/bin/pg_resetwal/Makefile
+++ b/src/bin/pg_resetwal/Makefile
@@ -15,13 +15,19 @@ subdir = src/bin/pg_resetwal
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-OBJS= pg_resetwal.o $(WIN32RES)
+override CPPFLAGS := -DFRONTEND $(CPPFLAGS)
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
+
+OBJS= pg_resetwal.o encryption.o $(WIN32RES)
all: pg_resetwal
pg_resetwal: $(OBJS) | submake-libpgport
$(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+encryption.c: % : $(top_srcdir)/src/backend/storage/file/%
+ rm -f $@ && $(LN_S) $< .
+
install: all installdirs
$(INSTALL_PROGRAM) pg_resetwal$(X) '$(DESTDIR)$(bindir)/pg_resetwal$(X)'
@@ -32,7 +38,7 @@ uninstall:
rm -f '$(DESTDIR)$(bindir)/pg_resetwal$(X)'
clean distclean maintainer-clean:
- rm -f pg_resetwal$(X) $(OBJS)
+ rm -f pg_resetwal$(X) $(OBJS) encryption.c
rm -rf tmp_check
check:
diff --git a/src/bin/pg_resetwal/pg_resetwal.c b/src/bin/pg_resetwal/pg_resetwal.c
index ff0f8ea5e7..43921132d4 100644
--- a/src/bin/pg_resetwal/pg_resetwal.c
+++ b/src/bin/pg_resetwal/pg_resetwal.c
@@ -54,11 +54,12 @@
#include "common/file_perm.h"
#include "common/logging.h"
#include "common/restricted_token.h"
+#include "fe_utils/encryption.h"
+#include "storage/encryption.h"
#include "storage/large_object.h"
#include "pg_getopt.h"
#include "getopt_long.h"
-
static ControlFileData ControlFile; /* pg_control values */
static XLogSegNo newXlogSegNo; /* new XLOG segment # */
static bool guessed = false; /* T if we had to guess at any values */
@@ -135,7 +136,7 @@ main(int argc, char *argv[])
}
- while ((c = getopt_long(argc, argv, "c:D:e:fl:m:no:O:x:", long_options, NULL)) != -1)
+ while ((c = getopt_long(argc, argv, "c:D:e:fK:l:m:no:O:x:", long_options, NULL)) != -1)
{
switch (c)
{
@@ -277,6 +278,12 @@ main(int argc, char *argv[])
}
break;
+#ifdef USE_ENCRYPTION
+ case 'K':
+ encryption_key_command = strdup(optarg);
+ break;
+#endif /* USE_ENCRYPTION */
+
case 'l':
if (strspn(optarg, "01234567890ABCDEFabcdef") != XLOG_FNAME_LEN)
{
@@ -408,6 +415,28 @@ main(int argc, char *argv[])
XLogFromFileName(log_fname, &minXlogTli, &minXlogSegNo, WalSegSz);
/*
+ * If the data is encrypted, we also might need to encrypt the XLOG record
+ * below.
+ */
+ if (ControlFile.data_cipher > PG_CIPHER_NONE && !noupdate)
+ {
+ if (encryption_key_command)
+ run_encryption_key_command(encryption_key, DataDir);
+ else
+ {
+ /*
+ * If executed by pg_upgrade, we don't want pg_resetwal to run the
+ * encryption key command (possibly interactive application) w/o
+ * access to terminal.
+ */
+ read_encryption_key_fe(stdin);
+ }
+
+ setup_encryption();
+ data_encrypted = true;
+ }
+
+ /*
* Also look at existing segment files to set up newXlogSegNo
*/
FindEndOfXLOG();
@@ -814,6 +843,12 @@ PrintControlValues(bool guessed)
(ControlFile.float8ByVal ? _("by value") : _("by reference")));
printf(_("Data page checksum version: %u\n"),
ControlFile.data_checksum_version);
+ if (ControlFile.data_cipher > PG_CIPHER_NONE)
+ printf(_("Data encryption fingerprint: %08X%08X%08X%08X\n"),
+ htonl(((uint32 *) ControlFile.encryption_verification)[0]),
+ htonl(((uint32 *) ControlFile.encryption_verification)[1]),
+ htonl(((uint32 *) ControlFile.encryption_verification)[2]),
+ htonl(((uint32 *) ControlFile.encryption_verification)[3]));
}
@@ -1164,6 +1199,14 @@ WriteEmptyXLOG(void)
FIN_CRC32C(crc);
record->xl_crc = crc;
+ if (data_encrypted)
+ {
+ char tweak[TWEAK_SIZE];
+
+ XLogEncryptionTweak(tweak, page->xlp_tli, newXlogSegNo, 0);
+ encrypt_block(buffer.data, buffer.data, XLOG_BLCKSZ, tweak, true);
+ }
+
/* Write the first page */
XLogFilePath(path, ControlFile.checkPointCopy.ThisTimeLineID,
newXlogSegNo, WalSegSz);
@@ -1224,6 +1267,10 @@ usage(void)
printf(_(" [-D, --pgdata=]DATADIR data directory\n"));
printf(_(" -e, --epoch=XIDEPOCH set next transaction ID epoch\n"));
printf(_(" -f, --force force update to be done\n"));
+#ifdef USE_ENCRYPTION
+ printf(_(" -K, --encryption-key-command\n"
+ " command that returns encryption key\n"));
+#endif /* USE_ENCRYPTION */
printf(_(" -l, --next-wal-file=WALFILE set minimum starting location for new WAL\n"));
printf(_(" -m, --multixact-ids=MXID,MXID set next and oldest multitransaction ID\n"));
printf(_(" -n, --dry-run no update, just show what would be done\n"));
diff --git a/src/bin/pg_upgrade/controldata.c b/src/bin/pg_upgrade/controldata.c
index 38236415be..dc15b979bc 100644
--- a/src/bin/pg_upgrade/controldata.c
+++ b/src/bin/pg_upgrade/controldata.c
@@ -485,6 +485,33 @@ get_control_data(ClusterInfo *cluster, bool live_check)
cluster->controldata.data_checksum_version = str2uint(p);
got_data_checksum_version = true;
}
+ else if ((p = strstr(bufin, "encryption fingerprint")) != NULL)
+ {
+ int i;
+
+ p = strchr(p, ':');
+
+ if (p == NULL || strlen(p) <= 1)
+ pg_fatal("%d: controldata retrieval problem\n", __LINE__);
+
+ cluster->controldata.data_encrypted = true;
+
+ /* Skip the colon and any whitespace after it */
+ p = strchr(p, ':');
+ if (p == NULL || strlen(p) <= 1)
+ pg_fatal("%d: controldata retrieval problem\n", __LINE__);
+ p = strpbrk(p, "01234567890ABCDEF");
+ if (p == NULL || strlen(p) <= 1)
+ pg_fatal("%d: controldata retrieval problem\n", __LINE__);
+
+ /* Make sure it looks like a valid finerprint */
+ if (strspn(p, "0123456789ABCDEF") != 32)
+ pg_fatal("%d: controldata retrieval problem\n", __LINE__);
+
+ for (i = 0; i < ENCRYPTION_SAMPLE_SIZE; i++)
+ sscanf(p + 2 * i, "%2hhx",
+ cluster->controldata.encryption_verification + i);
+ }
}
pclose(output);
@@ -669,6 +696,18 @@ check_control_data(ControlData *oldctrl,
pg_fatal("old cluster uses data checksums but the new one does not\n");
else if (oldctrl->data_checksum_version != newctrl->data_checksum_version)
pg_fatal("old and new cluster pg_controldata checksum versions do not match\n");
+
+ if (oldctrl->data_encrypted && !newctrl->data_encrypted)
+ pg_fatal("old cluster is encrypted, but the new one is not\n");
+ else if (!oldctrl->data_encrypted && newctrl->data_encrypted)
+ pg_fatal("old cluster is not encrypted, but the new one is\n");
+ else if (oldctrl->data_encrypted)
+ {
+ if (memcmp(oldctrl->encryption_verification,
+ newctrl->encryption_verification,
+ ENCRYPTION_SAMPLE_SIZE) != 0)
+ pg_fatal("encryption of the new cluster is not compatible with encryption of the old one\n");
+ }
}
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index c1429fe4bf..6fcf9126a4 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -22,7 +22,7 @@ generate_old_dump(void)
prep_status("Creating dump of global objects");
/* run new pg_dumpall binary for globals */
- exec_prog(UTILITY_LOG_FILE, NULL, true, true,
+ exec_prog(UTILITY_LOG_FILE, NULL, true, true, NULL,
"\"%s/pg_dumpall\" %s --globals-only --quote-all-identifiers "
"--binary-upgrade %s -f %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
diff --git a/src/bin/pg_upgrade/exec.c b/src/bin/pg_upgrade/exec.c
index 0363309328..cd6c68bdd9 100644
--- a/src/bin/pg_upgrade/exec.c
+++ b/src/bin/pg_upgrade/exec.c
@@ -74,11 +74,17 @@ get_bin_version(ClusterInfo *cluster)
* If the command fails, an error message is optionally written to the specified
* log_file, and the program optionally exits.
*
+ * If encryption_key is passed, popen() is used and the key is sent to stdin
+ * of the command.
+ *
* The code requires it be called first from the primary thread on Windows.
+ *
+ * TODO Consolidate the use of popen() with Windows. Or use only popen()?
*/
bool
exec_prog(const char *log_file, const char *opt_log_file,
- bool report_error, bool exit_on_error, const char *fmt,...)
+ bool report_error, bool exit_on_error, unsigned char *encryption_key,
+ const char *fmt,...)
{
int result = 0;
int written;
@@ -170,7 +176,27 @@ exec_prog(const char *log_file, const char *opt_log_file,
/* see comment above */
if (mainThreadId == GetCurrentThreadId())
#endif
- result = system(cmd);
+ {
+ if (!encryption_key)
+ result = system(cmd);
+ else
+ {
+ FILE *fp;
+ int i;
+
+ fp = popen(cmd, "w");
+ if (fp == NULL)
+ pg_fatal("Failed to execute \"%s\"\n", cmd);
+
+ /* Send the key. */
+ for (i = 0; i < ENCRYPTION_KEY_LENGTH; i++)
+ fprintf(fp, "%.2x", encryption_key[i]);
+ fputc('\n', fp);
+
+ if (pclose(fp))
+ pg_fatal("\"%s\" returned non-zero code\n", cmd);
+ }
+ }
if (result != 0 && report_error)
{
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index 73f395f2a3..e2bb0e7c8e 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"encryption-key-command", required_argument, NULL, 'K'},
{NULL, 0, NULL, 0}
};
@@ -101,7 +102,7 @@ parseCommandLine(int argc, char *argv[])
if (os_user_effective_id == 0)
pg_fatal("%s: cannot be run as root\n", os_info.progname);
- while ((option = getopt_long(argc, argv, "d:D:b:B:cj:ko:O:p:P:rs:U:v",
+ while ((option = getopt_long(argc, argv, "d:D:b:B:cj:K:ko:O:p:P:rs:U:v",
long_options, &optindex)) != -1)
{
switch (option)
@@ -209,6 +210,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 'K':
+ encryption_key_command = pg_strdup(optarg);
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -309,6 +314,9 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+#ifdef USE_ENCRYPTION
+ printf(_(" -K, --encryption-key-command command that returns encryption key\n\n"));
+#endif /* USE_ENCRYPTION */
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/parallel.c b/src/bin/pg_upgrade/parallel.c
index 80ab1b8609..be3fd51f15 100644
--- a/src/bin/pg_upgrade/parallel.c
+++ b/src/bin/pg_upgrade/parallel.c
@@ -79,7 +79,7 @@ parallel_exec_prog(const char *log_file, const char *opt_log_file,
if (user_opts.jobs <= 1)
/* exit_on_error must be true to allow jobs */
- exec_prog(log_file, opt_log_file, true, true, "%s", cmd);
+ exec_prog(log_file, opt_log_file, true, true, NULL, "%s", cmd);
else
{
/* parallel */
@@ -122,7 +122,8 @@ parallel_exec_prog(const char *log_file, const char *opt_log_file,
child = fork();
if (child == 0)
/* use _exit to skip atexit() functions */
- _exit(!exec_prog(log_file, opt_log_file, true, true, "%s", cmd));
+ _exit(!exec_prog(log_file, opt_log_file, true, true, NULL, "%s",
+ cmd));
else if (child < 0)
/* fork failed */
pg_fatal("could not create worker process: %s\n", strerror(errno));
@@ -160,7 +161,7 @@ win32_exec_prog(exec_thread_arg *args)
{
int ret;
- ret = !exec_prog(args->log_file, args->opt_log_file, true, true, "%s", args->cmd);
+ ret = !exec_prog(args->log_file, args->opt_log_file, true, true, NULL, "%s", args->cmd);
/* terminates thread */
return ret;
diff --git a/src/bin/pg_upgrade/pg_upgrade.c b/src/bin/pg_upgrade/pg_upgrade.c
index 336df7378f..c464f90cf9 100644
--- a/src/bin/pg_upgrade/pg_upgrade.c
+++ b/src/bin/pg_upgrade/pg_upgrade.c
@@ -73,6 +73,12 @@ char *output_files[] = {
NULL
};
+/*
+ * Declare these locally so we don't have to link storage/file/encryption.c
+ * here.
+ */
+bool encryption_setup_done = false;
+unsigned char encryption_key[ENCRYPTION_KEY_LENGTH];
int
main(int argc, char **argv)
@@ -94,6 +100,20 @@ main(int argc, char **argv)
adjust_data_dir(&old_cluster);
adjust_data_dir(&new_cluster);
+ /*
+ * The encryption key is needed to start the clusters.
+ */
+ if (encryption_key_command)
+ {
+ /*
+ * Both clusters should have the same KDF parameters, so we can pass
+ * pgdata of any one.
+ */
+ run_encryption_key_command(encryption_key, old_cluster.pgdata);
+
+ encryption_setup_done = true;
+ }
+
setup(argv[0], &live_check);
output_check_banner(live_check);
@@ -139,6 +159,16 @@ main(int argc, char **argv)
copy_xact_xlog_xid();
+ if (encryption_setup_done)
+ {
+ /*
+ * Copy KDF file so that the old cluster encryption password works for
+ * the new cluster.
+ */
+ read_kdf_file(old_cluster.pgdata);
+ write_kdf_file(new_cluster.pgdata);
+ }
+
/* New now using xids of the old system */
/* -- NEW -- */
@@ -170,13 +200,15 @@ main(int argc, char **argv)
*/
prep_status("Setting next OID for new cluster");
exec_prog(UTILITY_LOG_FILE, NULL, true, true,
+ encryption_setup_done ? encryption_key : NULL,
"\"%s/pg_resetwal\" -o %u \"%s\"",
- new_cluster.bindir, old_cluster.controldata.chkpnt_nxtoid,
+ new_cluster.bindir,
+ old_cluster.controldata.chkpnt_nxtoid,
new_cluster.pgdata);
check_ok();
prep_status("Sync data directory to disk");
- exec_prog(UTILITY_LOG_FILE, NULL, true, true,
+ exec_prog(UTILITY_LOG_FILE, NULL, true, true, NULL,
"\"%s/initdb\" --sync-only \"%s\"", new_cluster.bindir,
new_cluster.pgdata);
check_ok();
@@ -270,7 +302,7 @@ prepare_new_cluster(void)
* --analyze so autovacuum doesn't update statistics later
*/
prep_status("Analyzing all rows in the new cluster");
- exec_prog(UTILITY_LOG_FILE, NULL, true, true,
+ exec_prog(UTILITY_LOG_FILE, NULL, true, true, NULL,
"\"%s/vacuumdb\" %s --all --analyze %s",
new_cluster.bindir, cluster_conn_opts(&new_cluster),
log_opts.verbose ? "--verbose" : "");
@@ -283,7 +315,7 @@ prepare_new_cluster(void)
* counter later.
*/
prep_status("Freezing all rows in the new cluster");
- exec_prog(UTILITY_LOG_FILE, NULL, true, true,
+ exec_prog(UTILITY_LOG_FILE, NULL, true, true, NULL,
"\"%s/vacuumdb\" %s --all --freeze %s",
new_cluster.bindir, cluster_conn_opts(&new_cluster),
log_opts.verbose ? "--verbose" : "");
@@ -304,7 +336,7 @@ prepare_new_globals(void)
*/
prep_status("Restoring global objects in the new cluster");
- exec_prog(UTILITY_LOG_FILE, NULL, true, true,
+ exec_prog(UTILITY_LOG_FILE, NULL, true, true, NULL,
"\"%s/psql\" " EXEC_PSQL_ARGS " %s -f \"%s\"",
new_cluster.bindir, cluster_conn_opts(&new_cluster),
GLOBALS_DUMP_FILE);
@@ -351,6 +383,7 @@ create_new_objects(void)
NULL,
true,
true,
+ NULL,
"\"%s/pg_restore\" %s %s --exit-on-error --verbose "
"--dbname postgres \"%s\"",
new_cluster.bindir,
@@ -448,7 +481,7 @@ copy_subdir_files(const char *old_subdir, const char *new_subdir)
prep_status("Copying old %s to new server", old_subdir);
- exec_prog(UTILITY_LOG_FILE, NULL, true, true,
+ exec_prog(UTILITY_LOG_FILE, NULL, true, true, NULL,
#ifndef WIN32
"cp -Rf \"%s\" \"%s\"",
#else
@@ -475,15 +508,20 @@ copy_xact_xlog_xid(void)
/* set the next transaction id and epoch of the new cluster */
prep_status("Setting next transaction ID and epoch for new cluster");
exec_prog(UTILITY_LOG_FILE, NULL, true, true,
+ encryption_setup_done ? encryption_key : NULL,
"\"%s/pg_resetwal\" -f -x %u \"%s\"",
- new_cluster.bindir, old_cluster.controldata.chkpnt_nxtxid,
+ new_cluster.bindir,
+ old_cluster.controldata.chkpnt_nxtxid,
new_cluster.pgdata);
exec_prog(UTILITY_LOG_FILE, NULL, true, true,
+ encryption_setup_done ? encryption_key : NULL,
"\"%s/pg_resetwal\" -f -e %u \"%s\"",
- new_cluster.bindir, old_cluster.controldata.chkpnt_nxtepoch,
+ new_cluster.bindir,
+ old_cluster.controldata.chkpnt_nxtepoch,
new_cluster.pgdata);
/* must reset commit timestamp limits also */
exec_prog(UTILITY_LOG_FILE, NULL, true, true,
+ encryption_setup_done ? encryption_key : NULL,
"\"%s/pg_resetwal\" -f -c %u,%u \"%s\"",
new_cluster.bindir,
old_cluster.controldata.chkpnt_nxtxid,
@@ -510,6 +548,7 @@ copy_xact_xlog_xid(void)
* counters here and the oldest multi present on system.
*/
exec_prog(UTILITY_LOG_FILE, NULL, true, true,
+ encryption_setup_done ? encryption_key : NULL,
"\"%s/pg_resetwal\" -O %u -m %u,%u \"%s\"",
new_cluster.bindir,
old_cluster.controldata.chkpnt_nxtmxoff,
@@ -538,6 +577,7 @@ copy_xact_xlog_xid(void)
* next=MaxMultiXactId, but multixact.c can cope with that just fine.
*/
exec_prog(UTILITY_LOG_FILE, NULL, true, true,
+ encryption_setup_done ? encryption_key : NULL,
"\"%s/pg_resetwal\" -m %u,%u \"%s\"",
new_cluster.bindir,
old_cluster.controldata.chkpnt_nxtmulti + 1,
@@ -549,6 +589,7 @@ copy_xact_xlog_xid(void)
/* now reset the wal archives in the new cluster */
prep_status("Resetting WAL archives");
exec_prog(UTILITY_LOG_FILE, NULL, true, true,
+ encryption_setup_done ? encryption_key : NULL,
/* use timeline 1 to match controldata and no WAL history file */
"\"%s/pg_resetwal\" -l 00000001%s \"%s\"", new_cluster.bindir,
old_cluster.controldata.nextxlogfile + 8,
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 5d31750d86..e89f6c08a0 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -10,7 +10,10 @@
#include <sys/stat.h>
#include <sys/time.h>
+#include "fe_utils/encryption.h"
#include "libpq-fe.h"
+#include "storage/relfilenode.h"
+#include "storage/encryption.h"
/* Use port in the private/dynamic port number range */
#define DEF_PGUPORT 50432
@@ -227,6 +230,8 @@ typedef struct
bool date_is_int;
bool float8_pass_by_value;
bool data_checksum_version;
+ bool data_encrypted;
+ uint8 encryption_verification[ENCRYPTION_SAMPLE_SIZE];
} ControlData;
/*
@@ -367,7 +372,9 @@ void generate_old_dump(void);
#define EXEC_PSQL_ARGS "--echo-queries --set ON_ERROR_STOP=on --no-psqlrc --dbname=template1"
bool exec_prog(const char *log_file, const char *opt_log_file,
- bool report_error, bool exit_on_error, const char *fmt,...) pg_attribute_printf(5, 6);
+ bool report_error, bool exit_on_error,
+ unsigned char *encryption_key, const char *fmt,...)
+ pg_attribute_printf(6, 7);
void verify_directories(void);
bool pid_lock_file_exists(const char *datadir);
diff --git a/src/bin/pg_upgrade/server.c b/src/bin/pg_upgrade/server.c
index 6ad4b14e16..93cb518cc5 100644
--- a/src/bin/pg_upgrade/server.c
+++ b/src/bin/pg_upgrade/server.c
@@ -252,6 +252,41 @@ start_postmaster(ClusterInfo *cluster, bool report_and_exit_on_error)
cluster->pgopts ? cluster->pgopts : "", socket_string);
/*
+ * If encryption key needs to be sent, run a separate process now and let
+ * it send the password to the postmaster. We cannot send the key later
+ * in the current process because the exec_prog call below blocks until
+ * the postmaster succeeds or fails to start (and it will definitely fail
+ * if it receives no key).
+ */
+ if (encryption_setup_done)
+ {
+#ifndef WIN32
+ pid_t sender;
+
+ sender = fork();
+ if (sender == 0)
+ {
+ char port_str[6];
+
+ snprintf(port_str, sizeof(port_str), "%d", cluster->port);
+
+ /* in child process */
+ send_key_to_postmaster(cluster->sockdir, port_str,
+ encryption_key);
+ exit(EXIT_SUCCESS);
+ }
+ else if (sender < 0)
+ {
+ pg_fatal("could not create key sender process");
+ exit(EXIT_FAILURE);
+ }
+#else
+ /* TODO */
+ #error "W32 not implemented yet"
+#endif
+ }
+
+ /*
* Don't throw an error right away, let connecting throw the error because
* it might supply a reason for the failure.
*/
@@ -260,7 +295,7 @@ start_postmaster(ClusterInfo *cluster, bool report_and_exit_on_error)
(strcmp(SERVER_LOG_FILE,
SERVER_START_LOG_FILE) != 0) ?
SERVER_LOG_FILE : NULL,
- report_and_exit_on_error, false,
+ report_and_exit_on_error, false, NULL,
"%s", cmd);
/* Did it fail and we are just testing if the server could be started? */
@@ -336,7 +371,7 @@ stop_postmaster(bool in_atexit)
else
return; /* no cluster running */
- exec_prog(SERVER_STOP_LOG_FILE, NULL, !in_atexit, !in_atexit,
+ exec_prog(SERVER_STOP_LOG_FILE, NULL, !in_atexit, !in_atexit, NULL,
"\"%s/pg_ctl\" -w -D \"%s\" -o \"%s\" %s stop",
cluster->bindir, cluster->pgconfig,
cluster->pgopts ? cluster->pgopts : "",
diff --git a/src/fe_utils/encryption.c b/src/fe_utils/encryption.c
index 134b3bde9b..2b4a6fadad 100644
--- a/src/fe_utils/encryption.c
+++ b/src/fe_utils/encryption.c
@@ -259,8 +259,6 @@ run_encryption_key_command(unsigned char *encryption_key, char *data_dir)
FILE *fp;
char cmd[MAXPGPATH];
char *sp, *dp, *endp;
- char *buf;
- int read_len, i, c;
Assert(encryption_key_command != NULL &&
strlen(encryption_key_command) > 0);
@@ -312,15 +310,25 @@ run_encryption_key_command(unsigned char *encryption_key, char *data_dir)
exit(EXIT_FAILURE);
}
+ /* Read the key. */
+ read_encryption_key_fe(fp);
+
+ pclose(fp);
+}
+
+/*
+ * Frontend counterpart of read_encryption_key().
+ */
+void
+read_encryption_key_fe(FILE *f)
+{
+ char *buf;
+ int read_len, i, c;
+
buf = (char *) palloc(ENCRYPTION_KEY_CHARS);
- /*
- * Read the key. This is very similar to backend's read_encryption_key()
- * but there seems to be no straightforward way to call the function from
- * here.
- */
read_len = 0;
- while ((c = fgetc(fp)) != EOF && c != '\n')
+ while ((c = fgetc(f)) != EOF && c != '\n')
{
if (read_len >= ENCRYPTION_KEY_CHARS)
{
@@ -349,7 +357,6 @@ run_encryption_key_command(unsigned char *encryption_key, char *data_dir)
}
pfree(buf);
- pclose(fp);
}
/*
diff --git a/src/include/fe_utils/encryption.h b/src/include/fe_utils/encryption.h
index da302fe494..2c6032df07 100644
--- a/src/include/fe_utils/encryption.h
+++ b/src/include/fe_utils/encryption.h
@@ -22,5 +22,6 @@ extern void derive_key_from_password(unsigned char *encryption_key,
const char *password, int len);
extern void run_encryption_key_command(unsigned char *encryption_key,
char *data_dir);
+extern void read_encryption_key_fe(FILE *f);
extern bool send_key_to_postmaster(const char *host, const char *port,
const unsigned char *encryption_Key);
--
2.13.7
v04-0016-Teach-the-remaining-frontend-application-about-encry.patch (text/x-diff, 23.6K) download
From 26dc3cd48f082de47198e0975fbb7f1f971e2896 Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Fri, 5 Jul 2019 16:24:02 +0200
Subject: [PATCH 16/17] Teach the remaining frontend application about
encryption.
pg_checksums, pg_rewind, pg_waldump
---
src/bin/pg_checksums/Makefile | 9 +-
src/bin/pg_checksums/pg_checksums.c | 158 +++++++++++++++++++++++++++++++++---
src/bin/pg_rewind/Makefile | 9 +-
src/bin/pg_rewind/parsexlog.c | 9 ++
src/bin/pg_rewind/pg_rewind.c | 57 ++++++++++++-
src/bin/pg_waldump/Makefile | 9 +-
src/bin/pg_waldump/pg_waldump.c | 90 +++++++++++++++++---
7 files changed, 309 insertions(+), 32 deletions(-)
diff --git a/src/bin/pg_checksums/Makefile b/src/bin/pg_checksums/Makefile
index 278b7a0f2e..74e66ea0b7 100644
--- a/src/bin/pg_checksums/Makefile
+++ b/src/bin/pg_checksums/Makefile
@@ -15,13 +15,19 @@ subdir = src/bin/pg_checksums
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-OBJS= pg_checksums.o $(WIN32RES)
+OBJS= encryption.o pg_checksums.o $(WIN32RES)
+
+override CPPFLAGS := -DFRONTEND $(CPPFLAGS)
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
all: pg_checksums
pg_checksums: $(OBJS) | submake-libpgport
$(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+encryption.c: % : $(top_srcdir)/src/backend/storage/file/%
+ rm -f $@ && $(LN_S) $< .
+
install: all installdirs
$(INSTALL_PROGRAM) pg_checksums$(X) '$(DESTDIR)$(bindir)/pg_checksums$(X)'
@@ -34,6 +40,7 @@ uninstall:
clean distclean maintainer-clean:
rm -f pg_checksums$(X) $(OBJS)
rm -rf tmp_check
+ rm -f encryption.c
check:
$(prove_check)
diff --git a/src/bin/pg_checksums/pg_checksums.c b/src/bin/pg_checksums/pg_checksums.c
index b591fcc864..587132faf9 100644
--- a/src/bin/pg_checksums/pg_checksums.c
+++ b/src/bin/pg_checksums/pg_checksums.c
@@ -20,15 +20,18 @@
#include <unistd.h>
#include "access/xlog_internal.h"
+#include "catalog/pg_tablespace_d.h"
#include "common/controldata_utils.h"
#include "common/file_perm.h"
#include "common/file_utils.h"
#include "common/logging.h"
+#include "fe_utils/encryption.h"
#include "getopt_long.h"
#include "pg_getopt.h"
#include "storage/bufpage.h"
#include "storage/checksum.h"
#include "storage/checksum_impl.h"
+#include "storage/encryption.h"
static int64 files = 0;
@@ -80,6 +83,10 @@ usage(void)
printf(_(" -c, --check check data checksums (default)\n"));
printf(_(" -d, --disable disable data checksums\n"));
printf(_(" -e, --enable enable data checksums\n"));
+#ifdef USE_ENCRYPTION
+ printf(_(" -K, --encryption-key-command=COMMAND\n"
+ " command that returns encryption key\n"));
+#endif /* USE_ENCRYPTION */
printf(_(" -f, --filenode=FILENODE check only relation with specified filenode\n"));
printf(_(" -N, --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" -P, --progress show progress information\n"));
@@ -167,7 +174,8 @@ skipfile(const char *fn)
}
static void
-scan_file(const char *fn, BlockNumber segmentno)
+scan_file(const char *fn, Oid relnode, BlockNumber segmentno,
+ ForkNumber forkno, Oid tbspace, Oid db)
{
PGAlignedBlock buf;
PageHeader header = (PageHeader) buf.data;
@@ -193,6 +201,7 @@ scan_file(const char *fn, BlockNumber segmentno)
{
uint16 csum;
int r = read(f, buf.data, BLCKSZ);
+ char tweak[TWEAK_SIZE];
if (r == 0)
break;
@@ -204,6 +213,18 @@ scan_file(const char *fn, BlockNumber segmentno)
}
blocks++;
+ if (data_encrypted)
+ {
+ RelFileNode node;
+
+ node.spcNode = tbspace;
+ node.dbNode = db;
+ node.relNode = relnode;
+
+ mdtweak(tweak, &node, forkno, blockno);
+ decrypt_block(buf.data, buf.data, BLCKSZ, tweak, false);
+ }
+
/* New pages have no checksum yet */
if (PageIsNew(header))
continue;
@@ -232,6 +253,9 @@ scan_file(const char *fn, BlockNumber segmentno)
exit(1);
}
+ if (data_encrypted)
+ encrypt_block(buf.data, buf.data, BLCKSZ, tweak, false);
+
/* Write block with checksum */
if (write(f, buf.data, BLCKSZ) != BLCKSZ)
{
@@ -262,9 +286,13 @@ scan_file(const char *fn, BlockNumber segmentno)
* all the items which have checksums is computed and returned back
* to the caller without operating on the files. This is used to compile
* the total size of the data directory for progress reports.
+ *
+ * If db is a valid pointer, *db contains database OID. If it's NULL, the
+ * database OID needs to be recognized, possibly by recursive call.
*/
static int64
-scan_directory(const char *basedir, const char *subdir, bool sizeonly)
+scan_directory(const char *basedir, const char *subdir, bool sizeonly,
+ Oid tbspace, Oid *db)
{
int64 dirsize = 0;
char path[MAXPGPATH];
@@ -282,6 +310,9 @@ scan_directory(const char *basedir, const char *subdir, bool sizeonly)
{
char fn[MAXPGPATH];
struct stat st;
+ bool tbsp_identified = false;
+ Oid database;
+ bool db_identified = false;
if (strcmp(de->d_name, ".") == 0 ||
strcmp(de->d_name, "..") == 0)
@@ -305,12 +336,58 @@ scan_directory(const char *basedir, const char *subdir, bool sizeonly)
pg_log_error("could not stat file \"%s\": %m", fn);
exit(1);
}
+
+ if (tbspace != InvalidOid && db == NULL)
+ {
+ /*
+ * If tablespace is passed by caller or identified by upper call
+ * of this function, and if caller could not identify the database
+ * OID, try to do it now.
+ */
+ if (strcmp(de->d_name, TABLESPACE_VERSION_DIRECTORY) == 0)
+ {
+ /*
+ * Major-version-specific tablespace subdirectory needs no
+ * special attention, we'll recurse into it below.
+ */
+ }
+ else
+ {
+ errno = 0;
+ database = strtol(de->d_name, NULL, 10);
+ if (errno != 0 || database == InvalidOid)
+ {
+ fprintf(stderr, _("%s: invalid database oid \"%s\"\n"),
+ progname, de->d_name);
+ exit(1);
+ }
+ db_identified = true;
+ }
+ }
+ else if (tbspace == InvalidOid)
+ {
+ /*
+ * This entry should be a direct subdirectory of pg_tblspc, so the
+ * name is supposedly tablespace oid.
+ */
+ errno = 0;
+ tbspace = strtol(de->d_name, NULL, 10);
+ if (errno != 0 || tbspace == InvalidOid)
+ {
+ fprintf(stderr, _("%s: invalid tablespace oid \"%s\"\n"),
+ progname, de->d_name);
+ exit(1);
+ }
+ tbsp_identified = true;
+ }
+
if (S_ISREG(st.st_mode))
{
char fnonly[MAXPGPATH];
char *forkpath,
*segmentpath;
BlockNumber segmentno = 0;
+ ForkNumber forkno;
if (skipfile(de->d_name))
continue;
@@ -337,10 +414,16 @@ scan_directory(const char *basedir, const char *subdir, bool sizeonly)
forkpath = strchr(fnonly, '_');
if (forkpath != NULL)
+ {
*forkpath++ = '\0';
+ forkno = forkname_to_number(forkpath);
+ }
+ else
+ forkno = MAIN_FORKNUM;
+
if (only_filenode && strcmp(only_filenode, fnonly) != 0)
- /* filenode not to be included */
+ /* Relfilenode not to be included */
continue;
dirsize += st.st_size;
@@ -350,14 +433,31 @@ scan_directory(const char *basedir, const char *subdir, bool sizeonly)
* the items in the data folder.
*/
if (!sizeonly)
- scan_file(fn, segmentno);
+ {
+ Oid relnode = atoi(fnonly);
+
+ scan_file(fn, relnode, segmentno, forkno, tbspace, *db);
+ }
}
#ifndef WIN32
else if (S_ISDIR(st.st_mode) || S_ISLNK(st.st_mode))
#else
else if (S_ISDIR(st.st_mode) || pgwin32_is_junction(fn))
#endif
- dirsize += scan_directory(path, de->d_name, sizeonly);
+
+ /*
+ * If database OID is not passed by caller, use the one we
+ * identified ourselves.
+ */
+ dirsize = scan_directory(path, de->d_name, sizeonly, tbspace,
+ db_identified ? &database : db);
+
+ /*
+ * If tablespace is not passed by caller, forget what we found out so
+ * that next directory entry is examined in the same way.
+ */
+ if (tbsp_identified)
+ tbspace = InvalidOid;
}
closedir(dir);
return dirsize;
@@ -371,6 +471,9 @@ main(int argc, char *argv[])
{"pgdata", required_argument, NULL, 'D'},
{"disable", no_argument, NULL, 'd'},
{"enable", no_argument, NULL, 'e'},
+#ifdef USE_ENCRYPTION
+ {"encryption-key-command", required_argument, NULL, 'K'},
+#endif /* USE_ENCRYPTION */
{"filenode", required_argument, NULL, 'f'},
{"no-sync", no_argument, NULL, 'N'},
{"progress", no_argument, NULL, 'P'},
@@ -401,7 +504,7 @@ main(int argc, char *argv[])
}
}
- while ((c = getopt_long(argc, argv, "cD:deNPf:v", long_options, &option_index)) != -1)
+ while ((c = getopt_long(argc, argv, "cD:deNK:Pf:v", long_options, &option_index)) != -1)
{
switch (c)
{
@@ -422,6 +525,12 @@ main(int argc, char *argv[])
}
only_filenode = pstrdup(optarg);
break;
+#ifdef USE_ENCRYPTION
+ case 'K':
+ encryption_key_command = pg_strdup(optarg);
+ data_encrypted = true;
+ break;
+#endif /* USE_ENCRYPTION */
case 'N':
do_sync = false;
break;
@@ -483,6 +592,24 @@ main(int argc, char *argv[])
exit(1);
}
+ if (ControlFile->data_cipher > PG_CIPHER_NONE)
+ {
+ if (encryption_key_command == NULL)
+ {
+#ifdef USE_ENCRYPTION
+ fprintf(stderr, _("%s: please specify command to retrieve encryption key\n"),
+ progname);
+#else
+ fprintf(stderr, _("%s: compile postgres with --with-openssl to use encryption\n"),
+ progname);
+#endif /* USE_ENCRYPTION */
+ exit(1);
+ }
+
+ run_encryption_key_command(encryption_key, DataDir);
+ setup_encryption();
+ }
+
if (ControlFile->pg_control_version != PG_CONTROL_VERSION)
{
pg_log_error("cluster is not compatible with this version of pg_checksums");
@@ -533,6 +660,8 @@ main(int argc, char *argv[])
/* Operate on all files if checking or enabling checksums */
if (mode == PG_MODE_CHECK || mode == PG_MODE_ENABLE)
{
+ Oid db_shared = 0;
+
/*
* If progress status information is requested, we need to scan the
* directory tree twice: once to know how much total data needs to be
@@ -540,14 +669,19 @@ main(int argc, char *argv[])
*/
if (showprogress)
{
- total_size = scan_directory(DataDir, "global", true);
- total_size += scan_directory(DataDir, "base", true);
- total_size += scan_directory(DataDir, "pg_tblspc", true);
+ total_size = scan_directory(DataDir, "global", true,
+ GLOBALTABLESPACE_OID, &db_shared);
+ total_size += scan_directory(DataDir, "base", true,
+ DEFAULTTABLESPACE_OID, NULL);
+ total_size += scan_directory(DataDir, "pg_tblspc", true,
+ InvalidOid, NULL);
}
- (void) scan_directory(DataDir, "global", false);
- (void) scan_directory(DataDir, "base", false);
- (void) scan_directory(DataDir, "pg_tblspc", false);
+ (void) scan_directory(DataDir, "global", false, GLOBALTABLESPACE_OID,
+ &db_shared);
+ (void) scan_directory(DataDir, "base", false, DEFAULTTABLESPACE_OID,
+ NULL);
+ (void) scan_directory(DataDir, "pg_tblspc", false, InvalidOid, NULL);
if (showprogress)
{
diff --git a/src/bin/pg_rewind/Makefile b/src/bin/pg_rewind/Makefile
index 859d3abc41..e41d2035f4 100644
--- a/src/bin/pg_rewind/Makefile
+++ b/src/bin/pg_rewind/Makefile
@@ -16,10 +16,10 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
override CPPFLAGS := -I$(libpq_srcdir) -DFRONTEND $(CPPFLAGS)
-LDFLAGS_INTERNAL += $(libpq_pgport)
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
OBJS = pg_rewind.o parsexlog.o xlogreader.o datapagemap.o timeline.o \
- fetch.o file_ops.o copy_fetch.o libpq_fetch.o filemap.o \
+ fetch.o file_ops.o copy_fetch.o libpq_fetch.o filemap.o encryption.o\
$(WIN32RES)
EXTRA_CLEAN = xlogreader.c
@@ -32,6 +32,9 @@ pg_rewind: $(OBJS) | submake-libpq submake-libpgport
xlogreader.c: % : $(top_srcdir)/src/backend/access/transam/%
rm -f $@ && $(LN_S) $< .
+encryption.c: % : $(top_srcdir)/src/backend/storage/file/%
+ rm -f $@ && $(LN_S) $< .
+
install: all installdirs
$(INSTALL_PROGRAM) pg_rewind$(X) '$(DESTDIR)$(bindir)/pg_rewind$(X)'
@@ -42,7 +45,7 @@ uninstall:
rm -f '$(DESTDIR)$(bindir)/pg_rewind$(X)'
clean distclean maintainer-clean:
- rm -f pg_rewind$(X) $(OBJS) xlogreader.c
+ rm -f pg_rewind$(X) $(OBJS) xlogreader.c encryption.c
rm -rf tmp_check
check:
diff --git a/src/bin/pg_rewind/parsexlog.c b/src/bin/pg_rewind/parsexlog.c
index 287af60c4e..0624c4c5ef 100644
--- a/src/bin/pg_rewind/parsexlog.c
+++ b/src/bin/pg_rewind/parsexlog.c
@@ -320,6 +320,15 @@ SimpleXLogPageRead(XLogReaderState *xlogreader, XLogRecPtr targetPagePtr,
return -1;
}
+ if (data_encrypted)
+ {
+ char tweak[TWEAK_SIZE];
+
+ XLogEncryptionTweak(tweak, targetHistory[private->tliIndex].tli,
+ xlogreadsegno, targetPageOff);
+ decrypt_block(readBuf, readBuf, XLOG_BLCKSZ, tweak, true);
+ }
+
Assert(targetSegNo == xlogreadsegno);
*pageTLI = targetHistory[private->tliIndex].tli;
diff --git a/src/bin/pg_rewind/pg_rewind.c b/src/bin/pg_rewind/pg_rewind.c
index d378053de4..58c192adad 100644
--- a/src/bin/pg_rewind/pg_rewind.c
+++ b/src/bin/pg_rewind/pg_rewind.c
@@ -27,8 +27,10 @@
#include "common/file_perm.h"
#include "common/file_utils.h"
#include "common/restricted_token.h"
+#include "fe_utils/encryption.h"
#include "getopt_long.h"
#include "storage/bufpage.h"
+#include "storage/encryption.h"
static void usage(const char *progname);
@@ -75,6 +77,10 @@ usage(const char *progname)
printf(_(" -D, --target-pgdata=DIRECTORY existing data directory to modify\n"));
printf(_(" --source-pgdata=DIRECTORY source data directory to synchronize with\n"));
printf(_(" --source-server=CONNSTR source server to synchronize with\n"));
+#ifdef USE_ENCRYPTION
+ printf(_(" -K, --encryption-key-command=COMMAND\n"
+ " command that returns encryption key\n"));
+#endif /* USE_ENCRYPTION */
printf(_(" -n, --dry-run stop before modifying anything\n"));
printf(_(" -N, --no-sync do not wait for changes to be written\n"
" safely to disk\n"));
@@ -99,6 +105,9 @@ main(int argc, char **argv)
{"no-sync", no_argument, NULL, 'N'},
{"progress", no_argument, NULL, 'P'},
{"debug", no_argument, NULL, 3},
+#ifdef USE_ENCRYPTION
+ {"encryption-key-command", required_argument, NULL, 'K'},
+#endif /* USE_ENCRYPTION */
{NULL, 0, NULL, 0}
};
int option_index;
@@ -134,7 +143,7 @@ main(int argc, char **argv)
}
}
- while ((c = getopt_long(argc, argv, "D:nNP", long_options, &option_index)) != -1)
+ while ((c = getopt_long(argc, argv, "D:K:nNP", long_options, &option_index)) != -1)
{
switch (c)
{
@@ -169,6 +178,12 @@ main(int argc, char **argv)
case 2: /* --source-server */
connstr_source = pg_strdup(optarg);
break;
+#ifdef USE_ENCRYPTION
+ case 4: /* --encryption-key-command */
+ case 'K':
+ encryption_key_command = strdup(optarg);
+ break;
+#endif /* USE_ENCRYPTION */
}
}
@@ -248,6 +263,28 @@ main(int argc, char **argv)
sanityChecks();
/*
+ * Setup encryption if it's obvious that we'll have to deal with encrypted
+ * XLOG.
+ */
+ if (ControlFile_target.data_cipher > PG_CIPHER_NONE)
+ {
+ if (encryption_key_command == NULL)
+ {
+ pg_log_error("-K option must be passed for encrypted cluster");
+ exit(EXIT_FAILURE);
+ }
+
+ /*
+ * It should not matter whether we pass the source or target data
+ * directory. It should have been checked earlier that both clusters
+ * are encrypted using the same key.
+ */
+ run_encryption_key_command(encryption_key, datadir_source);
+ setup_encryption();
+ data_encrypted = true;
+ }
+
+ /*
* If both clusters are already on the same timeline, there's nothing to
* do.
*/
@@ -446,6 +483,24 @@ sanityChecks(void)
ControlFile_source.state != DB_SHUTDOWNED &&
ControlFile_source.state != DB_SHUTDOWNED_IN_RECOVERY)
pg_fatal("source data directory must be shut down cleanly");
+
+ /*
+ * Since slave receives XLOG stream encrypted by master, handling
+ * differently encrypted clusters is not the typical use case for
+ * pg_rewind. Yet we should check the encryption.
+ */
+ if (ControlFile_source.data_cipher > PG_CIPHER_NONE ||
+ ControlFile_target.data_cipher > PG_CIPHER_NONE)
+ {
+ if (ControlFile_source.data_cipher !=
+ ControlFile_target.data_cipher)
+ pg_fatal("source and target server must be both unencrypted or both encrypted\n");
+
+ if (memcmp(ControlFile_source.encryption_verification,
+ ControlFile_target.encryption_verification,
+ ENCRYPTION_SAMPLE_SIZE))
+ pg_fatal("both source and target server must use the same encryption key");
+ }
}
/*
diff --git a/src/bin/pg_waldump/Makefile b/src/bin/pg_waldump/Makefile
index 135979cef3..9eb1c2f01d 100644
--- a/src/bin/pg_waldump/Makefile
+++ b/src/bin/pg_waldump/Makefile
@@ -7,10 +7,11 @@ subdir = src/bin/pg_waldump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-OBJS = pg_waldump.o compat.o xlogreader.o rmgrdesc.o \
+OBJS = pg_waldump.o compat.o xlogreader.o rmgrdesc.o encryption.o \
$(RMGRDESCOBJS) $(WIN32RES)
override CPPFLAGS := -DFRONTEND $(CPPFLAGS)
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
RMGRDESCSOURCES = $(sort $(notdir $(wildcard $(top_srcdir)/src/backend/access/rmgrdesc/*desc.c)))
RMGRDESCOBJS = $(patsubst %.c,%.o,$(RMGRDESCSOURCES))
@@ -27,6 +28,9 @@ xlogreader.c: % : $(top_srcdir)/src/backend/access/transam/%
$(RMGRDESCSOURCES): % : $(top_srcdir)/src/backend/access/rmgrdesc/%
rm -f $@ && $(LN_S) $< .
+encryption.c: % : $(top_srcdir)/src/backend/storage/file/%
+ rm -f $@ && $(LN_S) $< .
+
install: all installdirs
$(INSTALL_PROGRAM) pg_waldump$(X) '$(DESTDIR)$(bindir)/pg_waldump$(X)'
@@ -37,7 +41,8 @@ uninstall:
rm -f '$(DESTDIR)$(bindir)/pg_waldump$(X)'
clean distclean maintainer-clean:
- rm -f pg_waldump$(X) $(OBJS) $(RMGRDESCSOURCES) xlogreader.c
+ rm -f pg_waldump$(X) $(OBJS) $(RMGRDESCSOURCES) xlogreader.c \
+encryption.c
rm -rf tmp_check
check:
diff --git a/src/bin/pg_waldump/pg_waldump.c b/src/bin/pg_waldump/pg_waldump.c
index b95d467805..4df84efb01 100644
--- a/src/bin/pg_waldump/pg_waldump.c
+++ b/src/bin/pg_waldump/pg_waldump.c
@@ -22,6 +22,8 @@
#include "access/transam.h"
#include "common/fe_memutils.h"
#include "common/logging.h"
+#include "fe_utils/encryption.h"
+#include "storage/encryption.h"
#include "getopt_long.h"
#include "rmgrdesc.h"
@@ -194,27 +196,59 @@ search_directory(const char *directory, const char *fname)
PGAlignedXLogBlock buf;
int r;
- r = read(fd, buf.data, XLOG_BLCKSZ);
- if (r == XLOG_BLCKSZ)
+ if (data_encrypted)
{
- XLogLongPageHeader longhdr = (XLogLongPageHeader) buf.data;
-
- WalSegSz = longhdr->xlp_seg_size;
+ /*
+ * Segment size affects calculation of segNo and thus also the
+ * encryption tweak, so we cannot get the size from the header
+ * until the page is decrypted. Here we need to take more
+ * expensive approach and really check the file size.
+ */
+ WalSegSz = (int) lseek(fd, 0, SEEK_END);
+ if (WalSegSz <= 0)
+ fatal_error("Could not determine size of WAL segment \"%s\"", fname);
+ /*
+ * Verification of the file size is the only useful thing we can
+ * do. If anything else is wrong, the XLOG reader should find out
+ * after decryption.
+ */
if (!IsValidWalSegSize(WalSegSz))
- fatal_error(ngettext("WAL segment size must be a power of two between 1 MB and 1 GB, but the WAL file \"%s\" header specifies %d byte",
- "WAL segment size must be a power of two between 1 MB and 1 GB, but the WAL file \"%s\" header specifies %d bytes",
+ fatal_error(ngettext("WAL segment size must be a power of two between 1 MB and 1 GB, but size of the WAL file \"%s\" is %d byte",
+ "WAL segment size must be a power of two between 1 MB and 1 GB, but size of the WAL file \"%s\" is %d bytes",
WalSegSz),
fname, WalSegSz);
}
else
{
- if (errno != 0)
- fatal_error("could not read file \"%s\": %s",
- fname, strerror(errno));
+ r = read(fd, buf.data, XLOG_BLCKSZ);
+ if (r == XLOG_BLCKSZ)
+ {
+ XLogLongPageHeader longhdr;
+
+ longhdr = (XLogLongPageHeader) buf.data;
+
+ if (longhdr->std.xlp_magic != XLOG_PAGE_MAGIC)
+ fatal_error(gettext("WAL segment \"%s\" has an incorrect magic number. If it's encrypted, use the -K option to pass encryption credentials"),
+ fname);
+
+ WalSegSz = longhdr->xlp_seg_size;
+
+ if (!IsValidWalSegSize(WalSegSz))
+ fatal_error(ngettext("WAL segment size must be a power of two between 1 MB and 1 GB, but the WAL file \"%s\" header specifies %d byte",
+ "WAL segment size must be a power of two between 1 MB and 1 GB, but the WAL file \"%s\" header specifies %d bytes",
+ WalSegSz),
+ fname, WalSegSz);
+ }
else
- fatal_error("could not read file \"%s\": read %d of %zu",
- fname, r, (Size) XLOG_BLCKSZ);
+ {
+ if (errno != 0)
+ fatal_error("could not read file \"%s\": %s",
+ fname, strerror(errno));
+ else
+ fatal_error("could not read file \"%s\": read %d of %zu",
+ fname, r, (Size) XLOG_BLCKSZ);
+ }
}
close(fd);
return true;
@@ -444,6 +478,19 @@ XLogDumpReadPage(XLogReaderState *state, XLogRecPtr targetPagePtr, int reqLen,
XLogDumpXLogRead(private->inpath, private->timeline, targetPagePtr,
readBuff, count);
+ if (data_encrypted)
+ {
+ char tweak[TWEAK_SIZE];
+ XLogSegNo readSegNo;
+ uint32 readSegOff;
+
+ XLByteToSeg(targetPagePtr, readSegNo, WalSegSz);
+ readSegOff = targetPagePtr % WalSegSz;
+
+ XLogEncryptionTweak(tweak, private->timeline, readSegNo, readSegOff);
+ decrypt_block(readBuff, readBuff, count, tweak, true);
+ }
+
return count;
}
@@ -792,6 +839,10 @@ usage(void)
printf(_(" -b, --bkp-details output detailed information about backup blocks\n"));
printf(_(" -e, --end=RECPTR stop reading at WAL location RECPTR\n"));
printf(_(" -f, --follow keep retrying after reaching end of WAL\n"));
+#ifdef USE_ENCRYPTION
+ printf(_(" -K, --encryption-key-command=COMMAND\n"
+ " command that returns encryption key\n"));
+#endif /* USE_ENCRYPTION */
printf(_(" -n, --limit=N number of records to display\n"));
printf(_(" -p, --path=PATH directory in which to find log segment files or a\n"
" directory with a ./pg_wal that contains such files\n"
@@ -827,6 +878,7 @@ main(int argc, char **argv)
{"end", required_argument, NULL, 'e'},
{"follow", no_argument, NULL, 'f'},
{"help", no_argument, NULL, '?'},
+ {"encryption-key-command", required_argument, NULL, 'K'},
{"limit", required_argument, NULL, 'n'},
{"path", required_argument, NULL, 'p'},
{"rmgr", required_argument, NULL, 'r'},
@@ -884,7 +936,7 @@ main(int argc, char **argv)
goto bad_argument;
}
- while ((option = getopt_long(argc, argv, "be:fn:p:r:s:t:x:z",
+ while ((option = getopt_long(argc, argv, "be:fK:n:p:r:s:t:x:z",
long_options, &optindex)) != -1)
{
switch (option)
@@ -904,6 +956,12 @@ main(int argc, char **argv)
case 'f':
config.follow = true;
break;
+#ifdef USE_ENCRYPTION
+ case 'K':
+ encryption_key_command = pg_strdup(optarg);
+ data_encrypted = true;
+ break;
+#endif /* USE_ENCRYPTION */
case 'n':
if (sscanf(optarg, "%d", &config.stop_after_records) != 1)
{
@@ -1099,6 +1157,12 @@ main(int argc, char **argv)
/* done with argument parsing, do the actual work */
+ if (data_encrypted)
+ {
+ run_encryption_key_command(encryption_key, NULL);
+ setup_encryption();
+ }
+
/* we have everything we need, start reading */
xlogreader_state = XLogReaderAllocate(WalSegSz, XLogDumpReadPage,
&private);
--
2.13.7
v04-0017-User-and-development-documentation.patch (text/x-diff, 30.7K) download
From 96bd043f2f61e0378674216eb6e4c02ec5f4a963 Mon Sep 17 00:00:00 2001
From: Antonin Houska <ah@cybertec.at>
Date: Fri, 5 Jul 2019 16:24:02 +0200
Subject: [PATCH 17/17] User and development documentation.
---
doc/src/sgml/config.sgml | 14 ++
doc/src/sgml/encryption.sgml | 95 ++++++++++++++
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/installation.sgml | 14 +-
doc/src/sgml/postgres.sgml | 1 +
doc/src/sgml/ref/allfiles.sgml | 1 +
doc/src/sgml/ref/initdb.sgml | 11 ++
doc/src/sgml/ref/pg_checksums.sgml | 13 +-
doc/src/sgml/ref/pg_ctl-ref.sgml | 11 ++
doc/src/sgml/ref/pg_keytool.sgml | 121 +++++++++++++++++
doc/src/sgml/ref/pg_resetwal.sgml | 11 ++
doc/src/sgml/ref/pg_rewind.sgml | 11 ++
doc/src/sgml/ref/pg_waldump.sgml | 11 ++
doc/src/sgml/ref/pgupgrade.sgml | 31 +++++
doc/src/sgml/reference.sgml | 1 +
src/backend/storage/file/README.encryption | 202 +++++++++++++++++++++++++++++
16 files changed, 542 insertions(+), 7 deletions(-)
create mode 100644 doc/src/sgml/encryption.sgml
create mode 100644 doc/src/sgml/ref/pg_keytool.sgml
create mode 100644 src/backend/storage/file/README.encryption
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index c91e3e1550..5d64736a03 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -9004,6 +9004,20 @@ dynamic_library_path = 'C:\tools\postgresql;H:\my_project\lib;$libdir'
</listitem>
</varlistentry>
+ <varlistentry id="guc-data-encryption" xreflabel="data_encryption">
+ <term><varname>data_encryption</varname> (<type>boolean</type>)
+ <indexterm>
+ <primary><varname>data_encryption</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Reports whether data encryption is enabled for this cluster.
+ See <xref linkend="encryption"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-data-directory-mode" xreflabel="data_directory_mode">
<term><varname>data_directory_mode</varname> (<type>integer</type>)
<indexterm>
diff --git a/doc/src/sgml/encryption.sgml b/doc/src/sgml/encryption.sgml
new file mode 100644
index 0000000000..5913883bc0
--- /dev/null
+++ b/doc/src/sgml/encryption.sgml
@@ -0,0 +1,95 @@
+<!-- doc/src/sgml/encryption.sgml -->
+
+<chapter id="encryption">
+ <title>Transparent Cluster Encryption</title>
+
+ <para>
+ Cluster encryption can be used if <acronym>DBA</acronym> cannot or does not
+ want to rely on the filesystem in terms of data confidentiality. If this
+ feature is enabled, <productname>PostgreSQL</productname> encrypts data
+ (both relations and write-ahead-log) when writing it to disk, and decrypts
+ when reading it. The encryption is transparent, so applications see no
+ difference between encrypted and unencrypted cluster.
+ </para>
+
+ <para>
+ To create encrypted cluster, use the <option>-K</option> option to pass the
+ <xref linkend="app-initdb"/> utility path to the command that will retrieve
+ the encryption key. For example:
+<screen>
+<prompt>$</prompt> <userinput>initdb -D /usr/local/pgsql/data -K /usr/local/pgsql/fetch_key_cmd</userinput>
+</screen>
+ Here <filename>/usr/local/pgsql/fetch_key_cmd</filename> is an executable
+ file that writes the encryption key to its standard output and returns
+ zero. The simplest command looks like:
+
+<programlisting>
+echo <your key>
+</programlisting>
+ </para>
+
+ <para>
+ The encryption key is expected in hexadecimal format, two characters
+ (hexadecimal digits) per
+ byte. Since <productname>PostgreSQL</productname> currently uses key
+ of length 32 bytes (256 bits), the expected length of the key string
+ is 64 characters. For example:
+ <computeroutput>
+ 882fb7c12e80280fd664c69d2d636913e86c381ba487c82f77653c0fac8ffc69
+ </computeroutput>
+ </para>
+
+ <para>
+ If the DBA wants to enter password, he can use
+ <xref linkend="app-pg-keytool"/> utility to derive ecryption key from
+ it. For example, if the <option>-K</option> option looks like in the
+ following example, the DBA will be asked for password interactively:
+
+<screen>
+<prompt>$</prompt> <userinput>initdb -K 'read -sp "Cluster encryption password: " PGENCRPWD; echo $PGENCRPWD
+| pg_keytool -D %D -w' -D data
+</userinput>
+</screen>
+
+ </para>
+
+ <para>
+ As long as the application is aware of the data directory, you can
+ put <literal>%D</literal> into the command and the application will replace
+ it with the actual directory path.
+ </para>
+
+ <para>
+ Likewise, the same command can be passed to <xref linkend="app-pg-ctl"/> in
+ order to start the cluster:
+
+<screen>
+<prompt>$</prompt> <userinput>pg_ctl -K 'read -sp "Cluster encryption password: " PGENCRPWD; echo $PGENCRPWD
+| pg_keytool -D %D -w' -D data start
+</userinput>
+</screen>
+ </para>
+
+ <para>
+ If encryption is enabled, <xref linkend="guc-full-page-writes"/> must be
+ turned on, otherwise the server refuses to start. This is because the
+ encryption introduces dependencies between data within a page, and thus
+ server crash during disk write can result in more serious damage of the page
+ than it would do without encryption. The whole page needs to be retrieved
+ from WAL in such a case to ensure reliable recovery.
+ </para>
+
+ <para>
+ Once the <productname>PostgreSQL</productname> server is running, client
+ applications should recognize no difference from an unencrypted cluster,
+ except that <xref linkend="guc-data-encryption"/> variable is set.
+ </para>
+
+ <para>
+ Since WAL is encrypted, any replication solution based on log shipping
+ (<xref linkend="warm-standby"/>) assume that all standby servers are
+ encrypted using the same key as their primary server. On the other hand,
+ <xref linkend="logical-replication"/> allows replication between encrypted
+ and unencrypted clusters, or between clusters encrypted with different keys.
+ </para>
+</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 3da2365ea9..d21dabf03b 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -48,6 +48,7 @@
<!ENTITY wal SYSTEM "wal.sgml">
<!ENTITY logical-replication SYSTEM "logical-replication.sgml">
<!ENTITY jit SYSTEM "jit.sgml">
+<!ENTITY encryption SYSTEM "encryption.sgml">
<!-- programmer's guide -->
<!ENTITY bgworker SYSTEM "bgworker.sgml">
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index 449386243b..a2daf33e6f 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -245,8 +245,9 @@ su - postgres
<listitem>
<para>
You need <productname>OpenSSL</productname>, if you want to support
- encrypted client connections. <productname>OpenSSL</productname> is
- also required for random number generation on platforms that do not
+ on-disk data encryption or encrypted client
+ connections. <productname>OpenSSL</productname> is also required for
+ random number generation on platforms that do not
have <filename>/dev/urandom</filename> (except Windows). The minimum
version required is 0.9.8.
</para>
@@ -835,10 +836,11 @@ su - postgres
</term>
<listitem>
<para>
- Build with support for <acronym>SSL</acronym> (encrypted)
- connections. This requires the <productname>OpenSSL</productname>
- package to be installed. <filename>configure</filename> will check
- for the required header files and libraries to make sure that
+ Build with support for on-disk data encryption
+ or <acronym>SSL</acronym> (encrypted) connections. This requires
+ the <productname>OpenSSL</productname> package to be
+ installed. <filename>configure</filename> will check for the
+ required header files and libraries to make sure that
your <productname>OpenSSL</productname> installation is sufficient
before proceeding.
</para>
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 3e115f1c76..b0d35600b2 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -163,6 +163,7 @@
&wal;
&logical-replication;
&jit;
+ &encryption;
®ress;
</part>
diff --git a/doc/src/sgml/ref/allfiles.sgml b/doc/src/sgml/ref/allfiles.sgml
index 8d91f3529e..a60b02e23f 100644
--- a/doc/src/sgml/ref/allfiles.sgml
+++ b/doc/src/sgml/ref/allfiles.sgml
@@ -206,6 +206,7 @@ Complete list of usable sgml source files in this directory.
<!ENTITY pgDump SYSTEM "pg_dump.sgml">
<!ENTITY pgDumpall SYSTEM "pg_dumpall.sgml">
<!ENTITY pgIsready SYSTEM "pg_isready.sgml">
+<!ENTITY pgKeytool SYSTEM "pg_keytool.sgml">
<!ENTITY pgReceivewal SYSTEM "pg_receivewal.sgml">
<!ENTITY pgRecvlogical SYSTEM "pg_recvlogical.sgml">
<!ENTITY pgResetwal SYSTEM "pg_resetwal.sgml">
diff --git a/doc/src/sgml/ref/initdb.sgml b/doc/src/sgml/ref/initdb.sgml
index 7fc3152c6d..ce19151521 100644
--- a/doc/src/sgml/ref/initdb.sgml
+++ b/doc/src/sgml/ref/initdb.sgml
@@ -209,6 +209,17 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry id="app-initdb-data-encr-cmd" xreflabel="data encryption">
+ <term><option>-K</option></term>
+ <term><option>--encryption-key-command=<replaceable class="parameter">command</replaceable></option></term>
+ <listitem>
+ <para>
+ Encrypt the cluster data using a key retrieved from the command
+ specified here. See <xref linkend="encryption"/> for details.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="app-initdb-data-checksums" xreflabel="data checksums">
<term><option>-k</option></term>
<term><option>--data-checksums</option></term>
diff --git a/doc/src/sgml/ref/pg_checksums.sgml b/doc/src/sgml/ref/pg_checksums.sgml
index 33706d1d97..7a2cae5e57 100644
--- a/doc/src/sgml/ref/pg_checksums.sgml
+++ b/doc/src/sgml/ref/pg_checksums.sgml
@@ -106,7 +106,18 @@ PostgreSQL documentation
<listitem>
<para>
Only validate checksums in the relation with filenode
- <replaceable>filenode</replaceable>.
+ <replaceable>filenode</replaceable>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-K</option></term>
+ <term><option>--encryption-key-command=<replaceable class="parameter">command</replaceable></option></term>
+ <listitem>
+ <para>
+ If the cluster is encrypted, run this command to retrieve the
+ encryption key. See <xref linkend="encryption"/> for details.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/pg_ctl-ref.sgml b/doc/src/sgml/ref/pg_ctl-ref.sgml
index e31275a04e..48966eb58e 100644
--- a/doc/src/sgml/ref/pg_ctl-ref.sgml
+++ b/doc/src/sgml/ref/pg_ctl-ref.sgml
@@ -293,6 +293,17 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-K</option></term>
+ <term><option>--encryption-key-command=<replaceable class="parameter">command</replaceable></option></term>
+ <listitem>
+ <para>
+ If the cluster is encrypted, run this command to retrieve the
+ encryption key. See <xref linkend="encryption"/> for details.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-l <replaceable class="parameter">filename</replaceable></option></term>
<term><option>--log=<replaceable class="parameter">filename</replaceable></option></term>
<listitem>
diff --git a/doc/src/sgml/ref/pg_keytool.sgml b/doc/src/sgml/ref/pg_keytool.sgml
new file mode 100644
index 0000000000..3c15910885
--- /dev/null
+++ b/doc/src/sgml/ref/pg_keytool.sgml
@@ -0,0 +1,121 @@
+<!--
+doc/src/sgml/ref/pg_keytool.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="app-pg-keytool">
+ <indexterm zone="app-pg-keytool">
+ <primary>pg_keysetup</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle><application>pg_keytool</application></refentrytitle>
+ <manvolnum>1</manvolnum>
+ <refmiscinfo>Application</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>pg_keytool</refname>
+ <refpurpose>produce cluster encryption key, possibly deriving it from a
+ passphrase</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+ <cmdsynopsis>
+ <command>pg_keysetup</command>
+ <arg rep="repeat"><replaceable class="parameter">option</replaceable></arg>
+ </cmdsynopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>Description</title>
+ <para>
+ <application>pg_keytool</application> reads encryption key or password from
+ the standard input. If it received password, it runs <firstterm>key
+ derivation function</firstterm> on it in order to derive the key. The
+ encryption key is written either to the standard output or sent to via
+ frontend/backend protocol to postmaster process which is being started up.
+ </para>
+
+ <para>
+ One common use case is that <application>pg_keytool</application> is used
+ within encryption key command, see the examples in
+ <xref linkend="encryption"/>. In this case it sends the key to the
+ standard output.
+ </para>
+
+ <para>
+ The other use case is that <productname>PostgreSQL</productname> is
+ started other way than using <xref linkend="app-pg-ctl"/>, typically
+ via <firstterm>systemd</firstterm> or a custom script. In this
+ case <application>pg_keytool</application> can be used to send the key to
+ specified TCP/IP port or to an Unix socket, on which the server
+ listens. This is the same port or socket to which applications will
+ eventually connect, but when the encrypted cluster starts up, it only
+ accepts the key.
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>Options</title>
+
+ <para>
+ <variablelist>
+ <varlistentry>
+ <term><option>-D <replaceable>directory</replaceable></option></term>
+ <listitem>
+ <para>
+ Specifies the directory where the database cluster is
+ stored. Different directory generally means that a different key will
+ be derived from the same password.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-h <replaceable class="parameter">hostname</replaceable></option></term>
+ <term><option>--host=<replaceable class="parameter">hostname</replaceable></option></term>
+ <listitem>
+ <para>
+ Specifies the host name of the machine on which the server is
+ listening. If the value begins with a slash, it is used as the
+ directory for the Unix-domain socket.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-p <replaceable class="parameter">port</replaceable></option></term>
+ <listitem>
+ <para>
+ Specifies the TCP port or the local Unix-domain socket file
+ extension on which the server is listening for
+ connections. Defaults to the value of the <envar>PGPORT</envar>
+ environment variable or, if not set, to the port specified at
+ compile time, usually 5432.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-s</option></term>
+ <listitem>
+ <para>
+ Send the key to <productname>PostgreSQL</productname> server rather
+ than to standard output.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-w</option></term>
+ <listitem>
+ <para>
+ Expect password on standard input rather than encryption key.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ </refsect1>
+</refentry>
diff --git a/doc/src/sgml/ref/pg_resetwal.sgml b/doc/src/sgml/ref/pg_resetwal.sgml
index 8a9e22d050..613d14e1cc 100644
--- a/doc/src/sgml/ref/pg_resetwal.sgml
+++ b/doc/src/sgml/ref/pg_resetwal.sgml
@@ -100,6 +100,17 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-K</option></term>
+ <term><option>--encryption-key-command=<replaceable class="parameter">command</replaceable></option></term>
+ <listitem>
+ <para>
+ If the cluster is encrypted, run this command to retrieve the
+ encryption key. See <xref linkend="encryption"/> for details.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-n</option></term>
<term><option>--dry-run</option></term>
<listitem>
diff --git a/doc/src/sgml/ref/pg_rewind.sgml b/doc/src/sgml/ref/pg_rewind.sgml
index 4d91eeb0ff..0e4f79b652 100644
--- a/doc/src/sgml/ref/pg_rewind.sgml
+++ b/doc/src/sgml/ref/pg_rewind.sgml
@@ -140,6 +140,17 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-K</option></term>
+ <term><option>--encryption-key-command=<replaceable class="parameter">command</replaceable></option></term>
+ <listitem>
+ <para>
+ If the cluster is encrypted, run this command to retrieve the
+ encryption key. See <xref linkend="encryption"/> for details.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>--source-pgdata=<replaceable class="parameter">directory</replaceable></option></term>
<listitem>
<para>
diff --git a/doc/src/sgml/ref/pg_waldump.sgml b/doc/src/sgml/ref/pg_waldump.sgml
index 329c10e430..ca1e4e1468 100644
--- a/doc/src/sgml/ref/pg_waldump.sgml
+++ b/doc/src/sgml/ref/pg_waldump.sgml
@@ -101,6 +101,17 @@ PostgreSQL documentation
</varlistentry>
<varlistentry>
+ <term><option>-K</option></term>
+ <term><option>--encryption-key-command=<replaceable class="parameter">command</replaceable></option></term>
+ <listitem>
+ <para>
+ If the cluster is encrypted, run this command to retrieve the
+ encryption key. See <xref linkend="encryption"/> for details.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-n <replaceable>limit</replaceable></option></term>
<term><option>--limit=<replaceable>limit</replaceable></option></term>
<listitem>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 82886760f1..0c04990d51 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -120,6 +120,37 @@
</varlistentry>
<varlistentry>
+ <term><option>-K</option></term>
+ <term><option>--encryption-key-command=<replaceable class="parameter">command</replaceable></option></term>
+ <listitem>
+ <para>
+ If the clusters are encrypted, run this command to retrieve the
+ encryption key. This is the same command that was passed to
+ <xref linkend="app-initdb"/> for cluster creation. For the upgrade to
+ be successful, both old and new clusters must use the same key.
+ Therefore, unless the command contains the <literal>%D</literal>
+ string, it does not matter which cluster the command comes from.
+ </para>
+
+ <note>
+ <para>
+ If you are using <xref linkend="app-pg-keytool"/> to derive the
+ encryption key (see <xref linkend="app-initdb-data-encr-cmd"/>),
+ make sure the <option>-D</option> points to
+ the <emphasis>old</emphasis> cluster. If it pointed to the new
+ one then the new cluster would derive a different key from the same
+ password and thus <xref linkend="pgupgrade"/> would fail to start
+ the new cluster.
+ </para>
+ </note>
+
+ <para>
+ See <xref linkend="encryption"/> for details on the command.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-k</option></term>
<term><option>--link</option></term>
<listitem><para>use hard links instead of copying files to the new
diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml
index cef09dd38b..5eed489f27 100644
--- a/doc/src/sgml/reference.sgml
+++ b/doc/src/sgml/reference.sgml
@@ -279,6 +279,7 @@
&pgChecksums;
&pgControldata;
&pgCtl;
+ &pgKeytool;
&pgResetwal;
&pgRewind;
&pgtestfsync;
diff --git a/src/backend/storage/file/README.encryption b/src/backend/storage/file/README.encryption
new file mode 100644
index 0000000000..5ffd587cf5
--- /dev/null
+++ b/src/backend/storage/file/README.encryption
@@ -0,0 +1,202 @@
+src/backend/storage/file/README.encryption
+
+Transparent Cluster Encryption
+==============================
+
+When creating a new cluster (instance), user can choose to have his data
+encrypted on disk. If this feature is active, data is encrypted before it's
+written to disk and decrypted after it has been read, however the data is
+unencrypted in memory (data-at-rest encryption). The following characteristics
+should be considered by anyone who is interested in the feature:
+
+ 1. The encryption is transparent from application's point of view.
+
+ 2. A single key is used to encrypt the whole cluster.
+
+The full instance encryption feature helps to ensure data confidentiality,
+especially when user cannot rely on confidentiality on filesystem level. On
+the other hand, it does not ensure data integrity, i.e. it does not help to
+detect whether an adversary wrote his cipher data to the disk. The block
+cipher methods generally do not protect data integrity, and it'd probably be
+hard anyway because we encrypt postgres data pages (typically of size 8 kB)
+separate from each other. If the attacker only changes part of the page, this
+can be detected if data checksums feature is enabled. And specifically for
+XLOG, each XLOG record has its own checksum.
+
+Since the data is stored on a disk, we naturally base our approach on "Disk
+encryption theory" [1]. For each kind of file we use the AES cipher in the
+appropriate mode of operation. The AES cipher itself encrypts / decrypts
+individual blocks ("encryption blocks") of 16 bytes (128 bits), while the mode
+of operation defines the rules how to apply the cipher to data which spans
+multiple encryption blocks.
+
+Relations
+---------
+
+AES cipher in CBC mode [2] is used to encrypt relation files, one relation
+block (page) at a time. The important characteristic of this mode is that, if
+a single encryption block changes, the next run of the encryption produces an
+encrypted page where not only that block is different, but also all the
+following blocks. Furthermore, as the postgres page starts with LSN, even if
+only the last encryption block of the page changes, the whole cipher text of
+the page will be different after the next encryption.
+
+This propagation of changes across encryption blocks makes it harder for
+adversary to deduce what happens in the database. Without the propagation he
+might be able to see data changes with higher granularity. For example, if
+only the beginning of index page changes, it gives some information on the
+keys inserted.
+
+On the other hand, the same change propagation can cause problems if only a
+part of a new version of encrypted page is written to disk (torn page
+write). If such a page is decrypted during crash recovery, part of the plain
+text can become garbage. This can affect even those parts of the page that
+haven't been modified recently, and therefore XLOG records to fix those parts
+are not available. Therefore, if the cluster encryption is enabled,
+full_page_writes configuration variable must be set too, otherwise the server
+refuses to start. With this setting, the damage described here can be fixed by
+retrieving the full-page-image (FPI) from XLOG and applying it.
+
+However, even if full_page_writes is set, postgres can still avoid writing FPI
+XLOG record if only hint bit(s) changed on a page and if wal_log_hints
+configuration variable is off. The problem with encryption is that even a
+single bit change is propagated to all the following encryption blocks during
+the next encryption of the page. Again, if only part of the new cipher text is
+written, the remaining portion of the page becomes garbage after decryption,
+and we might not be able to fix it during recovery unless we have the FPI in
+the XLOG. So if encryption is enabled, postgres behaves as if wal_log_hints
+was always set.
+
+XLOG
+----
+
+The specific problem of XLOG is that record must not be changed once it has
+been flushed to disk. However if we used a block cipher, and if a new XLOG
+record started in the same encryption block in which the previous record ends,
+that encryption block would become completely different after the next run of
+encryption of the containing page. Torn write of such a block (e.g. if memory
+page boundary crosses the encryption block) is likely to make decryption
+produce garbage, which will also appear in the already-flushed record.
+
+Therefore we encrypt XLOG using a stream cipher, or rather block cipher in
+stream mode of operation. Stream cipher uses XOR operation to combine a "key
+stream" with the input stream, and it does not matter if the length of the
+input stream is aligned to any value. In particular, the CTR mode [3] was
+chosen because it allows for both read and write operations to be
+parallelized.
+
+The XLOG is encrypted / decrypted one XLOG page (typically 8 kB) at a time, so
+rather than a single stream, the encrypted XLOG is actually a sequence of
+per-page streams.
+
+Since stream ciphers are susceptible to "reused key attack" [4], we must
+ensure that the unused part of the last XLOG page (filled with zeroes) is
+never encrypted.
+
+Temporary files
+---------------
+
+BufFileWrite() and BufFileRead() functions (see buffile.c) hide the encryption
+/ decryption from caller. The encryption / decryption processes one buffer at
+a time so that the buffers can be retrieved independent from each other.
+
+If the encryption is enabled, the following requirements need to be taken into
+account:
+
+1. The file buffer cannot be positioned at arbitrary offset of the file. If
+the encryption routine starts at some position of the file, decryption must
+not start elsewhere because there's no way to determine which initialization
+vector was used internally for the corresponding encryption blocks (of 16
+bytes) during encryption. It makes sense to position the buffer at file offset
+that is whole multiple of buffer size.
+
+2. In general, the useful (written) data does not fill whole multiple of
+encryption blocks, but we must write the whole blocks for decryption to
+succeed. This implies that we need to fill the unused part of the last block
+with zeroes and also remember the amount of useful bytes in the segment file.
+(In fact we align the segment file size to file buffers instead of encryption
+blocks, which probably makes the implementation a bit simpler.)
+
+Stream cipher might seem like a solution of the padding problem, but we cannot
+use it here because parts of the temporary file can be rewritten. That would
+expose the temporary file to "reused key attack" [4].
+
+Auxiliary files
+---------------
+
+To store other kinds of data encrypted than the ones above, developers are
+advised to use BufFileWriteTransient() and BufFileReadTransient() functions
+(also located in buffile.c). These are especially useful if some data
+structure should be written to an encrypted file and user does not want to
+care whether the next write position is at encryption block boundary.
+
+Wherever the write starts, BufFileWriteTransient() first reads the whole
+encryption block that contains the start positions, then decrypts it, adds the
+new data to it, encrypts the whole block and writes it back to the
+file. Likewise, BufFileReadTransient() ensures that only the whole encryption
+blocks are read from disk and decrypted, whether the next read position is at
+block boundary or not.
+
+Serialization of data changes during logical decoding (reorderbuffer.c) is the
+typical use case for this API.
+
+Initialization vector (IV), encryption tweak
+--------------------------------------------
+
+Besides the input data and the key, both block and stream cipher used for the
+postgres cluster encryption require an initialization vector. It should make
+analysis of the encrypted data more difficult. Also according to [1], the same
+data should be encrypted differently if located elsewhere on the disk. Term
+"encryption tweak" is more common in the context of disk encryption.
+
+When encrypting relations, each encryption unit (i.e. page) has an unique
+tweak, which consists of RelFileNode structure, buffer number and fork
+number. As a consequence, we need to decrypt page using the existing tweak and
+encrypt it with a new tweak if copying it from one relation to
+another. Typically this happens when a new database is being created from
+template database.
+
+To avoid "reencryption" during pg_upgrade, all the OIDs contained in the
+RelFileNode structure (tablespace OID, database OID and relation file OID)
+must be preserved. If pg_upgrade had to reencrypt files, it could not be used
+with the --link option.
+
+XLOG encryption tweak consists of timeline, segment number and offset at which
+the XLOG page starts in the segment. The "reencryption" takes place when XLOG
+page is copied from one timeline to another, typicially at the end of
+recovery.
+
+As for temporary files, PID of the owning backend, file / fileset number and
+block number (where block is of the same size as a relation page) within the
+file provide sufficient uniqueness, so we use these to generate the tweak.
+
+Auxiliary files can be closed and reopened by another backend, so there's no
+PID strictly associated with them. Therefore we generate the tweak by hashing
+the file path, and appending block number to the hash.
+
+Replication
+-----------
+
+During streaming replication, the walsender process sends the XLOG encrypted
+and walreceiver just writes it. Decryption is performed before the slave
+cluster tries to apply the changes. That implies that the same encryption key
+must be used both master and slave. If it should be possible someday to stream
+the XLOG unencrypted. One implication is that pg_basebackup should then also
+receive the relation data unencrypted.
+
+As for logical replication, the only change introduced by this feature is that
+the XLOG has to be decrypted before the contained data changes can be
+decoded. The data changes are transferred to the subscribing database /
+cluster unencrypted, so both master and slave can use different encryption
+keys.
+
+References
+----------
+
+[1] https://en.wikipedia.org/wiki/Disk_encryption_theory
+
+[2] https://en.wikipedia.org/wiki/Disk_encryption_theory#Cipher-block_chaining_(CBC)
+
+[3] https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Counter_(CTR)
+
+[4] https://en.wikipedia.org/wiki/Stream_cipher_attacks#Reused_key_attack
--
2.13.7
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-07-06 02:33 Bruce Momjian <bruce@momjian.us>
parent: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
1 sibling, 0 replies; 37+ messages in thread
From: Bruce Momjian @ 2019-07-06 02:33 UTC (permalink / raw)
To: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>; +Cc: Antonin Houska <ah@cybertec.at>; Robert Haas <robertmhaas@gmail.com>; Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
On Tue, Jun 25, 2019 at 02:28:00PM +0200, Peter Eisentraut wrote:
> On 2019-06-17 11:23, Antonin Houska wrote:
> > I'm thinking how to teach postmaster to accept FEBE protocol connections
> > temporarily, just to receive the key. The user applications like pg_ctl,
> > initdb or pg_upgrade would retrieve the key / password from the DBA, then
> > start postmaster and send it the key.
> >
> > Perhaps the message format should be a bit generic so that extensions like
> > this can use it to receive their keys too.
> >
> > (The idea of an unix socket or named pipe I proposed upthread is not good
> > because it's harder to implement in a portable way.)
>
> How are the requirements here different from ssl_passphrase_command?
> Why do we need a new mechanism?
Agreed. My pgcryptokey prompting shell script was mostly a
proof-of-concept.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
^ permalink raw reply [nested|flat] 37+ messages in thread
* Re: "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
@ 2019-07-08 01:07 Thomas Munro <thomas.munro@gmail.com>
parent: Antonin Houska <ah@cybertec.at>
0 siblings, 0 replies; 37+ messages in thread
From: Thomas Munro @ 2019-07-08 01:07 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Robert Haas <robertmhaas@gmail.com>; Toshi Harada <harada.toshi@po.ntt-tx.co.jp>; pgsql-hackers
On Sat, Jul 6, 2019 at 2:42 AM Antonin Houska <ah@cybertec.at> wrote:
> I've reworked the way key is passed to postmaster (v04-0003-...) so it's
> easier to call the encryption key command interactively, adjusted pg_upgrade
> so that it preserves database, tablespace and relfilenode OIDs (v04-0014-...),
> reorganized the code a bit and split the code into more diffs.
Hi Antonin,
Some robotic feedback:
1. On Linux, an assertion failed:
ExceptionalCondition (conditionName=conditionName@entry=0x973891
"!(decrypt_p == p)", errorType=errorType@entry=0x928d7d
"FailedAssertion", fileName=fileName@entry=0x973827 "xlogutils.c",
lineNumber=lineNumber@entry=815) at assert.c:54
See full stack here:
https://travis-ci.org/postgresql-cfbot/postgresql/builds/555550833
2. On Windows the build failed:
https://ci.appveyor.com/project/postgresql-cfbot/postgresql/build/1.0.46469
--
Thomas Munro
https://enterprisedb.com
^ permalink raw reply [nested|flat] 37+ messages in thread
end of thread, other threads:[~2019-07-08 01:07 UTC | newest]
Thread overview: 37+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2018-08-13 05:56 "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3 Toshi Harada <harada.toshi@po.ntt-tx.co.jp>
2018-08-13 06:06 ` Antonin Houska <ah@cybertec.at>
2018-08-15 02:09 ` Toshi Harada <harada.toshi@po.ntt-tx.co.jp>
2018-09-14 14:05 ` Antonin Houska <ah@cybertec.at>
2019-03-01 21:17 ` Robert Haas <robertmhaas@gmail.com>
2019-03-14 13:28 ` Antonin Houska <ah@cybertec.at>
2019-03-15 15:34 ` Robert Haas <robertmhaas@gmail.com>
2019-03-15 21:10 ` Tom Lane <tgl@sss.pgh.pa.us>
2019-03-15 21:48 ` Robert Haas <robertmhaas@gmail.com>
2019-03-21 11:46 ` Antonin Houska <ah@cybertec.at>
2019-03-21 20:07 ` Robert Haas <robertmhaas@gmail.com>
2019-04-05 15:22 ` Antonin Houska <ah@cybertec.at>
2019-04-05 19:57 ` Robert Haas <robertmhaas@gmail.com>
2019-04-06 13:29 ` Antonin Houska <ah@cybertec.at>
2019-04-12 20:17 ` Bruce Momjian <bruce@momjian.us>
2019-04-04 13:57 ` Antonin Houska <ah@cybertec.at>
2019-04-04 14:55 ` Robert Haas <robertmhaas@gmail.com>
2019-04-04 15:52 ` Antonin Houska <ah@cybertec.at>
2019-04-04 18:09 ` PostgreSQL - Hans-Jürgen Schönig <postgres@cybertec.at>
2019-04-04 19:47 ` Robert Haas <robertmhaas@gmail.com>
2019-04-24 14:48 ` Antonin Houska <ah@cybertec.at>
2019-05-22 15:16 ` Robert Haas <robertmhaas@gmail.com>
2019-05-28 15:26 ` Antonin Houska <ah@cybertec.at>
2019-05-29 21:51 ` Robert Haas <robertmhaas@gmail.com>
2019-05-31 06:59 ` Antonin Houska <ah@cybertec.at>
2019-06-03 16:04 ` Robert Haas <robertmhaas@gmail.com>
2019-06-05 12:32 ` Antonin Houska <ah@cybertec.at>
2019-06-06 10:58 ` Pavlo Golub <pavlo.golub@cybertec.at>
2019-06-14 01:14 ` Bruce Momjian <bruce@momjian.us>
2019-06-14 13:02 ` Bruce Momjian <bruce@momjian.us>
2019-06-17 09:23 ` Antonin Houska <ah@cybertec.at>
2019-06-25 12:28 ` Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2019-06-25 13:39 ` Robert Haas <robertmhaas@gmail.com>
2019-06-26 13:05 ` Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2019-07-06 02:33 ` Bruce Momjian <bruce@momjian.us>
2019-07-05 14:41 ` Antonin Houska <ah@cybertec.at>
2019-07-08 01:07 ` Thomas Munro <thomas.munro@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox