pg.ddx.io pgbackrest/pgbackrest GitHub issues and pull requests
help / color / mirror / Atom feedFrom: qinruyao (@qinruyao) <noreply+qinruyao@github.com>
To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>
Subject: [pgbackrest/pgbackrest] PR #2896: Add S3 KMS encryption context option.
Date: Tue, 03 Nov 2026 19:02:14 +0000
Message-ID: <gh-pgbackrest-pgbackrest-2896@github.com> (raw)
KMS grants can require an encryption context in addition to a key ID. Add `repo-s3-kms-encryption-context` so S3 repository writes can supply that context.
The optional hash uses repeated `key=value` entries, like `repo-storage-tag`, and requires `repo-s3-kms-key-id`. The S3 driver serializes the entries as sorted JSON and Base64-encodes the result once when it creates the storage object. It sends `x-amz-server-side-encryption-context` with the existing KMS headers on `PutObject` and `CreateMultipartUpload`.
When the option is unset, requests retain their existing behavior. Reads and upload parts do not send the context header. The documentation explains that context values are not secret.
```ini
repo1-s3-kms-key-id=bceb4f13-6939-4be3-910d-df54dee817b7
repo1-s3-kms-encryption-context=cluster_id=pg-prod-01
repo1-s3-kms-encryption-context=tenant_id=tenant-01
```
Local validation passed on Ubuntu 24.04, ARM64:
- Formatting and source lint.
- Unity build with warnings treated as errors.
- `storage/s3`, `config/parse`, `command/help`, and `doc/command/build/reference` tests with full coverage for the reported modules.
- Integration-test harness compilation.
- Command and configuration HTML generation with execution enabled, plus inspection of the option description, examples, and links.
- Inspection of the new option's help from the built binary.
The tests cover missing-key and malformed-entry errors, plus sorted JSON with escaped characters and UTF-8. They check signed upload headers and omission on reads and upload parts. They also retain KMS-without-context and non-KMS write coverage.
Integration workloads, Valgrind, and live AWS KMS validation have not run locally. The full documentation set was rendered with `--no-exe`; the executable user-guide examples and Debian/RHEL documentation workflow have not run locally.
Fixes https://github.com/pgbackrest/pgbackrest/issues/2891.
inline patch 1: build/config.yaml, doc/xml/reference.xml, src/config/config.auto.h and 7 more (+200/-10 lines) download inline-1.patch
diff --git a/build/config.yaml b/build/config.yaml
index e20b77be03..b086b65216 100644
--- a/build/config.yaml
+++ b/build/config.yaml
@@ -2433,6 +2433,13 @@ option:
deprecate:
repo-s3-key-secret: {}
+ repo-s3-kms-encryption-context:
+ inherit: repo-s3-bucket
+ type: hash
+ required: false
+ depend:
+ option: repo-s3-kms-key-id
+
repo-s3-kms-key-id:
inherit: repo-s3-bucket
required: false
diff --git a/doc/xml/reference.xml b/doc/xml/reference.xml
index 716f9fdef6..051de1b5a9 100644
--- a/doc/xml/reference.xml
+++ b/doc/xml/reference.xml
@@ -1060,6 +1060,23 @@
<example>AQoDYXdzEPT//////////wEXAMPLEtc764bNrC9SAPBSM22 ...</example>
</config-key>
+ <config-key id="repo-s3-kms-encryption-context" name="S3 Repository KMS Encryption Context">
+ <summary>S3 repository KMS encryption context.</summary>
+
+ <text>
+ <p>This option adds an encryption context for S3 server-side encryption with AWS KMS. It requires <br-option>repo-s3-kms-key-id</br-option>.</p>
+
+ <p>Each entry is a key/value pair, for example <setting>repo1-s3-kms-encryption-context=cluster_id=pg-prod-01</setting>. Multiple entries require a separate option for each entry.</p>
+
+ <p><backrest/> sends the context when it creates objects, including multipart uploads. This option does not change existing objects. S3 stores the context with each object and uses it to decrypt the object.</p>
+
+ <p>Encryption context is not secret and must not contain sensitive information.</p>
+ </text>
+
+ <example>cluster_id=pg-prod-01</example>
+ <example>tenant_id=tenant-01</example>
+ </config-key>
+
<config-key id="repo-s3-kms-key-id" name="S3 Repository KMS Key ID">
<summary>S3 repository KMS key.</summary>
diff --git a/src/config/config.auto.h b/src/config/config.auto.h
index 07ea853524..c64d7a51c7 100644
--- a/src/config/config.auto.h
+++ b/src/config/config.auto.h
@@ -146,7 +146,7 @@ Option constants
#define CFGOPT_VERBOSE "verbose"
#define CFGOPT_VERSION "version"
-#define CFG_OPTION_TOTAL 202
+#define CFG_OPTION_TOTAL 203
/***********************************************************************************************************************************
Option value constants
@@ -624,6 +624,7 @@ typedef enum
cfgOptRepoS3Key,
cfgOptRepoS3KeySecret,
cfgOptRepoS3KeyType,
+ cfgOptRepoS3KmsEncryptionContext,
cfgOptRepoS3KmsKeyId,
cfgOptRepoS3ProcessCmd,
cfgOptRepoS3Region,
diff --git a/src/config/parse.auto.c.inc b/src/config/parse.auto.c.inc
index b736d687ad..04fc88f05b 100644
--- a/src/config/parse.auto.c.inc
+++ b/src/config/parse.auto.c.inc
@@ -8035,6 +8035,83 @@ static const ParseRuleOption parseRuleOption[CFG_OPTION_TOTAL] =
), // opt/repo-s3-key-type
), // opt/repo-s3-key-type
// -----------------------------------------------------------------------------------------------------------------------------
+ PARSE_RULE_OPTION // opt/repo-s3-kms-encryption-context
+ ( // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_NAME("repo-s3-kms-encryption-context"), // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_TYPE(Hash), // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_RESET(true), // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_REQUIRED(false), // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_SECTION(Global), // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_MULTI(true), // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_GROUP_ID(Repo), // opt/repo-s3-kms-encryption-context
+ // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND_ROLE_MAIN_VALID_LIST // opt/repo-s3-kms-encryption-context
+ ( // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(Annotate) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(ArchiveGet) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(ArchivePush) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(Backup) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(Check) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(Expire) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(Info) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(Manifest) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(RepoGet) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(RepoLs) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(RepoRm) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(Restore) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(StanzaCreate) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(StanzaDelete) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(StanzaUpgrade) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(Verify) // opt/repo-s3-kms-encryption-context
+ ), // opt/repo-s3-kms-encryption-context
+ // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND_ROLE_ASYNC_VALID_LIST // opt/repo-s3-kms-encryption-context
+ ( // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(ArchiveGet) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(ArchivePush) // opt/repo-s3-kms-encryption-context
+ ), // opt/repo-s3-kms-encryption-context
+ // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND_ROLE_LOCAL_VALID_LIST // opt/repo-s3-kms-encryption-context
+ ( // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(ArchiveGet) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(ArchivePush) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(Backup) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(Restore) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(Verify) // opt/repo-s3-kms-encryption-context
+ ), // opt/repo-s3-kms-encryption-context
+ // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND_ROLE_REMOTE_VALID_LIST // opt/repo-s3-kms-encryption-context
+ ( // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(Annotate) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(ArchiveGet) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(ArchivePush) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(Backup) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(Check) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(Expire) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(Info) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(Manifest) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(RepoGet) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(RepoLs) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(RepoRm) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(Restore) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(StanzaCreate) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(StanzaDelete) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(StanzaUpgrade) // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTION_COMMAND(Verify) // opt/repo-s3-kms-encryption-context
+ ), // opt/repo-s3-kms-encryption-context
+ // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTIONAL // opt/repo-s3-kms-encryption-context
+ ( // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTIONAL_GROUP // opt/repo-s3-kms-encryption-context
+ ( // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_OPTIONAL_DEPEND // opt/repo-s3-kms-encryption-context
+ ( // opt/repo-s3-kms-encryption-context
+ PARSE_RULE_VAL_OPT(RepoS3KmsKeyId), // opt/repo-s3-kms-encryption-context
+ ), // opt/repo-s3-kms-encryption-context
+ ), // opt/repo-s3-kms-encryption-context
+ ), // opt/repo-s3-kms-encryption-context
+ ), // opt/repo-s3-kms-encryption-context
+ // -----------------------------------------------------------------------------------------------------------------------------
PARSE_RULE_OPTION // opt/repo-s3-kms-key-id
( // opt/repo-s3-kms-key-id
PARSE_RULE_OPTION_NAME("repo-s3-kms-key-id"), // opt/repo-s3-kms-key-id
@@ -12338,5 +12415,6 @@ static const uint8_t optionResolveOrder[] =
cfgOptRepoHostPort, // opt-resolve-order
cfgOptRepoS3Key, // opt-resolve-order
cfgOptRepoS3KeySecret, // opt-resolve-order
+ cfgOptRepoS3KmsEncryptionContext, // opt-resolve-order
cfgOptRepoSftpHostFingerprint, // opt-resolve-order
};
diff --git a/src/storage/s3/helper.c b/src/storage/s3/helper.c
index 54eedbe51d..fda75bebc4 100644
--- a/src/storage/s3/helper.c
+++ b/src/storage/s3/helper.c
@@ -117,7 +117,8 @@ storageS3Helper(const unsigned int repoIdx, const bool write, StoragePathExpress
cfgOptionIdxStr(cfgOptRepoS3Region, repoIdx), cfgOptionIdxStr(cfgOptRepoS3Service, repoIdx), keyType,
(StorageS3UriStyle)cfgOptionIdxSeq(cfgOptRepoS3UriStyle, repoIdx), cfgOptionIdxStrNull(cfgOptRepoS3Key, repoIdx),
cfgOptionIdxStrNull(cfgOptRepoS3KeySecret, repoIdx), cfgOptionIdxStrNull(cfgOptRepoS3Token, repoIdx),
- cfgOptionIdxStrNull(cfgOptRepoS3KmsKeyId, repoIdx), cfgOptionIdxStrNull(cfgOptRepoS3SseCustomerKey, repoIdx), role,
+ cfgOptionIdxStrNull(cfgOptRepoS3KmsKeyId, repoIdx), cfgOptionIdxKvNull(cfgOptRepoS3KmsEncryptionContext, repoIdx),
+ cfgOptionIdxStrNull(cfgOptRepoS3SseCustomerKey, repoIdx), role,
tokenFile, credUrl, credCmd, cfgOptionIdxStrNull(cfgOptRepoS3StsHost, repoIdx),
(size_t)cfgOptionIdxUInt64(cfgOptRepoStorageUploadChunkSize, repoIdx),
cfgOptionIdxKvNull(cfgOptRepoStorageTag, repoIdx), host, port, ioTimeoutMs(), protocolType,
diff --git a/src/storage/s3/storage.c b/src/storage/s3/storage.c
index 039492b794..19ec83b198 100644
--- a/src/storage/s3/storage.c
+++ b/src/storage/s3/storage.c
@@ -35,6 +35,7 @@ STRING_STATIC(S3_HEADER_TOKEN_STR, "x-amz-secur
STRING_STATIC(S3_HEADER_SRVSDENC_STR, "x-amz-server-side-encryption");
STRING_STATIC(S3_HEADER_SRVSDENC_KMS_STR, "aws:kms");
STRING_STATIC(S3_HEADER_SRVSDENC_KMSKEYID_STR, "x-amz-server-side-encryption-aws-kms-key-id");
+STRING_STATIC(S3_HEADER_SRVSDENC_CONTEXT_STR, "x-amz-server-side-encryption-context");
STRING_STATIC(S3_HEADER_SSECUSTKEY_ALGO_STR, "x-amz-server-side-encryption-customer-algorithm");
STRING_STATIC(S3_HEADER_SSECUSTKEY_AES256_STR, "AES256");
STRING_STATIC(S3_HEADER_SSECUSTKEY_KEY_STR, "x-amz-server-side-encryption-customer-key");
@@ -110,6 +111,7 @@ struct StorageS3
String *secretAccessKey; // Secret access key
String *securityToken; // Security token, if any
const String *kmsKeyId; // Server-side encryption key
+ const String *kmsEncryptionContext; // Base64 JSON of the KMS encryption context
const String *sseCustomerKey; // Base64 of SSE-C encryption key
const String *sseCustomerKeyMd5; // Base64 of MD5 of SSE-C key
size_t partSize; // Part size for multi-part upload
@@ -598,6 +600,9 @@ storageS3RequestAsync(StorageS3 *const this, const String *const verb, const Str
{
httpHeaderPut(requestHeader, S3_HEADER_SRVSDENC_STR, S3_HEADER_SRVSDENC_KMS_STR);
httpHeaderPut(requestHeader, S3_HEADER_SRVSDENC_KMSKEYID_STR, this->kmsKeyId);
+
+ if (this->kmsEncryptionContext != NULL)
+ httpHeaderPut(requestHeader, S3_HEADER_SRVSDENC_CONTEXT_STR, this->kmsEncryptionContext);
}
// Set SSE-C headers when requested
@@ -1327,7 +1332,8 @@ storageS3New(
const String *const bucket, const String *const endPoint, const String *const region, const String *const service,
const StorageS3KeyType keyType, const StorageS3UriStyle uriStyle, const String *const accessKey,
const String *const secretAccessKey, const String *const securityToken, const String *const kmsKeyId,
- const String *sseCustomerKey, const String *const credRole, const String *const tokenFile, const String *const credUrl,
+ const KeyValue *const kmsEncryptionContext, const String *sseCustomerKey, const String *const credRole,
+ const String *const tokenFile, const String *const credUrl,
const StringList *const credCmd, const String *const stsHost, const size_t partSize, const KeyValue *const tag,
const String *host, const unsigned int port, const TimeMSec timeout, const HttpProtocolType protocolType,
const bool verifyPeer, const String *const caFile, const String *const caPath, const bool requesterPays,
@@ -1349,6 +1355,7 @@ storageS3New(
FUNCTION_TEST_PARAM(STRING, secretAccessKey);
FUNCTION_TEST_PARAM(STRING, securityToken);
FUNCTION_TEST_PARAM(STRING, kmsKeyId);
+ FUNCTION_LOG_PARAM(KEY_VALUE, kmsEncryptionContext);
FUNCTION_TEST_PARAM(STRING, sseCustomerKey);
FUNCTION_TEST_PARAM(STRING, credRole);
FUNCTION_TEST_PARAM(STRING, tokenFile);
@@ -1397,6 +1404,21 @@ storageS3New(
.signingKeyDate = YYYYMMDD_STR,
};
+ if (kmsEncryptionContext != NULL)
+ {
+ MEM_CONTEXT_TEMP_BEGIN()
+ {
+ const String *const context = jsonFromVar(varNewKv(kvDup(kmsEncryptionContext)));
+
+ MEM_CONTEXT_PRIOR_BEGIN()
+ {
+ this->kmsEncryptionContext = strNewEncode(encodingBase64, BUFSTR(context));
+ }
+ MEM_CONTEXT_PRIOR_END();
+ }
+ MEM_CONTEXT_TEMP_END();
+ }
+
// Set prefetch and read over
this->interface.prefetch = prefetch;
this->interface.readOver = readOver;
diff --git a/src/storage/s3/storage.h b/src/storage/s3/storage.h
index 6b251b0fb5..d11b008d73 100644
--- a/src/storage/s3/storage.h
+++ b/src/storage/s3/storage.h
@@ -40,9 +40,9 @@ FN_EXTERN Storage *storageS3New(
const String *path, bool write, time_t targetTime, StoragePathExpressionCallback pathExpressionFunction, const String *bucket,
const String *endPoint, const String *region, const String *service, StorageS3KeyType keyType, StorageS3UriStyle uriStyle,
const String *accessKey, const String *secretAccessKey, const String *securityToken, const String *kmsKeyId,
- const String *sseCustomerKey, const String *credRole, const String *tokenFile, const String *credUrl,
- const StringList *credCmd, const String *stsHost, size_t partSize, const KeyValue *tag, const String *host,
- unsigned int port, TimeMSec timeout, HttpProtocolType protocolType, bool verifyPeer, const String *caFile,
+ const KeyValue *kmsEncryptionContext, const String *sseCustomerKey, const String *credRole, const String *tokenFile,
+ const String *credUrl, const StringList *credCmd, const String *stsHost, size_t partSize, const KeyValue *tag,
+ const String *host, unsigned int port, TimeMSec timeout, HttpProtocolType protocolType, bool verifyPeer, const String *caFile,
const String *caPath, bool requesterPays, unsigned int prefetch, uint64_t readOver);
#endif
diff --git a/test/src/harness/host.c b/test/src/harness/host.c
index 6492c946d8..f84866efaf 100644
--- a/test/src/harness/host.c
+++ b/test/src/harness/host.c
@@ -789,7 +789,7 @@ hrnHostConfig(HrnHost *const this)
hrnHostRepo1Path(this), true, 0, NULL, STRDEF(HRN_HOST_S3_BUCKET), STRDEF(HRN_HOST_S3_ENDPOINT),
STR(HRN_HOST_S3_REGION), STRDEF("s3"), storageS3KeyTypeShared, storageS3UriStyleHost,
STRDEF(HRN_HOST_S3_ACCESS_KEY), STRDEF(HRN_HOST_S3_ACCESS_SECRET_KEY), NULL, NULL, NULL, NULL, NULL,
- NULL, NULL, NULL, 5 * 1024 * 1024, NULL, hrnHostIp(s3), 443, ioTimeoutMs(), httpProtocolTypeHttps,
+ NULL, NULL, NULL, NULL, 5 * 1024 * 1024, NULL, hrnHostIp(s3), 443, ioTimeoutMs(), httpProtocolTypeHttps,
false, NULL, NULL, NULL, 2, 8192);
}
MEM_CONTEXT_OBJ_END();
diff --git a/test/src/module/command/helpTest.c b/test/src/module/command/helpTest.c
index 205f2d1884..7ac678c226 100644
--- a/test/src/module/command/helpTest.c
+++ b/test/src/module/command/helpTest.c
@@ -340,6 +340,7 @@ testRun(void)
" --repo-s3-key S3 repository access key\n"
" --repo-s3-key-secret S3 repository secret access key\n"
" --repo-s3-key-type S3 repository key type\n"
+ " --repo-s3-kms-encryption-context S3 repository KMS encryption context\n"
" --repo-s3-kms-key-id S3 repository KMS key\n"
" --repo-s3-process-cmd S3 authentication process command\n"
" --repo-s3-region S3 repository region\n"
diff --git a/test/src/module/storage/s3Test.c b/test/src/module/storage/s3Test.c
index 9cb8de6d0a..5e23aa763f 100644
--- a/test/src/module/storage/s3Test.c
+++ b/test/src/module/storage/s3Test.c
@@ -17,6 +17,8 @@ Test S3 Storage
Constants
***********************************************************************************************************************************/
#define S3_TEST_HOST "s3.amazonaws.com"
+#define S3_TEST_KMS_CONTEXT \
+ "eyJjbHVzdGVyX2lkIjoicGctcHJvZC0wMSIsInRlbmFudF9pZCI6InRlbmFudC0wMSJ9"
/***********************************************************************************************************************************
Calculate splitMax. This defines the point at which the maximum chunk size must be used in order to be able to upload the maximum
@@ -78,6 +80,7 @@ typedef struct TestRequestParam
const char *authorization;
const char *range;
const char *kms;
+ const char *kmsContext;
const char *sseC;
const char *ttl;
const char *token;
@@ -136,6 +139,9 @@ testRequest(IoWrite *write, Storage *s3, const char *verb, const char *path, Tes
if (param.kms != NULL)
strCatZ(request, ";x-amz-server-side-encryption;x-amz-server-side-encryption-aws-kms-key-id");
+ if (param.kmsContext != NULL)
+ strCatZ(request, ";x-amz-server-side-encryption-context");
+
if (param.sseC != NULL)
{
strCatZ(
@@ -206,6 +212,9 @@ testRequest(IoWrite *write, Storage *s3, const char *verb, const char *path, Tes
strCatFmt(request, "x-amz-server-side-encryption-aws-kms-key-id:%s\r\n", param.kms);
}
+ if (param.kmsContext != NULL)
+ strCatFmt(request, "x-amz-server-side-encryption-context:%s\r\n", param.kmsContext);
+
// Add sseC key
if (param.sseC != NULL)
{
@@ -371,6 +380,7 @@ testRun(void)
TEST_RESULT_STR(driver->accessKey, accessKey, "check access key");
TEST_RESULT_STR(driver->secretAccessKey, secretAccessKey, "check secret access key");
TEST_RESULT_STR(driver->securityToken, NULL, "check security token");
+ TEST_RESULT_STR(driver->kmsEncryptionContext, NULL, "check encryption context is unset");
TEST_RESULT_VOID(
FUNCTION_LOG_OBJECT_FORMAT(driver->httpClient, httpClientToLog, logBuf, sizeof(logBuf)), "httpClientToLog");
@@ -527,6 +537,40 @@ testRun(void)
driver = (StorageS3 *)storageDriver(storageRepoGet(0, false));
TEST_RESULT_STR_Z(driver->signingService, "s3", "check signing service is s3 for default repo-s3-service");
+
+ // -------------------------------------------------------------------------------------------------------------------------
+ TEST_TITLE("KMS encryption context requires a key");
+
+ argList = strLstDup(commonArgList);
+ hrnCfgArgRawZ(argList, cfgOptRepoS3KmsEncryptionContext, "cluster_id=pg-prod-01");
+ TEST_ERROR(
+ HRN_CFG_LOAD(cfgCmdArchivePush, argList), OptionInvalidError,
+ "option 'repo1-s3-kms-encryption-context' not valid without option 'repo1-s3-kms-key-id'");
+
+ // -------------------------------------------------------------------------------------------------------------------------
+ TEST_TITLE("KMS encryption context requires key/value pairs");
+
+ argList = strLstDup(commonArgList);
+ hrnCfgArgRawZ(argList, cfgOptRepoS3KmsKeyId, "kmskey1");
+ hrnCfgArgRawZ(argList, cfgOptRepoS3KmsEncryptionContext, "invalid");
+ TEST_ERROR(
+ HRN_CFG_LOAD(cfgCmdArchivePush, argList), OptionInvalidError,
+ "key/value 'invalid' not valid for 'repo1-s3-kms-encryption-context' option");
+
+ // -------------------------------------------------------------------------------------------------------------------------
+ TEST_TITLE("KMS encryption context sorts keys and escapes JSON");
+
+ argList = strLstDup(commonArgList);
+ hrnCfgArgRawZ(argList, cfgOptRepoS3KmsKeyId, "kmskey1");
+ hrnCfgArgRawZ(argList, cfgOptRepoS3KmsEncryptionContext, "tenant_id=tenant\"\\\xc3\xa9");
+ hrnCfgArgRawZ(argList, cfgOptRepoS3KmsEncryptionContext, "cluster_id=pg-prod-01");
+ HRN_CFG_LOAD(cfgCmdArchivePush, argList);
+
+ driver = (StorageS3 *)storageDriver(storageRepoGet(0, false));
+
+ TEST_RESULT_STR_Z(
+ strNewBuf(bufNewDecode(encodingBase64, driver->kmsEncryptionContext)),
+ "{\"cluster_id\":\"pg-prod-01\",\"tenant_id\":\"tenant\\\"\\\\\xc3\xa9\"}", "check encryption context JSON");
}
// *****************************************************************************************************************************
@@ -778,6 +822,8 @@ testRun(void)
hrnCfgArgRaw(argList, cfgOptRepoS3Role, credRole);
hrnCfgArgRawZ(argList, cfgOptRepoS3KeyType, "auto");
hrnCfgArgRawZ(argList, cfgOptRepoS3KmsKeyId, "kmskey1");
+ hrnCfgArgRawZ(argList, cfgOptRepoS3KmsEncryptionContext, "tenant_id=tenant-01");
+ hrnCfgArgRawZ(argList, cfgOptRepoS3KmsEncryptionContext, "cluster_id=pg-prod-01");
hrnCfgEnvRawZ(cfgOptRepoS3SseCustomerKey, "rA1P");
hrnCfgArgRawZ(argList, cfgOptRepoStorageTag, "Key1=Value1");
hrnCfgArgRawZ(argList, cfgOptRepoStorageTag, " Key 2= Value 2");
@@ -1004,7 +1050,8 @@ testRun(void)
testRequestP(
service, s3, HTTP_VERB_PUT, "/file.txt", .content = "ABCD", .accessKey = "xx", .securityToken = "zz",
- .kms = "kmskey1", .sseC = "rA1P", .tag = "%20Key%202=%20Value%202&Key1=Value1");
+ .kms = "kmskey1", .kmsContext = S3_TEST_KMS_CONTEXT, .sseC = "rA1P",
+ .tag = "%20Key%202=%20Value%202&Key1=Value1");
testResponseP(service);
// Make a copy of the signing key to verify that it gets changed when the keys are updated
@@ -1034,7 +1081,18 @@ testRun(void)
TEST_RESULT_BOOL(bufEq(driver->signingKey, oldSigningKey), false, "signing key changed");
// -----------------------------------------------------------------------------------------------------------------
- TEST_TITLE("write zero-length file");
+ TEST_TITLE("read with KMS encryption context configured");
+
+ testRequestP(service, s3, HTTP_VERB_GET, "/file.txt", .sseC = "rA1P");
+ testResponseP(service, .content = "ABCD");
+
+ TEST_RESULT_STR_Z(strNewBuf(storageGetP(storageNewReadP(s3, STRDEF("file.txt")))), "ABCD", "read file");
+
+ // -----------------------------------------------------------------------------------------------------------------
+ TEST_TITLE("write zero-length file with KMS but without encryption context");
+
+ const String *const kmsEncryptionContext = driver->kmsEncryptionContext;
+ driver->kmsEncryptionContext = NULL;
testRequestP(
service, s3, HTTP_VERB_PUT, "/file.txt", .content = "", .kms = "kmskey1", .sseC = "rA1P",
@@ -1047,8 +1105,11 @@ testRun(void)
// -----------------------------------------------------------------------------------------------------------------
TEST_TITLE("write file in chunks with nothing left over on close");
+ driver->kmsEncryptionContext = kmsEncryptionContext;
+
testRequestP(
- service, s3, HTTP_VERB_POST, "/file.txt?uploads=", .kms = "kmskey1", .sseC = "rA1P",
+ service, s3, HTTP_VERB_POST, "/file.txt?uploads=", .kms = "kmskey1", .kmsContext = S3_TEST_KMS_CONTEXT,
+ .sseC = "rA1P",
.tag = "%20Key%202=%20Value%202&Key1=Value1");
testResponseP(
service,
@@ -1090,6 +1151,8 @@ testRun(void)
// -----------------------------------------------------------------------------------------------------------------
TEST_TITLE("error in success response of multipart upload");
+ driver->kmsEncryptionContext = NULL;
+
// Stop writing tags
driver->tag = NULL;
view thread (18+ messages) latest in thread
Message-ID: <gh-pgbackrest-pgbackrest-2896@github.com>
Permalink: ../gh-pgbackrest-pgbackrest-2896@github.com/
· ·
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox