pg.ddx.io  pgbackrest/pgbackrest GitHub issues and pull requests  
help / color / mirror / Atom feed
[pgbackrest/pgbackrest] PR #2915: Add a minimum repository format for options.
11+ messages / 2 participants
[nested] [flat]

* [pgbackrest/pgbackrest] PR #2915: Add a minimum repository format for options.
@ 2026-11-10 12:26 "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
  0 siblings, 0 replies; 11+ messages in thread

From: dwsteele (@dwsteele) @ 2026-11-10 12:26 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

Options that are only valid for some repository formats were checked in command code, e.g. expire warned when repo-cipher-rotate was set for a repository at format 5. The documentation also had to state the required format by hand.

Add a format key to config.yaml that sets the minimum repository format for an option in the repo group. The option is parsed as usual but cannot be accessed until the format of its repository is set with cfgRepoFormatSet(). When the repository format is lower, an option set on the command line is an error and otherwise the option is unset with a warning if the source is not default. The minimum format is shown in the option reference and in option command-line help.

Set the minimum format of repo-cipher-rotate to 6. Expire sets the repository format after loading backup.info and archive.info and checking that they are at the same format. An interrupted stanza-upgrade leaves archive.info at the new format and backup.info at the prior format, which is now an error for the repository with a hint to complete the upgrade, as backup and check already do. Since archive.info is now loaded before backups are expired, a missing archive.info also stops expire for the repository before any backups are expired.

Options that are not valid on a remote are no longer considered when building the remote parameters, since a remote is started before the repository format is known.

^ permalink  raw  [nested|flat] 11+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2915: Add a minimum repository format for options.
@ 2026-11-10 12:27 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
  9 siblings, 0 replies; 11+ messages in thread

From: dwsteele (@dwsteele) @ 2026-11-10 12:27 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

@dwsteele assigned @dwsteele.

^ permalink  raw  [nested|flat] 11+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2915: Add a minimum repository format for options.
@ 2026-11-10 12:27 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
  9 siblings, 0 replies; 11+ messages in thread

From: dwsteele (@dwsteele) @ 2026-11-10 12:27 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

@dwsteele applied label "enhancement".

^ permalink  raw  [nested|flat] 11+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2915: Add a minimum repository format for options.
@ 2026-11-10 12:27 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
  9 siblings, 0 replies; 11+ messages in thread

From: dwsteele (@dwsteele) @ 2026-11-10 12:27 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

@dwsteele applied label "module [core]".

^ permalink  raw  [nested|flat] 11+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2915: Add a minimum repository format for options.
@ 2026-11-10 12:27 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
  9 siblings, 0 replies; 11+ messages in thread

From: dwsteele (@dwsteele) @ 2026-11-10 12:27 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

@dwsteele set milestone to "2.61.0".

^ permalink  raw  [nested|flat] 11+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2915: Add a minimum repository format for options.
@ 2026-11-10 13:57 ` "pgstef (@pgstef)" <noreply+pgstef@github.com>
  9 siblings, 0 replies; 11+ messages in thread

From: pgstef (@pgstef) @ 2026-11-10 13:57 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

The overall approach looks good to me! I noticed two minor things worth considering:

**1. Archive key rotation format check**

If I read it correctly, the format check previously used `archive.info`, but now relies on `backup.info` through `cfgRepoFormatSet()`. After an interrupted `stanza-upgrade`, it's possible to have `archive.info` at format 6 while `backup.info` is still at format 5, since `archive.info` is saved first.

In that case, archive key rotation would be skipped even though the archive format supports it. Depending on how `repo-cipher-rotate` is configured, this could happen silently (default), produce a misleading warning (config/env), or fail the expire operation (command line).

This would be a rare situation that `stanza-upgrade` can repair, but perhaps we should preserve the archive-specific format check in `expireArchiveCipherRotate()`?

I was able to reproduce this on the PR with the following test, added after the "new key is current" check in the existing rotation tests. It reuses `argList`, `backupInfoContent`, and `timeRotate`, and passes as-is, confirming that the archive key is not rotated:
```c
// backup.info back to format 5 (archive.info saved first by stanza-upgrade, backup.info save interrupted)
HRN_INFO_PUT(storageRepoIdxWrite(1), INFO_BACKUP_PATH_FILE, strZ(backupInfoContent), .cipherSpec = TEST_CIPHER_SPEC);
HRN_INFO_PUT(
    storageRepoIdxWrite(1), INFO_ARCHIVE_PATH_FILE, TEST_ARCHIVE_INFO_FORMAT_6(timeRotate - 31 * SEC_PER_DAY),
    .format = REPOSITORY_FORMAT_6, .header = true, .cipherSpec = TEST_CIPHER_SPEC);

// Command-line repo2-cipher-rotate (argList)
HRN_CFG_LOAD(cfgCmdExpire, argList);

TEST_ERROR(cmdExpire(), CommandError, CFGCMD_EXPIRE " command encountered 1 error(s), check the log file for details");
TEST_RESULT_LOG(
    "P00   WARN: backup '20181119-152850F_20181119-152252D' missing manifest removed from backup.info\n"
    "P00  ERROR: [031]: repo2: option 'repo2-cipher-rotate' requires repository format 6 or higher");

TEST_ASSIGN(
    infoArchive, infoArchiveLoadFile(storageRepoIdx(1), INFO_ARCHIVE_PATH_FILE_STR, TEST_CIPHER_SPEC), "load archive info");
TEST_RESULT_STR_Z(cipherSpecMapIdCurrent(infoArchiveCipherSpecMap(infoArchive)), "1", "cmdline: key NOT rotated");

// Env-var repo2-cipher-rotate (config/env source)
StringList *argListEnv = strLstDup(argListAvoidWarn);
hrnCfgArgRawZ(argListEnv, cfgOptRepo, "2");
hrnCfgArgKeyRawZ(argListEnv, cfgOptRepoPath, 2, TEST_PATH "/repo2");
hrnCfgArgKeyRawZ(argListEnv, cfgOptRepoRetentionFull, 2, "1");
hrnCfgArgKeyRawStrId(argListEnv, cfgOptRepoCipherType, 2, cipherTypeAes256Cbc);
hrnCfgEnvKeyRawZ(cfgOptRepoCipherRotate, 2, "30d");
HRN_CFG_LOAD(cfgCmdExpire, argListEnv);

TEST_RESULT_VOID(cmdExpire(), "expire with env rotate");
TEST_RESULT_LOG(
    "P00   WARN: backup '20181119-152850F_20181119-152252D' missing manifest removed from backup.info\n"
    "P00   WARN: option 'repo2-cipher-rotate' requires repository format 6 or higher\n"
    "P00 DETAIL: repo2: 12-2 archive retention on backup 20181119-152850F, start = 000000010000000000000002\n"
    "P00   INFO: repo2: 12-2 no archive to remove");
hrnCfgEnvKeyRemoveRaw(cfgOptRepoCipherRotate, 2);

TEST_ASSIGN(
    infoArchive, infoArchiveLoadFile(storageRepoIdx(1), INFO_ARCHIVE_PATH_FILE_STR, TEST_CIPHER_SPEC), "load archive info");
TEST_RESULT_STR_Z(cipherSpecMapIdCurrent(infoArchiveCipherSpecMap(infoArchive)), "1", "env: key NOT rotated");
```

**2. Remote parameter regression test**

The existing "remote pg server, backup params for remote" test already catches the regression fixed by `ea62757d0`, but only because `cfgOptionIdxTest()` checks the repository format before checking whether the option is set.

Would it be worth adding a test with an encrypted repository and `repo1-cipher-rotate` explicitly configured? This would make the regression coverage more explicit and help protect the fix if that ordering changes in the future.

Something like the following, after the existing test in `protocolTest.c` (passes with the fix):

```c
// -------------------------------------------------------------------------------------------------------------------------
TEST_TITLE("remote pg server, backup params for remote with repo-cipher-rotate");

argList = strLstNew();
hrnCfgArgRawZ(argList, cfgOptStanza, "test1");
hrnCfgArgRawZ(argList, cfgOptPgPath, "/path/to/1");
hrnCfgArgRawZ(argList, cfgOptPgHost, "pg1-host");
hrnCfgArgKeyRawZ(argList, cfgOptRepoCipherType, 1, "aes-256-cbc");
hrnCfgArgKeyRawZ(argList, cfgOptRepoCipherRotate, 1, "30d");
hrnCfgArgKeyRawZ(argList, cfgOptRepoRetentionFull, 1, "1");
hrnCfgEnvKeyRawZ(cfgOptRepoCipherPass, 1, TEST_CIPHER_PASS);
HRN_CFG_LOAD(cfgCmdBackup, argList, .noStd = true);
hrnCfgEnvKeyRemoveRaw(cfgOptRepoCipherPass, 1);

// Repo format is not known, but repo-cipher-rotate is not valid on the remote
TEST_RESULT_STRLST_Z(
    protocolRemoteParamSsh(protocolStorageTypePg, 0),
    "-o\nLogLevel=error\n-o\nCompression=no\n-o\nPasswordAuthentication=no\npostgres@pg1-host\n"
    TEST_PROJECT_EXE " --exec-id=1-test --lock=test1-backup-1.lock --log-level-console=off --log-level-file=off"
    " --log-level-stderr=error --pg1-path=/path/to/1 --process=0 --remote-type=pg --stanza=test1 backup:remote\n",
    "check config");
```

^ permalink  raw  [nested|flat] 11+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2915: Add a minimum repository format for options.
@ 2026-11-10 14:48 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
  9 siblings, 0 replies; 11+ messages in thread

From: dwsteele (@dwsteele) @ 2026-11-10 14:48 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

> Archive key rotation format check

I think the right thing to do is error when the info files do not match (9f88d530a), just as check and backup do. That forces the user to complete the stanza-upgrade. Expire can't do much anyway if backup is not running unless expire options are changed.

Since archive-push/get only read archive.info they can keep running until the stanza is fully upgraded. We definitely do not want to stop these commands if we can help it.

> Remote parameter regression test

Sure, but instead of adding an entirely new text updated an existing one in 350b7cf90a77.


^ permalink  raw  [nested|flat] 11+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2915: Add a minimum repository format for options.
@ 2026-11-10 15:18 ` "pgstef (@pgstef)" <noreply+pgstef@github.com>
  9 siblings, 0 replies; 11+ messages in thread

From: pgstef (@pgstef) @ 2026-11-10 15:18 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

Agreed, erroring is better than working around a half-finished upgrade.
The updated test looks good too.
Thanks!

^ permalink  raw  [nested|flat] 11+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2915 (approved): Add a minimum repository format for options.
@ 2026-11-10 15:18 ` "pgstef (@pgstef)" <noreply+pgstef@github.com>
  9 siblings, 0 replies; 11+ messages in thread

From: pgstef (@pgstef) @ 2026-11-10 15:18 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>



^ permalink  raw  [nested|flat] 11+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2915: Add a minimum repository format for options.
@ 2026-11-10 17:30 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
  9 siblings, 0 replies; 11+ messages in thread

From: dwsteele (@dwsteele) @ 2026-11-10 17:30 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

@dwsteele closed this.

^ permalink  raw  [nested|flat] 11+ messages in thread

* Re: [pgbackrest/pgbackrest] PR #2915: Add a minimum repository format for options.
@ 2026-11-10 17:30 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
  9 siblings, 0 replies; 11+ messages in thread

From: dwsteele (@dwsteele) @ 2026-11-10 17:30 UTC (permalink / raw)
  To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>

@dwsteele merged this (commit 35a4591dd638).

^ permalink  raw  [nested|flat] 11+ messages in thread


end of thread, other threads:[~2026-11-10 17:30 UTC | newest]

Thread overview: 11+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-11-10 12:26 [pgbackrest/pgbackrest] PR #2915: Add a minimum repository format for options. "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
2026-11-10 12:27 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
2026-11-10 12:27 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
2026-11-10 12:27 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
2026-11-10 12:27 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
2026-11-10 13:57 ` "pgstef (@pgstef)" <noreply+pgstef@github.com>
2026-11-10 14:48 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
2026-11-10 15:18 ` "pgstef (@pgstef)" <noreply+pgstef@github.com>
2026-11-10 15:18 ` Re: [pgbackrest/pgbackrest] PR #2915 (approved): Add a minimum repository format for options. "pgstef (@pgstef)" <noreply+pgstef@github.com>
2026-11-10 17:30 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>
2026-11-10 17:30 ` "dwsteele (@dwsteele)" <noreply+dwsteele@github.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox