From: Jyrgal (@Jyrgal) <noreply+Jyrgal@github.com>
To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>
Subject: [pgbackrest/pgbackrest] issue #2913: Proposal: Checkpoint saves in resume feature override for S3 object lock
Date: Mon, 09 Nov 2026 02:45:54 +0000
Message-ID: <gh-pgbackrest-pgbackrest-2913@github.com> (raw)
## Problem
When `resume` is enabled (the default), the backup command saves the manifest to `backup.manifest.copy` periodically during a backup, controlled by `manifest-save-threshold`. On S3 each save overwrites the object, and on a bucket with Object Lock enabled every overwrite creates a new object version.
If the bucket has a default retention rule, every one of these versions is locked for the full default period. That period is usually sized for the backups themselves (e.g. 30 days to several years), not for temporary checkpoints.
So:
- A long backup can produce many locked checkpoint versions of `backup.manifest.copy`. Only the final version is needed once the backup completes.
- These versions can't be deleted (in compliance mode, by anyone) until retention expires, so they add storage cost and clutter for the whole retention period.
- Lifecycle rules can't clean them up early, because Object Lock takes precedence over lifecycle expiration.
## Current workarounds
- `resume=n` stops the checkpoint saves but gives up resuming a failed backup, which matters most for large and long backups.
- Raising `manifest-save-threshold` cuts the number of versions but doesn't avoid the full-length lock on each one. - Lowering the bucket default retention weakens protection for the actual backup files.
None of these lets you keep both resume and strong retention on the backup itself.
## Proposal
S3 lets a PutObject / CreateMultipartUpload request set explicit retention on the object version it creates with `x-amz-object-lock-mode` and `x-amz-object-lock-retain-until-date`. Explicit retention on a version overrides the bucket default ([AWS docs](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html)).
Add per-repo options to set a separate, usually shorter, retention on checkpoint saves only:
- `repo-s3-manifest-save-object-lock`: retention period for checkpoint copies of `backup.manifest.copy` (e.g. `1d`). Unset by default, so current behavior doesn't change.
- `repo-s3-manifest-save-object-lock-mode`: `governance` (default) or `compliance`.
The final `backup.manifest.copy` and `backup.manifest` would not be affected and would keep the bucket default retention.
Obviously, this configuration assumes the bucket have Object Lock enabled on the bucket and the `s3:PutObjectRetention` permission for the credentials.
I have an implementation ready and will open a PR referencing this issue if we are happy with making this change.