From: dwsteele (@dwsteele) <noreply+dwsteele@github.com>
To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>
Subject: Re: [pgbackrest/pgbackrest] issue #2888: Proposal: experimental `syncfs` option for fresh restores
Date: Tue, 27 Oct 2026 11:14:03 +0000
Message-ID: <gh-pgbackrest-pgbackrest-2888-c6440497789@github.com> (raw)
In-Reply-To: <gh-pgbackrest-pgbackrest-2888@github.com>
References: <gh-pgbackrest-pgbackrest-2888@github.com>
Thanks for the detailed proposal and benchmark. This seems reasonable to me. PostgreSQL already supports syncfs (`recovery_init_sync_method=syncfs` in PG14 and `--sync-method=syncfs` for the client tools in PG17). Opening the directory before writes start and keeping it open until `syncfs()` is the right approach to get writeback errors on Linux 5.8+.
Please go ahead and submit a PR. Some things to address:
- **Tablespaces and links:** `syncfs()` only syncs the filesystem that contains the fd. All manifest targets need to be covered: pgdata, tablespaces, path links such as `pg_wal` and the parent paths of file links. After clean/build and before the restore jobs start, open each target, dedupe by `st_dev` and keep one fd per filesystem. After the jobs finish, call `syncfs()` on each fd. With that in place there is no need to limit the WAL mapping to the same filesystem or to defer tablespaces. Large clusters often have tablespaces, so an option that refuses them would be much less useful.
- **Delta:** I don't see a strong reason to exclude delta restores. Files that delta matches are not written or synced today, so syncfs would not weaken anything.
- **pg_control:** all `syncfs()` calls must complete before `pg_control` is renamed. The `global` path sync after the rename stays as it is.
- **Path sync:** the per-path sync loop in `cmdRestore()` can be skipped for paths on filesystems that were synced with `syncfs()`.
- **Platform:** syncfs is Linux only, so it needs a meson check and the option should error on other platforms. On kernels before 5.8 `syncfs()` does not report I/O errors. Distro backports make a kernel version check unreliable, so I would document this the way PostgreSQL does rather than try to detect it.
- **Option:** I prefer an enum such as `restore-sync-method=fsync|syncfs` over a boolean. It should be marked `beta: true` in `build/config.yaml` so it requires `--beta` while it is experimental. I
- **Tests:** unit tests in `test/src/module/command/restoreTest.c` with full coverage. See CONTRIBUTING.md.
Also note that PostgreSQL runs `SyncDataDirectory()` at startup when `pg_control` does not show a clean shutdown, and that is always true for a restored online backup. With this many files the first start pays for another full sync unless `recovery_init_sync_method=syncfs` is set. It's not part of this change but it may affect your timings.