From: Noah Misch <noah@leadboat.com>
To: hlinnaka@iki.fi
To: orlovmg@gmail.com
Cc: pgsql-hackers@postgresql.org
Subject: 64-bit MultiXactOffset vs. 9.3->9.4 upgrade, pg_resetwal, "wraparound" msg
Date: Thu, 27 Aug 2026 16:17:57 -0700
Message-ID: <20260827231757.78.noahmisch@microsoft.com> (raw)
commit bd8d9c9 wrote:
> Commit: Heikki Linnakangas <heikki.linnakangas@iki.fi>
> CommitDate: Tue Dec 9 13:53:03 2025 +0200
>
> Widen MultiXactOffset to 64 bits
> --- a/src/bin/pg_upgrade/pg_upgrade.c
> +++ b/src/bin/pg_upgrade/pg_upgrade.c
> + nxtmulti = old_cluster.controldata.chkpnt_nxtmulti;
> + if (old_cluster.controldata.cat_ver >= MULTIXACT_FORMATCHANGE_CAT_VER)
> + {
> + /* Versions 9.3 - 18: convert all multixids */
> + oldstMulti = old_cluster.controldata.chkpnt_oldstMulti;
If a cluster's upgrade history includes an upgrade from 9.3 to early 9.4, it
may have a wrong value here. Specifically, upgrades done before a61daa14
(2014-07 commit) have that hazard. We still have backend code to detect such
cases and reduce damage:
ereport(LOG,
(errmsg("cannot truncate up to MultiXact %u because it does not exist on disk, skipping truncation",
newOldestMulti)));
However, the pg_upgrade side from the v19 commit lacks such protection. If
heap tuples still reference older multixacts than the faulty control data
suggests, pg_upgrade will copy too small a range, making affected tuples
unreadable.
I also had Opus 4.8 look for defects in this change and write test cases. It
didn't find the above problem, but it had other findings. I'm attaching the
full report. I recommend fixing at least these before release:
> +| 5 | `resetwal-nextmxoff-zero` | `src/bin/pg_resetwal/pg_resetwal.c:703`, and `-O` with no zero check at 297-307 | `pg_resetwal -f` (or `-O 0`) leaves `nextMultiOffset = 0`, the reserved "invalid offset"; first multixact created afterwards is permanently unreadable: `ERROR: MultiXact 1 has invalid offset` | Yes — `src/test/modules/test_slru/t/003_multixact_offset.pl` |
> +| 7 | `members-truncation-apparent-wraparound` | `src/backend/access/transam/multixact.c:2642-2647` | When `nextOffset` lands on a members page boundary, truncation logs `could not truncate directory "pg_multixact/members": apparent wraparound` (impossible for a 64-bit counter) and reclaims nothing | Yes — `003_multixact_offset.pl` |
Others are more optional or already reported. In particular, two other
findings were already reported and apparently fixed after the Opus run, in
thread "pg_upgrade silently truncates nextMultiOffset to 32 bits":
https://www.postgresql.org/message-id/CAD21AoCvzerscfU8o4ARQ793yAGHpQ72r2x5apeC_W2-k%3DSLCQ%40mail.g...
commit bdff567 (cqla/mxactoffset64-defect-tests)
Author: Noah Misch <noah@leadboat.com>
AuthorDate: Tue Jul 21 17:52:05 2026 +0000
Commit: Noah Misch <noah@leadboat.com>
CommitDate: Tue Jul 21 17:52:05 2026 +0000
Test user-visible defects left by the 64-bit MultiXactOffset widening
Commit bd8d9c9 widened MultiXactOffset from 32 to 64 bits. Four of the
user-visible defects it left behind are still present in master; these
tests demonstrate all four. Every test fails on master today and passes
once the corresponding product-code defect is fixed.
src/test/modules/test_slru/t/003_multixact_offset.pl covers three:
* bd8d9c9 made offset 0 the reserved "invalid offset" marker, changing
BootStrapXLOG() to start nextMultiOffset at 1 instead of 0, but left
pg_resetwal's GuessControlValues() assigning 0 and left -O accepting 0
with no zero check, unlike -m. A cluster reset that way cannot read
back the first multixact it creates: "ERROR: MultiXact 1 has invalid
offset".
* pg_control_checkpoint() still runs the widened field through
TransactionIdGetDatum(), and pg_proc.dat still declares the
next_multi_offset output column as xid, so SQL reports the value
modulo 2^32 while pg_controldata prints it in full. Observed: control
file 4294967896, SQL 600.
* PerformMembersTruncation() passes MXOffsetToMemberPage(newOldestOffset)
to SimpleLruTruncate() with no step-back, unlike PerformOffsetsTruncation()
directly below it, whose comment names exactly this hazard. When
nextOffset lands on a members page boundary the cutoff page is one past
the last page ever written, so SimpleLruTruncate()'s endpoint check
fires: the server logs 'could not truncate directory
"pg_multixact/members": apparent wraparound' -- impossible for a
counter that cannot wrap -- and no obsolete segment is reclaimed.
src/bin/pg_upgrade/t/007_multixact_conversion.pl gains a fourth: while
chkpnt_nxtmxoff became uint64, get_control_data() still reads
NextMultiOffset with str2uint(), which returns unsigned int. Upgrading a
cluster whose offset has passed 2^32 gives the new cluster an offset
modulo 2^32, pointing back into members space that is still in use. No
SLRU conversion is involved, so this case runs regardless of $oldinstall.
bd8d9c9-untested-defects.md describes four further confirmed defects that
are not covered here, with the reason each is impractical to test: two
stale documentation claims in maintenance.sgml, pg_get_multixact_stats()
reporting lifetime rather than retained members on a hot standby, and a
server LOG message that reports the wrong subsystem as disabled. It also
records three candidates that adversarial verification refuted, so they
are not re-chased.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WxWnwpyK9G3DohkEZJxsLU
---
bd8d9c9-untested-defects.md | 529 +++++++++++++++++++++
src/bin/pg_upgrade/t/007_multixact_conversion.pl | 68 +++
src/test/modules/test_slru/meson.build | 3 +-
.../modules/test_slru/t/003_multixact_offset.pl | 254 ++++++++++
4 files changed, 853 insertions(+), 1 deletion(-)
diff --git a/bd8d9c9-untested-defects.md b/bd8d9c9-untested-defects.mdnew file mode 100644index 0000000..0740d89--- /dev/null+++ b/bd8d9c9-untested-defects.md@@ -0,0 +1,529 @@+# Defects in bd8d9c9 ("Widen MultiXactOffset to 64 bits") still present in master — the ones the companion test does not cover++Target commit: `bd8d9c9bdfa0c2168bb37edca6fa88168cacbbaa`, Heikki Linnakangas, 2025-12-09+(author Maxim Orlov), "Widen MultiXactOffset to 64 bits".+Tree audited: `/home/nm/src/pg/postgresql` at `c90c967` — 2028 commits after bd8d9c9.+Every line quoted below was re-read from that working tree while writing this report.++---++## 1. Scope and method++### What was audited++bd8d9c9 as it stands **today in master**, not as it was committed. The commit widened+`MultiXactOffset` from `uint32` to `uint64` (`src/include/c.h:807`), deleted the members-space+wraparound apparatus (`offsetStopLimit`, `MultiXactOffsetWouldWrap`, the emergency+"multixact members limit exceeded" ERROR — `grep -rn 'offsetStopLimit|MultiXactOffsetWouldWrap' src/`+now returns nothing), kept the members-driven clamp on vacuum's multixid freeze cutoff, and added+a pg_upgrade path that rewrites v18 32-bit SLRU files into the new format.++Areas covered: `multixact.c` (allocation, startup/trim, truncation, freeze-threshold),+`multixact.h`/`multixact_internal.h` (on-disk and WAL layout), pg_upgrade's conversion and control+data handling, pg_resetwal, pg_controldata (both the frontend program and the SQL function),+vacuum/autovacuum freeze scheduling, `pg_get_multixact_stats()`, and the SGML documentation that+describes any of the above.++### How++Fan-out static analysis: 7 areas x 4 independent reading lenses, each producing candidate findings+with a mandatory verbatim source quote and a concrete trigger. Every candidate then went through+adversarial 3-vote verification: three independent attempts to **refute** it, each required to read+master's source itself and to check `git log bd8d9c9..master -- <path>` for a follow-up fix. Anything+that lost 2 or more votes was dropped. Survivors were then handed to a reproduction pass that built+real clusters from a prebuilt worktree at the same commit (`/home/nm/src/pg/pg-master-mxact64`,+`--enable-cassert`) and tried to observe the symptom end to end.++### Standard applied++A finding counts only if **all** of the following hold:++1. It is present in master's working tree today (not merely in the bd8d9c9 diff — ~50 follow-up+ commits, e.g. `ac94ce8`, `d4b7bde`, `ecb553a`, `87a350e`, `6aa26be`, `516310e`, `102bdaa`,+ `ccae90a`, already repaired much of this code).+2. It is attributable to bd8d9c9: introduced by it, a regression versus pre-bd8d9c9 behavior, or a+ hole in code bd8d9c9 added. Where attribution is arguable, this report says so explicitly.+3. It produces something a user or DBA can **observe**: a wrong number, a false message, an ERROR+ where success is required, data left on disk that should be gone, documentation that contradicts+ the code.+4. It is reachable without hand-corrupting files into a state the code otherwise prevents.++"Risk areas", style, comment quality and unreachable overflow were rejected by construction. A prior+audit of this commit produced 26 "CRITICAL risk areas" that were all vapor; the refutation pass here+killed three more (section 5).++---++## 2. Summary table++| # | id | location | symptom | covered by test? |+|---|----|----------|---------|------------------|+| 1 | `maintenance-sgml-32bit-members` | `doc/src/sgml/maintenance.sgml:782` | Manual still says the pg_multixact members storage area "uses a 32-bit counter"; `MultiXactOffset` is now `uint64` and members cannot wrap | **No** — doc-only; nothing to assert at runtime |+| 2 | `no-launcher-signal-for-member-space` | `doc/src/sgml/maintenance.sgml:817-823` (behavior at `src/backend/access/transam/multixact.c:1009,2161-2183`) | Docs promise member-space-driven aggressive vacuums "even if autovacuum is nominally disabled"; since bd8d9c9 no code path starts the launcher on member growth | **No** — needs >2e9 member entries (~10 GB) to observe at runtime |+| 3 | `standby-oldestoffset-zero-stats` | `src/backend/access/transam/multixact.c:2555`, `src/backend/utils/adt/multixactfuncs.c:128-130` | On a hot standby `pg_get_multixact_stats()` reports `num_members`/`members_size` as lifetime allocation, not retained members; disagrees with its own primary | **No** — needs a primary/standby pair; attribution partly to a later commit (see §3.3) |+| 4 | `false-truncation-disabled-log` | `src/backend/access/transam/multixact.c:2477-2479` | LOG says "MultiXact member truncation is disabled …"; truncation is in fact unaffected, and the real damage (stale `oldestOffset`) is never mentioned | **No** — needs a hand-staged offsets-segment/oldestMulti mismatch |+| 5 | `resetwal-nextmxoff-zero` | `src/bin/pg_resetwal/pg_resetwal.c:703`, and `-O` with no zero check at 297-307 | `pg_resetwal -f` (or `-O 0`) leaves `nextMultiOffset = 0`, the reserved "invalid offset"; first multixact created afterwards is permanently unreadable: `ERROR: MultiXact 1 has invalid offset` | Yes — `src/test/modules/test_slru/t/003_multixact_offset.pl` |+| 6 | `pg-control-checkpoint-next-multi-offset-xid` | `src/backend/utils/misc/pg_controldata.c:133`, `src/include/catalog/pg_proc.dat:12374` | `pg_control_checkpoint().next_multi_offset` reports the value mod 2^32 (column still typed `xid`), disagreeing with `pg_controldata` | Yes — `003_multixact_offset.pl` |+| 7 | `members-truncation-apparent-wraparound` | `src/backend/access/transam/multixact.c:2642-2647` | When `nextOffset` lands on a members page boundary, truncation logs `could not truncate directory "pg_multixact/members": apparent wraparound` (impossible for a 64-bit counter) and reclaims nothing | Yes — `003_multixact_offset.pl` |+| 8 | `pgupgrade-nxtmxoff-str2uint` | `src/bin/pg_upgrade/controldata.c:292` | pg_upgrade parses the widened `NextMultiOffset` with 32-bit `str2uint()`; a v19+ source past 2^32 yields a new cluster with offset mod 2^32 — either fails to start past the "must re-initdb" point, or silently overwrites live members | Yes — `src/bin/pg_upgrade/t/007_multixact_conversion.pl` |++Rows 5-8 are documented in the committed test (`003_multixact_offset.pl`, `meson.build`, and the+additions to `007_multixact_conversion.pl`) and are not re-litigated here. Rows 1-4 have no test;+they are detailed below.++---++## 3. Per-defect detail — the untested findings++### 3.1 `maintenance-sgml-32bit-members` — the manual still describes a 32-bit members counter++**Location:** `doc/src/sgml/maintenance.sgml:778-786`, section "Multixacts and Wraparound".++```+778 Like transaction IDs, multixact IDs are implemented as a+779 32-bit counter and corresponding storage, all of which requires+780 careful aging management, storage cleanup, and wraparound handling.+781 There is a separate storage area which holds the list of members in+782 each multixact, which also uses a 32-bit counter and which must also+783 be managed. The system function+784 <function>pg_get_multixact_members()</function> described in+785 <xref linkend="functions-pg-snapshot"/> can be used to examine the+786 transaction IDs associated with a multixact ID.+```++**What the code says.** The members storage area is addressed by `MultiXactOffset`, and+`src/include/c.h:807` is now `typedef uint64 MultiXactOffset;`. `multixact.c:1106-1112`:++> Offsets are 64-bit integers and will never wrap around. Firstly, it would take an unrealistic+> amount of time and resources to consume 2^64 offsets. Secondly, multixid creation is WAL-logged,+> so you would run out of LSNs before reaching offset wraparound.++and `multixact.c:2826-2835` reduces `MultiXactMemberPagePrecedes()` to `return page1 < page2;`+with the comment "members never wrap around".++**What is and is not wrong.** Precisely one clause is false: "which also uses a 32-bit counter"+(line 782). Two neighbouring claims are still correct and must not be reported as bugs:++* Line 778-780, about multixact **IDs** being a 32-bit counter needing wraparound handling, is still+ true — `src/include/c.h:805` is `typedef TransactionId MultiXactId;` and multixids still wrap.+* "and which must also be managed" is still true — members space still drives SLRU truncation and the+ 2-billion / 4-billion-entry aggressive-vacuum thresholds documented at lines 814-827.++**Trigger.** Read the v19 manual: "Routine Database Maintenance Tasks" → "Preventing Transaction ID+Wraparound Failures" → "Multixacts and Wraparound", first paragraph.++**User-visible consequence.** A DBA planning capacity concludes there is a hard ~2^32-member / ~20 GB+ceiling on `pg_multixact/members` beyond which the cluster will refuse to assign multixids — the+pre-v19 behavior. There is no such ceiling in v19; the 4-billion figure at line 824 is only a vacuum+aggressiveness threshold. The stated remedy (watch for members wraparound) is for a hazard that no+longer exists.++**Verification evidence.**+* `git show --stat bd8d9c9 -- doc/` shows the commit touched exactly one SGML file,+ `doc/src/sgml/ref/pg_resetwal.sgml` (updating the `-m` multiplier from 65536 to 32768 for the now+ 8-byte offsets). `maintenance.sgml` was not touched.+* `git blame -L 776,786 doc/src/sgml/maintenance.sgml` attributes lines 781-782 to `53bb309d`+ ("Teach autovacuum about multixact member wraparound", 2014), where the sentence was correct.+ Nothing in `bd8d9c9..master` has changed it.+* Interesting corroboration that this is an oversight, not a deliberate retention: the *sibling*+ paragraph was updated after bd8d9c9. `git blame -L 814,827` attributes lines 817-827 to `97b10177`+ (2025-12-30). `git show bd8d9c9^:doc/src/sgml/maintenance.sgml` shows the pre-commit text there+ read "The members storage area can grow up to about 20GB before reaching wraparound"; that+ sentence is gone in master. So someone already scrubbed one stale wraparound reference from this+ section and missed line 782.++**Why it is not covered by a test.** It is a documentation defect: there is no runtime assertion that+can fail. PostgreSQL has no doc/code consistency harness that could catch a prose claim about an+integer width. The fix is a one-line SGML edit, not a test.++**Test someone should write:** none. Fix the sentence, e.g. "There is a separate storage area which+holds the list of members in each multixact; it does not wrap around, but its disk usage must still+be managed." A reviewer should also re-read the whole sect3 for other pre-64-bit residue while there.++---++### 3.2 `no-launcher-signal-for-member-space` — docs promise member-driven vacuums with autovacuum off; the code no longer delivers them++**Location of the false claim:** `doc/src/sgml/maintenance.sgml:814-827`.++```+814 <para>+815 As a safety device, an aggressive vacuum scan will+816 occur for any table whose multixact-age is greater than <xref+817 linkend="guc-autovacuum-multixact-freeze-max-age"/>. Also, if the number+818 of multixact member entries created exceeds approximately 2 billion+819 entries (occupying roughly 10GB in the+820 <literal>pg_multixact/members</literal> directory), aggressive vacuum+821 scans will occur more often for all tables, starting with those that+822 have the oldest multixact-age. Both of these kinds of aggressive+823 scans will occur even if autovacuum is nominally disabled. At approximately+824 4 billion entries (occupying roughly 20GB in the+825 <literal>pg_multixact/members</literal> directory), even more aggressive+826 vacuum scans are triggered to reclaim member storage space.+827 </para>+```++The word **"Both"** on line 822 is now false for the second kind of scan.++**What the code does.** `grep -rn PMSIGNAL_START_AUTOVAC_LAUNCHER src/backend/` returns exactly six+sites; the three in multixact.c are:++* `multixact.c:1034` and `multixact.c:1059`, both inside+ `if (!MultiXactIdPrecedes(result, MultiXactState->multiVacLimit))` at line 1009 — keyed on+ multixact **age** (`multiVacLimit`, `multiStopLimit`, `result % 65536`);+* `multixact.c:2183`, `if (MultiXactIdPrecedes(multiVacLimit, curMulti) && IsUnderPostmaster)` —+ also age.++None of them reads `nextOffset` or `oldestOffset`. bd8d9c9 removed the two that did: the+`MULTIXACT_MEMBER_SAFE_THRESHOLD` arm of `GetNewMultiXactId()` and the `needs_offset_vacuum` term in+`SetMultiXactIdLimit()` (verifiable in `git show bd8d9c9 -- src/backend/access/transam/multixact.c`).++**This part of the change was deliberate**, and the reasoning is still in the tree at+`multixact.c:2166-2172`:++```+2166 /*+2167 * Offsets are 64-bits wide and never wrap around, so we don't need to+2168 * consider them for emergency autovacuum purposes. But now that we're in+2169 * a consistent state, determine MultiXactState->oldestOffset. It will be+2170 * used to adjust the freezing cutoff, to keep the offsets disk usage in+2171 * check.+2172 */+2173 SetOldestOffset();+```++So the finding is **not** "the trigger was accidentally dropped from multixact.c". It is that the+documentation still asserts the deleted behavior, and — pointedly — the paragraph was *rewritten+after* bd8d9c9 by `97b10177` (2025-12-30), which introduced the new "2 billion / 4 billion entries"+wording and carried the "even if autovacuum is nominally disabled" clause forward unexamined. Before+bd8d9c9 the sentence was true.++**What still works, so the report is not overstated.** `MultiXactMemberFreezeThreshold()`+(`multixact.c:2589-2636`, using `MULTIXACT_MEMBER_LOW_THRESHOLD` = 2000000000 and+`MULTIXACT_MEMBER_HIGH_THRESHOLD` = 4000000000 at `multixact.c:99-100`) is alive and consulted by:++* manual `VACUUM` — `src/backend/commands/vacuum.c:1159`;+* `do_start_worker()` — `src/backend/postmaster/autovacuum.c:1182`+ (`multiForceLimit = recentMulti - MultiXactMemberFreezeThreshold();`);+* `relation_needs_vacanalyze()`, with `force_vacuum` bypassing `av_enabled`+ (`autovacuum.c:3246`, `if (force_vacuum) *dovacuum = true;`).++So with `autovacuum = on`, or for a manual VACUUM, member-pressure clamping behaves exactly as+documented. The regression is confined to `autovacuum = off`: nothing then starts the launcher on+member growth. The launcher's emergency path is `autovacuum.c:593-601`:++```+593 if (!AutoVacuumingActive())+594 {+595 if (!ShutdownRequestPending)+596 do_start_worker();+597 proc_exit(0); /* done */+598 }+```++— reachable only when the postmaster sets `start_autovac_launcher` from+`PMSIGNAL_START_AUTOVAC_LAUNCHER` (`postmaster.c:3832`). No member-space signal, no launcher, no+clamp applied.++**Trigger.** `autovacuum = off`, `autovacuum_multixact_freeze_max_age` at its 400,000,000 default.+Run a workload that creates multixacts with many members each but consumes few multixids and few+XIDs — e.g. transactions that each take `FOR KEY SHARE` on tens of thousands of rows that are already+share-locked — until `nextOffset - oldestOffset` exceeds 2,000,000,000. At 200 members/multixact that+is ~10 million multixacts, far under the 400M multixact age that trips the surviving age-based+trigger.++**User-visible consequence.** `pg_multixact/members` grows past the 10 GB and 20 GB figures the manual+cites without any autovacuum being launched on that account. It is not literally unbounded — growth+continues until multixact age crosses `autovacuum_multixact_freeze_max_age`, or until an+XID-wraparound emergency launcher fires from `varsup.c:136` — but that ceiling is enormously higher+than what the docs describe: at ~200 members/multixact, roughly 80 billion member entries+(hundreds of GB) versus v18's ~2^31-member relief point. No wrong results, no crash; a disk-space+surprise plus a false promise in the manual.++**Ancillary (comment-only, does not qualify on its own):** the block comment bd8d9c9 left at+`multixact.c:1000-1002` still says "If we're past multiVacLimit **or the safe threshold for member+storage space, or we don't know what the safe threshold for member storage is**, start trying to+force autovacuum cycles" — describing code the same commit deleted.++**Why it is not covered by a test.** Reaching `MULTIXACT_MEMBER_LOW_THRESHOLD` requires materialising+two billion real member entries (~10 GB of `pg_multixact/members`) — hours of runtime and 10+ GB of+disk on a machine that has 2 cores. There is no debug hook to lower the threshold: 2000000000 is a+compile-time `#define`, not a GUC, and no injection point exists in `MultiXactMemberFreezeThreshold()`+or in the launcher path. `pg_resetwal -O` can jump `nextOffset` forward cheaply, but that does not+help: the missing signal is in `GetNewMultiXactId()`/`SetMultiXactIdLimit()`, which would have to be+*reached* with the launcher stopped, and there is nothing left there to observe — the assertion would+be about a signal that is simply never sent.++**Test someone with the resources should write.** Two tiers:++1. *Cheap and honest (recommended):* fix the documentation and add nothing. The behavior change is+ intentional; only the manual is wrong.+2. *If the behavior is deemed a regression to fix:* add a `test_slru` TAP test that (a) sets+ `autovacuum = off`, (b) uses a debug-build-only GUC or injection point that lowers+ `MULTIXACT_MEMBER_LOW_THRESHOLD` to something like 100000, (c) creates enough members via+ `test_create_multixact()` to cross it, and (d) asserts from the postmaster log that an autovacuum+ worker started, plus that `pg_multixact/members` shrank. Introducing the threshold override is the+ real work; without it the test cannot exist at reasonable cost.++---++### 3.3 `standby-oldestoffset-zero-stats` — `pg_get_multixact_stats()` on a hot standby reports lifetime members, not retained members++**Location:** `src/backend/access/transam/multixact.c:2545-2559` and its SQL consumer+`src/backend/utils/adt/multixactfuncs.c:127-130`.++```+2545 void+2546 GetMultiXactInfo(uint32 *multixacts, MultiXactOffset *nextOffset,+2547 MultiXactId *oldestMultiXactId, MultiXactOffset *oldestOffset)+2548 {+2549 MultiXactId nextMultiXactId;+2550 +2551 LWLockAcquire(MultiXactGenLock, LW_SHARED);+2552 *nextOffset = MultiXactState->nextOffset;+2553 *oldestMultiXactId = MultiXactState->oldestMultiXactId;+2554 nextMultiXactId = MultiXactState->nextMXact;+2555 *oldestOffset = MultiXactState->oldestOffset;+2556 LWLockRelease(MultiXactGenLock);+```++```+127 GetMultiXactInfo(&multixacts, &nextOffset, &oldestMultiXactId, &oldestOffset);+128 members = nextOffset - oldestOffset;+129 membersBytes = MultiXactOffsetStorageSize(nextOffset, oldestOffset);+```++**Mechanism.** `MultiXactState->oldestOffset` is assigned in exactly two places:+`SetOldestOffset()` at `multixact.c:2488`, and `TruncateMultiXact()` at `multixact.c:2788`. Neither+runs in recovery:++* `TruncateMultiXact()` opens with `Assert(!RecoveryInProgress());` (`multixact.c:2685`).+* `SetOldestOffset()` is called only from `SetMultiXactIdLimit()` at `multixact.c:2173`, which is+ *after* the early return at `multixact.c:2161-2162`:+ `if (!MultiXactState->finishedStartup) return;`. `finishedStartup` is set only by+ `TrimMultiXact()` (`multixact.c:2006`), whose sole caller is `xlog.c:6527`, at end of recovery.+* The redo path (`multixact.c:2985-2990`) calls `SetMultiXactIdLimit(xlrec.oldestMulti, …)` — which+ correctly stores `oldestMultiXactId` at `multixact.c:2140` before the early return — and then+ `PerformMembersTruncation(xlrec.oldestOffset)` / `PerformOffsetsTruncation()`, using the WAL+ record's value directly. It never stores `xlrec.oldestOffset` into shared memory.++So on a never-promoted standby, `MultiXactState->oldestOffset` keeps its zeroed shmem value for the+life of the server while `nextOffset` is advanced continuously by redo. `num_members` therefore comes+out as `nextOffset - 0` and `members_size` as `5 * nextOffset` bytes+(`MultiXactOffsetStorageSize()`, `multixact_internal.h:125-134`: `MULTIXACT_MEMBERGROUP_SIZE /+MULTIXACT_MEMBERS_PER_MEMBERGROUP` = 20/4 = 5). `num_mxids` and `oldest_multixact` are correct.++The overreport is exactly the primary's current `oldestOffset`, and since bd8d9c9 made offsets+64-bit and monotonic, that quantity climbs for the life of the installation. Meanwhile redo *does*+physically delete the members segments, so the standby reports gigabytes of members it does not have.++**Trigger (reproduced).** Primary with `wal_level = replica`; create a table, take `FOR SHARE` from+concurrent sessions to build multixacts; `pg_basebackup -R` a standby and start it; on the primary+`ALTER DATABASE template0 ALLOW_CONNECTIONS true`, `VACUUM FREEZE` in every database, `CHECKPOINT`,+so `TruncateMultiXact()` advances `oldestOffset`. Observed: primary `num_members` 0, `members_size` 0;+standby `num_members` 3, `members_size` 15, `pg_is_in_recovery()` true. A larger run (1000 multixacts+of 2 members, `nextOffset` 2001) showed the standby reporting 2001/10005 against a primary reporting+0/0. Promoting the standby corrects it immediately, confirming the `finishedStartup` mechanism.++**User-visible consequence.** `SELECT * FROM pg_get_multixact_stats()` on a hot standby contradicts+the same query on its primary for byte-identical data, and contradicts+`doc/src/sgml/func/func-info.sgml:3010-3013`, which defines `num_members` as "the total number of+multixact member entries **currently present in the system**" and `members_size` as "the storage+occupied by `num_members` in the `pg_multixact/members` directory". Monitoring built on this function+sees a standby that appears to be holding hundreds of GB of members it does not hold.++**Attribution — stated honestly.** This is the weakest attribution in the set, and it survived+2/3 refutation votes rather than 3/3. The dissenting reading is correct on the facts and should be+recorded: the `finishedStartup` gate and the redo path are **identical in `bd8d9c9^`**, so the+standby's stuck `oldestOffset` predates the commit; pre-commit it was harmless because the only+consumer, `MultiXactMemberFreezeThreshold()`, never runs in recovery. What bd8d9c9 contributed is+deleting `MultiXactStateData.oldestOffsetKnown` and changing `GetMultiXactInfo()` from returning+`bool` to `void` — it used to short-circuit with `*members = 0; … return false;` so callers could+distinguish "unknown" from "zero". `97b10177` (2025-12-30) then wired the now-unqualified value into+SQL with no `RecoveryInProgress()` guard and no NULL path. A committer may reasonably file this+against `97b10177`; the missing "unknown" signal is bd8d9c9's.++**Why it is not covered by a test.** Testing it needs a full streaming primary/standby pair plus the+`template0`-connectable `VACUUM FREEZE` dance to move the primary's `oldestOffset` off zero — that is+a `src/test/recovery`-style test, not something that fits in `test_slru`'s single-node harness or in+`007_multixact_conversion.pl`. It also cannot be asserted until the intended semantics are decided:+should the standby return NULL for these two columns, or should redo store `xlrec.oldestOffset`?+Writing a test first would bake in whichever answer the test author guessed.++**Test someone should write** (in `src/test/recovery/t/`, once the semantics are settled):++```+init primary (wal_level=replica), create t, produce N multixacts via concurrent FOR SHARE+pg_basebackup -R -> standby, start, wait for catchup+on primary: ALTER DATABASE template0 ALLOW_CONNECTIONS true; VACUUM FREEZE in all dbs; CHECKPOINT+wait_for_catchup+is(standby: SELECT num_members FROM pg_get_multixact_stats(),+ primary: same) # or: expect NULL on the standby+promote standby; re-run; expect it to match+```++The middle assertion is the whole test; it fails today with standby = `nextOffset`,+primary = `nextOffset - oldestOffset`.++---++### 3.4 `false-truncation-disabled-log` — a LOG that names the wrong subsystem and hides the real degradation++**Location:** `src/backend/access/transam/multixact.c:2469-2479`, in `SetOldestOffset()`.++```+2469 oldestOffsetKnown =+2470 find_multixact_start(oldestMultiXactId, &oldestOffset);+2471 +2472 if (oldestOffsetKnown)+2473 ereport(DEBUG1,+2474 (errmsg_internal("oldest MultiXactId member is at offset %" PRIu64,+2475 oldestOffset)));+2476 else+2477 ereport(LOG,+2478 (errmsg("MultiXact member truncation is disabled because oldest checkpointed MultiXact %u does not exist on disk",+2479 oldestMultiXactId)));+```++**Why the message is false.** `MultiXactState->oldestOffset` — the only thing this failure leaves+unset — is written at `multixact.c:2488` and `multixact.c:2788`, and read only through+`GetMultiXactInfo()` (`multixact.c:2555`). Its consumers are exactly two:+`MultiXactMemberFreezeThreshold()` (`multixact.c:2603`) and `pg_get_multixact_stats()`+(`multixactfuncs.c:128`). **No truncation path reads it.** `TruncateMultiXact()` calls+`find_multixact_start(newOldestMulti, &newOldestOffset)` for itself (`multixact.c:2723`), emits its+own distinct messages for its own failures (`multixact.c:2725-2730` "cannot truncate up to MultiXact+%u because it does not exist on disk, skipping truncation" and `multixact.c:2740-2747` "… because it+has invalid offset …"), and hands its own local `newOldestOffset` to `PerformMembersTruncation()`+(`multixact.c:2792`). Member truncation continues normally after this LOG fires.++The stale comment bd8d9c9 added directly above asserts the same nonexistent invariant+(`multixact.c:2457-2462`): "oldestOffset is initialized to zero at system startup, **which prevents+truncating members until a proper value is calculated**".++**What actually degrades.** A stale-or-zero `oldestOffset`, which:++* makes `MultiXactMemberFreezeThreshold()` compute `members = nextOffset - 0` — an *inflated* figure,+ so vacuum freezes **more** aggressively, not less (clamped to `freeze_max_age` 0 once the inflated+ count passes `MULTIXACT_MEMBER_HIGH_THRESHOLD`). This is the opposite of the "members will grow+ unbounded" reading the message invites;+* makes `pg_get_multixact_stats()` report inflated `num_members`/`members_size`.++Neither is mentioned. bd8d9c9 also deleted the paired "MultiXact member wraparound protections are+now enabled" LOG without a replacement, so nothing announces when the condition clears.++**Attribution.** `git show bd8d9c9 -- src/backend/access/transam/multixact.c` changes this line from+"MultiXact member **wraparound protections** are disabled because oldest checkpointed MultiXact %u+does not exist on disk" to the current text, and in the same commit deletes `offsetStopLimit` and+`oldestOffsetKnown` from `MultiXactStateData`. The old wording was accurate for `bd8d9c9^`, where+`oldestOffsetKnown` genuinely gated `offsetStopLimit`; the replacement asserts something that has+never been true post-commit. Only `ef6a95c` (translation catalogs) has touched the string since.++**Trigger (reproduced).** Any state where the `pg_multixact/offsets` page holding pg_control's+`oldestMulti` is absent. Reproduced without touching SLRU *content*: a cluster driven to truncate+away its low offsets segments (30000 multixacts via `test_slru`'s `test_create_multixact()`, then+`VACUUM FREEZE` in all three databases, which unlinked `offsets/0001` and+`members/000000000000000`), then stopped and restarted after+`pg_resetwal --multixact-ids 95535,65535` — a stale "oldest" value of the kind an operator gets by+following the pg_resetwal recovery recipe with a figure from an out-of-date source. On restart+`TrimMultiXact` → `SetMultiXactIdLimit` → `SetOldestOffset` → `find_multixact_start` →+`SimpleLruDoesPhysicalPageExist` false → the LOG fires. 40000 more multixacts plus another+`VACUUM FREEZE` then **deleted a members segment**, with the "truncation is disabled" message still+in force. `pg_get_multixact_stats()` reported inflated members throughout.++Scope honesty: normal operation does not reach this state. `TruncateMultiXact()` updates+`MultiXactState->oldestMultiXactId` inside the same critical section as the unlink, WAL-logs it and+holds `DELAY_CHKPT_START`; the redo path skips `SetOldestOffset()` entirely. Reaching it takes either+legacy 9.3/9.4-era corruption (which the surrounding comment at `multixact.c:2464-2467` still+anticipates) or an operator-supplied `pg_resetwal -m` whose second value is older than the smallest+surviving segment. Note that following `doc/src/sgml/ref/pg_resetwal.sgml`'s documented recipe+literally yields a value whose page *does* exist, so the documented procedure does not hit it. This+is a wrong-message defect on a recovery path, not an everyday hazard.++**User-visible consequence.** A DBA recovering a damaged cluster reads a server LOG stating that+multixact member truncation has been disabled — implying `pg_multixact/members` will grow without+bound until they intervene — when truncation is working fine, and is given no hint about the actual+consequence (an inflated members count driving over-aggressive freezing and wrong+`pg_get_multixact_stats()` output). The message also never retracts itself.++**Why it is not covered by a test.** The state requires staging a mismatch between pg_control's+`oldestMulti` and the surviving offsets segments — a `pg_resetwal --multixact-ids` with a+deliberately stale value, after a real truncation, on a cluster whose multixid counter was+pre-positioned near a segment boundary. That is ~7 setup steps including a `test_slru` module load and+a pgbench run, and the payload assertion is "a LOG line's text is wrong", which the project does not+normally assert. The genuinely testable half — that truncation still runs while the message claims it+is disabled — requires the same staging plus log scraping. The proportionate fix is to reword the+message and delete the false comment; a test would cost far more than the bug.++**Test someone should write, if one is wanted** (`src/test/modules/test_slru/t/`):++```+initdb; pg_resetwal --multixact-ids 0xFFFF,0xFFFF; hand-place a zeroed offsets/0001, rm offsets/0000+load test_slru; create ~30k multixacts via test_create_multixact()+make template0 connectable; VACUUM FREEZE all dbs # truncation unlinks offsets/0001+stop; pg_resetwal --multixact-ids <new nextMulti>,<stale oldest inside the unlinked segment>; start+expect the LOG line+create more multixacts; VACUUM FREEZE all dbs+assert a members segment WAS unlinked # i.e. the LOG's claim is false+assert pg_get_multixact_stats().num_members is inflated relative to reality+```++---++## 4. What the companion test does cover++For completeness, `src/test/modules/test_slru/t/003_multixact_offset.pl` (plus its `meson.build`+entry) and the additions to `src/bin/pg_upgrade/t/007_multixact_conversion.pl` cover rows 5-8 of the+summary table:++* **`resetwal-nextmxoff-zero`** — `GuessControlValues()` at `pg_resetwal.c:703` assigns+ `nextMultiOffset = 0`, and `case 'O'` (`pg_resetwal.c:297-307`) accepts 0 with no zero check,+ unlike `-o` (`:263-264`) and `-m` (`:290-293`). The resulting cluster cannot read back the first+ multixact it creates: `ERROR: MultiXact 1 has invalid offset`.+* **`pg-control-checkpoint-next-multi-offset-xid`** — `pg_controldata.c:133` (backend) funnels the+ now-`uint64` field through `TransactionIdGetDatum()`, and `pg_proc.dat:12374` still declares the+ column `xid` (`proallargtypes` has `xid` in that slot; `func-info.sgml:3460-3461` documents it as+ `xid`). Control file holds 4294967896, SQL reports 600.+* **`members-truncation-apparent-wraparound`** — `PerformMembersTruncation()`+ (`multixact.c:2642-2647`) passes `MXOffsetToMemberPage(newOldestOffset)` to `SimpleLruTruncate()`+ with no step-back, unlike `PerformOffsetsTruncation()` immediately below it+ (`multixact.c:2652-2664`), whose comment names exactly this hazard.+* **`pgupgrade-nxtmxoff-str2uint`** — `controldata.c:292` reads the field with `str2uint()`+ (`util.c:351-355`, returns `unsigned int`) into the `uint64 chkpnt_nxtmxoff`+ (`pg_upgrade.h:221`). Old 4294967896 → new 600.++---++## 5. Refuted candidates — do not re-chase these++* **"pg_upgrade's multixact conversion `pg_fatal`s on a missing offsets segment that both servers+ tolerate" (`src/bin/pg_upgrade/slru_io.c:112`)** — refuted 2/3. The code reading is right, but no+ normal-operation path produces the state: offsets segments are removed only by+ `TruncateMultiXact()`, which never deletes the segment containing `oldestMulti`. Reaching it needs+ a hand-set pg_control on an already-corrupt cluster, and the claimed consequences were overstated.++* **"Widening `moff` to 64 bits opened a 4-byte padding hole that ships uninitialized stack in every+ CREATE_ID WAL record" (`src/include/access/multixact.h:75`)** — refuted 2/3. The layout claim is+ accurate (`mid`@0, `moff`@8, `nmembers`@16, `members`@20 with `MultiXactOffset` = `uint64`), but no+ observable behavior follows: nothing reads the padding, WAL is not compared byte-wise across+ systems, and no user-visible symptom could be stated. Fails the "user-visible defect" bar.++* **"`SetOldestOffset()` installs a zero `oldestOffset`, defeating `102bdaa`'s guard and clamping+ vacuum's multixact freeze age cluster-wide" (`multixact.c:2485`)** — refuted 3/3. The code is+ really at `multixact.c:2484-2490` and the mechanical chain is right, but the value is only+ installed `if (oldestOffsetKnown)`, and the arithmetic that was claimed to clamp+ `freeze_max_age` bottoms out at `if (fraction >= 1.0) return 0;` only after+ `members > MULTIXACT_MEMBER_LOW_THRESHOLD`, which the described state does not reach. (The related,+ *real* problem in that function is the false LOG message, §3.4.)++---++## 6. Bottom line++Four confirmed, still-unfixed, user-observable defects attributable to bd8d9c9 have no test coverage:+two documentation claims falsified by the commit (§3.1, §3.2), one wrong-value report on hot standbys+(§3.3, with attribution partly to the later `97b10177`), and one server LOG message that names the+wrong subsystem and conceals the actual degradation (§3.4). None of the four is testable at+proportionate cost on ordinary hardware without first adding a threshold override+(§3.2), building a recovery-suite test and settling the intended standby semantics (§3.3), or+accepting a seven-step SLRU staging fixture to assert a message's text (§3.4). §3.1 is a one-line+SGML fix that no test could ever have caught.diff --git a/src/bin/pg_upgrade/t/007_multixact_conversion.pl b/src/bin/pg_upgrade/t/007_multixact_conversion.plindex 867a062..85742b9 100644--- a/src/bin/pg_upgrade/t/007_multixact_conversion.pl+++ b/src/bin/pg_upgrade/t/007_multixact_conversion.pl@@ -440,4 +440,72 @@ SKIP:
upgrade_and_compare($tag, $old, $new);
}
+# High-offset scenario: the old cluster's 64-bit NextMultiOffset has+# passed 2^32. No SLRU conversion is involved here, so both clusters+# use the current version regardless of $ENV{oldinstall}: the value is+# copied verbatim from the old cluster's control file to the new one,+# which is exactly the code path we want to check.+#+# get_control_data() in controldata.c reads "Latest checkpoint's+# NextMultiOffset" with str2uint(), which returns unsigned int, even+# though chkpnt_nxtmxoff became uint64 when offsets were widened. The+# new cluster therefore gets NextMultiOffset modulo 2^32, i.e. an offset+# pointing back into members space that is still in use.+{+ my $tag = 'highoffset';+ my $old = PostgreSQL::Test::Cluster->new("${tag}_oldnode");+ my $new = PostgreSQL::Test::Cluster->new("${tag}_newnode");++ $old->init;+ $new->init;++ # MULTIXACT_MEMBERS_PER_PAGE, as computed in+ # reset_mxid_mxoffset_pre_v19() above.+ my ($out, undef) =+ run_command([ 'pg_resetwal', '--dry-run', $old->data_dir ]);+ $out =~ /^Database block size: *(\d+)$/m or die;+ my $blcksz = $1;+ my $multixact_members_per_page = int($blcksz / 20) * 4;++ # Smallest members-page-aligned offset at or above 2^32. Page+ # alignment matters because pg_upgrade starts the old cluster to+ # dump its schema: with an aligned nextOffset, TrimMultiXact() does+ # not read the members page, so the old cluster starts without a+ # multi-gigabyte members segment on disk. Alignment also puts+ # ($target mod 2^32) inside members page 0, which the copied segment+ # does contain, so pg_upgrade itself still succeeds and the test+ # fails on a clean value mismatch rather than on command_ok().+ my $two_to_32 = Math::BigInt->new(2)**32;+ my $target =+ ($two_to_32 + $multixact_members_per_page - 1)+ ->bdiv($multixact_members_per_page)+ ->bmul($multixact_members_per_page);++ command_ok(+ [+ 'pg_resetwal',+ '--multixact-offset' => "$target",+ $old->data_dir+ ],+ 'set old cluster NextMultiOffset past 2^32');++ command_ok(+ [+ 'pg_upgrade', '--no-sync',+ '--old-datadir' => $old->data_dir,+ '--new-datadir' => $new->data_dir,+ '--old-bindir' => $old->config_data('--bindir'),+ '--new-bindir' => $new->config_data('--bindir'),+ '--socketdir' => $new->host,+ '--old-port' => $old->port,+ '--new-port' => $new->port,+ ],+ 'run of pg_upgrade with a NextMultiOffset above 2^32');++ my (undef, undef, $old_mxoff) = read_multixid_fields($old);+ my (undef, undef, $new_mxoff) = read_multixid_fields($new);+ is($new_mxoff, $old_mxoff,+ 'NextMultiOffset is preserved across pg_upgrade');+}+
done_testing();
diff --git a/src/test/modules/test_slru/meson.build b/src/test/modules/test_slru/meson.buildindex 00f3ee3..f4536ea 100644--- a/src/test/modules/test_slru/meson.build+++ b/src/test/modules/test_slru/meson.build@@ -39,7 +39,8 @@ tests += {
},
'tests': [
't/001_multixact.pl',
- 't/002_multixact_wraparound.pl'+ 't/002_multixact_wraparound.pl',+ 't/003_multixact_offset.pl'
],
},
}
diff --git a/src/test/modules/test_slru/t/003_multixact_offset.pl b/src/test/modules/test_slru/t/003_multixact_offset.plnew file mode 100644index 0000000..c6771dd--- /dev/null+++ b/src/test/modules/test_slru/t/003_multixact_offset.pl@@ -0,0 +1,254 @@+# Copyright (c) 2026, PostgreSQL Global Development Group++# Tests for the 64-bit MultiXactOffset counter (nextMultiOffset), which+# commit bd8d9c9bdf widened from 32 bits. Three independent scenarios,+# each with its own cluster:+#+# 1. pg_resetwal must never leave nextMultiOffset at 0. Now that+# offsets are 64-bit and no longer wrap, 0 is reserved as the+# "invalid offset" marker: GetMultiXactIdMembers() in+# access/transam/multixact.c errors out with "MultiXact %u has+# invalid offset" when it reads a zero offset. But+# GuessControlValues() in bin/pg_resetwal/pg_resetwal.c still assigns+# nextMultiOffset = 0, and the -O option still accepts 0, unlike -m+# which rejects it. A cluster left in that state cannot read back+# the first multixact it creates.+#+# 2. pg_control_checkpoint() must report the full 64-bit+# NextMultiOffset. pg_control_checkpoint() in+# backend/utils/misc/pg_controldata.c still runs the field through+# TransactionIdGetDatum(), and pg_proc.dat still declares the+# next_multi_offset output column as xid, so the value is silently+# reported modulo 2^32.+#+# 3. Members truncation must not report a bogus "apparent wraparound"+# and must actually reclaim obsolete segments.+# PerformMembersTruncation() in access/transam/multixact.c passes+# MXOffsetToMemberPage(newOldestOffset) to SimpleLruTruncate()+# without stepping back one member, unlike PerformOffsetsTruncation()+# directly below it. When nextOffset happens to sit exactly on a+# members page boundary, the cutoff page is one past the last page+# ever written, so SimpleLruTruncate()'s endpoint-page safety check+# fires and nothing is removed. Before the widening,+# PerformMembersTruncation() unlinked the segments itself and had no+# such check.++use strict;+use warnings FATAL => 'all';++use PostgreSQL::Test::Cluster;+use PostgreSQL::Test::Utils;++use Test::More;++# Read "Latest checkpoint's NextMultiOffset" out of pg_controldata.+sub next_multi_offset+{+ my $node = shift;++ my $out = (run_command([ 'pg_controldata', $node->data_dir ]))[0];+ $out =~ /^Latest checkpoint's NextMultiOffset: *(\d+)$/m+ or die "could not find NextMultiOffset in: $out";+ return $1;+}++# Extract the SLRU geometry from pg_resetwal --dry-run output, like+# 002_multixact_wraparound.pl does.+sub slru_geometry+{+ my $node = shift;++ my $out =+ (run_command([ 'pg_resetwal', '--dry-run', $node->data_dir ]))[0];+ $out =~ /^Database block size: *(\d+)$/m or die;+ my $blcksz = $1;+ $out =~ /^Pages per SLRU segment: *(\d+)$/m or die;+ my $pages_per_seg = $1;++ # MULTIXACT_MEMBERS_PER_PAGE: a member group is+ # 4 * sizeof(TransactionId) + 4 == 20 bytes and holds 4 members.+ # See MULTIXACT_MEMBERGROUP_SIZE in access/multixact_internal.h.+ my $members_per_page = int($blcksz / 20) * 4;++ return ($blcksz, $pages_per_seg, $members_per_page);+}++#+# 1. pg_resetwal must not produce nextMultiOffset == 0.+#+{+ my $node = PostgreSQL::Test::Cluster->new('mxoff_zero');+ $node->init;+ $node->append_conf('postgresql.conf',+ "shared_preload_libraries = 'test_slru'\n" . "autovacuum = off\n");++ # Use run_command(), not command_ok(): a fix for this may well make+ # pg_resetwal reject -O 0 outright, and this test should pass either+ # way. All that matters is what ends up in the control file.+ run_command(+ [ 'pg_resetwal', '--multixact-offset' => 0, $node->data_dir ]);++ cmp_ok(next_multi_offset($node), '>=', 1,+ 'pg_resetwal -O 0 does not store the reserved offset 0');++ # Whatever the control file says, the cluster must be able to create+ # and read back multixacts.+ $node->start;+ $node->safe_psql('postgres', 'CREATE EXTENSION test_slru');++ my $mx1 = $node->safe_psql('postgres', 'SELECT test_create_multixact()');+ my ($rc, undef, $err) =+ $node->psql('postgres', "SELECT test_read_multixact('$mx1')");+ is($rc, 0, 'the first multixact created after pg_resetwal is readable')+ or diag("multixact $mx1: $err");++ # Negative control: a later multixact must be fine regardless, so a+ # failure above is really about the first one and not about a+ # generally broken cluster.+ my $mx2 = $node->safe_psql('postgres', 'SELECT test_create_multixact()');+ ($rc, undef, $err) =+ $node->psql('postgres', "SELECT test_read_multixact('$mx2')");+ is($rc, 0, 'a subsequent multixact is readable')+ or diag("multixact $mx2: $err");++ $node->stop;++ # The producer of a zero offset that needs no bad user input is+ # GuessControlValues(), used when pg_control cannot be read at all.+ # That is the documented "pg_resetwal -f" last-resort recovery. No+ # server start here: a guessed control file also rewinds nextXid, so+ # the cluster is not usable without more hand-picked values.+ my $pg_control = $node->data_dir . '/global/pg_control';+ my $size = -s $pg_control;+ open my $fh, '>', $pg_control or die "could not open pg_control: $!";+ binmode $fh;+ print $fh pack("x[$size]");+ close $fh;++ command_ok([ 'pg_resetwal', '--force', $node->data_dir ],+ 'pg_resetwal -f on an unreadable pg_control');+ cmp_ok(next_multi_offset($node), '>=', 1,+ 'pg_resetwal -f does not guess the reserved offset 0');+}++#+# 2. pg_control_checkpoint() must not truncate NextMultiOffset to 32 bits.+#+{+ my $node = PostgreSQL::Test::Cluster->new('mxoff_64bit');+ $node->init;+ $node->append_conf('postgresql.conf', "autovacuum = off\n");++ my (undef, undef, $members_per_page) = slru_geometry($node);++ # Smallest members-page-aligned offset at or above 2^32. Page+ # alignment matters: TrimMultiXact() then skips reading the members+ # page, so the cluster starts without a huge segment on disk.+ my $target =+ int((4294967296 + $members_per_page - 1) / $members_per_page) *+ $members_per_page;++ command_ok(+ [ 'pg_resetwal', '--multixact-offset' => $target, $node->data_dir ],+ 'set NextMultiOffset past 2^32');++ # pg_resetwal's own -O parser is 64-bit; check that the control file+ # really holds the large value before blaming SQL for the mismatch.+ my $from_controldata = next_multi_offset($node);+ is($from_controldata, "$target", 'pg_controldata reports the full value');++ $node->start;++ # Compare against the string pg_controldata printed, and cast to+ # text: a complete fix needs Int64GetDatum() *and* a change of the+ # column's declared type from xid to int8, and this comparison stays+ # valid across that change.+ is( $node->safe_psql(+ 'postgres',+ 'SELECT next_multi_offset::text FROM pg_control_checkpoint()'),+ $from_controldata,+ 'pg_control_checkpoint() reports the full 64-bit NextMultiOffset');++ $node->stop;+}++#+# 3. Members truncation must not claim "apparent wraparound".+#+{+ my $node = PostgreSQL::Test::Cluster->new('mxoff_members');+ $node->init;+ $node->append_conf('postgresql.conf',+ "shared_preload_libraries = 'test_slru'\n" . "autovacuum = off\n");++ my ($blcksz, $pages_per_seg, $members_per_page) = slru_geometry($node);+ my $members_per_seg = $members_per_page * $pages_per_seg;++ # Aim to land nextOffset exactly on a members segment boundary, two+ # segments in. test_create_multixact() creates exactly one multixact+ # with exactly 2 members, so start 2 short of the boundary.+ my $target = 2 * $members_per_seg;++ command_ok(+ [+ 'pg_resetwal',+ '--multixact-offset' => $target - 2,+ $node->data_dir+ ],+ 'set NextMultiOffset just below a members segment boundary');++ # pg_resetwal only updates the control file, so create the members+ # segments that the pre-seeded offset lives in, filled with zeros, as+ # 002_multixact_wraparound.pl does for the offsets SLRU. (In the+ # field the server would have written them itself.) Segment 0 must+ # exist too, or there would be nothing for the truncation to reclaim.+ # The members SLRU uses long segment names.+ my $bytes_per_seg = $pages_per_seg * $blcksz;+ foreach my $segno (0, 1)+ {+ my $path =+ sprintf('%s/pg_multixact/members/%015X', $node->data_dir, $segno);+ open my $fh, '>', $path or die "could not open \"$path\": $!";+ binmode $fh;+ syswrite($fh, "\0" x $bytes_per_seg) == $bytes_per_seg+ or die "could not write to \"$path\": $!";+ close $fh;+ }++ $node->start;+ $node->safe_psql('postgres', 'CREATE EXTENSION test_slru');++ # One multixact of 2 members: nextOffset becomes exactly $target,+ # i.e. exactly on a members segment (hence page) boundary.+ $node->safe_psql('postgres', 'SELECT test_create_multixact()');++ # Do not restart from here on. StartupMultiXact()/TrimMultiXact()+ # would re-derive the members SLRU's latest_page_number from+ # nextOffset and mask the problem.++ my $log_offset = -s $node->logfile;++ # Drive every datminmxid up to nextMulti, so that TruncateMultiXact()+ # takes the "there are NO MultiXacts" branch and uses nextOffset as+ # the members cutoff.+ $node->safe_psql('postgres',+ 'ALTER DATABASE template0 ALLOW_CONNECTIONS true');+ foreach my $db ('postgres', 'template1', 'template0')+ {+ $node->safe_psql($db, 'VACUUM FREEZE');+ }++ ok( !$node->log_contains(+ qr{pg_multixact/members": apparent wraparound}, $log_offset),+ 'members truncation does not report a bogus wraparound');++ my $seg0 = $node->data_dir . '/pg_multixact/members/000000000000000';+ my @segs = map { (split m{/})[-1] }+ glob($node->data_dir . '/pg_multixact/members/*');+ ok(!-f $seg0, 'obsolete members segment is reclaimed')+ or diag('members segments left behind: ' . join(' ', sort @segs));++ $node->stop;+}++done_testing();Attachments:
[text/plain] bd8d9c9-MultiXactOffset-tests-v0.patch (50.8K, ../20260827231757.78.noahmisch@microsoft.com/2-bd8d9c9-MultiXactOffset-tests-v0.patch)
download | inline diff:
commit bdff567 (cqla/mxactoffset64-defect-tests)
Author: Noah Misch <noah@leadboat.com>
AuthorDate: Tue Jul 21 17:52:05 2026 +0000
Commit: Noah Misch <noah@leadboat.com>
CommitDate: Tue Jul 21 17:52:05 2026 +0000
Test user-visible defects left by the 64-bit MultiXactOffset widening
Commit bd8d9c9 widened MultiXactOffset from 32 to 64 bits. Four of the
user-visible defects it left behind are still present in master; these
tests demonstrate all four. Every test fails on master today and passes
once the corresponding product-code defect is fixed.
src/test/modules/test_slru/t/003_multixact_offset.pl covers three:
* bd8d9c9 made offset 0 the reserved "invalid offset" marker, changing
BootStrapXLOG() to start nextMultiOffset at 1 instead of 0, but left
pg_resetwal's GuessControlValues() assigning 0 and left -O accepting 0
with no zero check, unlike -m. A cluster reset that way cannot read
back the first multixact it creates: "ERROR: MultiXact 1 has invalid
offset".
* pg_control_checkpoint() still runs the widened field through
TransactionIdGetDatum(), and pg_proc.dat still declares the
next_multi_offset output column as xid, so SQL reports the value
modulo 2^32 while pg_controldata prints it in full. Observed: control
file 4294967896, SQL 600.
* PerformMembersTruncation() passes MXOffsetToMemberPage(newOldestOffset)
to SimpleLruTruncate() with no step-back, unlike PerformOffsetsTruncation()
directly below it, whose comment names exactly this hazard. When
nextOffset lands on a members page boundary the cutoff page is one past
the last page ever written, so SimpleLruTruncate()'s endpoint check
fires: the server logs 'could not truncate directory
"pg_multixact/members": apparent wraparound' -- impossible for a
counter that cannot wrap -- and no obsolete segment is reclaimed.
src/bin/pg_upgrade/t/007_multixact_conversion.pl gains a fourth: while
chkpnt_nxtmxoff became uint64, get_control_data() still reads
NextMultiOffset with str2uint(), which returns unsigned int. Upgrading a
cluster whose offset has passed 2^32 gives the new cluster an offset
modulo 2^32, pointing back into members space that is still in use. No
SLRU conversion is involved, so this case runs regardless of $oldinstall.
bd8d9c9-untested-defects.md describes four further confirmed defects that
are not covered here, with the reason each is impractical to test: two
stale documentation claims in maintenance.sgml, pg_get_multixact_stats()
reporting lifetime rather than retained members on a hot standby, and a
server LOG message that reports the wrong subsystem as disabled. It also
records three candidates that adversarial verification refuted, so they
are not re-chased.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WxWnwpyK9G3DohkEZJxsLU
---
bd8d9c9-untested-defects.md | 529 +++++++++++++++++++++
src/bin/pg_upgrade/t/007_multixact_conversion.pl | 68 +++
src/test/modules/test_slru/meson.build | 3 +-
.../modules/test_slru/t/003_multixact_offset.pl | 254 ++++++++++
4 files changed, 853 insertions(+), 1 deletion(-)
diff --git a/bd8d9c9-untested-defects.md b/bd8d9c9-untested-defects.mdnew file mode 100644index 0000000..0740d89--- /dev/null+++ b/bd8d9c9-untested-defects.md@@ -0,0 +1,529 @@+# Defects in bd8d9c9 ("Widen MultiXactOffset to 64 bits") still present in master — the ones the companion test does not cover++Target commit: `bd8d9c9bdfa0c2168bb37edca6fa88168cacbbaa`, Heikki Linnakangas, 2025-12-09+(author Maxim Orlov), "Widen MultiXactOffset to 64 bits".+Tree audited: `/home/nm/src/pg/postgresql` at `c90c967` — 2028 commits after bd8d9c9.+Every line quoted below was re-read from that working tree while writing this report.++---++## 1. Scope and method++### What was audited++bd8d9c9 as it stands **today in master**, not as it was committed. The commit widened+`MultiXactOffset` from `uint32` to `uint64` (`src/include/c.h:807`), deleted the members-space+wraparound apparatus (`offsetStopLimit`, `MultiXactOffsetWouldWrap`, the emergency+"multixact members limit exceeded" ERROR — `grep -rn 'offsetStopLimit|MultiXactOffsetWouldWrap' src/`+now returns nothing), kept the members-driven clamp on vacuum's multixid freeze cutoff, and added+a pg_upgrade path that rewrites v18 32-bit SLRU files into the new format.++Areas covered: `multixact.c` (allocation, startup/trim, truncation, freeze-threshold),+`multixact.h`/`multixact_internal.h` (on-disk and WAL layout), pg_upgrade's conversion and control+data handling, pg_resetwal, pg_controldata (both the frontend program and the SQL function),+vacuum/autovacuum freeze scheduling, `pg_get_multixact_stats()`, and the SGML documentation that+describes any of the above.++### How++Fan-out static analysis: 7 areas x 4 independent reading lenses, each producing candidate findings+with a mandatory verbatim source quote and a concrete trigger. Every candidate then went through+adversarial 3-vote verification: three independent attempts to **refute** it, each required to read+master's source itself and to check `git log bd8d9c9..master -- <path>` for a follow-up fix. Anything+that lost 2 or more votes was dropped. Survivors were then handed to a reproduction pass that built+real clusters from a prebuilt worktree at the same commit (`/home/nm/src/pg/pg-master-mxact64`,+`--enable-cassert`) and tried to observe the symptom end to end.++### Standard applied++A finding counts only if **all** of the following hold:++1. It is present in master's working tree today (not merely in the bd8d9c9 diff — ~50 follow-up+ commits, e.g. `ac94ce8`, `d4b7bde`, `ecb553a`, `87a350e`, `6aa26be`, `516310e`, `102bdaa`,+ `ccae90a`, already repaired much of this code).+2. It is attributable to bd8d9c9: introduced by it, a regression versus pre-bd8d9c9 behavior, or a+ hole in code bd8d9c9 added. Where attribution is arguable, this report says so explicitly.+3. It produces something a user or DBA can **observe**: a wrong number, a false message, an ERROR+ where success is required, data left on disk that should be gone, documentation that contradicts+ the code.+4. It is reachable without hand-corrupting files into a state the code otherwise prevents.++"Risk areas", style, comment quality and unreachable overflow were rejected by construction. A prior+audit of this commit produced 26 "CRITICAL risk areas" that were all vapor; the refutation pass here+killed three more (section 5).++---++## 2. Summary table++| # | id | location | symptom | covered by test? |+|---|----|----------|---------|------------------|+| 1 | `maintenance-sgml-32bit-members` | `doc/src/sgml/maintenance.sgml:782` | Manual still says the pg_multixact members storage area "uses a 32-bit counter"; `MultiXactOffset` is now `uint64` and members cannot wrap | **No** — doc-only; nothing to assert at runtime |+| 2 | `no-launcher-signal-for-member-space` | `doc/src/sgml/maintenance.sgml:817-823` (behavior at `src/backend/access/transam/multixact.c:1009,2161-2183`) | Docs promise member-space-driven aggressive vacuums "even if autovacuum is nominally disabled"; since bd8d9c9 no code path starts the launcher on member growth | **No** — needs >2e9 member entries (~10 GB) to observe at runtime |+| 3 | `standby-oldestoffset-zero-stats` | `src/backend/access/transam/multixact.c:2555`, `src/backend/utils/adt/multixactfuncs.c:128-130` | On a hot standby `pg_get_multixact_stats()` reports `num_members`/`members_size` as lifetime allocation, not retained members; disagrees with its own primary | **No** — needs a primary/standby pair; attribution partly to a later commit (see §3.3) |+| 4 | `false-truncation-disabled-log` | `src/backend/access/transam/multixact.c:2477-2479` | LOG says "MultiXact member truncation is disabled …"; truncation is in fact unaffected, and the real damage (stale `oldestOffset`) is never mentioned | **No** — needs a hand-staged offsets-segment/oldestMulti mismatch |+| 5 | `resetwal-nextmxoff-zero` | `src/bin/pg_resetwal/pg_resetwal.c:703`, and `-O` with no zero check at 297-307 | `pg_resetwal -f` (or `-O 0`) leaves `nextMultiOffset = 0`, the reserved "invalid offset"; first multixact created afterwards is permanently unreadable: `ERROR: MultiXact 1 has invalid offset` | Yes — `src/test/modules/test_slru/t/003_multixact_offset.pl` |+| 6 | `pg-control-checkpoint-next-multi-offset-xid` | `src/backend/utils/misc/pg_controldata.c:133`, `src/include/catalog/pg_proc.dat:12374` | `pg_control_checkpoint().next_multi_offset` reports the value mod 2^32 (column still typed `xid`), disagreeing with `pg_controldata` | Yes — `003_multixact_offset.pl` |+| 7 | `members-truncation-apparent-wraparound` | `src/backend/access/transam/multixact.c:2642-2647` | When `nextOffset` lands on a members page boundary, truncation logs `could not truncate directory "pg_multixact/members": apparent wraparound` (impossible for a 64-bit counter) and reclaims nothing | Yes — `003_multixact_offset.pl` |+| 8 | `pgupgrade-nxtmxoff-str2uint` | `src/bin/pg_upgrade/controldata.c:292` | pg_upgrade parses the widened `NextMultiOffset` with 32-bit `str2uint()`; a v19+ source past 2^32 yields a new cluster with offset mod 2^32 — either fails to start past the "must re-initdb" point, or silently overwrites live members | Yes — `src/bin/pg_upgrade/t/007_multixact_conversion.pl` |++Rows 5-8 are documented in the committed test (`003_multixact_offset.pl`, `meson.build`, and the+additions to `007_multixact_conversion.pl`) and are not re-litigated here. Rows 1-4 have no test;+they are detailed below.++---++## 3. Per-defect detail — the untested findings++### 3.1 `maintenance-sgml-32bit-members` — the manual still describes a 32-bit members counter++**Location:** `doc/src/sgml/maintenance.sgml:778-786`, section "Multixacts and Wraparound".++```+778 Like transaction IDs, multixact IDs are implemented as a+779 32-bit counter and corresponding storage, all of which requires+780 careful aging management, storage cleanup, and wraparound handling.+781 There is a separate storage area which holds the list of members in+782 each multixact, which also uses a 32-bit counter and which must also+783 be managed. The system function+784 <function>pg_get_multixact_members()</function> described in+785 <xref linkend="functions-pg-snapshot"/> can be used to examine the+786 transaction IDs associated with a multixact ID.+```++**What the code says.** The members storage area is addressed by `MultiXactOffset`, and+`src/include/c.h:807` is now `typedef uint64 MultiXactOffset;`. `multixact.c:1106-1112`:++> Offsets are 64-bit integers and will never wrap around. Firstly, it would take an unrealistic+> amount of time and resources to consume 2^64 offsets. Secondly, multixid creation is WAL-logged,+> so you would run out of LSNs before reaching offset wraparound.++and `multixact.c:2826-2835` reduces `MultiXactMemberPagePrecedes()` to `return page1 < page2;`+with the comment "members never wrap around".++**What is and is not wrong.** Precisely one clause is false: "which also uses a 32-bit counter"+(line 782). Two neighbouring claims are still correct and must not be reported as bugs:++* Line 778-780, about multixact **IDs** being a 32-bit counter needing wraparound handling, is still+ true — `src/include/c.h:805` is `typedef TransactionId MultiXactId;` and multixids still wrap.+* "and which must also be managed" is still true — members space still drives SLRU truncation and the+ 2-billion / 4-billion-entry aggressive-vacuum thresholds documented at lines 814-827.++**Trigger.** Read the v19 manual: "Routine Database Maintenance Tasks" → "Preventing Transaction ID+Wraparound Failures" → "Multixacts and Wraparound", first paragraph.++**User-visible consequence.** A DBA planning capacity concludes there is a hard ~2^32-member / ~20 GB+ceiling on `pg_multixact/members` beyond which the cluster will refuse to assign multixids — the+pre-v19 behavior. There is no such ceiling in v19; the 4-billion figure at line 824 is only a vacuum+aggressiveness threshold. The stated remedy (watch for members wraparound) is for a hazard that no+longer exists.++**Verification evidence.**+* `git show --stat bd8d9c9 -- doc/` shows the commit touched exactly one SGML file,+ `doc/src/sgml/ref/pg_resetwal.sgml` (updating the `-m` multiplier from 65536 to 32768 for the now+ 8-byte offsets). `maintenance.sgml` was not touched.+* `git blame -L 776,786 doc/src/sgml/maintenance.sgml` attributes lines 781-782 to `53bb309d`+ ("Teach autovacuum about multixact member wraparound", 2014), where the sentence was correct.+ Nothing in `bd8d9c9..master` has changed it.+* Interesting corroboration that this is an oversight, not a deliberate retention: the *sibling*+ paragraph was updated after bd8d9c9. `git blame -L 814,827` attributes lines 817-827 to `97b10177`+ (2025-12-30). `git show bd8d9c9^:doc/src/sgml/maintenance.sgml` shows the pre-commit text there+ read "The members storage area can grow up to about 20GB before reaching wraparound"; that+ sentence is gone in master. So someone already scrubbed one stale wraparound reference from this+ section and missed line 782.++**Why it is not covered by a test.** It is a documentation defect: there is no runtime assertion that+can fail. PostgreSQL has no doc/code consistency harness that could catch a prose claim about an+integer width. The fix is a one-line SGML edit, not a test.++**Test someone should write:** none. Fix the sentence, e.g. "There is a separate storage area which+holds the list of members in each multixact; it does not wrap around, but its disk usage must still+be managed." A reviewer should also re-read the whole sect3 for other pre-64-bit residue while there.++---++### 3.2 `no-launcher-signal-for-member-space` — docs promise member-driven vacuums with autovacuum off; the code no longer delivers them++**Location of the false claim:** `doc/src/sgml/maintenance.sgml:814-827`.++```+814 <para>+815 As a safety device, an aggressive vacuum scan will+816 occur for any table whose multixact-age is greater than <xref+817 linkend="guc-autovacuum-multixact-freeze-max-age"/>. Also, if the number+818 of multixact member entries created exceeds approximately 2 billion+819 entries (occupying roughly 10GB in the+820 <literal>pg_multixact/members</literal> directory), aggressive vacuum+821 scans will occur more often for all tables, starting with those that+822 have the oldest multixact-age. Both of these kinds of aggressive+823 scans will occur even if autovacuum is nominally disabled. At approximately+824 4 billion entries (occupying roughly 20GB in the+825 <literal>pg_multixact/members</literal> directory), even more aggressive+826 vacuum scans are triggered to reclaim member storage space.+827 </para>+```++The word **"Both"** on line 822 is now false for the second kind of scan.++**What the code does.** `grep -rn PMSIGNAL_START_AUTOVAC_LAUNCHER src/backend/` returns exactly six+sites; the three in multixact.c are:++* `multixact.c:1034` and `multixact.c:1059`, both inside+ `if (!MultiXactIdPrecedes(result, MultiXactState->multiVacLimit))` at line 1009 — keyed on+ multixact **age** (`multiVacLimit`, `multiStopLimit`, `result % 65536`);+* `multixact.c:2183`, `if (MultiXactIdPrecedes(multiVacLimit, curMulti) && IsUnderPostmaster)` —+ also age.++None of them reads `nextOffset` or `oldestOffset`. bd8d9c9 removed the two that did: the+`MULTIXACT_MEMBER_SAFE_THRESHOLD` arm of `GetNewMultiXactId()` and the `needs_offset_vacuum` term in+`SetMultiXactIdLimit()` (verifiable in `git show bd8d9c9 -- src/backend/access/transam/multixact.c`).++**This part of the change was deliberate**, and the reasoning is still in the tree at+`multixact.c:2166-2172`:++```+2166 /*+2167 * Offsets are 64-bits wide and never wrap around, so we don't need to+2168 * consider them for emergency autovacuum purposes. But now that we're in+2169 * a consistent state, determine MultiXactState->oldestOffset. It will be+2170 * used to adjust the freezing cutoff, to keep the offsets disk usage in+2171 * check.+2172 */+2173 SetOldestOffset();+```++So the finding is **not** "the trigger was accidentally dropped from multixact.c". It is that the+documentation still asserts the deleted behavior, and — pointedly — the paragraph was *rewritten+after* bd8d9c9 by `97b10177` (2025-12-30), which introduced the new "2 billion / 4 billion entries"+wording and carried the "even if autovacuum is nominally disabled" clause forward unexamined. Before+bd8d9c9 the sentence was true.++**What still works, so the report is not overstated.** `MultiXactMemberFreezeThreshold()`+(`multixact.c:2589-2636`, using `MULTIXACT_MEMBER_LOW_THRESHOLD` = 2000000000 and+`MULTIXACT_MEMBER_HIGH_THRESHOLD` = 4000000000 at `multixact.c:99-100`) is alive and consulted by:++* manual `VACUUM` — `src/backend/commands/vacuum.c:1159`;+* `do_start_worker()` — `src/backend/postmaster/autovacuum.c:1182`+ (`multiForceLimit = recentMulti - MultiXactMemberFreezeThreshold();`);+* `relation_needs_vacanalyze()`, with `force_vacuum` bypassing `av_enabled`+ (`autovacuum.c:3246`, `if (force_vacuum) *dovacuum = true;`).++So with `autovacuum = on`, or for a manual VACUUM, member-pressure clamping behaves exactly as+documented. The regression is confined to `autovacuum = off`: nothing then starts the launcher on+member growth. The launcher's emergency path is `autovacuum.c:593-601`:++```+593 if (!AutoVacuumingActive())+594 {+595 if (!ShutdownRequestPending)+596 do_start_worker();+597 proc_exit(0); /* done */+598 }+```++— reachable only when the postmaster sets `start_autovac_launcher` from+`PMSIGNAL_START_AUTOVAC_LAUNCHER` (`postmaster.c:3832`). No member-space signal, no launcher, no+clamp applied.++**Trigger.** `autovacuum = off`, `autovacuum_multixact_freeze_max_age` at its 400,000,000 default.+Run a workload that creates multixacts with many members each but consumes few multixids and few+XIDs — e.g. transactions that each take `FOR KEY SHARE` on tens of thousands of rows that are already+share-locked — until `nextOffset - oldestOffset` exceeds 2,000,000,000. At 200 members/multixact that+is ~10 million multixacts, far under the 400M multixact age that trips the surviving age-based+trigger.++**User-visible consequence.** `pg_multixact/members` grows past the 10 GB and 20 GB figures the manual+cites without any autovacuum being launched on that account. It is not literally unbounded — growth+continues until multixact age crosses `autovacuum_multixact_freeze_max_age`, or until an+XID-wraparound emergency launcher fires from `varsup.c:136` — but that ceiling is enormously higher+than what the docs describe: at ~200 members/multixact, roughly 80 billion member entries+(hundreds of GB) versus v18's ~2^31-member relief point. No wrong results, no crash; a disk-space+surprise plus a false promise in the manual.++**Ancillary (comment-only, does not qualify on its own):** the block comment bd8d9c9 left at+`multixact.c:1000-1002` still says "If we're past multiVacLimit **or the safe threshold for member+storage space, or we don't know what the safe threshold for member storage is**, start trying to+force autovacuum cycles" — describing code the same commit deleted.++**Why it is not covered by a test.** Reaching `MULTIXACT_MEMBER_LOW_THRESHOLD` requires materialising+two billion real member entries (~10 GB of `pg_multixact/members`) — hours of runtime and 10+ GB of+disk on a machine that has 2 cores. There is no debug hook to lower the threshold: 2000000000 is a+compile-time `#define`, not a GUC, and no injection point exists in `MultiXactMemberFreezeThreshold()`+or in the launcher path. `pg_resetwal -O` can jump `nextOffset` forward cheaply, but that does not+help: the missing signal is in `GetNewMultiXactId()`/`SetMultiXactIdLimit()`, which would have to be+*reached* with the launcher stopped, and there is nothing left there to observe — the assertion would+be about a signal that is simply never sent.++**Test someone with the resources should write.** Two tiers:++1. *Cheap and honest (recommended):* fix the documentation and add nothing. The behavior change is+ intentional; only the manual is wrong.+2. *If the behavior is deemed a regression to fix:* add a `test_slru` TAP test that (a) sets+ `autovacuum = off`, (b) uses a debug-build-only GUC or injection point that lowers+ `MULTIXACT_MEMBER_LOW_THRESHOLD` to something like 100000, (c) creates enough members via+ `test_create_multixact()` to cross it, and (d) asserts from the postmaster log that an autovacuum+ worker started, plus that `pg_multixact/members` shrank. Introducing the threshold override is the+ real work; without it the test cannot exist at reasonable cost.++---++### 3.3 `standby-oldestoffset-zero-stats` — `pg_get_multixact_stats()` on a hot standby reports lifetime members, not retained members++**Location:** `src/backend/access/transam/multixact.c:2545-2559` and its SQL consumer+`src/backend/utils/adt/multixactfuncs.c:127-130`.++```+2545 void+2546 GetMultiXactInfo(uint32 *multixacts, MultiXactOffset *nextOffset,+2547 MultiXactId *oldestMultiXactId, MultiXactOffset *oldestOffset)+2548 {+2549 MultiXactId nextMultiXactId;+2550 +2551 LWLockAcquire(MultiXactGenLock, LW_SHARED);+2552 *nextOffset = MultiXactState->nextOffset;+2553 *oldestMultiXactId = MultiXactState->oldestMultiXactId;+2554 nextMultiXactId = MultiXactState->nextMXact;+2555 *oldestOffset = MultiXactState->oldestOffset;+2556 LWLockRelease(MultiXactGenLock);+```++```+127 GetMultiXactInfo(&multixacts, &nextOffset, &oldestMultiXactId, &oldestOffset);+128 members = nextOffset - oldestOffset;+129 membersBytes = MultiXactOffsetStorageSize(nextOffset, oldestOffset);+```++**Mechanism.** `MultiXactState->oldestOffset` is assigned in exactly two places:+`SetOldestOffset()` at `multixact.c:2488`, and `TruncateMultiXact()` at `multixact.c:2788`. Neither+runs in recovery:++* `TruncateMultiXact()` opens with `Assert(!RecoveryInProgress());` (`multixact.c:2685`).+* `SetOldestOffset()` is called only from `SetMultiXactIdLimit()` at `multixact.c:2173`, which is+ *after* the early return at `multixact.c:2161-2162`:+ `if (!MultiXactState->finishedStartup) return;`. `finishedStartup` is set only by+ `TrimMultiXact()` (`multixact.c:2006`), whose sole caller is `xlog.c:6527`, at end of recovery.+* The redo path (`multixact.c:2985-2990`) calls `SetMultiXactIdLimit(xlrec.oldestMulti, …)` — which+ correctly stores `oldestMultiXactId` at `multixact.c:2140` before the early return — and then+ `PerformMembersTruncation(xlrec.oldestOffset)` / `PerformOffsetsTruncation()`, using the WAL+ record's value directly. It never stores `xlrec.oldestOffset` into shared memory.++So on a never-promoted standby, `MultiXactState->oldestOffset` keeps its zeroed shmem value for the+life of the server while `nextOffset` is advanced continuously by redo. `num_members` therefore comes+out as `nextOffset - 0` and `members_size` as `5 * nextOffset` bytes+(`MultiXactOffsetStorageSize()`, `multixact_internal.h:125-134`: `MULTIXACT_MEMBERGROUP_SIZE /+MULTIXACT_MEMBERS_PER_MEMBERGROUP` = 20/4 = 5). `num_mxids` and `oldest_multixact` are correct.++The overreport is exactly the primary's current `oldestOffset`, and since bd8d9c9 made offsets+64-bit and monotonic, that quantity climbs for the life of the installation. Meanwhile redo *does*+physically delete the members segments, so the standby reports gigabytes of members it does not have.++**Trigger (reproduced).** Primary with `wal_level = replica`; create a table, take `FOR SHARE` from+concurrent sessions to build multixacts; `pg_basebackup -R` a standby and start it; on the primary+`ALTER DATABASE template0 ALLOW_CONNECTIONS true`, `VACUUM FREEZE` in every database, `CHECKPOINT`,+so `TruncateMultiXact()` advances `oldestOffset`. Observed: primary `num_members` 0, `members_size` 0;+standby `num_members` 3, `members_size` 15, `pg_is_in_recovery()` true. A larger run (1000 multixacts+of 2 members, `nextOffset` 2001) showed the standby reporting 2001/10005 against a primary reporting+0/0. Promoting the standby corrects it immediately, confirming the `finishedStartup` mechanism.++**User-visible consequence.** `SELECT * FROM pg_get_multixact_stats()` on a hot standby contradicts+the same query on its primary for byte-identical data, and contradicts+`doc/src/sgml/func/func-info.sgml:3010-3013`, which defines `num_members` as "the total number of+multixact member entries **currently present in the system**" and `members_size` as "the storage+occupied by `num_members` in the `pg_multixact/members` directory". Monitoring built on this function+sees a standby that appears to be holding hundreds of GB of members it does not hold.++**Attribution — stated honestly.** This is the weakest attribution in the set, and it survived+2/3 refutation votes rather than 3/3. The dissenting reading is correct on the facts and should be+recorded: the `finishedStartup` gate and the redo path are **identical in `bd8d9c9^`**, so the+standby's stuck `oldestOffset` predates the commit; pre-commit it was harmless because the only+consumer, `MultiXactMemberFreezeThreshold()`, never runs in recovery. What bd8d9c9 contributed is+deleting `MultiXactStateData.oldestOffsetKnown` and changing `GetMultiXactInfo()` from returning+`bool` to `void` — it used to short-circuit with `*members = 0; … return false;` so callers could+distinguish "unknown" from "zero". `97b10177` (2025-12-30) then wired the now-unqualified value into+SQL with no `RecoveryInProgress()` guard and no NULL path. A committer may reasonably file this+against `97b10177`; the missing "unknown" signal is bd8d9c9's.++**Why it is not covered by a test.** Testing it needs a full streaming primary/standby pair plus the+`template0`-connectable `VACUUM FREEZE` dance to move the primary's `oldestOffset` off zero — that is+a `src/test/recovery`-style test, not something that fits in `test_slru`'s single-node harness or in+`007_multixact_conversion.pl`. It also cannot be asserted until the intended semantics are decided:+should the standby return NULL for these two columns, or should redo store `xlrec.oldestOffset`?+Writing a test first would bake in whichever answer the test author guessed.++**Test someone should write** (in `src/test/recovery/t/`, once the semantics are settled):++```+init primary (wal_level=replica), create t, produce N multixacts via concurrent FOR SHARE+pg_basebackup -R -> standby, start, wait for catchup+on primary: ALTER DATABASE template0 ALLOW_CONNECTIONS true; VACUUM FREEZE in all dbs; CHECKPOINT+wait_for_catchup+is(standby: SELECT num_members FROM pg_get_multixact_stats(),+ primary: same) # or: expect NULL on the standby+promote standby; re-run; expect it to match+```++The middle assertion is the whole test; it fails today with standby = `nextOffset`,+primary = `nextOffset - oldestOffset`.++---++### 3.4 `false-truncation-disabled-log` — a LOG that names the wrong subsystem and hides the real degradation++**Location:** `src/backend/access/transam/multixact.c:2469-2479`, in `SetOldestOffset()`.++```+2469 oldestOffsetKnown =+2470 find_multixact_start(oldestMultiXactId, &oldestOffset);+2471 +2472 if (oldestOffsetKnown)+2473 ereport(DEBUG1,+2474 (errmsg_internal("oldest MultiXactId member is at offset %" PRIu64,+2475 oldestOffset)));+2476 else+2477 ereport(LOG,+2478 (errmsg("MultiXact member truncation is disabled because oldest checkpointed MultiXact %u does not exist on disk",+2479 oldestMultiXactId)));+```++**Why the message is false.** `MultiXactState->oldestOffset` — the only thing this failure leaves+unset — is written at `multixact.c:2488` and `multixact.c:2788`, and read only through+`GetMultiXactInfo()` (`multixact.c:2555`). Its consumers are exactly two:+`MultiXactMemberFreezeThreshold()` (`multixact.c:2603`) and `pg_get_multixact_stats()`+(`multixactfuncs.c:128`). **No truncation path reads it.** `TruncateMultiXact()` calls+`find_multixact_start(newOldestMulti, &newOldestOffset)` for itself (`multixact.c:2723`), emits its+own distinct messages for its own failures (`multixact.c:2725-2730` "cannot truncate up to MultiXact+%u because it does not exist on disk, skipping truncation" and `multixact.c:2740-2747` "… because it+has invalid offset …"), and hands its own local `newOldestOffset` to `PerformMembersTruncation()`+(`multixact.c:2792`). Member truncation continues normally after this LOG fires.++The stale comment bd8d9c9 added directly above asserts the same nonexistent invariant+(`multixact.c:2457-2462`): "oldestOffset is initialized to zero at system startup, **which prevents+truncating members until a proper value is calculated**".++**What actually degrades.** A stale-or-zero `oldestOffset`, which:++* makes `MultiXactMemberFreezeThreshold()` compute `members = nextOffset - 0` — an *inflated* figure,+ so vacuum freezes **more** aggressively, not less (clamped to `freeze_max_age` 0 once the inflated+ count passes `MULTIXACT_MEMBER_HIGH_THRESHOLD`). This is the opposite of the "members will grow+ unbounded" reading the message invites;+* makes `pg_get_multixact_stats()` report inflated `num_members`/`members_size`.++Neither is mentioned. bd8d9c9 also deleted the paired "MultiXact member wraparound protections are+now enabled" LOG without a replacement, so nothing announces when the condition clears.++**Attribution.** `git show bd8d9c9 -- src/backend/access/transam/multixact.c` changes this line from+"MultiXact member **wraparound protections** are disabled because oldest checkpointed MultiXact %u+does not exist on disk" to the current text, and in the same commit deletes `offsetStopLimit` and+`oldestOffsetKnown` from `MultiXactStateData`. The old wording was accurate for `bd8d9c9^`, where+`oldestOffsetKnown` genuinely gated `offsetStopLimit`; the replacement asserts something that has+never been true post-commit. Only `ef6a95c` (translation catalogs) has touched the string since.++**Trigger (reproduced).** Any state where the `pg_multixact/offsets` page holding pg_control's+`oldestMulti` is absent. Reproduced without touching SLRU *content*: a cluster driven to truncate+away its low offsets segments (30000 multixacts via `test_slru`'s `test_create_multixact()`, then+`VACUUM FREEZE` in all three databases, which unlinked `offsets/0001` and+`members/000000000000000`), then stopped and restarted after+`pg_resetwal --multixact-ids 95535,65535` — a stale "oldest" value of the kind an operator gets by+following the pg_resetwal recovery recipe with a figure from an out-of-date source. On restart+`TrimMultiXact` → `SetMultiXactIdLimit` → `SetOldestOffset` → `find_multixact_start` →+`SimpleLruDoesPhysicalPageExist` false → the LOG fires. 40000 more multixacts plus another+`VACUUM FREEZE` then **deleted a members segment**, with the "truncation is disabled" message still+in force. `pg_get_multixact_stats()` reported inflated members throughout.++Scope honesty: normal operation does not reach this state. `TruncateMultiXact()` updates+`MultiXactState->oldestMultiXactId` inside the same critical section as the unlink, WAL-logs it and+holds `DELAY_CHKPT_START`; the redo path skips `SetOldestOffset()` entirely. Reaching it takes either+legacy 9.3/9.4-era corruption (which the surrounding comment at `multixact.c:2464-2467` still+anticipates) or an operator-supplied `pg_resetwal -m` whose second value is older than the smallest+surviving segment. Note that following `doc/src/sgml/ref/pg_resetwal.sgml`'s documented recipe+literally yields a value whose page *does* exist, so the documented procedure does not hit it. This+is a wrong-message defect on a recovery path, not an everyday hazard.++**User-visible consequence.** A DBA recovering a damaged cluster reads a server LOG stating that+multixact member truncation has been disabled — implying `pg_multixact/members` will grow without+bound until they intervene — when truncation is working fine, and is given no hint about the actual+consequence (an inflated members count driving over-aggressive freezing and wrong+`pg_get_multixact_stats()` output). The message also never retracts itself.++**Why it is not covered by a test.** The state requires staging a mismatch between pg_control's+`oldestMulti` and the surviving offsets segments — a `pg_resetwal --multixact-ids` with a+deliberately stale value, after a real truncation, on a cluster whose multixid counter was+pre-positioned near a segment boundary. That is ~7 setup steps including a `test_slru` module load and+a pgbench run, and the payload assertion is "a LOG line's text is wrong", which the project does not+normally assert. The genuinely testable half — that truncation still runs while the message claims it+is disabled — requires the same staging plus log scraping. The proportionate fix is to reword the+message and delete the false comment; a test would cost far more than the bug.++**Test someone should write, if one is wanted** (`src/test/modules/test_slru/t/`):++```+initdb; pg_resetwal --multixact-ids 0xFFFF,0xFFFF; hand-place a zeroed offsets/0001, rm offsets/0000+load test_slru; create ~30k multixacts via test_create_multixact()+make template0 connectable; VACUUM FREEZE all dbs # truncation unlinks offsets/0001+stop; pg_resetwal --multixact-ids <new nextMulti>,<stale oldest inside the unlinked segment>; start+expect the LOG line+create more multixacts; VACUUM FREEZE all dbs+assert a members segment WAS unlinked # i.e. the LOG's claim is false+assert pg_get_multixact_stats().num_members is inflated relative to reality+```++---++## 4. What the companion test does cover++For completeness, `src/test/modules/test_slru/t/003_multixact_offset.pl` (plus its `meson.build`+entry) and the additions to `src/bin/pg_upgrade/t/007_multixact_conversion.pl` cover rows 5-8 of the+summary table:++* **`resetwal-nextmxoff-zero`** — `GuessControlValues()` at `pg_resetwal.c:703` assigns+ `nextMultiOffset = 0`, and `case 'O'` (`pg_resetwal.c:297-307`) accepts 0 with no zero check,+ unlike `-o` (`:263-264`) and `-m` (`:290-293`). The resulting cluster cannot read back the first+ multixact it creates: `ERROR: MultiXact 1 has invalid offset`.+* **`pg-control-checkpoint-next-multi-offset-xid`** — `pg_controldata.c:133` (backend) funnels the+ now-`uint64` field through `TransactionIdGetDatum()`, and `pg_proc.dat:12374` still declares the+ column `xid` (`proallargtypes` has `xid` in that slot; `func-info.sgml:3460-3461` documents it as+ `xid`). Control file holds 4294967896, SQL reports 600.+* **`members-truncation-apparent-wraparound`** — `PerformMembersTruncation()`+ (`multixact.c:2642-2647`) passes `MXOffsetToMemberPage(newOldestOffset)` to `SimpleLruTruncate()`+ with no step-back, unlike `PerformOffsetsTruncation()` immediately below it+ (`multixact.c:2652-2664`), whose comment names exactly this hazard.+* **`pgupgrade-nxtmxoff-str2uint`** — `controldata.c:292` reads the field with `str2uint()`+ (`util.c:351-355`, returns `unsigned int`) into the `uint64 chkpnt_nxtmxoff`+ (`pg_upgrade.h:221`). Old 4294967896 → new 600.++---++## 5. Refuted candidates — do not re-chase these++* **"pg_upgrade's multixact conversion `pg_fatal`s on a missing offsets segment that both servers+ tolerate" (`src/bin/pg_upgrade/slru_io.c:112`)** — refuted 2/3. The code reading is right, but no+ normal-operation path produces the state: offsets segments are removed only by+ `TruncateMultiXact()`, which never deletes the segment containing `oldestMulti`. Reaching it needs+ a hand-set pg_control on an already-corrupt cluster, and the claimed consequences were overstated.++* **"Widening `moff` to 64 bits opened a 4-byte padding hole that ships uninitialized stack in every+ CREATE_ID WAL record" (`src/include/access/multixact.h:75`)** — refuted 2/3. The layout claim is+ accurate (`mid`@0, `moff`@8, `nmembers`@16, `members`@20 with `MultiXactOffset` = `uint64`), but no+ observable behavior follows: nothing reads the padding, WAL is not compared byte-wise across+ systems, and no user-visible symptom could be stated. Fails the "user-visible defect" bar.++* **"`SetOldestOffset()` installs a zero `oldestOffset`, defeating `102bdaa`'s guard and clamping+ vacuum's multixact freeze age cluster-wide" (`multixact.c:2485`)** — refuted 3/3. The code is+ really at `multixact.c:2484-2490` and the mechanical chain is right, but the value is only+ installed `if (oldestOffsetKnown)`, and the arithmetic that was claimed to clamp+ `freeze_max_age` bottoms out at `if (fraction >= 1.0) return 0;` only after+ `members > MULTIXACT_MEMBER_LOW_THRESHOLD`, which the described state does not reach. (The related,+ *real* problem in that function is the false LOG message, §3.4.)++---++## 6. Bottom line++Four confirmed, still-unfixed, user-observable defects attributable to bd8d9c9 have no test coverage:+two documentation claims falsified by the commit (§3.1, §3.2), one wrong-value report on hot standbys+(§3.3, with attribution partly to the later `97b10177`), and one server LOG message that names the+wrong subsystem and conceals the actual degradation (§3.4). None of the four is testable at+proportionate cost on ordinary hardware without first adding a threshold override+(§3.2), building a recovery-suite test and settling the intended standby semantics (§3.3), or+accepting a seven-step SLRU staging fixture to assert a message's text (§3.4). §3.1 is a one-line+SGML fix that no test could ever have caught.diff --git a/src/bin/pg_upgrade/t/007_multixact_conversion.pl b/src/bin/pg_upgrade/t/007_multixact_conversion.plindex 867a062..85742b9 100644--- a/src/bin/pg_upgrade/t/007_multixact_conversion.pl+++ b/src/bin/pg_upgrade/t/007_multixact_conversion.pl@@ -440,4 +440,72 @@ SKIP:
upgrade_and_compare($tag, $old, $new);
}
+# High-offset scenario: the old cluster's 64-bit NextMultiOffset has+# passed 2^32. No SLRU conversion is involved here, so both clusters+# use the current version regardless of $ENV{oldinstall}: the value is+# copied verbatim from the old cluster's control file to the new one,+# which is exactly the code path we want to check.+#+# get_control_data() in controldata.c reads "Latest checkpoint's+# NextMultiOffset" with str2uint(), which returns unsigned int, even+# though chkpnt_nxtmxoff became uint64 when offsets were widened. The+# new cluster therefore gets NextMultiOffset modulo 2^32, i.e. an offset+# pointing back into members space that is still in use.+{+ my $tag = 'highoffset';+ my $old = PostgreSQL::Test::Cluster->new("${tag}_oldnode");+ my $new = PostgreSQL::Test::Cluster->new("${tag}_newnode");++ $old->init;+ $new->init;++ # MULTIXACT_MEMBERS_PER_PAGE, as computed in+ # reset_mxid_mxoffset_pre_v19() above.+ my ($out, undef) =+ run_command([ 'pg_resetwal', '--dry-run', $old->data_dir ]);+ $out =~ /^Database block size: *(\d+)$/m or die;+ my $blcksz = $1;+ my $multixact_members_per_page = int($blcksz / 20) * 4;++ # Smallest members-page-aligned offset at or above 2^32. Page+ # alignment matters because pg_upgrade starts the old cluster to+ # dump its schema: with an aligned nextOffset, TrimMultiXact() does+ # not read the members page, so the old cluster starts without a+ # multi-gigabyte members segment on disk. Alignment also puts+ # ($target mod 2^32) inside members page 0, which the copied segment+ # does contain, so pg_upgrade itself still succeeds and the test+ # fails on a clean value mismatch rather than on command_ok().+ my $two_to_32 = Math::BigInt->new(2)**32;+ my $target =+ ($two_to_32 + $multixact_members_per_page - 1)+ ->bdiv($multixact_members_per_page)+ ->bmul($multixact_members_per_page);++ command_ok(+ [+ 'pg_resetwal',+ '--multixact-offset' => "$target",+ $old->data_dir+ ],+ 'set old cluster NextMultiOffset past 2^32');++ command_ok(+ [+ 'pg_upgrade', '--no-sync',+ '--old-datadir' => $old->data_dir,+ '--new-datadir' => $new->data_dir,+ '--old-bindir' => $old->config_data('--bindir'),+ '--new-bindir' => $new->config_data('--bindir'),+ '--socketdir' => $new->host,+ '--old-port' => $old->port,+ '--new-port' => $new->port,+ ],+ 'run of pg_upgrade with a NextMultiOffset above 2^32');++ my (undef, undef, $old_mxoff) = read_multixid_fields($old);+ my (undef, undef, $new_mxoff) = read_multixid_fields($new);+ is($new_mxoff, $old_mxoff,+ 'NextMultiOffset is preserved across pg_upgrade');+}+
done_testing();
diff --git a/src/test/modules/test_slru/meson.build b/src/test/modules/test_slru/meson.buildindex 00f3ee3..f4536ea 100644--- a/src/test/modules/test_slru/meson.build+++ b/src/test/modules/test_slru/meson.build@@ -39,7 +39,8 @@ tests += {
},
'tests': [
't/001_multixact.pl',
- 't/002_multixact_wraparound.pl'+ 't/002_multixact_wraparound.pl',+ 't/003_multixact_offset.pl'
],
},
}
diff --git a/src/test/modules/test_slru/t/003_multixact_offset.pl b/src/test/modules/test_slru/t/003_multixact_offset.plnew file mode 100644index 0000000..c6771dd--- /dev/null+++ b/src/test/modules/test_slru/t/003_multixact_offset.pl@@ -0,0 +1,254 @@+# Copyright (c) 2026, PostgreSQL Global Development Group++# Tests for the 64-bit MultiXactOffset counter (nextMultiOffset), which+# commit bd8d9c9bdf widened from 32 bits. Three independent scenarios,+# each with its own cluster:+#+# 1. pg_resetwal must never leave nextMultiOffset at 0. Now that+# offsets are 64-bit and no longer wrap, 0 is reserved as the+# "invalid offset" marker: GetMultiXactIdMembers() in+# access/transam/multixact.c errors out with "MultiXact %u has+# invalid offset" when it reads a zero offset. But+# GuessControlValues() in bin/pg_resetwal/pg_resetwal.c still assigns+# nextMultiOffset = 0, and the -O option still accepts 0, unlike -m+# which rejects it. A cluster left in that state cannot read back+# the first multixact it creates.+#+# 2. pg_control_checkpoint() must report the full 64-bit+# NextMultiOffset. pg_control_checkpoint() in+# backend/utils/misc/pg_controldata.c still runs the field through+# TransactionIdGetDatum(), and pg_proc.dat still declares the+# next_multi_offset output column as xid, so the value is silently+# reported modulo 2^32.+#+# 3. Members truncation must not report a bogus "apparent wraparound"+# and must actually reclaim obsolete segments.+# PerformMembersTruncation() in access/transam/multixact.c passes+# MXOffsetToMemberPage(newOldestOffset) to SimpleLruTruncate()+# without stepping back one member, unlike PerformOffsetsTruncation()+# directly below it. When nextOffset happens to sit exactly on a+# members page boundary, the cutoff page is one past the last page+# ever written, so SimpleLruTruncate()'s endpoint-page safety check+# fires and nothing is removed. Before the widening,+# PerformMembersTruncation() unlinked the segments itself and had no+# such check.++use strict;+use warnings FATAL => 'all';++use PostgreSQL::Test::Cluster;+use PostgreSQL::Test::Utils;++use Test::More;++# Read "Latest checkpoint's NextMultiOffset" out of pg_controldata.+sub next_multi_offset+{+ my $node = shift;++ my $out = (run_command([ 'pg_controldata', $node->data_dir ]))[0];+ $out =~ /^Latest checkpoint's NextMultiOffset: *(\d+)$/m+ or die "could not find NextMultiOffset in: $out";+ return $1;+}++# Extract the SLRU geometry from pg_resetwal --dry-run output, like+# 002_multixact_wraparound.pl does.+sub slru_geometry+{+ my $node = shift;++ my $out =+ (run_command([ 'pg_resetwal', '--dry-run', $node->data_dir ]))[0];+ $out =~ /^Database block size: *(\d+)$/m or die;+ my $blcksz = $1;+ $out =~ /^Pages per SLRU segment: *(\d+)$/m or die;+ my $pages_per_seg = $1;++ # MULTIXACT_MEMBERS_PER_PAGE: a member group is+ # 4 * sizeof(TransactionId) + 4 == 20 bytes and holds 4 members.+ # See MULTIXACT_MEMBERGROUP_SIZE in access/multixact_internal.h.+ my $members_per_page = int($blcksz / 20) * 4;++ return ($blcksz, $pages_per_seg, $members_per_page);+}++#+# 1. pg_resetwal must not produce nextMultiOffset == 0.+#+{+ my $node = PostgreSQL::Test::Cluster->new('mxoff_zero');+ $node->init;+ $node->append_conf('postgresql.conf',+ "shared_preload_libraries = 'test_slru'\n" . "autovacuum = off\n");++ # Use run_command(), not command_ok(): a fix for this may well make+ # pg_resetwal reject -O 0 outright, and this test should pass either+ # way. All that matters is what ends up in the control file.+ run_command(+ [ 'pg_resetwal', '--multixact-offset' => 0, $node->data_dir ]);++ cmp_ok(next_multi_offset($node), '>=', 1,+ 'pg_resetwal -O 0 does not store the reserved offset 0');++ # Whatever the control file says, the cluster must be able to create+ # and read back multixacts.+ $node->start;+ $node->safe_psql('postgres', 'CREATE EXTENSION test_slru');++ my $mx1 = $node->safe_psql('postgres', 'SELECT test_create_multixact()');+ my ($rc, undef, $err) =+ $node->psql('postgres', "SELECT test_read_multixact('$mx1')");+ is($rc, 0, 'the first multixact created after pg_resetwal is readable')+ or diag("multixact $mx1: $err");++ # Negative control: a later multixact must be fine regardless, so a+ # failure above is really about the first one and not about a+ # generally broken cluster.+ my $mx2 = $node->safe_psql('postgres', 'SELECT test_create_multixact()');+ ($rc, undef, $err) =+ $node->psql('postgres', "SELECT test_read_multixact('$mx2')");+ is($rc, 0, 'a subsequent multixact is readable')+ or diag("multixact $mx2: $err");++ $node->stop;++ # The producer of a zero offset that needs no bad user input is+ # GuessControlValues(), used when pg_control cannot be read at all.+ # That is the documented "pg_resetwal -f" last-resort recovery. No+ # server start here: a guessed control file also rewinds nextXid, so+ # the cluster is not usable without more hand-picked values.+ my $pg_control = $node->data_dir . '/global/pg_control';+ my $size = -s $pg_control;+ open my $fh, '>', $pg_control or die "could not open pg_control: $!";+ binmode $fh;+ print $fh pack("x[$size]");+ close $fh;++ command_ok([ 'pg_resetwal', '--force', $node->data_dir ],+ 'pg_resetwal -f on an unreadable pg_control');+ cmp_ok(next_multi_offset($node), '>=', 1,+ 'pg_resetwal -f does not guess the reserved offset 0');+}++#+# 2. pg_control_checkpoint() must not truncate NextMultiOffset to 32 bits.+#+{+ my $node = PostgreSQL::Test::Cluster->new('mxoff_64bit');+ $node->init;+ $node->append_conf('postgresql.conf', "autovacuum = off\n");++ my (undef, undef, $members_per_page) = slru_geometry($node);++ # Smallest members-page-aligned offset at or above 2^32. Page+ # alignment matters: TrimMultiXact() then skips reading the members+ # page, so the cluster starts without a huge segment on disk.+ my $target =+ int((4294967296 + $members_per_page - 1) / $members_per_page) *+ $members_per_page;++ command_ok(+ [ 'pg_resetwal', '--multixact-offset' => $target, $node->data_dir ],+ 'set NextMultiOffset past 2^32');++ # pg_resetwal's own -O parser is 64-bit; check that the control file+ # really holds the large value before blaming SQL for the mismatch.+ my $from_controldata = next_multi_offset($node);+ is($from_controldata, "$target", 'pg_controldata reports the full value');++ $node->start;++ # Compare against the string pg_controldata printed, and cast to+ # text: a complete fix needs Int64GetDatum() *and* a change of the+ # column's declared type from xid to int8, and this comparison stays+ # valid across that change.+ is( $node->safe_psql(+ 'postgres',+ 'SELECT next_multi_offset::text FROM pg_control_checkpoint()'),+ $from_controldata,+ 'pg_control_checkpoint() reports the full 64-bit NextMultiOffset');++ $node->stop;+}++#+# 3. Members truncation must not claim "apparent wraparound".+#+{+ my $node = PostgreSQL::Test::Cluster->new('mxoff_members');+ $node->init;+ $node->append_conf('postgresql.conf',+ "shared_preload_libraries = 'test_slru'\n" . "autovacuum = off\n");++ my ($blcksz, $pages_per_seg, $members_per_page) = slru_geometry($node);+ my $members_per_seg = $members_per_page * $pages_per_seg;++ # Aim to land nextOffset exactly on a members segment boundary, two+ # segments in. test_create_multixact() creates exactly one multixact+ # with exactly 2 members, so start 2 short of the boundary.+ my $target = 2 * $members_per_seg;++ command_ok(+ [+ 'pg_resetwal',+ '--multixact-offset' => $target - 2,+ $node->data_dir+ ],+ 'set NextMultiOffset just below a members segment boundary');++ # pg_resetwal only updates the control file, so create the members+ # segments that the pre-seeded offset lives in, filled with zeros, as+ # 002_multixact_wraparound.pl does for the offsets SLRU. (In the+ # field the server would have written them itself.) Segment 0 must+ # exist too, or there would be nothing for the truncation to reclaim.+ # The members SLRU uses long segment names.+ my $bytes_per_seg = $pages_per_seg * $blcksz;+ foreach my $segno (0, 1)+ {+ my $path =+ sprintf('%s/pg_multixact/members/%015X', $node->data_dir, $segno);+ open my $fh, '>', $path or die "could not open \"$path\": $!";+ binmode $fh;+ syswrite($fh, "\0" x $bytes_per_seg) == $bytes_per_seg+ or die "could not write to \"$path\": $!";+ close $fh;+ }++ $node->start;+ $node->safe_psql('postgres', 'CREATE EXTENSION test_slru');++ # One multixact of 2 members: nextOffset becomes exactly $target,+ # i.e. exactly on a members segment (hence page) boundary.+ $node->safe_psql('postgres', 'SELECT test_create_multixact()');++ # Do not restart from here on. StartupMultiXact()/TrimMultiXact()+ # would re-derive the members SLRU's latest_page_number from+ # nextOffset and mask the problem.++ my $log_offset = -s $node->logfile;++ # Drive every datminmxid up to nextMulti, so that TruncateMultiXact()+ # takes the "there are NO MultiXacts" branch and uses nextOffset as+ # the members cutoff.+ $node->safe_psql('postgres',+ 'ALTER DATABASE template0 ALLOW_CONNECTIONS true');+ foreach my $db ('postgres', 'template1', 'template0')+ {+ $node->safe_psql($db, 'VACUUM FREEZE');+ }++ ok( !$node->log_contains(+ qr{pg_multixact/members": apparent wraparound}, $log_offset),+ 'members truncation does not report a bogus wraparound');++ my $seg0 = $node->data_dir . '/pg_multixact/members/000000000000000';+ my @segs = map { (split m{/})[-1] }+ glob($node->data_dir . '/pg_multixact/members/*');+ ok(!-f $seg0, 'obsolete members segment is reclaimed')+ or diag('members segments left behind: ' . join(' ', sort @segs));++ $node->stop;+}++done_testing();
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: noah@leadboat.com, hlinnaka@iki.fi, orlovmg@gmail.com
Subject: Re: 64-bit MultiXactOffset vs. 9.3->9.4 upgrade, pg_resetwal, "wraparound" msg
In-Reply-To: <20260827231757.78.noahmisch@microsoft.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox