public inbox for [email protected]help / color / mirror / Atom feed
[PATCH v13 1/6] snapshot scalability: Don't compute global horizons when building snapshots. 2+ messages / 2 participants [nested] [flat]
* [PATCH v13 1/6] snapshot scalability: Don't compute global horizons when building snapshots. @ 2020-07-15 22:35 Andres Freund <[email protected]> 0 siblings, 0 replies; 2+ messages in thread From: Andres Freund @ 2020-07-15 22:35 UTC (permalink / raw) To make GetSnapshotData() more scalable, it cannot not look at at each proc's xmin (see Discussion link below). Due to the frequency at which xmins are updated, that just does not scale. Without accessing xmins GetSnapshotData() cannot calculate accurate thresholds as it has so far. But we don't really have to: The horizons don't actually change that much between GetSnapshotData() calls. Nor are the horizons actually used every time a snapshot is called. The use of RecentGlobal[Data]Xmin to decide whether a row version could be removed has been replaces with new GlobalVisTest* functions. These use two thresholds to determine whether a row can be pruned: 1) definitely_needed, indicating that rows deleted by XIDs >= definitely_needed are definitely still visible. 2) maybe_needed, indicating that rows deleted by XIDs < maybe_needed can definitely be removed GetSnapshotData() updates definitely_needed to be the xmin of the computed snapshot. When testing whether a row can be removed (with GlobalVisTestIsRemovableXid()) and the tested XID falls in between the two (i.e. XID >= maybe_needed && XID < definitely_needed) the boundaries can be recomputed to be more accurate. As it is not cheap to compute accurate boundaries, we limit the number of times that happens in short succession. As the boundaries used by GlobalVisTestIsRemovableXid() are never reset (with maybe_needed updated byGetSnapshotData()), it is likely that further test can benefit from an earlier computation of accurate horizons. To avoid regressing performance when old_snapshot_threshold is set (as that requires an accurate horizon to be computed), heap_page_prune_opt() doesn't unconditionally call TransactionIdLimitedForOldSnapshots() anymore. Both the computation of the limited horizon, and the triggering of errors (with SetOldSnapshotThresholdTimestamp()) is now only done when necessary to remove tuples. Subsequent commits will take further advantage of the fact that GetSnapshotData() will not need to access xmins anymore. Note: This contains a workaround in heap_page_prune_opt() to keep the snapshot_too_old tests working. While that workaround is ugly, the tests currently are not meaningful, and it seems best to address them separately. Author: Andres Freund Reviewed-By: Robert Haas, Thomas Munro, David Rowley Discussion: https://postgr.es/m/[email protected] --- src/include/access/ginblock.h | 4 +- src/include/access/heapam.h | 10 +- src/include/access/transam.h | 79 +- src/include/storage/bufpage.h | 6 - src/include/storage/proc.h | 8 - src/include/storage/procarray.h | 32 +- src/include/utils/snapmgr.h | 37 +- src/include/utils/snapshot.h | 6 + src/backend/access/gin/ginvacuum.c | 26 + src/backend/access/gist/gistutil.c | 8 +- src/backend/access/gist/gistxlog.c | 10 +- src/backend/access/heap/heapam.c | 15 +- src/backend/access/heap/heapam_handler.c | 24 +- src/backend/access/heap/heapam_visibility.c | 79 +- src/backend/access/heap/pruneheap.c | 207 ++++- src/backend/access/heap/vacuumlazy.c | 24 +- src/backend/access/index/indexam.c | 3 +- src/backend/access/nbtree/README | 10 +- src/backend/access/nbtree/nbtpage.c | 4 +- src/backend/access/nbtree/nbtree.c | 28 +- src/backend/access/nbtree/nbtxlog.c | 10 +- src/backend/access/spgist/spgvacuum.c | 6 +- src/backend/access/transam/README | 78 +- src/backend/access/transam/xlog.c | 4 +- src/backend/commands/analyze.c | 2 +- src/backend/commands/vacuum.c | 41 +- src/backend/postmaster/autovacuum.c | 4 + src/backend/replication/logical/launcher.c | 4 + src/backend/replication/walreceiver.c | 17 +- src/backend/replication/walsender.c | 15 +- src/backend/storage/ipc/procarray.c | 902 ++++++++++++++++---- src/backend/utils/adt/selfuncs.c | 20 +- src/backend/utils/init/postinit.c | 4 + src/backend/utils/time/snapmgr.c | 258 +++--- contrib/amcheck/verify_nbtree.c | 8 +- contrib/pg_visibility/pg_visibility.c | 18 +- contrib/pgstattuple/pgstatapprox.c | 2 +- src/tools/pgindent/typedefs.list | 2 + 38 files changed, 1449 insertions(+), 566 deletions(-) diff --git a/src/include/access/ginblock.h b/src/include/access/ginblock.h index 3f64fd572e3..fe66a95226b 100644 --- a/src/include/access/ginblock.h +++ b/src/include/access/ginblock.h @@ -12,6 +12,7 @@ #include "access/transam.h" #include "storage/block.h" +#include "storage/bufpage.h" #include "storage/itemptr.h" #include "storage/off.h" @@ -134,8 +135,7 @@ typedef struct GinMetaPageData */ #define GinPageGetDeleteXid(page) ( ((PageHeader) (page))->pd_prune_xid ) #define GinPageSetDeleteXid(page, xid) ( ((PageHeader) (page))->pd_prune_xid = xid) -#define GinPageIsRecyclable(page) ( PageIsNew(page) || (GinPageIsDeleted(page) \ - && TransactionIdPrecedes(GinPageGetDeleteXid(page), RecentGlobalXmin))) +extern bool GinPageIsRecyclable(Page page); /* * We use our own ItemPointerGet(BlockNumber|OffsetNumber) diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h index b31de389106..ebb79428d16 100644 --- a/src/include/access/heapam.h +++ b/src/include/access/heapam.h @@ -172,9 +172,12 @@ extern TransactionId heap_compute_xid_horizon_for_tuples(Relation rel, int nitems); /* in heap/pruneheap.c */ +struct GlobalVisState; extern void heap_page_prune_opt(Relation relation, Buffer buffer); extern int heap_page_prune(Relation relation, Buffer buffer, - TransactionId OldestXmin, + struct GlobalVisState *vistest, + TransactionId limited_oldest_xmin, + TimestampTz limited_oldest_ts, bool report_stats, TransactionId *latestRemovedXid); extern void heap_page_prune_execute(Buffer buffer, OffsetNumber *redirected, int nredirected, @@ -195,11 +198,14 @@ extern TM_Result HeapTupleSatisfiesUpdate(HeapTuple stup, CommandId curcid, Buffer buffer); extern HTSV_Result HeapTupleSatisfiesVacuum(HeapTuple stup, TransactionId OldestXmin, Buffer buffer); +extern HTSV_Result HeapTupleSatisfiesVacuumHorizon(HeapTuple stup, Buffer buffer, + TransactionId *dead_after); extern void HeapTupleSetHintBits(HeapTupleHeader tuple, Buffer buffer, uint16 infomask, TransactionId xid); extern bool HeapTupleHeaderIsOnlyLocked(HeapTupleHeader tuple); extern bool XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot); -extern bool HeapTupleIsSurelyDead(HeapTuple htup, TransactionId OldestXmin); +extern bool HeapTupleIsSurelyDead(struct GlobalVisState *vistest, + HeapTuple htup); /* * To avoid leaking too much knowledge about reorderbuffer implementation diff --git a/src/include/access/transam.h b/src/include/access/transam.h index 8db326ad1b5..b32044153b0 100644 --- a/src/include/access/transam.h +++ b/src/include/access/transam.h @@ -95,15 +95,6 @@ FullTransactionIdFromU64(uint64 value) (dest) = FirstNormalTransactionId; \ } while(0) -/* advance a FullTransactionId variable, stepping over special XIDs */ -static inline void -FullTransactionIdAdvance(FullTransactionId *dest) -{ - dest->value++; - while (XidFromFullTransactionId(*dest) < FirstNormalTransactionId) - dest->value++; -} - /* * Retreat a FullTransactionId variable, stepping over xids that would appear * to be special only when viewed as 32bit XIDs. @@ -129,6 +120,23 @@ FullTransactionIdRetreat(FullTransactionId *dest) dest->value--; } +/* + * Advance a FullTransactionId variable, stepping over xids that would appear + * to be special only when viewed as 32bit XIDs. + */ +static inline void +FullTransactionIdAdvance(FullTransactionId *dest) +{ + dest->value++; + + /* see FullTransactionIdAdvance() */ + if (FullTransactionIdPrecedes(*dest, FirstNormalFullTransactionId)) + return; + + while (XidFromFullTransactionId(*dest) < FirstNormalTransactionId) + dest->value++; +} + /* back up a transaction ID variable, handling wraparound correctly */ #define TransactionIdRetreat(dest) \ do { \ @@ -293,6 +301,59 @@ ReadNewTransactionId(void) return XidFromFullTransactionId(ReadNextFullTransactionId()); } +/* return transaction ID backed up by amount, handling wraparound correctly */ +static inline TransactionId +TransactionIdRetreatedBy(TransactionId xid, uint32 amount) +{ + xid -= amount; + + while (xid < FirstNormalTransactionId) + xid--; + + return xid; +} + +/* return the older of the two IDs */ +static inline TransactionId +TransactionIdOlder(TransactionId a, TransactionId b) +{ + if (!TransactionIdIsValid(a)) + return b; + + if (!TransactionIdIsValid(b)) + return a; + + if (TransactionIdPrecedes(a, b)) + return a; + return b; +} + +/* return the older of the two IDs, assuming they're both normal */ +static inline TransactionId +NormalTransactionIdOlder(TransactionId a, TransactionId b) +{ + Assert(TransactionIdIsNormal(a)); + Assert(TransactionIdIsNormal(b)); + if (NormalTransactionIdPrecedes(a, b)) + return a; + return b; +} + +/* return the newer of the two IDs */ +static inline FullTransactionId +FullTransactionIdNewer(FullTransactionId a, FullTransactionId b) +{ + if (!FullTransactionIdIsValid(a)) + return b; + + if (!FullTransactionIdIsValid(b)) + return a; + + if (FullTransactionIdFollows(a, b)) + return a; + return b; +} + #endif /* FRONTEND */ #endif /* TRANSAM_H */ diff --git a/src/include/storage/bufpage.h b/src/include/storage/bufpage.h index 3f88683a059..51b8f994ac0 100644 --- a/src/include/storage/bufpage.h +++ b/src/include/storage/bufpage.h @@ -389,12 +389,6 @@ PageValidateSpecialPointer(Page page) #define PageClearAllVisible(page) \ (((PageHeader) (page))->pd_flags &= ~PD_ALL_VISIBLE) -#define PageIsPrunable(page, oldestxmin) \ -( \ - AssertMacro(TransactionIdIsNormal(oldestxmin)), \ - TransactionIdIsValid(((PageHeader) (page))->pd_prune_xid) && \ - TransactionIdPrecedes(((PageHeader) (page))->pd_prune_xid, oldestxmin) \ -) #define PageSetPrunable(page, xid) \ do { \ Assert(TransactionIdIsNormal(xid)); \ diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h index 5ceb2494bae..52ff43cabaa 100644 --- a/src/include/storage/proc.h +++ b/src/include/storage/proc.h @@ -42,20 +42,12 @@ struct XidCache /* * Flags for PGXACT->vacuumFlags - * - * Note: If you modify these flags, you need to modify PROCARRAY_XXX flags - * in src/include/storage/procarray.h. - * - * PROC_RESERVED may later be assigned for use in vacuumFlags, but its value is - * used for PROCARRAY_SLOTS_XMIN in procarray.h, so GetOldestXmin won't be able - * to match and ignore processes with this flag set. */ #define PROC_IS_AUTOVACUUM 0x01 /* is it an autovac worker? */ #define PROC_IN_VACUUM 0x02 /* currently running lazy vacuum */ #define PROC_VACUUM_FOR_WRAPAROUND 0x08 /* set by autovac only */ #define PROC_IN_LOGICAL_DECODING 0x10 /* currently doing logical * decoding outside xact */ -#define PROC_RESERVED 0x20 /* reserved for procarray */ /* flags reset at EOXact */ #define PROC_VACUUM_STATE_MASK \ diff --git a/src/include/storage/procarray.h b/src/include/storage/procarray.h index 01040d76e12..ea8a876ca45 100644 --- a/src/include/storage/procarray.h +++ b/src/include/storage/procarray.h @@ -20,34 +20,6 @@ #include "utils/snapshot.h" -/* - * These are to implement PROCARRAY_FLAGS_XXX - * - * Note: These flags are cloned from PROC_XXX flags in src/include/storage/proc.h - * to avoid forcing to include proc.h when including procarray.h. So if you modify - * PROC_XXX flags, you need to modify these flags. - */ -#define PROCARRAY_VACUUM_FLAG 0x02 /* currently running lazy - * vacuum */ -#define PROCARRAY_LOGICAL_DECODING_FLAG 0x10 /* currently doing logical - * decoding outside xact */ - -#define PROCARRAY_SLOTS_XMIN 0x20 /* replication slot xmin, - * catalog_xmin */ -/* - * Only flags in PROCARRAY_PROC_FLAGS_MASK are considered when matching - * PGXACT->vacuumFlags. Other flags are used for different purposes and - * have no corresponding PROC flag equivalent. - */ -#define PROCARRAY_PROC_FLAGS_MASK (PROCARRAY_VACUUM_FLAG | \ - PROCARRAY_LOGICAL_DECODING_FLAG) - -/* Use the following flags as an input "flags" to GetOldestXmin function */ -/* Consider all backends except for logical decoding ones which manage xmin separately */ -#define PROCARRAY_FLAGS_DEFAULT PROCARRAY_LOGICAL_DECODING_FLAG -/* Ignore vacuum backends */ -#define PROCARRAY_FLAGS_VACUUM PROCARRAY_FLAGS_DEFAULT | PROCARRAY_VACUUM_FLAG - extern Size ProcArrayShmemSize(void); extern void CreateSharedProcArray(void); extern void ProcArrayAdd(PGPROC *proc); @@ -81,9 +53,11 @@ extern RunningTransactions GetRunningTransactionData(void); extern bool TransactionIdIsInProgress(TransactionId xid); extern bool TransactionIdIsActive(TransactionId xid); -extern TransactionId GetOldestXmin(Relation rel, int flags); +extern TransactionId GetOldestNonRemovableTransactionId(Relation rel); +extern TransactionId GetOldestTransactionIdConsideredRunning(void); extern TransactionId GetOldestActiveTransactionId(void); extern TransactionId GetOldestSafeDecodingTransactionId(bool catalogOnly); +extern void GetReplicationHorizons(TransactionId *slot_xmin, TransactionId *catalog_xmin); extern VirtualTransactionId *GetVirtualXIDsDelayingChkpt(int *nvxids); extern bool HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids); diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index ffb4ba3adfb..b6b403e2931 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -52,13 +52,12 @@ extern Size SnapMgrShmemSize(void); extern void SnapMgrInit(void); extern TimestampTz GetSnapshotCurrentTimestamp(void); extern TimestampTz GetOldSnapshotThresholdTimestamp(void); +extern void SnapshotTooOldMagicForTest(void); extern bool FirstSnapshotSet; extern PGDLLIMPORT TransactionId TransactionXmin; extern PGDLLIMPORT TransactionId RecentXmin; -extern PGDLLIMPORT TransactionId RecentGlobalXmin; -extern PGDLLIMPORT TransactionId RecentGlobalDataXmin; /* Variables representing various special snapshot semantics */ extern PGDLLIMPORT SnapshotData SnapshotSelfData; @@ -78,11 +77,12 @@ extern PGDLLIMPORT SnapshotData CatalogSnapshotData; /* * Similarly, some initialization is required for a NonVacuumable snapshot. - * The caller must supply the xmin horizon to use (e.g., RecentGlobalXmin). + * The caller must supply the visibility cutoff state to use (c.f. + * GlobalVisTestFor()). */ -#define InitNonVacuumableSnapshot(snapshotdata, xmin_horizon) \ +#define InitNonVacuumableSnapshot(snapshotdata, vistestp) \ ((snapshotdata).snapshot_type = SNAPSHOT_NON_VACUUMABLE, \ - (snapshotdata).xmin = (xmin_horizon)) + (snapshotdata).vistest = (vistestp)) /* * Similarly, some initialization is required for SnapshotToast. We need @@ -98,6 +98,11 @@ extern PGDLLIMPORT SnapshotData CatalogSnapshotData; ((snapshot)->snapshot_type == SNAPSHOT_MVCC || \ (snapshot)->snapshot_type == SNAPSHOT_HISTORIC_MVCC) +static inline bool +OldSnapshotThresholdActive(void) +{ + return old_snapshot_threshold >= 0; +} extern Snapshot GetTransactionSnapshot(void); extern Snapshot GetLatestSnapshot(void); @@ -121,8 +126,6 @@ extern void UnregisterSnapshot(Snapshot snapshot); extern Snapshot RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner); extern void UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner); -extern FullTransactionId GetFullRecentGlobalXmin(void); - extern void AtSubCommit_Snapshot(int level); extern void AtSubAbort_Snapshot(int level); extern void AtEOXact_Snapshot(bool isCommit, bool resetXmin); @@ -131,13 +134,29 @@ extern void ImportSnapshot(const char *idstr); extern bool XactHasExportedSnapshots(void); extern void DeleteAllExportedSnapshotFiles(void); extern bool ThereAreNoPriorRegisteredSnapshots(void); -extern TransactionId TransactionIdLimitedForOldSnapshots(TransactionId recentXmin, - Relation relation); +extern bool TransactionIdLimitedForOldSnapshots(TransactionId recentXmin, + Relation relation, + TransactionId *limit_xid, + TimestampTz *limit_ts); +extern void SetOldSnapshotThresholdTimestamp(TimestampTz ts, TransactionId xlimit); extern void MaintainOldSnapshotTimeMapping(TimestampTz whenTaken, TransactionId xmin); extern char *ExportSnapshot(Snapshot snapshot); +/* + * These live in procarray.c because they're intimately linked to the + * procarray contents, but thematically they better fit into snapmgr.h. + */ +typedef struct GlobalVisState GlobalVisState; +extern GlobalVisState *GlobalVisTestFor(Relation rel); +extern bool GlobalVisTestIsRemovableXid(GlobalVisState *state, TransactionId xid); +extern bool GlobalVisTestIsRemovableFullXid(GlobalVisState *state, FullTransactionId fxid); +extern FullTransactionId GlobalVisTestNonRemovableFullHorizon(GlobalVisState *state); +extern TransactionId GlobalVisTestNonRemovableHorizon(GlobalVisState *state); +extern bool GlobalVisCheckRemovableXid(Relation rel, TransactionId xid); +extern bool GlobalVisIsRemovableFullXid(Relation rel, FullTransactionId fxid); + /* * Utility functions for implementing visibility routines in table AMs. */ diff --git a/src/include/utils/snapshot.h b/src/include/utils/snapshot.h index 4796edb63aa..35b1f05bea6 100644 --- a/src/include/utils/snapshot.h +++ b/src/include/utils/snapshot.h @@ -192,6 +192,12 @@ typedef struct SnapshotData */ uint32 speculativeToken; + /* + * For SNAPSHOT_NON_VACUUMABLE (and hopefully more in the future) this is + * used to determine whether row could be vacuumed. + */ + struct GlobalVisState *vistest; + /* * Book-keeping information, used by the snapshot manager */ diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c index 8ae4fd95a7b..9cd6638df62 100644 --- a/src/backend/access/gin/ginvacuum.c +++ b/src/backend/access/gin/ginvacuum.c @@ -793,3 +793,29 @@ ginvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) return stats; } + +/* + * Return whether Page can safely be recycled. + */ +bool +GinPageIsRecyclable(Page page) +{ + TransactionId delete_xid; + + if (PageIsNew(page)) + return true; + + if (!GinPageIsDeleted(page)) + return false; + + delete_xid = GinPageGetDeleteXid(page); + + if (!TransactionIdIsValid(delete_xid)) + return true; + + /* + * If no backend still could view delete_xid as in running, all scans + * concurrent with ginDeletePage() must have finished. + */ + return GlobalVisCheckRemovableXid(NULL, delete_xid); +} diff --git a/src/backend/access/gist/gistutil.c b/src/backend/access/gist/gistutil.c index 765329bbcd4..bfda7fbe3d5 100644 --- a/src/backend/access/gist/gistutil.c +++ b/src/backend/access/gist/gistutil.c @@ -891,15 +891,13 @@ gistPageRecyclable(Page page) * As long as that can happen, we must keep the deleted page around as * a tombstone. * - * Compare the deletion XID with RecentGlobalXmin. If deleteXid < - * RecentGlobalXmin, then no scan that's still in progress could have + * For that check if the deletion XID could still be visible to + * anyone. If not, then no scan that's still in progress could have * seen its downlink, and we can recycle it. */ FullTransactionId deletexid_full = GistPageGetDeleteXid(page); - FullTransactionId recentxmin_full = GetFullRecentGlobalXmin(); - if (FullTransactionIdPrecedes(deletexid_full, recentxmin_full)) - return true; + return GlobalVisIsRemovableFullXid(NULL, deletexid_full); } return false; } diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index 7b5d1e98b70..a63b05388c5 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -387,11 +387,11 @@ gistRedoPageReuse(XLogReaderState *record) * PAGE_REUSE records exist to provide a conflict point when we reuse * pages in the index via the FSM. That's all they do though. * - * latestRemovedXid was the page's deleteXid. The deleteXid < - * RecentGlobalXmin test in gistPageRecyclable() conceptually mirrors the - * pgxact->xmin > limitXmin test in GetConflictingVirtualXIDs(). - * Consequently, one XID value achieves the same exclusion effect on - * primary and standby. + * latestRemovedXid was the page's deleteXid. The + * GlobalVisIsRemovableFullXid(deleteXid) test in gistPageRecyclable() + * conceptually mirrors the pgxact->xmin > limitXmin test in + * GetConflictingVirtualXIDs(). Consequently, one XID value achieves the + * same exclusion effect on primary and standby. */ if (InHotStandby) { diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index 00169006fb1..0a89e741a15 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -1517,6 +1517,7 @@ heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer, bool at_chain_start; bool valid; bool skip; + GlobalVisState *vistest = NULL; /* If this is not the first call, previous call returned a (live!) tuple */ if (all_dead) @@ -1527,7 +1528,8 @@ heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer, at_chain_start = first_call; skip = !first_call; - Assert(TransactionIdIsValid(RecentGlobalXmin)); + /* XXX: we should assert that a snapshot is pushed or registered */ + Assert(TransactionIdIsValid(RecentXmin)); Assert(BufferGetBlockNumber(buffer) == blkno); /* Scan through possible multiple members of HOT-chain */ @@ -1616,9 +1618,14 @@ heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer, * Note: if you change the criterion here for what is "dead", fix the * planner's get_actual_variable_range() function to match. */ - if (all_dead && *all_dead && - !HeapTupleIsSurelyDead(heapTuple, RecentGlobalXmin)) - *all_dead = false; + if (all_dead && *all_dead) + { + if (!vistest) + vistest = GlobalVisTestFor(relation); + + if (!HeapTupleIsSurelyDead(vistest, heapTuple)) + *all_dead = false; + } /* * Check to see if HOT chain continues past this tuple; if so fetch diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 267a6ee25a7..e3e41fb7516 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -1203,7 +1203,7 @@ heapam_index_build_range_scan(Relation heapRelation, /* okay to ignore lazy VACUUMs here */ if (!IsBootstrapProcessingMode() && !indexInfo->ii_Concurrent) - OldestXmin = GetOldestXmin(heapRelation, PROCARRAY_FLAGS_VACUUM); + OldestXmin = GetOldestNonRemovableTransactionId(heapRelation); if (!scan) { @@ -1244,6 +1244,17 @@ heapam_index_build_range_scan(Relation heapRelation, hscan = (HeapScanDesc) scan; + /* + * Must have called GetOldestNonRemovableTransactionId() if using + * SnapshotAny. Shouldn't have for an MVCC snapshot. (It's especially + * worth checking this for parallel builds, since ambuild routines that + * support parallel builds must work these details out for themselves.) + */ + Assert(snapshot == SnapshotAny || IsMVCCSnapshot(snapshot)); + Assert(snapshot == SnapshotAny ? TransactionIdIsValid(OldestXmin) : + !TransactionIdIsValid(OldestXmin)); + Assert(snapshot == SnapshotAny || !anyvisible); + /* Publish number of blocks to scan */ if (progress) { @@ -1263,17 +1274,6 @@ heapam_index_build_range_scan(Relation heapRelation, nblocks); } - /* - * Must call GetOldestXmin() with SnapshotAny. Should never call - * GetOldestXmin() with MVCC snapshot. (It's especially worth checking - * this for parallel builds, since ambuild routines that support parallel - * builds must work these details out for themselves.) - */ - Assert(snapshot == SnapshotAny || IsMVCCSnapshot(snapshot)); - Assert(snapshot == SnapshotAny ? TransactionIdIsValid(OldestXmin) : - !TransactionIdIsValid(OldestXmin)); - Assert(snapshot == SnapshotAny || !anyvisible); - /* set our scan endpoints */ if (!allow_sync) heap_setscanlimits(scan, start_blockno, numblocks); diff --git a/src/backend/access/heap/heapam_visibility.c b/src/backend/access/heap/heapam_visibility.c index c77128087cf..f117ee160a3 100644 --- a/src/backend/access/heap/heapam_visibility.c +++ b/src/backend/access/heap/heapam_visibility.c @@ -1154,19 +1154,56 @@ HeapTupleSatisfiesMVCC(HeapTuple htup, Snapshot snapshot, * we mainly want to know is if a tuple is potentially visible to *any* * running transaction. If so, it can't be removed yet by VACUUM. * - * OldestXmin is a cutoff XID (obtained from GetOldestXmin()). Tuples - * deleted by XIDs >= OldestXmin are deemed "recently dead"; they might - * still be visible to some open transaction, so we can't remove them, - * even if we see that the deleting transaction has committed. + * OldestXmin is a cutoff XID (obtained from + * GetOldestNonRemovableTransactionId()). Tuples deleted by XIDs >= + * OldestXmin are deemed "recently dead"; they might still be visible to some + * open transaction, so we can't remove them, even if we see that the deleting + * transaction has committed. */ HTSV_Result HeapTupleSatisfiesVacuum(HeapTuple htup, TransactionId OldestXmin, Buffer buffer) +{ + TransactionId dead_after = InvalidTransactionId; + HTSV_Result res; + + res = HeapTupleSatisfiesVacuumHorizon(htup, buffer, &dead_after); + + if (res == HEAPTUPLE_RECENTLY_DEAD) + { + Assert(TransactionIdIsValid(dead_after)); + + if (TransactionIdPrecedes(dead_after, OldestXmin)) + res = HEAPTUPLE_DEAD; + } + else + Assert(!TransactionIdIsValid(dead_after)); + + return res; +} + +/* + * Work horse for HeapTupleSatisfiesVacuum and similar routines. + * + * In contrast to HeapTupleSatisfiesVacuum this routine, when encountering a + * tuple that could still be visible to some backend, stores the xid that + * needs to be compared with the horizon in *dead_after, and returns + * HEAPTUPLE_RECENTLY_DEAD. The caller then can perform the comparison with + * the horizon. This is e.g. useful when comparing with different horizons. + * + * Note: HEAPTUPLE_DEAD can still be returned here, e.g. if the inserting + * transaction aborted. + */ +HTSV_Result +HeapTupleSatisfiesVacuumHorizon(HeapTuple htup, Buffer buffer, TransactionId *dead_after) { HeapTupleHeader tuple = htup->t_data; Assert(ItemPointerIsValid(&htup->t_self)); Assert(htup->t_tableOid != InvalidOid); + Assert(dead_after != NULL); + + *dead_after = InvalidTransactionId; /* * Has inserting transaction committed? @@ -1323,17 +1360,15 @@ HeapTupleSatisfiesVacuum(HeapTuple htup, TransactionId OldestXmin, else if (TransactionIdDidCommit(xmax)) { /* - * The multixact might still be running due to lockers. If the - * updater is below the xid horizon, we have to return DEAD - * regardless -- otherwise we could end up with a tuple where the - * updater has to be removed due to the horizon, but is not pruned - * away. It's not a problem to prune that tuple, because any - * remaining lockers will also be present in newer tuple versions. + * The multixact might still be running due to lockers. Need to + * allow for pruning if below the xid horizon regardless -- + * otherwise we could end up with a tuple where the updater has to + * be removed due to the horizon, but is not pruned away. It's + * not a problem to prune that tuple, because any remaining + * lockers will also be present in newer tuple versions. */ - if (!TransactionIdPrecedes(xmax, OldestXmin)) - return HEAPTUPLE_RECENTLY_DEAD; - - return HEAPTUPLE_DEAD; + *dead_after = xmax; + return HEAPTUPLE_RECENTLY_DEAD; } else if (!MultiXactIdIsRunning(HeapTupleHeaderGetRawXmax(tuple), false)) { @@ -1372,14 +1407,11 @@ HeapTupleSatisfiesVacuum(HeapTuple htup, TransactionId OldestXmin, } /* - * Deleter committed, but perhaps it was recent enough that some open - * transactions could still see the tuple. + * Deleter committed, allow caller to check if it was recent enough that + * some open transactions could still see the tuple. */ - if (!TransactionIdPrecedes(HeapTupleHeaderGetRawXmax(tuple), OldestXmin)) - return HEAPTUPLE_RECENTLY_DEAD; - - /* Otherwise, it's dead and removable */ - return HEAPTUPLE_DEAD; + *dead_after = HeapTupleHeaderGetRawXmax(tuple); + return HEAPTUPLE_RECENTLY_DEAD; } @@ -1418,7 +1450,7 @@ HeapTupleSatisfiesNonVacuumable(HeapTuple htup, Snapshot snapshot, * if the tuple is removable. */ bool -HeapTupleIsSurelyDead(HeapTuple htup, TransactionId OldestXmin) +HeapTupleIsSurelyDead(GlobalVisState *vistest, HeapTuple htup) { HeapTupleHeader tuple = htup->t_data; @@ -1459,7 +1491,8 @@ HeapTupleIsSurelyDead(HeapTuple htup, TransactionId OldestXmin) return false; /* Deleter committed, so tuple is dead if the XID is old enough. */ - return TransactionIdPrecedes(HeapTupleHeaderGetRawXmax(tuple), OldestXmin); + return GlobalVisTestIsRemovableXid(vistest, + HeapTupleHeaderGetRawXmax(tuple)); } /* diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 256df4de105..00a3cb106aa 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -23,12 +23,30 @@ #include "miscadmin.h" #include "pgstat.h" #include "storage/bufmgr.h" +#include "utils/snapmgr.h" #include "utils/rel.h" #include "utils/snapmgr.h" /* Working data for heap_page_prune and subroutines */ typedef struct { + Relation rel; + + /* tuple visibility test, initialized for the relation */ + GlobalVisState *vistest; + + /* + * Thresholds set by TransactionIdLimitedForOldSnapshots() if they have + * been computed (done on demand, and only if + * OldSnapshotThresholdActive()). The first time a tuple is about to be + * removed based on the limited horizon, old_snap_used is set to true, and + * SetOldSnapshotThresholdTimestamp() is called. See + * heap_prune_satisfies_vacuum(). + */ + TimestampTz old_snap_ts; + TransactionId old_snap_xmin; + bool old_snap_used; + TransactionId new_prune_xid; /* new prune hint value for page */ TransactionId latestRemovedXid; /* latest xid to be removed by this prune */ int nredirected; /* numbers of entries in arrays below */ @@ -43,9 +61,8 @@ typedef struct } PruneState; /* Local functions */ -static int heap_prune_chain(Relation relation, Buffer buffer, +static int heap_prune_chain(Buffer buffer, OffsetNumber rootoffnum, - TransactionId OldestXmin, PruneState *prstate); static void heap_prune_record_prunable(PruneState *prstate, TransactionId xid); static void heap_prune_record_redirect(PruneState *prstate, @@ -65,16 +82,16 @@ static void heap_prune_record_unused(PruneState *prstate, OffsetNumber offnum); * if there's not any use in pruning. * * Caller must have pin on the buffer, and must *not* have a lock on it. - * - * OldestXmin is the cutoff XID used to distinguish whether tuples are DEAD - * or RECENTLY_DEAD (see HeapTupleSatisfiesVacuum). */ void heap_page_prune_opt(Relation relation, Buffer buffer) { Page page = BufferGetPage(buffer); + TransactionId prune_xid; + GlobalVisState *vistest; + TransactionId limited_xmin = InvalidTransactionId; + TimestampTz limited_ts = 0; Size minfree; - TransactionId OldestXmin; /* * We can't write WAL in recovery mode, so there's no point trying to @@ -85,37 +102,55 @@ heap_page_prune_opt(Relation relation, Buffer buffer) return; /* - * Use the appropriate xmin horizon for this relation. If it's a proper - * catalog relation or a user defined, additional, catalog relation, we - * need to use the horizon that includes slots, otherwise the data-only - * horizon can be used. Note that the toast relation of user defined - * relations are *not* considered catalog relations. + * XXX: Magic to keep old_snapshot_threshold tests appear "working". They + * currently are broken, and discussion of what to do about them is + * ongoing. See + * https://www.postgresql.org/message-id/20200403001235.e6jfdll3gh2ygbuc%40alap3.anarazel.de + */ + if (old_snapshot_threshold == 0) + SnapshotTooOldMagicForTest(); + + /* + * First check whether there's any chance there's something to prune, + * determining the appropriate horizon is a waste if there's no prune_xid + * (i.e. no updates/deletes left potentially dead tuples around). + */ + prune_xid = ((PageHeader) page)->pd_prune_xid; + if (!TransactionIdIsValid(prune_xid)) + return; + + /* + * Check whether prune_xid indicates that there may be dead rows that can + * be cleaned up. * - * It is OK to apply the old snapshot limit before acquiring the cleanup + * It is OK to check the old snapshot limit before acquiring the cleanup * lock because the worst that can happen is that we are not quite as * aggressive about the cleanup (by however many transaction IDs are * consumed between this point and acquiring the lock). This allows us to * save significant overhead in the case where the page is found not to be * prunable. - */ - if (IsCatalogRelation(relation) || - RelationIsAccessibleInLogicalDecoding(relation)) - OldestXmin = RecentGlobalXmin; - else - OldestXmin = - TransactionIdLimitedForOldSnapshots(RecentGlobalDataXmin, - relation); - - Assert(TransactionIdIsValid(OldestXmin)); - - /* - * Let's see if we really need pruning. * - * Forget it if page is not hinted to contain something prunable that's - * older than OldestXmin. + * Even if old_snapshot_threshold is set, we first check whether the page + * can be pruned without. Both because + * TransactionIdLimitedForOldSnapshots() is not cheap, and because not + * unnecessarily relying on old_snapshot_threshold avoids causing + * conflicts. */ - if (!PageIsPrunable(page, OldestXmin)) - return; + vistest = GlobalVisTestFor(relation); + + if (!GlobalVisTestIsRemovableXid(vistest, prune_xid)) + { + if (!OldSnapshotThresholdActive()) + return; + + if (!TransactionIdLimitedForOldSnapshots(GlobalVisTestNonRemovableHorizon(vistest), + relation, + &limited_xmin, &limited_ts)) + return; + + if (!TransactionIdPrecedes(prune_xid, limited_xmin)) + return; + } /* * We prune when a previous UPDATE failed to find enough space on the page @@ -151,7 +186,9 @@ heap_page_prune_opt(Relation relation, Buffer buffer) * needed */ /* OK to prune */ - (void) heap_page_prune(relation, buffer, OldestXmin, true, &ignore); + (void) heap_page_prune(relation, buffer, vistest, + limited_xmin, limited_ts, + true, &ignore); } /* And release buffer lock */ @@ -165,8 +202,11 @@ heap_page_prune_opt(Relation relation, Buffer buffer) * * Caller must have pin and buffer cleanup lock on the page. * - * OldestXmin is the cutoff XID used to distinguish whether tuples are DEAD - * or RECENTLY_DEAD (see HeapTupleSatisfiesVacuum). + * vistest is used to distinguish whether tuples are DEAD or RECENTLY_DEAD + * (see heap_prune_satisfies_vacuum and + * HeapTupleSatisfiesVacuum). old_snap_xmin / old_snap_ts need to + * either have been set by TransactionIdLimitedForOldSnapshots, or + * InvalidTransactionId/0 respectively. * * If report_stats is true then we send the number of reclaimed heap-only * tuples to pgstats. (This must be false during vacuum, since vacuum will @@ -177,7 +217,10 @@ heap_page_prune_opt(Relation relation, Buffer buffer) * latestRemovedXid. */ int -heap_page_prune(Relation relation, Buffer buffer, TransactionId OldestXmin, +heap_page_prune(Relation relation, Buffer buffer, + GlobalVisState *vistest, + TransactionId old_snap_xmin, + TimestampTz old_snap_ts, bool report_stats, TransactionId *latestRemovedXid) { int ndeleted = 0; @@ -198,6 +241,11 @@ heap_page_prune(Relation relation, Buffer buffer, TransactionId OldestXmin, * initialize the rest of our working state. */ prstate.new_prune_xid = InvalidTransactionId; + prstate.rel = relation; + prstate.vistest = vistest; + prstate.old_snap_xmin = old_snap_xmin; + prstate.old_snap_ts = old_snap_ts; + prstate.old_snap_used = false; prstate.latestRemovedXid = *latestRemovedXid; prstate.nredirected = prstate.ndead = prstate.nunused = 0; memset(prstate.marked, 0, sizeof(prstate.marked)); @@ -220,9 +268,7 @@ heap_page_prune(Relation relation, Buffer buffer, TransactionId OldestXmin, continue; /* Process this item or chain of items */ - ndeleted += heap_prune_chain(relation, buffer, offnum, - OldestXmin, - &prstate); + ndeleted += heap_prune_chain(buffer, offnum, &prstate); } /* Any error while applying the changes is critical */ @@ -323,6 +369,85 @@ heap_page_prune(Relation relation, Buffer buffer, TransactionId OldestXmin, } +/* + * Perform visiblity checks for heap pruning. + * + * This is more complicated than just using GlobalVisTestIsRemovableXid() + * because of old_snapshot_threshold. We only want to increase the threshold + * that triggers errors for old snapshots when we actually decide to remove a + * row based on the limited horizon. + * + * Due to its cost we also only want to call + * TransactionIdLimitedForOldSnapshots() if necessary, i.e. we might not have + * done so in heap_hot_prune_opt() if pd_prune_xid was old enough. But we + * still want to be able to remove rows that are too new to be removed + * according to prstate->vistest, but that can be removed based on + * old_snapshot_threshold. So we call TransactionIdLimitedForOldSnapshots() on + * demand in here, if appropriate. + */ +static HTSV_Result +heap_prune_satisfies_vacuum(PruneState *prstate, HeapTuple tup, Buffer buffer) +{ + HTSV_Result res; + TransactionId dead_after; + + res = HeapTupleSatisfiesVacuumHorizon(tup, buffer, &dead_after); + + if (res != HEAPTUPLE_RECENTLY_DEAD) + return res; + + /* + * If we are already relying on the limited xmin, there is no need to + * delay doing so anymore. + */ + if (prstate->old_snap_used) + { + Assert(TransactionIdIsValid(prstate->old_snap_xmin)); + + if (TransactionIdPrecedes(dead_after, prstate->old_snap_xmin)) + res = HEAPTUPLE_DEAD; + return res; + } + + /* + * First check if GlobalVisTestIsRemovableXid() is sufficient to find the + * row dead. If not, and old_snapshot_threshold is enabled, try to use the + * lowered horizon. + */ + if (GlobalVisTestIsRemovableXid(prstate->vistest, dead_after)) + res = HEAPTUPLE_DEAD; + else if (OldSnapshotThresholdActive()) + { + /* haven't determined limited horizon yet, requests */ + if (!TransactionIdIsValid(prstate->old_snap_xmin)) + { + TransactionId horizon = + GlobalVisTestNonRemovableHorizon(prstate->vistest); + + TransactionIdLimitedForOldSnapshots(horizon, prstate->rel, + &prstate->old_snap_xmin, + &prstate->old_snap_ts); + } + + if (TransactionIdIsValid(prstate->old_snap_xmin) && + TransactionIdPrecedes(dead_after, prstate->old_snap_xmin)) + { + /* + * About to remove row based on snapshot_too_old. Need to raise + * the threshold so problematic accesses would error. + */ + Assert(!prstate->old_snap_used); + SetOldSnapshotThresholdTimestamp(prstate->old_snap_ts, + prstate->old_snap_xmin); + prstate->old_snap_used = true; + res = HEAPTUPLE_DEAD; + } + } + + return res; +} + + /* * Prune specified line pointer or a HOT chain originating at line pointer. * @@ -349,9 +474,7 @@ heap_page_prune(Relation relation, Buffer buffer, TransactionId OldestXmin, * Returns the number of tuples (to be) deleted from the page. */ static int -heap_prune_chain(Relation relation, Buffer buffer, OffsetNumber rootoffnum, - TransactionId OldestXmin, - PruneState *prstate) +heap_prune_chain(Buffer buffer, OffsetNumber rootoffnum, PruneState *prstate) { int ndeleted = 0; Page dp = (Page) BufferGetPage(buffer); @@ -366,7 +489,7 @@ heap_prune_chain(Relation relation, Buffer buffer, OffsetNumber rootoffnum, i; HeapTupleData tup; - tup.t_tableOid = RelationGetRelid(relation); + tup.t_tableOid = RelationGetRelid(prstate->rel); rootlp = PageGetItemId(dp, rootoffnum); @@ -401,7 +524,7 @@ heap_prune_chain(Relation relation, Buffer buffer, OffsetNumber rootoffnum, * either here or while following a chain below. Whichever path * gets there first will mark the tuple unused. */ - if (HeapTupleSatisfiesVacuum(&tup, OldestXmin, buffer) + if (heap_prune_satisfies_vacuum(prstate, &tup, buffer) == HEAPTUPLE_DEAD && !HeapTupleHeaderIsHotUpdated(htup)) { heap_prune_record_unused(prstate, rootoffnum); @@ -485,7 +608,7 @@ heap_prune_chain(Relation relation, Buffer buffer, OffsetNumber rootoffnum, */ tupdead = recent_dead = false; - switch (HeapTupleSatisfiesVacuum(&tup, OldestXmin, buffer)) + switch (heap_prune_satisfies_vacuum(prstate, &tup, buffer)) { case HEAPTUPLE_DEAD: tupdead = true; diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c index 1bbc4598f75..44e2224dd55 100644 --- a/src/backend/access/heap/vacuumlazy.c +++ b/src/backend/access/heap/vacuumlazy.c @@ -788,6 +788,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats, PROGRESS_VACUUM_MAX_DEAD_TUPLES }; int64 initprog_val[3]; + GlobalVisState *vistest; pg_rusage_init(&ru0); @@ -816,6 +817,8 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats, vacrelstats->nonempty_pages = 0; vacrelstats->latestRemovedXid = InvalidTransactionId; + vistest = GlobalVisTestFor(onerel); + /* * Initialize state for a parallel vacuum. As of now, only one worker can * be used for an index, so we invoke parallelism only if there are at @@ -1239,7 +1242,8 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats, * * We count tuples removed by the pruning step as removed by VACUUM. */ - tups_vacuumed += heap_page_prune(onerel, buf, OldestXmin, false, + tups_vacuumed += heap_page_prune(onerel, buf, vistest, false, + InvalidTransactionId, 0, &vacrelstats->latestRemovedXid); /* @@ -1596,14 +1600,16 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats, } /* - * It's possible for the value returned by GetOldestXmin() to move - * backwards, so it's not wrong for us to see tuples that appear to - * not be visible to everyone yet, while PD_ALL_VISIBLE is already - * set. The real safe xmin value never moves backwards, but - * GetOldestXmin() is conservative and sometimes returns a value - * that's unnecessarily small, so if we see that contradiction it just - * means that the tuples that we think are not visible to everyone yet - * actually are, and the PD_ALL_VISIBLE flag is correct. + * It's possible for the value returned by + * GetOldestNonRemovableTransactionId() to move backwards, so it's not + * wrong for us to see tuples that appear to not be visible to + * everyone yet, while PD_ALL_VISIBLE is already set. The real safe + * xmin value never moves backwards, but + * GetOldestNonRemovableTransactionId() is conservative and sometimes + * returns a value that's unnecessarily small, so if we see that + * contradiction it just means that the tuples that we think are not + * visible to everyone yet actually are, and the PD_ALL_VISIBLE flag + * is correct. * * There should never be dead tuples on a page with PD_ALL_VISIBLE * set, however. diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c index 6b9750c244a..3fb8688f8f4 100644 --- a/src/backend/access/index/indexam.c +++ b/src/backend/access/index/indexam.c @@ -519,7 +519,8 @@ index_getnext_tid(IndexScanDesc scan, ScanDirection direction) SCAN_CHECKS; CHECK_SCAN_PROCEDURE(amgettuple); - Assert(TransactionIdIsValid(RecentGlobalXmin)); + /* XXX: we should assert that a snapshot is pushed or registered */ + Assert(TransactionIdIsValid(RecentXmin)); /* * The AM's amgettuple proc finds the next index entry matching the scan diff --git a/src/backend/access/nbtree/README b/src/backend/access/nbtree/README index abce31a5a96..781a8f1932d 100644 --- a/src/backend/access/nbtree/README +++ b/src/backend/access/nbtree/README @@ -342,9 +342,9 @@ snapshots and registered snapshots as of the deletion are gone; which is overly strong, but is simple to implement within Postgres. When marked dead, a deleted page is labeled with the next-transaction counter value. VACUUM can reclaim the page for re-use when this transaction number is -older than RecentGlobalXmin. As collateral damage, this implementation -also waits for running XIDs with no snapshots and for snapshots taken -until the next transaction to allocate an XID commits. +guaranteed to be "visible to everyone". As collateral damage, this +implementation also waits for running XIDs with no snapshots and for +snapshots taken until the next transaction to allocate an XID commits. Reclaiming a page doesn't actually change its state on disk --- we simply record it in the shared-memory free space map, from which it will be @@ -411,8 +411,8 @@ page and also the correct place to hold the current value. We can avoid the cost of walking down the tree in such common cases. The optimization works on the assumption that there can only be one -non-ignorable leaf rightmost page, and so even a RecentGlobalXmin style -interlock isn't required. We cannot fail to detect that our hint was +non-ignorable leaf rightmost page, and so not even a visible-to-everyone +style interlock required. We cannot fail to detect that our hint was invalidated, because there can only be one such page in the B-Tree at any time. It's possible that the page will be deleted and recycled without a backend's cached page also being detected as invalidated, but diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index d5db9aaa3a1..74be3807bb7 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -1097,7 +1097,7 @@ _bt_page_recyclable(Page page) */ opaque = (BTPageOpaque) PageGetSpecialPointer(page); if (P_ISDELETED(opaque) && - TransactionIdPrecedes(opaque->btpo.xact, RecentGlobalXmin)) + GlobalVisCheckRemovableXid(NULL, opaque->btpo.xact)) return true; return false; } @@ -2318,7 +2318,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * updated links to the target, ReadNewTransactionId() suffices as an * upper bound. Any scan having retained a now-stale link is advertising * in its PGXACT an xmin less than or equal to the value we read here. It - * will continue to do so, holding back RecentGlobalXmin, for the duration + * will continue to do so, holding back the xmin horizon, for the duration * of that scan. */ page = BufferGetPage(buf); diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c index 49a8a9708e3..8fa6ac7296b 100644 --- a/src/backend/access/nbtree/nbtree.c +++ b/src/backend/access/nbtree/nbtree.c @@ -808,6 +808,12 @@ _bt_vacuum_needs_cleanup(IndexVacuumInfo *info) metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); + /* + * XXX: If IndexVacuumInfo contained the heap relation, we could be more + * aggressive about vacuuming non catalog relations by passing the table + * to GlobalVisCheckRemovableXid(). + */ + if (metad->btm_version < BTREE_NOVAC_VERSION) { /* @@ -817,13 +823,12 @@ _bt_vacuum_needs_cleanup(IndexVacuumInfo *info) result = true; } else if (TransactionIdIsValid(metad->btm_oldest_btpo_xact) && - TransactionIdPrecedes(metad->btm_oldest_btpo_xact, - RecentGlobalXmin)) + GlobalVisCheckRemovableXid(NULL, metad->btm_oldest_btpo_xact)) { /* * If any oldest btpo.xact from a previously deleted page in the index - * is older than RecentGlobalXmin, then at least one deleted page can - * be recycled -- don't skip cleanup. + * is visible to everyone, then at least one deleted page can be + * recycled -- don't skip cleanup. */ result = true; } @@ -1276,14 +1281,13 @@ backtrack: * own conflict now.) * * Backends with snapshots acquired after a VACUUM starts but - * before it finishes could have a RecentGlobalXmin with a - * later xid than the VACUUM's OldestXmin cutoff. These - * backends might happen to opportunistically mark some index - * tuples LP_DEAD before we reach them, even though they may - * be after our cutoff. We don't try to kill these "extra" - * index tuples in _bt_delitems_vacuum(). This keep things - * simple, and allows us to always avoid generating our own - * conflicts. + * before it finishes could have visibility cutoff with a + * later xid than VACUUM's OldestXmin cutoff. These backends + * might happen to opportunistically mark some index tuples + * LP_DEAD before we reach them, even though they may be after + * our cutoff. We don't try to kill these "extra" index + * tuples in _bt_delitems_vacuum(). This keep things simple, + * and allows us to always avoid generating our own conflicts. */ Assert(!BTreeTupleIsPivot(itup)); if (!BTreeTupleIsPosting(itup)) diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index dbec58d5249..bda9be23489 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -948,11 +948,11 @@ btree_xlog_reuse_page(XLogReaderState *record) * Btree reuse_page records exist to provide a conflict point when we * reuse pages in the index via the FSM. That's all they do though. * - * latestRemovedXid was the page's btpo.xact. The btpo.xact < - * RecentGlobalXmin test in _bt_page_recyclable() conceptually mirrors the - * pgxact->xmin > limitXmin test in GetConflictingVirtualXIDs(). - * Consequently, one XID value achieves the same exclusion effect on - * primary and standby. + * latestRemovedXid was the page's btpo.xact. The + * GlobalVisCheckRemovableXid test in _bt_page_recyclable() conceptually + * mirrors the pgxact->xmin > limitXmin test in + * GetConflictingVirtualXIDs(). Consequently, one XID value achieves the + * same exclusion effect on primary and standby. */ if (InHotStandby) { diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c index bd98707f3c0..e1c58933f97 100644 --- a/src/backend/access/spgist/spgvacuum.c +++ b/src/backend/access/spgist/spgvacuum.c @@ -501,10 +501,14 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) OffsetNumber itemToPlaceholder[MaxIndexTuplesPerPage]; OffsetNumber itemnos[MaxIndexTuplesPerPage]; spgxlogVacuumRedirect xlrec; + GlobalVisState *vistest; xlrec.nToPlaceholder = 0; xlrec.newestRedirectXid = InvalidTransactionId; + /* XXX: providing heap relation would allow more pruning */ + vistest = GlobalVisTestFor(NULL); + START_CRIT_SECTION(); /* @@ -521,7 +525,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) dt = (SpGistDeadTuple) PageGetItem(page, PageGetItemId(page, i)); if (dt->tupstate == SPGIST_REDIRECT && - TransactionIdPrecedes(dt->xid, RecentGlobalXmin)) + GlobalVisTestIsRemovableXid(vistest, dt->xid)) { dt->tupstate = SPGIST_PLACEHOLDER; Assert(opaque->nRedirection > 0); diff --git a/src/backend/access/transam/README b/src/backend/access/transam/README index eb9aac5fd39..fffe0783295 100644 --- a/src/backend/access/transam/README +++ b/src/backend/access/transam/README @@ -293,42 +293,50 @@ once, rather than assume they can read it multiple times and get the same answer each time. (Use volatile-qualified pointers when doing this, to ensure that the C compiler does exactly what you tell it to.) -Another important activity that uses the shared ProcArray is GetOldestXmin, -which must determine a lower bound for the oldest xmin of any active MVCC -snapshot, system-wide. Each individual backend advertises the smallest -xmin of its own snapshots in MyPgXact->xmin, or zero if it currently has no -live snapshots (eg, if it's between transactions or hasn't yet set a -snapshot for a new transaction). GetOldestXmin takes the MIN() of the -valid xmin fields. It does this with only shared lock on ProcArrayLock, -which means there is a potential race condition against other backends -doing GetSnapshotData concurrently: we must be certain that a concurrent -backend that is about to set its xmin does not compute an xmin less than -what GetOldestXmin returns. We ensure that by including all the active -XIDs into the MIN() calculation, along with the valid xmins. The rule that -transactions can't exit without taking exclusive ProcArrayLock ensures that -concurrent holders of shared ProcArrayLock will compute the same minimum of -currently-active XIDs: no xact, in particular not the oldest, can exit -while we hold shared ProcArrayLock. So GetOldestXmin's view of the minimum -active XID will be the same as that of any concurrent GetSnapshotData, and -so it can't produce an overestimate. If there is no active transaction at -all, GetOldestXmin returns latestCompletedXid + 1, which is a lower bound -for the xmin that might be computed by concurrent or later GetSnapshotData -calls. (We know that no XID less than this could be about to appear in -the ProcArray, because of the XidGenLock interlock discussed above.) +Another important activity that uses the shared ProcArray is +ComputeXidHorizons, which must determine a lower bound for the oldest xmin +of any active MVCC snapshot, system-wide. Each individual backend +advertises the smallest xmin of its own snapshots in MyPgXact->xmin, or zero +if it currently has no live snapshots (eg, if it's between transactions or +hasn't yet set a snapshot for a new transaction). ComputeXidHorizons takes +the MIN() of the valid xmin fields. It does this with only shared lock on +ProcArrayLock, which means there is a potential race condition against other +backends doing GetSnapshotData concurrently: we must be certain that a +concurrent backend that is about to set its xmin does not compute an xmin +less than what ComputeXidHorizons determines. We ensure that by including +all the active XIDs into the MIN() calculation, along with the valid xmins. +The rule that transactions can't exit without taking exclusive ProcArrayLock +ensures that concurrent holders of shared ProcArrayLock will compute the +same minimum of currently-active XIDs: no xact, in particular not the +oldest, can exit while we hold shared ProcArrayLock. So +ComputeXidHorizons's view of the minimum active XID will be the same as that +of any concurrent GetSnapshotData, and so it can't produce an overestimate. +If there is no active transaction at all, ComputeXidHorizons uses +latestCompletedXid + 1, which is a lower bound for the xmin that might +be computed by concurrent or later GetSnapshotData calls. (We know that no +XID less than this could be about to appear in the ProcArray, because of the +XidGenLock interlock discussed above.) -GetSnapshotData also performs an oldest-xmin calculation (which had better -match GetOldestXmin's) and stores that into RecentGlobalXmin, which is used -for some tuple age cutoff checks where a fresh call of GetOldestXmin seems -too expensive. Note that while it is certain that two concurrent -executions of GetSnapshotData will compute the same xmin for their own -snapshots, as argued above, it is not certain that they will arrive at the -same estimate of RecentGlobalXmin. This is because we allow XID-less -transactions to clear their MyPgXact->xmin asynchronously (without taking -ProcArrayLock), so one execution might see what had been the oldest xmin, -and another not. This is OK since RecentGlobalXmin need only be a valid -lower bound. As noted above, we are already assuming that fetch/store -of the xid fields is atomic, so assuming it for xmin as well is no extra -risk. +As GetSnapshotData is performance critical, it does not perform an accurate +oldest-xmin calculation (it used to, until v13). The contents of a snapshot +only depend on the xids of other backends, not their xmin. As backend's xmin +changes much more often than its xid, having GetSnapshotData look at xmins +can lead to a lot of unnecessary cacheline ping-pong. Instead +GetSnapshotData updates approximate thresholds (one that guarantees that all +deleted rows older than it can be removed, another determining that deleted +rows newer than it can not be removed). GlobalVisTest* uses those threshold +to make invisibility decision, falling back to ComputeXidHorizons if +necessary. + +Note that while it is certain that two concurrent executions of +GetSnapshotData will compute the same xmin for their own snapshots, there is +no such guarantee for the horizons computed by ComputeXidHorizons. This is +because we allow XID-less transactions to clear their MyPgXact->xmin +asynchronously (without taking ProcArrayLock), so one execution might see +what had been the oldest xmin, and another not. This is OK since the +thresholds need only be a valid lower bound. As noted above, we are already +assuming that fetch/store of the xid fields is atomic, so assuming it for +xmin as well is no extra risk. pg_xact and pg_subtrans diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 8f72faee82c..09c01ed4ae4 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -9096,7 +9096,7 @@ CreateCheckPoint(int flags) * StartupSUBTRANS hasn't been called yet. */ if (!RecoveryInProgress()) - TruncateSUBTRANS(GetOldestXmin(NULL, PROCARRAY_FLAGS_DEFAULT)); + TruncateSUBTRANS(GetOldestTransactionIdConsideredRunning()); /* Real work is done, but log and update stats before releasing lock. */ LogCheckpointEnd(false); @@ -9456,7 +9456,7 @@ CreateRestartPoint(int flags) * this because StartupSUBTRANS hasn't been called yet. */ if (EnableHotStandby) - TruncateSUBTRANS(GetOldestXmin(NULL, PROCARRAY_FLAGS_DEFAULT)); + TruncateSUBTRANS(GetOldestTransactionIdConsideredRunning()); /* Real work is done, but log and update before releasing lock. */ LogCheckpointEnd(true); diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index e0fa73ba790..8af12b5c6b2 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -1045,7 +1045,7 @@ acquire_sample_rows(Relation onerel, int elevel, totalblocks = RelationGetNumberOfBlocks(onerel); /* Need a cutoff xmin for HeapTupleSatisfiesVacuum */ - OldestXmin = GetOldestXmin(onerel, PROCARRAY_FLAGS_VACUUM); + OldestXmin = GetOldestNonRemovableTransactionId(onerel); /* Prepare for sampling block numbers */ nblocks = BlockSampler_Init(&bs, totalblocks, targrows, random()); diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index 576c7e63e99..22228f5684f 100644 --- a/src/backend/commands/vacuum.c +++ b/src/backend/commands/vacuum.c @@ -955,8 +955,25 @@ vacuum_set_xid_limits(Relation rel, * working on a particular table at any time, and that each vacuum is * always an independent transaction. */ - *oldestXmin = - TransactionIdLimitedForOldSnapshots(GetOldestXmin(rel, PROCARRAY_FLAGS_VACUUM), rel); + *oldestXmin = GetOldestNonRemovableTransactionId(rel); + + if (OldSnapshotThresholdActive()) + { + TransactionId limit_xmin; + TimestampTz limit_ts; + + if (TransactionIdLimitedForOldSnapshots(*oldestXmin, rel, &limit_xmin, &limit_ts)) + { + /* + * TODO: We should only set the threshold if we are pruning on the + * basis of the increased limits. Not as crucial here as it is for + * opportunistic pruning (which often happens at a much higher + * frequency), but would still be a significant improvement. + */ + SetOldSnapshotThresholdTimestamp(limit_ts, limit_xmin); + *oldestXmin = limit_xmin; + } + } Assert(TransactionIdIsNormal(*oldestXmin)); @@ -1345,12 +1362,13 @@ vac_update_datfrozenxid(void) bool dirty = false; /* - * Initialize the "min" calculation with GetOldestXmin, which is a - * reasonable approximation to the minimum relfrozenxid for not-yet- - * committed pg_class entries for new tables; see AddNewRelationTuple(). - * So we cannot produce a wrong minimum by starting with this. + * Initialize the "min" calculation with + * GetOldestNonRemovableTransactionId(), which is a reasonable + * approximation to the minimum relfrozenxid for not-yet-committed + * pg_class entries for new tables; see AddNewRelationTuple(). So we + * cannot produce a wrong minimum by starting with this. */ - newFrozenXid = GetOldestXmin(NULL, PROCARRAY_FLAGS_VACUUM); + newFrozenXid = GetOldestNonRemovableTransactionId(NULL); /* * Similarly, initialize the MultiXact "min" with the value that would be @@ -1681,8 +1699,9 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params) StartTransactionCommand(); /* - * Functions in indexes may want a snapshot set. Also, setting a snapshot - * ensures that RecentGlobalXmin is kept truly recent. + * Need to acquire a snapshot to prevent pg_subtrans from being truncated, + * cutoff xids in local memory wrapping around, and to have updated xmin + * horizons. */ PushActiveSnapshot(GetTransactionSnapshot()); @@ -1705,8 +1724,8 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params) * * Note: these flags remain set until CommitTransaction or * AbortTransaction. We don't want to clear them until we reset - * MyPgXact->xid/xmin, else OldestXmin might appear to go backwards, - * which is probably Not Good. + * MyPgXact->xid/xmin, otherwise GetOldestNonRemovableTransactionId() + * might appear to go backwards, which is probably Not Good. */ LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE); MyPgXact->vacuumFlags |= PROC_IN_VACUUM; diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c index 9c7d4b0c60e..ac97e28be19 100644 --- a/src/backend/postmaster/autovacuum.c +++ b/src/backend/postmaster/autovacuum.c @@ -1877,6 +1877,10 @@ get_database_list(void) * the secondary effect that it sets RecentGlobalXmin. (This is critical * for anything that reads heap pages, because HOT may decide to prune * them even if the process doesn't attempt to modify any tuples.) + * + * FIXME: This comment is inaccurate / the code buggy. A snapshot that is + * not pushed/active does not reliably prevent HOT pruning (->xmin could + * e.g. be cleared when cache invalidations are processed). */ StartTransactionCommand(); (void) GetTransactionSnapshot(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index ff985b9b24c..bdaf0312d63 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -122,6 +122,10 @@ get_subscription_list(void) * the secondary effect that it sets RecentGlobalXmin. (This is critical * for anything that reads heap pages, because HOT may decide to prune * them even if the process doesn't attempt to modify any tuples.) + * + * FIXME: This comment is inaccurate / the code buggy. A snapshot that is + * not pushed/active does not reliably prevent HOT pruning (->xmin could + * e.g. be cleared when cache invalidations are processed). */ StartTransactionCommand(); (void) GetTransactionSnapshot(); diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c index d5a9b568a68..7c11e1ab44c 100644 --- a/src/backend/replication/walreceiver.c +++ b/src/backend/replication/walreceiver.c @@ -1181,22 +1181,7 @@ XLogWalRcvSendHSFeedback(bool immed) */ if (hot_standby_feedback) { - TransactionId slot_xmin; - - /* - * Usually GetOldestXmin() would include both global replication slot - * xmin and catalog_xmin in its calculations, but we want to derive - * separate values for each of those. So we ask for an xmin that - * excludes the catalog_xmin. - */ - xmin = GetOldestXmin(NULL, - PROCARRAY_FLAGS_DEFAULT | PROCARRAY_SLOTS_XMIN); - - ProcArrayGetReplicationSlotXmin(&slot_xmin, &catalog_xmin); - - if (TransactionIdIsValid(slot_xmin) && - TransactionIdPrecedes(slot_xmin, xmin)) - xmin = slot_xmin; + GetReplicationHorizons(&xmin, &catalog_xmin); } else { diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index d13220c1400..460ca3f947f 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -2113,9 +2113,10 @@ ProcessStandbyHSFeedbackMessage(void) /* * Set the WalSender's xmin equal to the standby's requested xmin, so that - * the xmin will be taken into account by GetOldestXmin. This will hold - * back the removal of dead rows and thereby prevent the generation of - * cleanup conflicts on the standby server. + * the xmin will be taken into account by GetSnapshotData() / + * ComputeXidHorizons(). This will hold back the removal of dead rows and + * thereby prevent the generation of cleanup conflicts on the standby + * server. * * There is a small window for a race condition here: although we just * checked that feedbackXmin precedes nextXid, the nextXid could have @@ -2128,10 +2129,10 @@ ProcessStandbyHSFeedbackMessage(void) * own xmin would prevent nextXid from advancing so far. * * We don't bother taking the ProcArrayLock here. Setting the xmin field - * is assumed atomic, and there's no real need to prevent a concurrent - * GetOldestXmin. (If we're moving our xmin forward, this is obviously - * safe, and if we're moving it backwards, well, the data is at risk - * already since a VACUUM could have just finished calling GetOldestXmin.) + * is assumed atomic, and there's no real need to prevent concurrent + * horizon determinations. (If we're moving our xmin forward, this is + * obviously safe, and if we're moving it backwards, well, the data is at + * risk already since a VACUUM could already have determined the horizon.) * * If we're using a replication slot we reserve the xmin via that, * otherwise via the walsender's PGXACT entry. We can only track the diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c index 522518695ee..360e6e9da07 100644 --- a/src/backend/storage/ipc/procarray.c +++ b/src/backend/storage/ipc/procarray.c @@ -99,6 +99,142 @@ typedef struct ProcArrayStruct int pgprocnos[FLEXIBLE_ARRAY_MEMBER]; } ProcArrayStruct; +/* + * State for the GlobalVisTest* family of functions. Those functions can + * e.g. be used to decide if a deleted row can be removed without violating + * MVCC semantics: If the deleted row's xmax is not considered to be running + * by anyone, the row can be removed. + * + * To avoid slowing down GetSnapshotData(), we don't calculate a precise + * cutoff XID while building a snapshot (looking at the frequently changing + * xmins scales badly). Instead we compute two boundaries while building the + * snapshot: + * + * 1) definitely_needed, indicating that rows deleted by XIDs >= + * definitely_needed are definitely still visible. + * + * 2) maybe_needed, indicating that rows deleted by XIDs < maybe_needed can + * definitely be removed + * + * When testing an XID that falls in between the two (i.e. XID >= maybe_needed + * && XID < definitely_needed), the boundaries can be recomputed (using + * ComputeXidHorizons()) to get a more accurate answer. This is cheaper than + * maintaining an accurate value all the time. + * + * As it is not cheap to compute accurate boundaries, we limit the number of + * times that happens in short succession. See GlobalVisTestShouldUpdate(). + * + * + * There are three backend lifetime instances of this struct, optimized for + * different types of relations. As e.g. a normal user defined table in one + * database is inaccessible to backends connected to another database, a test + * specific to a relation can be more aggressive than a test for a shared + * relation. Currently we track three different states: + * + * 1) GlobalVisSharedRels, which only considers an XID's + * effects visible-to-everyone if neither snapshots in any database, nor a + * replication slot's xmin, nor a replication slot's catalog_xmin might + * still consider XID as running. + * + * 2) GlobalVisCatalogRels, which only considers an XID's + * effects visible-to-everyone if neither snapshots in the current + * database, nor a replication slot's xmin, nor a replication slot's + * catalog_xmin might still consider XID as running. + * + * I.e. the difference to GlobalVisSharedRels is that + * snapshot in other databases are ignored. + * + * 3) GlobalVisCatalogRels, which only considers an XID's + * effects visible-to-everyone if neither snapshots in the current + * database, nor a replication slot's xmin consider XID as running. + * + * I.e. the difference to GlobalVisCatalogRels is that + * replication slot's catalog_xmin is not taken into account. + * + * GlobalVisTestFor(relation) returns the appropriate state + * for the relation. + * + * The boundaries are FullTransactionIds instead of TransactionIds to avoid + * wraparound dangers. There e.g. would otherwise exist no procarray state to + * prevent maybe_needed to become old enough after the GetSnapshotData() + * call. + * + * The typedef is in the header. + */ +struct GlobalVisState +{ + /* XIDs >= are considered running by some backend */ + FullTransactionId definitely_needed; + + /* XIDs < are not considered to be running by any backend */ + FullTransactionId maybe_needed; +}; + +/* + * Result of ComputeXidHorizons(). + */ +typedef struct ComputeXidHorizonsResult +{ + /* + * The value of ShmemVariableCache->latestCompletedXid when + * ComputeXidHorizons() held ProcArrayLock. + */ + FullTransactionId latest_completed; + + /* + * The same for procArray->replication_slot_xmin and. + * procArray->replication_slot_catalog_xmin. + */ + TransactionId slot_xmin; + TransactionId slot_catalog_xmin; + + /* + * Oldest xid that any backend might still consider running. This needs to + * include processes running VACUUM, in contrast to the normal visibility + * cutoffs, as vacuum needs to be able to perform pg_subtrans lookups when + * determining visibility, but doesn't care about rows above its xmin to + * be removed. + * + * This likely should only be needed to determine whether pg_subtrans can + * be truncated. It currently includes the effects of replications slots, + * for historical reasons. But that could likely be changed. + */ + TransactionId oldest_considered_running; + + /* + * Oldest xid for which deleted tuples need to be retained in shared + * tables. + * + * This includes the effects of replications lots. If that's not desired, + * look at shared_oldest_nonremovable_raw; + */ + TransactionId shared_oldest_nonremovable; + + /* + * Oldest xid that may be necessary to retain in shared tables. This is + * the same as shared_oldest_nonremovable, except that is not affected by + * replication slot's catalog_xmin. + * + * This is mainly useful to be able to send the catalog_xmin to upstream + * streaming replication servers via hot_standby_feedback, so they can + * apply the limit only when accessing catalog tables. + */ + TransactionId shared_oldest_nonremovable_raw; + + /* + * Oldest xid for which deleted tuples need to be retained in non-shared + * catalog tables. + */ + TransactionId catalog_oldest_nonremovable; + + /* + * Oldest xid for which deleted tuples need to be retained in normal user + * defined tables. + */ + TransactionId data_oldest_nonremovable; +} ComputeXidHorizonsResult; + + static ProcArrayStruct *procArray; static PGPROC *allProcs; @@ -118,6 +254,22 @@ static TransactionId latestObservedXid = InvalidTransactionId; */ static TransactionId standbySnapshotPendingXmin; +/* + * State for visibility checks on different types of relations. See struct + * GlobalVisState for details. As shared, catalog, and user defined + * relations can have different horizons, one such state exists for each. + */ +static GlobalVisState GlobalVisSharedRels; +static GlobalVisState GlobalVisCatalogRels; +static GlobalVisState GlobalVisDataRels; + +/* + * This backend's RecentXmin at the last time the accurate xmin horizon was + * recomputed, or InvalidTransactionId if it has not. Used to limit how many + * times accurate horizons are recomputed. See GlobalVisTestShouldUpdate(). + */ +static TransactionId ComputeXidHorizonsResultLastXmin; + #ifdef XIDCACHE_DEBUG /* counters for XidCache measurement */ @@ -180,6 +332,7 @@ static void MaintainLatestCompletedXidRecovery(TransactionId latestXid); static inline FullTransactionId FullXidRelativeTo(FullTransactionId rel, TransactionId xid); +static void GlobalVisUpdateApply(ComputeXidHorizonsResult *horizons); /* * Report shared-memory space needed by CreateSharedProcArray. @@ -1302,159 +1455,191 @@ TransactionIdIsActive(TransactionId xid) /* - * GetOldestXmin -- returns oldest transaction that was running - * when any current transaction was started. + * Determine XID horizons. * - * If rel is NULL or a shared relation, all backends are considered, otherwise - * only backends running in this database are considered. + * This is used by wrapper functions like GetOldestNonRemovableTransactionId() + * (for VACUUM), GetReplicationHorizons() (for hot_standby_feedback), etc as + * well as "internally" by GlobalVisUpdate() (see comment above struct + * GlobalVisState). * - * The flags are used to ignore the backends in calculation when any of the - * corresponding flags is set. Typically, if you want to ignore ones with - * PROC_IN_VACUUM flag, you can use PROCARRAY_FLAGS_VACUUM. + * See the definition of ComputedXidHorizonsResult for the various computed + * horizons. * - * PROCARRAY_SLOTS_XMIN causes GetOldestXmin to ignore the xmin and - * catalog_xmin of any replication slots that exist in the system when - * calculating the oldest xmin. + * For VACUUM separate horizons (used to to decide which deleted tuples must + * be preserved), for shared and non-shared tables are computed. For shared + * relations backends in all databases must be considered, but for non-shared + * relations that's not required, since only backends in my own database could + * ever see the tuples in them. Also, we can ignore concurrently running lazy + * VACUUMs because (a) they must be working on other tables, and (b) they + * don't need to do snapshot-based lookups. * - * This is used by VACUUM to decide which deleted tuples must be preserved in - * the passed in table. For shared relations backends in all databases must be - * considered, but for non-shared relations that's not required, since only - * backends in my own database could ever see the tuples in them. Also, we can - * ignore concurrently running lazy VACUUMs because (a) they must be working - * on other tables, and (b) they don't need to do snapshot-based lookups. - * - * This is also used to determine where to truncate pg_subtrans. For that - * backends in all databases have to be considered, so rel = NULL has to be - * passed in. + * This also computes a horizon used to truncate pg_subtrans. For that + * backends in all databases have to be considered, and concurrently running + * lazy VACUUMs cannot be ignored, as they still may perform pg_subtrans + * accesses. * * Note: we include all currently running xids in the set of considered xids. * This ensures that if a just-started xact has not yet set its snapshot, * when it does set the snapshot it cannot set xmin less than what we compute. * See notes in src/backend/access/transam/README. * - * Note: despite the above, it's possible for the calculated value to move - * backwards on repeated calls. The calculated value is conservative, so that - * anything older is definitely not considered as running by anyone anymore, - * but the exact value calculated depends on a number of things. For example, - * if rel = NULL and there are no transactions running in the current - * database, GetOldestXmin() returns latestCompletedXid. If a transaction + * Note: despite the above, it's possible for the calculated values to move + * backwards on repeated calls. The calculated values are conservative, so + * that anything older is definitely not considered as running by anyone + * anymore, but the exact values calculated depend on a number of things. For + * example, if there are no transactions running in the current database, the + * horizon for normal tables will be latestCompletedXid. If a transaction * begins after that, its xmin will include in-progress transactions in other * databases that started earlier, so another call will return a lower value. * Nonetheless it is safe to vacuum a table in the current database with the * first result. There are also replication-related effects: a walsender * process can set its xmin based on transactions that are no longer running * on the primary but are still being replayed on the standby, thus possibly - * making the GetOldestXmin reading go backwards. In this case there is a - * possibility that we lose data that the standby would like to have, but - * unless the standby uses a replication slot to make its xmin persistent - * there is little we can do about that --- data is only protected if the - * walsender runs continuously while queries are executed on the standby. - * (The Hot Standby code deals with such cases by failing standby queries - * that needed to access already-removed data, so there's no integrity bug.) - * The return value is also adjusted with vacuum_defer_cleanup_age, so - * increasing that setting on the fly is another easy way to make - * GetOldestXmin() move backwards, with no consequences for data integrity. + * making the values go backwards. In this case there is a possibility that + * we lose data that the standby would like to have, but unless the standby + * uses a replication slot to make its xmin persistent there is little we can + * do about that --- data is only protected if the walsender runs continuously + * while queries are executed on the standby. (The Hot Standby code deals + * with such cases by failing standby queries that needed to access + * already-removed data, so there's no integrity bug.) The computed values + * are also adjusted with vacuum_defer_cleanup_age, so increasing that setting + * on the fly is another easy way to make horizons move backwards, with no + * consequences for data integrity. + * + * Note: the approximate horizons (see definition of GlobalVisState) are + * updated by the computations done here. That's currently required for + * correctness and a small optimization. Without doing so it's possible that + * heap vacuum's call to heap_page_prune() uses a more conservative horizon + * than later when deciding which tuples can be removed - which the code + * doesn't expect (breaking HOT). */ -TransactionId -GetOldestXmin(Relation rel, int flags) +static void +ComputeXidHorizons(ComputeXidHorizonsResult *h) { ProcArrayStruct *arrayP = procArray; - TransactionId result; - int index; - bool allDbs; + TransactionId kaxmin; + bool in_recovery = RecoveryInProgress(); - TransactionId replication_slot_xmin = InvalidTransactionId; - TransactionId replication_slot_catalog_xmin = InvalidTransactionId; - - /* - * If we're not computing a relation specific limit, or if a shared - * relation has been passed in, backends in all databases have to be - * considered. - */ - allDbs = rel == NULL || rel->rd_rel->relisshared; - - /* Cannot look for individual databases during recovery */ - Assert(allDbs || !RecoveryInProgress()); + /* inferred after ProcArrayLock is released */ + h->catalog_oldest_nonremovable = InvalidTransactionId; LWLockAcquire(ProcArrayLock, LW_SHARED); + h->latest_completed = ShmemVariableCache->latestCompletedXid; + /* * We initialize the MIN() calculation with latestCompletedXid + 1. This * is a lower bound for the XIDs that might appear in the ProcArray later, * and so protects us against overestimating the result due to future * additions. */ - result = XidFromFullTransactionId(ShmemVariableCache->latestCompletedXid); - TransactionIdAdvance(result); - Assert(TransactionIdIsNormal(result)); + { + TransactionId initial; - for (index = 0; index < arrayP->numProcs; index++) + initial = XidFromFullTransactionId(h->latest_completed); + Assert(TransactionIdIsValid(initial)); + TransactionIdAdvance(initial); + + h->oldest_considered_running = initial; + h->shared_oldest_nonremovable = initial; + h->data_oldest_nonremovable = initial; + } + + /* + * Fetch slot horizons while ProcArrayLock is held - the + * LWLockAcquire/LWLockRelease are a barrier, ensuring this happens inside + * the lock. + */ + h->slot_xmin = procArray->replication_slot_xmin; + h->slot_catalog_xmin = procArray->replication_slot_catalog_xmin; + + for (int index = 0; index < arrayP->numProcs; index++) { int pgprocno = arrayP->pgprocnos[index]; PGPROC *proc = &allProcs[pgprocno]; PGXACT *pgxact = &allPgXact[pgprocno]; + TransactionId xid; + TransactionId xmin; - if (pgxact->vacuumFlags & (flags & PROCARRAY_PROC_FLAGS_MASK)) + /* Fetch xid just once - see GetNewTransactionId */ + xid = UINT32_ACCESS_ONCE(pgxact->xid); + xmin = UINT32_ACCESS_ONCE(pgxact->xmin); + + /* + * Consider both the transaction's Xmin, and its Xid. + * + * We must check both because a transaction might have an Xmin but not + * (yet) an Xid; conversely, if it has an Xid, that could determine + * some not-yet-set Xmin. + */ + xmin = TransactionIdOlder(xmin, xid); + + /* if neither is set, this proc doesn't influence the horizon */ + if (!TransactionIdIsValid(xmin)) continue; - if (allDbs || + /* + * Don't ignore any procs when determining which transactions might be + * considered running. While slots should ensure logical decoding + * backends are protected even without this check, it can't hurt to + * include them here as well.. + */ + h->oldest_considered_running = + TransactionIdOlder(h->oldest_considered_running, xmin); + + /* + * Skip over backends either vacuuming (which is ok with rows being + * removed, as long as pg_subtrans is not truncated) or doing logical + * decoding (which manages xmin separately, check below). + */ + if (pgxact->vacuumFlags & (PROC_IN_VACUUM | PROC_IN_LOGICAL_DECODING)) + continue; + + /* shared tables need to take backends in all database into account */ + h->shared_oldest_nonremovable = + TransactionIdOlder(h->shared_oldest_nonremovable, xmin); + + /* + * Normally queries in other databases are ignored for anything but + * the shared horizon. But in recovery we cannot compute an accurate + * per-database horizon as all xids are managed via the + * KnownAssignedXids machinery. + */ + if (in_recovery || proc->databaseId == MyDatabaseId || proc->databaseId == 0) /* always include WalSender */ { - /* Fetch xid just once - see GetNewTransactionId */ - TransactionId xid = UINT32_ACCESS_ONCE(pgxact->xid); - - /* First consider the transaction's own Xid, if any */ - if (TransactionIdIsNormal(xid) && - TransactionIdPrecedes(xid, result)) - result = xid; - - /* - * Also consider the transaction's Xmin, if set. - * - * We must check both Xid and Xmin because a transaction might - * have an Xmin but not (yet) an Xid; conversely, if it has an - * Xid, that could determine some not-yet-set Xmin. - */ - xid = UINT32_ACCESS_ONCE(pgxact->xmin); - if (TransactionIdIsNormal(xid) && - TransactionIdPrecedes(xid, result)) - result = xid; + h->data_oldest_nonremovable = + TransactionIdOlder(h->data_oldest_nonremovable, xmin); } } /* - * Fetch into local variable while ProcArrayLock is held - the - * LWLockRelease below is a barrier, ensuring this happens inside the - * lock. + * If in recovery fetch oldest xid in KnownAssignedXids, will be applied + * after lock is released. */ - replication_slot_xmin = procArray->replication_slot_xmin; - replication_slot_catalog_xmin = procArray->replication_slot_catalog_xmin; + if (in_recovery) + kaxmin = KnownAssignedXidsGetOldestXmin(); - if (RecoveryInProgress()) + /* + * No other information needed, so release the lock immediately. The rest + * of the computations can be done without a lock. + */ + LWLockRelease(ProcArrayLock); + + if (in_recovery) { - /* - * Check to see whether KnownAssignedXids contains an xid value older - * than the main procarray. - */ - TransactionId kaxmin = KnownAssignedXidsGetOldestXmin(); - - LWLockRelease(ProcArrayLock); - - if (TransactionIdIsNormal(kaxmin) && - TransactionIdPrecedes(kaxmin, result)) - result = kaxmin; + h->oldest_considered_running = + TransactionIdOlder(h->oldest_considered_running, kaxmin); + h->shared_oldest_nonremovable = + TransactionIdOlder(h->shared_oldest_nonremovable, kaxmin); + h->data_oldest_nonremovable = + TransactionIdOlder(h->data_oldest_nonremovable, kaxmin); } else { /* - * No other information needed, so release the lock immediately. - */ - LWLockRelease(ProcArrayLock); - - /* - * Compute the cutoff XID by subtracting vacuum_defer_cleanup_age, - * being careful not to generate a "permanent" XID. + * Compute the cutoff XID by subtracting vacuum_defer_cleanup_age. * * vacuum_defer_cleanup_age provides some additional "slop" for the * benefit of hot standby queries on standby servers. This is quick @@ -1466,34 +1651,146 @@ GetOldestXmin(Relation rel, int flags) * in varsup.c. Also note that we intentionally don't apply * vacuum_defer_cleanup_age on standby servers. */ - result -= vacuum_defer_cleanup_age; - if (!TransactionIdIsNormal(result)) - result = FirstNormalTransactionId; + h->oldest_considered_running = + TransactionIdRetreatedBy(h->oldest_considered_running, + vacuum_defer_cleanup_age); + h->shared_oldest_nonremovable = + TransactionIdRetreatedBy(h->shared_oldest_nonremovable, + vacuum_defer_cleanup_age); + h->data_oldest_nonremovable = + TransactionIdRetreatedBy(h->data_oldest_nonremovable, + vacuum_defer_cleanup_age); } /* * Check whether there are replication slots requiring an older xmin. */ - if (!(flags & PROCARRAY_SLOTS_XMIN) && - TransactionIdIsValid(replication_slot_xmin) && - NormalTransactionIdPrecedes(replication_slot_xmin, result)) - result = replication_slot_xmin; + h->shared_oldest_nonremovable = + TransactionIdOlder(h->shared_oldest_nonremovable, h->slot_xmin); + h->data_oldest_nonremovable = + TransactionIdOlder(h->data_oldest_nonremovable, h->slot_xmin); /* - * After locks have been released and vacuum_defer_cleanup_age has been - * applied, check whether we need to back up further to make logical - * decoding possible. We need to do so if we're computing the global limit - * (rel = NULL) or if the passed relation is a catalog relation of some - * kind. + * The only difference between catalog / data horizons is that the slot's + * catalog xmin is applied to the catalog one (so catalogs can be accessed + * for logical decoding). Initialize with data horizon, and then back up + * further if necessary. Have to back up the shared horizon as well, since + * that also can contain catalogs. */ - if (!(flags & PROCARRAY_SLOTS_XMIN) && - (rel == NULL || - RelationIsAccessibleInLogicalDecoding(rel)) && - TransactionIdIsValid(replication_slot_catalog_xmin) && - NormalTransactionIdPrecedes(replication_slot_catalog_xmin, result)) - result = replication_slot_catalog_xmin; + h->shared_oldest_nonremovable_raw = h->shared_oldest_nonremovable; + h->shared_oldest_nonremovable = + TransactionIdOlder(h->shared_oldest_nonremovable, + h->slot_catalog_xmin); + h->catalog_oldest_nonremovable = h->data_oldest_nonremovable; + h->catalog_oldest_nonremovable = + TransactionIdOlder(h->catalog_oldest_nonremovable, + h->slot_catalog_xmin); - return result; + /* + * It's possible that slots / vacuum_defer_cleanup_age backed up the + * horizons further than oldest_considered_running. Fix. + */ + h->oldest_considered_running = + TransactionIdOlder(h->oldest_considered_running, + h->shared_oldest_nonremovable); + h->oldest_considered_running = + TransactionIdOlder(h->oldest_considered_running, + h->catalog_oldest_nonremovable); + h->oldest_considered_running = + TransactionIdOlder(h->oldest_considered_running, + h->data_oldest_nonremovable); + + /* + * shared horizons have to be at least as old as the oldest visible in + * current db + */ + Assert(TransactionIdPrecedesOrEquals(h->shared_oldest_nonremovable, + h->data_oldest_nonremovable)); + Assert(TransactionIdPrecedesOrEquals(h->shared_oldest_nonremovable, + h->catalog_oldest_nonremovable)); + + /* + * Horizons need to ensure that pg_subtrans access is still possible for + * the relevant backends. + */ + Assert(TransactionIdPrecedesOrEquals(h->oldest_considered_running, + h->shared_oldest_nonremovable)); + Assert(TransactionIdPrecedesOrEquals(h->oldest_considered_running, + h->catalog_oldest_nonremovable)); + Assert(TransactionIdPrecedesOrEquals(h->oldest_considered_running, + h->data_oldest_nonremovable)); + Assert(!TransactionIdIsValid(h->slot_xmin) || + TransactionIdPrecedesOrEquals(h->oldest_considered_running, + h->slot_xmin)); + Assert(!TransactionIdIsValid(h->slot_catalog_xmin) || + TransactionIdPrecedesOrEquals(h->oldest_considered_running, + h->slot_catalog_xmin)); + + /* update approximate horizons with the computed horizons */ + GlobalVisUpdateApply(h); +} + +/* + * Return the oldest XID for which deleted tuples must be preserved in the + * passed table. + * + * If rel is not NULL the horizon may be considerably more recent than + * otherwise (i.e. fewer tuples will be removable). In the NULL case a horizon + * that is correct (but not optimal) for all relations will be returned. + * + * This is used by VACUUM to decide which deleted tuples must be preserved in + * the passed in table. + */ +TransactionId +GetOldestNonRemovableTransactionId(Relation rel) +{ + ComputeXidHorizonsResult horizons; + + ComputeXidHorizons(&horizons); + + /* select horizon appropriate for relation */ + if (rel == NULL || rel->rd_rel->relisshared) + return horizons.shared_oldest_nonremovable; + else if (RelationIsAccessibleInLogicalDecoding(rel)) + return horizons.catalog_oldest_nonremovable; + else + return horizons.data_oldest_nonremovable; +} + +/* + * Return the oldest transaction id any currently running backend might still + * consider running. This should not be used for visibility / pruning + * determinations (see GetOldestNonRemovableTransactionId()), but for + * decisions like up to where pg_subtrans can be truncated. + */ +TransactionId +GetOldestTransactionIdConsideredRunning(void) +{ + ComputeXidHorizonsResult horizons; + + ComputeXidHorizons(&horizons); + + return horizons.oldest_considered_running; +} + +/* + * Return the visibility horizons for a hot standby feedback message. + */ +void +GetReplicationHorizons(TransactionId *xmin, TransactionId *catalog_xmin) +{ + ComputeXidHorizonsResult horizons; + + ComputeXidHorizons(&horizons); + + /* + * Don't want to use shared_oldest_nonremovable here, as that contains the + * effect of replication slot's catalog_xmin. We want to send a separate + * feedback for the catalog horizon, so the primary can remove data table + * contents more aggressively. + */ + *xmin = horizons.shared_oldest_nonremovable_raw; + *catalog_xmin = horizons.slot_catalog_xmin; } /* @@ -1544,12 +1841,10 @@ GetMaxSnapshotSubxidCount(void) * current transaction (this is the same as MyPgXact->xmin). * RecentXmin: the xmin computed for the most recent snapshot. XIDs * older than this are known not running any more. - * RecentGlobalXmin: the global xmin (oldest TransactionXmin across all - * running transactions, except those running LAZY VACUUM). This is - * the same computation done by - * GetOldestXmin(NULL, PROCARRAY_FLAGS_VACUUM). - * RecentGlobalDataXmin: the global xmin for non-catalog tables - * >= RecentGlobalXmin + * + * And try to advance the bounds of GlobalVisSharedRels, + * GlobalVisCatalogRels, GlobalVisDataRels for + * the benefit GlobalVis*. * * Note: this function should probably not be called with an argument that's * not statically allocated (see xip allocation below). @@ -1560,12 +1855,12 @@ GetSnapshotData(Snapshot snapshot) ProcArrayStruct *arrayP = procArray; TransactionId xmin; TransactionId xmax; - TransactionId globalxmin; int index; int count = 0; int subcount = 0; bool suboverflowed = false; FullTransactionId latest_completed; + TransactionId oldestxid; TransactionId replication_slot_xmin = InvalidTransactionId; TransactionId replication_slot_catalog_xmin = InvalidTransactionId; @@ -1610,13 +1905,15 @@ GetSnapshotData(Snapshot snapshot) LWLockAcquire(ProcArrayLock, LW_SHARED); latest_completed = ShmemVariableCache->latestCompletedXid; + oldestxid = ShmemVariableCache->oldestXid; + /* xmax is always latestCompletedXid + 1 */ xmax = XidFromFullTransactionId(latest_completed); TransactionIdAdvance(xmax); Assert(TransactionIdIsNormal(xmax)); /* initialize xmin calculation with xmax */ - globalxmin = xmin = xmax; + xmin = xmax; snapshot->takenDuringRecovery = RecoveryInProgress(); @@ -1645,12 +1942,6 @@ GetSnapshotData(Snapshot snapshot) (PROC_IN_LOGICAL_DECODING | PROC_IN_VACUUM)) continue; - /* Update globalxmin to be the smallest valid xmin */ - xid = UINT32_ACCESS_ONCE(pgxact->xmin); - if (TransactionIdIsNormal(xid) && - NormalTransactionIdPrecedes(xid, globalxmin)) - globalxmin = xid; - /* Fetch xid just once - see GetNewTransactionId */ xid = UINT32_ACCESS_ONCE(pgxact->xid); @@ -1766,34 +2057,78 @@ GetSnapshotData(Snapshot snapshot) LWLockRelease(ProcArrayLock); - /* - * Update globalxmin to include actual process xids. This is a slightly - * different way of computing it than GetOldestXmin uses, but should give - * the same result. - */ - if (TransactionIdPrecedes(xmin, globalxmin)) - globalxmin = xmin; + /* maintain state for GlobalVis* */ + { + TransactionId def_vis_xid; + TransactionId def_vis_xid_data; + FullTransactionId def_vis_fxid; + FullTransactionId def_vis_fxid_data; + FullTransactionId oldestfxid; - /* Update global variables too */ - RecentGlobalXmin = globalxmin - vacuum_defer_cleanup_age; - if (!TransactionIdIsNormal(RecentGlobalXmin)) - RecentGlobalXmin = FirstNormalTransactionId; + /* + * Converting oldestXid is only safe when xid horizon cannot advance, + * i.e. holding locks. While we don't hold the lock anymore, all the + * necessary data has been gathered with lock held. + */ + oldestfxid = FullXidRelativeTo(latest_completed, oldestxid); - /* Check whether there's a replication slot requiring an older xmin. */ - if (TransactionIdIsValid(replication_slot_xmin) && - NormalTransactionIdPrecedes(replication_slot_xmin, RecentGlobalXmin)) - RecentGlobalXmin = replication_slot_xmin; + /* apply vacuum_defer_cleanup_age */ + def_vis_xid_data = + TransactionIdRetreatedBy(xmin, vacuum_defer_cleanup_age); - /* Non-catalog tables can be vacuumed if older than this xid */ - RecentGlobalDataXmin = RecentGlobalXmin; + /* Check whether there's a replication slot requiring an older xmin. */ + def_vis_xid_data = + TransactionIdOlder(def_vis_xid_data, replication_slot_xmin); - /* - * Check whether there's a replication slot requiring an older catalog - * xmin. - */ - if (TransactionIdIsNormal(replication_slot_catalog_xmin) && - NormalTransactionIdPrecedes(replication_slot_catalog_xmin, RecentGlobalXmin)) - RecentGlobalXmin = replication_slot_catalog_xmin; + /* + * Rows in non-shared, non-catalog tables possibly could be vacuumed + * if older than this xid. + */ + def_vis_xid = def_vis_xid_data; + + /* + * Check whether there's a replication slot requiring an older catalog + * xmin. + */ + def_vis_xid = + TransactionIdOlder(replication_slot_catalog_xmin, def_vis_xid); + + def_vis_fxid = FullXidRelativeTo(latest_completed, def_vis_xid); + def_vis_fxid_data = FullXidRelativeTo(latest_completed, def_vis_xid_data); + + /* + * Check if we can increase upper bound. As a previous + * GlobalVisUpdate() might have computed more aggressive values, don't + * overwrite them if so. + */ + GlobalVisSharedRels.definitely_needed = + FullTransactionIdNewer(def_vis_fxid, + GlobalVisSharedRels.definitely_needed); + GlobalVisCatalogRels.definitely_needed = + FullTransactionIdNewer(def_vis_fxid, + GlobalVisCatalogRels.definitely_needed); + GlobalVisDataRels.definitely_needed = + FullTransactionIdNewer(def_vis_fxid_data, + GlobalVisDataRels.definitely_needed); + + /* + * Check if we know that we can initialize or increase the lower + * bound. Currently the only cheap way to do so is to use + * ShmemVariableCache->oldestXid as input. + * + * We should definitely be able to do better. We could e.g. put a + * global lower bound value into ShmemVariableCache. + */ + GlobalVisSharedRels.maybe_needed = + FullTransactionIdNewer(GlobalVisSharedRels.maybe_needed, + oldestfxid); + GlobalVisCatalogRels.maybe_needed = + FullTransactionIdNewer(GlobalVisCatalogRels.maybe_needed, + oldestfxid); + GlobalVisDataRels.maybe_needed = + FullTransactionIdNewer(GlobalVisDataRels.maybe_needed, + oldestfxid); + } RecentXmin = xmin; @@ -3291,6 +3626,255 @@ DisplayXidCache(void) } #endif /* XIDCACHE_DEBUG */ +/* + * If rel != NULL, return test state appropriate for relation, otherwise + * return state usable for all relations. The latter may consider XIDs as + * not-yet-visible-to-everyone that a state for a specific relation would + * already consider visible-to-everyone. + * + * This needs to be called while a snapshot is active or registered, otherwise + * there are wraparound and other dangers. + * + * See comment for GlobalVisState for details. + */ +GlobalVisState * +GlobalVisTestFor(Relation rel) +{ + bool need_shared; + bool need_catalog; + GlobalVisState *state; + + /* XXX: we should assert that a snapshot is pushed or registered */ + Assert(RecentXmin); + + if (!rel) + need_shared = need_catalog = true; + else + { + /* + * Other kinds currently don't contain xids, nor always the necessary + * logical decoding markers. + */ + Assert(rel->rd_rel->relkind == RELKIND_RELATION || + rel->rd_rel->relkind == RELKIND_MATVIEW || + rel->rd_rel->relkind == RELKIND_TOASTVALUE); + + need_shared = rel->rd_rel->relisshared || RecoveryInProgress(); + need_catalog = IsCatalogRelation(rel) || RelationIsAccessibleInLogicalDecoding(rel); + } + + if (need_shared) + state = &GlobalVisSharedRels; + else if (need_catalog) + state = &GlobalVisCatalogRels; + else + state = &GlobalVisDataRels; + + Assert(FullTransactionIdIsValid(state->definitely_needed) && + FullTransactionIdIsValid(state->maybe_needed)); + + return state; +} + +/* + * Return true if it's worth updating the accurate maybe_needed boundary. + * + * As it is somewhat expensive to determine xmin horizons, we don't want to + * repeatedly do so when there is a low likelihood of it being beneficial. + * + * The current heuristic is that we update only if RecentXmin has changed + * since the last update. If the oldest currently running transaction has not + * finished, it is unlikely that recomputing the horizon would be useful. + */ +static bool +GlobalVisTestShouldUpdate(GlobalVisState *state) +{ + /* hasn't been updated yet */ + if (!TransactionIdIsValid(ComputeXidHorizonsResultLastXmin)) + return true; + + /* + * If the maybe_needed/definitely_needed boundaries are the same, it's + * unlikely to be beneficial to refresh boundaries. + */ + if (FullTransactionIdFollowsOrEquals(state->maybe_needed, + state->definitely_needed)) + return false; + + /* does the last snapshot built have a different xmin? */ + return RecentXmin != ComputeXidHorizonsResultLastXmin; +} + +static void +GlobalVisUpdateApply(ComputeXidHorizonsResult *horizons) +{ + GlobalVisSharedRels.maybe_needed = + FullXidRelativeTo(horizons->latest_completed, + horizons->shared_oldest_nonremovable); + GlobalVisCatalogRels.maybe_needed = + FullXidRelativeTo(horizons->latest_completed, + horizons->catalog_oldest_nonremovable); + GlobalVisDataRels.maybe_needed = + FullXidRelativeTo(horizons->latest_completed, + horizons->data_oldest_nonremovable); + + /* + * In longer running transactions it's possible that transactions we + * previously needed to treat as running aren't around anymore. So update + * definitely_needed to not be earlier than maybe_needed. + */ + GlobalVisSharedRels.definitely_needed = + FullTransactionIdNewer(GlobalVisSharedRels.maybe_needed, + GlobalVisSharedRels.definitely_needed); + GlobalVisCatalogRels.definitely_needed = + FullTransactionIdNewer(GlobalVisCatalogRels.maybe_needed, + GlobalVisCatalogRels.definitely_needed); + GlobalVisDataRels.definitely_needed = + FullTransactionIdNewer(GlobalVisDataRels.maybe_needed, + GlobalVisDataRels.definitely_needed); + + ComputeXidHorizonsResultLastXmin = RecentXmin; +} + +/* + * Update boundaries in GlobalVis{Shared,Catalog, Data}Rels + * using ComputeXidHorizons(). + */ +static void +GlobalVisUpdate(void) +{ + ComputeXidHorizonsResult horizons; + + /* updates the horizons as a side-effect */ + ComputeXidHorizons(&horizons); +} + +/* + * Return true if no snapshot still considers fxid to be running. + * + * The state passed needs to have been initialized for the relation fxid is + * from (NULL is also OK), otherwise the result may not be correct. + * + * See comment for GlobalVisState for details. + */ +bool +GlobalVisTestIsRemovableFullXid(GlobalVisState *state, + FullTransactionId fxid) +{ + /* + * If fxid is older than maybe_needed bound, it definitely is visible to + * everyone. + */ + if (FullTransactionIdPrecedes(fxid, state->maybe_needed)) + return true; + + /* + * If fxid is >= definitely_needed bound, it is very likely to still be + * considered running. + */ + if (FullTransactionIdFollowsOrEquals(fxid, state->definitely_needed)) + return false; + + /* + * fxid is between maybe_needed and definitely_needed, i.e. there might or + * might not exist a snapshot considering fxid running. If it makes sense, + * update boundaries and recheck. + */ + if (GlobalVisTestShouldUpdate(state)) + { + GlobalVisUpdate(); + + Assert(FullTransactionIdPrecedes(fxid, state->definitely_needed)); + + return FullTransactionIdPrecedes(fxid, state->maybe_needed); + } + else + return false; +} + +/* + * Wrapper around GlobalVisTestIsRemovableFullXid() for 32bit xids. + * + * It is crucial that this only gets called for xids from a source that + * protects against xid wraparounds (e.g. from a table and thus protected by + * relfrozenxid). + */ +bool +GlobalVisTestIsRemovableXid(GlobalVisState *state, TransactionId xid) +{ + FullTransactionId fxid; + + /* + * Convert 32 bit argument to FullTransactionId. We can do so safely + * because we know the xid has to, at the very least, be between + * [oldestXid, nextFullXid), i.e. within 2 billion of xid. To avoid taking + * a lock to determine either, we can just compare with + * state->definitely_needed, which was based on those value at the time + * the current snapshot was built. + */ + fxid = FullXidRelativeTo(state->definitely_needed, xid); + + return GlobalVisTestIsRemovableFullXid(state, fxid); +} + +/* + * Return FullTransactionId below which all transactions are not considered + * running anymore. + * + * Note: This is less efficient than testing with + * GlobalVisTestIsRemovableFullXid as it likely requires building an accurate + * cutoff, even in the case all the XIDs compared with the cutoff are outside + * [maybe_needed, definitely_needed). + */ +FullTransactionId +GlobalVisTestNonRemovableFullHorizon(GlobalVisState *state) +{ + /* acquire accurate horizon if not already done */ + if (GlobalVisTestShouldUpdate(state)) + GlobalVisUpdate(); + + return state->maybe_needed; +} + +/* Convenience wrapper around GlobalVisTestNonRemovableFullHorizon */ +TransactionId +GlobalVisTestNonRemovableHorizon(GlobalVisState *state) +{ + FullTransactionId cutoff; + + cutoff = GlobalVisTestNonRemovableFullHorizon(state); + + return XidFromFullTransactionId(cutoff); +} + +/* + * Convenience wrapper around GlobalVisTestFor() and + * GlobalVisTestIsRemovableFullXid(), see their comments. + */ +bool +GlobalVisIsRemovableFullXid(Relation rel, FullTransactionId fxid) +{ + GlobalVisState *state; + + state = GlobalVisTestFor(rel); + + return GlobalVisTestIsRemovableFullXid(state, fxid); +} + +/* + * Convenience wrapper around GlobalVisTestFor() and + * GlobalVisTestIsRemovableXid(), see their comments. + */ +bool +GlobalVisCheckRemovableXid(Relation rel, TransactionId xid) +{ + GlobalVisState *state; + + state = GlobalVisTestFor(rel); + + return GlobalVisTestIsRemovableXid(state, xid); +} + /* * Convert a 32 bit transaction id into 64 bit transaction id, by assuming it * is within MaxTransactionId / 2 of XidFromFullTransactionId(rel). diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c index 53d974125fd..00c7afc66fc 100644 --- a/src/backend/utils/adt/selfuncs.c +++ b/src/backend/utils/adt/selfuncs.c @@ -5786,14 +5786,15 @@ get_actual_variable_endpoint(Relation heapRel, * recent); that case motivates not using SnapshotAny here. * * A crucial point here is that SnapshotNonVacuumable, with - * RecentGlobalXmin as horizon, yields the inverse of the condition that - * the indexscan will use to decide that index entries are killable (see - * heap_hot_search_buffer()). Therefore, if the snapshot rejects a tuple - * (or more precisely, all tuples of a HOT chain) and we have to continue - * scanning past it, we know that the indexscan will mark that index entry - * killed. That means that the next get_actual_variable_endpoint() call - * will not have to re-consider that index entry. In this way we avoid - * repetitive work when this function is used a lot during planning. + * GlobalVisTestFor(heapRel) as horizon, yields the inverse of the + * condition that the indexscan will use to decide that index entries are + * killable (see heap_hot_search_buffer()). Therefore, if the snapshot + * rejects a tuple (or more precisely, all tuples of a HOT chain) and we + * have to continue scanning past it, we know that the indexscan will mark + * that index entry killed. That means that the next + * get_actual_variable_endpoint() call will not have to re-consider that + * index entry. In this way we avoid repetitive work when this function + * is used a lot during planning. * * But using SnapshotNonVacuumable creates a hazard of its own. In a * recently-created index, some index entries may point at "broken" HOT @@ -5805,7 +5806,8 @@ get_actual_variable_endpoint(Relation heapRel, * or could even be NULL. We avoid this hazard because we take the data * from the index entry not the heap. */ - InitNonVacuumableSnapshot(SnapshotNonVacuumable, RecentGlobalXmin); + InitNonVacuumableSnapshot(SnapshotNonVacuumable, + GlobalVisTestFor(heapRel)); index_scan = index_beginscan(heapRel, indexRel, &SnapshotNonVacuumable, diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c index f4247ea70d5..893be2f3ddb 100644 --- a/src/backend/utils/init/postinit.c +++ b/src/backend/utils/init/postinit.c @@ -722,6 +722,10 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username, * is critical for anything that reads heap pages, because HOT may decide * to prune them even if the process doesn't attempt to modify any * tuples.) + * + * FIXME: This comment is inaccurate / the code buggy. A snapshot that is + * not pushed/active does not reliably prevent HOT pruning (->xmin could + * e.g. be cleared when cache invalidations are processed). */ if (!bootstrap) { diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 6b6c8571e23..76578868cf9 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -157,16 +157,9 @@ static Snapshot HistoricSnapshot = NULL; * These are updated by GetSnapshotData. We initialize them this way * for the convenience of TransactionIdIsInProgress: even in bootstrap * mode, we don't want it to say that BootstrapTransactionId is in progress. - * - * RecentGlobalXmin and RecentGlobalDataXmin are initialized to - * InvalidTransactionId, to ensure that no one tries to use a stale - * value. Readers should ensure that it has been set to something else - * before using it. */ TransactionId TransactionXmin = FirstNormalTransactionId; TransactionId RecentXmin = FirstNormalTransactionId; -TransactionId RecentGlobalXmin = InvalidTransactionId; -TransactionId RecentGlobalDataXmin = InvalidTransactionId; /* (table, ctid) => (cmin, cmax) mapping during timetravel */ static HTAB *tuplecid_data = NULL; @@ -581,9 +574,7 @@ SetTransactionSnapshot(Snapshot sourcesnap, VirtualTransactionId *sourcevxid, * Even though we are not going to use the snapshot it computes, we must * call GetSnapshotData, for two reasons: (1) to be sure that * CurrentSnapshotData's XID arrays have been allocated, and (2) to update - * RecentXmin and RecentGlobalXmin. (We could alternatively include those - * two variables in exported snapshot files, but it seems better to have - * snapshot importers compute reasonably up-to-date values for them.) + * the state for GlobalVis*. */ CurrentSnapshot = GetSnapshotData(&CurrentSnapshotData); @@ -956,36 +947,6 @@ xmin_cmp(const pairingheap_node *a, const pairingheap_node *b, void *arg) return 0; } -/* - * Get current RecentGlobalXmin value, as a FullTransactionId. - */ -FullTransactionId -GetFullRecentGlobalXmin(void) -{ - FullTransactionId nextxid_full; - uint32 nextxid_epoch; - TransactionId nextxid_xid; - uint32 epoch; - - Assert(TransactionIdIsNormal(RecentGlobalXmin)); - - /* - * Compute the epoch from the next XID's epoch. This relies on the fact - * that RecentGlobalXmin must be within the 2 billion XID horizon from the - * next XID. - */ - nextxid_full = ReadNextFullTransactionId(); - nextxid_epoch = EpochFromFullTransactionId(nextxid_full); - nextxid_xid = XidFromFullTransactionId(nextxid_full); - - if (RecentGlobalXmin > nextxid_xid) - epoch = nextxid_epoch - 1; - else - epoch = nextxid_epoch; - - return FullTransactionIdFromEpochAndXid(epoch, RecentGlobalXmin); -} - /* * SnapshotResetXmin * @@ -1753,106 +1714,157 @@ GetOldSnapshotThresholdTimestamp(void) return threshold_timestamp; } -static void +void SetOldSnapshotThresholdTimestamp(TimestampTz ts, TransactionId xlimit) { SpinLockAcquire(&oldSnapshotControl->mutex_threshold); + Assert(oldSnapshotControl->threshold_timestamp <= ts); + Assert(TransactionIdPrecedesOrEquals(oldSnapshotControl->threshold_xid, xlimit)); oldSnapshotControl->threshold_timestamp = ts; oldSnapshotControl->threshold_xid = xlimit; SpinLockRelease(&oldSnapshotControl->mutex_threshold); } +/* + * XXX: Magic to keep old_snapshot_threshold tests appear "working". They + * currently are broken, and discussion of what to do about them is + * ongoing. See + * https://www.postgresql.org/message-id/20200403001235.e6jfdll3gh2ygbuc%40alap3.anarazel.de + */ +void +SnapshotTooOldMagicForTest(void) +{ + TimestampTz ts = GetSnapshotCurrentTimestamp(); + + Assert(old_snapshot_threshold == 0); + + ts -= 5 * USECS_PER_SEC; + + SpinLockAcquire(&oldSnapshotControl->mutex_threshold); + oldSnapshotControl->threshold_timestamp = ts; + SpinLockRelease(&oldSnapshotControl->mutex_threshold); +} + +/* + * If there is a valid mapping for the timestamp, set *xlimitp to + * that. Returns whether there is such a mapping. + */ +static bool +GetOldSnapshotFromTimeMapping(TimestampTz ts, TransactionId *xlimitp) +{ + bool in_mapping = false; + + Assert(ts == AlignTimestampToMinuteBoundary(ts)); + + LWLockAcquire(OldSnapshotTimeMapLock, LW_SHARED); + + if (oldSnapshotControl->count_used > 0 + && ts >= oldSnapshotControl->head_timestamp) + { + int offset; + + offset = ((ts - oldSnapshotControl->head_timestamp) + / USECS_PER_MINUTE); + if (offset > oldSnapshotControl->count_used - 1) + offset = oldSnapshotControl->count_used - 1; + offset = (oldSnapshotControl->head_offset + offset) + % OLD_SNAPSHOT_TIME_MAP_ENTRIES; + + *xlimitp = oldSnapshotControl->xid_by_minute[offset]; + + in_mapping = true; + } + + LWLockRelease(OldSnapshotTimeMapLock); + + return in_mapping; +} + /* * TransactionIdLimitedForOldSnapshots * - * Apply old snapshot limit, if any. This is intended to be called for page - * pruning and table vacuuming, to allow old_snapshot_threshold to override - * the normal global xmin value. Actual testing for snapshot too old will be - * based on whether a snapshot timestamp is prior to the threshold timestamp - * set in this function. + * Apply old snapshot limit. This is intended to be called for page pruning + * and table vacuuming, to allow old_snapshot_threshold to override the normal + * global xmin value. Actual testing for snapshot too old will be based on + * whether a snapshot timestamp is prior to the threshold timestamp set in + * this function. + * + * If the limited horizon allows a cleanup action that otherwise would not be + * possible, SetOldSnapshotThresholdTimestamp(*limit_ts, *limit_xid) needs to + * be called before that cleanup action. */ -TransactionId +bool TransactionIdLimitedForOldSnapshots(TransactionId recentXmin, - Relation relation) + Relation relation, + TransactionId *limit_xid, + TimestampTz *limit_ts) { - if (TransactionIdIsNormal(recentXmin) - && old_snapshot_threshold >= 0 - && RelationAllowsEarlyPruning(relation)) + TimestampTz ts; + TransactionId xlimit = recentXmin; + TransactionId latest_xmin; + TimestampTz next_map_update_ts; + TransactionId threshold_timestamp; + TransactionId threshold_xid; + + Assert(TransactionIdIsNormal(recentXmin)); + Assert(OldSnapshotThresholdActive()); + Assert(limit_ts != NULL && limit_xid != NULL); + + if (!RelationAllowsEarlyPruning(relation)) + return false; + + ts = GetSnapshotCurrentTimestamp(); + + SpinLockAcquire(&oldSnapshotControl->mutex_latest_xmin); + latest_xmin = oldSnapshotControl->latest_xmin; + next_map_update_ts = oldSnapshotControl->next_map_update; + SpinLockRelease(&oldSnapshotControl->mutex_latest_xmin); + + /* + * Zero threshold always overrides to latest xmin, if valid. Without + * some heuristic it will find its own snapshot too old on, for + * example, a simple UPDATE -- which would make it useless for most + * testing, but there is no principled way to ensure that it doesn't + * fail in this way. Use a five-second delay to try to get useful + * testing behavior, but this may need adjustment. + */ + if (old_snapshot_threshold == 0) { - TimestampTz ts = GetSnapshotCurrentTimestamp(); - TransactionId xlimit = recentXmin; - TransactionId latest_xmin; - TimestampTz update_ts; - bool same_ts_as_threshold = false; - - SpinLockAcquire(&oldSnapshotControl->mutex_latest_xmin); - latest_xmin = oldSnapshotControl->latest_xmin; - update_ts = oldSnapshotControl->next_map_update; - SpinLockRelease(&oldSnapshotControl->mutex_latest_xmin); - - /* - * Zero threshold always overrides to latest xmin, if valid. Without - * some heuristic it will find its own snapshot too old on, for - * example, a simple UPDATE -- which would make it useless for most - * testing, but there is no principled way to ensure that it doesn't - * fail in this way. Use a five-second delay to try to get useful - * testing behavior, but this may need adjustment. - */ - if (old_snapshot_threshold == 0) - { - if (TransactionIdPrecedes(latest_xmin, MyPgXact->xmin) - && TransactionIdFollows(latest_xmin, xlimit)) - xlimit = latest_xmin; - - ts -= 5 * USECS_PER_SEC; - SetOldSnapshotThresholdTimestamp(ts, xlimit); - - return xlimit; - } + if (TransactionIdPrecedes(latest_xmin, MyPgXact->xmin) + && TransactionIdFollows(latest_xmin, xlimit)) + xlimit = latest_xmin; + ts -= 5 * USECS_PER_SEC; + } + else + { ts = AlignTimestampToMinuteBoundary(ts) - (old_snapshot_threshold * USECS_PER_MINUTE); /* Check for fast exit without LW locking. */ SpinLockAcquire(&oldSnapshotControl->mutex_threshold); - if (ts == oldSnapshotControl->threshold_timestamp) - { - xlimit = oldSnapshotControl->threshold_xid; - same_ts_as_threshold = true; - } + threshold_timestamp = oldSnapshotControl->threshold_timestamp; + threshold_xid = oldSnapshotControl->threshold_xid; SpinLockRelease(&oldSnapshotControl->mutex_threshold); - if (!same_ts_as_threshold) + if (ts == threshold_timestamp) + { + /* + * Current timestamp is in same bucket as the the last limit that + * was applied. Reuse. + */ + xlimit = threshold_xid; + } + else if (ts == next_map_update_ts) + { + /* + * FIXME: This branch is super iffy - but that should probably + * fixed separately. + */ + xlimit = latest_xmin; + } + else if (GetOldSnapshotFromTimeMapping(ts, &xlimit)) { - if (ts == update_ts) - { - xlimit = latest_xmin; - if (NormalTransactionIdFollows(xlimit, recentXmin)) - SetOldSnapshotThresholdTimestamp(ts, xlimit); - } - else - { - LWLockAcquire(OldSnapshotTimeMapLock, LW_SHARED); - - if (oldSnapshotControl->count_used > 0 - && ts >= oldSnapshotControl->head_timestamp) - { - int offset; - - offset = ((ts - oldSnapshotControl->head_timestamp) - / USECS_PER_MINUTE); - if (offset > oldSnapshotControl->count_used - 1) - offset = oldSnapshotControl->count_used - 1; - offset = (oldSnapshotControl->head_offset + offset) - % OLD_SNAPSHOT_TIME_MAP_ENTRIES; - xlimit = oldSnapshotControl->xid_by_minute[offset]; - - if (NormalTransactionIdFollows(xlimit, recentXmin)) - SetOldSnapshotThresholdTimestamp(ts, xlimit); - } - - LWLockRelease(OldSnapshotTimeMapLock); - } } /* @@ -1867,12 +1879,18 @@ TransactionIdLimitedForOldSnapshots(TransactionId recentXmin, if (TransactionIdIsNormal(latest_xmin) && TransactionIdPrecedes(latest_xmin, xlimit)) xlimit = latest_xmin; - - if (NormalTransactionIdFollows(xlimit, recentXmin)) - return xlimit; } - return recentXmin; + if (TransactionIdIsValid(xlimit) && + TransactionIdFollowsOrEquals(xlimit, recentXmin)) + { + *limit_ts = ts; + *limit_xid = xlimit; + + return true; + } + + return false; } /* diff --git a/contrib/amcheck/verify_nbtree.c b/contrib/amcheck/verify_nbtree.c index 635ece73b35..5f3de3c0b7f 100644 --- a/contrib/amcheck/verify_nbtree.c +++ b/contrib/amcheck/verify_nbtree.c @@ -434,10 +434,10 @@ bt_check_every_level(Relation rel, Relation heaprel, bool heapkeyspace, RelationGetRelationName(rel)); /* - * RecentGlobalXmin assertion matches index_getnext_tid(). See note on - * RecentGlobalXmin/B-Tree page deletion. + * This assertion matches the one in index_getnext_tid(). See page + * recycling/"visible to everyone" notes in nbtree README. */ - Assert(TransactionIdIsValid(RecentGlobalXmin)); + Assert(TransactionIdIsValid(RecentXmin)); /* * Initialize state for entire verification operation @@ -1581,7 +1581,7 @@ bt_right_page_check_scankey(BtreeCheckState *state) * does not occur until no possible index scan could land on the page. * Index scans can follow links with nothing more than their snapshot as * an interlock and be sure of at least that much. (See page - * recycling/RecentGlobalXmin notes in nbtree README.) + * recycling/"visible to everyone" notes in nbtree README.) * * Furthermore, it's okay if we follow a rightlink and find a half-dead or * dead (ignorable) page one or more times. There will either be a diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c index e731161734a..e8cdea7e283 100644 --- a/contrib/pg_visibility/pg_visibility.c +++ b/contrib/pg_visibility/pg_visibility.c @@ -563,17 +563,14 @@ collect_corrupt_items(Oid relid, bool all_visible, bool all_frozen) BufferAccessStrategy bstrategy = GetAccessStrategy(BAS_BULKREAD); TransactionId OldestXmin = InvalidTransactionId; - if (all_visible) - { - /* Don't pass rel; that will fail in recovery. */ - OldestXmin = GetOldestXmin(NULL, PROCARRAY_FLAGS_VACUUM); - } - rel = relation_open(relid, AccessShareLock); /* Only some relkinds have a visibility map */ check_relation_relkind(rel); + if (all_visible) + OldestXmin = GetOldestNonRemovableTransactionId(rel); + nblocks = RelationGetNumberOfBlocks(rel); /* @@ -679,11 +676,12 @@ collect_corrupt_items(Oid relid, bool all_visible, bool all_frozen) * From a concurrency point of view, it sort of sucks to * retake ProcArrayLock here while we're holding the buffer * exclusively locked, but it should be safe against - * deadlocks, because surely GetOldestXmin() should never take - * a buffer lock. And this shouldn't happen often, so it's - * worth being careful so as to avoid false positives. + * deadlocks, because surely GetOldestNonRemovableTransactionId() + * should never take a buffer lock. And this shouldn't happen + * often, so it's worth being careful so as to avoid false + * positives. */ - RecomputedOldestXmin = GetOldestXmin(NULL, PROCARRAY_FLAGS_VACUUM); + RecomputedOldestXmin = GetOldestNonRemovableTransactionId(rel); if (!TransactionIdPrecedes(OldestXmin, RecomputedOldestXmin)) record_corrupt_item(items, &tuple.t_self); diff --git a/contrib/pgstattuple/pgstatapprox.c b/contrib/pgstattuple/pgstatapprox.c index dbc0fa11f61..3a99333d443 100644 --- a/contrib/pgstattuple/pgstatapprox.c +++ b/contrib/pgstattuple/pgstatapprox.c @@ -71,7 +71,7 @@ statapprox_heap(Relation rel, output_type *stat) BufferAccessStrategy bstrategy; TransactionId OldestXmin; - OldestXmin = GetOldestXmin(rel, PROCARRAY_FLAGS_VACUUM); + OldestXmin = GetOldestNonRemovableTransactionId(rel); bstrategy = GetAccessStrategy(BAS_BULKREAD); nblocks = RelationGetNumberOfBlocks(rel); diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 7eaaad1e140..b4948ac675f 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -395,6 +395,7 @@ CompositeTypeStmt CompoundAffixFlag CompressionAlgorithm CompressorState +ComputeXidHorizonsResult ConditionVariable ConditionalStack ConfigData @@ -930,6 +931,7 @@ GistSplitVector GistTsVectorOptions GistVacState GlobalTransaction +GlobalVisState GrantRoleStmt GrantStmt GrantTargetType -- 2.25.0.114.g5b0ca878e0 --4il4azyv6rmablac Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13-0002-snapshot-scalability-Move-PGXACT-xmin-back-to-PG.patch" ^ permalink raw reply [nested|flat] 2+ messages in thread
* Re: DSA_ALLOC_NO_OOM doesn't work @ 2024-02-21 19:19 Heikki Linnakangas <[email protected]> 0 siblings, 0 replies; 2+ messages in thread From: Heikki Linnakangas @ 2024-02-21 19:19 UTC (permalink / raw) To: Robert Haas <[email protected]>; +Cc: pgsql-hackers; Thomas Munro <[email protected]> On 14/02/2024 09:23, Robert Haas wrote: > On Tue, Feb 13, 2024 at 7:53 PM Heikki Linnakangas <[email protected]> wrote: >> However, I must say that the dsm_impl_op() interface is absolutely >> insane. It's like someone looked at ioctl() and thought, "hey that's a >> great idea!". > > As the person who wrote that code, this made me laugh. > > I agree it's not the prettiest interface, but I thought that was OK > considering that it should only ever have a very limited number of > callers. I believe I did it this way in the interest of code > compactness. Since there are four DSM implementations, I wanted the > implementation-specific code to be short and all in one place, and > jamming it all into one function served that purpose. Also, there's a > bunch of logic that is shared by multiple operations - detach and > destroy tend to be similar, and so do create and attach, and there are > even things that are shared across all operations, like the snprintf > at the top of dsm_impl_posix() or the slightly larger amount of > boilerplate at the top of dsm_impl_sysv(). > > I'm not particularly opposed to refactoring this to make it nicer, but > my judgement was that splitting it up into one function per operation > per implementation, say, would have involved a lot of duplication of > small bits of code that might then get out of sync with each other > over time. By doing it this way, the logic is a bit tangled -- or > maybe more than a bit -- but there's very little duplication because > each implementation gets jammed into the smallest possible box. I'm OK > with somebody deciding that I got the trade-offs wrong there, but I > will be interested to see the number of lines added vs. removed in any > future refactoring patch. That's fair, I can see those reasons. Nevertheless, I do think it was a bad tradeoff. A little bit of repetition would be better here, or we can extract the common parts to smaller functions. I came up with the attached: 25 files changed, 1710 insertions(+), 1113 deletions(-) So yeah, it's more code, and there's some repetition, but I think this is more readable. Some of that is extra boilerplate because I split the implementations to separate files, and I also added tests. I'm not 100% wedded on all of the decisions here, but I think this is the right direction overall. For example, I decided to separate dsm_handle used by the high-level interface in dsm.c and the dsm_impl_handle used by the low-level interace in dsm_impl_*.c (more on that below). That feels slightly better to me, but could be left out if we don't want that. Notable changes: - Split the single multiplexed dsm_impl_op() function into multiple functions for different operations. This allows more natural function signatures for the different operations. - The create() function is now responsible for generating the handle, instead of having the caller generate it. Those implementations that need to generate a random handle and retry if it's already in use, now do that retry within the implementation. - The destroy() function no longer detaches the segment; you must call detach() first if the segment is still attached. This avoids having to pass "junk" values when destroying a segment that's not attached, and in case of error, makes it more clear what failed. - Separate dsm_handle, used by backend code to interact with the high level interface in dsm.c, from dsm_impl_handle, which is used to interact with the low-level functions in dsm_impl.c. This gets rid of the convention in dsm.c of reserving odd numbers for DSM segments stored in the main shmem area. There is now an explicit flag for that the control slot. For generating dsm_handles, we now use the same scheme we used to use for main-area shm segments for all DSM segments, which includes the slot number in the dsm_handle. The implementations use their own mechanisms for generating the low-level dsm_impl_handles (all but the SysV implementation generate a random handle and retry on collision). - Use IPC_PRIVATE in the SysV implementation to have the OS create a unique identifier for us. Use the shmid directly as the (low-level) handle, so that we don't need to use shmget() to convert a key to shmid, and don't need the "cache" for that. - create() no longer returns the mapped_size. The old Windows implementation had some code to read the actual mapped size after creating the mapping, and returned that in *mapped_size. Others just returned the requested size. In principle reading the actual size might be useful; the caller might be able to make use of the whole mapped size when it's larger than requested. In practice, the callers didn't do that. Also, POSIX shmem on FreeBSD has similar round-up-to-page-size behavior but the implementation did not query the actual mapped size after creating the segment, so you could not rely on it. - Added a test that exercises basic create, detach, attach functionality using all the different implementations supported on the current platform. - Change datatype of the opaque types in dsm_impl.c from "void *" to typedefs over uintptr_t. It's easy to make mistakes with "void *", as you can pass any pointer without getting warnings from the compiler. Dedicated typedefs give a bit more type checking. (This is in the first commit, all the other changes are bundled together in the second commit.) Overall, I don't think this is backpatchable. The handle changes and use of IPC_PRIVATE in particular: they could lead to failure to clean up old segments if you upgraded the binary without a clean shutdown. A slightly different version of this possibly would be, but I'd like to focus on what's best for master for now. -- Heikki Linnakangas Neon (https://neon.tech) Attachments: [text/x-patch] v1-0001-Change-datatype-of-the-opaque-types-in-dsm_impl.c.patch (12.2K, ../../[email protected]/2-v1-0001-Change-datatype-of-the-opaque-types-in-dsm_impl.c.patch) download | inline diff: From 7b29e4818bb3b780664fde2e8f5726b4f0b54b7b Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas <[email protected]> Date: Wed, 21 Feb 2024 17:15:22 +0200 Subject: [PATCH v1 1/2] Change datatype of the opaque types in dsm_impl.c It's easy to make mistakes with "void *", as you can pass any pointer without getting warnings from the compiler. Switch to a typedef over uintptr_t to get a bit more type checking. --- src/backend/storage/ipc/dsm.c | 28 +++++++++++------------ src/backend/storage/ipc/dsm_impl.c | 36 +++++++++++++++--------------- src/include/storage/dsm.h | 2 +- src/include/storage/dsm_impl.h | 21 +++++++++++++---- 4 files changed, 50 insertions(+), 37 deletions(-) diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index 6b12108dd10..d3f982c5b2a 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -70,7 +70,7 @@ struct dsm_segment ResourceOwner resowner; /* Resource owner. */ dsm_handle handle; /* Segment name. */ uint32 control_slot; /* Slot in control segment. */ - void *impl_private; /* Implementation-specific private data. */ + dsm_impl_private impl_private; /* Implementation-specific private data. */ void *mapped_address; /* Mapping address, or NULL if unmapped. */ Size mapped_size; /* Size of our mapping. */ slist_head on_detach; /* On-detach callbacks. */ @@ -83,7 +83,7 @@ typedef struct dsm_control_item uint32 refcnt; /* 2+ = active, 1 = moribund, 0 = gone */ size_t first_page; size_t npages; - void *impl_private_pm_handle; /* only needed on Windows */ + dsm_impl_private_pm_handle impl_private_pm_handle; /* only needed on Windows */ bool pinned; } dsm_control_item; @@ -140,7 +140,7 @@ static dlist_head dsm_segment_list = DLIST_STATIC_INIT(dsm_segment_list); static dsm_handle dsm_control_handle; static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; -static void *dsm_control_impl_private = NULL; +static dsm_impl_private dsm_control_impl_private = 0; /* ResourceOwner callbacks to hold DSM segments */ @@ -240,8 +240,8 @@ dsm_cleanup_using_control_segment(dsm_handle old_control_handle) { void *mapped_address = NULL; void *junk_mapped_address = NULL; - void *impl_private = NULL; - void *junk_impl_private = NULL; + dsm_impl_private impl_private = 0; + dsm_impl_private junk_impl_private = 0; Size mapped_size = 0; Size junk_mapped_size = 0; uint32 nitems; @@ -362,7 +362,7 @@ dsm_postmaster_shutdown(int code, Datum arg) uint32 i; void *dsm_control_address; void *junk_mapped_address = NULL; - void *junk_impl_private = NULL; + dsm_impl_private junk_impl_private = 0; Size junk_mapped_size = 0; PGShmemHeader *shim = (PGShmemHeader *) DatumGetPointer(arg); @@ -598,7 +598,7 @@ dsm_create(Size size, int flags) dsm_control->item[i].handle = seg->handle; /* refcnt of 1 triggers destruction, so start at 2 */ dsm_control->item[i].refcnt = 2; - dsm_control->item[i].impl_private_pm_handle = NULL; + dsm_control->item[i].impl_private_pm_handle = 0; dsm_control->item[i].pinned = false; seg->control_slot = i; LWLockRelease(DynamicSharedMemoryControlLock); @@ -637,7 +637,7 @@ dsm_create(Size size, int flags) dsm_control->item[nitems].handle = seg->handle; /* refcnt of 1 triggers destruction, so start at 2 */ dsm_control->item[nitems].refcnt = 2; - dsm_control->item[nitems].impl_private_pm_handle = NULL; + dsm_control->item[nitems].impl_private_pm_handle = 0; dsm_control->item[nitems].pinned = false; seg->control_slot = nitems; dsm_control->nitems++; @@ -842,7 +842,7 @@ dsm_detach(dsm_segment *seg) if (!is_main_region_dsm_handle(seg->handle)) dsm_impl_op(DSM_OP_DETACH, seg->handle, 0, &seg->impl_private, &seg->mapped_address, &seg->mapped_size, WARNING); - seg->impl_private = NULL; + seg->impl_private = 0; seg->mapped_address = NULL; seg->mapped_size = 0; } @@ -955,7 +955,7 @@ dsm_unpin_mapping(dsm_segment *seg) void dsm_pin_segment(dsm_segment *seg) { - void *handle = NULL; + dsm_impl_private_pm_handle pm_handle = 0; /* * Bump reference count for this segment in shared memory. This will @@ -967,10 +967,10 @@ dsm_pin_segment(dsm_segment *seg) if (dsm_control->item[seg->control_slot].pinned) elog(ERROR, "cannot pin a segment that is already pinned"); if (!is_main_region_dsm_handle(seg->handle)) - dsm_impl_pin_segment(seg->handle, seg->impl_private, &handle); + dsm_impl_pin_segment(seg->handle, seg->impl_private, &pm_handle); dsm_control->item[seg->control_slot].pinned = true; dsm_control->item[seg->control_slot].refcnt++; - dsm_control->item[seg->control_slot].impl_private_pm_handle = handle; + dsm_control->item[seg->control_slot].impl_private_pm_handle = pm_handle; LWLockRelease(DynamicSharedMemoryControlLock); } @@ -1039,7 +1039,7 @@ dsm_unpin_segment(dsm_handle handle) /* Clean up resources if that was the last reference. */ if (destroy) { - void *junk_impl_private = NULL; + dsm_impl_private junk_impl_private = 0; void *junk_mapped_address = NULL; Size junk_mapped_size = 0; @@ -1211,7 +1211,7 @@ dsm_create_descriptor(void) /* seg->handle must be initialized by the caller */ seg->control_slot = INVALID_CONTROL_SLOT; - seg->impl_private = NULL; + seg->impl_private = 0; seg->mapped_address = NULL; seg->mapped_size = 0; diff --git a/src/backend/storage/ipc/dsm_impl.c b/src/backend/storage/ipc/dsm_impl.c index 8dd669e0ce9..4478c58bb72 100644 --- a/src/backend/storage/ipc/dsm_impl.c +++ b/src/backend/storage/ipc/dsm_impl.c @@ -71,23 +71,23 @@ #ifdef USE_DSM_POSIX static bool dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size, - void **impl_private, void **mapped_address, + dsm_impl_private *impl_private, void **mapped_address, Size *mapped_size, int elevel); static int dsm_impl_posix_resize(int fd, off_t size); #endif #ifdef USE_DSM_SYSV static bool dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size, - void **impl_private, void **mapped_address, + dsm_impl_private *impl_private, void **mapped_address, Size *mapped_size, int elevel); #endif #ifdef USE_DSM_WINDOWS static bool dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size, - void **impl_private, void **mapped_address, + dsm_impl_private *impl_private, void **mapped_address, Size *mapped_size, int elevel); #endif #ifdef USE_DSM_MMAP static bool dsm_impl_mmap(dsm_op op, dsm_handle handle, Size request_size, - void **impl_private, void **mapped_address, + dsm_impl_private *impl_private, void **mapped_address, Size *mapped_size, int elevel); #endif static int errcode_for_dynamic_shared_memory(void); @@ -157,7 +157,7 @@ int min_dynamic_shared_memory; */ bool dsm_impl_op(dsm_op op, dsm_handle handle, Size request_size, - void **impl_private, void **mapped_address, Size *mapped_size, + dsm_impl_private *impl_private, void **mapped_address, Size *mapped_size, int elevel) { Assert(op == DSM_OP_CREATE || request_size == 0); @@ -210,7 +210,7 @@ dsm_impl_op(dsm_op op, dsm_handle handle, Size request_size, */ static bool dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size, - void **impl_private, void **mapped_address, Size *mapped_size, + dsm_impl_private *impl_private, void **mapped_address, Size *mapped_size, int elevel) { char name[64]; @@ -421,7 +421,7 @@ dsm_impl_posix_resize(int fd, off_t size) */ static bool dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size, - void **impl_private, void **mapped_address, Size *mapped_size, + dsm_impl_private *impl_private, void **mapped_address, Size *mapped_size, int elevel) { key_t key; @@ -477,9 +477,9 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size, * the shared memory key to a shared memory identifier using shmget(). To * avoid repeated lookups, we store the key using impl_private. */ - if (*impl_private != NULL) + if (*impl_private != 0) { - ident_cache = *impl_private; + ident_cache = (int *) *impl_private; ident = *ident_cache; } else @@ -522,14 +522,14 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size, } *ident_cache = ident; - *impl_private = ident_cache; + *impl_private = (uintptr_t) ident_cache; } /* Handle teardown cases. */ if (op == DSM_OP_DETACH || op == DSM_OP_DESTROY) { pfree(ident_cache); - *impl_private = NULL; + *impl_private = 0; if (*mapped_address != NULL && shmdt(*mapped_address) != 0) { ereport(elevel, @@ -608,7 +608,7 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size, */ static bool dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size, - void **impl_private, void **mapped_address, + dsm_impl_private *impl_private, void **mapped_address, Size *mapped_size, int elevel) { char *address; @@ -790,7 +790,7 @@ dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size, */ static bool dsm_impl_mmap(dsm_op op, dsm_handle handle, Size request_size, - void **impl_private, void **mapped_address, Size *mapped_size, + dsm_impl_private *impl_private, void **mapped_address, Size *mapped_size, int elevel) { char name[64]; @@ -960,8 +960,8 @@ dsm_impl_mmap(dsm_op op, dsm_handle handle, Size request_size, * do anything to receive the handle; Windows transfers it automatically. */ void -dsm_impl_pin_segment(dsm_handle handle, void *impl_private, - void **impl_private_pm_handle) +dsm_impl_pin_segment(dsm_handle handle, dsm_impl_private impl_private, + dsm_impl_private_pm_handle *pm_handle) { switch (dynamic_shared_memory_type) { @@ -992,7 +992,7 @@ dsm_impl_pin_segment(dsm_handle handle, void *impl_private, * matter. We're just holding onto it so that, if the segment * is unpinned, dsm_impl_unpin_segment can close it. */ - *impl_private_pm_handle = hmap; + *pm_handle = hmap; } break; #endif @@ -1011,7 +1011,7 @@ dsm_impl_pin_segment(dsm_handle handle, void *impl_private, * postmaster's process space. */ void -dsm_impl_unpin_segment(dsm_handle handle, void **impl_private) +dsm_impl_unpin_segment(dsm_handle handle, dsm_impl_private_pm_handle *pm_handle) { switch (dynamic_shared_memory_type) { @@ -1034,7 +1034,7 @@ dsm_impl_unpin_segment(dsm_handle handle, void **impl_private) name))); } - *impl_private = NULL; + *pm_handle = 0; } break; #endif diff --git a/src/include/storage/dsm.h b/src/include/storage/dsm.h index 1a22b32df1a..35ae4eb164e 100644 --- a/src/include/storage/dsm.h +++ b/src/include/storage/dsm.h @@ -13,7 +13,7 @@ #ifndef DSM_H #define DSM_H -#include "storage/dsm_impl.h" +#include "dsm_impl.h" typedef struct dsm_segment dsm_segment; diff --git a/src/include/storage/dsm_impl.h b/src/include/storage/dsm_impl.h index 882269603da..f2bfb2a1a5c 100644 --- a/src/include/storage/dsm_impl.h +++ b/src/include/storage/dsm_impl.h @@ -66,14 +66,27 @@ typedef enum DSM_OP_DESTROY, } dsm_op; +/* + * When a segment is created or attached, the caller provides this space to + * hold implementation-specific information about the attachment. It is opaque + * to the caller, and is passed back to the implementation when detaching. + */ +typedef uintptr_t dsm_impl_private; + +/* + * Similar caller-provided space for implementation-specific information held + * when a segment is pinned. + */ +typedef uintptr_t dsm_impl_private_pm_handle; + /* Create, attach to, detach from, resize, or destroy a segment. */ extern bool dsm_impl_op(dsm_op op, dsm_handle handle, Size request_size, - void **impl_private, void **mapped_address, Size *mapped_size, + dsm_impl_private *impl_private, void **mapped_address, Size *mapped_size, int elevel); /* Implementation-dependent actions required to keep segment until shutdown. */ -extern void dsm_impl_pin_segment(dsm_handle handle, void *impl_private, - void **impl_private_pm_handle); -extern void dsm_impl_unpin_segment(dsm_handle handle, void **impl_private); +extern void dsm_impl_pin_segment(dsm_handle handle, dsm_impl_private impl_private, + dsm_impl_private_pm_handle *impl_private_pm_handle); +extern void dsm_impl_unpin_segment(dsm_handle handle, dsm_impl_private_pm_handle *pm_handle); #endif /* DSM_IMPL_H */ -- 2.39.2 [text/x-patch] v1-0002-Rewrite-the-dsm_impl-interface.patch (107.6K, ../../[email protected]/3-v1-0002-Rewrite-the-dsm_impl-interface.patch) download | inline diff: From ce0b0ae4ca4cc6046ec20baaf627bba1a9cc7112 Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas <[email protected]> Date: Wed, 21 Feb 2024 21:09:59 +0200 Subject: [PATCH v1 2/2] Rewrite the dsm_impl interface Notable changes: - Split the single multiplexed dsm_impl_op() function into multiple functions for different operations. This allows more natural function signatures for the different operations. - The create() function is now responsible for generating the handle, instead of having the caller generate it. Those implementations that need to generate a random handle and retry if it's already in use now do that retry within the implementation. - The destroy() function no longer detaches the segment; you must call detach() first, if the segment is still attached. This avoids having to pass "junk" values when destoying a segment that's not attached, and in case of error, makes it more clear what failed. - Separate dsm_handle, used by backend code to interact with the high-level interface in dsm.c, from dsm_impl_handle, which is used to interact with the low-level functions in dsm_impl.c. This gets rid of the convention in dsm.c of reserving odd numbers for DSM segments stored in the main shmem area. There is now an explicit flag for that the control slot. For generating dsm_handles, we now use the same scheme we used to use for main-area shm segments for all DSM segments, which includes the slot number in the dsm_handle. The implementations use their own mechanisms for generating the low-level dsm_impl_handles (all but the SysV implementation generate a random handle and retry on collision). - Use IPC_PRIVATE in the SysV implementation to have the OS create a unique identifier for us. Use the shmid directly as the (low-level) handle, so that we don't need to use shmget() to convert a key to shmid, and don't need the cache for that. - create() no longer returns the mapped_size. The old Windows implementation had some code to read the actual mapped size after creating the mapping, and returned that in *mapped_size. In principle that might be useful; the caller might be able to make use of the whole mapped size when it's larger than requested. In practice, the callers didn't do that. Also, POSIX shmem on FreeBSD has similar round-up-to-page-size behavior but the implementation did not query the actual mapped size after creating the segment, so you could not rely on it. - Added a test that exercises basic create, detach, attach functionality using all the different implementations supported on the current platform. Discussion: https://www.postgresql.org/message-id/[email protected] --- src/backend/port/sysv_shmem.c | 4 +- src/backend/port/win32_shmem.c | 2 +- src/backend/storage/ipc/Makefile | 10 + src/backend/storage/ipc/dsm.c | 273 ++--- src/backend/storage/ipc/dsm_impl.c | 984 +------------------ src/backend/storage/ipc/dsm_impl_mmap.c | 300 ++++++ src/backend/storage/ipc/dsm_impl_posix.c | 341 +++++++ src/backend/storage/ipc/dsm_impl_sysv.c | 224 +++++ src/backend/storage/ipc/dsm_impl_windows.c | 345 +++++++ src/backend/storage/ipc/meson.build | 12 + src/backend/utils/misc/guc_tables.c | 2 +- src/include/storage/dsm.h | 10 +- src/include/storage/dsm_impl.h | 99 +- src/include/storage/pg_shmem.h | 2 +- src/include/utils/guc_hooks.h | 2 + src/test/modules/Makefile | 1 + src/test/modules/meson.build | 1 + src/test/modules/test_dsm/.gitignore | 3 + src/test/modules/test_dsm/Makefile | 27 + src/test/modules/test_dsm/meson.build | 33 + src/test/modules/test_dsm/t/001_dsm_basic.pl | 61 ++ src/test/modules/test_dsm/test_dsm--1.0.sql | 9 + src/test/modules/test_dsm/test_dsm.c | 75 ++ src/test/modules/test_dsm/test_dsm.control | 4 + src/tools/pgindent/typedefs.list | 3 + 25 files changed, 1714 insertions(+), 1113 deletions(-) create mode 100644 src/backend/storage/ipc/dsm_impl_mmap.c create mode 100644 src/backend/storage/ipc/dsm_impl_posix.c create mode 100644 src/backend/storage/ipc/dsm_impl_sysv.c create mode 100644 src/backend/storage/ipc/dsm_impl_windows.c create mode 100644 src/test/modules/test_dsm/.gitignore create mode 100644 src/test/modules/test_dsm/Makefile create mode 100644 src/test/modules/test_dsm/meson.build create mode 100644 src/test/modules/test_dsm/t/001_dsm_basic.pl create mode 100644 src/test/modules/test_dsm/test_dsm--1.0.sql create mode 100644 src/test/modules/test_dsm/test_dsm.c create mode 100644 src/test/modules/test_dsm/test_dsm.control diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c index 9a96329bf25..b9bf2979f5d 100644 --- a/src/backend/port/sysv_shmem.c +++ b/src/backend/port/sysv_shmem.c @@ -827,7 +827,7 @@ PGSharedMemoryCreate(Size size, * if some other process creates the same shmem key before we * do, in which case we'll try the next key. */ - if (oldhdr->dsm_control != 0) + if (oldhdr->dsm_control != DSM_IMPL_HANDLE_INVALID) dsm_cleanup_using_control_segment(oldhdr->dsm_control); if (shmctl(shmid, IPC_RMID, NULL) < 0) NextShmemSegID++; @@ -842,7 +842,7 @@ PGSharedMemoryCreate(Size size, hdr = (PGShmemHeader *) memAddress; hdr->creatorPID = getpid(); hdr->magic = PGShmemMagic; - hdr->dsm_control = 0; + hdr->dsm_control = DSM_IMPL_HANDLE_INVALID; /* Fill in the data directory ID info, too */ hdr->device = statbuf.st_dev; diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c index 90bed0146dd..b23686dd49d 100644 --- a/src/backend/port/win32_shmem.c +++ b/src/backend/port/win32_shmem.c @@ -390,7 +390,7 @@ retry: */ hdr->totalsize = size; hdr->freeoffset = MAXALIGN(sizeof(PGShmemHeader)); - hdr->dsm_control = 0; + hdr->dsm_control = DSM_IMPL_HANDLE_INVALID; /* Save info for possible future use */ UsedShmemSegAddr = memAddress; diff --git a/src/backend/storage/ipc/Makefile b/src/backend/storage/ipc/Makefile index d8a1653eb6a..4e600f77d1c 100644 --- a/src/backend/storage/ipc/Makefile +++ b/src/backend/storage/ipc/Makefile @@ -27,4 +27,14 @@ OBJS = \ sinvaladt.o \ standby.o +ifeq ($(PORTNAME), win32) +OBJS += dsm_impl_windows.o +else +# TODO: Only build the POSIX and System V implementatiions on +# platforms where they are available. Currently there are large #ifdef +# blocks in the source files, but would be nicer to skip compiling +# them altogether. +OBJS += dsm_impl_posix.o dsm_impl_sysv.o dsm_impl_mmap.o +endif + include $(top_srcdir)/src/backend/common.mk diff --git a/src/backend/storage/ipc/dsm.c b/src/backend/storage/ipc/dsm.c index d3f982c5b2a..e0775d8a984 100644 --- a/src/backend/storage/ipc/dsm.c +++ b/src/backend/storage/ipc/dsm.c @@ -14,6 +14,18 @@ * hard postmaster crash, remaining segments will be removed, if they * still exist, at the next postmaster startup. * + * These services manage two kinds of segments: + * + * 1. Segments carved out of one "main" DSM segment + * 2. Segments backed by a separate low-level DSM segments + * + * Each segment is identified by a 32-bit integer handle (dsm_handle), + * whether it's carved out of the main region or created as a separate + * segment. The handle can be used by other processes to find and attach + * to the segment. Segments that are backed by a dedicated low-level + * segment also have a separate dsm_impl_handle, which is used with the + * low-level functions in dsm_impl.c. + * * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group * Portions Copyright (c) 1994, Regents of the University of California * @@ -68,7 +80,10 @@ struct dsm_segment { dlist_node node; /* List link in dsm_segment_list. */ ResourceOwner resowner; /* Resource owner. */ - dsm_handle handle; /* Segment name. */ + dsm_handle handle; /* Handle for other backends to find this + * segment. */ + dsm_impl_handle impl_handle; /* Implementation-specific handle. */ + bool in_main_region; /* Is this stored in the main shm region? */ uint32 control_slot; /* Slot in control segment. */ dsm_impl_private impl_private; /* Implementation-specific private data. */ void *mapped_address; /* Mapping address, or NULL if unmapped. */ @@ -80,10 +95,13 @@ struct dsm_segment typedef struct dsm_control_item { dsm_handle handle; + dsm_impl_handle impl_handle; + bool in_main_region; uint32 refcnt; /* 2+ = active, 1 = moribund, 0 = gone */ size_t first_page; size_t npages; - dsm_impl_private_pm_handle impl_private_pm_handle; /* only needed on Windows */ + dsm_impl_private_pm_handle impl_private_pm_handle; /* only needed on + * Windows */ bool pinned; } dsm_control_item; @@ -102,8 +120,7 @@ static dsm_segment *dsm_create_descriptor(void); static bool dsm_control_segment_sane(dsm_control_header *control, Size mapped_size); static uint64 dsm_control_bytes_needed(uint32 nitems); -static inline dsm_handle make_main_region_dsm_handle(int slot); -static inline bool is_main_region_dsm_handle(dsm_handle handle); +static inline dsm_handle make_dsm_handle(int slot); /* Has this backend initialized the dynamic shared memory system yet? */ static bool dsm_init_done = false; @@ -137,7 +154,7 @@ static dlist_head dsm_segment_list = DLIST_STATIC_INIT(dsm_segment_list); * reference counted; instead, it lasts for the postmaster's entire * life cycle. For simplicity, it doesn't have a dsm_segment object either. */ -static dsm_handle dsm_control_handle; +static dsm_impl_handle dsm_control_handle; static dsm_control_header *dsm_control; static Size dsm_control_mapped_size = 0; static dsm_impl_private dsm_control_impl_private = 0; @@ -199,32 +216,19 @@ dsm_postmaster_startup(PGShmemHeader *shim) maxitems); segsize = dsm_control_bytes_needed(maxitems); - /* - * Loop until we find an unused identifier for the new control segment. We - * sometimes use DSM_HANDLE_INVALID as a sentinel value indicating "no - * control segment", so avoid generating that value for a real handle. - */ - for (;;) - { - Assert(dsm_control_address == NULL); - Assert(dsm_control_mapped_size == 0); - /* Use even numbers only */ - dsm_control_handle = pg_prng_uint32(&pg_global_prng_state) << 1; - if (dsm_control_handle == DSM_HANDLE_INVALID) - continue; - if (dsm_impl_op(DSM_OP_CREATE, dsm_control_handle, segsize, - &dsm_control_impl_private, &dsm_control_address, - &dsm_control_mapped_size, ERROR)) - break; - } + /* Create the control segment. */ + dsm_control_handle = dsm_impl->create(segsize, + &dsm_control_impl_private, &dsm_control_address, + ERROR); dsm_control = dsm_control_address; + dsm_control_mapped_size = segsize; on_shmem_exit(dsm_postmaster_shutdown, PointerGetDatum(shim)); elog(DEBUG2, "created dynamic shared memory control segment %u (%zu bytes)", dsm_control_handle, segsize); shim->dsm_control = dsm_control_handle; - /* Initialize control segment. */ + /* Initialize it. */ dsm_control->magic = PG_DYNSHMEM_CONTROL_MAGIC; dsm_control->nitems = 0; dsm_control->maxitems = maxitems; @@ -236,14 +240,11 @@ dsm_postmaster_startup(PGShmemHeader *shim) * segments to which it refers, and then the control segment itself. */ void -dsm_cleanup_using_control_segment(dsm_handle old_control_handle) +dsm_cleanup_using_control_segment(dsm_impl_handle old_control_handle) { void *mapped_address = NULL; - void *junk_mapped_address = NULL; dsm_impl_private impl_private = 0; - dsm_impl_private junk_impl_private = 0; Size mapped_size = 0; - Size junk_mapped_size = 0; uint32 nitems; uint32 i; dsm_control_header *old_control; @@ -254,8 +255,8 @@ dsm_cleanup_using_control_segment(dsm_handle old_control_handle) * exists, or an unrelated process has used the same shm ID. So just fall * out quietly. */ - if (!dsm_impl_op(DSM_OP_ATTACH, old_control_handle, 0, &impl_private, - &mapped_address, &mapped_size, DEBUG1)) + if (!dsm_impl->attach(old_control_handle, &impl_private, + &mapped_address, &mapped_size, DEBUG1)) return; /* @@ -265,8 +266,8 @@ dsm_cleanup_using_control_segment(dsm_handle old_control_handle) old_control = (dsm_control_header *) mapped_address; if (!dsm_control_segment_sane(old_control, mapped_size)) { - dsm_impl_op(DSM_OP_DETACH, old_control_handle, 0, &impl_private, - &mapped_address, &mapped_size, LOG); + dsm_impl->detach(old_control_handle, impl_private, + mapped_address, mapped_size, LOG); return; } @@ -277,7 +278,7 @@ dsm_cleanup_using_control_segment(dsm_handle old_control_handle) nitems = old_control->nitems; for (i = 0; i < nitems; ++i) { - dsm_handle handle; + dsm_impl_handle handle; uint32 refcnt; /* If the reference count is 0, the slot is actually unused. */ @@ -286,8 +287,8 @@ dsm_cleanup_using_control_segment(dsm_handle old_control_handle) continue; /* If it was using the main shmem area, there is nothing to do. */ - handle = old_control->item[i].handle; - if (is_main_region_dsm_handle(handle)) + handle = old_control->item[i].impl_handle; + if (old_control->item[i].in_main_region) continue; /* Log debugging information. */ @@ -295,16 +296,16 @@ dsm_cleanup_using_control_segment(dsm_handle old_control_handle) handle, refcnt); /* Destroy the referenced segment. */ - dsm_impl_op(DSM_OP_DESTROY, handle, 0, &junk_impl_private, - &junk_mapped_address, &junk_mapped_size, LOG); + dsm_impl->destroy(handle, LOG); } /* Destroy the old control segment, too. */ elog(DEBUG2, "cleaning up dynamic shared memory control segment with ID %u", old_control_handle); - dsm_impl_op(DSM_OP_DESTROY, old_control_handle, 0, &impl_private, - &mapped_address, &mapped_size, LOG); + dsm_impl->detach(old_control_handle, impl_private, + mapped_address, mapped_size, LOG); + dsm_impl->destroy(old_control_handle, LOG); } /* @@ -361,9 +362,6 @@ dsm_postmaster_shutdown(int code, Datum arg) uint32 nitems; uint32 i; void *dsm_control_address; - void *junk_mapped_address = NULL; - dsm_impl_private junk_impl_private = 0; - Size junk_mapped_size = 0; PGShmemHeader *shim = (PGShmemHeader *) DatumGetPointer(arg); /* @@ -384,23 +382,23 @@ dsm_postmaster_shutdown(int code, Datum arg) /* Remove any remaining segments. */ for (i = 0; i < nitems; ++i) { - dsm_handle handle; + dsm_impl_handle handle; /* If the reference count is 0, the slot is actually unused. */ if (dsm_control->item[i].refcnt == 0) continue; - handle = dsm_control->item[i].handle; - if (is_main_region_dsm_handle(handle)) + if (dsm_control->item[i].in_main_region) continue; + handle = dsm_control->item[i].impl_handle; + /* Log debugging information. */ elog(DEBUG2, "cleaning up orphaned dynamic shared memory with ID %u", handle); /* Destroy the segment. */ - dsm_impl_op(DSM_OP_DESTROY, handle, 0, &junk_impl_private, - &junk_mapped_address, &junk_mapped_size, LOG); + dsm_impl->destroy(handle, LOG); } /* Remove the control segment itself. */ @@ -408,11 +406,13 @@ dsm_postmaster_shutdown(int code, Datum arg) "cleaning up dynamic shared memory control segment with ID %u", dsm_control_handle); dsm_control_address = dsm_control; - dsm_impl_op(DSM_OP_DESTROY, dsm_control_handle, 0, - &dsm_control_impl_private, &dsm_control_address, - &dsm_control_mapped_size, LOG); - dsm_control = dsm_control_address; - shim->dsm_control = 0; + dsm_impl->detach(dsm_control_handle, + dsm_control_impl_private, dsm_control_address, + dsm_control_mapped_size, LOG); + dsm_impl->destroy(dsm_control_handle, LOG); + dsm_control = NULL; + dsm_control_mapped_size = 0; + shim->dsm_control = DSM_IMPL_HANDLE_INVALID; } /* @@ -429,17 +429,17 @@ dsm_backend_startup(void) void *control_address = NULL; /* Attach control segment. */ - Assert(dsm_control_handle != 0); - dsm_impl_op(DSM_OP_ATTACH, dsm_control_handle, 0, - &dsm_control_impl_private, &control_address, - &dsm_control_mapped_size, ERROR); + Assert(dsm_control_handle != DSM_HANDLE_INVALID); + dsm_impl->attach(dsm_control_handle, + &dsm_control_impl_private, &control_address, + &dsm_control_mapped_size, ERROR); dsm_control = control_address; /* If control segment doesn't look sane, something is badly wrong. */ if (!dsm_control_segment_sane(dsm_control, dsm_control_mapped_size)) { - dsm_impl_op(DSM_OP_DETACH, dsm_control_handle, 0, - &dsm_control_impl_private, &control_address, - &dsm_control_mapped_size, WARNING); + dsm_impl->detach(dsm_control_handle, + dsm_control_impl_private, control_address, + dsm_control_mapped_size, WARNING); ereport(FATAL, (errcode(ERRCODE_INTERNAL_ERROR), errmsg("dynamic shared memory control segment is not valid"))); @@ -459,7 +459,8 @@ dsm_backend_startup(void) void dsm_set_control_handle(dsm_handle h) { - Assert(dsm_control_handle == 0 && h != 0); + Assert(dsm_control_handle == DSM_HANDLE_INVALID); + Assert(h != DSM_HANDLE_INVALID); dsm_control_handle = h; } #endif @@ -567,19 +568,15 @@ dsm_create(Size size, int flags) */ if (dsm_main_space_fpm) LWLockRelease(DynamicSharedMemoryControlLock); - for (;;) - { - Assert(seg->mapped_address == NULL && seg->mapped_size == 0); - /* Use even numbers only */ - seg->handle = pg_prng_uint32(&pg_global_prng_state) << 1; - if (seg->handle == DSM_HANDLE_INVALID) /* Reserve sentinel */ - continue; - if (dsm_impl_op(DSM_OP_CREATE, seg->handle, size, &seg->impl_private, - &seg->mapped_address, &seg->mapped_size, ERROR)) - break; - } + + seg->impl_handle = dsm_impl->create(size, &seg->impl_private, + &seg->mapped_address, ERROR); + seg->mapped_size = size; LWLockAcquire(DynamicSharedMemoryControlLock, LW_EXCLUSIVE); } + else + seg->impl_handle = DSM_IMPL_HANDLE_INVALID; + seg->in_main_region = using_main_dsm_region; /* Search the control segment for an unused slot. */ nitems = dsm_control->nitems; @@ -587,15 +584,16 @@ dsm_create(Size size, int flags) { if (dsm_control->item[i].refcnt == 0) { + seg->handle = make_dsm_handle(i); + seg->in_main_region = using_main_dsm_region; + dsm_control->item[i].in_main_region = using_main_dsm_region; + dsm_control->item[i].handle = seg->handle; if (using_main_dsm_region) { - seg->handle = make_main_region_dsm_handle(i); dsm_control->item[i].first_page = first_page; dsm_control->item[i].npages = npages; } - else - Assert(!is_main_region_dsm_handle(seg->handle)); - dsm_control->item[i].handle = seg->handle; + dsm_control->item[i].impl_handle = seg->impl_handle; /* refcnt of 1 triggers destruction, so start at 2 */ dsm_control->item[i].refcnt = 2; dsm_control->item[i].impl_private_pm_handle = 0; @@ -613,8 +611,13 @@ dsm_create(Size size, int flags) FreePageManagerPut(dsm_main_space_fpm, first_page, npages); LWLockRelease(DynamicSharedMemoryControlLock); if (!using_main_dsm_region) - dsm_impl_op(DSM_OP_DESTROY, seg->handle, 0, &seg->impl_private, - &seg->mapped_address, &seg->mapped_size, WARNING); + { + dsm_impl->detach(seg->impl_handle, seg->impl_private, + seg->mapped_address, seg->mapped_size, WARNING); + seg->mapped_address = NULL; + seg->mapped_size = 0; + dsm_impl->destroy(seg->impl_handle, WARNING); + } if (seg->resowner != NULL) ResourceOwnerForgetDSM(seg->resowner, seg); dlist_delete(&seg->node); @@ -628,13 +631,15 @@ dsm_create(Size size, int flags) } /* Enter the handle into a new array slot. */ + seg->handle = make_dsm_handle(nitems); + dsm_control->item[nitems].in_main_region = using_main_dsm_region; + dsm_control->item[nitems].handle = seg->handle; if (using_main_dsm_region) { - seg->handle = make_main_region_dsm_handle(nitems); - dsm_control->item[i].first_page = first_page; - dsm_control->item[i].npages = npages; + dsm_control->item[nitems].first_page = first_page; + dsm_control->item[nitems].npages = npages; } - dsm_control->item[nitems].handle = seg->handle; + dsm_control->item[nitems].impl_handle = seg->impl_handle; /* refcnt of 1 triggers destruction, so start at 2 */ dsm_control->item[nitems].refcnt = 2; dsm_control->item[nitems].impl_private_pm_handle = 0; @@ -719,7 +724,9 @@ dsm_attach(dsm_handle h) /* Otherwise we've found a match. */ dsm_control->item[i].refcnt++; seg->control_slot = i; - if (is_main_region_dsm_handle(seg->handle)) + seg->in_main_region = dsm_control->item[i].in_main_region; + seg->impl_handle = dsm_control->item[i].impl_handle; + if (seg->in_main_region) { seg->mapped_address = (char *) dsm_main_space_begin + dsm_control->item[i].first_page * FPM_PAGE_SIZE; @@ -742,9 +749,9 @@ dsm_attach(dsm_handle h) } /* Here's where we actually try to map the segment. */ - if (!is_main_region_dsm_handle(seg->handle)) - dsm_impl_op(DSM_OP_ATTACH, seg->handle, 0, &seg->impl_private, - &seg->mapped_address, &seg->mapped_size, ERROR); + if (!seg->in_main_region) + dsm_impl->attach(seg->impl_handle, &seg->impl_private, + &seg->mapped_address, &seg->mapped_size, ERROR); return seg; } @@ -786,9 +793,13 @@ dsm_detach_all(void) } if (control_address != NULL) - dsm_impl_op(DSM_OP_DETACH, dsm_control_handle, 0, - &dsm_control_impl_private, &control_address, - &dsm_control_mapped_size, ERROR); + { + dsm_impl->detach(dsm_control_handle, + dsm_control_impl_private, control_address, + dsm_control_mapped_size, ERROR); + dsm_control = NULL; + dsm_control_mapped_size = 0; + } } /* @@ -839,9 +850,9 @@ dsm_detach(dsm_segment *seg) */ if (seg->mapped_address != NULL) { - if (!is_main_region_dsm_handle(seg->handle)) - dsm_impl_op(DSM_OP_DETACH, seg->handle, 0, &seg->impl_private, - &seg->mapped_address, &seg->mapped_size, WARNING); + if (!seg->in_main_region) + dsm_impl->detach(seg->impl_handle, seg->impl_private, + seg->mapped_address, seg->mapped_size, WARNING); seg->impl_private = 0; seg->mapped_address = NULL; seg->mapped_size = 0; @@ -855,6 +866,8 @@ dsm_detach(dsm_segment *seg) LWLockAcquire(DynamicSharedMemoryControlLock, LW_EXCLUSIVE); Assert(dsm_control->item[control_slot].handle == seg->handle); + Assert(dsm_control->item[control_slot].in_main_region == seg->in_main_region); + Assert(dsm_control->item[control_slot].impl_handle == seg->impl_handle); Assert(dsm_control->item[control_slot].refcnt > 1); refcnt = --dsm_control->item[control_slot].refcnt; seg->control_slot = INVALID_CONTROL_SLOT; @@ -881,16 +894,17 @@ dsm_detach(dsm_segment *seg) * other reason, the postmaster may not have any better luck than * we did. There's not much we can do about that, though. */ - if (is_main_region_dsm_handle(seg->handle) || - dsm_impl_op(DSM_OP_DESTROY, seg->handle, 0, &seg->impl_private, - &seg->mapped_address, &seg->mapped_size, WARNING)) + if (seg->in_main_region || + dsm_impl->destroy(seg->impl_handle, WARNING)) { LWLockAcquire(DynamicSharedMemoryControlLock, LW_EXCLUSIVE); - if (is_main_region_dsm_handle(seg->handle)) + if (seg->in_main_region) FreePageManagerPut((FreePageManager *) dsm_main_space_begin, dsm_control->item[control_slot].first_page, dsm_control->item[control_slot].npages); Assert(dsm_control->item[control_slot].handle == seg->handle); + Assert(dsm_control->item[control_slot].in_main_region == seg->in_main_region); + Assert(dsm_control->item[control_slot].impl_handle == seg->impl_handle); Assert(dsm_control->item[control_slot].refcnt == 1); dsm_control->item[control_slot].refcnt = 0; LWLockRelease(DynamicSharedMemoryControlLock); @@ -966,8 +980,8 @@ dsm_pin_segment(dsm_segment *seg) LWLockAcquire(DynamicSharedMemoryControlLock, LW_EXCLUSIVE); if (dsm_control->item[seg->control_slot].pinned) elog(ERROR, "cannot pin a segment that is already pinned"); - if (!is_main_region_dsm_handle(seg->handle)) - dsm_impl_pin_segment(seg->handle, seg->impl_private, &pm_handle); + if (!seg->in_main_region) + dsm_impl->pin_segment(seg->impl_handle, seg->impl_private, &pm_handle); dsm_control->item[seg->control_slot].pinned = true; dsm_control->item[seg->control_slot].refcnt++; dsm_control->item[seg->control_slot].impl_private_pm_handle = pm_handle; @@ -991,6 +1005,8 @@ dsm_unpin_segment(dsm_handle handle) uint32 control_slot = INVALID_CONTROL_SLOT; bool destroy = false; uint32 i; + bool in_main_region = false; + dsm_impl_handle impl_handle = DSM_IMPL_HANDLE_INVALID; /* Find the control slot for the given handle. */ LWLockAcquire(DynamicSharedMemoryControlLock, LW_EXCLUSIVE); @@ -1004,6 +1020,8 @@ dsm_unpin_segment(dsm_handle handle) if (dsm_control->item[i].handle == handle) { control_slot = i; + in_main_region = dsm_control->item[i].in_main_region; + impl_handle = dsm_control->item[i].impl_handle; break; } } @@ -1024,9 +1042,12 @@ dsm_unpin_segment(dsm_handle handle) * releasing the lock, because impl_private_pm_handle may get modified by * dsm_impl_unpin_segment. */ - if (!is_main_region_dsm_handle(handle)) - dsm_impl_unpin_segment(handle, - &dsm_control->item[control_slot].impl_private_pm_handle); + if (!in_main_region) + { + dsm_impl->unpin_segment(impl_handle, + dsm_control->item[control_slot].impl_private_pm_handle); + dsm_control->item[control_slot].impl_private_pm_handle = 0; + } /* Note that 1 means no references (0 means unused slot). */ if (--dsm_control->item[control_slot].refcnt == 1) @@ -1039,26 +1060,15 @@ dsm_unpin_segment(dsm_handle handle) /* Clean up resources if that was the last reference. */ if (destroy) { - dsm_impl_private junk_impl_private = 0; - void *junk_mapped_address = NULL; - Size junk_mapped_size = 0; - /* * For an explanation of how error handling works in this case, see - * comments in dsm_detach. Note that if we reach this point, the - * current process certainly does not have the segment mapped, because - * if it did, the reference count would have still been greater than 1 - * even after releasing the reference count held by the pin. The fact - * that there can't be a dsm_segment for this handle makes it OK to - * pass the mapped size, mapped address, and private data as NULL - * here. + * comments in dsm_detach. */ - if (is_main_region_dsm_handle(handle) || - dsm_impl_op(DSM_OP_DESTROY, handle, 0, &junk_impl_private, - &junk_mapped_address, &junk_mapped_size, WARNING)) + if (in_main_region || + dsm_impl->destroy(impl_handle, WARNING)) { LWLockAcquire(DynamicSharedMemoryControlLock, LW_EXCLUSIVE); - if (is_main_region_dsm_handle(handle)) + if (in_main_region) FreePageManagerPut((FreePageManager *) dsm_main_space_begin, dsm_control->item[control_slot].first_page, dsm_control->item[control_slot].npages); @@ -1209,7 +1219,10 @@ dsm_create_descriptor(void) seg = MemoryContextAlloc(TopMemoryContext, sizeof(dsm_segment)); dlist_push_head(&dsm_segment_list, &seg->node); - /* seg->handle must be initialized by the caller */ + /* + * seg->handle, seg->in_main_region, and seg->impl_handle must be + * initialized by the caller. + */ seg->control_slot = INVALID_CONTROL_SLOT; seg->impl_private = 0; seg->mapped_address = NULL; @@ -1259,29 +1272,25 @@ dsm_control_bytes_needed(uint32 nitems) + sizeof(dsm_control_item) * (uint64) nitems; } +/* + * Generate a new handle that can be used by any backend process to identify a + * DSM segment. + */ static inline dsm_handle -make_main_region_dsm_handle(int slot) +make_dsm_handle(int slot) { dsm_handle handle; /* - * We need to create a handle that doesn't collide with any existing extra - * segment created by dsm_impl_op(), so we'll make it odd. It also - * mustn't collide with any other main area pseudo-segment, so we'll - * include the slot number in some of the bits. We also want to make an - * effort to avoid newly created and recently destroyed handles from being - * confused, so we'll make the rest of the bits random. + * It mustn't collide with any other segment, so we include the slot + * number in some of the bits. We also want to make an effort to avoid + * newly created and recently destroyed handles from being confused, so we + * make the rest of the bits random. */ - handle = 1; - handle |= slot << 1; - handle |= pg_prng_uint32(&pg_global_prng_state) << (pg_leftmost_one_pos32(dsm_control->maxitems) + 1); - return handle; -} + handle = slot; + handle |= pg_prng_uint32(&pg_global_prng_state) << (pg_leftmost_one_pos32(dsm_control->maxitems)); -static inline bool -is_main_region_dsm_handle(dsm_handle handle) -{ - return handle & 1; + return handle; } /* ResourceOwner callbacks */ diff --git a/src/backend/storage/ipc/dsm_impl.c b/src/backend/storage/ipc/dsm_impl.c index 4478c58bb72..ced298a8726 100644 --- a/src/backend/storage/ipc/dsm_impl.c +++ b/src/backend/storage/ipc/dsm_impl.c @@ -3,7 +3,7 @@ * dsm_impl.c * manage dynamic shared memory segments * - * This file provides low-level APIs for creating and destroying shared + * 'dsm_impl' provides low-level APIs for creating and destroying shared * memory segments using several different possible techniques. We refer * to these segments as dynamic because they can be created, altered, and * destroyed at any point during the server life cycle. This is unlike @@ -36,6 +36,11 @@ * * As ever, Windows requires its own implementation. * + * The different implementations are in separate source files in this + * directory. This file contains a few functions shared by different + * implementations and the GUC support to route calls to the currently + * active implementation. + * * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group * Portions Copyright (c) 1994, Regents of the University of California * @@ -48,49 +53,8 @@ #include "postgres.h" -#include <fcntl.h> -#include <signal.h> -#include <unistd.h> -#ifndef WIN32 -#include <sys/mman.h> -#include <sys/ipc.h> -#include <sys/shm.h> -#include <sys/stat.h> -#endif - -#include "common/file_perm.h" -#include "libpq/pqsignal.h" -#include "miscadmin.h" -#include "pgstat.h" -#include "portability/mem.h" -#include "postmaster/postmaster.h" #include "storage/dsm_impl.h" -#include "storage/fd.h" -#include "utils/guc.h" -#include "utils/memutils.h" - -#ifdef USE_DSM_POSIX -static bool dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size, - dsm_impl_private *impl_private, void **mapped_address, - Size *mapped_size, int elevel); -static int dsm_impl_posix_resize(int fd, off_t size); -#endif -#ifdef USE_DSM_SYSV -static bool dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size, - dsm_impl_private *impl_private, void **mapped_address, - Size *mapped_size, int elevel); -#endif -#ifdef USE_DSM_WINDOWS -static bool dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size, - dsm_impl_private *impl_private, void **mapped_address, - Size *mapped_size, int elevel); -#endif -#ifdef USE_DSM_MMAP -static bool dsm_impl_mmap(dsm_op op, dsm_handle handle, Size request_size, - dsm_impl_private *impl_private, void **mapped_address, - Size *mapped_size, int elevel); -#endif -static int errcode_for_dynamic_shared_memory(void); +#include "utils/guc_hooks.h" const struct config_enum_entry dynamic_shared_memory_options[] = { #ifdef USE_DSM_POSIX @@ -114,940 +78,60 @@ int dynamic_shared_memory_type = DEFAULT_DYNAMIC_SHARED_MEMORY_TYPE; /* Amount of space reserved for DSM segments in the main area. */ int min_dynamic_shared_memory; -/* Size of buffer to be used for zero-filling. */ -#define ZBUFFER_SIZE 8192 +const dsm_impl_ops *dsm_impl; -#define SEGMENT_NAME_PREFIX "Global/PostgreSQL" - -/*------ - * Perform a low-level shared memory operation in a platform-specific way, - * as dictated by the selected implementation. Each implementation is - * required to implement the following primitives. - * - * DSM_OP_CREATE. Create a segment whose size is the request_size and - * map it. - * - * DSM_OP_ATTACH. Map the segment, whose size must be the request_size. - * - * DSM_OP_DETACH. Unmap the segment. - * - * DSM_OP_DESTROY. Unmap the segment, if it is mapped. Destroy the - * segment. - * - * Arguments: - * op: The operation to be performed. - * handle: The handle of an existing object, or for DSM_OP_CREATE, the - * identifier for the new handle the caller wants created. - * request_size: For DSM_OP_CREATE, the requested size. Otherwise, 0. - * impl_private: Private, implementation-specific data. Will be a pointer - * to NULL for the first operation on a shared memory segment within this - * backend; thereafter, it will point to the value to which it was set - * on the previous call. - * mapped_address: Pointer to start of current mapping; pointer to NULL - * if none. Updated with new mapping address. - * mapped_size: Pointer to size of current mapping; pointer to 0 if none. - * Updated with new mapped size. - * elevel: Level at which to log errors. - * - * Return value: true on success, false on failure. When false is returned, - * a message should first be logged at the specified elevel, except in the - * case where DSM_OP_CREATE experiences a name collision, which should - * silently return false. - *----- - */ -bool -dsm_impl_op(dsm_op op, dsm_handle handle, Size request_size, - dsm_impl_private *impl_private, void **mapped_address, Size *mapped_size, - int elevel) +int +errcode_for_dynamic_shared_memory(void) { - Assert(op == DSM_OP_CREATE || request_size == 0); - Assert((op != DSM_OP_CREATE && op != DSM_OP_ATTACH) || - (*mapped_address == NULL && *mapped_size == 0)); + if (errno == EFBIG || errno == ENOMEM) + return errcode(ERRCODE_OUT_OF_MEMORY); + else + return errcode_for_file_access(); +} - switch (dynamic_shared_memory_type) +void +assign_dynamic_shared_memory_type(int new_dynamic_shared_memory_type, void *extra) +{ + switch (new_dynamic_shared_memory_type) { #ifdef USE_DSM_POSIX case DSM_IMPL_POSIX: - return dsm_impl_posix(op, handle, request_size, impl_private, - mapped_address, mapped_size, elevel); + dsm_impl = &dsm_impl_posix_ops; + break; #endif #ifdef USE_DSM_SYSV case DSM_IMPL_SYSV: - return dsm_impl_sysv(op, handle, request_size, impl_private, - mapped_address, mapped_size, elevel); + dsm_impl = &dsm_impl_sysv_ops; + break; #endif #ifdef USE_DSM_WINDOWS case DSM_IMPL_WINDOWS: - return dsm_impl_windows(op, handle, request_size, impl_private, - mapped_address, mapped_size, elevel); + dsm_impl = &dsm_impl_windows_ops; + break; #endif #ifdef USE_DSM_MMAP case DSM_IMPL_MMAP: - return dsm_impl_mmap(op, handle, request_size, impl_private, - mapped_address, mapped_size, elevel); + dsm_impl = &dsm_impl_mmap_ops; + break; #endif default: elog(ERROR, "unexpected dynamic shared memory type: %d", - dynamic_shared_memory_type); - return false; + new_dynamic_shared_memory_type); } } -#ifdef USE_DSM_POSIX /* - * Operating system primitives to support POSIX shared memory. - * - * POSIX shared memory segments are created and attached using shm_open() - * and shm_unlink(); other operations, such as sizing or mapping the - * segment, are performed as if the shared memory segments were files. - * - * Indeed, on some platforms, they may be implemented that way. While - * POSIX shared memory segments seem intended to exist in a flat namespace, - * some operating systems may implement them as files, even going so far - * to treat a request for /xyz as a request to create a file by that name - * in the root directory. Users of such broken platforms should select - * a different shared memory implementation. - */ -static bool -dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size, - dsm_impl_private *impl_private, void **mapped_address, Size *mapped_size, - int elevel) -{ - char name[64]; - int flags; - int fd; - char *address; - - snprintf(name, 64, "/PostgreSQL.%u", handle); - - /* Handle teardown cases. */ - if (op == DSM_OP_DETACH || op == DSM_OP_DESTROY) - { - if (*mapped_address != NULL - && munmap(*mapped_address, *mapped_size) != 0) - { - ereport(elevel, - (errcode_for_dynamic_shared_memory(), - errmsg("could not unmap shared memory segment \"%s\": %m", - name))); - return false; - } - *mapped_address = NULL; - *mapped_size = 0; - if (op == DSM_OP_DESTROY && shm_unlink(name) != 0) - { - ereport(elevel, - (errcode_for_dynamic_shared_memory(), - errmsg("could not remove shared memory segment \"%s\": %m", - name))); - return false; - } - return true; - } - - /* - * Create new segment or open an existing one for attach. - * - * Even though we will close the FD before returning, it seems desirable - * to use Reserve/ReleaseExternalFD, to reduce the probability of EMFILE - * failure. The fact that we won't hold the FD open long justifies using - * ReserveExternalFD rather than AcquireExternalFD, though. - */ - ReserveExternalFD(); - - flags = O_RDWR | (op == DSM_OP_CREATE ? O_CREAT | O_EXCL : 0); - if ((fd = shm_open(name, flags, PG_FILE_MODE_OWNER)) == -1) - { - ReleaseExternalFD(); - if (op == DSM_OP_ATTACH || errno != EEXIST) - ereport(elevel, - (errcode_for_dynamic_shared_memory(), - errmsg("could not open shared memory segment \"%s\": %m", - name))); - return false; - } - - /* - * If we're attaching the segment, determine the current size; if we are - * creating the segment, set the size to the requested value. - */ - if (op == DSM_OP_ATTACH) - { - struct stat st; - - if (fstat(fd, &st) != 0) - { - int save_errno; - - /* Back out what's already been done. */ - save_errno = errno; - close(fd); - ReleaseExternalFD(); - errno = save_errno; - - ereport(elevel, - (errcode_for_dynamic_shared_memory(), - errmsg("could not stat shared memory segment \"%s\": %m", - name))); - return false; - } - request_size = st.st_size; - } - else if (dsm_impl_posix_resize(fd, request_size) != 0) - { - int save_errno; - - /* Back out what's already been done. */ - save_errno = errno; - close(fd); - ReleaseExternalFD(); - shm_unlink(name); - errno = save_errno; - - ereport(elevel, - (errcode_for_dynamic_shared_memory(), - errmsg("could not resize shared memory segment \"%s\" to %zu bytes: %m", - name, request_size))); - return false; - } - - /* Map it. */ - address = mmap(NULL, request_size, PROT_READ | PROT_WRITE, - MAP_SHARED | MAP_HASSEMAPHORE | MAP_NOSYNC, fd, 0); - if (address == MAP_FAILED) - { - int save_errno; - - /* Back out what's already been done. */ - save_errno = errno; - close(fd); - ReleaseExternalFD(); - if (op == DSM_OP_CREATE) - shm_unlink(name); - errno = save_errno; - - ereport(elevel, - (errcode_for_dynamic_shared_memory(), - errmsg("could not map shared memory segment \"%s\": %m", - name))); - return false; - } - *mapped_address = address; - *mapped_size = request_size; - close(fd); - ReleaseExternalFD(); - - return true; -} - -/* - * Set the size of a virtual memory region associated with a file descriptor. - * If necessary, also ensure that virtual memory is actually allocated by the - * operating system, to avoid nasty surprises later. - * - * Returns non-zero if either truncation or allocation fails, and sets errno. - */ -static int -dsm_impl_posix_resize(int fd, off_t size) -{ - int rc; - int save_errno; - sigset_t save_sigmask; - - /* - * Block all blockable signals, except SIGQUIT. posix_fallocate() can run - * for quite a long time, and is an all-or-nothing operation. If we - * allowed SIGUSR1 to interrupt us repeatedly (for example, due to - * recovery conflicts), the retry loop might never succeed. - */ - if (IsUnderPostmaster) - sigprocmask(SIG_SETMASK, &BlockSig, &save_sigmask); - - pgstat_report_wait_start(WAIT_EVENT_DSM_ALLOCATE); -#if defined(HAVE_POSIX_FALLOCATE) && defined(__linux__) - - /* - * On Linux, a shm_open fd is backed by a tmpfs file. If we were to use - * ftruncate, the file would contain a hole. Accessing memory backed by a - * hole causes tmpfs to allocate pages, which fails with SIGBUS if there - * is no more tmpfs space available. So we ask tmpfs to allocate pages - * here, so we can fail gracefully with ENOSPC now rather than risking - * SIGBUS later. - * - * We still use a traditional EINTR retry loop to handle SIGCONT. - * posix_fallocate() doesn't restart automatically, and we don't want this - * to fail if you attach a debugger. - */ - do - { - rc = posix_fallocate(fd, 0, size); - } while (rc == EINTR); - - /* - * The caller expects errno to be set, but posix_fallocate() doesn't set - * it. Instead it returns error numbers directly. So set errno, even - * though we'll also return rc to indicate success or failure. - */ - errno = rc; -#else - /* Extend the file to the requested size. */ - do - { - rc = ftruncate(fd, size); - } while (rc < 0 && errno == EINTR); -#endif - pgstat_report_wait_end(); - - if (IsUnderPostmaster) - { - save_errno = errno; - sigprocmask(SIG_SETMASK, &save_sigmask, NULL); - errno = save_errno; - } - - return rc; -} - -#endif /* USE_DSM_POSIX */ - -#ifdef USE_DSM_SYSV -/* - * Operating system primitives to support System V shared memory. - * - * System V shared memory segments are manipulated using shmget(), shmat(), - * shmdt(), and shmctl(). As the default allocation limits for System V - * shared memory are usually quite low, the POSIX facilities may be - * preferable; but those are not supported everywhere. - */ -static bool -dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size, - dsm_impl_private *impl_private, void **mapped_address, Size *mapped_size, - int elevel) -{ - key_t key; - int ident; - char *address; - char name[64]; - int *ident_cache; - - /* - * POSIX shared memory and mmap-based shared memory identify segments with - * names. To avoid needless error message variation, we use the handle as - * the name. - */ - snprintf(name, 64, "%u", handle); - - /* - * The System V shared memory namespace is very restricted; names are of - * type key_t, which is expected to be some sort of integer data type, but - * not necessarily the same one as dsm_handle. Since we use dsm_handle to - * identify shared memory segments across processes, this might seem like - * a problem, but it's really not. If dsm_handle is bigger than key_t, - * the cast below might truncate away some bits from the handle the - * user-provided, but it'll truncate exactly the same bits away in exactly - * the same fashion every time we use that handle, which is all that - * really matters. Conversely, if dsm_handle is smaller than key_t, we - * won't use the full range of available key space, but that's no big deal - * either. - * - * We do make sure that the key isn't negative, because that might not be - * portable. - */ - key = (key_t) handle; - if (key < 1) /* avoid compiler warning if type is unsigned */ - key = -key; - - /* - * There's one special key, IPC_PRIVATE, which can't be used. If we end - * up with that value by chance during a create operation, just pretend it - * already exists, so that caller will retry. If we run into it anywhere - * else, the caller has passed a handle that doesn't correspond to - * anything we ever created, which should not happen. - */ - if (key == IPC_PRIVATE) - { - if (op != DSM_OP_CREATE) - elog(DEBUG4, "System V shared memory key may not be IPC_PRIVATE"); - errno = EEXIST; - return false; - } - - /* - * Before we can do anything with a shared memory segment, we have to map - * the shared memory key to a shared memory identifier using shmget(). To - * avoid repeated lookups, we store the key using impl_private. - */ - if (*impl_private != 0) - { - ident_cache = (int *) *impl_private; - ident = *ident_cache; - } - else - { - int flags = IPCProtection; - size_t segsize; - - /* - * Allocate the memory BEFORE acquiring the resource, so that we don't - * leak the resource if memory allocation fails. - */ - ident_cache = MemoryContextAlloc(TopMemoryContext, sizeof(int)); - - /* - * When using shmget to find an existing segment, we must pass the - * size as 0. Passing a non-zero size which is greater than the - * actual size will result in EINVAL. - */ - segsize = 0; - - if (op == DSM_OP_CREATE) - { - flags |= IPC_CREAT | IPC_EXCL; - segsize = request_size; - } - - if ((ident = shmget(key, segsize, flags)) == -1) - { - if (op == DSM_OP_ATTACH || errno != EEXIST) - { - int save_errno = errno; - - pfree(ident_cache); - errno = save_errno; - ereport(elevel, - (errcode_for_dynamic_shared_memory(), - errmsg("could not get shared memory segment: %m"))); - } - return false; - } - - *ident_cache = ident; - *impl_private = (uintptr_t) ident_cache; - } - - /* Handle teardown cases. */ - if (op == DSM_OP_DETACH || op == DSM_OP_DESTROY) - { - pfree(ident_cache); - *impl_private = 0; - if (*mapped_address != NULL && shmdt(*mapped_address) != 0) - { - ereport(elevel, - (errcode_for_dynamic_shared_memory(), - errmsg("could not unmap shared memory segment \"%s\": %m", - name))); - return false; - } - *mapped_address = NULL; - *mapped_size = 0; - if (op == DSM_OP_DESTROY && shmctl(ident, IPC_RMID, NULL) < 0) - { - ereport(elevel, - (errcode_for_dynamic_shared_memory(), - errmsg("could not remove shared memory segment \"%s\": %m", - name))); - return false; - } - return true; - } - - /* If we're attaching it, we must use IPC_STAT to determine the size. */ - if (op == DSM_OP_ATTACH) - { - struct shmid_ds shm; - - if (shmctl(ident, IPC_STAT, &shm) != 0) - { - ereport(elevel, - (errcode_for_dynamic_shared_memory(), - errmsg("could not stat shared memory segment \"%s\": %m", - name))); - return false; - } - request_size = shm.shm_segsz; - } - - /* Map it. */ - address = shmat(ident, NULL, PG_SHMAT_FLAGS); - if (address == (void *) -1) - { - int save_errno; - - /* Back out what's already been done. */ - save_errno = errno; - if (op == DSM_OP_CREATE) - shmctl(ident, IPC_RMID, NULL); - errno = save_errno; - - ereport(elevel, - (errcode_for_dynamic_shared_memory(), - errmsg("could not map shared memory segment \"%s\": %m", - name))); - return false; - } - *mapped_address = address; - *mapped_size = request_size; - - return true; -} -#endif - -#ifdef USE_DSM_WINDOWS -/* - * Operating system primitives to support Windows shared memory. - * - * Windows shared memory implementation is done using file mapping - * which can be backed by either physical file or system paging file. - * Current implementation uses system paging file as other effects - * like performance are not clear for physical file and it is used in similar - * way for main shared memory in windows. - * - * A memory mapping object is a kernel object - they always get deleted when - * the last reference to them goes away, either explicitly via a CloseHandle or - * when the process containing the reference exits. - */ -static bool -dsm_impl_windows(dsm_op op, dsm_handle handle, Size request_size, - dsm_impl_private *impl_private, void **mapped_address, - Size *mapped_size, int elevel) -{ - char *address; - HANDLE hmap; - char name[64]; - MEMORY_BASIC_INFORMATION info; - - /* - * Storing the shared memory segment in the Global\ namespace, can allow - * any process running in any session to access that file mapping object - * provided that the caller has the required access rights. But to avoid - * issues faced in main shared memory, we are using the naming convention - * similar to main shared memory. We can change here once issue mentioned - * in GetSharedMemName is resolved. - */ - snprintf(name, 64, "%s.%u", SEGMENT_NAME_PREFIX, handle); - - /* - * Handle teardown cases. Since Windows automatically destroys the object - * when no references remain, we can treat it the same as detach. - */ - if (op == DSM_OP_DETACH || op == DSM_OP_DESTROY) - { - if (*mapped_address != NULL - && UnmapViewOfFile(*mapped_address) == 0) - { - _dosmaperr(GetLastError()); - ereport(elevel, - (errcode_for_dynamic_shared_memory(), - errmsg("could not unmap shared memory segment \"%s\": %m", - name))); - return false; - } - if (*impl_private != NULL - && CloseHandle(*impl_private) == 0) - { - _dosmaperr(GetLastError()); - ereport(elevel, - (errcode_for_dynamic_shared_memory(), - errmsg("could not remove shared memory segment \"%s\": %m", - name))); - return false; - } - - *impl_private = NULL; - *mapped_address = NULL; - *mapped_size = 0; - return true; - } - - /* Create new segment or open an existing one for attach. */ - if (op == DSM_OP_CREATE) - { - DWORD size_high; - DWORD size_low; - DWORD errcode; - - /* Shifts >= the width of the type are undefined. */ -#ifdef _WIN64 - size_high = request_size >> 32; -#else - size_high = 0; -#endif - size_low = (DWORD) request_size; - - /* CreateFileMapping might not clear the error code on success */ - SetLastError(0); - - hmap = CreateFileMapping(INVALID_HANDLE_VALUE, /* Use the pagefile */ - NULL, /* Default security attrs */ - PAGE_READWRITE, /* Memory is read/write */ - size_high, /* Upper 32 bits of size */ - size_low, /* Lower 32 bits of size */ - name); - - errcode = GetLastError(); - if (errcode == ERROR_ALREADY_EXISTS || errcode == ERROR_ACCESS_DENIED) - { - /* - * On Windows, when the segment already exists, a handle for the - * existing segment is returned. We must close it before - * returning. However, if the existing segment is created by a - * service, then it returns ERROR_ACCESS_DENIED. We don't do - * _dosmaperr here, so errno won't be modified. - */ - if (hmap) - CloseHandle(hmap); - return false; - } - - if (!hmap) - { - _dosmaperr(errcode); - ereport(elevel, - (errcode_for_dynamic_shared_memory(), - errmsg("could not create shared memory segment \"%s\": %m", - name))); - return false; - } - } - else - { - hmap = OpenFileMapping(FILE_MAP_WRITE | FILE_MAP_READ, - FALSE, /* do not inherit the name */ - name); /* name of mapping object */ - if (!hmap) - { - _dosmaperr(GetLastError()); - ereport(elevel, - (errcode_for_dynamic_shared_memory(), - errmsg("could not open shared memory segment \"%s\": %m", - name))); - return false; - } - } - - /* Map it. */ - address = MapViewOfFile(hmap, FILE_MAP_WRITE | FILE_MAP_READ, - 0, 0, 0); - if (!address) - { - int save_errno; - - _dosmaperr(GetLastError()); - /* Back out what's already been done. */ - save_errno = errno; - CloseHandle(hmap); - errno = save_errno; - - ereport(elevel, - (errcode_for_dynamic_shared_memory(), - errmsg("could not map shared memory segment \"%s\": %m", - name))); - return false; - } - - /* - * VirtualQuery gives size in page_size units, which is 4K for Windows. We - * need size only when we are attaching, but it's better to get the size - * when creating new segment to keep size consistent both for - * DSM_OP_CREATE and DSM_OP_ATTACH. - */ - if (VirtualQuery(address, &info, sizeof(info)) == 0) - { - int save_errno; - - _dosmaperr(GetLastError()); - /* Back out what's already been done. */ - save_errno = errno; - UnmapViewOfFile(address); - CloseHandle(hmap); - errno = save_errno; - - ereport(elevel, - (errcode_for_dynamic_shared_memory(), - errmsg("could not stat shared memory segment \"%s\": %m", - name))); - return false; - } - - *mapped_address = address; - *mapped_size = info.RegionSize; - *impl_private = hmap; - - return true; -} -#endif - -#ifdef USE_DSM_MMAP -/* - * Operating system primitives to support mmap-based shared memory. - * - * Calling this "shared memory" is somewhat of a misnomer, because what - * we're really doing is creating a bunch of files and mapping them into - * our address space. The operating system may feel obliged to - * synchronize the contents to disk even if nothing is being paged out, - * which will not serve us well. The user can relocate the pg_dynshmem - * directory to a ramdisk to avoid this problem, if available. - */ -static bool -dsm_impl_mmap(dsm_op op, dsm_handle handle, Size request_size, - dsm_impl_private *impl_private, void **mapped_address, Size *mapped_size, - int elevel) -{ - char name[64]; - int flags; - int fd; - char *address; - - snprintf(name, 64, PG_DYNSHMEM_DIR "/" PG_DYNSHMEM_MMAP_FILE_PREFIX "%u", - handle); - - /* Handle teardown cases. */ - if (op == DSM_OP_DETACH || op == DSM_OP_DESTROY) - { - if (*mapped_address != NULL - && munmap(*mapped_address, *mapped_size) != 0) - { - ereport(elevel, - (errcode_for_dynamic_shared_memory(), - errmsg("could not unmap shared memory segment \"%s\": %m", - name))); - return false; - } - *mapped_address = NULL; - *mapped_size = 0; - if (op == DSM_OP_DESTROY && unlink(name) != 0) - { - ereport(elevel, - (errcode_for_dynamic_shared_memory(), - errmsg("could not remove shared memory segment \"%s\": %m", - name))); - return false; - } - return true; - } - - /* Create new segment or open an existing one for attach. */ - flags = O_RDWR | (op == DSM_OP_CREATE ? O_CREAT | O_EXCL : 0); - if ((fd = OpenTransientFile(name, flags)) == -1) - { - if (op == DSM_OP_ATTACH || errno != EEXIST) - ereport(elevel, - (errcode_for_dynamic_shared_memory(), - errmsg("could not open shared memory segment \"%s\": %m", - name))); - return false; - } - - /* - * If we're attaching the segment, determine the current size; if we are - * creating the segment, set the size to the requested value. - */ - if (op == DSM_OP_ATTACH) - { - struct stat st; - - if (fstat(fd, &st) != 0) - { - int save_errno; - - /* Back out what's already been done. */ - save_errno = errno; - CloseTransientFile(fd); - errno = save_errno; - - ereport(elevel, - (errcode_for_dynamic_shared_memory(), - errmsg("could not stat shared memory segment \"%s\": %m", - name))); - return false; - } - request_size = st.st_size; - } - else - { - /* - * Allocate a buffer full of zeros. - * - * Note: palloc zbuffer, instead of just using a local char array, to - * ensure it is reasonably well-aligned; this may save a few cycles - * transferring data to the kernel. - */ - char *zbuffer = (char *) palloc0(ZBUFFER_SIZE); - Size remaining = request_size; - bool success = true; - - /* - * Zero-fill the file. We have to do this the hard way to ensure that - * all the file space has really been allocated, so that we don't - * later seg fault when accessing the memory mapping. This is pretty - * pessimal. - */ - while (success && remaining > 0) - { - Size goal = remaining; - - if (goal > ZBUFFER_SIZE) - goal = ZBUFFER_SIZE; - pgstat_report_wait_start(WAIT_EVENT_DSM_FILL_ZERO_WRITE); - if (write(fd, zbuffer, goal) == goal) - remaining -= goal; - else - success = false; - pgstat_report_wait_end(); - } - - if (!success) - { - int save_errno; - - /* Back out what's already been done. */ - save_errno = errno; - CloseTransientFile(fd); - unlink(name); - errno = save_errno ? save_errno : ENOSPC; - - ereport(elevel, - (errcode_for_dynamic_shared_memory(), - errmsg("could not resize shared memory segment \"%s\" to %zu bytes: %m", - name, request_size))); - return false; - } - } - - /* Map it. */ - address = mmap(NULL, request_size, PROT_READ | PROT_WRITE, - MAP_SHARED | MAP_HASSEMAPHORE | MAP_NOSYNC, fd, 0); - if (address == MAP_FAILED) - { - int save_errno; - - /* Back out what's already been done. */ - save_errno = errno; - CloseTransientFile(fd); - if (op == DSM_OP_CREATE) - unlink(name); - errno = save_errno; - - ereport(elevel, - (errcode_for_dynamic_shared_memory(), - errmsg("could not map shared memory segment \"%s\": %m", - name))); - return false; - } - *mapped_address = address; - *mapped_size = request_size; - - if (CloseTransientFile(fd) != 0) - { - ereport(elevel, - (errcode_for_file_access(), - errmsg("could not close shared memory segment \"%s\": %m", - name))); - return false; - } - - return true; -} -#endif - -/* - * Implementation-specific actions that must be performed when a segment is to - * be preserved even when no backend has it attached. - * - * Except on Windows, we don't need to do anything at all. But since Windows - * cleans up segments automatically when no references remain, we duplicate - * the segment handle into the postmaster process. The postmaster needn't - * do anything to receive the handle; Windows transfers it automatically. + * No-op implementation of the pin_segment interface. Most implementations + * (all but Windows) don't need to do anything here. */ void -dsm_impl_pin_segment(dsm_handle handle, dsm_impl_private impl_private, - dsm_impl_private_pm_handle *pm_handle) +dsm_impl_noop_pin_segment(dsm_impl_handle handle, dsm_impl_private impl_private, + dsm_impl_private_pm_handle *pm_handle) { - switch (dynamic_shared_memory_type) - { -#ifdef USE_DSM_WINDOWS - case DSM_IMPL_WINDOWS: - if (IsUnderPostmaster) - { - HANDLE hmap; - - if (!DuplicateHandle(GetCurrentProcess(), impl_private, - PostmasterHandle, &hmap, 0, FALSE, - DUPLICATE_SAME_ACCESS)) - { - char name[64]; - - snprintf(name, 64, "%s.%u", SEGMENT_NAME_PREFIX, handle); - _dosmaperr(GetLastError()); - ereport(ERROR, - (errcode_for_dynamic_shared_memory(), - errmsg("could not duplicate handle for \"%s\": %m", - name))); - } - - /* - * Here, we remember the handle that we created in the - * postmaster process. This handle isn't actually usable in - * any process other than the postmaster, but that doesn't - * matter. We're just holding onto it so that, if the segment - * is unpinned, dsm_impl_unpin_segment can close it. - */ - *pm_handle = hmap; - } - break; -#endif - default: - break; - } } -/* - * Implementation-specific actions that must be performed when a segment is no - * longer to be preserved, so that it will be cleaned up when all backends - * have detached from it. - * - * Except on Windows, we don't need to do anything at all. For Windows, we - * close the extra handle that dsm_impl_pin_segment created in the - * postmaster's process space. - */ +/* no-op implementation of the unpin_segment interface */ void -dsm_impl_unpin_segment(dsm_handle handle, dsm_impl_private_pm_handle *pm_handle) -{ - switch (dynamic_shared_memory_type) - { -#ifdef USE_DSM_WINDOWS - case DSM_IMPL_WINDOWS: - if (IsUnderPostmaster) - { - if (*impl_private && - !DuplicateHandle(PostmasterHandle, *impl_private, - NULL, NULL, 0, FALSE, - DUPLICATE_CLOSE_SOURCE)) - { - char name[64]; - - snprintf(name, 64, "%s.%u", SEGMENT_NAME_PREFIX, handle); - _dosmaperr(GetLastError()); - ereport(ERROR, - (errcode_for_dynamic_shared_memory(), - errmsg("could not duplicate handle for \"%s\": %m", - name))); - } - - *pm_handle = 0; - } - break; -#endif - default: - break; - } -} - -static int -errcode_for_dynamic_shared_memory(void) +dsm_impl_noop_unpin_segment(dsm_impl_handle handle, dsm_impl_private_pm_handle pm_handle) { - if (errno == EFBIG || errno == ENOMEM) - return errcode(ERRCODE_OUT_OF_MEMORY); - else - return errcode_for_file_access(); } diff --git a/src/backend/storage/ipc/dsm_impl_mmap.c b/src/backend/storage/ipc/dsm_impl_mmap.c new file mode 100644 index 00000000000..8dd7d1eeb7f --- /dev/null +++ b/src/backend/storage/ipc/dsm_impl_mmap.c @@ -0,0 +1,300 @@ +/*------------------------------------------------------------------------- + * + * dsm_impl_mmap.c + * Operating system primitives to support mmap-based shared memory. + * + * Calling this "shared memory" is somewhat of a misnomer, because what + * we're really doing is creating a bunch of files and mapping them into + * our address space. The operating system may feel obliged to + * synchronize the contents to disk even if nothing is being paged out, + * which will not serve us well. The user can relocate the pg_dynshmem + * directory to a ramdisk to avoid this problem, if available. + * + * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * + * IDENTIFICATION + * src/backend/storage/ipc/dsm_impl_mmap.c + * + *------------------------------------------------------------------------- + */ +#include "postgres.h" + +#include <fcntl.h> +#include <unistd.h> +#ifndef WIN32 +#include <sys/mman.h> +#include <sys/stat.h> +#endif + +#include "common/file_perm.h" +#include "common/pg_prng.h" +#include "miscadmin.h" +#include "pgstat.h" +#include "portability/mem.h" +#include "storage/dsm_impl.h" +#include "storage/fd.h" + +#ifdef USE_DSM_MMAP + +/* Size of buffer to be used for zero-filling. */ +#define ZBUFFER_SIZE 8192 + +static dsm_impl_handle +dsm_impl_mmap_create(Size request_size, dsm_impl_private *impl_private, + void **mapped_address, int elevel) +{ + char name[64]; + int flags; + int fd; + char *address; + dsm_impl_handle handle; + + /* + * Create new segment, with a random name. If the name is already in use, + * retry until we find an unused name. + */ + for (;;) + { + do + { + handle = pg_prng_uint32(&pg_global_prng_state); + } while (handle == DSM_IMPL_HANDLE_INVALID); + + snprintf(name, 64, PG_DYNSHMEM_DIR "/" PG_DYNSHMEM_MMAP_FILE_PREFIX "%u", + handle); + + /* Create new segment or open an existing one for attach. */ + flags = O_RDWR | O_CREAT | O_EXCL; + if ((fd = OpenTransientFile(name, flags)) == -1) + { + if (errno == EEXIST) + continue; + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not open shared memory segment \"%s\": %m", + name))); + return DSM_IMPL_HANDLE_INVALID; + } + break; + } + + /* Enlarge it to the requested size. */ + { + /* + * Allocate a buffer full of zeros. + * + * Note: palloc zbuffer, instead of just using a local char array, to + * ensure it is reasonably well-aligned; this may save a few cycles + * transferring data to the kernel. + */ + char *zbuffer = (char *) palloc0(ZBUFFER_SIZE); + Size remaining = request_size; + bool success = true; + + /* + * Zero-fill the file. We have to do this the hard way to ensure that + * all the file space has really been allocated, so that we don't + * later seg fault when accessing the memory mapping. This is pretty + * pessimal. + */ + while (success && remaining > 0) + { + Size goal = remaining; + + if (goal > ZBUFFER_SIZE) + goal = ZBUFFER_SIZE; + pgstat_report_wait_start(WAIT_EVENT_DSM_FILL_ZERO_WRITE); + if (write(fd, zbuffer, goal) == goal) + remaining -= goal; + else + success = false; + pgstat_report_wait_end(); + } + + if (!success) + { + int save_errno; + + /* Back out what's already been done. */ + save_errno = errno; + CloseTransientFile(fd); + unlink(name); + errno = save_errno ? save_errno : ENOSPC; + + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not resize shared memory segment \"%s\" to %zu bytes: %m", + name, request_size))); + return DSM_IMPL_HANDLE_INVALID; + } + } + + /* Map it to this process's address space. */ + address = mmap(NULL, request_size, PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_HASSEMAPHORE | MAP_NOSYNC, fd, 0); + if (address == MAP_FAILED) + { + int save_errno; + + /* Back out what's already been done. */ + save_errno = errno; + CloseTransientFile(fd); + unlink(name); + errno = save_errno; + + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not map shared memory segment \"%s\": %m", + name))); + return DSM_IMPL_HANDLE_INVALID; + } + + /* Once it's mapped, we don't need the file descriptor for it anymore. */ + if (CloseTransientFile(fd) != 0) + { + ereport(elevel, + (errcode_for_file_access(), + errmsg("could not close shared memory segment \"%s\": %m", + name))); + return DSM_IMPL_HANDLE_INVALID; + } + + *mapped_address = address; + *impl_private = 0; /* not used by this implementation */ + return handle; +} + +static bool +dsm_impl_mmap_attach(dsm_impl_handle handle, + dsm_impl_private *impl_private, void **mapped_address, Size *mapped_size, + int elevel) +{ + char name[64]; + int flags; + int fd; + char *address; + Size size; + struct stat st; + + snprintf(name, 64, PG_DYNSHMEM_DIR "/" PG_DYNSHMEM_MMAP_FILE_PREFIX "%u", + handle); + + /* Open an existing file for attach. */ + flags = O_RDWR; + if ((fd = OpenTransientFile(name, flags)) == -1) + { + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not open shared memory segment \"%s\": %m", + name))); + return false; + } + + /* Determine the current size. */ + if (fstat(fd, &st) != 0) + { + int save_errno; + + /* Back out what's already been done. */ + save_errno = errno; + CloseTransientFile(fd); + errno = save_errno; + + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not stat shared memory segment \"%s\": %m", + name))); + return false; + } + size = st.st_size; + + /* Map it to this process's address space. */ + address = mmap(NULL, size, PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_HASSEMAPHORE | MAP_NOSYNC, fd, 0); + if (address == MAP_FAILED) + { + int save_errno; + + /* Back out what's already been done. */ + save_errno = errno; + CloseTransientFile(fd); + errno = save_errno; + + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not map shared memory segment \"%s\": %m", + name))); + return false; + } + + /* Once it's mapped, we don't need the file descriptor for it anymore */ + if (CloseTransientFile(fd) != 0) + { + ereport(elevel, + (errcode_for_file_access(), + errmsg("could not close shared memory segment \"%s\": %m", + name))); + return false; + } + + *mapped_address = address; + *mapped_size = size; + *impl_private = 0; /* not used by this implementation */ + return true; +} + +static bool +dsm_impl_mmap_detach(dsm_impl_handle handle, + dsm_impl_private impl_private, void *mapped_address, Size mapped_size, + int elevel) +{ + char name[64]; + + Assert(mapped_address != NULL); + Assert(impl_private == 0); /* not used by this implementation */ + + snprintf(name, 64, PG_DYNSHMEM_DIR "/" PG_DYNSHMEM_MMAP_FILE_PREFIX "%u", + handle); + + if (munmap(mapped_address, mapped_size) != 0) + { + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not unmap shared memory segment \"%s\": %m", + name))); + return false; + } + return true; +} + +static bool +dsm_impl_mmap_destroy(dsm_impl_handle handle, int elevel) +{ + char name[64]; + + snprintf(name, 64, PG_DYNSHMEM_DIR "/" PG_DYNSHMEM_MMAP_FILE_PREFIX "%u", + handle); + + if (unlink(name) != 0) + { + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not remove shared memory segment \"%s\": %m", + name))); + return false; + } + return true; +} + +const dsm_impl_ops dsm_impl_mmap_ops = { + .create = dsm_impl_mmap_create, + .attach = dsm_impl_mmap_attach, + .detach = dsm_impl_mmap_detach, + .destroy = dsm_impl_mmap_destroy, + .pin_segment = dsm_impl_noop_pin_segment, + .unpin_segment = dsm_impl_noop_unpin_segment, +}; + +#endif diff --git a/src/backend/storage/ipc/dsm_impl_posix.c b/src/backend/storage/ipc/dsm_impl_posix.c new file mode 100644 index 00000000000..f9c873cd961 --- /dev/null +++ b/src/backend/storage/ipc/dsm_impl_posix.c @@ -0,0 +1,341 @@ +/*------------------------------------------------------------------------- + * + * dsm_impl_posix.c + * Operating system primitives to support POSIX shared memory. + * + * POSIX shared memory segments are created and attached using shm_open() + * and shm_unlink(); other operations, such as sizing or mapping the + * segment, are performed as if the shared memory segments were files. + * + * Indeed, on some platforms, they may be implemented that way. While + * POSIX shared memory segments seem intended to exist in a flat namespace, + * some operating systems may implement them as files, even going so far + * to treat a request for /xyz as a request to create a file by that name + * in the root directory. Users of such broken platforms should select + * a different shared memory implementation. + * + * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * + * IDENTIFICATION + * src/backend/storage/ipc/dsm_impl_posix.c + * + *------------------------------------------------------------------------- + */ +#include "postgres.h" + +#include <fcntl.h> +#include <signal.h> +#include <unistd.h> +#ifndef WIN32 +#include <sys/mman.h> +#include <sys/stat.h> +#endif + +#include "common/file_perm.h" +#include "common/pg_prng.h" +#include "libpq/pqsignal.h" +#include "miscadmin.h" +#include "pgstat.h" +#include "portability/mem.h" +#include "storage/dsm_impl.h" +#include "storage/fd.h" +#include "utils/guc.h" +#include "utils/memutils.h" + +#ifdef USE_DSM_POSIX + +static int dsm_impl_posix_resize(int fd, off_t size); + +static dsm_impl_handle +dsm_impl_posix_create(Size request_size, dsm_impl_private *impl_private, + void **mapped_address, int elevel) +{ + char name[64]; + int flags; + int fd; + char *address; + dsm_impl_handle handle; + + /* + * Even though we will close the FD before returning, it seems desirable + * to use Reserve/ReleaseExternalFD, to reduce the probability of EMFILE + * failure. The fact that we won't hold the FD open long justifies using + * ReserveExternalFD rather than AcquireExternalFD, though. + */ + ReserveExternalFD(); + + /* + * Create new segment, with a random name. If the name is already in use, + * retry until we find an unused name. + */ + for (;;) + { + do + { + handle = pg_prng_uint32(&pg_global_prng_state); + } while (handle == DSM_IMPL_HANDLE_INVALID); + + snprintf(name, 64, "/PostgreSQL.%u", handle); + + flags = O_RDWR | O_CREAT | O_EXCL; + if ((fd = shm_open(name, flags, PG_FILE_MODE_OWNER)) == -1) + { + ReleaseExternalFD(); + if (errno == EEXIST) + continue; + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not open shared memory segment \"%s\": %m", + name))); + return DSM_IMPL_HANDLE_INVALID; + } + break; + } + + /* Enlarge it to the requested size. */ + if (dsm_impl_posix_resize(fd, request_size) != 0) + { + int save_errno; + + /* Back out what's already been done. */ + save_errno = errno; + close(fd); + ReleaseExternalFD(); + shm_unlink(name); + errno = save_errno; + + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not resize shared memory segment \"%s\" to %zu bytes: %m", + name, request_size))); + return DSM_IMPL_HANDLE_INVALID; + } + + /* Map it to this process's address space. */ + address = mmap(NULL, request_size, PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_HASSEMAPHORE | MAP_NOSYNC, fd, 0); + if (address == MAP_FAILED) + { + int save_errno; + + /* Back out what's already been done. */ + save_errno = errno; + close(fd); + ReleaseExternalFD(); + shm_unlink(name); + errno = save_errno; + + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not map shared memory segment \"%s\": %m", + name))); + return DSM_IMPL_HANDLE_INVALID; + } + + /* Once it's mapped, we don't need the file descriptor for it anymore. */ + close(fd); + ReleaseExternalFD(); + + *mapped_address = address; + *impl_private = 0; /* not used by this implementation */ + return handle; +} + +static bool +dsm_impl_posix_attach(dsm_impl_handle handle, dsm_impl_private *impl_private, + void **mapped_address, Size *mapped_size, + int elevel) +{ + char name[64]; + int flags; + int fd; + char *address; + Size size; + struct stat st; + + snprintf(name, 64, "/PostgreSQL.%u", handle); + + /* Like in dsm_impl_posix_attach, make sure an FD is available */ + ReserveExternalFD(); + + flags = O_RDWR; + if ((fd = shm_open(name, flags, PG_FILE_MODE_OWNER)) == -1) + { + ReleaseExternalFD(); + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not open shared memory segment \"%s\": %m", + name))); + return false; + } + + /* Determine the current size */ + if (fstat(fd, &st) != 0) + { + int save_errno; + + /* Back out what's already been done. */ + save_errno = errno; + close(fd); + ReleaseExternalFD(); + errno = save_errno; + + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not stat shared memory segment \"%s\": %m", + name))); + return false; + } + size = st.st_size; + + /* Map it to this process's address space. */ + address = mmap(NULL, size, PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_HASSEMAPHORE | MAP_NOSYNC, fd, 0); + if (address == MAP_FAILED) + { + int save_errno; + + /* Back out what's already been done. */ + save_errno = errno; + close(fd); + ReleaseExternalFD(); + errno = save_errno; + + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not map shared memory segment \"%s\": %m", + name))); + return false; + } + + /* Once it's mapped, we don't need the file descriptor for it anymore */ + close(fd); + ReleaseExternalFD(); + + *mapped_address = address; + *mapped_size = size; + *impl_private = 0; /* not used by this implementation */ + return true; +} + +static bool +dsm_impl_posix_detach(dsm_impl_handle handle, dsm_impl_private impl_private, + void *mapped_address, Size mapped_size, + int elevel) +{ + char name[64]; + + Assert(mapped_address != NULL); + Assert(impl_private == 0); /* not used by this implementation */ + + snprintf(name, 64, "/PostgreSQL.%u", handle); + + if (munmap(mapped_address, mapped_size) != 0) + { + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not unmap shared memory segment \"%s\": %m", + name))); + return false; + } + return true; +} + +static bool +dsm_impl_posix_destroy(dsm_impl_handle handle, int elevel) +{ + char name[64]; + + snprintf(name, 64, "/PostgreSQL.%u", handle); + + if (shm_unlink(name) != 0) + { + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not remove shared memory segment \"%s\": %m", + name))); + return false; + } + return true; +} + +/* + * Set the size of a virtual memory region associated with a file descriptor. + * If necessary, also ensure that virtual memory is actually allocated by the + * operating system, to avoid nasty surprises later. + * + * Returns non-zero if either truncation or allocation fails, and sets errno. + */ +static int +dsm_impl_posix_resize(int fd, off_t size) +{ + int rc; + int save_errno; + sigset_t save_sigmask; + + /* + * Block all blockable signals, except SIGQUIT. posix_fallocate() can run + * for quite a long time, and is an all-or-nothing operation. If we + * allowed SIGUSR1 to interrupt us repeatedly (for example, due to + * recovery conflicts), the retry loop might never succeed. + */ + if (IsUnderPostmaster) + sigprocmask(SIG_SETMASK, &BlockSig, &save_sigmask); + + pgstat_report_wait_start(WAIT_EVENT_DSM_ALLOCATE); +#if defined(HAVE_POSIX_FALLOCATE) && defined(__linux__) + + /* + * On Linux, a shm_open fd is backed by a tmpfs file. If we were to use + * ftruncate, the file would contain a hole. Accessing memory backed by a + * hole causes tmpfs to allocate pages, which fails with SIGBUS if there + * is no more tmpfs space available. So we ask tmpfs to allocate pages + * here, so we can fail gracefully with ENOSPC now rather than risking + * SIGBUS later. + * + * We still use a traditional EINTR retry loop to handle SIGCONT. + * posix_fallocate() doesn't restart automatically, and we don't want this + * to fail if you attach a debugger. + */ + do + { + rc = posix_fallocate(fd, 0, size); + } while (rc == EINTR); + + /* + * The caller expects errno to be set, but posix_fallocate() doesn't set + * it. Instead it returns error numbers directly. So set errno, even + * though we'll also return rc to indicate success or failure. + */ + errno = rc; +#else + /* Extend the file to the requested size. */ + do + { + rc = ftruncate(fd, size); + } while (rc < 0 && errno == EINTR); +#endif + pgstat_report_wait_end(); + + if (IsUnderPostmaster) + { + save_errno = errno; + sigprocmask(SIG_SETMASK, &save_sigmask, NULL); + errno = save_errno; + } + + return rc; +} + +const dsm_impl_ops dsm_impl_posix_ops = { + .create = dsm_impl_posix_create, + .attach = dsm_impl_posix_attach, + .detach = dsm_impl_posix_detach, + .destroy = dsm_impl_posix_destroy, + .pin_segment = dsm_impl_noop_pin_segment, + .unpin_segment = dsm_impl_noop_unpin_segment, +}; + +#endif /* USE_DSM_POSIX */ diff --git a/src/backend/storage/ipc/dsm_impl_sysv.c b/src/backend/storage/ipc/dsm_impl_sysv.c new file mode 100644 index 00000000000..cde742d4854 --- /dev/null +++ b/src/backend/storage/ipc/dsm_impl_sysv.c @@ -0,0 +1,224 @@ +/*------------------------------------------------------------------------- + * + * dsm_impl_sysv.c + * Operating system primitives to support System V shared memory. + * + * System V shared memory segments are manipulated using shmget(), shmat(), + * shmdt(), and shmctl(). As the default allocation limits for System V + * shared memory are usually quite low, the POSIX facilities may be + * preferable; but those are not supported everywhere. + * + * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * + * IDENTIFICATION + * src/backend/storage/ipc/dsm_impl_sysv.c + * + *------------------------------------------------------------------------- + */ + +#include "postgres.h" + +#include <fcntl.h> +#include <signal.h> +#include <unistd.h> +#include <sys/mman.h> +#include <sys/ipc.h> +#include <sys/shm.h> +#include <sys/stat.h> + +#include "common/file_perm.h" +#include "libpq/pqsignal.h" +#include "miscadmin.h" +#include "pgstat.h" +#include "portability/mem.h" +#include "postmaster/postmaster.h" +#include "storage/dsm_impl.h" +#include "storage/fd.h" +#include "utils/guc.h" +#include "utils/memutils.h" + +#ifdef USE_DSM_SYSV + +/* + * Return the segment's shmid as a handle. + * + * Because 0 is a valid shmid, but not a valid dsm_impl_handle, add one to + * avoid it. The valid range for shmid is from 0 to INT_MAX, so it fits in + * uint32 this way. + */ +static inline dsm_impl_handle +handle_from_shmid(int shmid) +{ + return ((uint32) shmid) + 1; +} + +static inline int +shmid_from_handle(dsm_impl_handle handle) +{ + return handle - 1; +} + +static dsm_impl_handle +dsm_impl_sysv_create(Size request_size, + dsm_impl_private *impl_private, void **mapped_address, + int elevel) +{ + int ident; + char *address; + char name[64]; + + /* + * Create a new shared memory segment. We let shmget() pick a unique + * identifier for us. + */ + if ((ident = shmget(IPC_PRIVATE, request_size, IPCProtection | IPC_CREAT | IPC_EXCL)) == -1) + { + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not create shared memory segment: %m"))); + return DSM_IMPL_HANDLE_INVALID; + } + + /* + * POSIX shared memory and mmap-based shared memory identify segments with + * names. To avoid needless error message variation, we use the shmid as + * the name. + */ + snprintf(name, 64, "shmid %d", ident); + + /* Map it to this process's address space. */ + address = shmat(ident, NULL, PG_SHMAT_FLAGS); + if (address == (void *) -1) + { + int save_errno; + + /* Back out what's already been done. */ + save_errno = errno; + shmctl(ident, IPC_RMID, NULL); + errno = save_errno; + + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not map shared memory segment \"%s\": %m", + name))); + return DSM_IMPL_HANDLE_INVALID; + } + *mapped_address = address; + *impl_private = 0; /* not used by this implementation */ + + return handle_from_shmid(ident); +} + +static bool +dsm_impl_sysv_attach(dsm_impl_handle handle, + dsm_impl_private *impl_private, void **mapped_address, Size *mapped_size, + int elevel) +{ + int ident = shmid_from_handle(handle); + char *address; + char name[64]; + struct shmid_ds shm; + Size size; + + /* + * POSIX shared memory and mmap-based shared memory identify segments with + * names. To avoid needless error message variation, we use the shmid as + * the name. + */ + snprintf(name, 64, "shmid %d", ident); + + /* Use IPC_STAT to determine the size. */ + if (shmctl(ident, IPC_STAT, &shm) != 0) + { + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not stat shared memory segment \"%s\": %m", + name))); + return false; + } + size = shm.shm_segsz; + + /* Map it to this process's address space. */ + address = shmat(ident, NULL, PG_SHMAT_FLAGS); + if (address == (void *) -1) + { + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not map shared memory segment \"%s\": %m", + name))); + return false; + } + *mapped_address = address; + *mapped_size = size; + *impl_private = 0; /* not used by this implementation */ + + return true; +} + + +static bool +dsm_impl_sysv_detach(dsm_impl_handle handle, + dsm_impl_private impl_private, void *mapped_address, Size mapped_size, + int elevel) +{ + int ident = shmid_from_handle(handle); + char name[64]; + + Assert(mapped_address != NULL); + Assert(impl_private == 0); /* not used by this implementation */ + + /* + * POSIX shared memory and mmap-based shared memory identify segments with + * names. To avoid needless error message variation, we use the shmid as + * the name. + */ + snprintf(name, 64, "shmid %d", ident); + + if (shmdt(mapped_address) != 0) + { + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not unmap shared memory segment \"%s\": %m", + name))); + return false; + } + return true; +} + +static bool +dsm_impl_sysv_destroy(dsm_impl_handle handle, int elevel) +{ + int ident = shmid_from_handle(handle); + char name[64]; + + /* + * POSIX shared memory and mmap-based shared memory identify segments with + * names. To avoid needless error message variation, we use the shmid as + * the name. + */ + snprintf(name, 64, "shmid %d", ident); + + /* Handle teardown cases. */ + if (shmctl(ident, IPC_RMID, NULL) < 0) + { + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not remove shared memory segment \"%s\": %m", + name))); + return false; + } + return true; +} + +const dsm_impl_ops dsm_impl_sysv_ops = { + .create = dsm_impl_sysv_create, + .attach = dsm_impl_sysv_attach, + .detach = dsm_impl_sysv_detach, + .destroy = dsm_impl_sysv_destroy, + .pin_segment = dsm_impl_noop_pin_segment, + .unpin_segment = dsm_impl_noop_unpin_segment, +}; + +#endif diff --git a/src/backend/storage/ipc/dsm_impl_windows.c b/src/backend/storage/ipc/dsm_impl_windows.c new file mode 100644 index 00000000000..f4b62f92c35 --- /dev/null +++ b/src/backend/storage/ipc/dsm_impl_windows.c @@ -0,0 +1,345 @@ +/*------------------------------------------------------------------------- + * + * dsm_impl_windows.c + * Operating system primitives to support Windows shared memory. + * + * Windows shared memory implementation is done using file mapping + * which can be backed by either physical file or system paging file. + * Current implementation uses system paging file as other effects + * like performance are not clear for physical file and it is used in + * similar way for main shared memory in windows. + * + * A memory mapping object is a kernel object - they always get deleted + * when the last reference to them goes away, either explicitly via a + * CloseHandle or when the process containing the reference exits. + * + * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group + * Portions Copyright (c) 1994, Regents of the University of California + * + * + * IDENTIFICATION + * src/backend/storage/ipc/dsm_impl_windows.c + * + *------------------------------------------------------------------------- + */ +#include "postgres.h" + +#include "common/file_perm.h" +#include "common/pg_prng.h" +#include "miscadmin.h" +#include "pgstat.h" +#include "portability/mem.h" +#include "postmaster/postmaster.h" +#include "storage/dsm_impl.h" +#include "storage/fd.h" +#include "utils/guc.h" + +#define SEGMENT_NAME_PREFIX "Global/PostgreSQL" + +static dsm_impl_handle +dsm_impl_windows_create(Size request_size, + dsm_impl_private *impl_private, void **mapped_address, + int elevel) +{ + char *address; + HANDLE hmap; + char name[64]; + DWORD size_high; + DWORD size_low; + DWORD errcode; + dsm_impl_handle handle; + + /* Shifts >= the width of the type are undefined. */ +#ifdef _WIN64 + size_high = request_size >> 32; +#else + size_high = 0; +#endif + size_low = (DWORD) request_size; + + /* + * Create new segment, with a random name. If the name is already in use, + * retry until we find an unused name. + */ + for (;;) + { + do + { + handle = pg_prng_uint32(&pg_global_prng_state); + } while (handle == DSM_IMPL_HANDLE_INVALID); + + /* + * Storing the shared memory segment in the Global\ namespace can + * allow any process running in any session to access that file + * mapping object provided that the caller has the required access + * rights. But to avoid issues faced in main shared memory, we are + * using the naming convention similar to main shared memory. We can + * change here once issue mentioned in GetSharedMemName is resolved. + */ + snprintf(name, 64, "%s.%u", SEGMENT_NAME_PREFIX, handle); + + /* CreateFileMapping might not clear the error code on success */ + SetLastError(0); + + hmap = CreateFileMapping(INVALID_HANDLE_VALUE, /* Use the pagefile */ + NULL, /* Default security attrs */ + PAGE_READWRITE, /* Memory is read/write */ + size_high, /* Upper 32 bits of size */ + size_low, /* Lower 32 bits of size */ + name); + + errcode = GetLastError(); + if (errcode == ERROR_ALREADY_EXISTS || errcode == ERROR_ACCESS_DENIED) + { + /* + * On Windows, when the segment already exists, a handle for the + * existing segment is returned. We must close it before + * retrying. However, if the existing segment is created by a + * service, then it returns ERROR_ACCESS_DENIED. We don't do + * _dosmaperr here, so errno won't be modified. + */ + if (hmap) + CloseHandle(hmap); + continue; + } + break; + } + + if (!hmap) + { + _dosmaperr(errcode); + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not create shared memory segment \"%s\": %m", + name))); + return DSM_IMPL_HANDLE_INVALID; + } + + /* Map it. */ + address = MapViewOfFile(hmap, FILE_MAP_WRITE | FILE_MAP_READ, + 0, 0, 0); + if (!address) + { + int save_errno; + + _dosmaperr(GetLastError()); + /* Back out what's already been done. */ + save_errno = errno; + CloseHandle(hmap); + errno = save_errno; + + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not map shared memory segment \"%s\": %m", + name))); + return DSM_IMPL_HANDLE_INVALID; + } + + *mapped_address = address; + *impl_private = (uintptr_t) hmap; + + return handle; +} + +static bool +dsm_impl_windows_attach(dsm_impl_handle handle, + dsm_impl_private *impl_private, void **mapped_address, + Size *mapped_size, int elevel) +{ + char *address; + HANDLE hmap; + char name[64]; + MEMORY_BASIC_INFORMATION info; + + snprintf(name, 64, "%s.%u", SEGMENT_NAME_PREFIX, handle); + + /* Open the existing mapping object. */ + hmap = OpenFileMapping(FILE_MAP_WRITE | FILE_MAP_READ, + FALSE, /* do not inherit the name */ + name); /* name of mapping object */ + if (!hmap) + { + _dosmaperr(GetLastError()); + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not open shared memory segment \"%s\": %m", + name))); + return false; + } + + /* Map it to this process's address space. */ + address = MapViewOfFile(hmap, FILE_MAP_WRITE | FILE_MAP_READ, + 0, 0, 0); + if (!address) + { + int save_errno; + + _dosmaperr(GetLastError()); + /* Back out what's already been done. */ + save_errno = errno; + CloseHandle(hmap); + errno = save_errno; + + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not map shared memory segment \"%s\": %m", + name))); + return false; + } + + /* + * Determine the size of the mapping. + * + * Note: Windows rounds up the mapping size to the 4K page size, so this + * can be larger than what was requested when the segment was created. + */ + if (VirtualQuery(address, &info, sizeof(info)) == 0) + { + int save_errno; + + _dosmaperr(GetLastError()); + /* Back out what's already been done. */ + save_errno = errno; + UnmapViewOfFile(address); + CloseHandle(hmap); + errno = save_errno; + + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not stat shared memory segment \"%s\": %m", + name))); + return false; + } + + *mapped_address = address; + *mapped_size = info.RegionSize; + *impl_private = (uintptr_t) hmap; + + return true; +} + +static bool +dsm_impl_windows_detach(dsm_impl_handle handle, + dsm_impl_private impl_private, void *mapped_address, + Size mapped_size, int elevel) +{ + HANDLE hmap; + char name[64]; + + Assert(mapped_address != NULL); + hmap = (HANDLE) impl_private; + + snprintf(name, 64, "%s.%u", SEGMENT_NAME_PREFIX, handle); + + /* + * Handle teardown cases. + */ + if (UnmapViewOfFile(mapped_address) == 0) + { + _dosmaperr(GetLastError()); + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not unmap shared memory segment \"%s\": %m", + name))); + return false; + } + if (CloseHandle(hmap) == 0) + { + _dosmaperr(GetLastError()); + ereport(elevel, + (errcode_for_dynamic_shared_memory(), + errmsg("could not remove shared memory segment \"%s\": %m", + name))); + return false; + } + + return true; +} + +/* + * Since Windows automatically destroys the object when no references remain, + * this is a no-op. + */ +static bool +dsm_impl_windows_destroy(dsm_impl_handle handle, int elevel) +{ + return true; +} + +/* + * Windows cleans up segments automatically when no references remain. That's + * handy, but if a segment needs to be preserved even when no backend has it + * attached, we need to take measures to prevent it from being cleaned up. To + * prevent it, we duplicate the segment handle into the postmaster process. + * The postmaster needn't do anything to receive the handle; Windows transfers + * it automatically. + */ +static void +dsm_impl_windows_pin_segment(dsm_impl_handle handle, dsm_impl_private impl_private, + dsm_impl_private_pm_handle *pm_handle) +{ + if (IsUnderPostmaster) + { + HANDLE hmap = (HANDLE) impl_private; + HANDLE pm_hmap; + + if (!DuplicateHandle(GetCurrentProcess(), hmap, + PostmasterHandle, &pm_hmap, 0, FALSE, + DUPLICATE_SAME_ACCESS)) + { + char name[64]; + + snprintf(name, 64, "%s.%u", SEGMENT_NAME_PREFIX, handle); + _dosmaperr(GetLastError()); + ereport(ERROR, + (errcode_for_dynamic_shared_memory(), + errmsg("could not duplicate handle for \"%s\": %m", + name))); + } + + /* + * Here, we remember the handle that we created in the postmaster + * process. This handle isn't actually usable in any process other + * than the postmaster, but that doesn't matter. We're just holding + * onto it so that, if the segment is unpinned, dsm_unpin_segment can + * close it. + */ + *pm_handle = (uintptr_t) pm_hmap; + } +} + +/* + * Close the extra handle that pin_segment created in the postmaster's process + * space. + */ +static void +dsm_impl_windows_unpin_segment(dsm_impl_handle handle, dsm_impl_private_pm_handle pm_handle) +{ + if (IsUnderPostmaster) + { + HANDLE pm_hmap = (HANDLE) pm_handle; + + if (!DuplicateHandle(PostmasterHandle, pm_hmap, + NULL, NULL, 0, FALSE, + DUPLICATE_CLOSE_SOURCE)) + { + char name[64]; + + snprintf(name, 64, "%s.%u", SEGMENT_NAME_PREFIX, handle); + _dosmaperr(GetLastError()); + ereport(ERROR, + (errcode_for_dynamic_shared_memory(), + errmsg("could not duplicate handle for \"%s\": %m", + name))); + } + } +} + +const dsm_impl_ops dsm_impl_windows_ops = { + .create = dsm_impl_windows_create, + .attach = dsm_impl_windows_attach, + .detach = dsm_impl_windows_detach, + .destroy = dsm_impl_windows_destroy, + .pin_segment = dsm_impl_windows_pin_segment, + .unpin_segment = dsm_impl_windows_unpin_segment, +}; diff --git a/src/backend/storage/ipc/meson.build b/src/backend/storage/ipc/meson.build index 5a936171f73..5acea5af250 100644 --- a/src/backend/storage/ipc/meson.build +++ b/src/backend/storage/ipc/meson.build @@ -20,3 +20,15 @@ backend_sources += files( 'standby.c', ) + +if host_system == 'windows' + backend_sources += files( + 'dsm_impl_windows.c', + ) +else + backend_sources += files( + 'dsm_impl_posix.c', + 'dsm_impl_sysv.c', + 'dsm_impl_mmap.c', + ) +endif diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c index 70652f0a3fc..1d809ba2c29 100644 --- a/src/backend/utils/misc/guc_tables.c +++ b/src/backend/utils/misc/guc_tables.c @@ -4885,7 +4885,7 @@ struct config_enum ConfigureNamesEnum[] = }, &dynamic_shared_memory_type, DEFAULT_DYNAMIC_SHARED_MEMORY_TYPE, dynamic_shared_memory_options, - NULL, NULL, NULL + NULL, assign_dynamic_shared_memory_type, NULL }, { diff --git a/src/include/storage/dsm.h b/src/include/storage/dsm.h index 35ae4eb164e..5d46361e4a0 100644 --- a/src/include/storage/dsm.h +++ b/src/include/storage/dsm.h @@ -17,11 +17,19 @@ typedef struct dsm_segment dsm_segment; +/* + * Note, must be meaningful even across restarts + */ +typedef uint32 dsm_handle; + +/* Sentinel value to use for invalid DSM handles. */ +#define DSM_HANDLE_INVALID 0 + #define DSM_CREATE_NULL_IF_MAXSEGMENTS 0x0001 /* Startup and shutdown functions. */ struct PGShmemHeader; /* avoid including pg_shmem.h */ -extern void dsm_cleanup_using_control_segment(dsm_handle old_control_handle); +extern void dsm_cleanup_using_control_segment(dsm_impl_handle old_control_handle); extern void dsm_postmaster_startup(struct PGShmemHeader *); extern void dsm_backend_shutdown(void); extern void dsm_detach_all(void); diff --git a/src/include/storage/dsm_impl.h b/src/include/storage/dsm_impl.h index f2bfb2a1a5c..0f351776773 100644 --- a/src/include/storage/dsm_impl.h +++ b/src/include/storage/dsm_impl.h @@ -13,6 +13,10 @@ #ifndef DSM_IMPL_H #define DSM_IMPL_H +typedef uint32 dsm_impl_handle; + +#define DSM_IMPL_HANDLE_INVALID 0 + /* Dynamic shared memory implementations. */ #define DSM_IMPL_POSIX 1 #define DSM_IMPL_SYSV 2 @@ -51,25 +55,11 @@ extern PGDLLIMPORT int min_dynamic_shared_memory; #define PG_DYNSHMEM_DIR "pg_dynshmem" #define PG_DYNSHMEM_MMAP_FILE_PREFIX "mmap." -/* A "name" for a dynamic shared memory segment. */ -typedef uint32 dsm_handle; - -/* Sentinel value to use for invalid DSM handles. */ -#define DSM_HANDLE_INVALID ((dsm_handle) 0) - -/* All the shared-memory operations we know about. */ -typedef enum -{ - DSM_OP_CREATE, - DSM_OP_ATTACH, - DSM_OP_DETACH, - DSM_OP_DESTROY, -} dsm_op; - /* * When a segment is created or attached, the caller provides this space to - * hold implementation-specific information about the attachment. It is opaque - * to the caller, and is passed back to the implementation when detaching. + * hold implementation-specific information about the attachment. It is + * opaque to the caller, and is passed back to the implementation when + * detaching. */ typedef uintptr_t dsm_impl_private; @@ -79,14 +69,73 @@ typedef uintptr_t dsm_impl_private; */ typedef uintptr_t dsm_impl_private_pm_handle; -/* Create, attach to, detach from, resize, or destroy a segment. */ -extern bool dsm_impl_op(dsm_op op, dsm_handle handle, Size request_size, - dsm_impl_private *impl_private, void **mapped_address, Size *mapped_size, - int elevel); +/* All the shared-memory operations we know about. */ +typedef struct +{ + /* + * Create a new DSM segment, and map it to the current process's address + * space. + * + * Note: the mapped_size can differ from requested size (currently only + * Windows) + * + */ + dsm_impl_handle(*create) (Size request_size, dsm_impl_private *impl_private, + void **mapped_address, int elevel); + + /* + * Map an existing DSM segment to current process's address space. + */ + bool (*attach) (dsm_impl_handle handle, dsm_impl_private *impl_private, + void **mapped_address, Size *mapped_size, int elevel); + + /* + * Unmap + */ + bool (*detach) (dsm_impl_handle handle, dsm_impl_private impl_private, + void *mapped_address, Size mapped_size, int elevel); + + /* + * Destroy a DSM segment. + * + * Note: must detach first + */ + bool (*destroy) (dsm_impl_handle handle, int elevel); + + /* + * Implementation-specific actions that must be performed when a segment + * is to be preserved even when no backend has it attached. + */ + void (*pin_segment) (dsm_impl_handle handle, dsm_impl_private impl_private, + dsm_impl_private_pm_handle *pm_handle); + + /* + * Implementation-specific actions that must be performed when a segment + * is no longer to be preserved, so that it will be cleaned up when all + * backends have detached from it. + */ + void (*unpin_segment) (dsm_impl_handle handle, dsm_impl_private_pm_handle pm_handle); +} dsm_impl_ops; + +extern PGDLLIMPORT const dsm_impl_ops *dsm_impl; + +#ifdef USE_DSM_POSIX +extern PGDLLIMPORT const dsm_impl_ops dsm_impl_posix_ops; +#endif +#ifdef USE_DSM_SYSV +extern PGDLLIMPORT const dsm_impl_ops dsm_impl_sysv_ops; +#endif +#ifdef USE_DSM_WINDOWS +extern PGDLLIMPORT const dsm_impl_ops dsm_impl_windows_ops; +#endif +#ifdef USE_DSM_MMAP +extern PGDLLIMPORT const dsm_impl_ops dsm_impl_mmap_ops; +#endif + +extern void dsm_impl_noop_pin_segment(dsm_impl_handle handle, dsm_impl_private impl_private, + dsm_impl_private_pm_handle *pm_handle); +extern void dsm_impl_noop_unpin_segment(dsm_impl_handle handle, dsm_impl_private_pm_handle pm_handle); -/* Implementation-dependent actions required to keep segment until shutdown. */ -extern void dsm_impl_pin_segment(dsm_handle handle, dsm_impl_private impl_private, - dsm_impl_private_pm_handle *impl_private_pm_handle); -extern void dsm_impl_unpin_segment(dsm_handle handle, dsm_impl_private_pm_handle *pm_handle); +extern int errcode_for_dynamic_shared_memory(void); #endif /* DSM_IMPL_H */ diff --git a/src/include/storage/pg_shmem.h b/src/include/storage/pg_shmem.h index 3065ff5be71..c319ad6208f 100644 --- a/src/include/storage/pg_shmem.h +++ b/src/include/storage/pg_shmem.h @@ -33,7 +33,7 @@ typedef struct PGShmemHeader /* standard header for all Postgres shmem */ pid_t creatorPID; /* PID of creating process (set but unread) */ Size totalsize; /* total size of segment */ Size freeoffset; /* offset to first free space */ - dsm_handle dsm_control; /* ID of dynamic shared memory control seg */ + dsm_impl_handle dsm_control; /* ID of dynamic shared memory control seg */ void *index; /* pointer to ShmemIndex table */ #ifndef WIN32 /* Windows doesn't have useful inode#s */ dev_t device; /* device data directory is on */ diff --git a/src/include/utils/guc_hooks.h b/src/include/utils/guc_hooks.h index 339c490300e..289b5c7addc 100644 --- a/src/include/utils/guc_hooks.h +++ b/src/include/utils/guc_hooks.h @@ -59,6 +59,8 @@ extern bool check_default_text_search_config(char **newval, void **extra, GucSou extern void assign_default_text_search_config(const char *newval, void *extra); extern bool check_default_with_oids(bool *newval, void **extra, GucSource source); +extern void assign_dynamic_shared_memory_type(int new_dynamic_shared_memory_type, + void *extra); extern bool check_effective_io_concurrency(int *newval, void **extra, GucSource source); extern bool check_huge_page_size(int *newval, void **extra, GucSource source); diff --git a/src/test/modules/Makefile b/src/test/modules/Makefile index 89aa41b5e31..8750eab03de 100644 --- a/src/test/modules/Makefile +++ b/src/test/modules/Makefile @@ -18,6 +18,7 @@ SUBDIRS = \ test_custom_rmgrs \ test_ddl_deparse \ test_dsa \ + test_dsm \ test_dsm_registry \ test_extensions \ test_ginpostinglist \ diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build index 8fbe742d385..54b1c26a7fa 100644 --- a/src/test/modules/meson.build +++ b/src/test/modules/meson.build @@ -17,6 +17,7 @@ subdir('test_copy_callbacks') subdir('test_custom_rmgrs') subdir('test_ddl_deparse') subdir('test_dsa') +subdir('test_dsm') subdir('test_dsm_registry') subdir('test_extensions') subdir('test_ginpostinglist') diff --git a/src/test/modules/test_dsm/.gitignore b/src/test/modules/test_dsm/.gitignore new file mode 100644 index 00000000000..9a737823e16 --- /dev/null +++ b/src/test/modules/test_dsm/.gitignore @@ -0,0 +1,3 @@ +# Generated subdirectories +/log/ +/tmp_check/ diff --git a/src/test/modules/test_dsm/Makefile b/src/test/modules/test_dsm/Makefile new file mode 100644 index 00000000000..12168490652 --- /dev/null +++ b/src/test/modules/test_dsm/Makefile @@ -0,0 +1,27 @@ +# src/test/modules/test_dsm/Makefile + +MODULE_big = test_dsm +OBJS = \ + $(WIN32RES) \ + test_dsm.o +PGFILEDESC = "test_dsm - test code for dynamic shared memory" + +EXTENSION = test_dsm +DATA = test_dsm--1.0.sql + +ifdef USE_PGXS +PG_CONFIG = pg_config +PGXS := $(shell $(PG_CONFIG) --pgxs) +include $(PGXS) +else +subdir = src/test/modules/test_dsm +top_builddir = ../../../.. +include $(top_builddir)/src/Makefile.global +include $(top_srcdir)/contrib/contrib-global.mk +endif + +check: + $(prove_check) + +installcheck: + $(prove_installcheck) diff --git a/src/test/modules/test_dsm/meson.build b/src/test/modules/test_dsm/meson.build new file mode 100644 index 00000000000..4321cabe6c4 --- /dev/null +++ b/src/test/modules/test_dsm/meson.build @@ -0,0 +1,33 @@ +# Copyright (c) 2022-2024, PostgreSQL Global Development Group + +test_dsm_sources = files( + 'test_dsm.c', +) + +if host_system == 'windows' + test_dsm_sources += rc_lib_gen.process(win32ver_rc, extra_args: [ + '--NAME', 'test_dsm', + '--FILEDESC', 'test_dsm - test code for dynamic shared memory',]) +endif + +test_dsm = shared_module('test_dsm', + test_dsm_sources, + kwargs: pg_test_mod_args, +) +test_install_libs += test_dsm + +test_install_data += files( + 'test_dsm.control', + 'test_dsm--1.0.sql', +) + +tests += { + 'name': 'test_dsm', + 'sd': meson.current_source_dir(), + 'bd': meson.current_build_dir(), + 'tap': { + 'tests': [ + 't/001_dsm_basic.pl', + ], + }, +} diff --git a/src/test/modules/test_dsm/t/001_dsm_basic.pl b/src/test/modules/test_dsm/t/001_dsm_basic.pl new file mode 100644 index 00000000000..dacc4e75ce1 --- /dev/null +++ b/src/test/modules/test_dsm/t/001_dsm_basic.pl @@ -0,0 +1,61 @@ + +# Copyright (c) 2024, PostgreSQL Global Development Group + +# Run the standard regression tests with streaming replication +use strict; +use warnings FATAL => 'all'; +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More; +use File::Basename; + +# Initialize primary node +my $node = PostgreSQL::Test::Cluster->new('main'); +$node->init(); + +# Run a SQL command and return psql's stderr (including debug messages) +sub run_dsm_basic_test +{ + local $Test::Builder::Level = $Test::Builder::Level + 1; + + my $dynamic_shared_memory_type = shift; + my $min_dynamic_shared_memory = shift; + + $node->adjust_conf('postgresql.conf', 'dynamic_shared_memory_type', $dynamic_shared_memory_type); + $node->adjust_conf('postgresql.conf', 'min_dynamic_shared_memory', $min_dynamic_shared_memory); + + $node->start; + + $node->safe_psql('postgres', 'CREATE EXTENSION IF NOT EXISTS test_dsm'); + + my $stderr; + my $cmdret = $node->psql('postgres', 'SELECT test_dsm_basic()', stderr => \$stderr); + ok($cmdret == 0, "$dynamic_shared_memory_type with minsize $min_dynamic_shared_memory"); + is($stderr, '', "$dynamic_shared_memory_type with minsize $min_dynamic_shared_memory"); + + $node->stop('fast'); +} + +# Test all the DSM implementations + +SKIP: +{ + skip "Skipping posix, sysv, mmap test on Windows", 2 if ($windows_os); + + run_dsm_basic_test('posix', 0); + run_dsm_basic_test('posix', 1000000); + run_dsm_basic_test('sysv', 0); + run_dsm_basic_test('sysv', 1000000); + run_dsm_basic_test('mmap', 0); + run_dsm_basic_test('mmap', 1000000); +} + +SKIP: +{ + skip "Windows dsm support", 2 if (!$windows_os); + + run_dsm_basic_test('windows', 0); + run_dsm_basic_test('windows', 1000000); +} + +done_testing(); diff --git a/src/test/modules/test_dsm/test_dsm--1.0.sql b/src/test/modules/test_dsm/test_dsm--1.0.sql new file mode 100644 index 00000000000..4dbdc811b81 --- /dev/null +++ b/src/test/modules/test_dsm/test_dsm--1.0.sql @@ -0,0 +1,9 @@ +/* src/test/modules/test_dsm/test_dsm--1.0.sql */ + +-- complain if script is sourced in psql, rather than via CREATE EXTENSION +\echo Use "CREATE EXTENSION test_dsm" to load this file. \quit + +CREATE FUNCTION test_dsm_basic() + RETURNS pg_catalog.void + AS 'MODULE_PATHNAME' LANGUAGE C; + diff --git a/src/test/modules/test_dsm/test_dsm.c b/src/test/modules/test_dsm/test_dsm.c new file mode 100644 index 00000000000..3b64d8a2f02 --- /dev/null +++ b/src/test/modules/test_dsm/test_dsm.c @@ -0,0 +1,75 @@ +/*-------------------------------------------------------------------------- + * + * test_dsm.c + * Test dynamic shared memory + * + * Copyright (c) 2022-2024, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/modules/test_dsm/test_dsm.c + * + * ------------------------------------------------------------------------- + */ +#include "postgres.h" + +#include "fmgr.h" +#include "storage/dsm.h" + +PG_MODULE_MAGIC; + + +/* Test basic DSM functionality */ +PG_FUNCTION_INFO_V1(test_dsm_basic); +Datum +test_dsm_basic(PG_FUNCTION_ARGS) +{ + dsm_segment *seg; + unsigned char *p; + Size requested_size; + Size created_size; + Size attached_size; + dsm_handle handle; + + /* Create a small DSM segment */ + requested_size = 1000; + seg = dsm_create(requested_size, 0); + + handle = dsm_segment_handle(seg); + created_size = dsm_segment_map_length(seg); + if (requested_size != created_size) + elog(ERROR, "DSM size mismatch, requested %lu but created as %lu", requested_size, created_size); + + /* Fill it with data */ + p = dsm_segment_address(seg); + memset(p, 0x12, requested_size); + + /* Pin the segment so that it's not destroyed when we detach it */ + dsm_pin_segment(seg); + + /* Detach and re-attach it */ + dsm_detach(seg); + seg = dsm_attach(handle); + + /* + * Check the size after re-attaching. It can be larger than what was + * requested originally, because some implementations round it up to the + * nearest page size. Tolerate that. + */ + attached_size = dsm_segment_map_length(seg); + if (attached_size < created_size) + elog(ERROR, "DSM size mismatch, created %lu but attached %lu", created_size, attached_size); + if (requested_size + 100000 < created_size) + elog(ERROR, "unexpectdly large size after attach: requested %lu but got %lu", requested_size, created_size); + + /* check contents */ + p = dsm_segment_address(seg); + for (Size i; i < created_size; i++) + { + if (p[i] != 0x12) + elog(ERROR, "DSM segment has unexpected content %u at offset %lu", p[i], i); + } + + dsm_detach(seg); + + PG_RETURN_VOID(); +} diff --git a/src/test/modules/test_dsm/test_dsm.control b/src/test/modules/test_dsm/test_dsm.control new file mode 100644 index 00000000000..2c25c4d55f2 --- /dev/null +++ b/src/test/modules/test_dsm/test_dsm.control @@ -0,0 +1,4 @@ +comment = 'Test code for dynamic shared memory' +default_version = '1.0' +module_pathname = '$libdir/test_dsm' +relocatable = true diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index d808aad8b05..39b11c88c9d 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -3322,6 +3322,9 @@ dshash_table_item dsm_control_header dsm_control_item dsm_handle +dsm_impl_ops +dsm_impl_private +dsm_impl_private_pm_handle dsm_op dsm_segment dsm_segment_detach_callback -- 2.39.2 ^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2024-02-21 19:19 UTC | newest] Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2020-07-15 22:35 [PATCH v13 1/6] snapshot scalability: Don't compute global horizons when building snapshots. Andres Freund <[email protected]> 2024-02-21 19:19 Re: DSA_ALLOC_NO_OOM doesn't work Heikki Linnakangas <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox