Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jvses-0004LA-Q2 for pgsql-hackers@arkaria.postgresql.org; Thu, 16 Jul 2020 01:26:00 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1jvser-0007AS-2B for pgsql-hackers@arkaria.postgresql.org; Thu, 16 Jul 2020 01:25:57 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jvseo-0007AL-Iy for pgsql-hackers@lists.postgresql.org; Thu, 16 Jul 2020 01:25:56 +0000 Received: from out2-smtp.messagingengine.com ([66.111.4.26]) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jvseX-0000lD-AS; Thu, 16 Jul 2020 01:25:53 +0000 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id 375FC5C0110; Wed, 15 Jul 2020 21:25:36 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Wed, 15 Jul 2020 21:25:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=anarazel.de; h= date:from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm2; bh=35F4ZPG5stg3dDW55EDMBsYi0xo iYFc4YiO7ASVP0HM=; b=DUefR1fErW+BYudQglrJ1wynTfpG5E7+hBQ/cvDi5SX Hgcsh4LDEpRoT6egEcnUC9x8E4ScIR0O2FKHGxuqziEv+qyf3/lKbV2OWqcqqAzo c97j+f74zW2qqRZTCtCredHB/wxiAxxSKiv43QTAi0j+WFe8HYiTUFaeQZ5Q1okA 2Nz6TQMZNb+myK1RaFwwB6Ew9MDz6cgdhJb4lrYjm4NFo2jloSwbv2sROnxT7okl dHrjcbZai/azyKPGoKkjvn7X5kqzA+KAaKpdD/ncRJiETVXiZBh6PTv3jz7jWdMN fPDtjIBt4GtH/7xUK0T+XPU23AJOWjCz/KizoYVtv1A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=35F4ZP G5stg3dDW55EDMBsYi0xoiYFc4YiO7ASVP0HM=; b=TygXdwee/W6F9Kekfjh75i qNoVYBgIY9ukFYfWlPFf0DGCeyw/Nwm5dL1EWiT2WuXtZcPQ93xQ6L7WgDhIjsrj pBeQYKZRfGiqcd/mWi91MwtFOTWCSHaBGCEn757ZGCulIBp2i5P5X9Ojr7CqHTpG CIoO4Oak6fU1tcDqziODTRt0emVVm4lRHqCxzTfMERMzjycxc60QaJg6c/thsRJO dY9STxG6RkE0QaVBz5e1RRnfwu55M7FyJ7AlObLtxsg6mrqQ6blLn01jN/ZUkdy+ VCofSiRz6j+kygYFrWsor4nxPcZ15qKSz/WPNCjDL89j9Lz1H1LzZ9z5ndv95RaQ == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduiedrfeefgdeghecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpeffhffvuffkfhggtggujgesmhdtreertddtvdenucfhrhhomheptehnughrvghs ucfhrhgvuhhnugcuoegrnhgurhgvshesrghnrghrrgiivghlrdguvgeqnecuggftrfgrth htvghrnhepgfeuheevheehtdeugfdvjeeuudejueekleetkeekveejudehgedvtdefveeh leeknecukfhppeeijedrudeitddrvddujedrvdehtdenucevlhhushhtvghrufhiiigvpe dtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegrnhgurhgvshesrghnrghrrgiivghlrdgu vg X-ME-Proxy: Received: from intern.anarazel.de (c-67-160-217-250.hsd1.ca.comcast.net [67.160.217.250]) by mail.messagingengine.com (Postfix) with ESMTPA id 506CF306005F; Wed, 15 Jul 2020 21:25:34 -0400 (EDT) Date: Wed, 15 Jul 2020 18:25:32 -0700 From: Andres Freund To: Daniel Gustafsson Cc: Robert Haas , "Jonathan S. Katz" , Alexander Korotkov , PostgreSQL Hackers , Peter Geoghegan , Alvaro Herrera , Bruce Momjian , Thomas Munro , David Rowley Subject: Re: Improving connection scalability: GetSnapshotData() Message-ID: <20200716012532.lfzpvg3w55d5rqrw@alap3.anarazel.de> References: <20200406133959.viql5fqecog6mppj@alap3.anarazel.de> <20200406235228.lnxcau2wgocnnn22@alap3.anarazel.de> <20200407121503.zltbpqmdesurflnm@alap3.anarazel.de> <20200408124318.amzvnpsqhy747wqp@alap3.anarazel.de> <07c6af6c-c30d-52e4-8716-e3688c3ef265@postgresql.org> <20200408222017.jnei3rc6e534aats@alap3.anarazel.de> <731B6D06-B8E0-44EB-BD63-3686F59168F9@yesql.se> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="kdyesixubxkl3b7r" Content-Disposition: inline In-Reply-To: <731B6D06-B8E0-44EB-BD63-3686F59168F9@yesql.se> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --kdyesixubxkl3b7r Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, On 2020-07-01 14:42:59 +0200, Daniel Gustafsson wrote: > This patch no longer applies to HEAD, please submit a rebased version. I've > marked it Waiting on Author in the meantime. Thanks! Here's a rebased version. There's a good bit of commit message polishing and some code and comment cleanup compared to the last version. Oh, and obviously the conflicts are resolved. It could make sense to split the conversion of VariableCacheData->latestCompletedXid to FullTransactionId out from 0001 into is own commit. Not sure... I've played with splitting 0003, to have the "infrastructure" pieces separate, but I think it's not worth it. Without a user the changes look weird and it's hard to have the comment make sense. Greetings, Andres Freund --kdyesixubxkl3b7r Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v11-0001-snapshot-scalability-Don-t-compute-global-horizo.patch" From 6f8b0d0cc0ac6fd882e4d3bb287ef6670d3da5ff Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Wed, 15 Jul 2020 15:35:07 -0700 Subject: [PATCH v11 1/6] snapshot scalability: Don't compute global horizons when building snapshots. To make GetSnapshotData() more scalable, it cannot not look at at each proc's xmin (see Discussion link below). Due to the frequency at which xmins are updated, that just does not scale. Without accessing xmins GetSnapshotData() cannot calculate accurate thresholds as it has so far. But we don't really have to: The horizons don't actually change that much between GetSnapshotData() calls. Nor are the horizons actually used every time a snapshot is called. The use of RecentGlobal[Data]Xmin to decide whether a row version could be removed has been replaces with new GlobalVisTest* functions. These use two thresholds to determine whether a row can be pruned: 1) definitely_needed, indicating that rows deleted by XIDs >= definitely_needed are definitely still visible. 2) maybe_needed, indicating that rows deleted by XIDs < maybe_needed can definitely be removed GetSnapshotData() updates definitely_needed to be the xmin of the computed snapshot. When testing whether a row can be removed (with GlobalVisTestIsRemovableXid()) and the tested XID falls in between the two (i.e. XID >= maybe_needed && XID < definitely_needed) the boundaries can be recomputed to be more accurate. As it is not cheap to compute accurate boundaries, we limit the number of times that happens in short succession. As the boundaries used by GlobalVisTestIsRemovableXid() are never reset (with maybe_needed updated byGetSnapshotData()), it is likely that further test can benefit from an earlier computation of accurate horizons. To avoid regressing performance when old_snapshot_threshold is set (as that requires an accurate horizon to be computed), heap_page_prune_opt() doesn't unconditionally call TransactionIdLimitedForOldSnapshots() anymore. Both the computation of the limited horizon, and the triggering of errors (with SetOldSnapshotThresholdTimestamp()) is now only done when necessary to remove tuples. Subsequent commits will take further advantage of the fact that GetSnapshotData() will not need to access xmins anymore. Note: This contains a workaround in heap_page_prune_opt() to keep the snapshot_too_old tests working. While that workaround is ugly, the tests currently are not meaningful, and it seems best to address them separately. Author: Andres Freund Reviewed-By: Robert Haas, Thomas Munro, David Rowley Discussion: https://postgr.es/m/20200301083601.ews6hz5dduc3w2se@alap3.anarazel.de --- src/include/access/ginblock.h | 4 +- src/include/access/heapam.h | 11 +- src/include/access/transam.h | 100 +- src/include/storage/bufpage.h | 6 - src/include/storage/proc.h | 8 - src/include/storage/procarray.h | 39 +- src/include/utils/snapmgr.h | 37 +- src/include/utils/snapshot.h | 6 + src/backend/access/gin/ginvacuum.c | 26 + src/backend/access/gist/gistutil.c | 8 +- src/backend/access/gist/gistxlog.c | 10 +- src/backend/access/heap/heapam.c | 15 +- src/backend/access/heap/heapam_handler.c | 24 +- src/backend/access/heap/heapam_visibility.c | 79 +- src/backend/access/heap/pruneheap.c | 207 +++- src/backend/access/heap/vacuumlazy.c | 24 +- src/backend/access/index/indexam.c | 3 +- src/backend/access/nbtree/README | 10 +- src/backend/access/nbtree/nbtpage.c | 4 +- src/backend/access/nbtree/nbtree.c | 28 +- src/backend/access/nbtree/nbtxlog.c | 10 +- src/backend/access/spgist/spgvacuum.c | 6 +- src/backend/access/transam/README | 92 +- src/backend/access/transam/varsup.c | 50 + src/backend/access/transam/xlog.c | 11 +- src/backend/commands/analyze.c | 2 +- src/backend/commands/vacuum.c | 41 +- src/backend/postmaster/autovacuum.c | 4 + src/backend/replication/logical/launcher.c | 4 + src/backend/replication/walreceiver.c | 17 +- src/backend/replication/walsender.c | 15 +- src/backend/storage/ipc/procarray.c | 1021 +++++++++++++++---- src/backend/utils/adt/selfuncs.c | 20 +- src/backend/utils/init/postinit.c | 4 + src/backend/utils/time/snapmgr.c | 258 ++--- contrib/amcheck/verify_nbtree.c | 8 +- contrib/pg_visibility/pg_visibility.c | 18 +- contrib/pgstattuple/pgstatapprox.c | 2 +- src/tools/pgindent/typedefs.list | 2 + 39 files changed, 1626 insertions(+), 608 deletions(-) diff --git a/src/include/access/ginblock.h b/src/include/access/ginblock.h index 3f64fd572e3..fe66a95226b 100644 --- a/src/include/access/ginblock.h +++ b/src/include/access/ginblock.h @@ -12,6 +12,7 @@ #include "access/transam.h" #include "storage/block.h" +#include "storage/bufpage.h" #include "storage/itemptr.h" #include "storage/off.h" @@ -134,8 +135,7 @@ typedef struct GinMetaPageData */ #define GinPageGetDeleteXid(page) ( ((PageHeader) (page))->pd_prune_xid ) #define GinPageSetDeleteXid(page, xid) ( ((PageHeader) (page))->pd_prune_xid = xid) -#define GinPageIsRecyclable(page) ( PageIsNew(page) || (GinPageIsDeleted(page) \ - && TransactionIdPrecedes(GinPageGetDeleteXid(page), RecentGlobalXmin))) +extern bool GinPageIsRecyclable(Page page); /* * We use our own ItemPointerGet(BlockNumber|OffsetNumber) diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h index f279edc4734..ef2fcb55a71 100644 --- a/src/include/access/heapam.h +++ b/src/include/access/heapam.h @@ -172,9 +172,12 @@ extern TransactionId heap_compute_xid_horizon_for_tuples(Relation rel, int nitems); /* in heap/pruneheap.c */ +struct GlobalVisState; extern void heap_page_prune_opt(Relation relation, Buffer buffer); extern int heap_page_prune(Relation relation, Buffer buffer, - TransactionId OldestXmin, + struct GlobalVisState *vistest, + TransactionId limited_oldest_xmin, + TimestampTz limited_oldest_ts, bool report_stats, TransactionId *latestRemovedXid); extern void heap_page_prune_execute(Buffer buffer, OffsetNumber *redirected, int nredirected, @@ -201,11 +204,15 @@ extern TM_Result HeapTupleSatisfiesUpdate(HeapTuple stup, CommandId curcid, Buffer buffer); extern HTSV_Result HeapTupleSatisfiesVacuum(HeapTuple stup, TransactionId OldestXmin, Buffer buffer); +extern HTSV_Result HeapTupleSatisfiesVacuumHorizon(HeapTuple stup, Buffer buffer, + TransactionId *dead_after); extern void HeapTupleSetHintBits(HeapTupleHeader tuple, Buffer buffer, uint16 infomask, TransactionId xid); extern bool HeapTupleHeaderIsOnlyLocked(HeapTupleHeader tuple); extern bool XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot); -extern bool HeapTupleIsSurelyDead(HeapTuple htup, TransactionId OldestXmin); +struct GlobalVisState; +extern bool HeapTupleIsSurelyDead(struct GlobalVisState *vistest, + HeapTuple htup); /* * To avoid leaking too much knowledge about reorderbuffer implementation diff --git a/src/include/access/transam.h b/src/include/access/transam.h index a91a0c7487d..6ec84b54599 100644 --- a/src/include/access/transam.h +++ b/src/include/access/transam.h @@ -54,6 +54,8 @@ #define FullTransactionIdFollowsOrEquals(a, b) ((a).value >= (b).value) #define FullTransactionIdIsValid(x) TransactionIdIsValid(XidFromFullTransactionId(x)) #define InvalidFullTransactionId FullTransactionIdFromEpochAndXid(0, InvalidTransactionId) +#define FirstNormalFullTransactionId FullTransactionIdFromEpochAndXid(0, FirstNormalTransactionId) +#define FullTransactionIdIsNormal(x) FullTransactionIdFollowsOrEquals(x, FirstNormalFullTransactionId) /* * A 64 bit value that contains an epoch and a TransactionId. This is @@ -93,15 +95,48 @@ FullTransactionIdFromU64(uint64 value) (dest) = FirstNormalTransactionId; \ } while(0) -/* advance a FullTransactionId variable, stepping over special XIDs */ +/* + * Advance a FullTransactionId variable, stepping over xids that would appear + * to be special only when viewed as 32bit XIDs. + */ static inline void FullTransactionIdAdvance(FullTransactionId *dest) { dest->value++; + + /* + * In contrast to 32bit XIDs don't step over the "actual" special xids. + * For 64bit xids these can't be reached as part of a wraparound as they + * can in the 32bit case. + */ + if (FullTransactionIdPrecedes(*dest, FirstNormalFullTransactionId)) + return; + + /* + * But we do need to step over XIDs that'd appear special only for 32bit + * XIDs. + */ while (XidFromFullTransactionId(*dest) < FirstNormalTransactionId) dest->value++; } +/* + * Retreat a FullTransactionId variable, stepping over xids that would appear + * to be special only when viewed as 32bit XIDs. + */ +static inline void +FullTransactionIdRetreat(FullTransactionId *dest) +{ + dest->value--; + + /* see FullTransactionIdAdvance() */ + if (FullTransactionIdPrecedes(*dest, FirstNormalFullTransactionId)) + return; + + while (XidFromFullTransactionId(*dest) < FirstNormalTransactionId) + dest->value--; +} + /* back up a transaction ID variable, handling wraparound correctly */ #define TransactionIdRetreat(dest) \ do { \ @@ -193,8 +228,8 @@ typedef struct VariableCacheData /* * These fields are protected by ProcArrayLock. */ - TransactionId latestCompletedXid; /* newest XID that has committed or - * aborted */ + FullTransactionId latestCompletedFullXid; /* newest full XID that has + * committed or aborted */ /* * These fields are protected by XactTruncationLock @@ -244,6 +279,12 @@ extern void AdvanceOldestClogXid(TransactionId oldest_datfrozenxid); extern bool ForceTransactionIdLimitUpdate(void); extern Oid GetNewObjectId(void); +#ifdef USE_ASSERT_CHECKING +extern void AssertTransactionInAllowableRange(TransactionId xid); +#else +#define AssertTransactionInAllowableRange(xid) ((void)true) +#endif + /* * Some frontend programs include this header. For compilers that emit static * inline functions even when they're unused, that leads to unsatisfied @@ -260,6 +301,59 @@ ReadNewTransactionId(void) return XidFromFullTransactionId(ReadNextFullTransactionId()); } +/* return transaction ID backed up by amount, handling wraparound correctly */ +static inline TransactionId +TransactionIdRetreatedBy(TransactionId xid, uint32 amount) +{ + xid -= amount; + + while (xid < FirstNormalTransactionId) + xid--; + + return xid; +} + +/* return the older of the two IDs */ +static inline TransactionId +TransactionIdOlder(TransactionId a, TransactionId b) +{ + if (!TransactionIdIsValid(a)) + return b; + + if (!TransactionIdIsValid(b)) + return a; + + if (TransactionIdPrecedes(a, b)) + return a; + return b; +} + +/* return the older of the two IDs, assuming they're both normal */ +static inline TransactionId +NormalTransactionIdOlder(TransactionId a, TransactionId b) +{ + Assert(TransactionIdIsNormal(a)); + Assert(TransactionIdIsNormal(b)); + if (NormalTransactionIdPrecedes(a, b)) + return a; + return b; +} + +/* return the newer of the two IDs */ +static inline FullTransactionId +FullTransactionIdNewer(FullTransactionId a, FullTransactionId b) +{ + if (!FullTransactionIdIsValid(a)) + return b; + + if (!FullTransactionIdIsValid(b)) + return a; + + if (FullTransactionIdFollows(a, b)) + return a; + return b; +} + #endif /* FRONTEND */ #endif /* TRANSAM_H */ diff --git a/src/include/storage/bufpage.h b/src/include/storage/bufpage.h index 3f88683a059..51b8f994ac0 100644 --- a/src/include/storage/bufpage.h +++ b/src/include/storage/bufpage.h @@ -389,12 +389,6 @@ PageValidateSpecialPointer(Page page) #define PageClearAllVisible(page) \ (((PageHeader) (page))->pd_flags &= ~PD_ALL_VISIBLE) -#define PageIsPrunable(page, oldestxmin) \ -( \ - AssertMacro(TransactionIdIsNormal(oldestxmin)), \ - TransactionIdIsValid(((PageHeader) (page))->pd_prune_xid) && \ - TransactionIdPrecedes(((PageHeader) (page))->pd_prune_xid, oldestxmin) \ -) #define PageSetPrunable(page, xid) \ do { \ Assert(TransactionIdIsNormal(xid)); \ diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h index b20e2ad4f6a..08f006f782e 100644 --- a/src/include/storage/proc.h +++ b/src/include/storage/proc.h @@ -42,13 +42,6 @@ struct XidCache /* * Flags for PGXACT->vacuumFlags - * - * Note: If you modify these flags, you need to modify PROCARRAY_XXX flags - * in src/include/storage/procarray.h. - * - * PROC_RESERVED may later be assigned for use in vacuumFlags, but its value is - * used for PROCARRAY_SLOTS_XMIN in procarray.h, so GetOldestXmin won't be able - * to match and ignore processes with this flag set. */ #define PROC_IS_AUTOVACUUM 0x01 /* is it an autovac worker? */ #define PROC_IN_VACUUM 0x02 /* currently running lazy vacuum */ @@ -56,7 +49,6 @@ struct XidCache #define PROC_VACUUM_FOR_WRAPAROUND 0x08 /* set by autovac only */ #define PROC_IN_LOGICAL_DECODING 0x10 /* currently doing logical * decoding outside xact */ -#define PROC_RESERVED 0x20 /* reserved for procarray */ /* flags reset at EOXact */ #define PROC_VACUUM_STATE_MASK \ diff --git a/src/include/storage/procarray.h b/src/include/storage/procarray.h index a5c7d0c0644..ea8a876ca45 100644 --- a/src/include/storage/procarray.h +++ b/src/include/storage/procarray.h @@ -20,41 +20,6 @@ #include "utils/snapshot.h" -/* - * These are to implement PROCARRAY_FLAGS_XXX - * - * Note: These flags are cloned from PROC_XXX flags in src/include/storage/proc.h - * to avoid forcing to include proc.h when including procarray.h. So if you modify - * PROC_XXX flags, you need to modify these flags. - */ -#define PROCARRAY_VACUUM_FLAG 0x02 /* currently running lazy - * vacuum */ -#define PROCARRAY_ANALYZE_FLAG 0x04 /* currently running - * analyze */ -#define PROCARRAY_LOGICAL_DECODING_FLAG 0x10 /* currently doing logical - * decoding outside xact */ - -#define PROCARRAY_SLOTS_XMIN 0x20 /* replication slot xmin, - * catalog_xmin */ -/* - * Only flags in PROCARRAY_PROC_FLAGS_MASK are considered when matching - * PGXACT->vacuumFlags. Other flags are used for different purposes and - * have no corresponding PROC flag equivalent. - */ -#define PROCARRAY_PROC_FLAGS_MASK (PROCARRAY_VACUUM_FLAG | \ - PROCARRAY_ANALYZE_FLAG | \ - PROCARRAY_LOGICAL_DECODING_FLAG) - -/* Use the following flags as an input "flags" to GetOldestXmin function */ -/* Consider all backends except for logical decoding ones which manage xmin separately */ -#define PROCARRAY_FLAGS_DEFAULT PROCARRAY_LOGICAL_DECODING_FLAG -/* Ignore vacuum backends */ -#define PROCARRAY_FLAGS_VACUUM PROCARRAY_FLAGS_DEFAULT | PROCARRAY_VACUUM_FLAG -/* Ignore analyze backends */ -#define PROCARRAY_FLAGS_ANALYZE PROCARRAY_FLAGS_DEFAULT | PROCARRAY_ANALYZE_FLAG -/* Ignore both vacuum and analyze backends */ -#define PROCARRAY_FLAGS_VACUUM_ANALYZE PROCARRAY_FLAGS_DEFAULT | PROCARRAY_VACUUM_FLAG | PROCARRAY_ANALYZE_FLAG - extern Size ProcArrayShmemSize(void); extern void CreateSharedProcArray(void); extern void ProcArrayAdd(PGPROC *proc); @@ -88,9 +53,11 @@ extern RunningTransactions GetRunningTransactionData(void); extern bool TransactionIdIsInProgress(TransactionId xid); extern bool TransactionIdIsActive(TransactionId xid); -extern TransactionId GetOldestXmin(Relation rel, int flags); +extern TransactionId GetOldestNonRemovableTransactionId(Relation rel); +extern TransactionId GetOldestTransactionIdConsideredRunning(void); extern TransactionId GetOldestActiveTransactionId(void); extern TransactionId GetOldestSafeDecodingTransactionId(bool catalogOnly); +extern void GetReplicationHorizons(TransactionId *slot_xmin, TransactionId *catalog_xmin); extern VirtualTransactionId *GetVirtualXIDsDelayingChkpt(int *nvxids); extern bool HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids); diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h index ffb4ba3adfb..b6b403e2931 100644 --- a/src/include/utils/snapmgr.h +++ b/src/include/utils/snapmgr.h @@ -52,13 +52,12 @@ extern Size SnapMgrShmemSize(void); extern void SnapMgrInit(void); extern TimestampTz GetSnapshotCurrentTimestamp(void); extern TimestampTz GetOldSnapshotThresholdTimestamp(void); +extern void SnapshotTooOldMagicForTest(void); extern bool FirstSnapshotSet; extern PGDLLIMPORT TransactionId TransactionXmin; extern PGDLLIMPORT TransactionId RecentXmin; -extern PGDLLIMPORT TransactionId RecentGlobalXmin; -extern PGDLLIMPORT TransactionId RecentGlobalDataXmin; /* Variables representing various special snapshot semantics */ extern PGDLLIMPORT SnapshotData SnapshotSelfData; @@ -78,11 +77,12 @@ extern PGDLLIMPORT SnapshotData CatalogSnapshotData; /* * Similarly, some initialization is required for a NonVacuumable snapshot. - * The caller must supply the xmin horizon to use (e.g., RecentGlobalXmin). + * The caller must supply the visibility cutoff state to use (c.f. + * GlobalVisTestFor()). */ -#define InitNonVacuumableSnapshot(snapshotdata, xmin_horizon) \ +#define InitNonVacuumableSnapshot(snapshotdata, vistestp) \ ((snapshotdata).snapshot_type = SNAPSHOT_NON_VACUUMABLE, \ - (snapshotdata).xmin = (xmin_horizon)) + (snapshotdata).vistest = (vistestp)) /* * Similarly, some initialization is required for SnapshotToast. We need @@ -98,6 +98,11 @@ extern PGDLLIMPORT SnapshotData CatalogSnapshotData; ((snapshot)->snapshot_type == SNAPSHOT_MVCC || \ (snapshot)->snapshot_type == SNAPSHOT_HISTORIC_MVCC) +static inline bool +OldSnapshotThresholdActive(void) +{ + return old_snapshot_threshold >= 0; +} extern Snapshot GetTransactionSnapshot(void); extern Snapshot GetLatestSnapshot(void); @@ -121,8 +126,6 @@ extern void UnregisterSnapshot(Snapshot snapshot); extern Snapshot RegisterSnapshotOnOwner(Snapshot snapshot, ResourceOwner owner); extern void UnregisterSnapshotFromOwner(Snapshot snapshot, ResourceOwner owner); -extern FullTransactionId GetFullRecentGlobalXmin(void); - extern void AtSubCommit_Snapshot(int level); extern void AtSubAbort_Snapshot(int level); extern void AtEOXact_Snapshot(bool isCommit, bool resetXmin); @@ -131,13 +134,29 @@ extern void ImportSnapshot(const char *idstr); extern bool XactHasExportedSnapshots(void); extern void DeleteAllExportedSnapshotFiles(void); extern bool ThereAreNoPriorRegisteredSnapshots(void); -extern TransactionId TransactionIdLimitedForOldSnapshots(TransactionId recentXmin, - Relation relation); +extern bool TransactionIdLimitedForOldSnapshots(TransactionId recentXmin, + Relation relation, + TransactionId *limit_xid, + TimestampTz *limit_ts); +extern void SetOldSnapshotThresholdTimestamp(TimestampTz ts, TransactionId xlimit); extern void MaintainOldSnapshotTimeMapping(TimestampTz whenTaken, TransactionId xmin); extern char *ExportSnapshot(Snapshot snapshot); +/* + * These live in procarray.c because they're intimately linked to the + * procarray contents, but thematically they better fit into snapmgr.h. + */ +typedef struct GlobalVisState GlobalVisState; +extern GlobalVisState *GlobalVisTestFor(Relation rel); +extern bool GlobalVisTestIsRemovableXid(GlobalVisState *state, TransactionId xid); +extern bool GlobalVisTestIsRemovableFullXid(GlobalVisState *state, FullTransactionId fxid); +extern FullTransactionId GlobalVisTestNonRemovableFullHorizon(GlobalVisState *state); +extern TransactionId GlobalVisTestNonRemovableHorizon(GlobalVisState *state); +extern bool GlobalVisCheckRemovableXid(Relation rel, TransactionId xid); +extern bool GlobalVisIsRemovableFullXid(Relation rel, FullTransactionId fxid); + /* * Utility functions for implementing visibility routines in table AMs. */ diff --git a/src/include/utils/snapshot.h b/src/include/utils/snapshot.h index 4796edb63aa..35b1f05bea6 100644 --- a/src/include/utils/snapshot.h +++ b/src/include/utils/snapshot.h @@ -192,6 +192,12 @@ typedef struct SnapshotData */ uint32 speculativeToken; + /* + * For SNAPSHOT_NON_VACUUMABLE (and hopefully more in the future) this is + * used to determine whether row could be vacuumed. + */ + struct GlobalVisState *vistest; + /* * Book-keeping information, used by the snapshot manager */ diff --git a/src/backend/access/gin/ginvacuum.c b/src/backend/access/gin/ginvacuum.c index 8ae4fd95a7b..9cd6638df62 100644 --- a/src/backend/access/gin/ginvacuum.c +++ b/src/backend/access/gin/ginvacuum.c @@ -793,3 +793,29 @@ ginvacuumcleanup(IndexVacuumInfo *info, IndexBulkDeleteResult *stats) return stats; } + +/* + * Return whether Page can safely be recycled. + */ +bool +GinPageIsRecyclable(Page page) +{ + TransactionId delete_xid; + + if (PageIsNew(page)) + return true; + + if (!GinPageIsDeleted(page)) + return false; + + delete_xid = GinPageGetDeleteXid(page); + + if (!TransactionIdIsValid(delete_xid)) + return true; + + /* + * If no backend still could view delete_xid as in running, all scans + * concurrent with ginDeletePage() must have finished. + */ + return GlobalVisCheckRemovableXid(NULL, delete_xid); +} diff --git a/src/backend/access/gist/gistutil.c b/src/backend/access/gist/gistutil.c index 765329bbcd4..bfda7fbe3d5 100644 --- a/src/backend/access/gist/gistutil.c +++ b/src/backend/access/gist/gistutil.c @@ -891,15 +891,13 @@ gistPageRecyclable(Page page) * As long as that can happen, we must keep the deleted page around as * a tombstone. * - * Compare the deletion XID with RecentGlobalXmin. If deleteXid < - * RecentGlobalXmin, then no scan that's still in progress could have + * For that check if the deletion XID could still be visible to + * anyone. If not, then no scan that's still in progress could have * seen its downlink, and we can recycle it. */ FullTransactionId deletexid_full = GistPageGetDeleteXid(page); - FullTransactionId recentxmin_full = GetFullRecentGlobalXmin(); - if (FullTransactionIdPrecedes(deletexid_full, recentxmin_full)) - return true; + return GlobalVisIsRemovableFullXid(NULL, deletexid_full); } return false; } diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index 3f0effd5e42..3167305ac00 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -387,11 +387,11 @@ gistRedoPageReuse(XLogReaderState *record) * PAGE_REUSE records exist to provide a conflict point when we reuse * pages in the index via the FSM. That's all they do though. * - * latestRemovedXid was the page's deleteXid. The deleteXid < - * RecentGlobalXmin test in gistPageRecyclable() conceptually mirrors the - * pgxact->xmin > limitXmin test in GetConflictingVirtualXIDs(). - * Consequently, one XID value achieves the same exclusion effect on - * primary and standby. + * latestRemovedXid was the page's deleteXid. The + * GlobalVisIsRemovableFullXid(deleteXid) test in gistPageRecyclable() + * conceptually mirrors the pgxact->xmin > limitXmin test in + * GetConflictingVirtualXIDs(). Consequently, one XID value achieves the + * same exclusion effect on primary and standby. */ if (InHotStandby) { diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c index d881f4cd46a..a8804351bee 100644 --- a/src/backend/access/heap/heapam.c +++ b/src/backend/access/heap/heapam.c @@ -1496,6 +1496,7 @@ heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer, bool at_chain_start; bool valid; bool skip; + GlobalVisState *vistest = NULL; /* If this is not the first call, previous call returned a (live!) tuple */ if (all_dead) @@ -1506,7 +1507,8 @@ heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer, at_chain_start = first_call; skip = !first_call; - Assert(TransactionIdIsValid(RecentGlobalXmin)); + /* XXX: we should assert that a snapshot is pushed or registered */ + Assert(TransactionIdIsValid(RecentXmin)); Assert(BufferGetBlockNumber(buffer) == blkno); /* Scan through possible multiple members of HOT-chain */ @@ -1595,9 +1597,14 @@ heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer, * Note: if you change the criterion here for what is "dead", fix the * planner's get_actual_variable_range() function to match. */ - if (all_dead && *all_dead && - !HeapTupleIsSurelyDead(heapTuple, RecentGlobalXmin)) - *all_dead = false; + if (all_dead && *all_dead) + { + if (!vistest) + vistest = GlobalVisTestFor(relation); + + if (!HeapTupleIsSurelyDead(vistest, heapTuple)) + *all_dead = false; + } /* * Check to see if HOT chain continues past this tuple; if so fetch diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c index 56b35622f1a..659fc4d8697 100644 --- a/src/backend/access/heap/heapam_handler.c +++ b/src/backend/access/heap/heapam_handler.c @@ -1201,7 +1201,7 @@ heapam_index_build_range_scan(Relation heapRelation, /* okay to ignore lazy VACUUMs here */ if (!IsBootstrapProcessingMode() && !indexInfo->ii_Concurrent) - OldestXmin = GetOldestXmin(heapRelation, PROCARRAY_FLAGS_VACUUM); + OldestXmin = GetOldestNonRemovableTransactionId(heapRelation); if (!scan) { @@ -1242,6 +1242,17 @@ heapam_index_build_range_scan(Relation heapRelation, hscan = (HeapScanDesc) scan; + /* + * Must have called GetOldestNonRemovableTransactionId() if using + * SnapshotAny. Shouldn't have for an MVCC snapshot. (It's especially + * worth checking this for parallel builds, since ambuild routines that + * support parallel builds must work these details out for themselves.) + */ + Assert(snapshot == SnapshotAny || IsMVCCSnapshot(snapshot)); + Assert(snapshot == SnapshotAny ? TransactionIdIsValid(OldestXmin) : + !TransactionIdIsValid(OldestXmin)); + Assert(snapshot == SnapshotAny || !anyvisible); + /* Publish number of blocks to scan */ if (progress) { @@ -1261,17 +1272,6 @@ heapam_index_build_range_scan(Relation heapRelation, nblocks); } - /* - * Must call GetOldestXmin() with SnapshotAny. Should never call - * GetOldestXmin() with MVCC snapshot. (It's especially worth checking - * this for parallel builds, since ambuild routines that support parallel - * builds must work these details out for themselves.) - */ - Assert(snapshot == SnapshotAny || IsMVCCSnapshot(snapshot)); - Assert(snapshot == SnapshotAny ? TransactionIdIsValid(OldestXmin) : - !TransactionIdIsValid(OldestXmin)); - Assert(snapshot == SnapshotAny || !anyvisible); - /* set our scan endpoints */ if (!allow_sync) heap_setscanlimits(scan, start_blockno, numblocks); diff --git a/src/backend/access/heap/heapam_visibility.c b/src/backend/access/heap/heapam_visibility.c index dba10890aab..b25b3e429ed 100644 --- a/src/backend/access/heap/heapam_visibility.c +++ b/src/backend/access/heap/heapam_visibility.c @@ -1154,19 +1154,56 @@ HeapTupleSatisfiesMVCC(HeapTuple htup, Snapshot snapshot, * we mainly want to know is if a tuple is potentially visible to *any* * running transaction. If so, it can't be removed yet by VACUUM. * - * OldestXmin is a cutoff XID (obtained from GetOldestXmin()). Tuples - * deleted by XIDs >= OldestXmin are deemed "recently dead"; they might - * still be visible to some open transaction, so we can't remove them, - * even if we see that the deleting transaction has committed. + * OldestXmin is a cutoff XID (obtained from + * GetOldestNonRemovableTransactionId()). Tuples deleted by XIDs >= + * OldestXmin are deemed "recently dead"; they might still be visible to some + * open transaction, so we can't remove them, even if we see that the deleting + * transaction has committed. */ HTSV_Result HeapTupleSatisfiesVacuum(HeapTuple htup, TransactionId OldestXmin, Buffer buffer) +{ + TransactionId dead_after = InvalidTransactionId; + HTSV_Result res; + + res = HeapTupleSatisfiesVacuumHorizon(htup, buffer, &dead_after); + + if (res == HEAPTUPLE_RECENTLY_DEAD) + { + Assert(TransactionIdIsValid(dead_after)); + + if (TransactionIdPrecedes(dead_after, OldestXmin)) + res = HEAPTUPLE_DEAD; + } + else + Assert(!TransactionIdIsValid(dead_after)); + + return res; +} + +/* + * Work horse for HeapTupleSatisfiesVacuum and similar routines. + * + * In contrast to HeapTupleSatisfiesVacuum this routine, when encountering a + * tuple that could still be visible to some backend, stores the xid that + * needs to be compared with the horizon in *dead_after, and returns + * HEAPTUPLE_RECENTLY_DEAD. The caller then can perform the comparison with + * the horizon. This is e.g. useful when comparing with different horizons. + * + * Note: HEAPTUPLE_DEAD can still be returned here, e.g. if the inserting + * transaction aborted. + */ +HTSV_Result +HeapTupleSatisfiesVacuumHorizon(HeapTuple htup, Buffer buffer, TransactionId *dead_after) { HeapTupleHeader tuple = htup->t_data; Assert(ItemPointerIsValid(&htup->t_self)); Assert(htup->t_tableOid != InvalidOid); + Assert(dead_after != NULL); + + *dead_after = InvalidTransactionId; /* * Has inserting transaction committed? @@ -1323,17 +1360,15 @@ HeapTupleSatisfiesVacuum(HeapTuple htup, TransactionId OldestXmin, else if (TransactionIdDidCommit(xmax)) { /* - * The multixact might still be running due to lockers. If the - * updater is below the xid horizon, we have to return DEAD - * regardless -- otherwise we could end up with a tuple where the - * updater has to be removed due to the horizon, but is not pruned - * away. It's not a problem to prune that tuple, because any - * remaining lockers will also be present in newer tuple versions. + * The multixact might still be running due to lockers. Need to + * allow for pruning if below the xid horizon regardless -- + * otherwise we could end up with a tuple where the updater has to + * be removed due to the horizon, but is not pruned away. It's + * not a problem to prune that tuple, because any remaining + * lockers will also be present in newer tuple versions. */ - if (!TransactionIdPrecedes(xmax, OldestXmin)) - return HEAPTUPLE_RECENTLY_DEAD; - - return HEAPTUPLE_DEAD; + *dead_after = xmax; + return HEAPTUPLE_RECENTLY_DEAD; } else if (!MultiXactIdIsRunning(HeapTupleHeaderGetRawXmax(tuple), false)) { @@ -1372,14 +1407,11 @@ HeapTupleSatisfiesVacuum(HeapTuple htup, TransactionId OldestXmin, } /* - * Deleter committed, but perhaps it was recent enough that some open - * transactions could still see the tuple. + * Deleter committed, allow caller to check if it was recent enough that + * some open transactions could still see the tuple. */ - if (!TransactionIdPrecedes(HeapTupleHeaderGetRawXmax(tuple), OldestXmin)) - return HEAPTUPLE_RECENTLY_DEAD; - - /* Otherwise, it's dead and removable */ - return HEAPTUPLE_DEAD; + *dead_after = HeapTupleHeaderGetRawXmax(tuple); + return HEAPTUPLE_RECENTLY_DEAD; } @@ -1418,7 +1450,7 @@ HeapTupleSatisfiesNonVacuumable(HeapTuple htup, Snapshot snapshot, * if the tuple is removable. */ bool -HeapTupleIsSurelyDead(HeapTuple htup, TransactionId OldestXmin) +HeapTupleIsSurelyDead(GlobalVisState *vistest, HeapTuple htup) { HeapTupleHeader tuple = htup->t_data; @@ -1459,7 +1491,8 @@ HeapTupleIsSurelyDead(HeapTuple htup, TransactionId OldestXmin) return false; /* Deleter committed, so tuple is dead if the XID is old enough. */ - return TransactionIdPrecedes(HeapTupleHeaderGetRawXmax(tuple), OldestXmin); + return GlobalVisTestIsRemovableXid(vistest, + HeapTupleHeaderGetRawXmax(tuple)); } /* diff --git a/src/backend/access/heap/pruneheap.c b/src/backend/access/heap/pruneheap.c index 256df4de105..00a3cb106aa 100644 --- a/src/backend/access/heap/pruneheap.c +++ b/src/backend/access/heap/pruneheap.c @@ -23,12 +23,30 @@ #include "miscadmin.h" #include "pgstat.h" #include "storage/bufmgr.h" +#include "utils/snapmgr.h" #include "utils/rel.h" #include "utils/snapmgr.h" /* Working data for heap_page_prune and subroutines */ typedef struct { + Relation rel; + + /* tuple visibility test, initialized for the relation */ + GlobalVisState *vistest; + + /* + * Thresholds set by TransactionIdLimitedForOldSnapshots() if they have + * been computed (done on demand, and only if + * OldSnapshotThresholdActive()). The first time a tuple is about to be + * removed based on the limited horizon, old_snap_used is set to true, and + * SetOldSnapshotThresholdTimestamp() is called. See + * heap_prune_satisfies_vacuum(). + */ + TimestampTz old_snap_ts; + TransactionId old_snap_xmin; + bool old_snap_used; + TransactionId new_prune_xid; /* new prune hint value for page */ TransactionId latestRemovedXid; /* latest xid to be removed by this prune */ int nredirected; /* numbers of entries in arrays below */ @@ -43,9 +61,8 @@ typedef struct } PruneState; /* Local functions */ -static int heap_prune_chain(Relation relation, Buffer buffer, +static int heap_prune_chain(Buffer buffer, OffsetNumber rootoffnum, - TransactionId OldestXmin, PruneState *prstate); static void heap_prune_record_prunable(PruneState *prstate, TransactionId xid); static void heap_prune_record_redirect(PruneState *prstate, @@ -65,16 +82,16 @@ static void heap_prune_record_unused(PruneState *prstate, OffsetNumber offnum); * if there's not any use in pruning. * * Caller must have pin on the buffer, and must *not* have a lock on it. - * - * OldestXmin is the cutoff XID used to distinguish whether tuples are DEAD - * or RECENTLY_DEAD (see HeapTupleSatisfiesVacuum). */ void heap_page_prune_opt(Relation relation, Buffer buffer) { Page page = BufferGetPage(buffer); + TransactionId prune_xid; + GlobalVisState *vistest; + TransactionId limited_xmin = InvalidTransactionId; + TimestampTz limited_ts = 0; Size minfree; - TransactionId OldestXmin; /* * We can't write WAL in recovery mode, so there's no point trying to @@ -85,37 +102,55 @@ heap_page_prune_opt(Relation relation, Buffer buffer) return; /* - * Use the appropriate xmin horizon for this relation. If it's a proper - * catalog relation or a user defined, additional, catalog relation, we - * need to use the horizon that includes slots, otherwise the data-only - * horizon can be used. Note that the toast relation of user defined - * relations are *not* considered catalog relations. + * XXX: Magic to keep old_snapshot_threshold tests appear "working". They + * currently are broken, and discussion of what to do about them is + * ongoing. See + * https://www.postgresql.org/message-id/20200403001235.e6jfdll3gh2ygbuc%40alap3.anarazel.de + */ + if (old_snapshot_threshold == 0) + SnapshotTooOldMagicForTest(); + + /* + * First check whether there's any chance there's something to prune, + * determining the appropriate horizon is a waste if there's no prune_xid + * (i.e. no updates/deletes left potentially dead tuples around). + */ + prune_xid = ((PageHeader) page)->pd_prune_xid; + if (!TransactionIdIsValid(prune_xid)) + return; + + /* + * Check whether prune_xid indicates that there may be dead rows that can + * be cleaned up. * - * It is OK to apply the old snapshot limit before acquiring the cleanup + * It is OK to check the old snapshot limit before acquiring the cleanup * lock because the worst that can happen is that we are not quite as * aggressive about the cleanup (by however many transaction IDs are * consumed between this point and acquiring the lock). This allows us to * save significant overhead in the case where the page is found not to be * prunable. - */ - if (IsCatalogRelation(relation) || - RelationIsAccessibleInLogicalDecoding(relation)) - OldestXmin = RecentGlobalXmin; - else - OldestXmin = - TransactionIdLimitedForOldSnapshots(RecentGlobalDataXmin, - relation); - - Assert(TransactionIdIsValid(OldestXmin)); - - /* - * Let's see if we really need pruning. * - * Forget it if page is not hinted to contain something prunable that's - * older than OldestXmin. + * Even if old_snapshot_threshold is set, we first check whether the page + * can be pruned without. Both because + * TransactionIdLimitedForOldSnapshots() is not cheap, and because not + * unnecessarily relying on old_snapshot_threshold avoids causing + * conflicts. */ - if (!PageIsPrunable(page, OldestXmin)) - return; + vistest = GlobalVisTestFor(relation); + + if (!GlobalVisTestIsRemovableXid(vistest, prune_xid)) + { + if (!OldSnapshotThresholdActive()) + return; + + if (!TransactionIdLimitedForOldSnapshots(GlobalVisTestNonRemovableHorizon(vistest), + relation, + &limited_xmin, &limited_ts)) + return; + + if (!TransactionIdPrecedes(prune_xid, limited_xmin)) + return; + } /* * We prune when a previous UPDATE failed to find enough space on the page @@ -151,7 +186,9 @@ heap_page_prune_opt(Relation relation, Buffer buffer) * needed */ /* OK to prune */ - (void) heap_page_prune(relation, buffer, OldestXmin, true, &ignore); + (void) heap_page_prune(relation, buffer, vistest, + limited_xmin, limited_ts, + true, &ignore); } /* And release buffer lock */ @@ -165,8 +202,11 @@ heap_page_prune_opt(Relation relation, Buffer buffer) * * Caller must have pin and buffer cleanup lock on the page. * - * OldestXmin is the cutoff XID used to distinguish whether tuples are DEAD - * or RECENTLY_DEAD (see HeapTupleSatisfiesVacuum). + * vistest is used to distinguish whether tuples are DEAD or RECENTLY_DEAD + * (see heap_prune_satisfies_vacuum and + * HeapTupleSatisfiesVacuum). old_snap_xmin / old_snap_ts need to + * either have been set by TransactionIdLimitedForOldSnapshots, or + * InvalidTransactionId/0 respectively. * * If report_stats is true then we send the number of reclaimed heap-only * tuples to pgstats. (This must be false during vacuum, since vacuum will @@ -177,7 +217,10 @@ heap_page_prune_opt(Relation relation, Buffer buffer) * latestRemovedXid. */ int -heap_page_prune(Relation relation, Buffer buffer, TransactionId OldestXmin, +heap_page_prune(Relation relation, Buffer buffer, + GlobalVisState *vistest, + TransactionId old_snap_xmin, + TimestampTz old_snap_ts, bool report_stats, TransactionId *latestRemovedXid) { int ndeleted = 0; @@ -198,6 +241,11 @@ heap_page_prune(Relation relation, Buffer buffer, TransactionId OldestXmin, * initialize the rest of our working state. */ prstate.new_prune_xid = InvalidTransactionId; + prstate.rel = relation; + prstate.vistest = vistest; + prstate.old_snap_xmin = old_snap_xmin; + prstate.old_snap_ts = old_snap_ts; + prstate.old_snap_used = false; prstate.latestRemovedXid = *latestRemovedXid; prstate.nredirected = prstate.ndead = prstate.nunused = 0; memset(prstate.marked, 0, sizeof(prstate.marked)); @@ -220,9 +268,7 @@ heap_page_prune(Relation relation, Buffer buffer, TransactionId OldestXmin, continue; /* Process this item or chain of items */ - ndeleted += heap_prune_chain(relation, buffer, offnum, - OldestXmin, - &prstate); + ndeleted += heap_prune_chain(buffer, offnum, &prstate); } /* Any error while applying the changes is critical */ @@ -323,6 +369,85 @@ heap_page_prune(Relation relation, Buffer buffer, TransactionId OldestXmin, } +/* + * Perform visiblity checks for heap pruning. + * + * This is more complicated than just using GlobalVisTestIsRemovableXid() + * because of old_snapshot_threshold. We only want to increase the threshold + * that triggers errors for old snapshots when we actually decide to remove a + * row based on the limited horizon. + * + * Due to its cost we also only want to call + * TransactionIdLimitedForOldSnapshots() if necessary, i.e. we might not have + * done so in heap_hot_prune_opt() if pd_prune_xid was old enough. But we + * still want to be able to remove rows that are too new to be removed + * according to prstate->vistest, but that can be removed based on + * old_snapshot_threshold. So we call TransactionIdLimitedForOldSnapshots() on + * demand in here, if appropriate. + */ +static HTSV_Result +heap_prune_satisfies_vacuum(PruneState *prstate, HeapTuple tup, Buffer buffer) +{ + HTSV_Result res; + TransactionId dead_after; + + res = HeapTupleSatisfiesVacuumHorizon(tup, buffer, &dead_after); + + if (res != HEAPTUPLE_RECENTLY_DEAD) + return res; + + /* + * If we are already relying on the limited xmin, there is no need to + * delay doing so anymore. + */ + if (prstate->old_snap_used) + { + Assert(TransactionIdIsValid(prstate->old_snap_xmin)); + + if (TransactionIdPrecedes(dead_after, prstate->old_snap_xmin)) + res = HEAPTUPLE_DEAD; + return res; + } + + /* + * First check if GlobalVisTestIsRemovableXid() is sufficient to find the + * row dead. If not, and old_snapshot_threshold is enabled, try to use the + * lowered horizon. + */ + if (GlobalVisTestIsRemovableXid(prstate->vistest, dead_after)) + res = HEAPTUPLE_DEAD; + else if (OldSnapshotThresholdActive()) + { + /* haven't determined limited horizon yet, requests */ + if (!TransactionIdIsValid(prstate->old_snap_xmin)) + { + TransactionId horizon = + GlobalVisTestNonRemovableHorizon(prstate->vistest); + + TransactionIdLimitedForOldSnapshots(horizon, prstate->rel, + &prstate->old_snap_xmin, + &prstate->old_snap_ts); + } + + if (TransactionIdIsValid(prstate->old_snap_xmin) && + TransactionIdPrecedes(dead_after, prstate->old_snap_xmin)) + { + /* + * About to remove row based on snapshot_too_old. Need to raise + * the threshold so problematic accesses would error. + */ + Assert(!prstate->old_snap_used); + SetOldSnapshotThresholdTimestamp(prstate->old_snap_ts, + prstate->old_snap_xmin); + prstate->old_snap_used = true; + res = HEAPTUPLE_DEAD; + } + } + + return res; +} + + /* * Prune specified line pointer or a HOT chain originating at line pointer. * @@ -349,9 +474,7 @@ heap_page_prune(Relation relation, Buffer buffer, TransactionId OldestXmin, * Returns the number of tuples (to be) deleted from the page. */ static int -heap_prune_chain(Relation relation, Buffer buffer, OffsetNumber rootoffnum, - TransactionId OldestXmin, - PruneState *prstate) +heap_prune_chain(Buffer buffer, OffsetNumber rootoffnum, PruneState *prstate) { int ndeleted = 0; Page dp = (Page) BufferGetPage(buffer); @@ -366,7 +489,7 @@ heap_prune_chain(Relation relation, Buffer buffer, OffsetNumber rootoffnum, i; HeapTupleData tup; - tup.t_tableOid = RelationGetRelid(relation); + tup.t_tableOid = RelationGetRelid(prstate->rel); rootlp = PageGetItemId(dp, rootoffnum); @@ -401,7 +524,7 @@ heap_prune_chain(Relation relation, Buffer buffer, OffsetNumber rootoffnum, * either here or while following a chain below. Whichever path * gets there first will mark the tuple unused. */ - if (HeapTupleSatisfiesVacuum(&tup, OldestXmin, buffer) + if (heap_prune_satisfies_vacuum(prstate, &tup, buffer) == HEAPTUPLE_DEAD && !HeapTupleHeaderIsHotUpdated(htup)) { heap_prune_record_unused(prstate, rootoffnum); @@ -485,7 +608,7 @@ heap_prune_chain(Relation relation, Buffer buffer, OffsetNumber rootoffnum, */ tupdead = recent_dead = false; - switch (HeapTupleSatisfiesVacuum(&tup, OldestXmin, buffer)) + switch (heap_prune_satisfies_vacuum(prstate, &tup, buffer)) { case HEAPTUPLE_DEAD: tupdead = true; diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c index 1bbc4598f75..44e2224dd55 100644 --- a/src/backend/access/heap/vacuumlazy.c +++ b/src/backend/access/heap/vacuumlazy.c @@ -788,6 +788,7 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats, PROGRESS_VACUUM_MAX_DEAD_TUPLES }; int64 initprog_val[3]; + GlobalVisState *vistest; pg_rusage_init(&ru0); @@ -816,6 +817,8 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats, vacrelstats->nonempty_pages = 0; vacrelstats->latestRemovedXid = InvalidTransactionId; + vistest = GlobalVisTestFor(onerel); + /* * Initialize state for a parallel vacuum. As of now, only one worker can * be used for an index, so we invoke parallelism only if there are at @@ -1239,7 +1242,8 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats, * * We count tuples removed by the pruning step as removed by VACUUM. */ - tups_vacuumed += heap_page_prune(onerel, buf, OldestXmin, false, + tups_vacuumed += heap_page_prune(onerel, buf, vistest, false, + InvalidTransactionId, 0, &vacrelstats->latestRemovedXid); /* @@ -1596,14 +1600,16 @@ lazy_scan_heap(Relation onerel, VacuumParams *params, LVRelStats *vacrelstats, } /* - * It's possible for the value returned by GetOldestXmin() to move - * backwards, so it's not wrong for us to see tuples that appear to - * not be visible to everyone yet, while PD_ALL_VISIBLE is already - * set. The real safe xmin value never moves backwards, but - * GetOldestXmin() is conservative and sometimes returns a value - * that's unnecessarily small, so if we see that contradiction it just - * means that the tuples that we think are not visible to everyone yet - * actually are, and the PD_ALL_VISIBLE flag is correct. + * It's possible for the value returned by + * GetOldestNonRemovableTransactionId() to move backwards, so it's not + * wrong for us to see tuples that appear to not be visible to + * everyone yet, while PD_ALL_VISIBLE is already set. The real safe + * xmin value never moves backwards, but + * GetOldestNonRemovableTransactionId() is conservative and sometimes + * returns a value that's unnecessarily small, so if we see that + * contradiction it just means that the tuples that we think are not + * visible to everyone yet actually are, and the PD_ALL_VISIBLE flag + * is correct. * * There should never be dead tuples on a page with PD_ALL_VISIBLE * set, however. diff --git a/src/backend/access/index/indexam.c b/src/backend/access/index/indexam.c index 6b9750c244a..3fb8688f8f4 100644 --- a/src/backend/access/index/indexam.c +++ b/src/backend/access/index/indexam.c @@ -519,7 +519,8 @@ index_getnext_tid(IndexScanDesc scan, ScanDirection direction) SCAN_CHECKS; CHECK_SCAN_PROCEDURE(amgettuple); - Assert(TransactionIdIsValid(RecentGlobalXmin)); + /* XXX: we should assert that a snapshot is pushed or registered */ + Assert(TransactionIdIsValid(RecentXmin)); /* * The AM's amgettuple proc finds the next index entry matching the scan diff --git a/src/backend/access/nbtree/README b/src/backend/access/nbtree/README index 32ad9e339a2..cf3dba96008 100644 --- a/src/backend/access/nbtree/README +++ b/src/backend/access/nbtree/README @@ -342,9 +342,9 @@ snapshots and registered snapshots as of the deletion are gone; which is overly strong, but is simple to implement within Postgres. When marked dead, a deleted page is labeled with the next-transaction counter value. VACUUM can reclaim the page for re-use when this transaction number is -older than RecentGlobalXmin. As collateral damage, this implementation -also waits for running XIDs with no snapshots and for snapshots taken -until the next transaction to allocate an XID commits. +guaranteed to be "visible to everyone". As collateral damage, this +implementation also waits for running XIDs with no snapshots and for +snapshots taken until the next transaction to allocate an XID commits. Reclaiming a page doesn't actually change its state on disk --- we simply record it in the shared-memory free space map, from which it will be @@ -411,8 +411,8 @@ page and also the correct place to hold the current value. We can avoid the cost of walking down the tree in such common cases. The optimization works on the assumption that there can only be one -non-ignorable leaf rightmost page, and so even a RecentGlobalXmin style -interlock isn't required. We cannot fail to detect that our hint was +non-ignorable leaf rightmost page, and so not even a visible-to-everyone +style interlock required. We cannot fail to detect that our hint was invalidated, because there can only be one such page in the B-Tree at any time. It's possible that the page will be deleted and recycled without a backend's cached page also being detected as invalidated, but diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index 75628e0eb98..9e6376f2c2b 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -990,7 +990,7 @@ _bt_page_recyclable(Page page) */ opaque = (BTPageOpaque) PageGetSpecialPointer(page); if (P_ISDELETED(opaque) && - TransactionIdPrecedes(opaque->btpo.xact, RecentGlobalXmin)) + GlobalVisCheckRemovableXid(NULL, opaque->btpo.xact)) return true; return false; } @@ -2209,7 +2209,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * updated links to the target, ReadNewTransactionId() suffices as an * upper bound. Any scan having retained a now-stale link is advertising * in its PGXACT an xmin less than or equal to the value we read here. It - * will continue to do so, holding back RecentGlobalXmin, for the duration + * will continue to do so, holding back the xmin horizon, for the duration * of that scan. */ page = BufferGetPage(buf); diff --git a/src/backend/access/nbtree/nbtree.c b/src/backend/access/nbtree/nbtree.c index e947addef6b..b59ba02a32f 100644 --- a/src/backend/access/nbtree/nbtree.c +++ b/src/backend/access/nbtree/nbtree.c @@ -807,6 +807,12 @@ _bt_vacuum_needs_cleanup(IndexVacuumInfo *info) metapg = BufferGetPage(metabuf); metad = BTPageGetMeta(metapg); + /* + * XXX: If IndexVacuumInfo contained the heap relation, we could be more + * aggressive about vacuuming non catalog relations by passing the table + * to GlobalVisCheckRemovableXid(). + */ + if (metad->btm_version < BTREE_NOVAC_VERSION) { /* @@ -816,13 +822,12 @@ _bt_vacuum_needs_cleanup(IndexVacuumInfo *info) result = true; } else if (TransactionIdIsValid(metad->btm_oldest_btpo_xact) && - TransactionIdPrecedes(metad->btm_oldest_btpo_xact, - RecentGlobalXmin)) + GlobalVisCheckRemovableXid(NULL, metad->btm_oldest_btpo_xact)) { /* * If any oldest btpo.xact from a previously deleted page in the index - * is older than RecentGlobalXmin, then at least one deleted page can - * be recycled -- don't skip cleanup. + * is visible to everyone, then at least one deleted page can be + * recycled -- don't skip cleanup. */ result = true; } @@ -1276,14 +1281,13 @@ backtrack: * own conflict now.) * * Backends with snapshots acquired after a VACUUM starts but - * before it finishes could have a RecentGlobalXmin with a - * later xid than the VACUUM's OldestXmin cutoff. These - * backends might happen to opportunistically mark some index - * tuples LP_DEAD before we reach them, even though they may - * be after our cutoff. We don't try to kill these "extra" - * index tuples in _bt_delitems_vacuum(). This keep things - * simple, and allows us to always avoid generating our own - * conflicts. + * before it finishes could have visibility cutoff with a + * later xid than VACUUM's OldestXmin cutoff. These backends + * might happen to opportunistically mark some index tuples + * LP_DEAD before we reach them, even though they may be after + * our cutoff. We don't try to kill these "extra" index + * tuples in _bt_delitems_vacuum(). This keep things simple, + * and allows us to always avoid generating our own conflicts. */ Assert(!BTreeTupleIsPivot(itup)); if (!BTreeTupleIsPosting(itup)) diff --git a/src/backend/access/nbtree/nbtxlog.c b/src/backend/access/nbtree/nbtxlog.c index 5d346da84fd..b097e98c3ba 100644 --- a/src/backend/access/nbtree/nbtxlog.c +++ b/src/backend/access/nbtree/nbtxlog.c @@ -928,11 +928,11 @@ btree_xlog_reuse_page(XLogReaderState *record) * Btree reuse_page records exist to provide a conflict point when we * reuse pages in the index via the FSM. That's all they do though. * - * latestRemovedXid was the page's btpo.xact. The btpo.xact < - * RecentGlobalXmin test in _bt_page_recyclable() conceptually mirrors the - * pgxact->xmin > limitXmin test in GetConflictingVirtualXIDs(). - * Consequently, one XID value achieves the same exclusion effect on - * primary and standby. + * latestRemovedXid was the page's btpo.xact. The + * GlobalVisCheckRemovableXid test in _bt_page_recyclable() conceptually + * mirrors the pgxact->xmin > limitXmin test in + * GetConflictingVirtualXIDs(). Consequently, one XID value achieves the + * same exclusion effect on primary and standby. */ if (InHotStandby) { diff --git a/src/backend/access/spgist/spgvacuum.c b/src/backend/access/spgist/spgvacuum.c index bd98707f3c0..e1c58933f97 100644 --- a/src/backend/access/spgist/spgvacuum.c +++ b/src/backend/access/spgist/spgvacuum.c @@ -501,10 +501,14 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) OffsetNumber itemToPlaceholder[MaxIndexTuplesPerPage]; OffsetNumber itemnos[MaxIndexTuplesPerPage]; spgxlogVacuumRedirect xlrec; + GlobalVisState *vistest; xlrec.nToPlaceholder = 0; xlrec.newestRedirectXid = InvalidTransactionId; + /* XXX: providing heap relation would allow more pruning */ + vistest = GlobalVisTestFor(NULL); + START_CRIT_SECTION(); /* @@ -521,7 +525,7 @@ vacuumRedirectAndPlaceholder(Relation index, Buffer buffer) dt = (SpGistDeadTuple) PageGetItem(page, PageGetItemId(page, i)); if (dt->tupstate == SPGIST_REDIRECT && - TransactionIdPrecedes(dt->xid, RecentGlobalXmin)) + GlobalVisTestIsRemovableXid(vistest, dt->xid)) { dt->tupstate = SPGIST_PLACEHOLDER; Assert(opaque->nRedirection > 0); diff --git a/src/backend/access/transam/README b/src/backend/access/transam/README index eb9aac5fd39..4e2178dabab 100644 --- a/src/backend/access/transam/README +++ b/src/backend/access/transam/README @@ -257,31 +257,31 @@ simultaneously, we have one backend take ProcArrayLock and clear the XIDs of multiple processes at once.) ProcArrayEndTransaction also holds the lock while advancing the shared -latestCompletedXid variable. This allows GetSnapshotData to use -latestCompletedXid + 1 as xmax for its snapshot: there can be no +latestCompletedFullXid variable. This allows GetSnapshotData to use +latestCompletedFullXid + 1 as xmax for its snapshot: there can be no transaction >= this xid value that the snapshot needs to consider as completed. In short, then, the rule is that no transaction may exit the set of -currently-running transactions between the time we fetch latestCompletedXid +currently-running transactions between the time we fetch latestCompletedFullXid and the time we finish building our snapshot. However, this restriction only applies to transactions that have an XID --- read-only transactions can end without acquiring ProcArrayLock, since they don't affect anyone -else's snapshot nor latestCompletedXid. +else's snapshot nor latestCompletedFullXid. Transaction start, per se, doesn't have any interlocking with these considerations, since we no longer assign an XID immediately at transaction start. But when we do decide to allocate an XID, GetNewTransactionId must store the new XID into the shared ProcArray before releasing XidGenLock. -This ensures that all top-level XIDs <= latestCompletedXid are either +This ensures that all top-level XIDs <= latestCompletedFullXid are either present in the ProcArray, or not running anymore. (This guarantee doesn't apply to subtransaction XIDs, because of the possibility that there's not room for them in the subxid array; instead we guarantee that they are present or the overflow flag is set.) If a backend released XidGenLock before storing its XID into MyPgXact, then it would be possible for another -backend to allocate and commit a later XID, causing latestCompletedXid to +backend to allocate and commit a later XID, causing latestCompletedFullXid to pass the first backend's XID, before that value became visible in the -ProcArray. That would break GetOldestXmin, as discussed below. +ProcArray. That would break ComputeXidHorizons, as discussed below. We allow GetNewTransactionId to store the XID into MyPgXact->xid (or the subxid array) without taking ProcArrayLock. This was once necessary to @@ -293,42 +293,50 @@ once, rather than assume they can read it multiple times and get the same answer each time. (Use volatile-qualified pointers when doing this, to ensure that the C compiler does exactly what you tell it to.) -Another important activity that uses the shared ProcArray is GetOldestXmin, -which must determine a lower bound for the oldest xmin of any active MVCC -snapshot, system-wide. Each individual backend advertises the smallest -xmin of its own snapshots in MyPgXact->xmin, or zero if it currently has no -live snapshots (eg, if it's between transactions or hasn't yet set a -snapshot for a new transaction). GetOldestXmin takes the MIN() of the -valid xmin fields. It does this with only shared lock on ProcArrayLock, -which means there is a potential race condition against other backends -doing GetSnapshotData concurrently: we must be certain that a concurrent -backend that is about to set its xmin does not compute an xmin less than -what GetOldestXmin returns. We ensure that by including all the active -XIDs into the MIN() calculation, along with the valid xmins. The rule that -transactions can't exit without taking exclusive ProcArrayLock ensures that -concurrent holders of shared ProcArrayLock will compute the same minimum of -currently-active XIDs: no xact, in particular not the oldest, can exit -while we hold shared ProcArrayLock. So GetOldestXmin's view of the minimum -active XID will be the same as that of any concurrent GetSnapshotData, and -so it can't produce an overestimate. If there is no active transaction at -all, GetOldestXmin returns latestCompletedXid + 1, which is a lower bound -for the xmin that might be computed by concurrent or later GetSnapshotData -calls. (We know that no XID less than this could be about to appear in -the ProcArray, because of the XidGenLock interlock discussed above.) +Another important activity that uses the shared ProcArray is +ComputeXidHorizons, which must determine a lower bound for the oldest xmin +of any active MVCC snapshot, system-wide. Each individual backend +advertises the smallest xmin of its own snapshots in MyPgXact->xmin, or zero +if it currently has no live snapshots (eg, if it's between transactions or +hasn't yet set a snapshot for a new transaction). ComputeXidHorizons takes +the MIN() of the valid xmin fields. It does this with only shared lock on +ProcArrayLock, which means there is a potential race condition against other +backends doing GetSnapshotData concurrently: we must be certain that a +concurrent backend that is about to set its xmin does not compute an xmin +less than what ComputeXidHorizons determines. We ensure that by including +all the active XIDs into the MIN() calculation, along with the valid xmins. +The rule that transactions can't exit without taking exclusive ProcArrayLock +ensures that concurrent holders of shared ProcArrayLock will compute the +same minimum of currently-active XIDs: no xact, in particular not the +oldest, can exit while we hold shared ProcArrayLock. So +ComputeXidHorizons's view of the minimum active XID will be the same as that +of any concurrent GetSnapshotData, and so it can't produce an overestimate. +If there is no active transaction at all, ComputeXidHorizons uses +latestCompletedFullXid + 1, which is a lower bound for the xmin that might +be computed by concurrent or later GetSnapshotData calls. (We know that no +XID less than this could be about to appear in the ProcArray, because of the +XidGenLock interlock discussed above.) -GetSnapshotData also performs an oldest-xmin calculation (which had better -match GetOldestXmin's) and stores that into RecentGlobalXmin, which is used -for some tuple age cutoff checks where a fresh call of GetOldestXmin seems -too expensive. Note that while it is certain that two concurrent -executions of GetSnapshotData will compute the same xmin for their own -snapshots, as argued above, it is not certain that they will arrive at the -same estimate of RecentGlobalXmin. This is because we allow XID-less -transactions to clear their MyPgXact->xmin asynchronously (without taking -ProcArrayLock), so one execution might see what had been the oldest xmin, -and another not. This is OK since RecentGlobalXmin need only be a valid -lower bound. As noted above, we are already assuming that fetch/store -of the xid fields is atomic, so assuming it for xmin as well is no extra -risk. +As GetSnapshotData is performance critical, it does not perform an accurate +oldest-xmin calculation (it used to, until v13). The contents of a snapshot +only depend on the xids of other backends, not their xmin. As backend's xmin +changes much more often than its xid, having GetSnapshotData look at xmins +can lead to a lot of unnecessary cacheline ping-pong. Instead +GetSnapshotData updates approximate thresholds (one that guarantees that all +deleted rows older than it can be removed, another determining that deleted +rows newer than it can not be removed). GlobalVisTest* uses those threshold +to make invisibility decision, falling back to ComputeXidHorizons if +necessary. + +Note that while it is certain that two concurrent executions of +GetSnapshotData will compute the same xmin for their own snapshots, there is +no such guarantee for the horizons computed by ComputeXidHorizons. This is +because we allow XID-less transactions to clear their MyPgXact->xmin +asynchronously (without taking ProcArrayLock), so one execution might see +what had been the oldest xmin, and another not. This is OK since the +thresholds need only be a valid lower bound. As noted above, we are already +assuming that fetch/store of the xid fields is atomic, so assuming it for +xmin as well is no extra risk. pg_xact and pg_subtrans diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c index e14b53bf9e3..00b8e4e50d7 100644 --- a/src/backend/access/transam/varsup.c +++ b/src/backend/access/transam/varsup.c @@ -566,3 +566,53 @@ GetNewObjectId(void) return result; } + + +#ifdef USE_ASSERT_CHECKING + +/* + * Assert that xid is between [oldestXid, nextFullXid], which is the range we + * expect XIDs coming from tables etc to be in. + * + * As ShmemVariableCache->oldestXid could change just after this call without + * further precautions, and as a wrapped-around xid could again fall within + * the valid range, this assertion can only detect if something is definitely + * wrong, but not establish correctness. + * + * This intentionally does not expose a return value, to avoid code being + * introduced that depends on the return value. + */ +void +AssertTransactionInAllowableRange(TransactionId xid) +{ + TransactionId oldest_xid; + TransactionId next_xid; + + Assert(TransactionIdIsValid(xid)); + + /* we may see bootstrap / frozen */ + if (!TransactionIdIsNormal(xid)) + return; + + /* + * We can't acquire XidGenLock, as this may be called with XidGenLock + * already held (or with other locks that don't allow XidGenLock to be + * nested). That's ok for our purposes though, since we already rely on + * 32bit reads to be atomic. While nextFullXid is 64 bit, we only look at + * the lower 32bit, so a skewed read doesn't hurt. + * + * There's no increased danger of falling outside [oldest, next] by + * accessing them without a lock. xid needs to have been created with + * GetNewTransactionId() in the originating session, and the locks there + * pair with the memory barrier below. We do however accept xid to be <= + * to next_xid, instead of just <, as xid could be from the procarray, + * before we see the updated nextFullXid value. + */ + pg_memory_barrier(); + oldest_xid = ShmemVariableCache->oldestXid; + next_xid = XidFromFullTransactionId(ShmemVariableCache->nextFullXid); + + Assert(TransactionIdFollowsOrEquals(xid, oldest_xid) || + TransactionIdPrecedesOrEquals(xid, next_xid)); +} +#endif diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 0a97b1d37fb..4ec414e93ef 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -7866,10 +7866,11 @@ StartupXLOG(void) XLogCtl->lastSegSwitchTime = (pg_time_t) time(NULL); XLogCtl->lastSegSwitchLSN = EndOfLog; - /* also initialize latestCompletedXid, to nextXid - 1 */ + /* also initialize latestCompletedFullXid, to nextFullXid - 1 */ LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE); - ShmemVariableCache->latestCompletedXid = XidFromFullTransactionId(ShmemVariableCache->nextFullXid); - TransactionIdRetreat(ShmemVariableCache->latestCompletedXid); + ShmemVariableCache->latestCompletedFullXid = + ShmemVariableCache->nextFullXid; + FullTransactionIdRetreat(&ShmemVariableCache->latestCompletedFullXid); LWLockRelease(ProcArrayLock); /* @@ -9099,7 +9100,7 @@ CreateCheckPoint(int flags) * StartupSUBTRANS hasn't been called yet. */ if (!RecoveryInProgress()) - TruncateSUBTRANS(GetOldestXmin(NULL, PROCARRAY_FLAGS_DEFAULT)); + TruncateSUBTRANS(GetOldestTransactionIdConsideredRunning()); /* Real work is done, but log and update stats before releasing lock. */ LogCheckpointEnd(false); @@ -9459,7 +9460,7 @@ CreateRestartPoint(int flags) * this because StartupSUBTRANS hasn't been called yet. */ if (EnableHotStandby) - TruncateSUBTRANS(GetOldestXmin(NULL, PROCARRAY_FLAGS_DEFAULT)); + TruncateSUBTRANS(GetOldestTransactionIdConsideredRunning()); /* Real work is done, but log and update before releasing lock. */ LogCheckpointEnd(true); diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index 924ef37c816..34b71b6c1c5 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -1056,7 +1056,7 @@ acquire_sample_rows(Relation onerel, int elevel, totalblocks = RelationGetNumberOfBlocks(onerel); /* Need a cutoff xmin for HeapTupleSatisfiesVacuum */ - OldestXmin = GetOldestXmin(onerel, PROCARRAY_FLAGS_VACUUM); + OldestXmin = GetOldestNonRemovableTransactionId(onerel); /* Prepare for sampling block numbers */ nblocks = BlockSampler_Init(&bs, totalblocks, targrows, random()); diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index 576c7e63e99..22228f5684f 100644 --- a/src/backend/commands/vacuum.c +++ b/src/backend/commands/vacuum.c @@ -955,8 +955,25 @@ vacuum_set_xid_limits(Relation rel, * working on a particular table at any time, and that each vacuum is * always an independent transaction. */ - *oldestXmin = - TransactionIdLimitedForOldSnapshots(GetOldestXmin(rel, PROCARRAY_FLAGS_VACUUM), rel); + *oldestXmin = GetOldestNonRemovableTransactionId(rel); + + if (OldSnapshotThresholdActive()) + { + TransactionId limit_xmin; + TimestampTz limit_ts; + + if (TransactionIdLimitedForOldSnapshots(*oldestXmin, rel, &limit_xmin, &limit_ts)) + { + /* + * TODO: We should only set the threshold if we are pruning on the + * basis of the increased limits. Not as crucial here as it is for + * opportunistic pruning (which often happens at a much higher + * frequency), but would still be a significant improvement. + */ + SetOldSnapshotThresholdTimestamp(limit_ts, limit_xmin); + *oldestXmin = limit_xmin; + } + } Assert(TransactionIdIsNormal(*oldestXmin)); @@ -1345,12 +1362,13 @@ vac_update_datfrozenxid(void) bool dirty = false; /* - * Initialize the "min" calculation with GetOldestXmin, which is a - * reasonable approximation to the minimum relfrozenxid for not-yet- - * committed pg_class entries for new tables; see AddNewRelationTuple(). - * So we cannot produce a wrong minimum by starting with this. + * Initialize the "min" calculation with + * GetOldestNonRemovableTransactionId(), which is a reasonable + * approximation to the minimum relfrozenxid for not-yet-committed + * pg_class entries for new tables; see AddNewRelationTuple(). So we + * cannot produce a wrong minimum by starting with this. */ - newFrozenXid = GetOldestXmin(NULL, PROCARRAY_FLAGS_VACUUM); + newFrozenXid = GetOldestNonRemovableTransactionId(NULL); /* * Similarly, initialize the MultiXact "min" with the value that would be @@ -1681,8 +1699,9 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params) StartTransactionCommand(); /* - * Functions in indexes may want a snapshot set. Also, setting a snapshot - * ensures that RecentGlobalXmin is kept truly recent. + * Need to acquire a snapshot to prevent pg_subtrans from being truncated, + * cutoff xids in local memory wrapping around, and to have updated xmin + * horizons. */ PushActiveSnapshot(GetTransactionSnapshot()); @@ -1705,8 +1724,8 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params) * * Note: these flags remain set until CommitTransaction or * AbortTransaction. We don't want to clear them until we reset - * MyPgXact->xid/xmin, else OldestXmin might appear to go backwards, - * which is probably Not Good. + * MyPgXact->xid/xmin, otherwise GetOldestNonRemovableTransactionId() + * might appear to go backwards, which is probably Not Good. */ LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE); MyPgXact->vacuumFlags |= PROC_IN_VACUUM; diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c index 9c7d4b0c60e..ac97e28be19 100644 --- a/src/backend/postmaster/autovacuum.c +++ b/src/backend/postmaster/autovacuum.c @@ -1877,6 +1877,10 @@ get_database_list(void) * the secondary effect that it sets RecentGlobalXmin. (This is critical * for anything that reads heap pages, because HOT may decide to prune * them even if the process doesn't attempt to modify any tuples.) + * + * FIXME: This comment is inaccurate / the code buggy. A snapshot that is + * not pushed/active does not reliably prevent HOT pruning (->xmin could + * e.g. be cleared when cache invalidations are processed). */ StartTransactionCommand(); (void) GetTransactionSnapshot(); diff --git a/src/backend/replication/logical/launcher.c b/src/backend/replication/logical/launcher.c index aec885e9871..158b2f3d73b 100644 --- a/src/backend/replication/logical/launcher.c +++ b/src/backend/replication/logical/launcher.c @@ -122,6 +122,10 @@ get_subscription_list(void) * the secondary effect that it sets RecentGlobalXmin. (This is critical * for anything that reads heap pages, because HOT may decide to prune * them even if the process doesn't attempt to modify any tuples.) + * + * FIXME: This comment is inaccurate / the code buggy. A snapshot that is + * not pushed/active does not reliably prevent HOT pruning (->xmin could + * e.g. be cleared when cache invalidations are processed). */ StartTransactionCommand(); (void) GetTransactionSnapshot(); diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c index d5a9b568a68..7c11e1ab44c 100644 --- a/src/backend/replication/walreceiver.c +++ b/src/backend/replication/walreceiver.c @@ -1181,22 +1181,7 @@ XLogWalRcvSendHSFeedback(bool immed) */ if (hot_standby_feedback) { - TransactionId slot_xmin; - - /* - * Usually GetOldestXmin() would include both global replication slot - * xmin and catalog_xmin in its calculations, but we want to derive - * separate values for each of those. So we ask for an xmin that - * excludes the catalog_xmin. - */ - xmin = GetOldestXmin(NULL, - PROCARRAY_FLAGS_DEFAULT | PROCARRAY_SLOTS_XMIN); - - ProcArrayGetReplicationSlotXmin(&slot_xmin, &catalog_xmin); - - if (TransactionIdIsValid(slot_xmin) && - TransactionIdPrecedes(slot_xmin, xmin)) - xmin = slot_xmin; + GetReplicationHorizons(&xmin, &catalog_xmin); } else { diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index 5e2210dd7bd..fd370d52b66 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -2116,9 +2116,10 @@ ProcessStandbyHSFeedbackMessage(void) /* * Set the WalSender's xmin equal to the standby's requested xmin, so that - * the xmin will be taken into account by GetOldestXmin. This will hold - * back the removal of dead rows and thereby prevent the generation of - * cleanup conflicts on the standby server. + * the xmin will be taken into account by GetSnapshotData() / + * ComputeXidHorizons(). This will hold back the removal of dead rows and + * thereby prevent the generation of cleanup conflicts on the standby + * server. * * There is a small window for a race condition here: although we just * checked that feedbackXmin precedes nextXid, the nextXid could have @@ -2131,10 +2132,10 @@ ProcessStandbyHSFeedbackMessage(void) * own xmin would prevent nextXid from advancing so far. * * We don't bother taking the ProcArrayLock here. Setting the xmin field - * is assumed atomic, and there's no real need to prevent a concurrent - * GetOldestXmin. (If we're moving our xmin forward, this is obviously - * safe, and if we're moving it backwards, well, the data is at risk - * already since a VACUUM could have just finished calling GetOldestXmin.) + * is assumed atomic, and there's no real need to prevent concurrent + * horizon determinations. (If we're moving our xmin forward, this is + * obviously safe, and if we're moving it backwards, well, the data is at + * risk already since a VACUUM could already have determined the horizon.) * * If we're using a replication slot we reserve the xmin via that, * otherwise via the walsender's PGXACT entry. We can only track the diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c index b4485335644..c011387ba90 100644 --- a/src/backend/storage/ipc/procarray.c +++ b/src/backend/storage/ipc/procarray.c @@ -99,6 +99,142 @@ typedef struct ProcArrayStruct int pgprocnos[FLEXIBLE_ARRAY_MEMBER]; } ProcArrayStruct; +/* + * State for the GlobalVisTest* familiy of functions. Those functions can + * e.g. be used to decide if a deleted row can be removed without violating + * MVCC semantics: If the deleted row's xmax is not considered to be running + * by anyone, the row can be removed. + * + * To avoid slowing down GetSnapshotData(), we don't calculate a precise + * cutoff XID while building a snapshot (looking at the frequently changing + * xmins scales badly). Instead we compute two boundaries while building the + * snapshot: + * + * 1) definitely_needed, indicating that rows deleted by XIDs >= + * definitely_needed are definitely still visible. + * + * 2) maybe_needed, indicating that rows deleted by XIDs < maybe_needed can + * definitely be removed + * + * When testing an XID that falls in between the two (i.e. XID >= maybe_needed + * && XID < definitely_needed), the boundaries can be recomputed (using + * ComputeXidHorizons()) to get a more accurate answer. This is cheaper than + * maintaining an accurate value all the time. + * + * As it is not cheap to compute accurate boundaries, we limit the number of + * times that happens in short succession. See GlobalVisTestShouldUpdate(). + * + * + * There are three backend lifetime instances of this struct, optimized for + * different types of relations. As e.g. a normal user defined table in one + * database is inaccessible to backends connected to another database, a test + * specific to a relation can be more aggressive than a test for a shared + * relation. Currently we track three different states: + * + * 1) GlobalVisSharedRels, which only considers an XID's + * effects visible-to-everyone if neither snapshots in any database, nor a + * replication slot's xmin, nor a replication slot's catalog_xmin might + * still consider XID as running. + * + * 2) GlobalVisCatalogRels, which only considers an XID's + * effects visible-to-everyone if neither snapshots in the current + * database, nor a replication slot's xmin, nor a replication slot's + * catalog_xmin might still consider XID as running. + * + * I.e. the difference to GlobalVisSharedRels is that + * snapshot in other databases are ignored. + * + * 3) GlobalVisCatalogRels, which only considers an XID's + * effects visible-to-everyone if neither snapshots in the current + * database, nor a replication slot's xmin consider XID as running. + * + * I.e. the difference to GlobalVisCatalogRels is that + * replication slot's catalog_xmin is not taken into account. + * + * GlobalVisTestFor(relation) returns the appropriate state + * for the relation. + * + * The boundaries are FullTransactionIds instead of TransactionIds to avoid + * wraparound dangers. There e.g. would otherwise exist no procarray state to + * prevent maybe_needed to become old enough after the GetSnapshotData() + * call. + * + * The typedef is in the header. + */ +struct GlobalVisState +{ + /* XIDs >= are considered running by some backend */ + FullTransactionId definitely_needed; + + /* XIDs < are not considered to be running by any backend */ + FullTransactionId maybe_needed; +}; + +/* + * Result of ComputeXidHorizons(). + */ +typedef struct ComputeXidHorizonsResult +{ + /* + * The value of ShmemVariableCache->latestCompletedFullXid when + * ComputeXidHorizons() held ProcArrayLock. + */ + FullTransactionId latest_completed; + + /* + * The same for procArray->replication_slot_xmin and. + * procArray->replication_slot_catalog_xmin. + */ + TransactionId slot_xmin; + TransactionId slot_catalog_xmin; + + /* + * Oldest xid that any backend might still consider running. This needs to + * include processes running VACUUM, in contrast to the normal visibility + * cutoffs, as vacuum needs to be able to perform pg_subtrans lookups when + * determining visibility, but doesn't care about rows above its xmin to + * be removed. + * + * This likely should only be needed to determine whether pg_subtrans can + * be truncated. It currently includes the effects of replications slots, + * for historical reasons. But that could likely be changed. + */ + TransactionId oldest_considered_running; + + /* + * Oldest xid for which deleted tuples need to be retained in shared + * tables. + * + * This includes the effects of replications lots. If that's not desired, + * look at shared_oldest_nonremovable_raw; + */ + TransactionId shared_oldest_nonremovable; + + /* + * Oldest xid that may be necessary to retain in shared tables. This is + * the same as shared_oldest_nonremovable, except that is not affected by + * replication slot's catalog_xmin. + * + * This is mainly useful to be able to send the catalog_xmin to upstream + * streaming replication servers via hot_standby_feedback, so they can + * apply the limit only when accessing catalog tables. + */ + TransactionId shared_oldest_nonremovable_raw; + + /* + * Oldest xid for which deleted tuples need to be retained in non-shared + * catalog tables. + */ + TransactionId catalog_oldest_nonremovable; + + /* + * Oldest xid for which deleted tuples need to be retained in normal user + * defined tables. + */ + TransactionId data_oldest_nonremovable; +} ComputeXidHorizonsResult; + + static ProcArrayStruct *procArray; static PGPROC *allProcs; @@ -118,6 +254,22 @@ static TransactionId latestObservedXid = InvalidTransactionId; */ static TransactionId standbySnapshotPendingXmin; +/* + * State for visibility checks on different types of relations. See struct + * GlobalVisState for details. As shared, catalog, and user defined + * relations can have different horizons, one such state exists for each. + */ +static GlobalVisState GlobalVisSharedRels; +static GlobalVisState GlobalVisCatalogRels; +static GlobalVisState GlobalVisDataRels; + +/* + * This backend's RecentXmin at the last time the accurate xmin horizon was + * recomputed, or InvalidTransactionId if it has not. Used to limit how many + * times accurate horizons are recomputed. See GlobalVisTestShouldUpdate(). + */ +static TransactionId ComputeXidHorizonsResultLastXmin; + #ifdef XIDCACHE_DEBUG /* counters for XidCache measurement */ @@ -175,6 +327,10 @@ static void KnownAssignedXidsReset(void); static inline void ProcArrayEndTransactionInternal(PGPROC *proc, PGXACT *pgxact, TransactionId latestXid); static void ProcArrayGroupClearXid(PGPROC *proc, TransactionId latestXid); +static void MaintainLatestCompletedXid(TransactionId latestXid); +static void MaintainLatestCompletedXidRecovery(TransactionId latestXid); + +static inline FullTransactionId FullXidViaRelative(FullTransactionId rel, TransactionId xid); /* * Report shared-memory space needed by CreateSharedProcArray. @@ -349,9 +505,7 @@ ProcArrayRemove(PGPROC *proc, TransactionId latestXid) Assert(TransactionIdIsValid(allPgXact[proc->pgprocno].xid)); /* Advance global latestCompletedXid while holding the lock */ - if (TransactionIdPrecedes(ShmemVariableCache->latestCompletedXid, - latestXid)) - ShmemVariableCache->latestCompletedXid = latestXid; + MaintainLatestCompletedXid(latestXid); } else { @@ -464,9 +618,7 @@ ProcArrayEndTransactionInternal(PGPROC *proc, PGXACT *pgxact, pgxact->overflowed = false; /* Also advance global latestCompletedXid while holding the lock */ - if (TransactionIdPrecedes(ShmemVariableCache->latestCompletedXid, - latestXid)) - ShmemVariableCache->latestCompletedXid = latestXid; + MaintainLatestCompletedXid(latestXid); } /* @@ -621,6 +773,58 @@ ProcArrayClearTransaction(PGPROC *proc) pgxact->overflowed = false; } +/* + * Update ShmemVariableCache->latestCompletedFullXid to point to latestXid if + * currently older. + */ +static void +MaintainLatestCompletedXid(TransactionId latestXid) +{ + FullTransactionId cur_latest = ShmemVariableCache->latestCompletedFullXid; + + Assert(LWLockHeldByMe(ProcArrayLock)); + Assert(FullTransactionIdIsValid(cur_latest)); + + if (TransactionIdPrecedes(XidFromFullTransactionId(cur_latest), latestXid)) + { + ShmemVariableCache->latestCompletedFullXid = + FullXidViaRelative(cur_latest, latestXid); + } + + Assert(IsBootstrapProcessingMode() || + FullTransactionIdIsNormal(ShmemVariableCache->latestCompletedFullXid)); +} + +/* + * Same as MaintainLatestCompletedXid, except for use during WAL replay. + */ +static void +MaintainLatestCompletedXidRecovery(TransactionId latestXid) +{ + FullTransactionId cur_latest = ShmemVariableCache->latestCompletedFullXid; + FullTransactionId rel; + + Assert(AmStartupProcess() || !IsUnderPostmaster); + Assert(LWLockHeldByMe(ProcArrayLock)); + + /* + * Need a FullTransactionId to compare latestXid with. Can't rely on + * latestCompletedFullXid to be initialized in recovery. But in recovery + * it's safe to access nextFullXid without a lock for the startup process. + */ + rel = ShmemVariableCache->nextFullXid; + Assert(FullTransactionIdIsValid(ShmemVariableCache->nextFullXid)); + + if (!FullTransactionIdIsValid(cur_latest) || + TransactionIdPrecedes(XidFromFullTransactionId(cur_latest), latestXid)) + { + ShmemVariableCache->latestCompletedFullXid = + FullXidViaRelative(rel, latestXid); + } + + Assert(FullTransactionIdIsNormal(ShmemVariableCache->latestCompletedFullXid)); +} + /* * ProcArrayInitRecovery -- initialize recovery xid mgmt environment * @@ -841,7 +1045,7 @@ ProcArrayApplyRecoveryInfo(RunningTransactions running) * Now we've got the running xids we need to set the global values that * are used to track snapshots as they evolve further. * - * - latestCompletedXid which will be the xmax for snapshots + * - latestCompletedFullXid which will be the xmax for snapshots * - lastOverflowedXid which shows whether snapshots overflow * - nextXid * @@ -867,14 +1071,11 @@ ProcArrayApplyRecoveryInfo(RunningTransactions running) /* * If a transaction wrote a commit record in the gap between taking and - * logging the snapshot then latestCompletedXid may already be higher than - * the value from the snapshot, so check before we use the incoming value. + * logging the snapshot then latestCompletedFullXid may already be higher + * than the value from the snapshot, so check before we use the incoming + * value. It also might not yet be set at all. */ - if (TransactionIdPrecedes(ShmemVariableCache->latestCompletedXid, - running->latestCompletedXid)) - ShmemVariableCache->latestCompletedXid = running->latestCompletedXid; - - Assert(TransactionIdIsNormal(ShmemVariableCache->latestCompletedXid)); + MaintainLatestCompletedXidRecovery(running->latestCompletedXid); LWLockRelease(ProcArrayLock); @@ -1048,10 +1249,11 @@ TransactionIdIsInProgress(TransactionId xid) LWLockAcquire(ProcArrayLock, LW_SHARED); /* - * Now that we have the lock, we can check latestCompletedXid; if the + * Now that we have the lock, we can check latestCompletedFullXid; if the * target Xid is after that, it's surely still running. */ - if (TransactionIdPrecedes(ShmemVariableCache->latestCompletedXid, xid)) + if (TransactionIdPrecedes(XidFromFullTransactionId(ShmemVariableCache->latestCompletedFullXid), + xid)) { LWLockRelease(ProcArrayLock); xc_by_latest_xid_inc(); @@ -1248,159 +1450,183 @@ TransactionIdIsActive(TransactionId xid) /* - * GetOldestXmin -- returns oldest transaction that was running - * when any current transaction was started. + * Determine XID horizons. * - * If rel is NULL or a shared relation, all backends are considered, otherwise - * only backends running in this database are considered. + * This is used by wrapper functions like GetOldestNonRemovableTransactionId() + * (for VACUUM), GetReplicationHorizons() (for hot_standby_feedback), etc as + * well as "internally" by GlobalVisUpdate() (see comment above struct + * GlobalVisState). * - * The flags are used to ignore the backends in calculation when any of the - * corresponding flags is set. Typically, if you want to ignore ones with - * PROC_IN_VACUUM flag, you can use PROCARRAY_FLAGS_VACUUM. + * See ComputedXidHorizonsResult for the various computed horizons. * - * PROCARRAY_SLOTS_XMIN causes GetOldestXmin to ignore the xmin and - * catalog_xmin of any replication slots that exist in the system when - * calculating the oldest xmin. + * For VACUUM separate horizons (used to to decide which deleted tuples must + * be preserved), for shared and non-shared tables are computed. For shared + * relations backends in all databases must be considered, but for non-shared + * relations that's not required, since only backends in my own database could + * ever see the tuples in them. Also, we can ignore concurrently running lazy + * VACUUMs because (a) they must be working on other tables, and (b) they + * don't need to do snapshot-based lookups. * - * This is used by VACUUM to decide which deleted tuples must be preserved in - * the passed in table. For shared relations backends in all databases must be - * considered, but for non-shared relations that's not required, since only - * backends in my own database could ever see the tuples in them. Also, we can - * ignore concurrently running lazy VACUUMs because (a) they must be working - * on other tables, and (b) they don't need to do snapshot-based lookups. - * - * This is also used to determine where to truncate pg_subtrans. For that - * backends in all databases have to be considered, so rel = NULL has to be - * passed in. + * This also computes a horizon used to truncate pg_subtrans. For that + * backends in all databases have to be considered, and concurrently running + * lazy VACUUMs cannot be ignored, as they still may perform pg_subtrans + * accesses. * * Note: we include all currently running xids in the set of considered xids. * This ensures that if a just-started xact has not yet set its snapshot, * when it does set the snapshot it cannot set xmin less than what we compute. * See notes in src/backend/access/transam/README. * - * Note: despite the above, it's possible for the calculated value to move - * backwards on repeated calls. The calculated value is conservative, so that - * anything older is definitely not considered as running by anyone anymore, - * but the exact value calculated depends on a number of things. For example, - * if rel = NULL and there are no transactions running in the current - * database, GetOldestXmin() returns latestCompletedXid. If a transaction + * Note: despite the above, it's possible for the calculated values to move + * backwards on repeated calls. The calculated values are conservative, so + * that anything older is definitely not considered as running by anyone + * anymore, but the exact values calculated depend on a number of things. For + * example, if there are no transactions running in the current database, the + * horizon for normal tables will be latestCompletedFullXid. If a transaction * begins after that, its xmin will include in-progress transactions in other * databases that started earlier, so another call will return a lower value. * Nonetheless it is safe to vacuum a table in the current database with the * first result. There are also replication-related effects: a walsender * process can set its xmin based on transactions that are no longer running * on the primary but are still being replayed on the standby, thus possibly - * making the GetOldestXmin reading go backwards. In this case there is a - * possibility that we lose data that the standby would like to have, but - * unless the standby uses a replication slot to make its xmin persistent - * there is little we can do about that --- data is only protected if the - * walsender runs continuously while queries are executed on the standby. - * (The Hot Standby code deals with such cases by failing standby queries - * that needed to access already-removed data, so there's no integrity bug.) - * The return value is also adjusted with vacuum_defer_cleanup_age, so - * increasing that setting on the fly is another easy way to make - * GetOldestXmin() move backwards, with no consequences for data integrity. + * making the values go backwards. In this case there is a possibility that + * we lose data that the standby would like to have, but unless the standby + * uses a replication slot to make its xmin persistent there is little we can + * do about that --- data is only protected if the walsender runs continuously + * while queries are executed on the standby. (The Hot Standby code deals + * with such cases by failing standby queries that needed to access + * already-removed data, so there's no integrity bug.) The computed values + * are also adjusted with vacuum_defer_cleanup_age, so increasing that setting + * on the fly is another easy way to make horizons move backwards, with no + * consequences for data integrity. */ -TransactionId -GetOldestXmin(Relation rel, int flags) +static void +ComputeXidHorizons(ComputeXidHorizonsResult *h) { ProcArrayStruct *arrayP = procArray; - TransactionId result; - int index; - bool allDbs; + TransactionId kaxmin; + bool in_recovery = RecoveryInProgress(); - TransactionId replication_slot_xmin = InvalidTransactionId; - TransactionId replication_slot_catalog_xmin = InvalidTransactionId; - - /* - * If we're not computing a relation specific limit, or if a shared - * relation has been passed in, backends in all databases have to be - * considered. - */ - allDbs = rel == NULL || rel->rd_rel->relisshared; - - /* Cannot look for individual databases during recovery */ - Assert(allDbs || !RecoveryInProgress()); + /* inferred after ProcArrayLock is released */ + h->catalog_oldest_nonremovable = InvalidTransactionId; LWLockAcquire(ProcArrayLock, LW_SHARED); - /* - * We initialize the MIN() calculation with latestCompletedXid + 1. This - * is a lower bound for the XIDs that might appear in the ProcArray later, - * and so protects us against overestimating the result due to future - * additions. - */ - result = ShmemVariableCache->latestCompletedXid; - Assert(TransactionIdIsNormal(result)); - TransactionIdAdvance(result); + h->latest_completed = ShmemVariableCache->latestCompletedFullXid; - for (index = 0; index < arrayP->numProcs; index++) + /* + * We initialize the MIN() calculation with latestCompletedFullXid + 1. + * This is a lower bound for the XIDs that might appear in the ProcArray + * later, and so protects us against overestimating the result due to + * future additions. + */ + { + TransactionId initial; + + initial = XidFromFullTransactionId(h->latest_completed); + Assert(TransactionIdIsValid(initial)); + TransactionIdAdvance(initial); + + h->oldest_considered_running = initial; + h->shared_oldest_nonremovable = initial; + h->data_oldest_nonremovable = initial; + } + + /* + * Fetch slot horizons while ProcArrayLock is held - the + * LWLockAcquire/LWLockRelease are a barrier, ensuring this happens inside + * the lock. + */ + h->slot_xmin = procArray->replication_slot_xmin; + h->slot_catalog_xmin = procArray->replication_slot_catalog_xmin; + + for (int index = 0; index < arrayP->numProcs; index++) { int pgprocno = arrayP->pgprocnos[index]; PGPROC *proc = &allProcs[pgprocno]; PGXACT *pgxact = &allPgXact[pgprocno]; + TransactionId xid; + TransactionId xmin; - if (pgxact->vacuumFlags & (flags & PROCARRAY_PROC_FLAGS_MASK)) + /* Fetch xid just once - see GetNewTransactionId */ + xid = UINT32_ACCESS_ONCE(pgxact->xid); + xmin = UINT32_ACCESS_ONCE(pgxact->xmin); + + /* + * Consider both the transaction's Xmin, and its Xid. + * + * We must check both because a transaction might have an Xmin but not + * (yet) an Xid; conversely, if it has an Xid, that could determine + * some not-yet-set Xmin. + */ + xmin = TransactionIdOlder(xmin, xid); + + /* if neither is set, this proc doesn't influence the horizon */ + if (!TransactionIdIsValid(xmin)) continue; - if (allDbs || + /* + * Don't ignore any procs when determining which transactions might be + * considered running. While slots should ensure logical decoding + * backends are protected even without this check, it can't hurt to + * include them here as well.. + */ + h->oldest_considered_running = + TransactionIdOlder(h->oldest_considered_running, xmin); + + /* + * Skip over backends either vacuuming (which is ok with rows being + * removed, as long as pg_subtrans is not truncated) or doing logical + * decoding (which manages xmin separately, check below). + */ + if (pgxact->vacuumFlags & (PROC_IN_VACUUM | PROC_IN_LOGICAL_DECODING)) + continue; + + /* shared tables need to take backends in all database into account */ + h->shared_oldest_nonremovable = + TransactionIdOlder(h->shared_oldest_nonremovable, xmin); + + /* + * Normally queries in other databases are ignored for anything but + * the shared horizon. But in recovery we cannot compute an accurate + * per-database horizon as all xids are managed via the + * KnownAssignedXids machinery. + */ + if (in_recovery || proc->databaseId == MyDatabaseId || proc->databaseId == 0) /* always include WalSender */ { - /* Fetch xid just once - see GetNewTransactionId */ - TransactionId xid = UINT32_ACCESS_ONCE(pgxact->xid); - - /* First consider the transaction's own Xid, if any */ - if (TransactionIdIsNormal(xid) && - TransactionIdPrecedes(xid, result)) - result = xid; - - /* - * Also consider the transaction's Xmin, if set. - * - * We must check both Xid and Xmin because a transaction might - * have an Xmin but not (yet) an Xid; conversely, if it has an - * Xid, that could determine some not-yet-set Xmin. - */ - xid = UINT32_ACCESS_ONCE(pgxact->xmin); - if (TransactionIdIsNormal(xid) && - TransactionIdPrecedes(xid, result)) - result = xid; + h->data_oldest_nonremovable = + TransactionIdOlder(h->data_oldest_nonremovable, xmin); } } /* - * Fetch into local variable while ProcArrayLock is held - the - * LWLockRelease below is a barrier, ensuring this happens inside the - * lock. + * If in recovery fetch oldest xid in KnownAssignedXids, will be applied + * after lock is released. */ - replication_slot_xmin = procArray->replication_slot_xmin; - replication_slot_catalog_xmin = procArray->replication_slot_catalog_xmin; + if (in_recovery) + kaxmin = KnownAssignedXidsGetOldestXmin(); - if (RecoveryInProgress()) + /* + * No other information needed, so release the lock immediately. The rest + * of the computations can be done without a lock. + */ + LWLockRelease(ProcArrayLock); + + if (in_recovery) { - /* - * Check to see whether KnownAssignedXids contains an xid value older - * than the main procarray. - */ - TransactionId kaxmin = KnownAssignedXidsGetOldestXmin(); - - LWLockRelease(ProcArrayLock); - - if (TransactionIdIsNormal(kaxmin) && - TransactionIdPrecedes(kaxmin, result)) - result = kaxmin; + h->oldest_considered_running = + TransactionIdOlder(h->oldest_considered_running, kaxmin); + h->shared_oldest_nonremovable = + TransactionIdOlder(h->shared_oldest_nonremovable, kaxmin); + h->data_oldest_nonremovable = + TransactionIdOlder(h->data_oldest_nonremovable, kaxmin); } else { /* - * No other information needed, so release the lock immediately. - */ - LWLockRelease(ProcArrayLock); - - /* - * Compute the cutoff XID by subtracting vacuum_defer_cleanup_age, - * being careful not to generate a "permanent" XID. + * Compute the cutoff XID by subtracting vacuum_defer_cleanup_age. * * vacuum_defer_cleanup_age provides some additional "slop" for the * benefit of hot standby queries on standby servers. This is quick @@ -1412,34 +1638,143 @@ GetOldestXmin(Relation rel, int flags) * in varsup.c. Also note that we intentionally don't apply * vacuum_defer_cleanup_age on standby servers. */ - result -= vacuum_defer_cleanup_age; - if (!TransactionIdIsNormal(result)) - result = FirstNormalTransactionId; + h->oldest_considered_running = + TransactionIdRetreatedBy(h->oldest_considered_running, + vacuum_defer_cleanup_age); + h->shared_oldest_nonremovable = + TransactionIdRetreatedBy(h->shared_oldest_nonremovable, + vacuum_defer_cleanup_age); + h->data_oldest_nonremovable = + TransactionIdRetreatedBy(h->data_oldest_nonremovable, + vacuum_defer_cleanup_age); } /* * Check whether there are replication slots requiring an older xmin. */ - if (!(flags & PROCARRAY_SLOTS_XMIN) && - TransactionIdIsValid(replication_slot_xmin) && - NormalTransactionIdPrecedes(replication_slot_xmin, result)) - result = replication_slot_xmin; + h->shared_oldest_nonremovable = + TransactionIdOlder(h->shared_oldest_nonremovable, h->slot_xmin); + h->data_oldest_nonremovable = + TransactionIdOlder(h->data_oldest_nonremovable, h->slot_xmin); /* - * After locks have been released and vacuum_defer_cleanup_age has been - * applied, check whether we need to back up further to make logical - * decoding possible. We need to do so if we're computing the global limit - * (rel = NULL) or if the passed relation is a catalog relation of some - * kind. + * The only difference between catalog / data horizons is that the slot's + * catalog xmin is applied to the catalog one (so catalogs can be accessed + * for logical decoding). Initialize with data horizon, and then back up + * further if necessary. Have to back up the shared horizon as well, since + * that also can contain catalogs. */ - if (!(flags & PROCARRAY_SLOTS_XMIN) && - (rel == NULL || - RelationIsAccessibleInLogicalDecoding(rel)) && - TransactionIdIsValid(replication_slot_catalog_xmin) && - NormalTransactionIdPrecedes(replication_slot_catalog_xmin, result)) - result = replication_slot_catalog_xmin; + h->shared_oldest_nonremovable_raw = h->shared_oldest_nonremovable; + h->shared_oldest_nonremovable = + TransactionIdOlder(h->shared_oldest_nonremovable, + h->slot_catalog_xmin); + h->catalog_oldest_nonremovable = h->data_oldest_nonremovable; + h->catalog_oldest_nonremovable = + TransactionIdOlder(h->catalog_oldest_nonremovable, + h->slot_catalog_xmin); - return result; + /* + * It's possible that slots / vacuum_defer_cleanup_age backed up the + * horizons further than oldest_considered_running. Fix. + */ + h->oldest_considered_running = + TransactionIdOlder(h->oldest_considered_running, + h->shared_oldest_nonremovable); + h->oldest_considered_running = + TransactionIdOlder(h->oldest_considered_running, + h->catalog_oldest_nonremovable); + h->oldest_considered_running = + TransactionIdOlder(h->oldest_considered_running, + h->data_oldest_nonremovable); + + /* + * shared horizons have to be at least as old as the oldest visible in + * current db + */ + Assert(TransactionIdPrecedesOrEquals(h->shared_oldest_nonremovable, + h->data_oldest_nonremovable)); + Assert(TransactionIdPrecedesOrEquals(h->shared_oldest_nonremovable, + h->catalog_oldest_nonremovable)); + + /* + * Horizons need to ensure that pg_subtrans access is still possible for + * the relevant backends. + */ + Assert(TransactionIdPrecedesOrEquals(h->oldest_considered_running, + h->shared_oldest_nonremovable)); + Assert(TransactionIdPrecedesOrEquals(h->oldest_considered_running, + h->catalog_oldest_nonremovable)); + Assert(TransactionIdPrecedesOrEquals(h->oldest_considered_running, + h->data_oldest_nonremovable)); + Assert(!TransactionIdIsValid(h->slot_xmin) || + TransactionIdPrecedesOrEquals(h->oldest_considered_running, + h->slot_xmin)); + Assert(!TransactionIdIsValid(h->slot_catalog_xmin) || + TransactionIdPrecedesOrEquals(h->oldest_considered_running, + h->slot_catalog_xmin)); +} + +/* + * Return the oldest XID for which deleted tuples must be preserved in the + * passed table. + * + * If rel is not NULL the horizon may be considerably more recent than + * otherwise (i.e. fewer tuples will be removable). In the NULL case a horizon + * that is correct (but not optimal) for all relations will be returned. + * + * This is used by VACUUM to decide which deleted tuples must be preserved in + * the passed in table. + */ +TransactionId +GetOldestNonRemovableTransactionId(Relation rel) +{ + ComputeXidHorizonsResult horizons; + + ComputeXidHorizons(&horizons); + + /* select horizon appropriate for relation */ + if (rel == NULL || rel->rd_rel->relisshared) + return horizons.shared_oldest_nonremovable; + else if (RelationIsAccessibleInLogicalDecoding(rel)) + return horizons.catalog_oldest_nonremovable; + else + return horizons.data_oldest_nonremovable; +} + +/* + * Return the oldest transaction id any currently running backend might still + * consider running. This should not be used for visibility / pruning + * determinations (see GetOldestNonRemovableTransactionId()), but for + * decisions like up to where pg_subtrans can be truncated. + */ +TransactionId +GetOldestTransactionIdConsideredRunning(void) +{ + ComputeXidHorizonsResult horizons; + + ComputeXidHorizons(&horizons); + + return horizons.oldest_considered_running; +} + +/* + * Return the visibility horizons for a hot standby feedback message. + */ +void +GetReplicationHorizons(TransactionId *xmin, TransactionId *catalog_xmin) +{ + ComputeXidHorizonsResult horizons; + + ComputeXidHorizons(&horizons); + + /* + * Don't want to use shared_oldest_nonremovable here, as that contains the + * effect of replication slot's catalog_xmin. We want to send a separate + * feedback for the catalog horizon, so the primary can remove data table + * contents more aggressively. + */ + *xmin = horizons.shared_oldest_nonremovable_raw; + *catalog_xmin = horizons.slot_catalog_xmin; } /* @@ -1490,12 +1825,10 @@ GetMaxSnapshotSubxidCount(void) * current transaction (this is the same as MyPgXact->xmin). * RecentXmin: the xmin computed for the most recent snapshot. XIDs * older than this are known not running any more. - * RecentGlobalXmin: the global xmin (oldest TransactionXmin across all - * running transactions, except those running LAZY VACUUM). This is - * the same computation done by - * GetOldestXmin(NULL, PROCARRAY_FLAGS_VACUUM). - * RecentGlobalDataXmin: the global xmin for non-catalog tables - * >= RecentGlobalXmin + * + * And try to advance the bounds of GlobalVisSharedRels, + * GlobalVisCatalogRels, GlobalVisDataRels for + * the benefit GlobalVis*. * * Note: this function should probably not be called with an argument that's * not statically allocated (see xip allocation below). @@ -1506,11 +1839,12 @@ GetSnapshotData(Snapshot snapshot) ProcArrayStruct *arrayP = procArray; TransactionId xmin; TransactionId xmax; - TransactionId globalxmin; int index; int count = 0; int subcount = 0; bool suboverflowed = false; + FullTransactionId latest_completed; + TransactionId oldestxid; TransactionId replication_slot_xmin = InvalidTransactionId; TransactionId replication_slot_catalog_xmin = InvalidTransactionId; @@ -1554,13 +1888,16 @@ GetSnapshotData(Snapshot snapshot) */ LWLockAcquire(ProcArrayLock, LW_SHARED); + latest_completed = ShmemVariableCache->latestCompletedFullXid; + oldestxid = ShmemVariableCache->oldestXid; + /* xmax is always latestCompletedXid + 1 */ - xmax = ShmemVariableCache->latestCompletedXid; - Assert(TransactionIdIsNormal(xmax)); + xmax = XidFromFullTransactionId(latest_completed); TransactionIdAdvance(xmax); + Assert(TransactionIdIsNormal(xmax)); /* initialize xmin calculation with xmax */ - globalxmin = xmin = xmax; + xmin = xmax; snapshot->takenDuringRecovery = RecoveryInProgress(); @@ -1589,12 +1926,6 @@ GetSnapshotData(Snapshot snapshot) (PROC_IN_LOGICAL_DECODING | PROC_IN_VACUUM)) continue; - /* Update globalxmin to be the smallest valid xmin */ - xid = UINT32_ACCESS_ONCE(pgxact->xmin); - if (TransactionIdIsNormal(xid) && - NormalTransactionIdPrecedes(xid, globalxmin)) - globalxmin = xid; - /* Fetch xid just once - see GetNewTransactionId */ xid = UINT32_ACCESS_ONCE(pgxact->xid); @@ -1710,34 +2041,78 @@ GetSnapshotData(Snapshot snapshot) LWLockRelease(ProcArrayLock); - /* - * Update globalxmin to include actual process xids. This is a slightly - * different way of computing it than GetOldestXmin uses, but should give - * the same result. - */ - if (TransactionIdPrecedes(xmin, globalxmin)) - globalxmin = xmin; + /* maintain state for GlobalVis* */ + { + TransactionId def_vis_xid; + TransactionId def_vis_xid_data; + FullTransactionId def_vis_fxid; + FullTransactionId def_vis_fxid_data; + FullTransactionId oldestfxid; - /* Update global variables too */ - RecentGlobalXmin = globalxmin - vacuum_defer_cleanup_age; - if (!TransactionIdIsNormal(RecentGlobalXmin)) - RecentGlobalXmin = FirstNormalTransactionId; + /* + * Converting oldestXid is only safe when xid horizon cannot advance, + * i.e. holding locks. While we don't hold the lock anymore, all the + * necessary data has been gathered with lock held. + */ + oldestfxid = FullXidViaRelative(latest_completed, oldestxid); - /* Check whether there's a replication slot requiring an older xmin. */ - if (TransactionIdIsValid(replication_slot_xmin) && - NormalTransactionIdPrecedes(replication_slot_xmin, RecentGlobalXmin)) - RecentGlobalXmin = replication_slot_xmin; + /* apply vacuum_defer_cleanup_age */ + def_vis_xid_data = + TransactionIdRetreatedBy(xmin, vacuum_defer_cleanup_age); - /* Non-catalog tables can be vacuumed if older than this xid */ - RecentGlobalDataXmin = RecentGlobalXmin; + /* Check whether there's a replication slot requiring an older xmin. */ + def_vis_xid_data = + TransactionIdOlder(def_vis_xid_data, replication_slot_xmin); - /* - * Check whether there's a replication slot requiring an older catalog - * xmin. - */ - if (TransactionIdIsNormal(replication_slot_catalog_xmin) && - NormalTransactionIdPrecedes(replication_slot_catalog_xmin, RecentGlobalXmin)) - RecentGlobalXmin = replication_slot_catalog_xmin; + /* + * Rows in non-shared, non-catalog tables possibly could be vacuumed + * if older than this xid. + */ + def_vis_xid = def_vis_xid_data; + + /* + * Check whether there's a replication slot requiring an older catalog + * xmin. + */ + def_vis_xid = + TransactionIdOlder(replication_slot_catalog_xmin, def_vis_xid); + + def_vis_fxid = FullXidViaRelative(latest_completed, def_vis_xid); + def_vis_fxid_data = FullXidViaRelative(latest_completed, def_vis_xid_data); + + /* + * Check if we can increase upper bound. As a previous + * GlobalVisUpdate() might have computed more aggressive values, don't + * overwrite them if so. + */ + GlobalVisSharedRels.definitely_needed = + FullTransactionIdNewer(def_vis_fxid, + GlobalVisSharedRels.definitely_needed); + GlobalVisCatalogRels.definitely_needed = + FullTransactionIdNewer(def_vis_fxid, + GlobalVisCatalogRels.definitely_needed); + GlobalVisDataRels.definitely_needed = + FullTransactionIdNewer(def_vis_fxid_data, + GlobalVisDataRels.definitely_needed); + + /* + * Check if we know that we can initialize or increase the lower + * bound. Currently the only cheap way to do so is to use + * ShmemVariableCache->oldestXid as input. + * + * We should definitely be able to do better. We could e.g. put a + * global lower bound value into ShmemVariableCache. + */ + GlobalVisSharedRels.maybe_needed = + FullTransactionIdNewer(GlobalVisSharedRels.maybe_needed, + oldestfxid); + GlobalVisCatalogRels.maybe_needed = + FullTransactionIdNewer(GlobalVisCatalogRels.maybe_needed, + oldestfxid); + GlobalVisDataRels.maybe_needed = + FullTransactionIdNewer(GlobalVisDataRels.maybe_needed, + oldestfxid); + } RecentXmin = xmin; @@ -1984,7 +2359,7 @@ GetRunningTransactionData(void) LWLockAcquire(ProcArrayLock, LW_SHARED); LWLockAcquire(XidGenLock, LW_SHARED); - latestCompletedXid = ShmemVariableCache->latestCompletedXid; + latestCompletedXid = XidFromFullTransactionId(ShmemVariableCache->latestCompletedFullXid); oldestRunningXid = XidFromFullTransactionId(ShmemVariableCache->nextFullXid); @@ -3207,9 +3582,7 @@ XidCacheRemoveRunningXids(TransactionId xid, elog(WARNING, "did not find subXID %u in MyProc", xid); /* Also advance global latestCompletedXid while holding the lock */ - if (TransactionIdPrecedes(ShmemVariableCache->latestCompletedXid, - latestXid)) - ShmemVariableCache->latestCompletedXid = latestXid; + MaintainLatestCompletedXid(latestXid); LWLockRelease(ProcArrayLock); } @@ -3236,6 +3609,276 @@ DisplayXidCache(void) } #endif /* XIDCACHE_DEBUG */ +/* + * If rel != NULL, return test state appropriate for relation, otherwise + * return state usable for all relations. The latter may consider XIDs as + * not-yet-visible-to-everyone that a state for a specific relation would + * already consider visible-to-everyone. + * + * This needs to be called while a snapshot is active or registered, otherwise + * there are wraparound and other dangers. + * + * See comment for GlobalVisState for details. + */ +GlobalVisState * +GlobalVisTestFor(Relation rel) +{ + bool need_shared; + bool need_catalog; + GlobalVisState *state; + + /* XXX: we should assert that a snapshot is pushed or registered */ + Assert(RecentXmin); + + if (!rel) + need_shared = need_catalog = true; + else + { + /* + * Other kinds currently don't contain xids, nor always the necessary + * logical decoding markers. + */ + Assert(rel->rd_rel->relkind == RELKIND_RELATION || + rel->rd_rel->relkind == RELKIND_MATVIEW || + rel->rd_rel->relkind == RELKIND_TOASTVALUE); + + need_shared = rel->rd_rel->relisshared || RecoveryInProgress(); + need_catalog = IsCatalogRelation(rel) || RelationIsAccessibleInLogicalDecoding(rel); + } + + if (need_shared) + state = &GlobalVisSharedRels; + else if (need_catalog) + state = &GlobalVisCatalogRels; + else + state = &GlobalVisDataRels; + + Assert(FullTransactionIdIsValid(state->definitely_needed) && + FullTransactionIdIsValid(state->maybe_needed)); + + return state; +} + +/* + * Return true if it's worth updating the accurate maybe_needed boundary. + * + * As it is somewhat expensive to determine xmin horizons, we don't want to + * repeatedly do so when there is a low likelihood of it being beneficial. + * + * The current heuristic is that we update only if RecentXmin has changed + * since the last update. If the oldest currently running transaction has not + * finished, it is unlikely that recomputing the horizon would be useful. + */ +static bool +GlobalVisTestShouldUpdate(GlobalVisState *state) +{ + /* hasn't been updated yet */ + if (!TransactionIdIsValid(ComputeXidHorizonsResultLastXmin)) + return true; + + /* + * If the maybe_needed/definitely_needed boundaries are the same, it's + * unlikely to be beneficial to refresh boundaries. + */ + if (FullTransactionIdFollowsOrEquals(state->maybe_needed, + state->definitely_needed)) + return false; + + /* does the last snapshot built have a different xmin? */ + return RecentXmin != ComputeXidHorizonsResultLastXmin; +} + +/* + * Update boundaries in GlobalVis{Shared,Catalog, Data}Rels + * using ComputeXidHorizons(). + */ +static void +GlobalVisUpdate(void) +{ + ComputeXidHorizonsResult horizons; + + ComputeXidHorizons(&horizons); + + GlobalVisSharedRels.maybe_needed = + FullXidViaRelative(horizons.latest_completed, + horizons.shared_oldest_nonremovable); + GlobalVisCatalogRels.maybe_needed = + FullXidViaRelative(horizons.latest_completed, + horizons.catalog_oldest_nonremovable); + GlobalVisDataRels.maybe_needed = + FullXidViaRelative(horizons.latest_completed, + horizons.data_oldest_nonremovable); + + /* + * In longer running transactions it's possible that transactions we + * previously needed to treat as running aren't around anymore. So update + * definitely_needed to not be earlier than maybe_needed. + */ + GlobalVisSharedRels.definitely_needed = + FullTransactionIdNewer(GlobalVisSharedRels.maybe_needed, + GlobalVisSharedRels.definitely_needed); + GlobalVisCatalogRels.definitely_needed = + FullTransactionIdNewer(GlobalVisCatalogRels.maybe_needed, + GlobalVisCatalogRels.definitely_needed); + GlobalVisDataRels.definitely_needed = + FullTransactionIdNewer(GlobalVisDataRels.maybe_needed, + GlobalVisDataRels.definitely_needed); + + ComputeXidHorizonsResultLastXmin = RecentXmin; +} + +/* + * Return true if no snapshot still considers fxid to be running. + * + * The state passed needs to have been initialized for the relation fxid is + * from (NULL is also OK), otherwise the result may not be correct. + * + * See comment for GlobalVisState for details. + */ +bool +GlobalVisTestIsRemovableFullXid(GlobalVisState *state, + FullTransactionId fxid) +{ + /* + * If fxid is older than maybe_needed bound, it definitely is visible to + * everyone. + */ + if (FullTransactionIdPrecedes(fxid, state->maybe_needed)) + return true; + + /* + * If fxid is >= definitely_needed bound, it is very likely to still be + * considered running. + */ + if (FullTransactionIdFollowsOrEquals(fxid, state->definitely_needed)) + return false; + + /* + * fxid is between maybe_needed and definitely_needed, i.e. there might or + * might not exist a snapshot considering fxid running. If it makes sense, + * update boundaries and recheck. + */ + if (GlobalVisTestShouldUpdate(state)) + { + GlobalVisUpdate(); + + Assert(FullTransactionIdPrecedes(fxid, state->definitely_needed)); + + return FullTransactionIdPrecedes(fxid, state->maybe_needed); + } + else + return false; +} + +/* + * Wrapper around GlobalVisTestIsRemovableFullXid() for 32bit xids. + * + * It is crucial that this only gets called for xids from a source that + * protects against xid wraparounds (e.g. from a table and thus protected by + * relfrozenxid). + */ +bool +GlobalVisTestIsRemovableXid(GlobalVisState *state, TransactionId xid) +{ + FullTransactionId fxid; + + /* + * Convert 32 bit argument to FullTransactionId. We can do so safely + * because we know the xid has to, at the very least, be between + * [oldestXid, nextFullXid), i.e. within 2 billion of xid. To avoid taking + * a lock to determine either, we can just compare with + * state->definitely_needed, which was based on those value at the time + * the current snapshot was built. + */ + fxid = FullXidViaRelative(state->definitely_needed, xid); + + return GlobalVisTestIsRemovableFullXid(state, fxid); +} + +/* + * Return FullTransactionId below which all transactions are not considered + * running anymore. + * + * Note: This is less efficient than testing with + * GlobalVisTestIsRemovableFullXid as it likely requires building an accurate + * cutoff, even in the case all the XIDs compared with the cutoff are outside + * [maybe_needed, definitely_needed). + */ +FullTransactionId +GlobalVisTestNonRemovableFullHorizon(GlobalVisState *state) +{ + /* acquire accurate horizon if not already done */ + if (GlobalVisTestShouldUpdate(state)) + GlobalVisUpdate(); + + return state->maybe_needed; +} + +/* Convenience wrapper around GlobalVisTestNonRemovableFullHorizon */ +TransactionId +GlobalVisTestNonRemovableHorizon(GlobalVisState *state) +{ + FullTransactionId cutoff; + + cutoff = GlobalVisTestNonRemovableFullHorizon(state); + + return XidFromFullTransactionId(cutoff); +} + +/* + * Convenience wrapper around GlobalVisTestFor() and + * GlobalVisTestIsRemovableFullXid(), see their comments. + */ +bool +GlobalVisIsRemovableFullXid(Relation rel, FullTransactionId fxid) +{ + GlobalVisState *state; + + state = GlobalVisTestFor(rel); + + return GlobalVisTestIsRemovableFullXid(state, fxid); +} + +/* + * Convenience wrapper around GlobalVisTestFor() and + * GlobalVisTestIsRemovableXid(), see their comments. + */ +bool +GlobalVisCheckRemovableXid(Relation rel, TransactionId xid) +{ + GlobalVisState *state; + + state = GlobalVisTestFor(rel); + + return GlobalVisTestIsRemovableXid(state, xid); +} + +/* + * Convert a 32 bit transaction id into 64 bit transaction id, by assuming it + * is within MaxTransactionId / 2 of XidFromFullTransactionId(rel). + * + * Be very careful about when to use this function. It can only safely be used + * when there is a guarantee that xid is within MaxTransactionId / 2 xids of + * rel. That e.g. can be guaranteed if the the caller assures a snapshot is + * held by the backend and xid is from a table (where vacuum/freezing ensures + * the xid has to be within that range), or if xid is from the procarray and + * prevents xid wraparound that way. + */ +static inline FullTransactionId +FullXidViaRelative(FullTransactionId rel, TransactionId xid) +{ + TransactionId rel_xid = XidFromFullTransactionId(rel); + + Assert(TransactionIdIsValid(xid)); + Assert(TransactionIdIsValid(rel_xid)); + + /* not guaranteed to find issues, but likely to catch mistakes */ + AssertTransactionInAllowableRange(xid); + + return FullTransactionIdFromU64(U64FromFullTransactionId(rel) + + (int32) (xid - rel_xid)); +} + /* ---------------------------------------------- * KnownAssignedTransactionIds sub-module @@ -3388,9 +4031,7 @@ ExpireTreeKnownAssignedTransactionIds(TransactionId xid, int nsubxids, KnownAssignedXidsRemoveTree(xid, nsubxids, subxids); /* As in ProcArrayEndTransaction, advance latestCompletedXid */ - if (TransactionIdPrecedes(ShmemVariableCache->latestCompletedXid, - max_xid)) - ShmemVariableCache->latestCompletedXid = max_xid; + MaintainLatestCompletedXidRecovery(max_xid); LWLockRelease(ProcArrayLock); } diff --git a/src/backend/utils/adt/selfuncs.c b/src/backend/utils/adt/selfuncs.c index be08eb48148..2d4ec92c2a1 100644 --- a/src/backend/utils/adt/selfuncs.c +++ b/src/backend/utils/adt/selfuncs.c @@ -5783,14 +5783,15 @@ get_actual_variable_endpoint(Relation heapRel, * recent); that case motivates not using SnapshotAny here. * * A crucial point here is that SnapshotNonVacuumable, with - * RecentGlobalXmin as horizon, yields the inverse of the condition that - * the indexscan will use to decide that index entries are killable (see - * heap_hot_search_buffer()). Therefore, if the snapshot rejects a tuple - * (or more precisely, all tuples of a HOT chain) and we have to continue - * scanning past it, we know that the indexscan will mark that index entry - * killed. That means that the next get_actual_variable_endpoint() call - * will not have to re-consider that index entry. In this way we avoid - * repetitive work when this function is used a lot during planning. + * GlobalVisTestFor(heapRel) as horizon, yields the inverse of the + * condition that the indexscan will use to decide that index entries are + * killable (see heap_hot_search_buffer()). Therefore, if the snapshot + * rejects a tuple (or more precisely, all tuples of a HOT chain) and we + * have to continue scanning past it, we know that the indexscan will mark + * that index entry killed. That means that the next + * get_actual_variable_endpoint() call will not have to re-consider that + * index entry. In this way we avoid repetitive work when this function + * is used a lot during planning. * * But using SnapshotNonVacuumable creates a hazard of its own. In a * recently-created index, some index entries may point at "broken" HOT @@ -5802,7 +5803,8 @@ get_actual_variable_endpoint(Relation heapRel, * or could even be NULL. We avoid this hazard because we take the data * from the index entry not the heap. */ - InitNonVacuumableSnapshot(SnapshotNonVacuumable, RecentGlobalXmin); + InitNonVacuumableSnapshot(SnapshotNonVacuumable, + GlobalVisTestFor(heapRel)); index_scan = index_beginscan(heapRel, indexRel, &SnapshotNonVacuumable, diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c index f4247ea70d5..893be2f3ddb 100644 --- a/src/backend/utils/init/postinit.c +++ b/src/backend/utils/init/postinit.c @@ -722,6 +722,10 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username, * is critical for anything that reads heap pages, because HOT may decide * to prune them even if the process doesn't attempt to modify any * tuples.) + * + * FIXME: This comment is inaccurate / the code buggy. A snapshot that is + * not pushed/active does not reliably prevent HOT pruning (->xmin could + * e.g. be cleared when cache invalidations are processed). */ if (!bootstrap) { diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 6b6c8571e23..76578868cf9 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -157,16 +157,9 @@ static Snapshot HistoricSnapshot = NULL; * These are updated by GetSnapshotData. We initialize them this way * for the convenience of TransactionIdIsInProgress: even in bootstrap * mode, we don't want it to say that BootstrapTransactionId is in progress. - * - * RecentGlobalXmin and RecentGlobalDataXmin are initialized to - * InvalidTransactionId, to ensure that no one tries to use a stale - * value. Readers should ensure that it has been set to something else - * before using it. */ TransactionId TransactionXmin = FirstNormalTransactionId; TransactionId RecentXmin = FirstNormalTransactionId; -TransactionId RecentGlobalXmin = InvalidTransactionId; -TransactionId RecentGlobalDataXmin = InvalidTransactionId; /* (table, ctid) => (cmin, cmax) mapping during timetravel */ static HTAB *tuplecid_data = NULL; @@ -581,9 +574,7 @@ SetTransactionSnapshot(Snapshot sourcesnap, VirtualTransactionId *sourcevxid, * Even though we are not going to use the snapshot it computes, we must * call GetSnapshotData, for two reasons: (1) to be sure that * CurrentSnapshotData's XID arrays have been allocated, and (2) to update - * RecentXmin and RecentGlobalXmin. (We could alternatively include those - * two variables in exported snapshot files, but it seems better to have - * snapshot importers compute reasonably up-to-date values for them.) + * the state for GlobalVis*. */ CurrentSnapshot = GetSnapshotData(&CurrentSnapshotData); @@ -956,36 +947,6 @@ xmin_cmp(const pairingheap_node *a, const pairingheap_node *b, void *arg) return 0; } -/* - * Get current RecentGlobalXmin value, as a FullTransactionId. - */ -FullTransactionId -GetFullRecentGlobalXmin(void) -{ - FullTransactionId nextxid_full; - uint32 nextxid_epoch; - TransactionId nextxid_xid; - uint32 epoch; - - Assert(TransactionIdIsNormal(RecentGlobalXmin)); - - /* - * Compute the epoch from the next XID's epoch. This relies on the fact - * that RecentGlobalXmin must be within the 2 billion XID horizon from the - * next XID. - */ - nextxid_full = ReadNextFullTransactionId(); - nextxid_epoch = EpochFromFullTransactionId(nextxid_full); - nextxid_xid = XidFromFullTransactionId(nextxid_full); - - if (RecentGlobalXmin > nextxid_xid) - epoch = nextxid_epoch - 1; - else - epoch = nextxid_epoch; - - return FullTransactionIdFromEpochAndXid(epoch, RecentGlobalXmin); -} - /* * SnapshotResetXmin * @@ -1753,106 +1714,157 @@ GetOldSnapshotThresholdTimestamp(void) return threshold_timestamp; } -static void +void SetOldSnapshotThresholdTimestamp(TimestampTz ts, TransactionId xlimit) { SpinLockAcquire(&oldSnapshotControl->mutex_threshold); + Assert(oldSnapshotControl->threshold_timestamp <= ts); + Assert(TransactionIdPrecedesOrEquals(oldSnapshotControl->threshold_xid, xlimit)); oldSnapshotControl->threshold_timestamp = ts; oldSnapshotControl->threshold_xid = xlimit; SpinLockRelease(&oldSnapshotControl->mutex_threshold); } +/* + * XXX: Magic to keep old_snapshot_threshold tests appear "working". They + * currently are broken, and discussion of what to do about them is + * ongoing. See + * https://www.postgresql.org/message-id/20200403001235.e6jfdll3gh2ygbuc%40alap3.anarazel.de + */ +void +SnapshotTooOldMagicForTest(void) +{ + TimestampTz ts = GetSnapshotCurrentTimestamp(); + + Assert(old_snapshot_threshold == 0); + + ts -= 5 * USECS_PER_SEC; + + SpinLockAcquire(&oldSnapshotControl->mutex_threshold); + oldSnapshotControl->threshold_timestamp = ts; + SpinLockRelease(&oldSnapshotControl->mutex_threshold); +} + +/* + * If there is a valid mapping for the timestamp, set *xlimitp to + * that. Returns whether there is such a mapping. + */ +static bool +GetOldSnapshotFromTimeMapping(TimestampTz ts, TransactionId *xlimitp) +{ + bool in_mapping = false; + + Assert(ts == AlignTimestampToMinuteBoundary(ts)); + + LWLockAcquire(OldSnapshotTimeMapLock, LW_SHARED); + + if (oldSnapshotControl->count_used > 0 + && ts >= oldSnapshotControl->head_timestamp) + { + int offset; + + offset = ((ts - oldSnapshotControl->head_timestamp) + / USECS_PER_MINUTE); + if (offset > oldSnapshotControl->count_used - 1) + offset = oldSnapshotControl->count_used - 1; + offset = (oldSnapshotControl->head_offset + offset) + % OLD_SNAPSHOT_TIME_MAP_ENTRIES; + + *xlimitp = oldSnapshotControl->xid_by_minute[offset]; + + in_mapping = true; + } + + LWLockRelease(OldSnapshotTimeMapLock); + + return in_mapping; +} + /* * TransactionIdLimitedForOldSnapshots * - * Apply old snapshot limit, if any. This is intended to be called for page - * pruning and table vacuuming, to allow old_snapshot_threshold to override - * the normal global xmin value. Actual testing for snapshot too old will be - * based on whether a snapshot timestamp is prior to the threshold timestamp - * set in this function. + * Apply old snapshot limit. This is intended to be called for page pruning + * and table vacuuming, to allow old_snapshot_threshold to override the normal + * global xmin value. Actual testing for snapshot too old will be based on + * whether a snapshot timestamp is prior to the threshold timestamp set in + * this function. + * + * If the limited horizon allows a cleanup action that otherwise would not be + * possible, SetOldSnapshotThresholdTimestamp(*limit_ts, *limit_xid) needs to + * be called before that cleanup action. */ -TransactionId +bool TransactionIdLimitedForOldSnapshots(TransactionId recentXmin, - Relation relation) + Relation relation, + TransactionId *limit_xid, + TimestampTz *limit_ts) { - if (TransactionIdIsNormal(recentXmin) - && old_snapshot_threshold >= 0 - && RelationAllowsEarlyPruning(relation)) + TimestampTz ts; + TransactionId xlimit = recentXmin; + TransactionId latest_xmin; + TimestampTz next_map_update_ts; + TransactionId threshold_timestamp; + TransactionId threshold_xid; + + Assert(TransactionIdIsNormal(recentXmin)); + Assert(OldSnapshotThresholdActive()); + Assert(limit_ts != NULL && limit_xid != NULL); + + if (!RelationAllowsEarlyPruning(relation)) + return false; + + ts = GetSnapshotCurrentTimestamp(); + + SpinLockAcquire(&oldSnapshotControl->mutex_latest_xmin); + latest_xmin = oldSnapshotControl->latest_xmin; + next_map_update_ts = oldSnapshotControl->next_map_update; + SpinLockRelease(&oldSnapshotControl->mutex_latest_xmin); + + /* + * Zero threshold always overrides to latest xmin, if valid. Without + * some heuristic it will find its own snapshot too old on, for + * example, a simple UPDATE -- which would make it useless for most + * testing, but there is no principled way to ensure that it doesn't + * fail in this way. Use a five-second delay to try to get useful + * testing behavior, but this may need adjustment. + */ + if (old_snapshot_threshold == 0) { - TimestampTz ts = GetSnapshotCurrentTimestamp(); - TransactionId xlimit = recentXmin; - TransactionId latest_xmin; - TimestampTz update_ts; - bool same_ts_as_threshold = false; - - SpinLockAcquire(&oldSnapshotControl->mutex_latest_xmin); - latest_xmin = oldSnapshotControl->latest_xmin; - update_ts = oldSnapshotControl->next_map_update; - SpinLockRelease(&oldSnapshotControl->mutex_latest_xmin); - - /* - * Zero threshold always overrides to latest xmin, if valid. Without - * some heuristic it will find its own snapshot too old on, for - * example, a simple UPDATE -- which would make it useless for most - * testing, but there is no principled way to ensure that it doesn't - * fail in this way. Use a five-second delay to try to get useful - * testing behavior, but this may need adjustment. - */ - if (old_snapshot_threshold == 0) - { - if (TransactionIdPrecedes(latest_xmin, MyPgXact->xmin) - && TransactionIdFollows(latest_xmin, xlimit)) - xlimit = latest_xmin; - - ts -= 5 * USECS_PER_SEC; - SetOldSnapshotThresholdTimestamp(ts, xlimit); - - return xlimit; - } + if (TransactionIdPrecedes(latest_xmin, MyPgXact->xmin) + && TransactionIdFollows(latest_xmin, xlimit)) + xlimit = latest_xmin; + ts -= 5 * USECS_PER_SEC; + } + else + { ts = AlignTimestampToMinuteBoundary(ts) - (old_snapshot_threshold * USECS_PER_MINUTE); /* Check for fast exit without LW locking. */ SpinLockAcquire(&oldSnapshotControl->mutex_threshold); - if (ts == oldSnapshotControl->threshold_timestamp) - { - xlimit = oldSnapshotControl->threshold_xid; - same_ts_as_threshold = true; - } + threshold_timestamp = oldSnapshotControl->threshold_timestamp; + threshold_xid = oldSnapshotControl->threshold_xid; SpinLockRelease(&oldSnapshotControl->mutex_threshold); - if (!same_ts_as_threshold) + if (ts == threshold_timestamp) + { + /* + * Current timestamp is in same bucket as the the last limit that + * was applied. Reuse. + */ + xlimit = threshold_xid; + } + else if (ts == next_map_update_ts) + { + /* + * FIXME: This branch is super iffy - but that should probably + * fixed separately. + */ + xlimit = latest_xmin; + } + else if (GetOldSnapshotFromTimeMapping(ts, &xlimit)) { - if (ts == update_ts) - { - xlimit = latest_xmin; - if (NormalTransactionIdFollows(xlimit, recentXmin)) - SetOldSnapshotThresholdTimestamp(ts, xlimit); - } - else - { - LWLockAcquire(OldSnapshotTimeMapLock, LW_SHARED); - - if (oldSnapshotControl->count_used > 0 - && ts >= oldSnapshotControl->head_timestamp) - { - int offset; - - offset = ((ts - oldSnapshotControl->head_timestamp) - / USECS_PER_MINUTE); - if (offset > oldSnapshotControl->count_used - 1) - offset = oldSnapshotControl->count_used - 1; - offset = (oldSnapshotControl->head_offset + offset) - % OLD_SNAPSHOT_TIME_MAP_ENTRIES; - xlimit = oldSnapshotControl->xid_by_minute[offset]; - - if (NormalTransactionIdFollows(xlimit, recentXmin)) - SetOldSnapshotThresholdTimestamp(ts, xlimit); - } - - LWLockRelease(OldSnapshotTimeMapLock); - } } /* @@ -1867,12 +1879,18 @@ TransactionIdLimitedForOldSnapshots(TransactionId recentXmin, if (TransactionIdIsNormal(latest_xmin) && TransactionIdPrecedes(latest_xmin, xlimit)) xlimit = latest_xmin; - - if (NormalTransactionIdFollows(xlimit, recentXmin)) - return xlimit; } - return recentXmin; + if (TransactionIdIsValid(xlimit) && + TransactionIdFollowsOrEquals(xlimit, recentXmin)) + { + *limit_ts = ts; + *limit_xid = xlimit; + + return true; + } + + return false; } /* diff --git a/contrib/amcheck/verify_nbtree.c b/contrib/amcheck/verify_nbtree.c index e4d501a85d1..76306976c2a 100644 --- a/contrib/amcheck/verify_nbtree.c +++ b/contrib/amcheck/verify_nbtree.c @@ -419,10 +419,10 @@ bt_check_every_level(Relation rel, Relation heaprel, bool heapkeyspace, RelationGetRelationName(rel)); /* - * RecentGlobalXmin assertion matches index_getnext_tid(). See note on - * RecentGlobalXmin/B-Tree page deletion. + * This assertion matches the one in index_getnext_tid(). See page + * recycling/"visible to everyone" notes in nbtree README. */ - Assert(TransactionIdIsValid(RecentGlobalXmin)); + Assert(TransactionIdIsValid(RecentXmin)); /* * Initialize state for entire verification operation @@ -1441,7 +1441,7 @@ bt_right_page_check_scankey(BtreeCheckState *state) * does not occur until no possible index scan could land on the page. * Index scans can follow links with nothing more than their snapshot as * an interlock and be sure of at least that much. (See page - * recycling/RecentGlobalXmin notes in nbtree README.) + * recycling/"visible to everyone" notes in nbtree README.) * * Furthermore, it's okay if we follow a rightlink and find a half-dead or * dead (ignorable) page one or more times. There will either be a diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c index 68d580ed1e0..37206c50a21 100644 --- a/contrib/pg_visibility/pg_visibility.c +++ b/contrib/pg_visibility/pg_visibility.c @@ -563,17 +563,14 @@ collect_corrupt_items(Oid relid, bool all_visible, bool all_frozen) BufferAccessStrategy bstrategy = GetAccessStrategy(BAS_BULKREAD); TransactionId OldestXmin = InvalidTransactionId; - if (all_visible) - { - /* Don't pass rel; that will fail in recovery. */ - OldestXmin = GetOldestXmin(NULL, PROCARRAY_FLAGS_VACUUM); - } - rel = relation_open(relid, AccessShareLock); /* Only some relkinds have a visibility map */ check_relation_relkind(rel); + if (all_visible) + OldestXmin = GetOldestNonRemovableTransactionId(rel); + nblocks = RelationGetNumberOfBlocks(rel); /* @@ -679,11 +676,12 @@ collect_corrupt_items(Oid relid, bool all_visible, bool all_frozen) * From a concurrency point of view, it sort of sucks to * retake ProcArrayLock here while we're holding the buffer * exclusively locked, but it should be safe against - * deadlocks, because surely GetOldestXmin() should never take - * a buffer lock. And this shouldn't happen often, so it's - * worth being careful so as to avoid false positives. + * deadlocks, because surely GetOldestNonRemovableTransactionId() + * should never take a buffer lock. And this shouldn't happen + * often, so it's worth being careful so as to avoid false + * positives. */ - RecomputedOldestXmin = GetOldestXmin(NULL, PROCARRAY_FLAGS_VACUUM); + RecomputedOldestXmin = GetOldestNonRemovableTransactionId(rel); if (!TransactionIdPrecedes(OldestXmin, RecomputedOldestXmin)) record_corrupt_item(items, &tuple.t_self); diff --git a/contrib/pgstattuple/pgstatapprox.c b/contrib/pgstattuple/pgstatapprox.c index dbc0fa11f61..3a99333d443 100644 --- a/contrib/pgstattuple/pgstatapprox.c +++ b/contrib/pgstattuple/pgstatapprox.c @@ -71,7 +71,7 @@ statapprox_heap(Relation rel, output_type *stat) BufferAccessStrategy bstrategy; TransactionId OldestXmin; - OldestXmin = GetOldestXmin(rel, PROCARRAY_FLAGS_VACUUM); + OldestXmin = GetOldestNonRemovableTransactionId(rel); bstrategy = GetAccessStrategy(BAS_BULKREAD); nblocks = RelationGetNumberOfBlocks(rel); diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 7eaaad1e140..b4948ac675f 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -395,6 +395,7 @@ CompositeTypeStmt CompoundAffixFlag CompressionAlgorithm CompressorState +ComputeXidHorizonsResult ConditionVariable ConditionalStack ConfigData @@ -930,6 +931,7 @@ GistSplitVector GistTsVectorOptions GistVacState GlobalTransaction +GlobalVisState GrantRoleStmt GrantStmt GrantTargetType -- 2.25.0.114.g5b0ca878e0 --kdyesixubxkl3b7r Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v11-0002-snapshot-scalability-Move-PGXACT-xmin-back-to-PG.patch" From ea877d637845b5941b7cbe63214c50334785f251 Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Wed, 15 Jul 2020 15:35:07 -0700 Subject: [PATCH v11 2/6] snapshot scalability: Move PGXACT->xmin back to PGPROC. Now that xmin isn't needed for GetSnapshotData() anymore, it leads to unnecessary cacheline ping-pong to have it in PGXACT as it is updated more frequently than the other PGXACT members. Author: Andres Freund Reviewed-By: Robert Haas, Thomas Munro, David Rowley Discussion: https://postgr.es/m/20200301083601.ews6hz5dduc3w2se@alap3.anarazel.de --- src/include/storage/proc.h | 10 +++--- src/backend/access/gist/gistxlog.c | 2 +- src/backend/access/nbtree/nbtpage.c | 2 +- src/backend/access/transam/README | 2 +- src/backend/access/transam/twophase.c | 2 +- src/backend/commands/indexcmds.c | 2 +- src/backend/replication/logical/snapbuild.c | 6 ++-- src/backend/replication/walsender.c | 10 +++--- src/backend/storage/ipc/procarray.c | 36 +++++++++------------ src/backend/storage/ipc/sinvaladt.c | 2 +- src/backend/storage/lmgr/proc.c | 4 +-- src/backend/utils/time/snapmgr.c | 28 ++++++++-------- 12 files changed, 51 insertions(+), 55 deletions(-) diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h index 08f006f782e..286c9a9aec3 100644 --- a/src/include/storage/proc.h +++ b/src/include/storage/proc.h @@ -102,6 +102,11 @@ struct PGPROC Latch procLatch; /* generic latch for process */ + TransactionId xmin; /* minimal running XID as it was when we were + * starting our xact, excluding LAZY VACUUM: + * vacuum must not remove tuples deleted by + * xid >= xmin ! */ + LocalTransactionId lxid; /* local id of top-level transaction currently * being executed by this proc, if running; * else InvalidLocalTransactionId */ @@ -224,11 +229,6 @@ typedef struct PGXACT * executed by this proc, if running and XID * is assigned; else InvalidTransactionId */ - TransactionId xmin; /* minimal running XID as it was when we were - * starting our xact, excluding LAZY VACUUM: - * vacuum must not remove tuples deleted by - * xid >= xmin ! */ - uint8 vacuumFlags; /* vacuum-related flags, see above */ bool overflowed; diff --git a/src/backend/access/gist/gistxlog.c b/src/backend/access/gist/gistxlog.c index 3167305ac00..b6603cd73cf 100644 --- a/src/backend/access/gist/gistxlog.c +++ b/src/backend/access/gist/gistxlog.c @@ -389,7 +389,7 @@ gistRedoPageReuse(XLogReaderState *record) * * latestRemovedXid was the page's deleteXid. The * GlobalVisIsRemovableFullXid(deleteXid) test in gistPageRecyclable() - * conceptually mirrors the pgxact->xmin > limitXmin test in + * conceptually mirrors the PGPROC->xmin > limitXmin test in * GetConflictingVirtualXIDs(). Consequently, one XID value achieves the * same exclusion effect on primary and standby. */ diff --git a/src/backend/access/nbtree/nbtpage.c b/src/backend/access/nbtree/nbtpage.c index 9e6376f2c2b..c88ca4221a4 100644 --- a/src/backend/access/nbtree/nbtpage.c +++ b/src/backend/access/nbtree/nbtpage.c @@ -2208,7 +2208,7 @@ _bt_unlink_halfdead_page(Relation rel, Buffer leafbuf, BlockNumber scanblkno, * we're in VACUUM and would not otherwise have an XID. Having already * updated links to the target, ReadNewTransactionId() suffices as an * upper bound. Any scan having retained a now-stale link is advertising - * in its PGXACT an xmin less than or equal to the value we read here. It + * in its PGPROC an xmin less than or equal to the value we read here. It * will continue to do so, holding back the xmin horizon, for the duration * of that scan. */ diff --git a/src/backend/access/transam/README b/src/backend/access/transam/README index 4e2178dabab..94d8f3fd0a2 100644 --- a/src/backend/access/transam/README +++ b/src/backend/access/transam/README @@ -331,7 +331,7 @@ necessary. Note that while it is certain that two concurrent executions of GetSnapshotData will compute the same xmin for their own snapshots, there is no such guarantee for the horizons computed by ComputeXidHorizons. This is -because we allow XID-less transactions to clear their MyPgXact->xmin +because we allow XID-less transactions to clear their MyProc->xmin asynchronously (without taking ProcArrayLock), so one execution might see what had been the oldest xmin, and another not. This is OK since the thresholds need only be a valid lower bound. As noted above, we are already diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c index 9b2e59bf0ec..ae7c1a4c172 100644 --- a/src/backend/access/transam/twophase.c +++ b/src/backend/access/transam/twophase.c @@ -464,7 +464,7 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid, /* We set up the gxact's VXID as InvalidBackendId/XID */ proc->lxid = (LocalTransactionId) xid; pgxact->xid = xid; - pgxact->xmin = InvalidTransactionId; + Assert(proc->xmin == InvalidTransactionId); proc->delayChkpt = false; pgxact->vacuumFlags = 0; proc->pid = 0; diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c index 2baca12c5f4..9d741aa03fa 100644 --- a/src/backend/commands/indexcmds.c +++ b/src/backend/commands/indexcmds.c @@ -1535,7 +1535,7 @@ DefineIndex(Oid relationId, StartTransactionCommand(); /* We should now definitely not be advertising any xmin. */ - Assert(MyPgXact->xmin == InvalidTransactionId); + Assert(MyProc->xmin == InvalidTransactionId); /* * The index is now valid in the sense that it contains all currently diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index 3089f0d5ddc..e9701ea7221 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -553,8 +553,8 @@ SnapBuildInitialSnapshot(SnapBuild *builder) elog(ERROR, "cannot build an initial slot snapshot, not all transactions are monitored anymore"); /* so we don't overwrite the existing value */ - if (TransactionIdIsValid(MyPgXact->xmin)) - elog(ERROR, "cannot build an initial slot snapshot when MyPgXact->xmin already is valid"); + if (TransactionIdIsValid(MyProc->xmin)) + elog(ERROR, "cannot build an initial slot snapshot when MyProc->xmin already is valid"); snap = SnapBuildBuildSnapshot(builder); @@ -575,7 +575,7 @@ SnapBuildInitialSnapshot(SnapBuild *builder) } #endif - MyPgXact->xmin = snap->xmin; + MyProc->xmin = snap->xmin; /* allocate in transaction context */ newxip = (TransactionId *) diff --git a/src/backend/replication/walsender.c b/src/backend/replication/walsender.c index fd370d52b66..06da4b4352a 100644 --- a/src/backend/replication/walsender.c +++ b/src/backend/replication/walsender.c @@ -1967,7 +1967,7 @@ PhysicalReplicationSlotNewXmin(TransactionId feedbackXmin, TransactionId feedbac ReplicationSlot *slot = MyReplicationSlot; SpinLockAcquire(&slot->mutex); - MyPgXact->xmin = InvalidTransactionId; + MyProc->xmin = InvalidTransactionId; /* * For physical replication we don't need the interlock provided by xmin @@ -2096,7 +2096,7 @@ ProcessStandbyHSFeedbackMessage(void) if (!TransactionIdIsNormal(feedbackXmin) && !TransactionIdIsNormal(feedbackCatalogXmin)) { - MyPgXact->xmin = InvalidTransactionId; + MyProc->xmin = InvalidTransactionId; if (MyReplicationSlot != NULL) PhysicalReplicationSlotNewXmin(feedbackXmin, feedbackCatalogXmin); return; @@ -2138,7 +2138,7 @@ ProcessStandbyHSFeedbackMessage(void) * risk already since a VACUUM could already have determined the horizon.) * * If we're using a replication slot we reserve the xmin via that, - * otherwise via the walsender's PGXACT entry. We can only track the + * otherwise via the walsender's PGPROC entry. We can only track the * catalog xmin separately when using a slot, so we store the least of the * two provided when not using a slot. * @@ -2151,9 +2151,9 @@ ProcessStandbyHSFeedbackMessage(void) { if (TransactionIdIsNormal(feedbackCatalogXmin) && TransactionIdPrecedes(feedbackCatalogXmin, feedbackXmin)) - MyPgXact->xmin = feedbackCatalogXmin; + MyProc->xmin = feedbackCatalogXmin; else - MyPgXact->xmin = feedbackXmin; + MyProc->xmin = feedbackXmin; } } diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c index c011387ba90..980ca2cc2af 100644 --- a/src/backend/storage/ipc/procarray.c +++ b/src/backend/storage/ipc/procarray.c @@ -585,9 +585,9 @@ ProcArrayEndTransaction(PGPROC *proc, TransactionId latestXid) Assert(!TransactionIdIsValid(allPgXact[proc->pgprocno].xid)); proc->lxid = InvalidLocalTransactionId; - pgxact->xmin = InvalidTransactionId; /* must be cleared with xid/xmin: */ pgxact->vacuumFlags &= ~PROC_VACUUM_STATE_MASK; + proc->xmin = InvalidTransactionId; proc->delayChkpt = false; /* be sure this is cleared in abort */ proc->recoveryConflictPending = false; @@ -607,9 +607,9 @@ ProcArrayEndTransactionInternal(PGPROC *proc, PGXACT *pgxact, { pgxact->xid = InvalidTransactionId; proc->lxid = InvalidLocalTransactionId; - pgxact->xmin = InvalidTransactionId; /* must be cleared with xid/xmin: */ pgxact->vacuumFlags &= ~PROC_VACUUM_STATE_MASK; + proc->xmin = InvalidTransactionId; proc->delayChkpt = false; /* be sure this is cleared in abort */ proc->recoveryConflictPending = false; @@ -761,7 +761,7 @@ ProcArrayClearTransaction(PGPROC *proc) */ pgxact->xid = InvalidTransactionId; proc->lxid = InvalidLocalTransactionId; - pgxact->xmin = InvalidTransactionId; + proc->xmin = InvalidTransactionId; proc->recoveryConflictPending = false; /* redundant, but just in case */ @@ -1550,7 +1550,7 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h) /* Fetch xid just once - see GetNewTransactionId */ xid = UINT32_ACCESS_ONCE(pgxact->xid); - xmin = UINT32_ACCESS_ONCE(pgxact->xmin); + xmin = UINT32_ACCESS_ONCE(proc->xmin); /* * Consider both the transaction's Xmin, and its Xid. @@ -1822,7 +1822,7 @@ GetMaxSnapshotSubxidCount(void) * * We also update the following backend-global variables: * TransactionXmin: the oldest xmin of any snapshot in use in the - * current transaction (this is the same as MyPgXact->xmin). + * current transaction (this is the same as MyProc->xmin). * RecentXmin: the xmin computed for the most recent snapshot. XIDs * older than this are known not running any more. * @@ -1884,7 +1884,7 @@ GetSnapshotData(Snapshot snapshot) /* * It is sufficient to get shared lock on ProcArrayLock, even if we are - * going to set MyPgXact->xmin. + * going to set MyProc->xmin. */ LWLockAcquire(ProcArrayLock, LW_SHARED); @@ -2036,8 +2036,8 @@ GetSnapshotData(Snapshot snapshot) replication_slot_xmin = procArray->replication_slot_xmin; replication_slot_catalog_xmin = procArray->replication_slot_catalog_xmin; - if (!TransactionIdIsValid(MyPgXact->xmin)) - MyPgXact->xmin = TransactionXmin = xmin; + if (!TransactionIdIsValid(MyProc->xmin)) + MyProc->xmin = TransactionXmin = xmin; LWLockRelease(ProcArrayLock); @@ -2157,7 +2157,7 @@ GetSnapshotData(Snapshot snapshot) } /* - * ProcArrayInstallImportedXmin -- install imported xmin into MyPgXact->xmin + * ProcArrayInstallImportedXmin -- install imported xmin into MyProc->xmin * * This is called when installing a snapshot imported from another * transaction. To ensure that OldestXmin doesn't go backwards, we must @@ -2210,7 +2210,7 @@ ProcArrayInstallImportedXmin(TransactionId xmin, /* * Likewise, let's just make real sure its xmin does cover us. */ - xid = UINT32_ACCESS_ONCE(pgxact->xmin); + xid = UINT32_ACCESS_ONCE(proc->xmin); if (!TransactionIdIsNormal(xid) || !TransactionIdPrecedesOrEquals(xid, xmin)) continue; @@ -2221,7 +2221,7 @@ ProcArrayInstallImportedXmin(TransactionId xmin, * GetSnapshotData first, we'll be overwriting a valid xmin here, so * we don't check that.) */ - MyPgXact->xmin = TransactionXmin = xmin; + MyProc->xmin = TransactionXmin = xmin; result = true; break; @@ -2233,7 +2233,7 @@ ProcArrayInstallImportedXmin(TransactionId xmin, } /* - * ProcArrayInstallRestoredXmin -- install restored xmin into MyPgXact->xmin + * ProcArrayInstallRestoredXmin -- install restored xmin into MyProc->xmin * * This is like ProcArrayInstallImportedXmin, but we have a pointer to the * PGPROC of the transaction from which we imported the snapshot, rather than @@ -2246,7 +2246,6 @@ ProcArrayInstallRestoredXmin(TransactionId xmin, PGPROC *proc) { bool result = false; TransactionId xid; - PGXACT *pgxact; Assert(TransactionIdIsNormal(xmin)); Assert(proc != NULL); @@ -2254,20 +2253,18 @@ ProcArrayInstallRestoredXmin(TransactionId xmin, PGPROC *proc) /* Get lock so source xact can't end while we're doing this */ LWLockAcquire(ProcArrayLock, LW_SHARED); - pgxact = &allPgXact[proc->pgprocno]; - /* * Be certain that the referenced PGPROC has an advertised xmin which is * no later than the one we're installing, so that the system-wide xmin * can't go backwards. Also, make sure it's running in the same database, * so that the per-database xmin cannot go backwards. */ - xid = UINT32_ACCESS_ONCE(pgxact->xmin); + xid = UINT32_ACCESS_ONCE(proc->xmin); if (proc->databaseId == MyDatabaseId && TransactionIdIsNormal(xid) && TransactionIdPrecedesOrEquals(xid, xmin)) { - MyPgXact->xmin = TransactionXmin = xmin; + MyProc->xmin = TransactionXmin = xmin; result = true; } @@ -2892,7 +2889,7 @@ GetCurrentVirtualXIDs(TransactionId limitXmin, bool excludeXmin0, if (allDbs || proc->databaseId == MyDatabaseId) { /* Fetch xmin just once - might change on us */ - TransactionId pxmin = UINT32_ACCESS_ONCE(pgxact->xmin); + TransactionId pxmin = UINT32_ACCESS_ONCE(proc->xmin); if (excludeXmin0 && !TransactionIdIsValid(pxmin)) continue; @@ -2978,7 +2975,6 @@ GetConflictingVirtualXIDs(TransactionId limitXmin, Oid dbOid) { int pgprocno = arrayP->pgprocnos[index]; PGPROC *proc = &allProcs[pgprocno]; - PGXACT *pgxact = &allPgXact[pgprocno]; /* Exclude prepared transactions */ if (proc->pid == 0) @@ -2988,7 +2984,7 @@ GetConflictingVirtualXIDs(TransactionId limitXmin, Oid dbOid) proc->databaseId == dbOid) { /* Fetch xmin just once - can't change on us, but good coding */ - TransactionId pxmin = UINT32_ACCESS_ONCE(pgxact->xmin); + TransactionId pxmin = UINT32_ACCESS_ONCE(proc->xmin); /* * We ignore an invalid pxmin because this means that backend has diff --git a/src/backend/storage/ipc/sinvaladt.c b/src/backend/storage/ipc/sinvaladt.c index e5c115b92f2..ad048bc85fa 100644 --- a/src/backend/storage/ipc/sinvaladt.c +++ b/src/backend/storage/ipc/sinvaladt.c @@ -420,7 +420,7 @@ BackendIdGetTransactionIds(int backendID, TransactionId *xid, TransactionId *xmi PGXACT *xact = &ProcGlobal->allPgXact[proc->pgprocno]; *xid = xact->xid; - *xmin = xact->xmin; + *xmin = proc->xmin; } } diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c index e57fcd25388..de346cd87fc 100644 --- a/src/backend/storage/lmgr/proc.c +++ b/src/backend/storage/lmgr/proc.c @@ -388,7 +388,7 @@ InitProcess(void) MyProc->fpVXIDLock = false; MyProc->fpLocalTransactionId = InvalidLocalTransactionId; MyPgXact->xid = InvalidTransactionId; - MyPgXact->xmin = InvalidTransactionId; + MyProc->xmin = InvalidTransactionId; MyProc->pid = MyProcPid; /* backendId, databaseId and roleId will be filled in later */ MyProc->backendId = InvalidBackendId; @@ -572,7 +572,7 @@ InitAuxiliaryProcess(void) MyProc->fpVXIDLock = false; MyProc->fpLocalTransactionId = InvalidLocalTransactionId; MyPgXact->xid = InvalidTransactionId; - MyPgXact->xmin = InvalidTransactionId; + MyProc->xmin = InvalidTransactionId; MyProc->backendId = InvalidBackendId; MyProc->databaseId = InvalidOid; MyProc->roleId = InvalidOid; diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 76578868cf9..689a3b6a597 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -27,11 +27,11 @@ * their lifetime is managed separately (as they live longer than one xact.c * transaction). * - * These arrangements let us reset MyPgXact->xmin when there are no snapshots + * These arrangements let us reset MyProc->xmin when there are no snapshots * referenced by this transaction, and advance it when the one with oldest * Xmin is no longer referenced. For simplicity however, only registered * snapshots not active snapshots participate in tracking which one is oldest; - * we don't try to change MyPgXact->xmin except when the active-snapshot + * we don't try to change MyProc->xmin except when the active-snapshot * stack is empty. * * @@ -187,7 +187,7 @@ static ActiveSnapshotElt *OldestActiveSnapshot = NULL; /* * Currently registered Snapshots. Ordered in a heap by xmin, so that we can - * quickly find the one with lowest xmin, to advance our MyPgXact->xmin. + * quickly find the one with lowest xmin, to advance our MyProc->xmin. */ static int xmin_cmp(const pairingheap_node *a, const pairingheap_node *b, void *arg); @@ -475,7 +475,7 @@ GetNonHistoricCatalogSnapshot(Oid relid) /* * Make sure the catalog snapshot will be accounted for in decisions - * about advancing PGXACT->xmin. We could apply RegisterSnapshot, but + * about advancing PGPROC->xmin. We could apply RegisterSnapshot, but * that would result in making a physical copy, which is overkill; and * it would also create a dependency on some resource owner, which we * do not want for reasons explained at the head of this file. Instead @@ -596,7 +596,7 @@ SetTransactionSnapshot(Snapshot sourcesnap, VirtualTransactionId *sourcevxid, /* NB: curcid should NOT be copied, it's a local matter */ /* - * Now we have to fix what GetSnapshotData did with MyPgXact->xmin and + * Now we have to fix what GetSnapshotData did with MyProc->xmin and * TransactionXmin. There is a race condition: to make sure we are not * causing the global xmin to go backwards, we have to test that the * source transaction is still running, and that has to be done @@ -950,13 +950,13 @@ xmin_cmp(const pairingheap_node *a, const pairingheap_node *b, void *arg) /* * SnapshotResetXmin * - * If there are no more snapshots, we can reset our PGXACT->xmin to InvalidXid. + * If there are no more snapshots, we can reset our PGPROC->xmin to InvalidXid. * Note we can do this without locking because we assume that storing an Xid * is atomic. * * Even if there are some remaining snapshots, we may be able to advance our - * PGXACT->xmin to some degree. This typically happens when a portal is - * dropped. For efficiency, we only consider recomputing PGXACT->xmin when + * PGPROC->xmin to some degree. This typically happens when a portal is + * dropped. For efficiency, we only consider recomputing PGPROC->xmin when * the active snapshot stack is empty; this allows us not to need to track * which active snapshot is oldest. * @@ -977,15 +977,15 @@ SnapshotResetXmin(void) if (pairingheap_is_empty(&RegisteredSnapshots)) { - MyPgXact->xmin = InvalidTransactionId; + MyProc->xmin = InvalidTransactionId; return; } minSnapshot = pairingheap_container(SnapshotData, ph_node, pairingheap_first(&RegisteredSnapshots)); - if (TransactionIdPrecedes(MyPgXact->xmin, minSnapshot->xmin)) - MyPgXact->xmin = minSnapshot->xmin; + if (TransactionIdPrecedes(MyProc->xmin, minSnapshot->xmin)) + MyProc->xmin = minSnapshot->xmin; } /* @@ -1132,13 +1132,13 @@ AtEOXact_Snapshot(bool isCommit, bool resetXmin) /* * During normal commit processing, we call ProcArrayEndTransaction() to - * reset the MyPgXact->xmin. That call happens prior to the call to + * reset the MyProc->xmin. That call happens prior to the call to * AtEOXact_Snapshot(), so we need not touch xmin here at all. */ if (resetXmin) SnapshotResetXmin(); - Assert(resetXmin || MyPgXact->xmin == 0); + Assert(resetXmin || MyProc->xmin == 0); } @@ -1830,7 +1830,7 @@ TransactionIdLimitedForOldSnapshots(TransactionId recentXmin, */ if (old_snapshot_threshold == 0) { - if (TransactionIdPrecedes(latest_xmin, MyPgXact->xmin) + if (TransactionIdPrecedes(latest_xmin, MyProc->xmin) && TransactionIdFollows(latest_xmin, xlimit)) xlimit = latest_xmin; -- 2.25.0.114.g5b0ca878e0 --kdyesixubxkl3b7r Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v11-0003-snapshot-scalability-Introduce-dense-array-of-in.patch" From 1609ab74104aa845df655807695ed341a0a5dbe1 Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Wed, 15 Jul 2020 15:35:07 -0700 Subject: [PATCH v11 3/6] snapshot scalability: Introduce dense array of in-progress xids. The new array contains the xids for all connected backends / in-use PGPROC entries in a dense manner (in contrast to the PGPROC/PGXACT arrays which can have unused entries interspersed). This improves performance because GetSnapshotData() always needs to scan the xids of all live procarray entries and now there's no need to go through the procArray->pgprocnos indirection anymore. As the set of running top-level xids changes rarely, compared to the number of snapshots taken, this substantially increases the likelihood of most data required for a snapshot being in l2 cache. In read-mostly workloads scanning the xids[] array will sufficient to build a snapshot, as most backends will not have an xid assigned. To keep the xid array dense ProcArrayRemove() needs to move entries behind the to-be-removed proc's one further up in the array. Obviously moving array entries cannot happen while a backend sets it xid. I.e. locking needs to prevent that array entries are moved while a backend modifies its xid. To avoid locking ProcArrayLock in GetNewTransactionId() - a fairly hot spot already - ProcArrayAdd() / ProcArrayRemove() now needs to hold XidGenLock in addition to ProcArrayLock. Adding / Removing a procarray entry is not a very frequent operation, even taking 2PC into account. Due to the above, the dense array entries can only be read or modified while holding ProcArrayLock and/or XidGenLock. This prevents a concurrent ProcArrayRemove() from shifting the dense array while it is accessed concurrently. While the new dense array is very good when needing to look at all xids it is less suitable when accessing a single backend's xid. In particular it would be problematic to have to acquire a lock to access a backend's own xid. Therefore a backend's xid is not just stored in the dense array, but also in PGPROC. This also allows a backend to only access the shared xid value when the backend had acquired an xid. The infrastructure added in this commit will be used for the remaining PGXACT fields in subsequent commits. They are kept separate to make review easier. Author: Andres Freund Reviewed-By: Robert Haas, Thomas Munro, David Rowley Discussion: https://postgr.es/m/20200301083601.ews6hz5dduc3w2se@alap3.anarazel.de --- src/include/storage/proc.h | 79 +++++- src/backend/access/heap/heapam_visibility.c | 8 +- src/backend/access/transam/README | 33 +-- src/backend/access/transam/clog.c | 8 +- src/backend/access/transam/twophase.c | 31 +-- src/backend/access/transam/varsup.c | 20 +- src/backend/commands/vacuum.c | 2 +- src/backend/storage/ipc/procarray.c | 282 +++++++++++++------- src/backend/storage/ipc/sinvaladt.c | 4 +- src/backend/storage/lmgr/lock.c | 3 +- src/backend/storage/lmgr/proc.c | 26 +- 11 files changed, 335 insertions(+), 161 deletions(-) diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h index 286c9a9aec3..b828cecd185 100644 --- a/src/include/storage/proc.h +++ b/src/include/storage/proc.h @@ -90,6 +90,17 @@ typedef enum * distinguished from a real one at need by the fact that it has pid == 0. * The semaphore and lock-activity fields in a prepared-xact PGPROC are unused, * but its myProcLocks[] lists are valid. + * + * Mirrored fields: + * + * Some fields in PGPROC (see "mirrored in ..." comment) are mirrored into an + * element of more densely packed ProcGlobal arrays. These arrays are indexed + * by PGPROC->pgxactoff. Both copies need to be maintained coherently. + * + * NB: The pgxactoff indexed value can *never* be accessed without holding + * locks. + * + * See PROC_HDR for details. */ struct PGPROC { @@ -102,6 +113,12 @@ struct PGPROC Latch procLatch; /* generic latch for process */ + + TransactionId xid; /* id of top-level transaction currently being + * executed by this proc, if running and XID + * is assigned; else InvalidTransactionId. + * mirrored in ProcGlobal->xids[pgxactoff] */ + TransactionId xmin; /* minimal running XID as it was when we were * starting our xact, excluding LAZY VACUUM: * vacuum must not remove tuples deleted by @@ -111,6 +128,9 @@ struct PGPROC * being executed by this proc, if running; * else InvalidLocalTransactionId */ int pid; /* Backend's process ID; 0 if prepared xact */ + + int pgxactoff; /* offset into various ProcGlobal->arrays + * with data mirrored from this PGPROC */ int pgprocno; /* These fields are zero while a backend is still starting up: */ @@ -225,10 +245,6 @@ extern PGDLLIMPORT struct PGXACT *MyPgXact; */ typedef struct PGXACT { - TransactionId xid; /* id of top-level transaction currently being - * executed by this proc, if running and XID - * is assigned; else InvalidTransactionId */ - uint8 vacuumFlags; /* vacuum-related flags, see above */ bool overflowed; @@ -237,6 +253,57 @@ typedef struct PGXACT /* * There is one ProcGlobal struct for the whole database cluster. + * + * Adding/Removing an entry into the procarray requires holding *both* + * ProcArrayLock and XidGenLock in exclusive mode (in that order). Both are + * needed because the dense arrays (see below) are accessed from + * GetNewTransactionId() and GetSnapshotData(), and we don't want to add + * further contention by both using the same lock. Adding/Removing a procarray + * entry is much less frequent. + * + * Some fields in PGPROC are mirrored into more densely packed arrays (like + * xids), with one entry for each backend. These arrays only contain entries + * for PGPROCs that have been added to the shared array with ProcArrayAdd() + * (in contrast to PGPROC array which has unused PGPROCs interspersed). + * + * The dense arrays are indexed indexed by PGPROC->pgxactoff. Any concurrent + * ProcArrayAdd() / ProcArrayRemove() can lead to pgxactoff of a procarray + * member to change. Therefore it is only safe to use PGPROC->pgxactoff to + * access the dense array while holding either ProcArrayLock or XidGenLock. + * + * As long as a PGPROC is in the procarray, the mirrored values need to be + * maintained in both places in a coherent manner. + * + * The denser separate arrays are beneficial for three main reasons: First, to + * allow for as tight loops accessing the data as possible. Second, to prevent + * updates of frequently changing data (e.g. xmin) from invalidating + * cachelines also containing less frequently changing data (e.g. xid, + * vacuumFlags). Third to condense frequently accessed data into as few + * cachelines as possible. + * + * There are two main reasons to have the data mirrored between these dense + * arrays and PGPROC. First, as explained above, a PGPROC's array entries can + * only be accessed with either ProcArrayLock or XidGenLock held, whereas the + * PGPROC entries do not require that (obviously there may still be locking + * requirements around the individual field, separate from the concerns + * here). That is particularly important for a backend to efficiently checks + * it own values, which it often can safely do without locking. Second, the + * PGPROC fields allow to avoid unnecessary accesses and modification to the + * dense arrays. A backend's own PGPROC is more likely to be in a local cache, + * whereas the cachelines for the dense array will be modified by other + * backends (often removing it from the cache for other cores/sockets). At + * commit/abort time a check of the PGPROC value can avoid accessing/dirtying + * the corresponding array value. + * + * Basically it makes sense to access the PGPROC variable when checking a + * single backend's data, especially when already looking at the PGPROC for + * other reasons already. It makes sense to look at the "dense" arrays if we + * need to look at many / most entries, because we then benefit from the + * reduced indirection and better cross-process cache-ability. + * + * When entering a PGPROC for 2PC transactions with ProcArrayAdd(), the data + * in the dense arrays is initialized from the PGPROC while it already holds + * ProcArrayLock. */ typedef struct PROC_HDR { @@ -244,6 +311,10 @@ typedef struct PROC_HDR PGPROC *allProcs; /* Array of PGXACT structures (not including dummies for prepared txns) */ PGXACT *allPgXact; + + /* Array mirroring PGPROC.xid for each PGPROC currently in the procarray */ + TransactionId *xids; + /* Length of allProcs array */ uint32 allProcCount; /* Head of list of free PGPROC structures */ diff --git a/src/backend/access/heap/heapam_visibility.c b/src/backend/access/heap/heapam_visibility.c index b25b3e429ed..10848649c0c 100644 --- a/src/backend/access/heap/heapam_visibility.c +++ b/src/backend/access/heap/heapam_visibility.c @@ -11,12 +11,12 @@ * shared buffer content lock on the buffer containing the tuple. * * NOTE: When using a non-MVCC snapshot, we must check - * TransactionIdIsInProgress (which looks in the PGXACT array) + * TransactionIdIsInProgress (which looks in the PGPROC array) * before TransactionIdDidCommit/TransactionIdDidAbort (which look in * pg_xact). Otherwise we have a race condition: we might decide that a * just-committed transaction crashed, because none of the tests succeed. * xact.c is careful to record commit/abort in pg_xact before it unsets - * MyPgXact->xid in the PGXACT array. That fixes that problem, but it + * MyProc->xid in the PGPROC array. That fixes that problem, but it * also means there is a window where TransactionIdIsInProgress and * TransactionIdDidCommit will both return true. If we check only * TransactionIdDidCommit, we could consider a tuple committed when a @@ -956,7 +956,7 @@ HeapTupleSatisfiesDirty(HeapTuple htup, Snapshot snapshot, * coding where we tried to set the hint bits as soon as possible, we instead * did TransactionIdIsInProgress in each call --- to no avail, as long as the * inserting/deleting transaction was still running --- which was more cycles - * and more contention on the PGXACT array. + * and more contention on ProcArrayLock. */ static bool HeapTupleSatisfiesMVCC(HeapTuple htup, Snapshot snapshot, @@ -1445,7 +1445,7 @@ HeapTupleSatisfiesNonVacuumable(HeapTuple htup, Snapshot snapshot, * HeapTupleSatisfiesMVCC) and, therefore, any hint bits that can be set * should already be set. We assume that if no hint bits are set, the xmin * or xmax transaction is still running. This is therefore faster than - * HeapTupleSatisfiesVacuum, because we don't consult PGXACT nor CLOG. + * HeapTupleSatisfiesVacuum, because we consult neither procarray nor CLOG. * It's okay to return false when in doubt, but we must return true only * if the tuple is removable. */ diff --git a/src/backend/access/transam/README b/src/backend/access/transam/README index 94d8f3fd0a2..c46fc3cc194 100644 --- a/src/backend/access/transam/README +++ b/src/backend/access/transam/README @@ -251,10 +251,10 @@ enforce, and it assists with some other issues as explained below.) The implementation of this is that GetSnapshotData takes the ProcArrayLock in shared mode (so that multiple backends can take snapshots in parallel), but ProcArrayEndTransaction must take the ProcArrayLock in exclusive mode -while clearing MyPgXact->xid at transaction end (either commit or abort). -(To reduce context switching, when multiple transactions commit nearly -simultaneously, we have one backend take ProcArrayLock and clear the XIDs -of multiple processes at once.) +while clearing the ProcGlobal->xids[] entry at transaction end (either +commit or abort). (To reduce context switching, when multiple transactions +commit nearly simultaneously, we have one backend take ProcArrayLock and +clear the XIDs of multiple processes at once.) ProcArrayEndTransaction also holds the lock while advancing the shared latestCompletedFullXid variable. This allows GetSnapshotData to use @@ -278,12 +278,13 @@ present in the ProcArray, or not running anymore. (This guarantee doesn't apply to subtransaction XIDs, because of the possibility that there's not room for them in the subxid array; instead we guarantee that they are present or the overflow flag is set.) If a backend released XidGenLock -before storing its XID into MyPgXact, then it would be possible for another -backend to allocate and commit a later XID, causing latestCompletedFullXid to -pass the first backend's XID, before that value became visible in the -ProcArray. That would break ComputeXidHorizons, as discussed below. +before storing its XID into ProcGlobal->xids[], then it would be possible for +another backend to allocate and commit a later XID, causing +latestCompletedFullXid to pass the first backend's XID, before that value +became visible in the ProcArray. That would break ComputeXidHorizons, +as discussed below. -We allow GetNewTransactionId to store the XID into MyPgXact->xid (or the +We allow GetNewTransactionId to store the XID into ProcGlobal->xids[] (or the subxid array) without taking ProcArrayLock. This was once necessary to avoid deadlock; while that is no longer the case, it's still beneficial for performance. We are thereby relying on fetch/store of an XID to be atomic, @@ -382,13 +383,13 @@ Top-level transactions do not have a parent, so they leave their pg_subtrans entries set to the default value of zero (InvalidTransactionId). pg_subtrans is used to check whether the transaction in question is still -running --- the main Xid of a transaction is recorded in the PGXACT struct, -but since we allow arbitrary nesting of subtransactions, we can't fit all Xids -in shared memory, so we have to store them on disk. Note, however, that for -each transaction we keep a "cache" of Xids that are known to be part of the -transaction tree, so we can skip looking at pg_subtrans unless we know the -cache has been overflowed. See storage/ipc/procarray.c for the gory details. - +running --- the main Xid of a transaction is recorded in ProcGlobal->xids[], +with a copy in PGPROC->xid, but since we allow arbitrary nesting of +subtransactions, we can't fit all Xids in shared memory, so we have to store +them on disk. Note, however, that for each transaction we keep a "cache" of +Xids that are known to be part of the transaction tree, so we can skip looking +at pg_subtrans unless we know the cache has been overflowed. See +storage/ipc/procarray.c for the gory details. slru.c is the supporting mechanism for both pg_xact and pg_subtrans. It implements the LRU policy for in-memory buffer pages. The high-level routines for pg_xact are implemented in transam.c, while the low-level functions are in diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c index f3da40ae017..5198a0cef68 100644 --- a/src/backend/access/transam/clog.c +++ b/src/backend/access/transam/clog.c @@ -285,15 +285,15 @@ TransactionIdSetPageStatus(TransactionId xid, int nsubxids, * updates for multiple backends so that the number of times XactSLRULock * needs to be acquired is reduced. * - * For this optimization to be safe, the XID in MyPgXact and the subxids - * in MyProc must be the same as the ones for which we're setting the - * status. Check that this is the case. + * For this optimization to be safe, the XID and subxids in MyProc must be + * the same as the ones for which we're setting the status. Check that + * this is the case. * * For this optimization to be efficient, we shouldn't have too many * sub-XIDs and all of the XIDs for which we're adjusting clog should be * on the same page. Check those conditions, too. */ - if (all_xact_same_page && xid == MyPgXact->xid && + if (all_xact_same_page && xid == MyProc->xid && nsubxids <= THRESHOLD_SUBTRANS_CLOG_OPT && nsubxids == MyPgXact->nxids && memcmp(subxids, MyProc->subxids.xids, diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c index ae7c1a4c172..d073eb07d23 100644 --- a/src/backend/access/transam/twophase.c +++ b/src/backend/access/transam/twophase.c @@ -351,7 +351,7 @@ AtAbort_Twophase(void) /* * This is called after we have finished transferring state to the prepared - * PGXACT entry. + * PGPROC entry. */ void PostPrepare_Twophase(void) @@ -463,7 +463,7 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid, proc->waitStatus = PROC_WAIT_STATUS_OK; /* We set up the gxact's VXID as InvalidBackendId/XID */ proc->lxid = (LocalTransactionId) xid; - pgxact->xid = xid; + proc->xid = xid; Assert(proc->xmin == InvalidTransactionId); proc->delayChkpt = false; pgxact->vacuumFlags = 0; @@ -768,7 +768,6 @@ pg_prepared_xact(PG_FUNCTION_ARGS) { GlobalTransaction gxact = &status->array[status->currIdx++]; PGPROC *proc = &ProcGlobal->allProcs[gxact->pgprocno]; - PGXACT *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno]; Datum values[5]; bool nulls[5]; HeapTuple tuple; @@ -783,7 +782,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS) MemSet(values, 0, sizeof(values)); MemSet(nulls, 0, sizeof(nulls)); - values[0] = TransactionIdGetDatum(pgxact->xid); + values[0] = TransactionIdGetDatum(proc->xid); values[1] = CStringGetTextDatum(gxact->gid); values[2] = TimestampTzGetDatum(gxact->prepared_at); values[3] = ObjectIdGetDatum(gxact->owner); @@ -829,9 +828,8 @@ TwoPhaseGetGXact(TransactionId xid, bool lock_held) for (i = 0; i < TwoPhaseState->numPrepXacts; i++) { GlobalTransaction gxact = TwoPhaseState->prepXacts[i]; - PGXACT *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno]; - if (pgxact->xid == xid) + if (gxact->xid == xid) { result = gxact; break; @@ -987,8 +985,7 @@ void StartPrepare(GlobalTransaction gxact) { PGPROC *proc = &ProcGlobal->allProcs[gxact->pgprocno]; - PGXACT *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno]; - TransactionId xid = pgxact->xid; + TransactionId xid = gxact->xid; TwoPhaseFileHeader hdr; TransactionId *children; RelFileNode *commitrels; @@ -1140,15 +1137,15 @@ EndPrepare(GlobalTransaction gxact) /* * Mark the prepared transaction as valid. As soon as xact.c marks - * MyPgXact as not running our XID (which it will do immediately after + * MyProc as not running our XID (which it will do immediately after * this function returns), others can commit/rollback the xact. * * NB: a side effect of this is to make a dummy ProcArray entry for the - * prepared XID. This must happen before we clear the XID from MyPgXact, - * else there is a window where the XID is not running according to - * TransactionIdIsInProgress, and onlookers would be entitled to assume - * the xact crashed. Instead we have a window where the same XID appears - * twice in ProcArray, which is OK. + * prepared XID. This must happen before we clear the XID from MyProc / + * ProcGlobal->xids[], else there is a window where the XID is not running + * according to TransactionIdIsInProgress, and onlookers would be entitled + * to assume the xact crashed. Instead we have a window where the same + * XID appears twice in ProcArray, which is OK. */ MarkAsPrepared(gxact, false); @@ -1404,7 +1401,6 @@ FinishPreparedTransaction(const char *gid, bool isCommit) { GlobalTransaction gxact; PGPROC *proc; - PGXACT *pgxact; TransactionId xid; char *buf; char *bufptr; @@ -1423,8 +1419,7 @@ FinishPreparedTransaction(const char *gid, bool isCommit) */ gxact = LockGXact(gid, GetUserId()); proc = &ProcGlobal->allProcs[gxact->pgprocno]; - pgxact = &ProcGlobal->allPgXact[gxact->pgprocno]; - xid = pgxact->xid; + xid = gxact->xid; /* * Read and validate 2PC state data. State data will typically be stored @@ -1726,7 +1721,7 @@ CheckPointTwoPhase(XLogRecPtr redo_horizon) for (i = 0; i < TwoPhaseState->numPrepXacts; i++) { /* - * Note that we are using gxact not pgxact so this works in recovery + * Note that we are using gxact not PGPROC so this works in recovery * also */ GlobalTransaction gxact = TwoPhaseState->prepXacts[i]; diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c index 00b8e4e50d7..ab376f2fe22 100644 --- a/src/backend/access/transam/varsup.c +++ b/src/backend/access/transam/varsup.c @@ -38,7 +38,8 @@ VariableCache ShmemVariableCache = NULL; * Allocate the next FullTransactionId for a new transaction or * subtransaction. * - * The new XID is also stored into MyPgXact before returning. + * The new XID is also stored into MyProc->xid/ProcGlobal->xids[] before + * returning. * * Note: when this is called, we are actually already inside a valid * transaction, since XIDs are now not allocated until the transaction @@ -65,7 +66,8 @@ GetNewTransactionId(bool isSubXact) if (IsBootstrapProcessingMode()) { Assert(!isSubXact); - MyPgXact->xid = BootstrapTransactionId; + MyProc->xid = BootstrapTransactionId; + ProcGlobal->xids[MyProc->pgxactoff] = BootstrapTransactionId; return FullTransactionIdFromEpochAndXid(0, BootstrapTransactionId); } @@ -190,10 +192,10 @@ GetNewTransactionId(bool isSubXact) * latestCompletedXid is present in the ProcArray, which is essential for * correct OldestXmin tracking; see src/backend/access/transam/README. * - * Note that readers of PGXACT xid fields should be careful to fetch the - * value only once, rather than assume they can read a value multiple - * times and get the same answer each time. Note we are assuming that - * TransactionId and int fetch/store are atomic. + * Note that readers of ProcGlobal->xids/PGPROC->xid should be careful + * to fetch the value for each proc only once, rather than assume they can + * read a value multiple times and get the same answer each time. Note we + * are assuming that TransactionId and int fetch/store are atomic. * * The same comments apply to the subxact xid count and overflow fields. * @@ -219,7 +221,11 @@ GetNewTransactionId(bool isSubXact) * answer later on when someone does have a reason to inquire.) */ if (!isSubXact) - MyPgXact->xid = xid; /* LWLockRelease acts as barrier */ + { + /* LWLockRelease acts as barrier */ + MyProc->xid = xid; + ProcGlobal->xids[MyProc->pgxactoff] = xid; + } else { int nxids = MyPgXact->nxids; diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index 22228f5684f..648e12c78d8 100644 --- a/src/backend/commands/vacuum.c +++ b/src/backend/commands/vacuum.c @@ -1724,7 +1724,7 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params) * * Note: these flags remain set until CommitTransaction or * AbortTransaction. We don't want to clear them until we reset - * MyPgXact->xid/xmin, otherwise GetOldestNonRemovableTransactionId() + * MyProc->xid/xmin, otherwise GetOldestNonRemovableTransactionId() * might appear to go backwards, which is probably Not Good. */ LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE); diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c index 980ca2cc2af..a9b32565367 100644 --- a/src/backend/storage/ipc/procarray.c +++ b/src/backend/storage/ipc/procarray.c @@ -9,8 +9,9 @@ * one is as a means of determining the set of currently running transactions. * * Because of various subtle race conditions it is critical that a backend - * hold the correct locks while setting or clearing its MyPgXact->xid field. - * See notes in src/backend/access/transam/README. + * hold the correct locks while setting or clearing its xid (in + * ProcGlobal->xids[]/MyProc->xid). See notes in + * src/backend/access/transam/README. * * The process arrays now also include structures representing prepared * transactions. The xid and subxids fields of these are valid, as are the @@ -434,7 +435,9 @@ ProcArrayAdd(PGPROC *proc) ProcArrayStruct *arrayP = procArray; int index; + /* See ProcGlobal comment explaining why both locks are held */ LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE); + LWLockAcquire(XidGenLock, LW_EXCLUSIVE); if (arrayP->numProcs >= arrayP->maxProcs) { @@ -443,7 +446,6 @@ ProcArrayAdd(PGPROC *proc) * fixed supply of PGPROC structs too, and so we should have failed * earlier.) */ - LWLockRelease(ProcArrayLock); ereport(FATAL, (errcode(ERRCODE_TOO_MANY_CONNECTIONS), errmsg("sorry, too many clients already"))); @@ -469,10 +471,25 @@ ProcArrayAdd(PGPROC *proc) } memmove(&arrayP->pgprocnos[index + 1], &arrayP->pgprocnos[index], - (arrayP->numProcs - index) * sizeof(int)); + (arrayP->numProcs - index) * sizeof(*arrayP->pgprocnos)); + memmove(&ProcGlobal->xids[index + 1], &ProcGlobal->xids[index], + (arrayP->numProcs - index) * sizeof(*ProcGlobal->xids)); + arrayP->pgprocnos[index] = proc->pgprocno; + ProcGlobal->xids[index] = proc->xid; + arrayP->numProcs++; + for (; index < arrayP->numProcs; index++) + { + allProcs[arrayP->pgprocnos[index]].pgxactoff = index; + } + + /* + * Release in reversed acquisition order, to reduce frequency of having to + * wait for XidGenLock while holding ProcArrayLock. + */ + LWLockRelease(XidGenLock); LWLockRelease(ProcArrayLock); } @@ -498,36 +515,59 @@ ProcArrayRemove(PGPROC *proc, TransactionId latestXid) DisplayXidCache(); #endif + /* See ProcGlobal comment explaining why both locks are held */ LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE); + LWLockAcquire(XidGenLock, LW_EXCLUSIVE); + + Assert(ProcGlobal->allProcs[arrayP->pgprocnos[proc->pgxactoff]].pgxactoff == proc->pgxactoff); if (TransactionIdIsValid(latestXid)) { - Assert(TransactionIdIsValid(allPgXact[proc->pgprocno].xid)); + Assert(TransactionIdIsValid(ProcGlobal->xids[proc->pgxactoff])); /* Advance global latestCompletedXid while holding the lock */ MaintainLatestCompletedXid(latestXid); + + ProcGlobal->xids[proc->pgxactoff] = 0; } else { /* Shouldn't be trying to remove a live transaction here */ - Assert(!TransactionIdIsValid(allPgXact[proc->pgprocno].xid)); + Assert(!TransactionIdIsValid(ProcGlobal->xids[proc->pgxactoff])); } + Assert(TransactionIdIsValid(ProcGlobal->xids[proc->pgxactoff] == 0)); + for (index = 0; index < arrayP->numProcs; index++) { if (arrayP->pgprocnos[index] == proc->pgprocno) { /* Keep the PGPROC array sorted. See notes above */ memmove(&arrayP->pgprocnos[index], &arrayP->pgprocnos[index + 1], - (arrayP->numProcs - index - 1) * sizeof(int)); + (arrayP->numProcs - index - 1) * sizeof(*arrayP->pgprocnos)); + memmove(&ProcGlobal->xids[index], &ProcGlobal->xids[index + 1], + (arrayP->numProcs - index - 1) * sizeof(*ProcGlobal->xids)); + arrayP->pgprocnos[arrayP->numProcs - 1] = -1; /* for debugging */ arrayP->numProcs--; + + for (; index < arrayP->numProcs; index++) + { + allProcs[arrayP->pgprocnos[index]].pgxactoff--; + } + + /* + * Release in reversed acquisition order, to reduce frequency of + * having to wait for XidGenLock while holding ProcArrayLock. + */ + LWLockRelease(XidGenLock); LWLockRelease(ProcArrayLock); return; } } /* Oops */ + LWLockRelease(XidGenLock); LWLockRelease(ProcArrayLock); elog(LOG, "failed to find proc %p in ProcArray", proc); @@ -560,7 +600,7 @@ ProcArrayEndTransaction(PGPROC *proc, TransactionId latestXid) * else is taking a snapshot. See discussion in * src/backend/access/transam/README. */ - Assert(TransactionIdIsValid(allPgXact[proc->pgprocno].xid)); + Assert(TransactionIdIsValid(proc->xid)); /* * If we can immediately acquire ProcArrayLock, we clear our own XID @@ -582,7 +622,7 @@ ProcArrayEndTransaction(PGPROC *proc, TransactionId latestXid) * anyone else's calculation of a snapshot. We might change their * estimate of global xmin, but that's OK. */ - Assert(!TransactionIdIsValid(allPgXact[proc->pgprocno].xid)); + Assert(!TransactionIdIsValid(proc->xid)); proc->lxid = InvalidLocalTransactionId; /* must be cleared with xid/xmin: */ @@ -605,7 +645,13 @@ static inline void ProcArrayEndTransactionInternal(PGPROC *proc, PGXACT *pgxact, TransactionId latestXid) { - pgxact->xid = InvalidTransactionId; + size_t pgxactoff = proc->pgxactoff; + + Assert(TransactionIdIsValid(ProcGlobal->xids[pgxactoff])); + Assert(ProcGlobal->xids[pgxactoff] == proc->xid); + + ProcGlobal->xids[pgxactoff] = InvalidTransactionId; + proc->xid = InvalidTransactionId; proc->lxid = InvalidLocalTransactionId; /* must be cleared with xid/xmin: */ pgxact->vacuumFlags &= ~PROC_VACUUM_STATE_MASK; @@ -641,7 +687,7 @@ ProcArrayGroupClearXid(PGPROC *proc, TransactionId latestXid) uint32 wakeidx; /* We should definitely have an XID to clear. */ - Assert(TransactionIdIsValid(allPgXact[proc->pgprocno].xid)); + Assert(TransactionIdIsValid(proc->xid)); /* Add ourselves to the list of processes needing a group XID clear. */ proc->procArrayGroupMember = true; @@ -746,20 +792,28 @@ ProcArrayGroupClearXid(PGPROC *proc, TransactionId latestXid) * This is used after successfully preparing a 2-phase transaction. We are * not actually reporting the transaction's XID as no longer running --- it * will still appear as running because the 2PC's gxact is in the ProcArray - * too. We just have to clear out our own PGXACT. + * too. We just have to clear out our own PGPROC. */ void ProcArrayClearTransaction(PGPROC *proc) { PGXACT *pgxact = &allPgXact[proc->pgprocno]; + size_t pgxactoff; /* - * We can skip locking ProcArrayLock here, because this action does not - * actually change anyone's view of the set of running XIDs: our entry is - * duplicate with the gxact that has already been inserted into the - * ProcArray. + * We can skip locking ProcArrayLock exclusively here, because this action + * does not actually change anyone's view of the set of running XIDs: our + * entry is duplicate with the gxact that has already been inserted into + * the ProcArray. But need it in shared mode for pgproc->pgxactoff to stay + * the same. */ - pgxact->xid = InvalidTransactionId; + LWLockAcquire(ProcArrayLock, LW_SHARED); + + pgxactoff = proc->pgxactoff; + + ProcGlobal->xids[pgxactoff] = InvalidTransactionId; + proc->xid = InvalidTransactionId; + proc->lxid = InvalidLocalTransactionId; proc->xmin = InvalidTransactionId; proc->recoveryConflictPending = false; @@ -771,6 +825,8 @@ ProcArrayClearTransaction(PGPROC *proc) /* Clear the subtransaction-XID cache too */ pgxact->nxids = 0; pgxact->overflowed = false; + + LWLockRelease(ProcArrayLock); } /* @@ -1164,7 +1220,7 @@ ProcArrayApplyXidAssignment(TransactionId topxid, * there are four possibilities for finding a running transaction: * * 1. The given Xid is a main transaction Id. We will find this out cheaply - * by looking at the PGXACT struct for each backend. + * by looking at ProcGlobal->xids. * * 2. The given Xid is one of the cached subxact Xids in the PGPROC array. * We can find this out cheaply too. @@ -1173,25 +1229,27 @@ ProcArrayApplyXidAssignment(TransactionId topxid, * if the Xid is running on the primary. * * 4. Search the SubTrans tree to find the Xid's topmost parent, and then see - * if that is running according to PGXACT or KnownAssignedXids. This is the - * slowest way, but sadly it has to be done always if the others failed, - * unless we see that the cached subxact sets are complete (none have + * if that is running according to ProcGlobal->xids[] or KnownAssignedXids. + * This is the slowest way, but sadly it has to be done always if the others + * failed, unless we see that the cached subxact sets are complete (none have * overflowed). * * ProcArrayLock has to be held while we do 1, 2, 3. If we save the top Xids * while doing 1 and 3, we can release the ProcArrayLock while we do 4. * This buys back some concurrency (and we can't retrieve the main Xids from - * PGXACT again anyway; see GetNewTransactionId). + * ProcGlobal->xids[] again anyway; see GetNewTransactionId). */ bool TransactionIdIsInProgress(TransactionId xid) { static TransactionId *xids = NULL; + static TransactionId *other_xids; int nxids = 0; ProcArrayStruct *arrayP = procArray; TransactionId topxid; - int i, - j; + int mypgxactoff; + size_t numProcs; + int j; /* * Don't bother checking a transaction older than RecentXmin; it could not @@ -1246,6 +1304,8 @@ TransactionIdIsInProgress(TransactionId xid) errmsg("out of memory"))); } + other_xids = ProcGlobal->xids; + LWLockAcquire(ProcArrayLock, LW_SHARED); /* @@ -1261,20 +1321,22 @@ TransactionIdIsInProgress(TransactionId xid) } /* No shortcuts, gotta grovel through the array */ - for (i = 0; i < arrayP->numProcs; i++) + mypgxactoff = MyProc->pgxactoff; + numProcs = arrayP->numProcs; + for (size_t pgxactoff = 0; pgxactoff < numProcs; pgxactoff++) { - int pgprocno = arrayP->pgprocnos[i]; - PGPROC *proc = &allProcs[pgprocno]; - PGXACT *pgxact = &allPgXact[pgprocno]; + int pgprocno; + PGXACT *pgxact; + PGPROC *proc; TransactionId pxid; int pxids; - /* Ignore my own proc --- dealt with it above */ - if (proc == MyProc) + /* Ignore ourselves --- dealt with it above */ + if (pgxactoff == mypgxactoff) continue; /* Fetch xid just once - see GetNewTransactionId */ - pxid = UINT32_ACCESS_ONCE(pgxact->xid); + pxid = UINT32_ACCESS_ONCE(other_xids[pgxactoff]); if (!TransactionIdIsValid(pxid)) continue; @@ -1299,8 +1361,12 @@ TransactionIdIsInProgress(TransactionId xid) /* * Step 2: check the cached child-Xids arrays */ + pgprocno = arrayP->pgprocnos[pgxactoff]; + pgxact = &allPgXact[pgprocno]; pxids = pgxact->nxids; pg_read_barrier(); /* pairs with barrier in GetNewTransactionId() */ + pgprocno = arrayP->pgprocnos[pgxactoff]; + proc = &allProcs[pgprocno]; for (j = pxids - 1; j >= 0; j--) { /* Fetch xid just once - see GetNewTransactionId */ @@ -1331,7 +1397,7 @@ TransactionIdIsInProgress(TransactionId xid) */ if (RecoveryInProgress()) { - /* none of the PGXACT entries should have XIDs in hot standby mode */ + /* none of the PGPROC entries should have XIDs in hot standby mode */ Assert(nxids == 0); if (KnownAssignedXidExists(xid)) @@ -1386,7 +1452,7 @@ TransactionIdIsInProgress(TransactionId xid) Assert(TransactionIdIsValid(topxid)); if (!TransactionIdEquals(topxid, xid)) { - for (i = 0; i < nxids; i++) + for (int i = 0; i < nxids; i++) { if (TransactionIdEquals(xids[i], topxid)) return true; @@ -1409,6 +1475,7 @@ TransactionIdIsActive(TransactionId xid) { bool result = false; ProcArrayStruct *arrayP = procArray; + TransactionId *other_xids = ProcGlobal->xids; int i; /* @@ -1424,11 +1491,10 @@ TransactionIdIsActive(TransactionId xid) { int pgprocno = arrayP->pgprocnos[i]; PGPROC *proc = &allProcs[pgprocno]; - PGXACT *pgxact = &allPgXact[pgprocno]; TransactionId pxid; /* Fetch xid just once - see GetNewTransactionId */ - pxid = UINT32_ACCESS_ONCE(pgxact->xid); + pxid = UINT32_ACCESS_ONCE(other_xids[i]); if (!TransactionIdIsValid(pxid)) continue; @@ -1506,6 +1572,7 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h) ProcArrayStruct *arrayP = procArray; TransactionId kaxmin; bool in_recovery = RecoveryInProgress(); + TransactionId *other_xids = ProcGlobal->xids; /* inferred after ProcArrayLock is released */ h->catalog_oldest_nonremovable = InvalidTransactionId; @@ -1549,7 +1616,7 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h) TransactionId xmin; /* Fetch xid just once - see GetNewTransactionId */ - xid = UINT32_ACCESS_ONCE(pgxact->xid); + xid = UINT32_ACCESS_ONCE(other_xids[pgprocno]); xmin = UINT32_ACCESS_ONCE(proc->xmin); /* @@ -1837,14 +1904,17 @@ Snapshot GetSnapshotData(Snapshot snapshot) { ProcArrayStruct *arrayP = procArray; + TransactionId *other_xids = ProcGlobal->xids; TransactionId xmin; TransactionId xmax; - int index; - int count = 0; + size_t count = 0; int subcount = 0; bool suboverflowed = false; FullTransactionId latest_completed; TransactionId oldestxid; + int mypgxactoff; + TransactionId myxid; + TransactionId replication_slot_xmin = InvalidTransactionId; TransactionId replication_slot_catalog_xmin = InvalidTransactionId; @@ -1889,6 +1959,10 @@ GetSnapshotData(Snapshot snapshot) LWLockAcquire(ProcArrayLock, LW_SHARED); latest_completed = ShmemVariableCache->latestCompletedFullXid; + mypgxactoff = MyProc->pgxactoff; + myxid = other_xids[mypgxactoff]; + Assert(myxid == MyProc->xid); + oldestxid = ShmemVariableCache->oldestXid; /* xmax is always latestCompletedXid + 1 */ @@ -1899,57 +1973,79 @@ GetSnapshotData(Snapshot snapshot) /* initialize xmin calculation with xmax */ xmin = xmax; + /* take own xid into account, saves a check inside the loop */ + if (TransactionIdIsNormal(myxid) && NormalTransactionIdPrecedes(myxid, xmin)) + xmin = myxid; + snapshot->takenDuringRecovery = RecoveryInProgress(); if (!snapshot->takenDuringRecovery) { + size_t numProcs = arrayP->numProcs; + TransactionId *xip = snapshot->xip; int *pgprocnos = arrayP->pgprocnos; - int numProcs; /* - * Spin over procArray checking xid, xmin, and subxids. The goal is - * to gather all active xids, find the lowest xmin, and try to record - * subxids. + * First collect set of pgxactoff/xids that need to be included in the + * snapshot. */ - numProcs = arrayP->numProcs; - for (index = 0; index < numProcs; index++) + for (size_t pgxactoff = 0; pgxactoff < numProcs; pgxactoff++) { - int pgprocno = pgprocnos[index]; - PGXACT *pgxact = &allPgXact[pgprocno]; - TransactionId xid; + /* Fetch xid just once - see GetNewTransactionId */ + TransactionId xid = UINT32_ACCESS_ONCE(other_xids[pgxactoff]); + int pgprocno; + PGXACT *pgxact; + uint8 vacuumFlags; + + Assert(allProcs[arrayP->pgprocnos[pgxactoff]].pgxactoff == pgxactoff); + + /* + * If the transaction has no XID assigned, we can skip it; it + * won't have sub-XIDs either. + */ + if (likely(xid == InvalidTransactionId)) + continue; + + /* + * We don't include our own XIDs (if any) in the snapshot. It + * needs to be includeded in the xmin computation, but we did so + * outside the loop. + */ + if (pgxactoff == mypgxactoff) + continue; + + /* + * The only way we are able to get here with a non-normal xid + * is during bootstrap - with this backend using + * BootstrapTransactionId. But the above test should filter + * that out. + */ + Assert(TransactionIdIsNormal(xid)); + + /* + * If the XID is >= xmax, we can skip it; such transactions will + * be treated as running anyway (and any sub-XIDs will also be >= + * xmax). + */ + if (!NormalTransactionIdPrecedes(xid, xmax)) + continue; + + pgprocno = pgprocnos[pgxactoff]; + pgxact = &allPgXact[pgprocno]; + vacuumFlags = pgxact->vacuumFlags; /* * Skip over backends doing logical decoding which manages xmin * separately (check below) and ones running LAZY VACUUM. */ - if (pgxact->vacuumFlags & - (PROC_IN_LOGICAL_DECODING | PROC_IN_VACUUM)) + if (vacuumFlags & (PROC_IN_LOGICAL_DECODING | PROC_IN_VACUUM)) continue; - /* Fetch xid just once - see GetNewTransactionId */ - xid = UINT32_ACCESS_ONCE(pgxact->xid); - - /* - * If the transaction has no XID assigned, we can skip it; it - * won't have sub-XIDs either. If the XID is >= xmax, we can also - * skip it; such transactions will be treated as running anyway - * (and any sub-XIDs will also be >= xmax). - */ - if (!TransactionIdIsNormal(xid) - || !NormalTransactionIdPrecedes(xid, xmax)) - continue; - - /* - * We don't include our own XIDs (if any) in the snapshot, but we - * must include them in xmin. - */ if (NormalTransactionIdPrecedes(xid, xmin)) xmin = xid; - if (pgxact == MyPgXact) - continue; /* Add XID to snapshot. */ - snapshot->xip[count++] = xid; + xip[count++] = xid; /* * Save subtransaction XIDs if possible (if we've already @@ -1972,9 +2068,9 @@ GetSnapshotData(Snapshot snapshot) suboverflowed = true; else { - int nxids = pgxact->nxids; + int nsubxids = pgxact->nxids; - if (nxids > 0) + if (nsubxids > 0) { PGPROC *proc = &allProcs[pgprocno]; @@ -1982,8 +2078,8 @@ GetSnapshotData(Snapshot snapshot) memcpy(snapshot->subxip + subcount, (void *) proc->subxids.xids, - nxids * sizeof(TransactionId)); - subcount += nxids; + nsubxids * sizeof(TransactionId)); + subcount += nsubxids; } } } @@ -2115,6 +2211,7 @@ GetSnapshotData(Snapshot snapshot) } RecentXmin = xmin; + Assert(TransactionIdPrecedesOrEquals(TransactionXmin, RecentXmin)); snapshot->xmin = xmin; snapshot->xmax = xmax; @@ -2277,7 +2374,7 @@ ProcArrayInstallRestoredXmin(TransactionId xmin, PGPROC *proc) * GetRunningTransactionData -- returns information about running transactions. * * Similar to GetSnapshotData but returns more information. We include - * all PGXACTs with an assigned TransactionId, even VACUUM processes and + * all PGPROCs with an assigned TransactionId, even VACUUM processes and * prepared transactions. * * We acquire XidGenLock and ProcArrayLock, but the caller is responsible for @@ -2292,7 +2389,7 @@ ProcArrayInstallRestoredXmin(TransactionId xmin, PGPROC *proc) * This is never executed during recovery so there is no need to look at * KnownAssignedXids. * - * Dummy PGXACTs from prepared transaction are included, meaning that this + * Dummy PGPROCs from prepared transaction are included, meaning that this * may return entries with duplicated TransactionId values coming from * transaction finishing to prepare. Nothing is done about duplicated * entries here to not hold on ProcArrayLock more than necessary. @@ -2311,6 +2408,7 @@ GetRunningTransactionData(void) static RunningTransactionsData CurrentRunningXactsData; ProcArrayStruct *arrayP = procArray; + TransactionId *other_xids = ProcGlobal->xids; RunningTransactions CurrentRunningXacts = &CurrentRunningXactsData; TransactionId latestCompletedXid; TransactionId oldestRunningXid; @@ -2370,7 +2468,7 @@ GetRunningTransactionData(void) TransactionId xid; /* Fetch xid just once - see GetNewTransactionId */ - xid = UINT32_ACCESS_ONCE(pgxact->xid); + xid = UINT32_ACCESS_ONCE(other_xids[index]); /* * We don't need to store transactions that don't have a TransactionId @@ -2467,7 +2565,7 @@ GetRunningTransactionData(void) * GetOldestActiveTransactionId() * * Similar to GetSnapshotData but returns just oldestActiveXid. We include - * all PGXACTs with an assigned TransactionId, even VACUUM processes. + * all PGPROCs with an assigned TransactionId, even VACUUM processes. * We look at all databases, though there is no need to include WALSender * since this has no effect on hot standby conflicts. * @@ -2482,6 +2580,7 @@ TransactionId GetOldestActiveTransactionId(void) { ProcArrayStruct *arrayP = procArray; + TransactionId *other_xids = ProcGlobal->xids; TransactionId oldestRunningXid; int index; @@ -2504,12 +2603,10 @@ GetOldestActiveTransactionId(void) LWLockAcquire(ProcArrayLock, LW_SHARED); for (index = 0; index < arrayP->numProcs; index++) { - int pgprocno = arrayP->pgprocnos[index]; - PGXACT *pgxact = &allPgXact[pgprocno]; TransactionId xid; /* Fetch xid just once - see GetNewTransactionId */ - xid = UINT32_ACCESS_ONCE(pgxact->xid); + xid = UINT32_ACCESS_ONCE(other_xids[index]); if (!TransactionIdIsNormal(xid)) continue; @@ -2587,8 +2684,8 @@ GetOldestSafeDecodingTransactionId(bool catalogOnly) * If we're not in recovery, we walk over the procarray and collect the * lowest xid. Since we're called with ProcArrayLock held and have * acquired XidGenLock, no entries can vanish concurrently, since - * PGXACT->xid is only set with XidGenLock held and only cleared with - * ProcArrayLock held. + * ProcGlobal->xids[i] is only set with XidGenLock held and only cleared + * with ProcArrayLock held. * * In recovery we can't lower the safe value besides what we've computed * above, so we'll have to wait a bit longer there. We unfortunately can @@ -2597,17 +2694,17 @@ GetOldestSafeDecodingTransactionId(bool catalogOnly) */ if (!recovery_in_progress) { + TransactionId *other_xids = ProcGlobal->xids; + /* - * Spin over procArray collecting all min(PGXACT->xid) + * Spin over procArray collecting min(ProcGlobal->xids[i]) */ for (index = 0; index < arrayP->numProcs; index++) { - int pgprocno = arrayP->pgprocnos[index]; - PGXACT *pgxact = &allPgXact[pgprocno]; TransactionId xid; /* Fetch xid just once - see GetNewTransactionId */ - xid = UINT32_ACCESS_ONCE(pgxact->xid); + xid = UINT32_ACCESS_ONCE(other_xids[index]); if (!TransactionIdIsNormal(xid)) continue; @@ -2795,6 +2892,7 @@ BackendXidGetPid(TransactionId xid) { int result = 0; ProcArrayStruct *arrayP = procArray; + TransactionId *other_xids = ProcGlobal->xids; int index; if (xid == InvalidTransactionId) /* never match invalid xid */ @@ -2806,9 +2904,8 @@ BackendXidGetPid(TransactionId xid) { int pgprocno = arrayP->pgprocnos[index]; PGPROC *proc = &allProcs[pgprocno]; - PGXACT *pgxact = &allPgXact[pgprocno]; - if (pgxact->xid == xid) + if (other_xids[index] == xid) { result = proc->pid; break; @@ -3088,7 +3185,6 @@ MinimumActiveBackends(int min) { int pgprocno = arrayP->pgprocnos[index]; PGPROC *proc = &allProcs[pgprocno]; - PGXACT *pgxact = &allPgXact[pgprocno]; /* * Since we're not holding a lock, need to be prepared to deal with @@ -3105,7 +3201,7 @@ MinimumActiveBackends(int min) continue; /* do not count deleted entries */ if (proc == MyProc) continue; /* do not count myself */ - if (pgxact->xid == InvalidTransactionId) + if (proc->xid == InvalidTransactionId) continue; /* do not count if no XID assigned */ if (proc->pid == 0) continue; /* do not count prepared xacts */ @@ -3531,8 +3627,8 @@ XidCacheRemoveRunningXids(TransactionId xid, * * Note that we do not have to be careful about memory ordering of our own * reads wrt. GetNewTransactionId() here - only this process can modify - * relevant fields of MyProc/MyPgXact. But we do have to be careful about - * our own writes being well ordered. + * relevant fields of MyProc/ProcGlobal->xids[]. But we do have to be + * careful about our own writes being well ordered. */ LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE); @@ -3885,7 +3981,7 @@ FullXidViaRelative(FullTransactionId rel, TransactionId xid) * In Hot Standby mode, we maintain a list of transactions that are (or were) * running on the primary at the current point in WAL. These XIDs must be * treated as running by standby transactions, even though they are not in - * the standby server's PGXACT array. + * the standby server's PGPROC array. * * We record all XIDs that we know have been assigned. That includes all the * XIDs seen in WAL records, plus all unobserved XIDs that we can deduce have diff --git a/src/backend/storage/ipc/sinvaladt.c b/src/backend/storage/ipc/sinvaladt.c index ad048bc85fa..a9477ccb4a3 100644 --- a/src/backend/storage/ipc/sinvaladt.c +++ b/src/backend/storage/ipc/sinvaladt.c @@ -417,9 +417,7 @@ BackendIdGetTransactionIds(int backendID, TransactionId *xid, TransactionId *xmi if (proc != NULL) { - PGXACT *xact = &ProcGlobal->allPgXact[proc->pgprocno]; - - *xid = xact->xid; + *xid = proc->xid; *xmin = proc->xmin; } } diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 95989ce79bd..d86566f4554 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -3974,9 +3974,8 @@ GetRunningTransactionLocks(int *nlocks) proclock->tag.myLock->tag.locktag_type == LOCKTAG_RELATION) { PGPROC *proc = proclock->tag.myProc; - PGXACT *pgxact = &ProcGlobal->allPgXact[proc->pgprocno]; LOCK *lock = proclock->tag.myLock; - TransactionId xid = pgxact->xid; + TransactionId xid = proc->xid; /* * Don't record locks for transactions if we know they have diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c index de346cd87fc..7fad49544ce 100644 --- a/src/backend/storage/lmgr/proc.c +++ b/src/backend/storage/lmgr/proc.c @@ -102,21 +102,18 @@ Size ProcGlobalShmemSize(void) { Size size = 0; + Size TotalProcs = + add_size(MaxBackends, add_size(NUM_AUXILIARY_PROCS, max_prepared_xacts)); /* ProcGlobal */ size = add_size(size, sizeof(PROC_HDR)); - /* MyProcs, including autovacuum workers and launcher */ - size = add_size(size, mul_size(MaxBackends, sizeof(PGPROC))); - /* AuxiliaryProcs */ - size = add_size(size, mul_size(NUM_AUXILIARY_PROCS, sizeof(PGPROC))); - /* Prepared xacts */ - size = add_size(size, mul_size(max_prepared_xacts, sizeof(PGPROC))); - /* ProcStructLock */ + size = add_size(size, mul_size(TotalProcs, sizeof(PGPROC))); size = add_size(size, sizeof(slock_t)); size = add_size(size, mul_size(MaxBackends, sizeof(PGXACT))); size = add_size(size, mul_size(NUM_AUXILIARY_PROCS, sizeof(PGXACT))); size = add_size(size, mul_size(max_prepared_xacts, sizeof(PGXACT))); + size = add_size(size, mul_size(TotalProcs, sizeof(*ProcGlobal->xids))); return size; } @@ -216,6 +213,17 @@ InitProcGlobal(void) MemSet(pgxacts, 0, TotalProcs * sizeof(PGXACT)); ProcGlobal->allPgXact = pgxacts; + /* + * Allocate arrays mirroring PGPROC fields in a dense manner. See + * PROC_HDR. + * + * XXX: It might make sense to increase padding for these arrays, given + * how hotly they are accessed. + */ + ProcGlobal->xids = + (TransactionId *) ShmemAlloc(TotalProcs * sizeof(*ProcGlobal->xids)); + MemSet(ProcGlobal->xids, 0, TotalProcs * sizeof(*ProcGlobal->xids)); + for (i = 0; i < TotalProcs; i++) { /* Common initialization for all PGPROCs, regardless of type. */ @@ -387,7 +395,7 @@ InitProcess(void) MyProc->lxid = InvalidLocalTransactionId; MyProc->fpVXIDLock = false; MyProc->fpLocalTransactionId = InvalidLocalTransactionId; - MyPgXact->xid = InvalidTransactionId; + MyProc->xid = InvalidTransactionId; MyProc->xmin = InvalidTransactionId; MyProc->pid = MyProcPid; /* backendId, databaseId and roleId will be filled in later */ @@ -571,7 +579,7 @@ InitAuxiliaryProcess(void) MyProc->lxid = InvalidLocalTransactionId; MyProc->fpVXIDLock = false; MyProc->fpLocalTransactionId = InvalidLocalTransactionId; - MyPgXact->xid = InvalidTransactionId; + MyProc->xid = InvalidTransactionId; MyProc->xmin = InvalidTransactionId; MyProc->backendId = InvalidBackendId; MyProc->databaseId = InvalidOid; -- 2.25.0.114.g5b0ca878e0 --kdyesixubxkl3b7r Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v11-0004-snapshot-scalability-Move-PGXACT-vacuumFlags-to-.patch" From f692283f4f8bd307830bffae7c826e10af113074 Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Wed, 15 Jul 2020 15:35:07 -0700 Subject: [PATCH v11 4/6] snapshot scalability: Move PGXACT->vacuumFlags to ProcGlobal->vacuumFlags. Similar to the previous commit this increases the chance that data frequently needed by GetSnapshotData() stays in l2 cache. As we now take care to not unnecessarily write to ProcGlobal->vacuumFlags, there should be very few modifications to the ProcGlobal->vacuumFlags array. Author: Andres Freund Reviewed-By: Robert Haas, Thomas Munro, David Rowley Discussion: https://postgr.es/m/20200301083601.ews6hz5dduc3w2se@alap3.anarazel.de --- src/include/storage/proc.h | 12 ++++- src/backend/access/transam/twophase.c | 2 +- src/backend/commands/analyze.c | 10 ++-- src/backend/commands/vacuum.c | 5 +- src/backend/postmaster/autovacuum.c | 6 +-- src/backend/replication/logical/logical.c | 3 +- src/backend/replication/slot.c | 3 +- src/backend/storage/ipc/procarray.c | 66 ++++++++++++++--------- src/backend/storage/lmgr/deadlock.c | 4 +- src/backend/storage/lmgr/proc.c | 16 +++--- 10 files changed, 79 insertions(+), 48 deletions(-) diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h index b828cecd185..ffb775939ed 100644 --- a/src/include/storage/proc.h +++ b/src/include/storage/proc.h @@ -41,7 +41,7 @@ struct XidCache }; /* - * Flags for PGXACT->vacuumFlags + * Flags for ProcGlobal->vacuumFlags[] */ #define PROC_IS_AUTOVACUUM 0x01 /* is it an autovac worker? */ #define PROC_IN_VACUUM 0x02 /* currently running lazy vacuum */ @@ -168,6 +168,9 @@ struct PGPROC bool delayChkpt; /* true if this proc delays checkpoint start */ + uint8 vacuumFlags; /* this backend's vacuum flags, see PROC_* + * above. mirrored in + * ProcGlobal->vacuumFlags[pgxactoff] */ /* * Info to allow us to wait for synchronous replication, if needed. * waitLSN is InvalidXLogRecPtr if not waiting; set only by user backend. @@ -245,7 +248,6 @@ extern PGDLLIMPORT struct PGXACT *MyPgXact; */ typedef struct PGXACT { - uint8 vacuumFlags; /* vacuum-related flags, see above */ bool overflowed; uint8 nxids; @@ -315,6 +317,12 @@ typedef struct PROC_HDR /* Array mirroring PGPROC.xid for each PGPROC currently in the procarray */ TransactionId *xids; + /* + * Array mirroring PGPROC.vacuumFlags for each PGPROC currently in the + * procarray. + */ + uint8 *vacuumFlags; + /* Length of allProcs array */ uint32 allProcCount; /* Head of list of free PGPROC structures */ diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c index d073eb07d23..3371ebd8896 100644 --- a/src/backend/access/transam/twophase.c +++ b/src/backend/access/transam/twophase.c @@ -466,7 +466,7 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid, proc->xid = xid; Assert(proc->xmin == InvalidTransactionId); proc->delayChkpt = false; - pgxact->vacuumFlags = 0; + proc->vacuumFlags = 0; proc->pid = 0; proc->backendId = InvalidBackendId; proc->databaseId = databaseid; diff --git a/src/backend/commands/analyze.c b/src/backend/commands/analyze.c index 34b71b6c1c5..2c1b956b76b 100644 --- a/src/backend/commands/analyze.c +++ b/src/backend/commands/analyze.c @@ -250,7 +250,8 @@ analyze_rel(Oid relid, RangeVar *relation, * OK, let's do it. First let other backends know I'm in ANALYZE. */ LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE); - MyPgXact->vacuumFlags |= PROC_IN_ANALYZE; + MyProc->vacuumFlags |= PROC_IN_ANALYZE; + ProcGlobal->vacuumFlags[MyProc->pgxactoff] = MyProc->vacuumFlags; LWLockRelease(ProcArrayLock); pgstat_progress_start_command(PROGRESS_COMMAND_ANALYZE, RelationGetRelid(onerel)); @@ -281,11 +282,12 @@ analyze_rel(Oid relid, RangeVar *relation, pgstat_progress_end_command(); /* - * Reset my PGXACT flag. Note: we need this here, and not in vacuum_rel, - * because the vacuum flag is cleared by the end-of-xact code. + * Reset vacuumFlags we set early. Note: we need this here, and not in + * vacuum_rel, because the vacuum flag is cleared by the end-of-xact code. */ LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE); - MyPgXact->vacuumFlags &= ~PROC_IN_ANALYZE; + MyProc->vacuumFlags &= ~PROC_IN_ANALYZE; + ProcGlobal->vacuumFlags[MyProc->pgxactoff] = MyProc->vacuumFlags; LWLockRelease(ProcArrayLock); } diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c index 648e12c78d8..aba13c31d1b 100644 --- a/src/backend/commands/vacuum.c +++ b/src/backend/commands/vacuum.c @@ -1728,9 +1728,10 @@ vacuum_rel(Oid relid, RangeVar *relation, VacuumParams *params) * might appear to go backwards, which is probably Not Good. */ LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE); - MyPgXact->vacuumFlags |= PROC_IN_VACUUM; + MyProc->vacuumFlags |= PROC_IN_VACUUM; if (params->is_wraparound) - MyPgXact->vacuumFlags |= PROC_VACUUM_FOR_WRAPAROUND; + MyProc->vacuumFlags |= PROC_VACUUM_FOR_WRAPAROUND; + ProcGlobal->vacuumFlags[MyProc->pgxactoff] = MyProc->vacuumFlags; LWLockRelease(ProcArrayLock); } diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c index ac97e28be19..c6ec657a936 100644 --- a/src/backend/postmaster/autovacuum.c +++ b/src/backend/postmaster/autovacuum.c @@ -2493,7 +2493,7 @@ do_autovacuum(void) tab->at_datname, tab->at_nspname, tab->at_relname); EmitErrorReport(); - /* this resets the PGXACT flags too */ + /* this resets ProcGlobal->vacuumFlags[i] too */ AbortOutOfAnyTransaction(); FlushErrorState(); MemoryContextResetAndDeleteChildren(PortalContext); @@ -2509,7 +2509,7 @@ do_autovacuum(void) did_vacuum = true; - /* the PGXACT flags are reset at the next end of transaction */ + /* ProcGlobal->vacuumFlags[i] are reset at the next end of xact */ /* be tidy */ deleted: @@ -2686,7 +2686,7 @@ perform_work_item(AutoVacuumWorkItem *workitem) cur_datname, cur_nspname, cur_relname); EmitErrorReport(); - /* this resets the PGXACT flags too */ + /* this resets ProcGlobal->vacuumFlags[i] too */ AbortOutOfAnyTransaction(); FlushErrorState(); MemoryContextResetAndDeleteChildren(PortalContext); diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c index 61902be3b0e..b416562ee2a 100644 --- a/src/backend/replication/logical/logical.c +++ b/src/backend/replication/logical/logical.c @@ -163,7 +163,8 @@ StartupDecodingContext(List *output_plugin_options, if (!IsTransactionOrTransactionBlock()) { LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE); - MyPgXact->vacuumFlags |= PROC_IN_LOGICAL_DECODING; + MyProc->vacuumFlags |= PROC_IN_LOGICAL_DECODING; + ProcGlobal->vacuumFlags[MyProc->pgxactoff] = MyProc->vacuumFlags; LWLockRelease(ProcArrayLock); } diff --git a/src/backend/replication/slot.c b/src/backend/replication/slot.c index 57bbb6288c6..ca46256f9d0 100644 --- a/src/backend/replication/slot.c +++ b/src/backend/replication/slot.c @@ -520,7 +520,8 @@ ReplicationSlotRelease(void) /* might not have been set when we've been a plain slot */ LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE); - MyPgXact->vacuumFlags &= ~PROC_IN_LOGICAL_DECODING; + MyProc->vacuumFlags &= ~PROC_IN_LOGICAL_DECODING; + ProcGlobal->vacuumFlags[MyProc->pgxactoff] = MyProc->vacuumFlags; LWLockRelease(ProcArrayLock); } diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c index a9b32565367..e72a0705abb 100644 --- a/src/backend/storage/ipc/procarray.c +++ b/src/backend/storage/ipc/procarray.c @@ -474,9 +474,12 @@ ProcArrayAdd(PGPROC *proc) (arrayP->numProcs - index) * sizeof(*arrayP->pgprocnos)); memmove(&ProcGlobal->xids[index + 1], &ProcGlobal->xids[index], (arrayP->numProcs - index) * sizeof(*ProcGlobal->xids)); + memmove(&ProcGlobal->vacuumFlags[index + 1], &ProcGlobal->vacuumFlags[index], + (arrayP->numProcs - index) * sizeof(*ProcGlobal->vacuumFlags)); arrayP->pgprocnos[index] = proc->pgprocno; ProcGlobal->xids[index] = proc->xid; + ProcGlobal->vacuumFlags[index] = proc->vacuumFlags; arrayP->numProcs++; @@ -537,6 +540,7 @@ ProcArrayRemove(PGPROC *proc, TransactionId latestXid) } Assert(TransactionIdIsValid(ProcGlobal->xids[proc->pgxactoff] == 0)); + ProcGlobal->vacuumFlags[proc->pgxactoff] = 0; for (index = 0; index < arrayP->numProcs; index++) { @@ -547,6 +551,8 @@ ProcArrayRemove(PGPROC *proc, TransactionId latestXid) (arrayP->numProcs - index - 1) * sizeof(*arrayP->pgprocnos)); memmove(&ProcGlobal->xids[index], &ProcGlobal->xids[index + 1], (arrayP->numProcs - index - 1) * sizeof(*ProcGlobal->xids)); + memmove(&ProcGlobal->vacuumFlags[index], &ProcGlobal->vacuumFlags[index + 1], + (arrayP->numProcs - index - 1) * sizeof(*ProcGlobal->vacuumFlags)); arrayP->pgprocnos[arrayP->numProcs - 1] = -1; /* for debugging */ arrayP->numProcs--; @@ -625,14 +631,24 @@ ProcArrayEndTransaction(PGPROC *proc, TransactionId latestXid) Assert(!TransactionIdIsValid(proc->xid)); proc->lxid = InvalidLocalTransactionId; - /* must be cleared with xid/xmin: */ - pgxact->vacuumFlags &= ~PROC_VACUUM_STATE_MASK; proc->xmin = InvalidTransactionId; proc->delayChkpt = false; /* be sure this is cleared in abort */ proc->recoveryConflictPending = false; Assert(pgxact->nxids == 0); Assert(pgxact->overflowed == false); + + /* must be cleared with xid/xmin: */ + /* avoid unnecessarily dirtying shared cachelines */ + if (proc->vacuumFlags & PROC_VACUUM_STATE_MASK) + { + Assert(!LWLockHeldByMe(ProcArrayLock)); + LWLockAcquire(ProcArrayLock, LW_SHARED); + Assert(proc->vacuumFlags == ProcGlobal->vacuumFlags[proc->pgxactoff]); + proc->vacuumFlags &= ~PROC_VACUUM_STATE_MASK; + ProcGlobal->vacuumFlags[proc->pgxactoff] = proc->vacuumFlags; + LWLockRelease(ProcArrayLock); + } } } @@ -653,12 +669,18 @@ ProcArrayEndTransactionInternal(PGPROC *proc, PGXACT *pgxact, ProcGlobal->xids[pgxactoff] = InvalidTransactionId; proc->xid = InvalidTransactionId; proc->lxid = InvalidLocalTransactionId; - /* must be cleared with xid/xmin: */ - pgxact->vacuumFlags &= ~PROC_VACUUM_STATE_MASK; proc->xmin = InvalidTransactionId; proc->delayChkpt = false; /* be sure this is cleared in abort */ proc->recoveryConflictPending = false; + /* must be cleared with xid/xmin: */ + /* avoid unnecessarily dirtying shared cachelines */ + if (proc->vacuumFlags & PROC_VACUUM_STATE_MASK) + { + proc->vacuumFlags &= ~PROC_VACUUM_STATE_MASK; + ProcGlobal->vacuumFlags[proc->pgxactoff] = proc->vacuumFlags; + } + /* Clear the subtransaction-XID cache too while holding the lock */ pgxact->nxids = 0; pgxact->overflowed = false; @@ -818,9 +840,8 @@ ProcArrayClearTransaction(PGPROC *proc) proc->xmin = InvalidTransactionId; proc->recoveryConflictPending = false; - /* redundant, but just in case */ - pgxact->vacuumFlags &= ~PROC_VACUUM_STATE_MASK; - proc->delayChkpt = false; + Assert(!(proc->vacuumFlags & PROC_VACUUM_STATE_MASK)); + Assert(!proc->delayChkpt); /* Clear the subtransaction-XID cache too */ pgxact->nxids = 0; @@ -1611,7 +1632,7 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h) { int pgprocno = arrayP->pgprocnos[index]; PGPROC *proc = &allProcs[pgprocno]; - PGXACT *pgxact = &allPgXact[pgprocno]; + int8 vacuumFlags = ProcGlobal->vacuumFlags[index]; TransactionId xid; TransactionId xmin; @@ -1628,10 +1649,6 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h) */ xmin = TransactionIdOlder(xmin, xid); - /* if neither is set, this proc doesn't influence the horizon */ - if (!TransactionIdIsValid(xmin)) - continue; - /* * Don't ignore any procs when determining which transactions might be * considered running. While slots should ensure logical decoding @@ -1646,7 +1663,7 @@ ComputeXidHorizons(ComputeXidHorizonsResult *h) * removed, as long as pg_subtrans is not truncated) or doing logical * decoding (which manages xmin separately, check below). */ - if (pgxact->vacuumFlags & (PROC_IN_VACUUM | PROC_IN_LOGICAL_DECODING)) + if (vacuumFlags & (PROC_IN_VACUUM | PROC_IN_LOGICAL_DECODING)) continue; /* shared tables need to take backends in all database into account */ @@ -1984,6 +2001,7 @@ GetSnapshotData(Snapshot snapshot) size_t numProcs = arrayP->numProcs; TransactionId *xip = snapshot->xip; int *pgprocnos = arrayP->pgprocnos; + uint8 *allVacuumFlags = ProcGlobal->vacuumFlags; /* * First collect set of pgxactoff/xids that need to be included in the @@ -1993,8 +2011,6 @@ GetSnapshotData(Snapshot snapshot) { /* Fetch xid just once - see GetNewTransactionId */ TransactionId xid = UINT32_ACCESS_ONCE(other_xids[pgxactoff]); - int pgprocno; - PGXACT *pgxact; uint8 vacuumFlags; Assert(allProcs[arrayP->pgprocnos[pgxactoff]].pgxactoff == pgxactoff); @@ -2030,14 +2046,11 @@ GetSnapshotData(Snapshot snapshot) if (!NormalTransactionIdPrecedes(xid, xmax)) continue; - pgprocno = pgprocnos[pgxactoff]; - pgxact = &allPgXact[pgprocno]; - vacuumFlags = pgxact->vacuumFlags; - /* * Skip over backends doing logical decoding which manages xmin * separately (check below) and ones running LAZY VACUUM. */ + vacuumFlags = allVacuumFlags[pgxactoff]; if (vacuumFlags & (PROC_IN_LOGICAL_DECODING | PROC_IN_VACUUM)) continue; @@ -2064,6 +2077,9 @@ GetSnapshotData(Snapshot snapshot) */ if (!suboverflowed) { + int pgprocno = pgprocnos[pgxactoff]; + PGXACT *pgxact = &allPgXact[pgprocno]; + if (pgxact->overflowed) suboverflowed = true; else @@ -2282,11 +2298,11 @@ ProcArrayInstallImportedXmin(TransactionId xmin, { int pgprocno = arrayP->pgprocnos[index]; PGPROC *proc = &allProcs[pgprocno]; - PGXACT *pgxact = &allPgXact[pgprocno]; + int vacuumFlags = ProcGlobal->vacuumFlags[index]; TransactionId xid; /* Ignore procs running LAZY VACUUM */ - if (pgxact->vacuumFlags & PROC_IN_VACUUM) + if (vacuumFlags & PROC_IN_VACUUM) continue; /* We are only interested in the specific virtual transaction. */ @@ -2975,12 +2991,12 @@ GetCurrentVirtualXIDs(TransactionId limitXmin, bool excludeXmin0, { int pgprocno = arrayP->pgprocnos[index]; PGPROC *proc = &allProcs[pgprocno]; - PGXACT *pgxact = &allPgXact[pgprocno]; + uint8 vacuumFlags = ProcGlobal->vacuumFlags[index]; if (proc == MyProc) continue; - if (excludeVacuum & pgxact->vacuumFlags) + if (excludeVacuum & vacuumFlags) continue; if (allDbs || proc->databaseId == MyDatabaseId) @@ -3395,7 +3411,7 @@ CountOtherDBBackends(Oid databaseId, int *nbackends, int *nprepared) { int pgprocno = arrayP->pgprocnos[index]; PGPROC *proc = &allProcs[pgprocno]; - PGXACT *pgxact = &allPgXact[pgprocno]; + uint8 vacuumFlags = ProcGlobal->vacuumFlags[index]; if (proc->databaseId != databaseId) continue; @@ -3409,7 +3425,7 @@ CountOtherDBBackends(Oid databaseId, int *nbackends, int *nprepared) else { (*nbackends)++; - if ((pgxact->vacuumFlags & PROC_IS_AUTOVACUUM) && + if ((vacuumFlags & PROC_IS_AUTOVACUUM) && nautovacs < MAXAUTOVACPIDS) autovac_pids[nautovacs++] = proc->pid; } diff --git a/src/backend/storage/lmgr/deadlock.c b/src/backend/storage/lmgr/deadlock.c index beedc7947db..e1246b8a4da 100644 --- a/src/backend/storage/lmgr/deadlock.c +++ b/src/backend/storage/lmgr/deadlock.c @@ -544,7 +544,6 @@ FindLockCycleRecurseMember(PGPROC *checkProc, { PGPROC *proc; LOCK *lock = checkProc->waitLock; - PGXACT *pgxact; PROCLOCK *proclock; SHM_QUEUE *procLocks; LockMethod lockMethodTable; @@ -582,7 +581,6 @@ FindLockCycleRecurseMember(PGPROC *checkProc, PGPROC *leader; proc = proclock->tag.myProc; - pgxact = &ProcGlobal->allPgXact[proc->pgprocno]; leader = proc->lockGroupLeader == NULL ? proc : proc->lockGroupLeader; /* A proc never blocks itself or any other lock group member */ @@ -630,7 +628,7 @@ FindLockCycleRecurseMember(PGPROC *checkProc, * ProcArrayLock. */ if (checkProc == MyProc && - pgxact->vacuumFlags & PROC_IS_AUTOVACUUM) + proc->vacuumFlags & PROC_IS_AUTOVACUUM) blocking_autovacuum_proc = proc; /* We're done looking at this proclock */ diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c index 7fad49544ce..f6113b2d243 100644 --- a/src/backend/storage/lmgr/proc.c +++ b/src/backend/storage/lmgr/proc.c @@ -114,6 +114,7 @@ ProcGlobalShmemSize(void) size = add_size(size, mul_size(NUM_AUXILIARY_PROCS, sizeof(PGXACT))); size = add_size(size, mul_size(max_prepared_xacts, sizeof(PGXACT))); size = add_size(size, mul_size(TotalProcs, sizeof(*ProcGlobal->xids))); + size = add_size(size, mul_size(TotalProcs, sizeof(*ProcGlobal->vacuumFlags))); return size; } @@ -223,6 +224,8 @@ InitProcGlobal(void) ProcGlobal->xids = (TransactionId *) ShmemAlloc(TotalProcs * sizeof(*ProcGlobal->xids)); MemSet(ProcGlobal->xids, 0, TotalProcs * sizeof(*ProcGlobal->xids)); + ProcGlobal->vacuumFlags = (uint8 *) ShmemAlloc(TotalProcs * sizeof(*ProcGlobal->vacuumFlags)); + MemSet(ProcGlobal->vacuumFlags, 0, TotalProcs * sizeof(*ProcGlobal->vacuumFlags)); for (i = 0; i < TotalProcs; i++) { @@ -405,10 +408,10 @@ InitProcess(void) MyProc->tempNamespaceId = InvalidOid; MyProc->isBackgroundWorker = IsBackgroundWorker; MyProc->delayChkpt = false; - MyPgXact->vacuumFlags = 0; + MyProc->vacuumFlags = 0; /* NB -- autovac launcher intentionally does not set IS_AUTOVACUUM */ if (IsAutoVacuumWorkerProcess()) - MyPgXact->vacuumFlags |= PROC_IS_AUTOVACUUM; + MyProc->vacuumFlags |= PROC_IS_AUTOVACUUM; MyProc->lwWaiting = false; MyProc->lwWaitMode = 0; MyProc->waitLock = NULL; @@ -587,7 +590,7 @@ InitAuxiliaryProcess(void) MyProc->tempNamespaceId = InvalidOid; MyProc->isBackgroundWorker = IsBackgroundWorker; MyProc->delayChkpt = false; - MyPgXact->vacuumFlags = 0; + MyProc->vacuumFlags = 0; MyProc->lwWaiting = false; MyProc->lwWaitMode = 0; MyProc->waitLock = NULL; @@ -1323,7 +1326,7 @@ ProcSleep(LOCALLOCK *locallock, LockMethod lockMethodTable) if (deadlock_state == DS_BLOCKED_BY_AUTOVACUUM && allow_autovacuum_cancel) { PGPROC *autovac = GetBlockingAutoVacuumPgproc(); - PGXACT *autovac_pgxact = &ProcGlobal->allPgXact[autovac->pgprocno]; + uint8 vacuumFlags; LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE); @@ -1331,8 +1334,9 @@ ProcSleep(LOCALLOCK *locallock, LockMethod lockMethodTable) * Only do it if the worker is not working to protect against Xid * wraparound. */ - if ((autovac_pgxact->vacuumFlags & PROC_IS_AUTOVACUUM) && - !(autovac_pgxact->vacuumFlags & PROC_VACUUM_FOR_WRAPAROUND)) + vacuumFlags = ProcGlobal->vacuumFlags[proc->pgxactoff]; + if ((vacuumFlags & PROC_IS_AUTOVACUUM) && + !(vacuumFlags & PROC_VACUUM_FOR_WRAPAROUND)) { int pid = autovac->pid; StringInfoData locktagbuf; -- 2.25.0.114.g5b0ca878e0 --kdyesixubxkl3b7r Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v11-0005-snapshot-scalability-Move-subxact-info-to-ProcGl.patch" From 8c58389951790366394c1ec82ac4ad1493e60030 Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Wed, 15 Jul 2020 15:35:07 -0700 Subject: [PATCH v11 5/6] snapshot scalability: Move subxact info to ProcGlobal, remove PGXACT. Similar to the previous changes this increases the chance that data frequently needed by GetSnapshotData() stays in l2 cache. In many workloads subtransactions are very rare, and this makes the check for that considerably cheaper. As this removes the last member of PGXACT, there is no need to keep it around anymore. Author: Andres Freund Reviewed-By: Robert Haas, Thomas Munro, David Rowley Discussion: https://postgr.es/m/20200301083601.ews6hz5dduc3w2se@alap3.anarazel.de --- src/include/storage/proc.h | 34 ++++--- src/backend/access/transam/clog.c | 7 +- src/backend/access/transam/twophase.c | 17 ++-- src/backend/access/transam/varsup.c | 15 ++- src/backend/storage/ipc/procarray.c | 128 ++++++++++++++------------ src/backend/storage/lmgr/proc.c | 24 +---- src/tools/pgindent/typedefs.list | 1 - 7 files changed, 113 insertions(+), 113 deletions(-) diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h index ffb775939ed..36fe5253a15 100644 --- a/src/include/storage/proc.h +++ b/src/include/storage/proc.h @@ -35,6 +35,14 @@ */ #define PGPROC_MAX_CACHED_SUBXIDS 64 /* XXX guessed-at value */ +typedef struct XidCacheStatus +{ + /* number of cached subxids, never more than PGPROC_MAX_CACHED_SUBXIDS */ + uint8 count; + /* has PGPROC->subxids overflowed */ + bool overflowed; +} XidCacheStatus; + struct XidCache { TransactionId xids[PGPROC_MAX_CACHED_SUBXIDS]; @@ -188,6 +196,8 @@ struct PGPROC */ SHM_QUEUE myProcLocks[NUM_LOCK_PARTITIONS]; + XidCacheStatus subxidStatus; /* mirrored with + * ProcGlobal->subxidStates[i] */ struct XidCache subxids; /* cache for subtransaction XIDs */ /* Support for group XID clearing. */ @@ -236,22 +246,6 @@ struct PGPROC extern PGDLLIMPORT PGPROC *MyProc; -extern PGDLLIMPORT struct PGXACT *MyPgXact; - -/* - * Prior to PostgreSQL 9.2, the fields below were stored as part of the - * PGPROC. However, benchmarking revealed that packing these particular - * members into a separate array as tightly as possible sped up GetSnapshotData - * considerably on systems with many CPU cores, by reducing the number of - * cache lines needing to be fetched. Thus, think very carefully before adding - * anything else here. - */ -typedef struct PGXACT -{ - bool overflowed; - - uint8 nxids; -} PGXACT; /* * There is one ProcGlobal struct for the whole database cluster. @@ -311,12 +305,16 @@ typedef struct PROC_HDR { /* Array of PGPROC structures (not including dummies for prepared txns) */ PGPROC *allProcs; - /* Array of PGXACT structures (not including dummies for prepared txns) */ - PGXACT *allPgXact; /* Array mirroring PGPROC.xid for each PGPROC currently in the procarray */ TransactionId *xids; + /* + * Array mirroring PGPROC.subxidStatus for each PGPROC currently in the + * procarray. + */ + XidCacheStatus *subxidStates; + /* * Array mirroring PGPROC.vacuumFlags for each PGPROC currently in the * procarray. diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c index 5198a0cef68..a3095ced3fb 100644 --- a/src/backend/access/transam/clog.c +++ b/src/backend/access/transam/clog.c @@ -295,7 +295,7 @@ TransactionIdSetPageStatus(TransactionId xid, int nsubxids, */ if (all_xact_same_page && xid == MyProc->xid && nsubxids <= THRESHOLD_SUBTRANS_CLOG_OPT && - nsubxids == MyPgXact->nxids && + nsubxids == MyProc->subxidStatus.count && memcmp(subxids, MyProc->subxids.xids, nsubxids * sizeof(TransactionId)) == 0) { @@ -510,16 +510,15 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status, while (nextidx != INVALID_PGPROCNO) { PGPROC *proc = &ProcGlobal->allProcs[nextidx]; - PGXACT *pgxact = &ProcGlobal->allPgXact[nextidx]; /* * Transactions with more than THRESHOLD_SUBTRANS_CLOG_OPT sub-XIDs * should not use group XID status update mechanism. */ - Assert(pgxact->nxids <= THRESHOLD_SUBTRANS_CLOG_OPT); + Assert(proc->subxidStatus.count <= THRESHOLD_SUBTRANS_CLOG_OPT); TransactionIdSetPageStatusInternal(proc->clogGroupMemberXid, - pgxact->nxids, + proc->subxidStatus.count, proc->subxids.xids, proc->clogGroupMemberXidStatus, proc->clogGroupMemberLsn, diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c index 3371ebd8896..2642eaf99de 100644 --- a/src/backend/access/transam/twophase.c +++ b/src/backend/access/transam/twophase.c @@ -21,9 +21,9 @@ * GIDs and aborts the transaction if there already is a global * transaction in prepared state with the same GID. * - * A global transaction (gxact) also has dummy PGXACT and PGPROC; this is - * what keeps the XID considered running by TransactionIdIsInProgress. - * It is also convenient as a PGPROC to hook the gxact's locks to. + * A global transaction (gxact) also has dummy PGPROC; this is what keeps + * the XID considered running by TransactionIdIsInProgress. It is also + * convenient as a PGPROC to hook the gxact's locks to. * * Information to recover prepared transactions in case of crash is * now stored in WAL for the common case. In some cases there will be @@ -447,14 +447,12 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid, TimestampTz prepared_at, Oid owner, Oid databaseid) { PGPROC *proc; - PGXACT *pgxact; int i; Assert(LWLockHeldByMeInMode(TwoPhaseStateLock, LW_EXCLUSIVE)); Assert(gxact != NULL); proc = &ProcGlobal->allProcs[gxact->pgprocno]; - pgxact = &ProcGlobal->allPgXact[gxact->pgprocno]; /* Initialize the PGPROC entry */ MemSet(proc, 0, sizeof(PGPROC)); @@ -480,8 +478,8 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid, for (i = 0; i < NUM_LOCK_PARTITIONS; i++) SHMQueueInit(&(proc->myProcLocks[i])); /* subxid data must be filled later by GXactLoadSubxactData */ - pgxact->overflowed = false; - pgxact->nxids = 0; + proc->subxidStatus.count = 0; + proc->subxidStatus.overflowed = 0; gxact->prepared_at = prepared_at; gxact->xid = xid; @@ -510,19 +508,18 @@ GXactLoadSubxactData(GlobalTransaction gxact, int nsubxacts, TransactionId *children) { PGPROC *proc = &ProcGlobal->allProcs[gxact->pgprocno]; - PGXACT *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno]; /* We need no extra lock since the GXACT isn't valid yet */ if (nsubxacts > PGPROC_MAX_CACHED_SUBXIDS) { - pgxact->overflowed = true; + proc->subxidStatus.overflowed = true; nsubxacts = PGPROC_MAX_CACHED_SUBXIDS; } if (nsubxacts > 0) { memcpy(proc->subxids.xids, children, nsubxacts * sizeof(TransactionId)); - pgxact->nxids = nsubxacts; + proc->subxidStatus.count = PGPROC_MAX_CACHED_SUBXIDS; } } diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c index ab376f2fe22..97471ffe488 100644 --- a/src/backend/access/transam/varsup.c +++ b/src/backend/access/transam/varsup.c @@ -222,22 +222,31 @@ GetNewTransactionId(bool isSubXact) */ if (!isSubXact) { + Assert(ProcGlobal->subxidStates[MyProc->pgxactoff].count == 0); + Assert(!ProcGlobal->subxidStates[MyProc->pgxactoff].overflowed); + Assert(MyProc->subxidStatus.count == 0); + Assert(!MyProc->subxidStatus.overflowed); + /* LWLockRelease acts as barrier */ MyProc->xid = xid; ProcGlobal->xids[MyProc->pgxactoff] = xid; } else { - int nxids = MyPgXact->nxids; + XidCacheStatus *substat = &ProcGlobal->subxidStates[MyProc->pgxactoff]; + int nxids = MyProc->subxidStatus.count; + + Assert(substat->count == MyProc->subxidStatus.count); + Assert(substat->overflowed == MyProc->subxidStatus.overflowed); if (nxids < PGPROC_MAX_CACHED_SUBXIDS) { MyProc->subxids.xids[nxids] = xid; pg_write_barrier(); - MyPgXact->nxids = nxids + 1; + MyProc->subxidStatus.count = substat->count = nxids + 1; } else - MyPgXact->overflowed = true; + MyProc->subxidStatus.overflowed = substat->overflowed = true; } LWLockRelease(XidGenLock); diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c index e72a0705abb..bf3e5b65dc7 100644 --- a/src/backend/storage/ipc/procarray.c +++ b/src/backend/storage/ipc/procarray.c @@ -4,9 +4,10 @@ * POSTGRES process array code. * * - * This module maintains arrays of the PGPROC and PGXACT structures for all - * active backends. Although there are several uses for this, the principal - * one is as a means of determining the set of currently running transactions. + * This module maintains arrays of PGPROC substructures, as well as associated + * arrays in ProcGlobal, for all active backends. Although there are several + * uses for this, the principal one is as a means of determining the set of + * currently running transactions. * * Because of various subtle race conditions it is critical that a backend * hold the correct locks while setting or clearing its xid (in @@ -85,7 +86,7 @@ typedef struct ProcArrayStruct /* * Highest subxid that has been removed from KnownAssignedXids array to * prevent overflow; or InvalidTransactionId if none. We track this for - * similar reasons to tracking overflowing cached subxids in PGXACT + * similar reasons to tracking overflowing cached subxids in PGPROC * entries. Must hold exclusive ProcArrayLock to change this, and shared * lock to read it. */ @@ -96,7 +97,7 @@ typedef struct ProcArrayStruct /* oldest catalog xmin of any replication slot */ TransactionId replication_slot_catalog_xmin; - /* indexes into allPgXact[], has PROCARRAY_MAXPROCS entries */ + /* indexes into allProcs[], has PROCARRAY_MAXPROCS entries */ int pgprocnos[FLEXIBLE_ARRAY_MEMBER]; } ProcArrayStruct; @@ -239,7 +240,6 @@ typedef struct ComputeXidHorizonsResult static ProcArrayStruct *procArray; static PGPROC *allProcs; -static PGXACT *allPgXact; /* * Bookkeeping for tracking emulated transactions in recovery @@ -325,8 +325,7 @@ static int KnownAssignedXidsGetAndSetXmin(TransactionId *xarray, static TransactionId KnownAssignedXidsGetOldestXmin(void); static void KnownAssignedXidsDisplay(int trace_level); static void KnownAssignedXidsReset(void); -static inline void ProcArrayEndTransactionInternal(PGPROC *proc, - PGXACT *pgxact, TransactionId latestXid); +static inline void ProcArrayEndTransactionInternal(PGPROC *proc, TransactionId latestXid); static void ProcArrayGroupClearXid(PGPROC *proc, TransactionId latestXid); static void MaintainLatestCompletedXid(TransactionId latestXid); static void MaintainLatestCompletedXidRecovery(TransactionId latestXid); @@ -409,7 +408,6 @@ CreateSharedProcArray(void) } allProcs = ProcGlobal->allProcs; - allPgXact = ProcGlobal->allPgXact; /* Create or attach to the KnownAssignedXids arrays too, if needed */ if (EnableHotStandby) @@ -474,11 +472,14 @@ ProcArrayAdd(PGPROC *proc) (arrayP->numProcs - index) * sizeof(*arrayP->pgprocnos)); memmove(&ProcGlobal->xids[index + 1], &ProcGlobal->xids[index], (arrayP->numProcs - index) * sizeof(*ProcGlobal->xids)); + memmove(&ProcGlobal->subxidStates[index + 1], &ProcGlobal->subxidStates[index], + (arrayP->numProcs - index) * sizeof(*ProcGlobal->subxidStates)); memmove(&ProcGlobal->vacuumFlags[index + 1], &ProcGlobal->vacuumFlags[index], (arrayP->numProcs - index) * sizeof(*ProcGlobal->vacuumFlags)); arrayP->pgprocnos[index] = proc->pgprocno; ProcGlobal->xids[index] = proc->xid; + ProcGlobal->subxidStates[index] = proc->subxidStatus; ProcGlobal->vacuumFlags[index] = proc->vacuumFlags; arrayP->numProcs++; @@ -532,6 +533,8 @@ ProcArrayRemove(PGPROC *proc, TransactionId latestXid) MaintainLatestCompletedXid(latestXid); ProcGlobal->xids[proc->pgxactoff] = 0; + ProcGlobal->subxidStates[proc->pgxactoff].overflowed = false; + ProcGlobal->subxidStates[proc->pgxactoff].count = 0; } else { @@ -540,6 +543,8 @@ ProcArrayRemove(PGPROC *proc, TransactionId latestXid) } Assert(TransactionIdIsValid(ProcGlobal->xids[proc->pgxactoff] == 0)); + Assert(TransactionIdIsValid(ProcGlobal->subxidStates[proc->pgxactoff].count == 0)); + Assert(TransactionIdIsValid(ProcGlobal->subxidStates[proc->pgxactoff].overflowed == false)); ProcGlobal->vacuumFlags[proc->pgxactoff] = 0; for (index = 0; index < arrayP->numProcs; index++) @@ -551,6 +556,8 @@ ProcArrayRemove(PGPROC *proc, TransactionId latestXid) (arrayP->numProcs - index - 1) * sizeof(*arrayP->pgprocnos)); memmove(&ProcGlobal->xids[index], &ProcGlobal->xids[index + 1], (arrayP->numProcs - index - 1) * sizeof(*ProcGlobal->xids)); + memmove(&ProcGlobal->subxidStates[index], &ProcGlobal->subxidStates[index + 1], + (arrayP->numProcs - index - 1) * sizeof(*ProcGlobal->subxidStates)); memmove(&ProcGlobal->vacuumFlags[index], &ProcGlobal->vacuumFlags[index + 1], (arrayP->numProcs - index - 1) * sizeof(*ProcGlobal->vacuumFlags)); @@ -596,8 +603,6 @@ ProcArrayRemove(PGPROC *proc, TransactionId latestXid) void ProcArrayEndTransaction(PGPROC *proc, TransactionId latestXid) { - PGXACT *pgxact = &allPgXact[proc->pgprocno]; - if (TransactionIdIsValid(latestXid)) { /* @@ -615,7 +620,7 @@ ProcArrayEndTransaction(PGPROC *proc, TransactionId latestXid) */ if (LWLockConditionalAcquire(ProcArrayLock, LW_EXCLUSIVE)) { - ProcArrayEndTransactionInternal(proc, pgxact, latestXid); + ProcArrayEndTransactionInternal(proc, latestXid); LWLockRelease(ProcArrayLock); } else @@ -629,15 +634,14 @@ ProcArrayEndTransaction(PGPROC *proc, TransactionId latestXid) * estimate of global xmin, but that's OK. */ Assert(!TransactionIdIsValid(proc->xid)); + Assert(proc->subxidStatus.count == 0); + Assert(!proc->subxidStatus.overflowed); proc->lxid = InvalidLocalTransactionId; proc->xmin = InvalidTransactionId; proc->delayChkpt = false; /* be sure this is cleared in abort */ proc->recoveryConflictPending = false; - Assert(pgxact->nxids == 0); - Assert(pgxact->overflowed == false); - /* must be cleared with xid/xmin: */ /* avoid unnecessarily dirtying shared cachelines */ if (proc->vacuumFlags & PROC_VACUUM_STATE_MASK) @@ -658,8 +662,7 @@ ProcArrayEndTransaction(PGPROC *proc, TransactionId latestXid) * We don't do any locking here; caller must handle that. */ static inline void -ProcArrayEndTransactionInternal(PGPROC *proc, PGXACT *pgxact, - TransactionId latestXid) +ProcArrayEndTransactionInternal(PGPROC *proc, TransactionId latestXid) { size_t pgxactoff = proc->pgxactoff; @@ -682,8 +685,15 @@ ProcArrayEndTransactionInternal(PGPROC *proc, PGXACT *pgxact, } /* Clear the subtransaction-XID cache too while holding the lock */ - pgxact->nxids = 0; - pgxact->overflowed = false; + Assert(ProcGlobal->subxidStates[pgxactoff].count == proc->subxidStatus.count && + ProcGlobal->subxidStates[pgxactoff].overflowed == proc->subxidStatus.overflowed); + if (proc->subxidStatus.count > 0 || proc->subxidStatus.overflowed) + { + ProcGlobal->subxidStates[pgxactoff].count = 0; + ProcGlobal->subxidStates[pgxactoff].overflowed = false; + proc->subxidStatus.count = 0; + proc->subxidStatus.overflowed = false; + } /* Also advance global latestCompletedXid while holding the lock */ MaintainLatestCompletedXid(latestXid); @@ -773,9 +783,8 @@ ProcArrayGroupClearXid(PGPROC *proc, TransactionId latestXid) while (nextidx != INVALID_PGPROCNO) { PGPROC *proc = &allProcs[nextidx]; - PGXACT *pgxact = &allPgXact[nextidx]; - ProcArrayEndTransactionInternal(proc, pgxact, proc->procArrayGroupMemberXid); + ProcArrayEndTransactionInternal(proc, proc->procArrayGroupMemberXid); /* Move to next proc in list. */ nextidx = pg_atomic_read_u32(&proc->procArrayGroupNext); @@ -819,7 +828,6 @@ ProcArrayGroupClearXid(PGPROC *proc, TransactionId latestXid) void ProcArrayClearTransaction(PGPROC *proc) { - PGXACT *pgxact = &allPgXact[proc->pgprocno]; size_t pgxactoff; /* @@ -844,8 +852,15 @@ ProcArrayClearTransaction(PGPROC *proc) Assert(!proc->delayChkpt); /* Clear the subtransaction-XID cache too */ - pgxact->nxids = 0; - pgxact->overflowed = false; + Assert(ProcGlobal->subxidStates[pgxactoff].count == proc->subxidStatus.count && + ProcGlobal->subxidStates[pgxactoff].overflowed == proc->subxidStatus.overflowed); + if (proc->subxidStatus.count > 0 || proc->subxidStatus.overflowed) + { + ProcGlobal->subxidStates[pgxactoff].count = 0; + ProcGlobal->subxidStates[pgxactoff].overflowed = false; + proc->subxidStatus.count = 0; + proc->subxidStatus.overflowed = false; + } LWLockRelease(ProcArrayLock); } @@ -1265,6 +1280,7 @@ TransactionIdIsInProgress(TransactionId xid) { static TransactionId *xids = NULL; static TransactionId *other_xids; + XidCacheStatus *other_subxidstates; int nxids = 0; ProcArrayStruct *arrayP = procArray; TransactionId topxid; @@ -1326,6 +1342,7 @@ TransactionIdIsInProgress(TransactionId xid) } other_xids = ProcGlobal->xids; + other_subxidstates = ProcGlobal->subxidStates; LWLockAcquire(ProcArrayLock, LW_SHARED); @@ -1347,7 +1364,6 @@ TransactionIdIsInProgress(TransactionId xid) for (size_t pgxactoff = 0; pgxactoff < numProcs; pgxactoff++) { int pgprocno; - PGXACT *pgxact; PGPROC *proc; TransactionId pxid; int pxids; @@ -1382,9 +1398,7 @@ TransactionIdIsInProgress(TransactionId xid) /* * Step 2: check the cached child-Xids arrays */ - pgprocno = arrayP->pgprocnos[pgxactoff]; - pgxact = &allPgXact[pgprocno]; - pxids = pgxact->nxids; + pxids = other_subxidstates[pgxactoff].count; pg_read_barrier(); /* pairs with barrier in GetNewTransactionId() */ pgprocno = arrayP->pgprocnos[pgxactoff]; proc = &allProcs[pgprocno]; @@ -1408,7 +1422,7 @@ TransactionIdIsInProgress(TransactionId xid) * we hold ProcArrayLock. So we can't miss an Xid that we need to * worry about.) */ - if (pgxact->overflowed) + if (other_subxidstates[pgxactoff].overflowed) xids[nxids++] = pxid; } @@ -2001,6 +2015,7 @@ GetSnapshotData(Snapshot snapshot) size_t numProcs = arrayP->numProcs; TransactionId *xip = snapshot->xip; int *pgprocnos = arrayP->pgprocnos; + XidCacheStatus *subxidStates = ProcGlobal->subxidStates; uint8 *allVacuumFlags = ProcGlobal->vacuumFlags; /* @@ -2077,17 +2092,16 @@ GetSnapshotData(Snapshot snapshot) */ if (!suboverflowed) { - int pgprocno = pgprocnos[pgxactoff]; - PGXACT *pgxact = &allPgXact[pgprocno]; - if (pgxact->overflowed) + if (subxidStates[pgxactoff].overflowed) suboverflowed = true; else { - int nsubxids = pgxact->nxids; + int nsubxids = subxidStates[pgxactoff].count; if (nsubxids > 0) { + int pgprocno = pgprocnos[pgxactoff]; PGPROC *proc = &allProcs[pgprocno]; pg_read_barrier(); /* pairs with GetNewTransactionId */ @@ -2479,8 +2493,6 @@ GetRunningTransactionData(void) */ for (index = 0; index < arrayP->numProcs; index++) { - int pgprocno = arrayP->pgprocnos[index]; - PGXACT *pgxact = &allPgXact[pgprocno]; TransactionId xid; /* Fetch xid just once - see GetNewTransactionId */ @@ -2501,7 +2513,7 @@ GetRunningTransactionData(void) if (TransactionIdPrecedes(xid, oldestRunningXid)) oldestRunningXid = xid; - if (pgxact->overflowed) + if (ProcGlobal->subxidStates[index].overflowed) suboverflowed = true; /* @@ -2521,27 +2533,28 @@ GetRunningTransactionData(void) */ if (!suboverflowed) { + XidCacheStatus *other_subxidstates = ProcGlobal->subxidStates; + for (index = 0; index < arrayP->numProcs; index++) { int pgprocno = arrayP->pgprocnos[index]; PGPROC *proc = &allProcs[pgprocno]; - PGXACT *pgxact = &allPgXact[pgprocno]; - int nxids; + int nsubxids; /* * Save subtransaction XIDs. Other backends can't add or remove * entries while we're holding XidGenLock. */ - nxids = pgxact->nxids; - if (nxids > 0) + nsubxids = other_subxidstates[index].count; + if (nsubxids > 0) { /* barrier not really required, as XidGenLock is held, but ... */ pg_read_barrier(); /* pairs with GetNewTransactionId */ memcpy(&xids[count], (void *) proc->subxids.xids, - nxids * sizeof(TransactionId)); - count += nxids; - subcount += nxids; + nsubxids * sizeof(TransactionId)); + count += nsubxids; + subcount += nsubxids; /* * Top-level XID of a transaction is always less than any of @@ -3608,14 +3621,6 @@ ProcArrayGetReplicationSlotXmin(TransactionId *xmin, LWLockRelease(ProcArrayLock); } - -#define XidCacheRemove(i) \ - do { \ - MyProc->subxids.xids[i] = MyProc->subxids.xids[MyPgXact->nxids - 1]; \ - pg_write_barrier(); \ - MyPgXact->nxids--; \ - } while (0) - /* * XidCacheRemoveRunningXids * @@ -3631,6 +3636,7 @@ XidCacheRemoveRunningXids(TransactionId xid, { int i, j; + XidCacheStatus *mysubxidstat; Assert(TransactionIdIsValid(xid)); @@ -3648,6 +3654,8 @@ XidCacheRemoveRunningXids(TransactionId xid, */ LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE); + mysubxidstat = &ProcGlobal->subxidStates[MyProc->pgxactoff]; + /* * Under normal circumstances xid and xids[] will be in increasing order, * as will be the entries in subxids. Scan backwards to avoid O(N^2) @@ -3657,11 +3665,14 @@ XidCacheRemoveRunningXids(TransactionId xid, { TransactionId anxid = xids[i]; - for (j = MyPgXact->nxids - 1; j >= 0; j--) + for (j = MyProc->subxidStatus.count - 1; j >= 0; j--) { if (TransactionIdEquals(MyProc->subxids.xids[j], anxid)) { - XidCacheRemove(j); + MyProc->subxids.xids[j] = MyProc->subxids.xids[MyProc->subxidStatus.count - 1]; + pg_write_barrier(); + mysubxidstat->count--; + MyProc->subxidStatus.count--; break; } } @@ -3673,20 +3684,23 @@ XidCacheRemoveRunningXids(TransactionId xid, * error during AbortSubTransaction. So instead of Assert, emit a * debug warning. */ - if (j < 0 && !MyPgXact->overflowed) + if (j < 0 && !MyProc->subxidStatus.overflowed) elog(WARNING, "did not find subXID %u in MyProc", anxid); } - for (j = MyPgXact->nxids - 1; j >= 0; j--) + for (j = MyProc->subxidStatus.count - 1; j >= 0; j--) { if (TransactionIdEquals(MyProc->subxids.xids[j], xid)) { - XidCacheRemove(j); + MyProc->subxids.xids[j] = MyProc->subxids.xids[MyProc->subxidStatus.count - 1]; + pg_write_barrier(); + mysubxidstat->count--; + MyProc->subxidStatus.count--; break; } } /* Ordinarily we should have found it, unless the cache has overflowed */ - if (j < 0 && !MyPgXact->overflowed) + if (j < 0 && !MyProc->subxidStatus.overflowed) elog(WARNING, "did not find subXID %u in MyProc", xid); /* Also advance global latestCompletedXid while holding the lock */ diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c index f6113b2d243..aa9fbd80545 100644 --- a/src/backend/storage/lmgr/proc.c +++ b/src/backend/storage/lmgr/proc.c @@ -63,9 +63,8 @@ int LockTimeout = 0; int IdleInTransactionSessionTimeout = 0; bool log_lock_waits = false; -/* Pointer to this process's PGPROC and PGXACT structs, if any */ +/* Pointer to this process's PGPROC struct, if any */ PGPROC *MyProc = NULL; -PGXACT *MyPgXact = NULL; /* * This spinlock protects the freelist of recycled PGPROC structures. @@ -110,10 +109,8 @@ ProcGlobalShmemSize(void) size = add_size(size, mul_size(TotalProcs, sizeof(PGPROC))); size = add_size(size, sizeof(slock_t)); - size = add_size(size, mul_size(MaxBackends, sizeof(PGXACT))); - size = add_size(size, mul_size(NUM_AUXILIARY_PROCS, sizeof(PGXACT))); - size = add_size(size, mul_size(max_prepared_xacts, sizeof(PGXACT))); size = add_size(size, mul_size(TotalProcs, sizeof(*ProcGlobal->xids))); + size = add_size(size, mul_size(TotalProcs, sizeof(*ProcGlobal->subxidStates))); size = add_size(size, mul_size(TotalProcs, sizeof(*ProcGlobal->vacuumFlags))); return size; @@ -161,7 +158,6 @@ void InitProcGlobal(void) { PGPROC *procs; - PGXACT *pgxacts; int i, j; bool found; @@ -202,18 +198,6 @@ InitProcGlobal(void) /* XXX allProcCount isn't really all of them; it excludes prepared xacts */ ProcGlobal->allProcCount = MaxBackends + NUM_AUXILIARY_PROCS; - /* - * Also allocate a separate array of PGXACT structures. This is separate - * from the main PGPROC array so that the most heavily accessed data is - * stored contiguously in memory in as few cache lines as possible. This - * provides significant performance benefits, especially on a - * multiprocessor system. There is one PGXACT structure for every PGPROC - * structure. - */ - pgxacts = (PGXACT *) ShmemAlloc(TotalProcs * sizeof(PGXACT)); - MemSet(pgxacts, 0, TotalProcs * sizeof(PGXACT)); - ProcGlobal->allPgXact = pgxacts; - /* * Allocate arrays mirroring PGPROC fields in a dense manner. See * PROC_HDR. @@ -224,6 +208,8 @@ InitProcGlobal(void) ProcGlobal->xids = (TransactionId *) ShmemAlloc(TotalProcs * sizeof(*ProcGlobal->xids)); MemSet(ProcGlobal->xids, 0, TotalProcs * sizeof(*ProcGlobal->xids)); + ProcGlobal->subxidStates = (XidCacheStatus *) ShmemAlloc(TotalProcs * sizeof(*ProcGlobal->subxidStates)); + MemSet(ProcGlobal->subxidStates, 0, TotalProcs * sizeof(*ProcGlobal->subxidStates)); ProcGlobal->vacuumFlags = (uint8 *) ShmemAlloc(TotalProcs * sizeof(*ProcGlobal->vacuumFlags)); MemSet(ProcGlobal->vacuumFlags, 0, TotalProcs * sizeof(*ProcGlobal->vacuumFlags)); @@ -372,7 +358,6 @@ InitProcess(void) (errcode(ERRCODE_TOO_MANY_CONNECTIONS), errmsg("sorry, too many clients already"))); } - MyPgXact = &ProcGlobal->allPgXact[MyProc->pgprocno]; /* * Cross-check that the PGPROC is of the type we expect; if this were not @@ -569,7 +554,6 @@ InitAuxiliaryProcess(void) ((volatile PGPROC *) auxproc)->pid = MyProcPid; MyProc = auxproc; - MyPgXact = &ProcGlobal->allPgXact[auxproc->pgprocno]; SpinLockRelease(ProcStructLock); diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index b4948ac675f..3d990463ce9 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -1536,7 +1536,6 @@ PGSetenvStatusType PGShmemHeader PGTransactionStatusType PGVerbosity -PGXACT PG_Locale_Strategy PG_Lock_Status PG_init_t -- 2.25.0.114.g5b0ca878e0 --kdyesixubxkl3b7r Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v11-0006-snapshot-scalability-cache-snapshots-using-a-xac.patch" From 1ad313d240fca83e4fd9f21d595792aaa3bbdade Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Wed, 15 Jul 2020 15:35:07 -0700 Subject: [PATCH v11 6/6] snapshot scalability: cache snapshots using a xact completion counter. Previous commits made it faster/more scalable to compute snapshots. But not building a snapshot is still faster. Now that GetSnapshotData() does not maintain RecentGlobal* anymore, that is actually not too hard: This commit introduces xactCompletionCount, which tracks the number of top-level transactions with xids (i.e. which may have modified the database) that completed in some form since the start of the server. We can avoid rebuilding the snapshot's contents whenever the current xactCompletionCount is the same as it was when the snapshot was originally built. Currently this check happens while holding ProcArrayLock. While it's likely possible to perform the check before acquiring ProcArrayLock, it's too complicated for now. Author: Andres Freund Reviewed-By: Robert Haas, Thomas Munro, David Rowley Discussion: https://postgr.es/m/20200301083601.ews6hz5dduc3w2se@alap3.anarazel.de --- src/include/access/transam.h | 9 ++ src/include/utils/snapshot.h | 7 ++ src/backend/replication/logical/snapbuild.c | 1 + src/backend/storage/ipc/procarray.c | 125 ++++++++++++++++---- src/backend/utils/time/snapmgr.c | 4 + 5 files changed, 126 insertions(+), 20 deletions(-) diff --git a/src/include/access/transam.h b/src/include/access/transam.h index 6ec84b54599..bc8d2b00bb7 100644 --- a/src/include/access/transam.h +++ b/src/include/access/transam.h @@ -231,6 +231,15 @@ typedef struct VariableCacheData FullTransactionId latestCompletedFullXid; /* newest full XID that has * committed or aborted */ + /* + * Number of top-level transactions with xids (i.e. which may have + * modified the database) that completed in some form since the start of + * the server. This currently is solely used to check whether + * GetSnapshotData() needs to recompute the contents of the snapshot, or + * not. There are likely other users of this. Always above 1. + */ + uint64 xactCompletionCount; + /* * These fields are protected by XactTruncationLock */ diff --git a/src/include/utils/snapshot.h b/src/include/utils/snapshot.h index 35b1f05bea6..dea072e5edf 100644 --- a/src/include/utils/snapshot.h +++ b/src/include/utils/snapshot.h @@ -207,6 +207,13 @@ typedef struct SnapshotData TimestampTz whenTaken; /* timestamp when snapshot was taken */ XLogRecPtr lsn; /* position in the WAL stream when taken */ + + /* + * The transaction completion count at the time GetSnapshotData() built + * this snapshot. Allows to avoid re-computing static snapshots when no + * transactions completed since the last GetSnapshotData(). + */ + uint64 snapXactCompletionCount; } SnapshotData; #endif /* SNAPSHOT_H */ diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c index e9701ea7221..9d5d68f3fa7 100644 --- a/src/backend/replication/logical/snapbuild.c +++ b/src/backend/replication/logical/snapbuild.c @@ -524,6 +524,7 @@ SnapBuildBuildSnapshot(SnapBuild *builder) snapshot->curcid = FirstCommandId; snapshot->active_count = 0; snapshot->regd_count = 0; + snapshot->snapXactCompletionCount = 0; return snapshot; } diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c index bf3e5b65dc7..cc88111a904 100644 --- a/src/backend/storage/ipc/procarray.c +++ b/src/backend/storage/ipc/procarray.c @@ -405,6 +405,7 @@ CreateSharedProcArray(void) procArray->lastOverflowedXid = InvalidTransactionId; procArray->replication_slot_xmin = InvalidTransactionId; procArray->replication_slot_catalog_xmin = InvalidTransactionId; + ShmemVariableCache->xactCompletionCount = 1; } allProcs = ProcGlobal->allProcs; @@ -532,6 +533,9 @@ ProcArrayRemove(PGPROC *proc, TransactionId latestXid) /* Advance global latestCompletedXid while holding the lock */ MaintainLatestCompletedXid(latestXid); + /* Same with xactCompletionCount */ + ShmemVariableCache->xactCompletionCount++; + ProcGlobal->xids[proc->pgxactoff] = 0; ProcGlobal->subxidStates[proc->pgxactoff].overflowed = false; ProcGlobal->subxidStates[proc->pgxactoff].count = 0; @@ -666,6 +670,7 @@ ProcArrayEndTransactionInternal(PGPROC *proc, TransactionId latestXid) { size_t pgxactoff = proc->pgxactoff; + Assert(LWLockHeldByMe(ProcArrayLock)); Assert(TransactionIdIsValid(ProcGlobal->xids[pgxactoff])); Assert(ProcGlobal->xids[pgxactoff] == proc->xid); @@ -697,6 +702,9 @@ ProcArrayEndTransactionInternal(PGPROC *proc, TransactionId latestXid) /* Also advance global latestCompletedXid while holding the lock */ MaintainLatestCompletedXid(latestXid); + + /* Same with xactCompletionCount */ + ShmemVariableCache->xactCompletionCount++; } /* @@ -1897,6 +1905,93 @@ GetMaxSnapshotSubxidCount(void) return TOTAL_MAX_CACHED_SUBXIDS; } +/* + * Initialize old_snapshot_threshold specific parts of a newly build snapshot. + */ +static void +GetSnapshotDataInitOldSnapshot(Snapshot snapshot) +{ + if (!OldSnapshotThresholdActive()) + { + /* + * If not using "snapshot too old" feature, fill related fields with + * dummy values that don't require any locking. + */ + snapshot->lsn = InvalidXLogRecPtr; + snapshot->whenTaken = 0; + } + else + { + /* + * Capture the current time and WAL stream location in case this + * snapshot becomes old enough to need to fall back on the special + * "old snapshot" logic. + */ + snapshot->lsn = GetXLogInsertRecPtr(); + snapshot->whenTaken = GetSnapshotCurrentTimestamp(); + MaintainOldSnapshotTimeMapping(snapshot->whenTaken, snapshot->xmin); + } +} + +/* + * Helper function for GetSnapshotData() that check if the bulk of the + * visibility information in the snapshot is still valid. If so, it updates + * the fields that need to change and returns true. Otherwise it returns + * false. + * + * This very likely can be evolved to not need ProcArrayLock held (at very + * least in the case we already hold a snapshot), but that's for another day. + */ +static bool +GetSnapshotDataReuse(Snapshot snapshot) +{ + uint64 curXactCompletionCount; + + Assert(LWLockHeldByMe(ProcArrayLock)); + + if (unlikely(snapshot->snapXactCompletionCount == 0)) + return false; + + curXactCompletionCount = ShmemVariableCache->xactCompletionCount; + if (curXactCompletionCount != snapshot->snapXactCompletionCount) + return false; + + /* + * If the current xactCompletionCount is still the same as it was at the + * time the snapshot was built, we can be sure that rebuilding the + * contents of the snapshot the hard way would result in the same snapshot + * contents: + * + * As explained in transam/README, the set of xids considered running by + * GetSnapshotData() cannot change while ProcArrayLock is held. Snapshot + * contents only depend on transactions with xids and xactCompletionCount + * is incremented whenever a transaction with an xid finishes (while + * holding ProcArrayLock) exclusively). Thus the xactCompletionCount check + * ensures we would detect if the snapshot would have changed. + * + * As the snapshot contents are the same as it was before, it is is safe + * to re-enter the snapshot's xmin into the PGPROC array. None of the rows + * visible under the snapshot could already have been removed (that'd + * require the set of running transactions to change) and it fulfills the + * requirement that concurrent GetSnapshotData() calls yield the same + * xmin. + */ + if (!TransactionIdIsValid(MyProc->xmin)) + MyProc->xmin = TransactionXmin = snapshot->xmin; + + RecentXmin = snapshot->xmin; + Assert(TransactionIdPrecedesOrEquals(TransactionXmin, RecentXmin)); + + snapshot->curcid = GetCurrentCommandId(false); + snapshot->active_count = 0; + snapshot->regd_count = 0; + snapshot->copied = false; + + GetSnapshotDataInitOldSnapshot(snapshot); + + return true; +} + /* * GetSnapshotData -- returns information about running transactions. * @@ -1945,6 +2040,7 @@ GetSnapshotData(Snapshot snapshot) TransactionId oldestxid; int mypgxactoff; TransactionId myxid; + uint64 curXactCompletionCount; TransactionId replication_slot_xmin = InvalidTransactionId; TransactionId replication_slot_catalog_xmin = InvalidTransactionId; @@ -1989,12 +2085,19 @@ GetSnapshotData(Snapshot snapshot) */ LWLockAcquire(ProcArrayLock, LW_SHARED); + if (GetSnapshotDataReuse(snapshot)) + { + LWLockRelease(ProcArrayLock); + return snapshot; + } + latest_completed = ShmemVariableCache->latestCompletedFullXid; mypgxactoff = MyProc->pgxactoff; myxid = other_xids[mypgxactoff]; Assert(myxid == MyProc->xid); oldestxid = ShmemVariableCache->oldestXid; + curXactCompletionCount = ShmemVariableCache->xactCompletionCount; /* xmax is always latestCompletedXid + 1 */ xmax = XidFromFullTransactionId(latest_completed); @@ -2248,6 +2351,7 @@ GetSnapshotData(Snapshot snapshot) snapshot->xcnt = count; snapshot->subxcnt = subcount; snapshot->suboverflowed = suboverflowed; + snapshot->snapXactCompletionCount = curXactCompletionCount; snapshot->curcid = GetCurrentCommandId(false); @@ -2259,26 +2363,7 @@ GetSnapshotData(Snapshot snapshot) snapshot->regd_count = 0; snapshot->copied = false; - if (old_snapshot_threshold < 0) - { - /* - * If not using "snapshot too old" feature, fill related fields with - * dummy values that don't require any locking. - */ - snapshot->lsn = InvalidXLogRecPtr; - snapshot->whenTaken = 0; - } - else - { - /* - * Capture the current time and WAL stream location in case this - * snapshot becomes old enough to need to fall back on the special - * "old snapshot" logic. - */ - snapshot->lsn = GetXLogInsertRecPtr(); - snapshot->whenTaken = GetSnapshotCurrentTimestamp(); - MaintainOldSnapshotTimeMapping(snapshot->whenTaken, xmin); - } + GetSnapshotDataInitOldSnapshot(snapshot); return snapshot; } diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c index 689a3b6a597..09ea03c2063 100644 --- a/src/backend/utils/time/snapmgr.c +++ b/src/backend/utils/time/snapmgr.c @@ -595,6 +595,8 @@ SetTransactionSnapshot(Snapshot sourcesnap, VirtualTransactionId *sourcevxid, CurrentSnapshot->takenDuringRecovery = sourcesnap->takenDuringRecovery; /* NB: curcid should NOT be copied, it's a local matter */ + CurrentSnapshot->snapXactCompletionCount = 0; + /* * Now we have to fix what GetSnapshotData did with MyProc->xmin and * TransactionXmin. There is a race condition: to make sure we are not @@ -670,6 +672,7 @@ CopySnapshot(Snapshot snapshot) newsnap->regd_count = 0; newsnap->active_count = 0; newsnap->copied = true; + newsnap->snapXactCompletionCount = 0; /* setup XID array */ if (snapshot->xcnt > 0) @@ -2207,6 +2210,7 @@ RestoreSnapshot(char *start_address) snapshot->curcid = serialized_snapshot.curcid; snapshot->whenTaken = serialized_snapshot.whenTaken; snapshot->lsn = serialized_snapshot.lsn; + snapshot->snapXactCompletionCount = 0; /* Copy XIDs, if present. */ if (serialized_snapshot.xcnt > 0) -- 2.25.0.114.g5b0ca878e0 --kdyesixubxkl3b7r--