agora inbox for [email protected]help / color / mirror / Atom feed
[PATCH 1/2] REPACK: do not require the user to have REPLICATION 255+ messages / 2 participants [nested] [flat]
* [PATCH 1/2] REPACK: do not require the user to have REPLICATION @ 2026-04-20 09:38 Álvaro Herrera <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Álvaro Herrera @ 2026-04-20 09:38 UTC (permalink / raw) Although REPACK (CONCURRENTLY) uses replication slots, there is no concern that the slot will leak data of other users, because the MAINTAIN privilege on the table is required anyway; requiring REPLICATION is user-unfriendly without providing any actual protection. A related aspect is that the REPLICATION attribute is not needed to prevent REPACK from stealing slots from logical replication, since commit e76d8c749c31 made REPACK use a separate pool of replication slots. Because there are now successful concurrent repack runs in the regression tests, we're forced to run test_plan_advice under wal_level=replica. Author: Antonin Houska <[email protected]> Reported-by: Justin Pryzby <[email protected]> Reviewed-by: Chao Li <[email protected]> Discussion: https://postgr.es/m/aeJHPNmL4vVy3oPw@pryzbyj2023 --- src/backend/commands/repack_worker.c | 1 - .../test_plan_advice/t/001_replan_regress.pl | 1 + src/test/regress/expected/cluster.out | 20 +++++++++++++++++-- src/test/regress/sql/cluster.sql | 11 ++++++++-- 4 files changed, 28 insertions(+), 5 deletions(-) diff --git a/src/backend/commands/repack_worker.c b/src/backend/commands/repack_worker.c index b17edd771e2..e4a4860805b 100644 --- a/src/backend/commands/repack_worker.c +++ b/src/backend/commands/repack_worker.c @@ -214,7 +214,6 @@ repack_setup_logical_decoding(Oid relid) /* * Make sure we can use logical decoding. */ - CheckSlotPermissions(); CheckLogicalDecodingRequirements(true); /* diff --git a/src/test/modules/test_plan_advice/t/001_replan_regress.pl b/src/test/modules/test_plan_advice/t/001_replan_regress.pl index 38ffa4d11ae..452b179a665 100644 --- a/src/test/modules/test_plan_advice/t/001_replan_regress.pl +++ b/src/test/modules/test_plan_advice/t/001_replan_regress.pl @@ -18,6 +18,7 @@ $node->init(); # Set up our desired configuration. $node->append_conf('postgresql.conf', <<EOM); shared_preload_libraries='test_plan_advice' +wal_level=replica pg_plan_advice.always_explain_supplied_advice=false pg_plan_advice.feedback_warnings=true EOM diff --git a/src/test/regress/expected/cluster.out b/src/test/regress/expected/cluster.out index 6127b215a86..e17bc91fae1 100644 --- a/src/test/regress/expected/cluster.out +++ b/src/test/regress/expected/cluster.out @@ -543,15 +543,17 @@ ERROR: REPACK (CONCURRENTLY) is not supported for partitioned tables HINT: Consider running the command on individual partitions. DROP TABLE clstrpart; -- Ownership of partitions is checked -CREATE TABLE ptnowner(i int unique) PARTITION BY LIST (i); +CREATE TABLE ptnowner(i int unique not null) PARTITION BY LIST (i); CREATE INDEX ptnowner_i_idx ON ptnowner(i); CREATE TABLE ptnowner1 PARTITION OF ptnowner FOR VALUES IN (1); -CREATE ROLE regress_ptnowner; +CREATE ROLE regress_ptnowner LOGIN; CREATE TABLE ptnowner2 PARTITION OF ptnowner FOR VALUES IN (2); ALTER TABLE ptnowner1 OWNER TO regress_ptnowner; SET SESSION AUTHORIZATION regress_ptnowner; CLUSTER ptnowner USING ptnowner_i_idx; ERROR: permission denied for table ptnowner +ALTER TABLE ptnowner1 REPLICA IDENTITY USING INDEX ptnowner1_i_key; +REPACK (CONCURRENTLY) ptnowner1; RESET SESSION AUTHORIZATION; ALTER TABLE ptnowner OWNER TO regress_ptnowner; CREATE TEMP TABLE ptnowner_oldnodes AS @@ -560,6 +562,11 @@ CREATE TEMP TABLE ptnowner_oldnodes AS SET SESSION AUTHORIZATION regress_ptnowner; CLUSTER ptnowner USING ptnowner_i_idx; WARNING: permission denied to execute CLUSTER on "ptnowner2", skipping it +-- still can't repack without a replica identity +ALTER TABLE ptnowner1 REPLICA IDENTITY DEFAULT; +REPACK (CONCURRENTLY) ptnowner1; +ERROR: cannot process relation "ptnowner1" +HINT: Relation "ptnowner1" has no identity index. RESET SESSION AUTHORIZATION; SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a JOIN ptnowner_oldnodes b USING (oid) ORDER BY a.relname COLLATE "C"; @@ -570,6 +577,15 @@ SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a ptnowner2 | t (3 rows) +SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a + JOIN ptnowner_oldnodes b USING (oid) ORDER BY a.relname COLLATE "C"; + relname | ?column? +-----------+---------- + ptnowner | t + ptnowner1 | f + ptnowner2 | t +(3 rows) + DROP TABLE ptnowner; DROP ROLE regress_ptnowner; -- Test CLUSTER with external tuplesorting diff --git a/src/test/regress/sql/cluster.sql b/src/test/regress/sql/cluster.sql index d14063a9683..1f471a8821a 100644 --- a/src/test/regress/sql/cluster.sql +++ b/src/test/regress/sql/cluster.sql @@ -254,14 +254,16 @@ REPACK (CONCURRENTLY) clstrpart; DROP TABLE clstrpart; -- Ownership of partitions is checked -CREATE TABLE ptnowner(i int unique) PARTITION BY LIST (i); +CREATE TABLE ptnowner(i int unique not null) PARTITION BY LIST (i); CREATE INDEX ptnowner_i_idx ON ptnowner(i); CREATE TABLE ptnowner1 PARTITION OF ptnowner FOR VALUES IN (1); -CREATE ROLE regress_ptnowner; +CREATE ROLE regress_ptnowner LOGIN; CREATE TABLE ptnowner2 PARTITION OF ptnowner FOR VALUES IN (2); ALTER TABLE ptnowner1 OWNER TO regress_ptnowner; SET SESSION AUTHORIZATION regress_ptnowner; CLUSTER ptnowner USING ptnowner_i_idx; +ALTER TABLE ptnowner1 REPLICA IDENTITY USING INDEX ptnowner1_i_key; +REPACK (CONCURRENTLY) ptnowner1; RESET SESSION AUTHORIZATION; ALTER TABLE ptnowner OWNER TO regress_ptnowner; CREATE TEMP TABLE ptnowner_oldnodes AS @@ -269,7 +271,12 @@ CREATE TEMP TABLE ptnowner_oldnodes AS JOIN pg_class AS c ON c.oid=tree.relid; SET SESSION AUTHORIZATION regress_ptnowner; CLUSTER ptnowner USING ptnowner_i_idx; +-- still can't repack without a replica identity +ALTER TABLE ptnowner1 REPLICA IDENTITY DEFAULT; +REPACK (CONCURRENTLY) ptnowner1; RESET SESSION AUTHORIZATION; +SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a + JOIN ptnowner_oldnodes b USING (oid) ORDER BY a.relname COLLATE "C"; SELECT a.relname, a.relfilenode=b.relfilenode FROM pg_class a JOIN ptnowner_oldnodes b USING (oid) ORDER BY a.relname COLLATE "C"; DROP TABLE ptnowner; -- 2.47.3 --m6zy3l65ushq557m Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename="0002-REPACK-do-not-require-LOGIN-privileges.patch" ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v2 9/9] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++---------------------- src/tools/pgindent/typedefs.list | 1 - 2 files changed, 17 insertions(+), 41 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 0608eee9eb2..c5943d7fc74 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3364,10 +3345,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4502,9 +4481,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 56c1f997f88..f4c989c8c30 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -846,7 +846,6 @@ FSMPageData FakeRelCacheEntry FakeRelCacheEntryData FastPathMeta -FastPathStrongRelationLockData FdwInfo FdwRoutine FetchDirection -- 2.50.1 (Apple Git-155) --Xf36GNcW+0xZNily-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v2 9/9] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++---------------------- src/tools/pgindent/typedefs.list | 1 - 2 files changed, 17 insertions(+), 41 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 0608eee9eb2..c5943d7fc74 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3364,10 +3345,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4502,9 +4481,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 56c1f997f88..f4c989c8c30 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -846,7 +846,6 @@ FSMPageData FakeRelCacheEntry FakeRelCacheEntryData FastPathMeta -FastPathStrongRelationLockData FdwInfo FdwRoutine FetchDirection -- 2.50.1 (Apple Git-155) --Xf36GNcW+0xZNily-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v2 9/9] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++---------------------- src/tools/pgindent/typedefs.list | 1 - 2 files changed, 17 insertions(+), 41 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 0608eee9eb2..c5943d7fc74 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3364,10 +3345,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4502,9 +4481,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 56c1f997f88..f4c989c8c30 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -846,7 +846,6 @@ FSMPageData FakeRelCacheEntry FakeRelCacheEntryData FastPathMeta -FastPathStrongRelationLockData FdwInfo FdwRoutine FetchDirection -- 2.50.1 (Apple Git-155) --Xf36GNcW+0xZNily-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v2 9/9] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++---------------------- src/tools/pgindent/typedefs.list | 1 - 2 files changed, 17 insertions(+), 41 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 0608eee9eb2..c5943d7fc74 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3364,10 +3345,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4502,9 +4481,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 56c1f997f88..f4c989c8c30 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -846,7 +846,6 @@ FSMPageData FakeRelCacheEntry FakeRelCacheEntryData FastPathMeta -FastPathStrongRelationLockData FdwInfo FdwRoutine FetchDirection -- 2.50.1 (Apple Git-155) --Xf36GNcW+0xZNily-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v2 9/9] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++---------------------- src/tools/pgindent/typedefs.list | 1 - 2 files changed, 17 insertions(+), 41 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 0608eee9eb2..c5943d7fc74 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3364,10 +3345,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4502,9 +4481,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 56c1f997f88..f4c989c8c30 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -846,7 +846,6 @@ FSMPageData FakeRelCacheEntry FakeRelCacheEntryData FastPathMeta -FastPathStrongRelationLockData FdwInfo FdwRoutine FetchDirection -- 2.50.1 (Apple Git-155) --Xf36GNcW+0xZNily-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v2 9/9] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++---------------------- src/tools/pgindent/typedefs.list | 1 - 2 files changed, 17 insertions(+), 41 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 0608eee9eb2..c5943d7fc74 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3364,10 +3345,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4502,9 +4481,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 56c1f997f88..f4c989c8c30 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -846,7 +846,6 @@ FSMPageData FakeRelCacheEntry FakeRelCacheEntryData FastPathMeta -FastPathStrongRelationLockData FdwInfo FdwRoutine FetchDirection -- 2.50.1 (Apple Git-155) --Xf36GNcW+0xZNily-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v2 9/9] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++---------------------- src/tools/pgindent/typedefs.list | 1 - 2 files changed, 17 insertions(+), 41 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 0608eee9eb2..c5943d7fc74 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3364,10 +3345,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4502,9 +4481,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 56c1f997f88..f4c989c8c30 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -846,7 +846,6 @@ FSMPageData FakeRelCacheEntry FakeRelCacheEntryData FastPathMeta -FastPathStrongRelationLockData FdwInfo FdwRoutine FetchDirection -- 2.50.1 (Apple Git-155) --Xf36GNcW+0xZNily-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v2 9/9] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++---------------------- src/tools/pgindent/typedefs.list | 1 - 2 files changed, 17 insertions(+), 41 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 0608eee9eb2..c5943d7fc74 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3364,10 +3345,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4502,9 +4481,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 56c1f997f88..f4c989c8c30 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -846,7 +846,6 @@ FSMPageData FakeRelCacheEntry FakeRelCacheEntryData FastPathMeta -FastPathStrongRelationLockData FdwInfo FdwRoutine FetchDirection -- 2.50.1 (Apple Git-155) --Xf36GNcW+0xZNily-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v2 9/9] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++---------------------- src/tools/pgindent/typedefs.list | 1 - 2 files changed, 17 insertions(+), 41 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 0608eee9eb2..c5943d7fc74 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3364,10 +3345,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4502,9 +4481,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 56c1f997f88..f4c989c8c30 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -846,7 +846,6 @@ FSMPageData FakeRelCacheEntry FakeRelCacheEntryData FastPathMeta -FastPathStrongRelationLockData FdwInfo FdwRoutine FetchDirection -- 2.50.1 (Apple Git-155) --Xf36GNcW+0xZNily-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v2 9/9] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++---------------------- src/tools/pgindent/typedefs.list | 1 - 2 files changed, 17 insertions(+), 41 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 0608eee9eb2..c5943d7fc74 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3364,10 +3345,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4502,9 +4481,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 56c1f997f88..f4c989c8c30 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -846,7 +846,6 @@ FSMPageData FakeRelCacheEntry FakeRelCacheEntryData FastPathMeta -FastPathStrongRelationLockData FdwInfo FdwRoutine FetchDirection -- 2.50.1 (Apple Git-155) --Xf36GNcW+0xZNily-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v2 9/9] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++---------------------- src/tools/pgindent/typedefs.list | 1 - 2 files changed, 17 insertions(+), 41 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 0608eee9eb2..c5943d7fc74 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3364,10 +3345,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4502,9 +4481,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 56c1f997f88..f4c989c8c30 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -846,7 +846,6 @@ FSMPageData FakeRelCacheEntry FakeRelCacheEntryData FastPathMeta -FastPathStrongRelationLockData FdwInfo FdwRoutine FetchDirection -- 2.50.1 (Apple Git-155) --Xf36GNcW+0xZNily-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v2 9/9] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++---------------------- src/tools/pgindent/typedefs.list | 1 - 2 files changed, 17 insertions(+), 41 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 0608eee9eb2..c5943d7fc74 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3364,10 +3345,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4502,9 +4481,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 56c1f997f88..f4c989c8c30 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -846,7 +846,6 @@ FSMPageData FakeRelCacheEntry FakeRelCacheEntryData FastPathMeta -FastPathStrongRelationLockData FdwInfo FdwRoutine FetchDirection -- 2.50.1 (Apple Git-155) --Xf36GNcW+0xZNily-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
* [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics @ 2026-07-09 20:38 Nathan Bossart <[email protected]> 0 siblings, 0 replies; 255+ messages in thread From: Nathan Bossart @ 2026-07-09 20:38 UTC (permalink / raw) --- src/backend/storage/lmgr/lock.c | 57 ++++++++++----------------------- 1 file changed, 17 insertions(+), 40 deletions(-) diff --git a/src/backend/storage/lmgr/lock.c b/src/backend/storage/lmgr/lock.c index 5ee80c7632e..035932197a4 100644 --- a/src/backend/storage/lmgr/lock.c +++ b/src/backend/storage/lmgr/lock.c @@ -40,11 +40,11 @@ #include "miscadmin.h" #include "pg_trace.h" #include "pgstat.h" +#include "port/atomics.h" #include "storage/lmgr.h" #include "storage/proc.h" #include "storage/procarray.h" #include "storage/shmem.h" -#include "storage/spin.h" #include "storage/standby.h" #include "storage/subsystems.h" #include "utils/memutils.h" @@ -306,13 +306,7 @@ static PROCLOCK *FastPathGetRelationLockEntry(LOCALLOCK *locallock); #define FastPathStrongLockHashPartition(hashcode) \ ((hashcode) % FAST_PATH_STRONG_LOCK_HASH_PARTITIONS) -typedef struct -{ - slock_t mutex; - uint32 count[FAST_PATH_STRONG_LOCK_HASH_PARTITIONS]; -} FastPathStrongRelationLockData; - -static FastPathStrongRelationLockData *FastPathStrongRelationLocks; +static pg_atomic_uint32 *FastPathStrongRelationLocks; static void LockManagerShmemRequest(void *arg); static void LockManagerShmemInit(void *arg); @@ -484,7 +478,8 @@ LockManagerShmemRequest(void *arg) ); ShmemRequestStruct(.name = "Fast Path Strong Relation Lock Data", - .size = sizeof(FastPathStrongRelationLockData), + .size = mul_size(sizeof(pg_atomic_uint32), + FAST_PATH_STRONG_LOCK_HASH_PARTITIONS), .ptr = (void **) (void *) &FastPathStrongRelationLocks, ); } @@ -492,7 +487,8 @@ LockManagerShmemRequest(void *arg) static void LockManagerShmemInit(void *arg) { - SpinLockInit(&FastPathStrongRelationLocks->mutex); + for (int i = 0; i < FAST_PATH_STRONG_LOCK_HASH_PARTITIONS; i++) + pg_atomic_init_u32(&FastPathStrongRelationLocks[i], 0); } /* @@ -992,11 +988,11 @@ LockAcquireExtended(const LOCKTAG *locktag, /* * LWLockAcquire acts as a memory sequencing point, so it's safe * to assume that any strong locker whose increment to - * FastPathStrongRelationLocks->counts becomes visible after we - * test it has yet to begin to transfer fast-path locks. + * FastPathStrongRelationLocks becomes visible after we test it + * has yet to begin to transfer fast-path locks. */ LWLockAcquire(&MyProc->fpInfoLock, LW_EXCLUSIVE); - if (FastPathStrongRelationLocks->count[fasthashcode] != 0) + if (pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) != 0) acquired = false; else acquired = FastPathGrantRelationLock(locktag->locktag_field2, @@ -1501,11 +1497,9 @@ RemoveLocalLock(LOCALLOCK *locallock) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } if (!hash_search(LockMethodLocalHash, @@ -1834,20 +1828,9 @@ BeginStrongLockAcquire(LOCALLOCK *locallock, uint32 fasthashcode) Assert(StrongLockInProgress == NULL); Assert(locallock->holdsStrongLockCount == false); - /* - * Adding to a memory location is not atomic, so we take a spinlock to - * ensure we don't collide with someone else trying to bump the count at - * the same time. - * - * XXX: It might be worth considering using an atomic fetch-and-add - * instruction here, on architectures where that is supported. - */ - - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = true; StrongLockInProgress = locallock; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -1875,12 +1858,10 @@ AbortStrongLockAcquire(void) fasthashcode = FastPathStrongLockHashPartition(locallock->hashcode); Assert(locallock->holdsStrongLockCount == true); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); locallock->holdsStrongLockCount = false; StrongLockInProgress = NULL; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); } /* @@ -3365,10 +3346,8 @@ LockRefindAndRelease(LockMethod lockMethodTable, PGPROC *proc, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - Assert(FastPathStrongRelationLocks->count[fasthashcode] > 0); - FastPathStrongRelationLocks->count[fasthashcode]--; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + Assert(pg_atomic_read_u32(&FastPathStrongRelationLocks[fasthashcode]) > 0); + pg_atomic_fetch_sub_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } } @@ -4504,9 +4483,7 @@ lock_twophase_recover(FullTransactionId fxid, uint16 info, { uint32 fasthashcode = FastPathStrongLockHashPartition(hashcode); - SpinLockAcquire(&FastPathStrongRelationLocks->mutex); - FastPathStrongRelationLocks->count[fasthashcode]++; - SpinLockRelease(&FastPathStrongRelationLocks->mutex); + pg_atomic_fetch_add_u32(&FastPathStrongRelationLocks[fasthashcode], 1); } LWLockRelease(partitionLock); -- 2.50.1 (Apple Git-155) --Ot5x3ffkWEuxilWO-- ^ permalink raw reply [nested|flat] 255+ messages in thread
end of thread, other threads:[~2026-07-09 20:38 UTC | newest] Thread overview: 255+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2026-04-20 09:38 [PATCH 1/2] REPACK: do not require the user to have REPLICATION Álvaro Herrera <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v2 9/9] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v2 9/9] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v2 9/9] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v2 9/9] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v2 9/9] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v2 9/9] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v2 9/9] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v2 9/9] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v2 9/9] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v2 9/9] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v2 9/9] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v2 9/9] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]> 2026-07-09 20:38 [PATCH v1 8/8] convert FastPathStrongRelationLocks to atomics Nathan Bossart <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox