pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feeduse a non-locking initial test in TAS_SPIN on AArch64
22+ messages / 6 participants
[nested] [flat]
* use a non-locking initial test in TAS_SPIN on AArch64
@ 2024-10-22 19:54 Nathan Bossart <nathandbossart@gmail.com>
2024-10-23 03:01 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Jingtang Zhang <mrdrivingduck@gmail.com>
2025-01-08 18:12 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 2 replies; 22+ messages in thread
From: Nathan Bossart @ 2024-10-22 19:54 UTC (permalink / raw)
To: pgsql-hackers; +Cc: dipiets@amazon.com
My colleague Salvatore Dipietro (CC'd) sent me a couple of profiles that
showed an enormous amount of s_lock() time going to the
__sync_lock_test_and_set() call in the AArch64 implementation of tas().
Upon closer inspection, I noticed that we don't implement a custom
TAS_SPIN() for this architecture, so I quickly hacked together the attached
patch and ran a couple of benchmarks that stressed the spinlock code. I
found no discussion about TAS_SPIN() on ARM in the archives, but I did
notice that the initial AArch64 support was added [0] before x86_64 started
using a non-locking test [1].
These benchmarks are for a c8g.24xlarge running a select-only pgbench with
256 clients and pg_stat_statements.track_planning enabled.
without the patch:
90.04% postgres [.] s_lock
1.07% pg_stat_statements.so [.] pgss_store
0.59% postgres [.] LWLockRelease
0.56% postgres [.] perform_spin_delay
0.31% [kernel] [k] arch_local_irq_enable
| while (TAS_SPIN(lock))
| {
| perform_spin_delay(&delayStatus);
0.12 |2c: -> bl perform_spin_delay
| tas():
| return __sync_lock_test_and_set(lock, 1);
0.01 |30: swpa w20, w1, [x19]
| s_lock():
99.87 | add x0, sp, #0x28
| while (TAS_SPIN(lock))
0.00 | ^ cbnz w1, 2c
tps = 74135.100891 (without initial connection time)
with the patch:
30.46% postgres [.] s_lock
5.88% postgres [.] perform_spin_delay
4.61% [kernel] [k] arch_local_irq_enable
3.31% [kernel] [k] next_uptodate_page
2.50% postgres [.] hash_search_with_hash_value
| while (TAS_SPIN(lock))
| {
| perform_spin_delay(&delayStatus);
0.63 |2c:+-->add x0, sp, #0x28
0.07 | |-> bl perform_spin_delay
| |while (TAS_SPIN(lock))
0.25 |34:| ldr w0, [x19]
65.19 | +--cbnz w0, 2c
| tas():
| return __sync_lock_test_and_set(lock, 1);
0.00 | swpa w20, w0, [x19]
| s_lock():
33.82 | ^ cbnz w0, 2c
tps = 549462.785554 (without initial connection time)
[0] https://postgr.es/c/5c7603c
[1] https://postgr.es/c/b03d196
--
nathan
From a69b7d38c40f0fb49f714c49bb45c292e7c8587d Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Tue, 22 Oct 2024 14:33:39 -0500
Subject: [PATCH v1 1/1] Use a non-locking initial test in TAS_SPIN on AArch64.
---
src/include/storage/s_lock.h | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/src/include/storage/s_lock.h b/src/include/storage/s_lock.h
index e94ed5f48b..07f343baf8 100644
--- a/src/include/storage/s_lock.h
+++ b/src/include/storage/s_lock.h
@@ -270,6 +270,12 @@ tas(volatile slock_t *lock)
*/
#if defined(__aarch64__)
+/*
+ * On ARM64, it's a win to use a non-locking test before the TAS proper. It
+ * may be a win on 32-bit ARM, too, but nobody's tested it yet.
+ */
+#define TAS_SPIN(lock) (*(lock) ? 1 : TAS(lock))
+
#define SPIN_DELAY() spin_delay()
static __inline__ void
--
2.39.5 (Apple Git-154)
Attachments:
[text/plain] v1-0001-Use-a-non-locking-initial-test-in-TAS_SPIN-on-AAr.patch (868B, ../../ZxgDEb_VpWyNZKB_@nathan/2-v1-0001-Use-a-non-locking-initial-test-in-TAS_SPIN-on-AAr.patch)
download | inline diff:
From a69b7d38c40f0fb49f714c49bb45c292e7c8587d Mon Sep 17 00:00:00 2001
From: Nathan Bossart <nathan@postgresql.org>
Date: Tue, 22 Oct 2024 14:33:39 -0500
Subject: [PATCH v1 1/1] Use a non-locking initial test in TAS_SPIN on AArch64.
---
src/include/storage/s_lock.h | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/src/include/storage/s_lock.h b/src/include/storage/s_lock.h
index e94ed5f48b..07f343baf8 100644
--- a/src/include/storage/s_lock.h
+++ b/src/include/storage/s_lock.h
@@ -270,6 +270,12 @@ tas(volatile slock_t *lock)
*/
#if defined(__aarch64__)
+/*
+ * On ARM64, it's a win to use a non-locking test before the TAS proper. It
+ * may be a win on 32-bit ARM, too, but nobody's tested it yet.
+ */
+#define TAS_SPIN(lock) (*(lock) ? 1 : TAS(lock))
+
#define SPIN_DELAY() spin_delay()
static __inline__ void
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: use a non-locking initial test in TAS_SPIN on AArch64
2024-10-22 19:54 use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
@ 2024-10-23 03:01 ` Jingtang Zhang <mrdrivingduck@gmail.com>
2024-10-23 14:46 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2025-01-15 11:50 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Jingtang Zhang <mrdrivingduck@gmail.com>
1 sibling, 2 replies; 22+ messages in thread
From: Jingtang Zhang @ 2024-10-23 03:01 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: pgsql-hackers@lists.postgresql.org
Hi~
Upon closer inspection, I noticed that we don't implement a custom
> TAS_SPIN() for this architecture, so I quickly hacked together the attached
> patch and ran a couple of benchmarks that stressed the spinlock code. I
> found no discussion about TAS_SPIN() on ARM in the archives, but I did
> notice that the initial AArch64 support was added [0] before x86_64 started
> using a non-locking test [1].
>
It reminds me of a discussion about improving spinlock performance on ARM
in 2020 [0], though the discussion is about CAS and TAS, not TAS_SPIN()
itself.
> tps = 74135.100891 (without initial connection time)
> tps = 549462.785554 (without initial connection time)
The result looks great, but the discussion in [0] shows that the result may
vary among different ARM chips. Could you provide the chip model of this
test? So that we can do a cross validation of this patch. Not sure if
compiler
version is necessary too. I'm willing to test it on Alibaba Cloud Yitian 710
if I have time.
[0]
https://www.postgresql.org/message-id/flat/CAPpHfdsGqVd6EJ4mr_RZVE5xSiCNBy4MuSvdTrKmTpM0eyWGpg%40mai...
- Regards
Jingtang
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: use a non-locking initial test in TAS_SPIN on AArch64
2024-10-22 19:54 use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2024-10-23 03:01 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Jingtang Zhang <mrdrivingduck@gmail.com>
@ 2024-10-23 14:46 ` Nathan Bossart <nathandbossart@gmail.com>
2024-10-23 15:18 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
1 sibling, 1 reply; 22+ messages in thread
From: Nathan Bossart @ 2024-10-23 14:46 UTC (permalink / raw)
To: Jingtang Zhang <mrdrivingduck@gmail.com>; +Cc: pgsql-hackers@lists.postgresql.org
On Wed, Oct 23, 2024 at 11:01:05AM +0800, Jingtang Zhang wrote:
> The result looks great, but the discussion in [0] shows that the result may
> vary among different ARM chips. Could you provide the chip model of this
> test? So that we can do a cross validation of this patch.
This is on a c8g.24xlarge, which is using Neoverse-V2 and Armv9.0-a [0].
> I'm willing to test it on Alibaba Cloud Yitian 710 if I have time.
That would be great. I have a couple of Apple M-series machines I can
test, too.
[0] https://github.com/aws/aws-graviton-getting-started/blob/main/README.md#building-for-graviton
--
nathan
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: use a non-locking initial test in TAS_SPIN on AArch64
2024-10-22 19:54 use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2024-10-23 03:01 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Jingtang Zhang <mrdrivingduck@gmail.com>
2024-10-23 14:46 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
@ 2024-10-23 15:18 ` Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 0 replies; 22+ messages in thread
From: Nathan Bossart @ 2024-10-23 15:18 UTC (permalink / raw)
To: Jingtang Zhang <mrdrivingduck@gmail.com>; +Cc: pgsql-hackers@lists.postgresql.org
On Wed, Oct 23, 2024 at 09:46:56AM -0500, Nathan Bossart wrote:
> I have a couple of Apple M-series machines I can test, too.
After some preliminary tests on an M3, I'm not seeing any gains outside the
noise range. That's not too surprising because it's likely more difficult
to create a lot of spinlock contention on these smaller machines. But, at
the very least, I'm not seeing a regression.
--
nathan
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: use a non-locking initial test in TAS_SPIN on AArch64
2024-10-22 19:54 use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2024-10-23 03:01 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Jingtang Zhang <mrdrivingduck@gmail.com>
@ 2025-01-15 11:50 ` Jingtang Zhang <mrdrivingduck@gmail.com>
2025-01-15 18:50 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
1 sibling, 1 reply; 22+ messages in thread
From: Jingtang Zhang @ 2025-01-15 11:50 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: pgsql-hackers@lists.postgresql.org
Hi, Nathan.
I just realized that I almost forgot about this thread :)
> The result looks great, but the discussion in [0] shows that the result may
> vary among different ARM chips. Could you provide the chip model of this
> test? So that we can do a cross validation of this patch. Not sure if compiler
> version is necessary too. I'm willing to test it on Alibaba Cloud Yitian 710
> if I have time.
I did some benchmark on Yitian 710.
On c8y.16xlarge (64 cores):
Without the patch:
80.31% postgres [.] __aarch64_swp4_acq
1.77% postgres [.] __aarch64_ldadd4_acq_rel
1.13% postgres [.] hash_search_with_hash_value
0.87% pg_stat_statements.so [.] __aarch64_swp4_acq
0.72% postgres [.] perform_spin_delay
0.44% postgres [.] _bt_compare
tps = 295272.628421 (including connections establishing)
tps = 295335.660323 (excluding connections establishing)
Patched:
9.94% postgres [.] s_lock
6.07% postgres [.] __aarch64_swp4_acq
5.73% postgres [.] hash_search_with_hash_value
2.81% postgres [.] perform_spin_delay
2.29% postgres [.] _bt_compare
2.15% postgres [.] PinBuffer
tps = 864519.764125 (including connections establishing)
tps = 864638.244443 (excluding connections establishing)
Seems that great performance could be gained if s_lock contention is severe.
This may be more likely to happen on bigger machines.
On c8y.2xlarge (8 cores), I failed to make s_lock contended severely, and
as a result this patch didn’t bring any difference outside the noise.
Regards,
Jingtang
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: use a non-locking initial test in TAS_SPIN on AArch64
2024-10-22 19:54 use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2024-10-23 03:01 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Jingtang Zhang <mrdrivingduck@gmail.com>
2025-01-15 11:50 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Jingtang Zhang <mrdrivingduck@gmail.com>
@ 2025-01-15 18:50 ` Nathan Bossart <nathandbossart@gmail.com>
2026-09-15 11:46 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Salvatore Dipietro <dipiets@amazon.it>
0 siblings, 1 reply; 22+ messages in thread
From: Nathan Bossart @ 2025-01-15 18:50 UTC (permalink / raw)
To: Jingtang Zhang <mrdrivingduck@gmail.com>; +Cc: pgsql-hackers@lists.postgresql.org
On Wed, Jan 15, 2025 at 07:50:38PM +0800, Jingtang Zhang wrote:
> Seems that great performance could be gained if s_lock contention is severe.
> This may be more likely to happen on bigger machines.
>
> On c8y.2xlarge (8 cores), I failed to make s_lock contended severely, and
> as a result this patch didn´t bring any difference outside the noise.
Thanks for sharing.
--
nathan
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: use a non-locking initial test in TAS_SPIN on AArch64
2024-10-22 19:54 use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2024-10-23 03:01 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Jingtang Zhang <mrdrivingduck@gmail.com>
2025-01-15 11:50 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Jingtang Zhang <mrdrivingduck@gmail.com>
2025-01-15 18:50 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
@ 2026-09-15 11:46 ` Salvatore Dipietro <dipiets@amazon.it>
2026-09-15 14:51 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 22+ messages in thread
From: Salvatore Dipietro @ 2026-09-15 11:46 UTC (permalink / raw)
To: nathandbossart@gmail.com; +Cc: pgsql-hackers@lists.postgresql.org; mrdrivingduck@gmail.com; andres@anarazel.de; tgl@sss.pgh.pa.us; dipietro.salvatore@gmail.com
I'd like to propose backporting this commit 3d0b4b1068 ("Use a
non-locking initial test in TAS_SPIN on AArch64.", Nathan Bossart,
2025-01-10), which first shipped in PostgreSQL 18, to the supported
back branches: REL_17_STABLE, REL_16_STABLE and REL_15_STABLE.
The trigger is a kernel change, not a PostgreSQL one: Linux made
PREEMPT_LAZY the default in v7.0. A backend can now be preempted while
holding a spinlock, causing a drop in performance.
Setup: AWS Graviton4 (m8g.24xlarge, 96 vCPU, 384 GB), Amazon Linux
2023, pgbench built-in simple-update, scale 8470, prepared protocol,
1024 clients, 1200 s per run, shared_buffers 95 GB. PostgreSQL 15.18,
16.14, 17.10, 18.4. Two kernels: AL2023 stock 6.1.182-227.379 (no
PREEMPT_LAZY) and linux-next 7.2.0-8d3ae5928 (PREEMPT_LAZY by default).
Kernel v6.5+ also carries a second, unrelated regression in the folio
allocation for which we have proposed a kernel fix separately [1]. Both
defects cap TPS at roughly the same level, so the paging one hides most
of the spinlock effect. Look at the "v7.2+fix" columns to see the
spinlock on its own.
TPS with huge_pages=off, and the TTAS-vs-stock gain per kernel:
| PG | 6.1 | v7.2 | v7.2+fix |
|-------------|--------:|-------:|---------:|
| 15.18 stock | 75,799 | 41,155 | 43,636 |
| 15.18 TTAS | 80,858 | 56,198 | 82,212 |
| 15.18 gain | +6.7% | +36.6% | +88% |
| 16.14 stock | 75,667 | 39,952 | 39,592 |
| 16.14 TTAS | 80,746 | 57,604 | 77,747 |
| 16.14 gain | +6.7% | +44.2% | +96% |
| 17.10 stock | 137,630 | 69,751 | 69,003 |
| 17.10 TTAS | 150,794 | 81,487 | 157,436 |
| 17.10 gain | +9.6% | +16.8% | +128% |
| 18.4 native | 148,467 | 79,820 | 143,473 |
So on a PREEMPT_LAZY kernel with the paging defect fixed, the one-line
change is worth +88% to +128% on PG 15-17. PG 17.10 with the backport
also tracks PG 18.4's native behaviour closely on both kernels.
With huge_pages=on both effects largely disappear: huge pages mask the
paging defect, and TTAS is neutral.
The change is a one-line, so a backport looks low-risk and would spare
pre-18 AArch64 users a severe regression as PREEMPT_LAZY kernels ship.
Thanks,
Salvatore
[1] https://lore.kernel.org/linux-mm/20260911142102.2294202-1-dipiets@amazon.it/T/#u
AMAZON DEVELOPMENT CENTER ITALY SRL, viale Monte Grappa 3/5, 20124 Milano, Italia, Registro delle Imprese di Milano Monza Brianza Lodi REA n. 2504859, Capitale Sociale: 10.000 EUR i.v., Cod. Fisc. e P.IVA 10100050961, Societa con Socio Unico
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: use a non-locking initial test in TAS_SPIN on AArch64
2024-10-22 19:54 use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2024-10-23 03:01 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Jingtang Zhang <mrdrivingduck@gmail.com>
2025-01-15 11:50 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Jingtang Zhang <mrdrivingduck@gmail.com>
2025-01-15 18:50 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2026-09-15 11:46 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Salvatore Dipietro <dipiets@amazon.it>
@ 2026-09-15 14:51 ` Nathan Bossart <nathandbossart@gmail.com>
2026-09-15 17:22 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Burd, Greg <greg@burd.me>
0 siblings, 1 reply; 22+ messages in thread
From: Nathan Bossart @ 2026-09-15 14:51 UTC (permalink / raw)
To: Salvatore Dipietro <dipiets@amazon.it>; +Cc: pgsql-hackers@lists.postgresql.org; mrdrivingduck@gmail.com; andres@anarazel.de; tgl@sss.pgh.pa.us; dipietro.salvatore@gmail.com
On Tue, Sep 15, 2026 at 11:46:23AM +0000, Salvatore Dipietro wrote:
> I'd like to propose backporting this commit 3d0b4b1068 ("Use a
> non-locking initial test in TAS_SPIN on AArch64.", Nathan Bossart,
> 2025-01-10), which first shipped in PostgreSQL 18, to the supported
> back branches: REL_17_STABLE, REL_16_STABLE and REL_15_STABLE.
I think that's fine. It was likely an oversight when AArch64 support was
first added to this file, anyway. We could also back-patch to v14, but
that would look a little different since it is missing commit a82a5eee31
("Use ISB as a spin-delay instruction on ARM64"). Since v14 is set to go
out-of-support in a couple months, I'd probably just stop at v15 and call
it a day, though.
> With huge_pages=on both effects largely disappear: huge pages mask the
> paging defect, and TTAS is neutral.
>
> The change is a one-line, so a backport looks low-risk and would spare
> pre-18 AArch64 users a severe regression as PREEMPT_LAZY kernels ship.
I'm curious why you are testing without huge pages enabled. My
understanding is that any serious workload on the kinds of machines you are
using for your benchmarks would use them.
--
nathan
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: use a non-locking initial test in TAS_SPIN on AArch64
2024-10-22 19:54 use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2024-10-23 03:01 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Jingtang Zhang <mrdrivingduck@gmail.com>
2025-01-15 11:50 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Jingtang Zhang <mrdrivingduck@gmail.com>
2025-01-15 18:50 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2026-09-15 11:46 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Salvatore Dipietro <dipiets@amazon.it>
2026-09-15 14:51 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
@ 2026-09-15 17:22 ` Burd, Greg <greg@burd.me>
2026-09-16 12:08 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Salvatore Dipietro <dipiets@amazon.it>
0 siblings, 1 reply; 22+ messages in thread
From: Burd, Greg @ 2026-09-15 17:22 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; Salvatore Dipietro <dipiets@amazon.it>; +Cc: pgsql-hackers@lists.postgresql.org; mrdrivingduck@gmail.com; andres@anarazel.de; tgl@sss.pgh.pa.us; dipietro.salvatore@gmail.com
> On Sep 15, 2026, at 10:51 AM, Nathan Bossart <nathandbossart@gmail.com> wrote:
>
> On Tue, Sep 15, 2026 at 11:46:23AM +0000, Salvatore Dipietro wrote:
>> I'd like to propose backporting this commit 3d0b4b1068 ("Use a
>> non-locking initial test in TAS_SPIN on AArch64.", Nathan Bossart,
>> 2025-01-10), which first shipped in PostgreSQL 18, to the supported
>> back branches: REL_17_STABLE, REL_16_STABLE and REL_15_STABLE.
>
> I think that's fine. It was likely an oversight when AArch64 support was
> first added to this file, anyway. We could also back-patch to v14, but
> that would look a little different since it is missing commit a82a5eee31
> ("Use ISB as a spin-delay instruction on ARM64"). Since v14 is set to go
> out-of-support in a couple months, I'd probably just stop at v15 and call
> it a day, though.
+1 for backpatching to v15
> With huge_pages=on both effects largely disappear: huge pages mask the
>> paging defect, and TTAS is neutral.
>>
>> The change is a one-line, so a backport looks low-risk and would spare
>> pre-18 AArch64 users a severe regression as PREEMPT_LAZY kernels ship.
>
> I'm curious why you are testing without huge pages enabled. My
> understanding is that any serious workload on the kinds of machines you are
> using for your benchmarks would use them.
I agree with Nathan, that should have a positive impact. 95GiB shared_buffers
on 4 KB pages is a TLB disaster. With ~4 KB pages, a buffer pool that size
needs ~25 million page-table entries; the TLB on Graviton4 cores covers only
a few thousand entries, so heavy concurrent access to shared memory causes
constant TLB misses and page-table walks. I bet you can measure for this
effect to validate that's happening.
2 MB (or larger) pages cut the translation footprint by ~500x, give it a
try.
best.
-greg
> --
> nathan
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: use a non-locking initial test in TAS_SPIN on AArch64
2024-10-22 19:54 use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2024-10-23 03:01 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Jingtang Zhang <mrdrivingduck@gmail.com>
2025-01-15 11:50 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Jingtang Zhang <mrdrivingduck@gmail.com>
2025-01-15 18:50 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2026-09-15 11:46 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Salvatore Dipietro <dipiets@amazon.it>
2026-09-15 14:51 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2026-09-15 17:22 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Burd, Greg <greg@burd.me>
@ 2026-09-16 12:08 ` Salvatore Dipietro <dipiets@amazon.it>
2026-10-06 19:37 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 22+ messages in thread
From: Salvatore Dipietro @ 2026-09-16 12:08 UTC (permalink / raw)
To: nathandbossart@gmail.com; greg@burd.me; +Cc: pgsql-hackers@lists.postgresql.org; mrdrivingduck@gmail.com; andres@anarazel.de; tgl@sss.pgh.pa.us; dipietro.salvatore@gmail.com
>> On Sep 15, 2026, at 10:51 AM, Nathan Bossart <nathandbossart(at)gmail(dot)com> wrote:
>> I'm curious why you are testing without huge pages enabled. My
>> understanding is that any serious workload on the kinds of machines you are
>> using for your benchmarks would use them.
>
> On Sep 15, 2026, at 13:51 AM, Burd, Greg <greg(at)burd(dot)me> wrote:
> I agree with Nathan, that should have a positive impact. 95GiB shared_buffers
> on 4 KB pages is a TLB disaster. With ~4 KB pages, a buffer pool that size
> needs ~25 million page-table entries; the TLB on Graviton4 cores covers only
> a few thousand entries, so heavy concurrent access to shared memory causes
> constant TLB misses and page-table walks. I bet you can measure for this
> effect to validate that's happening.
>
> 2 MB (or larger) pages cut the translation footprint by ~500x, give it a
> try.
We routinely test both huge_pages=on and huge_pages=off. The hp=off case
matters because huge_pages defaults to "try" and Linux does not
preallocate a hugepage pool, so unless the operator sets it manually the
server falls back to 4K pages. The regression is also more evident on
this instance because of the high core count and the contention that
comes with it; I would expect it on smaller instances too with a more
limited impact, though I have not measured that.
As expected, with hugepage=off the system is more under stress and TTAS
shows a bigger improvement, but we tested hugepage=on as well (2 MB
pages, preallocated pool of 54,612 pages for the 95 GB shared_buffers):
| PG | 6.1 | v7.2 | v7.2+fix |
|-------------|--------:|--------:|---------:|
| 17.10 stock | 144,162 | 176,377 | 156,885 |
| 17.10 TTAS | 148,997 | 185,961 | 185,524 |
| 17.10 gain | +3.4% | +5.4% | +18.3% |
Even with PG 17 and hugepages, TTAS still delivers some gains with
kernel v7.2, though the gain is much smaller than with hugepage=off.
For PG 15 and 16 with huge_pages=on, stock and TTAS come out close,
83-92k either way, so the limiter there is elsewhere.
Salvatore
AMAZON DEVELOPMENT CENTER ITALY SRL, viale Monte Grappa 3/5, 20124 Milano, Italia, Registro delle Imprese di Milano Monza Brianza Lodi REA n. 2504859, Capitale Sociale: 10.000 EUR i.v., Cod. Fisc. e P.IVA 10100050961, Societa con Socio Unico
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: use a non-locking initial test in TAS_SPIN on AArch64
2024-10-22 19:54 use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2024-10-23 03:01 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Jingtang Zhang <mrdrivingduck@gmail.com>
2025-01-15 11:50 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Jingtang Zhang <mrdrivingduck@gmail.com>
2025-01-15 18:50 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2026-09-15 11:46 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Salvatore Dipietro <dipiets@amazon.it>
2026-09-15 14:51 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2026-09-15 17:22 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Burd, Greg <greg@burd.me>
2026-09-16 12:08 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Salvatore Dipietro <dipiets@amazon.it>
@ 2026-10-06 19:37 ` Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 0 replies; 22+ messages in thread
From: Nathan Bossart @ 2026-10-06 19:37 UTC (permalink / raw)
To: Salvatore Dipietro <dipiets@amazon.it>; +Cc: greg@burd.me; pgsql-hackers@lists.postgresql.org; mrdrivingduck@gmail.com; andres@anarazel.de; tgl@sss.pgh.pa.us; dipietro.salvatore@gmail.com
I've back-patched commit 3d0b4b1068 down to v15.
--
nathan
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: use a non-locking initial test in TAS_SPIN on AArch64
2024-10-22 19:54 use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
@ 2025-01-08 18:12 ` Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 20:07 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Andres Freund <andres@anarazel.de>
2025-01-08 20:23 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 2 replies; 22+ messages in thread
From: Nathan Bossart @ 2025-01-08 18:12 UTC (permalink / raw)
To: pgsql-hackers; +Cc: dipiets@amazon.com
On Tue, Oct 22, 2024 at 02:54:57PM -0500, Nathan Bossart wrote:
> My colleague Salvatore Dipietro (CC'd) sent me a couple of profiles that
> showed an enormous amount of s_lock() time going to the
> __sync_lock_test_and_set() call in the AArch64 implementation of tas().
> Upon closer inspection, I noticed that we don't implement a custom
> TAS_SPIN() for this architecture, so I quickly hacked together the attached
> patch and ran a couple of benchmarks that stressed the spinlock code. I
> found no discussion about TAS_SPIN() on ARM in the archives, but I did
> notice that the initial AArch64 support was added [0] before x86_64 started
> using a non-locking test [1].
>
> These benchmarks are for a c8g.24xlarge running a select-only pgbench with
> 256 clients and pg_stat_statements.track_planning enabled.
>
> without the patch:
>
> [...]
>
> tps = 74135.100891 (without initial connection time)
>
> with the patch:
>
> [...]
>
> tps = 549462.785554 (without initial connection time)
Are there any objections to proceeding with this change? So far, it's been
tested on a c8g.24xlarge and an Apple M3 (which seems to be too small to
show any effect). If anyone has access to a larger ARM machine, additional
testing would be greatly appreciated. I think it would be unfortunate if
this slipped to v19.
--
nathan
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: use a non-locking initial test in TAS_SPIN on AArch64
2024-10-22 19:54 use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 18:12 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
@ 2025-01-08 20:07 ` Andres Freund <andres@anarazel.de>
2025-01-08 22:01 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
1 sibling, 1 reply; 22+ messages in thread
From: Andres Freund @ 2025-01-08 20:07 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: pgsql-hackers; dipiets@amazon.com
Hi,
On 2025-01-08 12:12:19 -0600, Nathan Bossart wrote:
> On Tue, Oct 22, 2024 at 02:54:57PM -0500, Nathan Bossart wrote:
> > My colleague Salvatore Dipietro (CC'd) sent me a couple of profiles that
> > showed an enormous amount of s_lock() time going to the
> > __sync_lock_test_and_set() call in the AArch64 implementation of tas().
> > Upon closer inspection, I noticed that we don't implement a custom
> > TAS_SPIN() for this architecture, so I quickly hacked together the attached
> > patch and ran a couple of benchmarks that stressed the spinlock code. I
> > found no discussion about TAS_SPIN() on ARM in the archives, but I did
> > notice that the initial AArch64 support was added [0] before x86_64 started
> > using a non-locking test [1].
> >
> > These benchmarks are for a c8g.24xlarge running a select-only pgbench with
> > 256 clients and pg_stat_statements.track_planning enabled.
> >
> > without the patch:
> >
> > [...]
> >
> > tps = 74135.100891 (without initial connection time)
> >
> > with the patch:
> >
> > [...]
> >
> > tps = 549462.785554 (without initial connection time)
>
> Are there any objections to proceeding with this change? So far, it's been
> tested on a c8g.24xlarge and an Apple M3 (which seems to be too small to
> show any effect). If anyone has access to a larger ARM machine, additional
> testing would be greatly appreciated. I think it would be unfortunate if
> this slipped to v19.
Seems reasonable. It's not surprising that spinning with an atomic operation
doesn't scale very well once a you have more than a handful cores.
Of course the whole way locking works in pg_stat_statements is a scalability
disaster, but that's a bigger project to fix.
Out of curiosity, have you measured whether this has a positive effect without
pg_stat_statements? I think it'll e.g. also affect lwlocks, as they also use
perform_spin_delay().
Greetings,
Andres
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: use a non-locking initial test in TAS_SPIN on AArch64
2024-10-22 19:54 use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 18:12 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 20:07 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Andres Freund <andres@anarazel.de>
@ 2025-01-08 22:01 ` Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 22:25 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Andres Freund <andres@anarazel.de>
2025-01-08 23:07 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 2 replies; 22+ messages in thread
From: Nathan Bossart @ 2025-01-08 22:01 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: pgsql-hackers; dipiets@amazon.com
On Wed, Jan 08, 2025 at 03:07:24PM -0500, Andres Freund wrote:
> Out of curiosity, have you measured whether this has a positive effect without
> pg_stat_statements? I think it'll e.g. also affect lwlocks, as they also use
> perform_spin_delay().
AFAICT TAS_SPIN() is only used for s_lock(), which doesn't appear to be
used by LWLocks. But I did retry my test from upthread without
pg_stat_statements and was surprised to find a reproducible 4-6%
regression. I'm not seeing any obvious differences in perf, but I do see
that the thread for adding TAS_SPIN() for PPC mentions a regression at
lower contention levels [0]. Perhaps the non-locked test is failing often
enough to hurt performance in this case... Whatever it is, it'll be mighty
frustrating to miss out on a >7x gain because of a 4% regression.
[0] https://postgr.es/me/26496.1325625436%40sss.pgh.pa.us
--
nathan
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: use a non-locking initial test in TAS_SPIN on AArch64
2024-10-22 19:54 use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 18:12 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 20:07 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Andres Freund <andres@anarazel.de>
2025-01-08 22:01 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
@ 2025-01-08 22:25 ` Andres Freund <andres@anarazel.de>
2025-01-08 22:38 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
1 sibling, 1 reply; 22+ messages in thread
From: Andres Freund @ 2025-01-08 22:25 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: pgsql-hackers; dipiets@amazon.com
Hi,
On 2025-01-08 16:01:19 -0600, Nathan Bossart wrote:
> On Wed, Jan 08, 2025 at 03:07:24PM -0500, Andres Freund wrote:
> > Out of curiosity, have you measured whether this has a positive effect without
> > pg_stat_statements? I think it'll e.g. also affect lwlocks, as they also use
> > perform_spin_delay().
>
> AFAICT TAS_SPIN() is only used for s_lock(), which doesn't appear to be
> used by LWLocks.
Brainfart on my part, sorry. I was thinking of SPIN_DELAY() for a moment...
> But I did retry my test from upthread without pg_stat_statements and was
> surprised to find a reproducible 4-6% regression.
Uh, huh. I assume this was readonly pgbench with 256 clients just as you had
tested upthread? I don't think there's any hot spinlock meaningfully involved
in that workload? A r/w workload is a different story, but upthread you
mentioned select-only.
Do you see any spinlock in profiles?
> I'm not seeing any obvious differences in perf, but I do see that the thread
> for adding TAS_SPIN() for PPC mentions a regression at lower contention
> levels [0]. Perhaps the non-locked test is failing often enough to hurt
> performance in this case... Whatever it is, it'll be mighty frustrating to
> miss out on a
> >7x gain because of a 4% regression.
I don't think the explanation can be that simple - even with TAS_SPIN defined,
we do try to acquire the lock once without using TAS_SPIN:
#if !defined(S_LOCK)
#define S_LOCK(lock) \
(TAS(lock) ? s_lock((lock), __FILE__, __LINE__, __func__) : 0)
#endif /* S_LOCK */
Only s_lock() then uses TAS_SPIN(lock).
I wonder if you're hitting an extreme case of binary-layout related effects?
I've never seen them at this magnitude though. I'd suggest using either lld
or mold as linker and comparing the numbers for a few
-Wl,--shuffle-sections=$seed seed values.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: use a non-locking initial test in TAS_SPIN on AArch64
2024-10-22 19:54 use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 18:12 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 20:07 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Andres Freund <andres@anarazel.de>
2025-01-08 22:01 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 22:25 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Andres Freund <andres@anarazel.de>
@ 2025-01-08 22:38 ` Nathan Bossart <nathandbossart@gmail.com>
2025-01-10 16:51 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 22+ messages in thread
From: Nathan Bossart @ 2025-01-08 22:38 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: pgsql-hackers; dipiets@amazon.com
On Wed, Jan 08, 2025 at 05:25:24PM -0500, Andres Freund wrote:
> On 2025-01-08 16:01:19 -0600, Nathan Bossart wrote:
>> But I did retry my test from upthread without pg_stat_statements and was
>> surprised to find a reproducible 4-6% regression.
>
> Uh, huh. I assume this was readonly pgbench with 256 clients just as you had
> tested upthread? I don't think there's any hot spinlock meaningfully involved
> in that workload? A r/w workload is a different story, but upthread you
> mentioned select-only.
>
> Do you see any spinlock in profiles?
Yes, this was using 256 clients. Looking closer, I don't see anything
spinlock related anywhere near the top of perf.
>> I'm not seeing any obvious differences in perf, but I do see that the thread
>> for adding TAS_SPIN() for PPC mentions a regression at lower contention
>> levels [0]. Perhaps the non-locked test is failing often enough to hurt
>> performance in this case... Whatever it is, it'll be mighty frustrating to
>> miss out on a
>> >7x gain because of a 4% regression.
>
> I don't think the explanation can be that simple - even with TAS_SPIN defined,
> we do try to acquire the lock once without using TAS_SPIN:
>
> #if !defined(S_LOCK)
> #define S_LOCK(lock) \
> (TAS(lock) ? s_lock((lock), __FILE__, __LINE__, __func__) : 0)
> #endif /* S_LOCK */
>
> Only s_lock() then uses TAS_SPIN(lock).
Ah, right. FWIW I tried setting a cap on the number of times we do a
non-locked test, and the results still showed the regression, which seems
to match your intuition here.
> I wonder if you're hitting an extreme case of binary-layout related effects?
> I've never seen them at this magnitude though. I'd suggest using either lld
> or mold as linker and comparing the numbers for a few
> -Wl,--shuffle-sections=$seed seed values.
Will do.
--
nathan
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: use a non-locking initial test in TAS_SPIN on AArch64
2024-10-22 19:54 use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 18:12 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 20:07 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Andres Freund <andres@anarazel.de>
2025-01-08 22:01 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 22:25 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Andres Freund <andres@anarazel.de>
2025-01-08 22:38 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
@ 2025-01-10 16:51 ` Nathan Bossart <nathandbossart@gmail.com>
2025-01-10 19:19 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 1 reply; 22+ messages in thread
From: Nathan Bossart @ 2025-01-10 16:51 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: pgsql-hackers; dipiets@amazon.com
On Wed, Jan 08, 2025 at 04:38:02PM -0600, Nathan Bossart wrote:
> On Wed, Jan 08, 2025 at 05:25:24PM -0500, Andres Freund wrote:
>> I wonder if you're hitting an extreme case of binary-layout related effects?
>> I've never seen them at this magnitude though. I'd suggest using either lld
>> or mold as linker and comparing the numbers for a few
>> -Wl,--shuffle-sections=$seed seed values.
>
> Will do.
Actually, I think I may have just had back luck and/or not warmed things up
enough. I just re-ran the test a few dozen times, carefully ensuring the
data was in the cache and periodically alternating between the binary with
the patch applied and the one without it. The results converged to within
1-2% of each other, with the patched version even winning about half the
time. The averages across all the runs showed a ~0.4% regression, which I
suspect is well within the noise range.
--
nathan
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: use a non-locking initial test in TAS_SPIN on AArch64
2024-10-22 19:54 use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 18:12 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 20:07 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Andres Freund <andres@anarazel.de>
2025-01-08 22:01 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 22:25 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Andres Freund <andres@anarazel.de>
2025-01-08 22:38 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2025-01-10 16:51 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
@ 2025-01-10 19:19 ` Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 0 replies; 22+ messages in thread
From: Nathan Bossart @ 2025-01-10 19:19 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: pgsql-hackers; dipiets@amazon.com
On Fri, Jan 10, 2025 at 10:51:40AM -0600, Nathan Bossart wrote:
> Actually, I think I may have just had back luck and/or not warmed things up
> enough. I just re-ran the test a few dozen times, carefully ensuring the
> data was in the cache and periodically alternating between the binary with
> the patch applied and the one without it. The results converged to within
> 1-2% of each other, with the patched version even winning about half the
> time. The averages across all the runs showed a ~0.4% regression, which I
> suspect is well within the noise range.
I went ahead and committed this patch. Please let me know if there are any
remaining concerns.
--
nathan
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: use a non-locking initial test in TAS_SPIN on AArch64
2024-10-22 19:54 use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 18:12 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 20:07 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Andres Freund <andres@anarazel.de>
2025-01-08 22:01 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
@ 2025-01-08 23:07 ` Tom Lane <tgl@sss.pgh.pa.us>
2025-01-09 18:15 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
1 sibling, 1 reply; 22+ messages in thread
From: Tom Lane @ 2025-01-08 23:07 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; dipiets@amazon.com
Nathan Bossart <nathandbossart@gmail.com> writes:
> AFAICT TAS_SPIN() is only used for s_lock(), which doesn't appear to be
> used by LWLocks. But I did retry my test from upthread without
> pg_stat_statements and was surprised to find a reproducible 4-6%
> regression.
On what hardware?
I just spent an hour beating on my M4 Pro (the 14-core variant)
and could not detect any outside-the-noise effect of this patch,
with or without pg_stat_statements loaded. There does seem to be
a small fraction-of-a-percent-ish benefit. But the run-to-run
variation with 60-second "pgbench -S" tests is a couple of percent,
so I can't say that that's real.
I do feel pretty sure that the patch doesn't hurt on this
class of hardware.
regards, tom lane
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: use a non-locking initial test in TAS_SPIN on AArch64
2024-10-22 19:54 use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 18:12 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 20:07 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Andres Freund <andres@anarazel.de>
2025-01-08 22:01 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 23:07 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Tom Lane <tgl@sss.pgh.pa.us>
@ 2025-01-09 18:15 ` Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 0 replies; 22+ messages in thread
From: Nathan Bossart @ 2025-01-09 18:15 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-hackers; dipiets@amazon.com
On Wed, Jan 08, 2025 at 06:07:44PM -0500, Tom Lane wrote:
> Nathan Bossart <nathandbossart@gmail.com> writes:
>> AFAICT TAS_SPIN() is only used for s_lock(), which doesn't appear to be
>> used by LWLocks. But I did retry my test from upthread without
>> pg_stat_statements and was surprised to find a reproducible 4-6%
>> regression.
>
> On what hardware?
This was on a c8g.24xlarge (Neoverse-V2, Armv9.0-a) [0].
> I just spent an hour beating on my M4 Pro (the 14-core variant)
> and could not detect any outside-the-noise effect of this patch,
> with or without pg_stat_statements loaded. There does seem to be
> a small fraction-of-a-percent-ish benefit. But the run-to-run
> variation with 60-second "pgbench -S" tests is a couple of percent,
> so I can't say that that's real.
>
> I do feel pretty sure that the patch doesn't hurt on this
> class of hardware.
Great. This matches what I saw on an M3.
[0] https://github.com/aws/aws-graviton-getting-started/blob/main/README.md#building-for-graviton
--
nathan
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: use a non-locking initial test in TAS_SPIN on AArch64
2024-10-22 19:54 use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 18:12 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
@ 2025-01-08 20:23 ` Tom Lane <tgl@sss.pgh.pa.us>
2025-01-08 20:30 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
1 sibling, 1 reply; 22+ messages in thread
From: Tom Lane @ 2025-01-08 20:23 UTC (permalink / raw)
To: Nathan Bossart <nathandbossart@gmail.com>; +Cc: pgsql-hackers; dipiets@amazon.com
Nathan Bossart <nathandbossart@gmail.com> writes:
> Are there any objections to proceeding with this change? So far, it's been
> tested on a c8g.24xlarge and an Apple M3 (which seems to be too small to
> show any effect). If anyone has access to a larger ARM machine, additional
> testing would be greatly appreciated. I think it would be unfortunate if
> this slipped to v19.
I just acquired an M4 Pro, which may also be too small to show any
effect, but perhaps running the test there would at least give us
more confidence that there's not a bad effect. Which test case(s)
would you recommend trying?
regards, tom lane
^ permalink raw reply [nested|flat] 22+ messages in thread
* Re: use a non-locking initial test in TAS_SPIN on AArch64
2024-10-22 19:54 use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 18:12 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 20:23 ` Re: use a non-locking initial test in TAS_SPIN on AArch64 Tom Lane <tgl@sss.pgh.pa.us>
@ 2025-01-08 20:30 ` Nathan Bossart <nathandbossart@gmail.com>
0 siblings, 0 replies; 22+ messages in thread
From: Nathan Bossart @ 2025-01-08 20:30 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-hackers; dipiets@amazon.com
On Wed, Jan 08, 2025 at 03:23:45PM -0500, Tom Lane wrote:
> I just acquired an M4 Pro, which may also be too small to show any
> effect, but perhaps running the test there would at least give us
> more confidence that there's not a bad effect. Which test case(s)
> would you recommend trying?
Thanks! A select-only pgbench with many clients (I used 256 upthread) and
pg_stat_statements.track_planning enabled seems to be a pretty easy way to
stress spinlocks. The same test without track_planning enabled might also
be interesting. I'm looking for a way to stress LWLocks, too, and I will
share here when I either find an existing test or write something of my
own.
--
nathan
^ permalink raw reply [nested|flat] 22+ messages in thread
end of thread, other threads:[~2026-10-06 19:37 UTC | newest]
Thread overview: 22+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2024-10-22 19:54 use a non-locking initial test in TAS_SPIN on AArch64 Nathan Bossart <nathandbossart@gmail.com>
2024-10-23 03:01 ` Jingtang Zhang <mrdrivingduck@gmail.com>
2024-10-23 14:46 ` Nathan Bossart <nathandbossart@gmail.com>
2024-10-23 15:18 ` Nathan Bossart <nathandbossart@gmail.com>
2025-01-15 11:50 ` Jingtang Zhang <mrdrivingduck@gmail.com>
2025-01-15 18:50 ` Nathan Bossart <nathandbossart@gmail.com>
2026-09-15 11:46 ` Salvatore Dipietro <dipiets@amazon.it>
2026-09-15 14:51 ` Nathan Bossart <nathandbossart@gmail.com>
2026-09-15 17:22 ` Burd, Greg <greg@burd.me>
2026-09-16 12:08 ` Salvatore Dipietro <dipiets@amazon.it>
2026-10-06 19:37 ` Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 18:12 ` Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 20:07 ` Andres Freund <andres@anarazel.de>
2025-01-08 22:01 ` Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 22:25 ` Andres Freund <andres@anarazel.de>
2025-01-08 22:38 ` Nathan Bossart <nathandbossart@gmail.com>
2025-01-10 16:51 ` Nathan Bossart <nathandbossart@gmail.com>
2025-01-10 19:19 ` Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 23:07 ` Tom Lane <tgl@sss.pgh.pa.us>
2025-01-09 18:15 ` Nathan Bossart <nathandbossart@gmail.com>
2025-01-08 20:23 ` Tom Lane <tgl@sss.pgh.pa.us>
2025-01-08 20:30 ` Nathan Bossart <nathandbossart@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox