public inbox for [email protected]
help / color / mirror / Atom feedAvoid stack frame setup in performance critical routines using tail calls
7+ messages / 2 participants
[nested] [flat]
* Avoid stack frame setup in performance critical routines using tail calls
@ 2021-07-19 19:59 Andres Freund <[email protected]>
0 siblings, 1 reply; 7+ messages in thread
From: Andres Freund @ 2021-07-19 19:59 UTC (permalink / raw)
To: pgsql-hackers; Robert Haas <[email protected]>; Michael Paquier <[email protected]>; +Cc: Tomas Vondra <[email protected]>
Hi,
We have a few routines that are taking up a meaningful share of nearly all
workloads. They are worth micro-optimizing, even though they rarely the most
expensive parts of a profile. The memory allocation infrastructure is an
example of that.
When looking at a profile one can often see that a measurable percentage of
the time is spent doing stack frame setup in functions like palloc(),
AllocSetAlloc(). E.g. here's a perf profile annotating palloc(), the first
column showing the percentage of the time the relevant instruction was
sampled:
│ void *
│ palloc(Size size)
│ {
11.62 │ push %rbp
5.89 │ mov %rsp,%rbp
11.79 │ push %r12
│ mov %rdi,%r12
6.07 │ push %rbx
│ /* duplicates MemoryContextAlloc to avoid increased overhead */
│ void *ret;
│ MemoryContext context = CurrentMemoryContext;
│ mov CurrentMemoryContext,%rbx
│
│ AssertArg(MemoryContextIsValid(context));
│ AssertNotInCriticalSection(context);
│
│ if (!AllocSizeIsValid(size))
5.86 │ cmp $0x3fffffff,%rdi
│ → ja 14fa5b <palloc.cold>
│ elog(ERROR, "invalid memory alloc request size %zu", size);
│
│ context->isReset = false;
17.71 │ movb $0x0,0x4(%rbx)
│
│ ret = context->methods->alloc(context, size);
5.98 │ mov 0x10(%rbx),%rax
│ mov %rdi,%rsi
│ mov %rbx,%rdi
35.08 │ → callq *(%rax)
The stack frame setup bit is the push ... bit.
At least on x86-64 unixoid systems, that overhead can be avoided in certain
circumstances! The simplest case is if the function doesn't do any function
calls of its own. If simple enough (i.e. no register spilling), the compiler
will just not set up a stack frame - nobody could need it.
The slightly more complicated case is that of a function that only does a
"tail call", i.e. the only function call is just before returning (there can
be multiple such paths though). If the function is simple enough, gcc/clang
will then not use the "call" instruction to call the function (which would
require a proper stack frame being set up), but instead just jump to the other
function. Which ends up reusing the current function's stack frame,
basically. When that called function returns using 'ret', it'll reuse the
location pushed onto the call stack by the caller of the "original" function,
and return to its caller. Having optimized away the need to maintain one stack
frame level, and one indirection when returning from the inner function (which
just would do its own ret).
For that to work, there obviously cannot be any instructions in the function
before calling the inner function. Which brings us back to the palloc example
from above.
As an experiment, if i change the code for palloc() to omit the if (ret == NULL)
check, the assembly (omitting source for brevity) from:
61c9a0: 55 push %rbp
61c9a1: 48 89 e5 mov %rsp,%rbp
61c9a4: 41 54 push %r12
61c9a6: 49 89 fc mov %rdi,%r12
61c9a9: 53 push %rbx
61c9aa: 48 8b 1d 2f f2 2a 00 mov 0x2af22f(%rip),%rbx # 8cbbe0 <CurrentMemoryContext>
61c9b1: 48 81 ff ff ff ff 3f cmp $0x3fffffff,%rdi
61c9b8: 0f 87 9d 30 b3 ff ja 14fa5b <palloc.cold>
61c9be: c6 43 04 00 movb $0x0,0x4(%rbx)
61c9c2: 48 8b 43 10 mov 0x10(%rbx),%rax
61c9c6: 48 89 fe mov %rdi,%rsi
61c9c9: 48 89 df mov %rbx,%rdi
61c9cc: ff 10 callq *(%rax)
61c9ce: 48 85 c0 test %rax,%rax
61c9d1: 0f 84 b9 30 b3 ff je 14fa90 <palloc.cold+0x35>
61c9d7: 5b pop %rbx
61c9d8: 41 5c pop %r12
61c9da: 5d pop %rbp
61c9db: c3 retq
to
61c8c0: 48 89 fe mov %rdi,%rsi
61c8c3: 48 8b 3d 16 f3 2a 00 mov 0x2af316(%rip),%rdi # 8cbbe0 <CurrentMemoryContext>
61c8ca: 48 81 fe ff ff ff 3f cmp $0x3fffffff,%rsi
61c8d1: 0f 87 c3 31 b3 ff ja 14fa9a <palloc.cold>
61c8d7: c6 47 04 00 movb $0x0,0x4(%rdi)
61c8db: 48 8b 47 10 mov 0x10(%rdi),%rax
61c8df: ff 20 jmpq *(%rax)
It's not hard to see why that would be faster, I think.
Of course, we cannot just omit that check. But I think this is an argument for
why it is not a great idea to have such a check in palloc() - it prevents the
use of the above optimization, and it adds a branch to a performance critical
function, though there already existing branches in aset.c etc that
specifically know about this case.
The code in palloc() does this check after context->methods->alloc() since
3d6d1b585524: Move out-of-memory error checks from aset.c to mcxt.c
Of course, that commit changed things for a reason: It allows
palloc_extended() to exist.
However, it seems that the above optimization, as well as the desire to avoid
redundant branches (checking for allocation failures in AllocSetAlloc() and
then again in palloc() etc) in critical paths, suggests pushing the handling
of MCXT_ALLOC_NO_OOM (and perhaps others) a layer down, into the memory
context implementations. Which of course means that we would need to pass down
MCXT_ALLOC_NO_OOM into at least MemoryContextMethods->alloc,realloc}. But that
seems like a good idea to me anyway. That way we could pass down further
information as well, e.g. about required alignment.
Of course it'd make sense to avoid duplicating the same error message across
all contexts, but that could be addressed using a mcxt.c helper function to
deal with the allocation failure case.
E.g the existing cases like
block = (AllocBlock) malloc(blksize);
if (block == NULL)
return NULL;
could become something like
block = (AllocBlock) malloc(blksize);
if (unlikely(block == NULL))
return MemoryContextAllocationFailure(context, size, flags);
The trick of avoiding stack frame setup does not just apply to wrapper
functions like palloc(). It even can apply to AllocSetAlloc() itself! If one
separates out the "slow paths" from the "fast paths" of AllocSetAlloc(), the
fast path can avoid needing the stack frame, for the price of the slow paths
being a tiny bit slower. Often the generated code turns out to be better,
because the register allocation pressure is lower in the fast path.
For that to work, the paths of AllocSetAlloc() that call malloc() need to be
separated out. As we obviously need to process malloc()'s result, the call to
malloc cannot be a tail call. So we need to split out two paths:
1) handling of large allocations
2) running out of space in the current block / having no block
To actually benefit from the optimization, those paths need to actually return
the allocated memory. And they need to be marked pg_noinline, otherwise the
compiler won't get the message...
I think this actually makes the aset.c code a good bit more readable, and
highlights where in AllocSetAlloc() adding instructions hurts, and where its
fine.
I have *not* carefully benchmarked this, but a quick implementation of this
does seem to increase readonly pgbench tps at a small scale by 2-3% (both
-Mprepared/simple). Despite not being an all that pgbench bound workload.
Rough prototype patch for the above attached.
Comments?
A slightly tangential improvement would be to move the memset() in palloc0()
et al do into a static inline. There's two benefits of that:
1) compilers can generate much better code for memset() if the length is known
- instead of a function call with length dispatch replace that with a
handful of instructions doing the zeroing for the precise length.
2) compilers can often optimize away [part of ]the overhead of needing to do
the memset, as many callers will go on to overwrite a good portion of the
zeroed data.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Avoid stack frame setup in performance critical routines using tail calls
@ 2021-07-20 04:50 David Rowley <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 7+ messages in thread
From: David Rowley @ 2021-07-20 04:50 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: pgsql-hackers; Robert Haas <[email protected]>; Michael Paquier <[email protected]>; Tomas Vondra <[email protected]>
On Tue, 20 Jul 2021 at 08:00, Andres Freund <[email protected]> wrote:
> I have *not* carefully benchmarked this, but a quick implementation of this
> does seem to increase readonly pgbench tps at a small scale by 2-3% (both
Interesting.
I've not taken the time to study the patch but I was running some
other benchmarks today on a small scale pgbench readonly test and I
took this patch for a spin to see if I could see the same performance
gains.
This is an AMD 3990x machine that seems to get the most throughput
from pgbench with 132 processes
I did: pgbench -T 240 -P 10 -c 132 -j 132 -S -M prepared
--random-seed=12345 postgres
master = dd498998a
Master: 3816959.53 tps
Patched: 3820723.252 tps
I didn't quite get the same 2-3% as you did, but it did come out
faster than on master.
David
master dense hash LockReleaseAll + aset tail call aset tail call dense hash LockReleaseAll
10 3758201.2 3741925.6 3737701.5 3713521.5
20 3810125.5 3861572.5 3830863.7 3844142.9
30 3806505.1 3851164.4 3832257.9 3848458
40 3816094.8 3855232.4 3832305.7 3855706.6
50 3820317.2 3846941 3829641.5 3851717.7
60 3827809 3849490.4 3812254.5 3851499.4
70 3828757.9 3844582.8 3829097.8 3849312
80 3824492.1 3843161.8 3821383 3852378.8
90 3816502.1 3851970.8 3825119.2 3854793.8
100 3819124.1 3839695.5 3839286.7 3860418.6
110 3816154.3 3851302.8 3821209.5 3845327.7
120 3817070.5 3852974.2 3833781 3845842.5
130 3815424.7 3854379.1 3830812.5 3847626
140 3823631.1 3852449.1 3825261.9 3846760.6
150 3820963.8 3837493.5 3820703.2 3840196.6
160 3827737 3837809.7 3835278.4 3841149.3
170 3827779.2 3851799.4 3818430.3 3840130.9
180 3829352 3853094 3823286.1 3842814.5
190 3825518.3 3854912.8 3816329.4 3841991
200 3823477.2 3838998.6 3816060.8 3839390.7
210 3809304.3 3845776.7 3814737.2 3836433.5
220 3814328.5 3841394.7 3818894 3842073.7
230 3811399.3 3839360.8 3811939 3843780.7
avg 3816959.53 3843368.809 3820723.252 3840672.478
100.69% 100.10% 100.62%
Attachments:
[text/plain] benchresults.txt (1.1K, ../../CAApHDvpQQc67NbYJkM0GZWVw_wZvUvOuuANCPiOyobV=q1=owA@mail.gmail.com/2-benchresults.txt)
download | inline:
master dense hash LockReleaseAll + aset tail call aset tail call dense hash LockReleaseAll
10 3758201.2 3741925.6 3737701.5 3713521.5
20 3810125.5 3861572.5 3830863.7 3844142.9
30 3806505.1 3851164.4 3832257.9 3848458
40 3816094.8 3855232.4 3832305.7 3855706.6
50 3820317.2 3846941 3829641.5 3851717.7
60 3827809 3849490.4 3812254.5 3851499.4
70 3828757.9 3844582.8 3829097.8 3849312
80 3824492.1 3843161.8 3821383 3852378.8
90 3816502.1 3851970.8 3825119.2 3854793.8
100 3819124.1 3839695.5 3839286.7 3860418.6
110 3816154.3 3851302.8 3821209.5 3845327.7
120 3817070.5 3852974.2 3833781 3845842.5
130 3815424.7 3854379.1 3830812.5 3847626
140 3823631.1 3852449.1 3825261.9 3846760.6
150 3820963.8 3837493.5 3820703.2 3840196.6
160 3827737 3837809.7 3835278.4 3841149.3
170 3827779.2 3851799.4 3818430.3 3840130.9
180 3829352 3853094 3823286.1 3842814.5
190 3825518.3 3854912.8 3816329.4 3841991
200 3823477.2 3838998.6 3816060.8 3839390.7
210 3809304.3 3845776.7 3814737.2 3836433.5
220 3814328.5 3841394.7 3818894 3842073.7
230 3811399.3 3839360.8 3811939 3843780.7
avg 3816959.53 3843368.809 3820723.252 3840672.478
100.69% 100.10% 100.62%
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Avoid stack frame setup in performance critical routines using tail calls
@ 2021-07-20 06:16 Andres Freund <[email protected]>
parent: David Rowley <[email protected]>
0 siblings, 1 reply; 7+ messages in thread
From: Andres Freund @ 2021-07-20 06:16 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: pgsql-hackers; Robert Haas <[email protected]>; Michael Paquier <[email protected]>; Tomas Vondra <[email protected]>
Hi,
On 2021-07-20 16:50:09 +1200, David Rowley wrote:
> I've not taken the time to study the patch but I was running some
> other benchmarks today on a small scale pgbench readonly test and I
> took this patch for a spin to see if I could see the same performance
> gains.
Thanks!
> This is an AMD 3990x machine that seems to get the most throughput
> from pgbench with 132 processes
>
> I did: pgbench -T 240 -P 10 -c 132 -j 132 -S -M prepared
> --random-seed=12345 postgres
>
> master = dd498998a
>
> Master: 3816959.53 tps
> Patched: 3820723.252 tps
>
> I didn't quite get the same 2-3% as you did, but it did come out
> faster than on master.
It would not at all be suprising to me if AMD in recent microarchitectures did
a better job at removing stack management overview (e.g. by better register
renaming, or by resolving dependencies on %rsp in a smarter way) than Intel
has. This was on a Cascade Lake CPU (xeon 5215), which, despite being released
in 2019, effectively is a moderately polished (or maybe shoehorned)
microarchitecture from 2015 due to all the Intel troubles. Whereas Zen2 is
from 2019.
It's also possible that my attempts at avoiding the stack management just
didn't work on your compiler. Either due to vendor (I know that gcc is better
at it than clang), version, or compiler flags (e.g. -fno-omit-frame-pointer
could make it harder, -fno-optimize-sibling-calls would disable it).
A third plausible explanation for the difference is that at a client count of
132, the bottlenecks are sufficiently elsewhere to just not show a meaningful
gain from memory management efficiency improvements.
Any chance you could show a `perf annotate AllocSetAlloc` and `perf annotate
palloc` from a patched run? And perhaps how high their percentages of the
total work are. E.g. using something like
perf report -g none|grep -E 'AllocSetAlloc|palloc|MemoryContextAlloc|pfree'
It'd be interesting to know where the bottlenecks on a zen2 machine are.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Avoid stack frame setup in performance critical routines using tail calls
@ 2021-07-20 06:53 David Rowley <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 7+ messages in thread
From: David Rowley @ 2021-07-20 06:53 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: pgsql-hackers; Robert Haas <[email protected]>; Michael Paquier <[email protected]>; Tomas Vondra <[email protected]>
On Tue, 20 Jul 2021 at 18:17, Andres Freund <[email protected]> wrote:
> Any chance you could show a `perf annotate AllocSetAlloc` and `perf annotate
> palloc` from a patched run? And perhaps how high their percentages of the
> total work are. E.g. using something like
> perf report -g none|grep -E 'AllocSetAlloc|palloc|MemoryContextAlloc|pfree'
Sure. See attached.
David
Percent | Source code & Disassembly of postgres for cycles (626 samples, percent: local period)
-----------------------------------------------------------------------------------------------------
:
:
:
: Disassembly of section .text:
:
: 000000000056bd80 <AllocSetAlloc>:
: AllocSetAlloc():
: * is marked, as mcxt.c will set it to UNDEFINED. In some paths we will
: * return space that is marked NOACCESS - AllocSetRealloc has to beware!
: */
: static void *
: AllocSetAlloc(MemoryContext context, Size size, int flags)
: {
7.66 : 56bd80: endbr64
:
: /*
: * If requested size exceeds maximum for chunks, allocate an entire block
: * for this request.
: */
: if (unlikely(size > set->allocChunkLimit))
2.68 : 56bd84: cmp %rsi,0xc8(%rdi)
3.34 : 56bd8b: jb 56be10 <AllocSetAlloc+0x90>
: AllocSetFreeIndex():
: idx = 0;
0.17 : 56bd91: xor %ecx,%ecx
: if (size > (1 << ALLOC_MINBITS))
0.00 : 56bd93: cmp $0x8,%rsi
0.16 : 56bd97: jbe 56bda9 <AllocSetAlloc+0x29>
: idx = 31 - __builtin_clz((uint32) size - 1) - ALLOC_MINBITS + 1;
0.00 : 56bd99: sub $0x1,%esi
0.47 : 56bd9c: mov $0x1d,%ecx
0.50 : 56bda1: bsr %esi,%esi
6.38 : 56bda4: xor $0x1f,%esi
1.92 : 56bda7: sub %esi,%ecx
: AllocSetAlloc():
: * corresponding free list to see if there is a free chunk we could reuse.
: * If one is found, remove it from the free list, make it again a member
: * of the alloc set and return its data address.
: */
: fidx = AllocSetFreeIndex(size);
: chunk = set->freelist[fidx];
1.44 : 56bda9: movslq %ecx,%rdx
1.88 : 56bdac: add $0xa,%rdx
1.11 : 56bdb0: mov 0x8(%rdi,%rdx,8),%rax
: if (chunk != NULL)
19.90 : 56bdb5: test %rax,%rax
0.32 : 56bdb8: je 56bdd0 <AllocSetAlloc+0x50>
: {
: Assert(chunk->size >= size);
:
: set->freelist[fidx] = (AllocChunk) chunk->aset;
0.79 : 56bdba: mov 0x8(%rax),%rcx
: AllocSetAllocReturnChunk():
: return AllocChunkGetPointer(chunk);
13.97 : 56bdbe: add $0x10,%rax
: AllocSetAlloc():
: set->freelist[fidx] = (AllocChunk) chunk->aset;
0.00 : 56bdc2: mov %rcx,0x8(%rdi,%rdx,8)
: AllocSetAllocReturnChunk():
: chunk->aset = (void *) set;
0.00 : 56bdc7: mov %rdi,-0x8(%rax)
: AllocSetAlloc():
:
: return AllocSetAllocReturnChunk(set, size, chunk, chunk->size);
0.16 : 56bdcb: ret
0.00 : 56bdcc: nopl 0x0(%rax)
: }
:
: /*
: * Choose the actual chunk size to allocate.
: */
: chunk_size = (1 << ALLOC_MINBITS) << fidx;
1.10 : 56bdd0: mov $0x8,%esi
:
: /*
: * If there is enough room in the active allocation block, we will put the
: * chunk into that block. Else must start a new one.
: */
: if ((block = set->blocks) != NULL)
0.32 : 56bdd5: mov 0x50(%rdi),%rdx
: chunk_size = (1 << ALLOC_MINBITS) << fidx;
0.32 : 56bdd9: shl %cl,%esi
0.00 : 56bddb: movslq %esi,%rsi
: if ((block = set->blocks) != NULL)
0.00 : 56bdde: test %rdx,%rdx
0.47 : 56bde1: je 56be18 <AllocSetAlloc+0x98>
: {
: Size availspace = block->endptr - block->freeptr;
0.00 : 56bde3: mov 0x18(%rdx),%rax
4.80 : 56bde7: mov 0x20(%rdx),%rcx
:
: if (unlikely(availspace < (chunk_size + ALLOC_CHUNKHDRSZ)))
1.75 : 56bdeb: lea 0x10(%rsi),%r8
: Size availspace = block->endptr - block->freeptr;
0.47 : 56bdef: sub %rax,%rcx
: if (unlikely(availspace < (chunk_size + ALLOC_CHUNKHDRSZ)))
0.16 : 56bdf2: cmp %rcx,%r8
1.90 : 56bdf5: ja 56be20 <AllocSetAlloc+0xa0>
: chunk = (AllocChunk) (block->freeptr);
:
: /* Prepare to initialize the chunk header. */
: VALGRIND_MAKE_MEM_UNDEFINED(chunk, ALLOC_CHUNKHDRSZ);
:
: chunk->size = chunk_size;
0.00 : 56bdf7: mov %rsi,(%rax)
: AllocSetAllocReturnChunk():
: return AllocChunkGetPointer(chunk);
21.14 : 56bdfa: add $0x10,%rax
: AllocSetAlloc():
:
: block->freeptr += (chunk_size + ALLOC_CHUNKHDRSZ);
0.00 : 56bdfe: add %r8,0x18(%rdx)
: AllocSetAllocReturnChunk():
: chunk->aset = (void *) set;
0.16 : 56be02: mov %rdi,-0x8(%rax)
: AllocSetAlloc():
: Assert(block->freeptr <= block->endptr);
:
: return AllocSetAllocReturnChunk(set, size, chunk, chunk_size);
: }
4.58 : 56be06: ret
0.00 : 56be07: nopw 0x0(%rax,%rax,1)
: return AllocSetAllocLarge(set, size, flags);
0.00 : 56be10: jmp 56bcc0 <AllocSetAllocLarge>
0.00 : 56be15: nopl (%rax)
: return AllocSetAllocFromNewBlock(set, size, chunk_size);
0.00 : 56be18: jmp 56bae0 <AllocSetAllocFromNewBlock.constprop.0>
0.00 : 56be1d: nopl (%rax)
: return AllocSetAllocCarveOldAndAlloc(set, size, chunk_size,
0.00 : 56be20: jmp 56bbd0 <AllocSetAllocCarveOldAndAlloc.isra.0>
Percent | Source code & Disassembly of postgres for cycles (81 samples, percent: local period)
----------------------------------------------------------------------------------------------------
:
:
:
: Disassembly of section .text:
:
: 0000000000571220 <palloc>:
: palloc():
: MemoryContextStatsDetail(TopMemoryContext, 100, false);
: }
:
: void *
: palloc(Size size)
: {
13.44 : 571220: endbr64
2.45 : 571224: mov %rdi,%rsi
: /* duplicates MemoryContextAlloc to avoid increased overhead */
: void *ret;
: MemoryContext context = CurrentMemoryContext;
2.44 : 571227: mov 0x2be9b2(%rip),%rdi # 82fbe0 <CurrentMemoryContext>
:
: AssertArg(MemoryContextIsValid(context));
: AssertNotInCriticalSection(context);
: context->isReset = false;
:
: ret = context->methods->alloc(context, size, 0);
11.11 : 57122e: xor %edx,%edx
1.21 : 571230: mov 0x10(%rdi),%rax
: context->isReset = false;
31.08 : 571234: movb $0x0,0x4(%rdi)
: ret = context->methods->alloc(context, size, 0);
3.67 : 571238: mov (%rax),%rax
34.59 : 57123b: jmp *%rax
2.38% postgres postgres [.] AllocSetAlloc
0.37% postgres postgres [.] pfree
0.31% postgres postgres [.] palloc
0.12% postgres postgres [.] MemoryContextAlloc
0.12% postgres postgres [.] AllocSetAllocCarveOldAndAlloc.isra.0
0.10% postgres postgres [.] MemoryContextAllocZero
0.10% postgres postgres [.] palloc0
0.06% postgres postgres [.] MemoryContextAllocZeroAligned
0.02% postgres postgres [.] AllocSetAllocLarge
Attachments:
[text/plain] AllocateSetAlloc.txt (6.1K, ../../CAApHDvpfr07JBykytxDXLxMjtExgmSKRUaR4dXkAt5xaGumUBA@mail.gmail.com/2-AllocateSetAlloc.txt)
download | inline:
Percent | Source code & Disassembly of postgres for cycles (626 samples, percent: local period)
-----------------------------------------------------------------------------------------------------
:
:
:
: Disassembly of section .text:
:
: 000000000056bd80 <AllocSetAlloc>:
: AllocSetAlloc():
: * is marked, as mcxt.c will set it to UNDEFINED. In some paths we will
: * return space that is marked NOACCESS - AllocSetRealloc has to beware!
: */
: static void *
: AllocSetAlloc(MemoryContext context, Size size, int flags)
: {
7.66 : 56bd80: endbr64
:
: /*
: * If requested size exceeds maximum for chunks, allocate an entire block
: * for this request.
: */
: if (unlikely(size > set->allocChunkLimit))
2.68 : 56bd84: cmp %rsi,0xc8(%rdi)
3.34 : 56bd8b: jb 56be10 <AllocSetAlloc+0x90>
: AllocSetFreeIndex():
: idx = 0;
0.17 : 56bd91: xor %ecx,%ecx
: if (size > (1 << ALLOC_MINBITS))
0.00 : 56bd93: cmp $0x8,%rsi
0.16 : 56bd97: jbe 56bda9 <AllocSetAlloc+0x29>
: idx = 31 - __builtin_clz((uint32) size - 1) - ALLOC_MINBITS + 1;
0.00 : 56bd99: sub $0x1,%esi
0.47 : 56bd9c: mov $0x1d,%ecx
0.50 : 56bda1: bsr %esi,%esi
6.38 : 56bda4: xor $0x1f,%esi
1.92 : 56bda7: sub %esi,%ecx
: AllocSetAlloc():
: * corresponding free list to see if there is a free chunk we could reuse.
: * If one is found, remove it from the free list, make it again a member
: * of the alloc set and return its data address.
: */
: fidx = AllocSetFreeIndex(size);
: chunk = set->freelist[fidx];
1.44 : 56bda9: movslq %ecx,%rdx
1.88 : 56bdac: add $0xa,%rdx
1.11 : 56bdb0: mov 0x8(%rdi,%rdx,8),%rax
: if (chunk != NULL)
19.90 : 56bdb5: test %rax,%rax
0.32 : 56bdb8: je 56bdd0 <AllocSetAlloc+0x50>
: {
: Assert(chunk->size >= size);
:
: set->freelist[fidx] = (AllocChunk) chunk->aset;
0.79 : 56bdba: mov 0x8(%rax),%rcx
: AllocSetAllocReturnChunk():
: return AllocChunkGetPointer(chunk);
13.97 : 56bdbe: add $0x10,%rax
: AllocSetAlloc():
: set->freelist[fidx] = (AllocChunk) chunk->aset;
0.00 : 56bdc2: mov %rcx,0x8(%rdi,%rdx,8)
: AllocSetAllocReturnChunk():
: chunk->aset = (void *) set;
0.00 : 56bdc7: mov %rdi,-0x8(%rax)
: AllocSetAlloc():
:
: return AllocSetAllocReturnChunk(set, size, chunk, chunk->size);
0.16 : 56bdcb: ret
0.00 : 56bdcc: nopl 0x0(%rax)
: }
:
: /*
: * Choose the actual chunk size to allocate.
: */
: chunk_size = (1 << ALLOC_MINBITS) << fidx;
1.10 : 56bdd0: mov $0x8,%esi
:
: /*
: * If there is enough room in the active allocation block, we will put the
: * chunk into that block. Else must start a new one.
: */
: if ((block = set->blocks) != NULL)
0.32 : 56bdd5: mov 0x50(%rdi),%rdx
: chunk_size = (1 << ALLOC_MINBITS) << fidx;
0.32 : 56bdd9: shl %cl,%esi
0.00 : 56bddb: movslq %esi,%rsi
: if ((block = set->blocks) != NULL)
0.00 : 56bdde: test %rdx,%rdx
0.47 : 56bde1: je 56be18 <AllocSetAlloc+0x98>
: {
: Size availspace = block->endptr - block->freeptr;
0.00 : 56bde3: mov 0x18(%rdx),%rax
4.80 : 56bde7: mov 0x20(%rdx),%rcx
:
: if (unlikely(availspace < (chunk_size + ALLOC_CHUNKHDRSZ)))
1.75 : 56bdeb: lea 0x10(%rsi),%r8
: Size availspace = block->endptr - block->freeptr;
0.47 : 56bdef: sub %rax,%rcx
: if (unlikely(availspace < (chunk_size + ALLOC_CHUNKHDRSZ)))
0.16 : 56bdf2: cmp %rcx,%r8
1.90 : 56bdf5: ja 56be20 <AllocSetAlloc+0xa0>
: chunk = (AllocChunk) (block->freeptr);
:
: /* Prepare to initialize the chunk header. */
: VALGRIND_MAKE_MEM_UNDEFINED(chunk, ALLOC_CHUNKHDRSZ);
:
: chunk->size = chunk_size;
0.00 : 56bdf7: mov %rsi,(%rax)
: AllocSetAllocReturnChunk():
: return AllocChunkGetPointer(chunk);
21.14 : 56bdfa: add $0x10,%rax
: AllocSetAlloc():
:
: block->freeptr += (chunk_size + ALLOC_CHUNKHDRSZ);
0.00 : 56bdfe: add %r8,0x18(%rdx)
: AllocSetAllocReturnChunk():
: chunk->aset = (void *) set;
0.16 : 56be02: mov %rdi,-0x8(%rax)
: AllocSetAlloc():
: Assert(block->freeptr <= block->endptr);
:
: return AllocSetAllocReturnChunk(set, size, chunk, chunk_size);
: }
4.58 : 56be06: ret
0.00 : 56be07: nopw 0x0(%rax,%rax,1)
: return AllocSetAllocLarge(set, size, flags);
0.00 : 56be10: jmp 56bcc0 <AllocSetAllocLarge>
0.00 : 56be15: nopl (%rax)
: return AllocSetAllocFromNewBlock(set, size, chunk_size);
0.00 : 56be18: jmp 56bae0 <AllocSetAllocFromNewBlock.constprop.0>
0.00 : 56be1d: nopl (%rax)
: return AllocSetAllocCarveOldAndAlloc(set, size, chunk_size,
0.00 : 56be20: jmp 56bbd0 <AllocSetAllocCarveOldAndAlloc.isra.0>
[text/plain] palloc.txt (1.5K, ../../CAApHDvpfr07JBykytxDXLxMjtExgmSKRUaR4dXkAt5xaGumUBA@mail.gmail.com/3-palloc.txt)
download | inline:
Percent | Source code & Disassembly of postgres for cycles (81 samples, percent: local period)
----------------------------------------------------------------------------------------------------
:
:
:
: Disassembly of section .text:
:
: 0000000000571220 <palloc>:
: palloc():
: MemoryContextStatsDetail(TopMemoryContext, 100, false);
: }
:
: void *
: palloc(Size size)
: {
13.44 : 571220: endbr64
2.45 : 571224: mov %rdi,%rsi
: /* duplicates MemoryContextAlloc to avoid increased overhead */
: void *ret;
: MemoryContext context = CurrentMemoryContext;
2.44 : 571227: mov 0x2be9b2(%rip),%rdi # 82fbe0 <CurrentMemoryContext>
:
: AssertArg(MemoryContextIsValid(context));
: AssertNotInCriticalSection(context);
: context->isReset = false;
:
: ret = context->methods->alloc(context, size, 0);
11.11 : 57122e: xor %edx,%edx
1.21 : 571230: mov 0x10(%rdi),%rax
: context->isReset = false;
31.08 : 571234: movb $0x0,0x4(%rdi)
: ret = context->methods->alloc(context, size, 0);
3.67 : 571238: mov (%rax),%rax
34.59 : 57123b: jmp *%rax
[text/plain] percent.txt (577B, ../../CAApHDvpfr07JBykytxDXLxMjtExgmSKRUaR4dXkAt5xaGumUBA@mail.gmail.com/4-percent.txt)
download | inline:
2.38% postgres postgres [.] AllocSetAlloc
0.37% postgres postgres [.] pfree
0.31% postgres postgres [.] palloc
0.12% postgres postgres [.] MemoryContextAlloc
0.12% postgres postgres [.] AllocSetAllocCarveOldAndAlloc.isra.0
0.10% postgres postgres [.] MemoryContextAllocZero
0.10% postgres postgres [.] palloc0
0.06% postgres postgres [.] MemoryContextAllocZeroAligned
0.02% postgres postgres [.] AllocSetAllocLarge
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Avoid stack frame setup in performance critical routines using tail calls
@ 2021-07-20 07:03 Andres Freund <[email protected]>
parent: David Rowley <[email protected]>
0 siblings, 1 reply; 7+ messages in thread
From: Andres Freund @ 2021-07-20 07:03 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: pgsql-hackers; Robert Haas <[email protected]>; Michael Paquier <[email protected]>; Tomas Vondra <[email protected]>
Hi,
On Mon, Jul 19, 2021, at 23:53, David Rowley wrote:
> On Tue, 20 Jul 2021 at 18:17, Andres Freund <[email protected]> wrote:
> > Any chance you could show a `perf annotate AllocSetAlloc` and `perf annotate
> > palloc` from a patched run? And perhaps how high their percentages of the
> > total work are. E.g. using something like
> > perf report -g none|grep -E 'AllocSetAlloc|palloc|MemoryContextAlloc|pfree'
>
> Sure. See attached.
>
> David
>
> Attachments:
> * AllocateSetAlloc.txt
> * palloc.txt
> * percent.txt
Huh, that's interesting. You have some control flow enforcement stuff turned on (the endbr64). And it looks like it has a non zero cost (or maybe it's just skid). Did you enable that intentionally? If not, what compiler/version/distro is it? I think at least on GCC that's -fcf-protection=...
Andres
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Avoid stack frame setup in performance critical routines using tail calls
@ 2021-07-20 07:37 David Rowley <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 7+ messages in thread
From: David Rowley @ 2021-07-20 07:37 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: pgsql-hackers; Robert Haas <[email protected]>; Michael Paquier <[email protected]>; Tomas Vondra <[email protected]>
On Tue, 20 Jul 2021 at 19:04, Andres Freund <[email protected]> wrote:
> > * AllocateSetAlloc.txt
> > * palloc.txt
> > * percent.txt
>
> Huh, that's interesting. You have some control flow enforcement stuff turned on (the endbr64). And it looks like it has a non zero cost (or maybe it's just skid). Did you enable that intentionally? If not, what compiler/version/distro is it? I think at least on GCC that's -fcf-protection=...
It's ubuntu 21.04 with gcc 10.3 (specifically gcc version 10.3.0
(Ubuntu 10.3.0-1ubuntu1)
I've attached the same results from compiling with clang 12
(12.0.0-3ubuntu1~21.04.1)
David
Percent | Source code & Disassembly of postgres for cycles (707 samples, percent: local period)
-----------------------------------------------------------------------------------------------------
:
:
:
: Disassembly of section .text:
:
: 00000000008e7c10 <AllocSetAlloc>:
: AllocSetAlloc():
:
: /*
: * If requested size exceeds maximum for chunks, allocate an entire block
: * for this request.
: */
: if (unlikely(size > set->allocChunkLimit))
7.48 : 8e7c10: cmp %rsi,0xc8(%rdi)
3.26 : 8e7c17: jb 8e7c81 <AllocSetAlloc+0x71>
0.00 : 8e7c19: xor %eax,%eax
: AllocSetFreeIndex():
: if (size > (1 << ALLOC_MINBITS))
0.44 : 8e7c1b: cmp $0x9,%rsi
0.00 : 8e7c1f: jb 8e7c2d <AllocSetAlloc+0x1d>
: idx = 31 - __builtin_clz((uint32) size - 1) - ALLOC_MINBITS + 1;
0.00 : 8e7c21: add $0xffffffff,%esi
0.98 : 8e7c24: bsr %esi,%eax
9.59 : 8e7c27: xor $0xffffffe0,%eax
1.44 : 8e7c2a: add $0x1e,%eax
: AllocSetAlloc():
: * corresponding free list to see if there is a free chunk we could reuse.
: * If one is found, remove it from the free list, make it again a member
: * of the alloc set and return its data address.
: */
: fidx = AllocSetFreeIndex(size);
: chunk = set->freelist[fidx];
1.67 : 8e7c2d: movslq %eax,%rcx
4.10 : 8e7c30: mov 0x58(%rdi,%rcx,8),%rax
: if (chunk != NULL)
15.97 : 8e7c35: test %rax,%rax
0.28 : 8e7c38: je 8e7c45 <AllocSetAlloc+0x35>
: {
: Assert(chunk->size >= size);
:
: set->freelist[fidx] = (AllocChunk) chunk->aset;
0.00 : 8e7c3a: mov 0x8(%rax),%rdx
13.33 : 8e7c3e: mov %rdx,0x58(%rdi,%rcx,8)
0.28 : 8e7c43: jmp 8e7c73 <AllocSetAlloc+0x63>
0.00 : 8e7c45: mov $0x8,%eax
: }
:
: /*
: * Choose the actual chunk size to allocate.
: */
: chunk_size = (1 << ALLOC_MINBITS) << fidx;
0.71 : 8e7c4a: shl %cl,%eax
0.15 : 8e7c4c: movslq %eax,%rsi
:
: /*
: * If there is enough room in the active allocation block, we will put the
: * chunk into that block. Else must start a new one.
: */
: if ((block = set->blocks) != NULL)
0.43 : 8e7c4f: mov 0x50(%rdi),%rdx
1.13 : 8e7c53: test %rdx,%rdx
0.14 : 8e7c56: je 8e7c7c <AllocSetAlloc+0x6c>
: {
: Size availspace = block->endptr - block->freeptr;
0.00 : 8e7c58: mov 0x18(%rdx),%rax
6.98 : 8e7c5c: mov 0x20(%rdx),%rcx
2.30 : 8e7c60: sub %rax,%rcx
:
: if (unlikely(availspace < (chunk_size + ALLOC_CHUNKHDRSZ)))
0.00 : 8e7c63: lea 0x10(%rsi),%r8
0.14 : 8e7c67: cmp %r8,%rcx
2.02 : 8e7c6a: jb 8e7c86 <AllocSetAlloc+0x76>
: chunk = (AllocChunk) (block->freeptr);
:
: /* Prepare to initialize the chunk header. */
: VALGRIND_MAKE_MEM_UNDEFINED(chunk, ALLOC_CHUNKHDRSZ);
:
: chunk->size = chunk_size;
2.04 : 8e7c6c: mov %rsi,(%rax)
:
: block->freeptr += (chunk_size + ALLOC_CHUNKHDRSZ);
20.16 : 8e7c6f: add %r8,0x18(%rdx)
0.28 : 8e7c73: mov %rdi,0x8(%rax)
4.70 : 8e7c77: add $0x10,%rax
: Assert(block->freeptr <= block->endptr);
:
: return AllocSetAllocReturnChunk(set, size, chunk, chunk_size);
: }
0.00 : 8e7c7b: ret
: return AllocSetAllocFromNewBlock(set, size, chunk_size);
0.00 : 8e7c7c: jmp 8e8470 <AllocSetAllocFromNewBlock>
: return AllocSetAllocLarge(set, size, flags);
0.00 : 8e7c81: jmp 8e8330 <AllocSetAllocLarge>
: return AllocSetAllocCarveOldAndAlloc(set, size, chunk_size,
0.00 : 8e7c86: jmp 8e83e0 <AllocSetAllocCarveOldAndAlloc>
Percent | Source code & Disassembly of postgres for cycles (123 samples, percent: local period)
-----------------------------------------------------------------------------------------------------
:
:
:
: Disassembly of section .text:
:
: 00000000008ee3a0 <palloc>:
: palloc():
: MemoryContextStatsDetail(TopMemoryContext, 100, false);
: }
:
: void *
: palloc(Size size)
: {
9.12 : 8ee3a0: mov %rdi,%rsi
: /* duplicates MemoryContextAlloc to avoid increased overhead */
: void *ret;
: MemoryContext context = CurrentMemoryContext;
2.47 : 8ee3a3: mov 0x295a86(%rip),%rdi # b83e30 <CurrentMemoryContext>
:
: AssertArg(MemoryContextIsValid(context));
: AssertNotInCriticalSection(context);
: context->isReset = false;
22.83 : 8ee3aa: movb $0x0,0x4(%rdi)
:
: ret = context->methods->alloc(context, size, 0);
34.25 : 8ee3ae: mov 0x10(%rdi),%rax
7.68 : 8ee3b2: mov (%rax),%rax
23.66 : 8ee3b5: xor %edx,%edx
0.00 : 8ee3b7: jmp *%rax
2.27% postgres postgres [.] AllocSetAlloc
0.59% postgres postgres [.] pfree
0.44% postgres postgres [.] MemoryContextAllocZero
0.39% postgres postgres [.] palloc
0.34% postgres postgres [.] MemoryContextAllocZeroAligned
0.27% postgres postgres [.] palloc0
0.17% postgres postgres [.] AllocSetAllocCarveOldAndAlloc
0.09% postgres postgres [.] MemoryContextAlloc
0.02% postgres postgres [.] AllocSetAllocLarge
0.01% postgres postgres [.] AllocSetAllocFromNewBlock
Attachments:
[text/plain] AllocSetAlloc_clang.txt (4.5K, ../../CAApHDvpmhKQTcbAT3ryr+_b8v9B53C1AXkV=EeQeUJ+mOQCH9Q@mail.gmail.com/2-AllocSetAlloc_clang.txt)
download | inline:
Percent | Source code & Disassembly of postgres for cycles (707 samples, percent: local period)
-----------------------------------------------------------------------------------------------------
:
:
:
: Disassembly of section .text:
:
: 00000000008e7c10 <AllocSetAlloc>:
: AllocSetAlloc():
:
: /*
: * If requested size exceeds maximum for chunks, allocate an entire block
: * for this request.
: */
: if (unlikely(size > set->allocChunkLimit))
7.48 : 8e7c10: cmp %rsi,0xc8(%rdi)
3.26 : 8e7c17: jb 8e7c81 <AllocSetAlloc+0x71>
0.00 : 8e7c19: xor %eax,%eax
: AllocSetFreeIndex():
: if (size > (1 << ALLOC_MINBITS))
0.44 : 8e7c1b: cmp $0x9,%rsi
0.00 : 8e7c1f: jb 8e7c2d <AllocSetAlloc+0x1d>
: idx = 31 - __builtin_clz((uint32) size - 1) - ALLOC_MINBITS + 1;
0.00 : 8e7c21: add $0xffffffff,%esi
0.98 : 8e7c24: bsr %esi,%eax
9.59 : 8e7c27: xor $0xffffffe0,%eax
1.44 : 8e7c2a: add $0x1e,%eax
: AllocSetAlloc():
: * corresponding free list to see if there is a free chunk we could reuse.
: * If one is found, remove it from the free list, make it again a member
: * of the alloc set and return its data address.
: */
: fidx = AllocSetFreeIndex(size);
: chunk = set->freelist[fidx];
1.67 : 8e7c2d: movslq %eax,%rcx
4.10 : 8e7c30: mov 0x58(%rdi,%rcx,8),%rax
: if (chunk != NULL)
15.97 : 8e7c35: test %rax,%rax
0.28 : 8e7c38: je 8e7c45 <AllocSetAlloc+0x35>
: {
: Assert(chunk->size >= size);
:
: set->freelist[fidx] = (AllocChunk) chunk->aset;
0.00 : 8e7c3a: mov 0x8(%rax),%rdx
13.33 : 8e7c3e: mov %rdx,0x58(%rdi,%rcx,8)
0.28 : 8e7c43: jmp 8e7c73 <AllocSetAlloc+0x63>
0.00 : 8e7c45: mov $0x8,%eax
: }
:
: /*
: * Choose the actual chunk size to allocate.
: */
: chunk_size = (1 << ALLOC_MINBITS) << fidx;
0.71 : 8e7c4a: shl %cl,%eax
0.15 : 8e7c4c: movslq %eax,%rsi
:
: /*
: * If there is enough room in the active allocation block, we will put the
: * chunk into that block. Else must start a new one.
: */
: if ((block = set->blocks) != NULL)
0.43 : 8e7c4f: mov 0x50(%rdi),%rdx
1.13 : 8e7c53: test %rdx,%rdx
0.14 : 8e7c56: je 8e7c7c <AllocSetAlloc+0x6c>
: {
: Size availspace = block->endptr - block->freeptr;
0.00 : 8e7c58: mov 0x18(%rdx),%rax
6.98 : 8e7c5c: mov 0x20(%rdx),%rcx
2.30 : 8e7c60: sub %rax,%rcx
:
: if (unlikely(availspace < (chunk_size + ALLOC_CHUNKHDRSZ)))
0.00 : 8e7c63: lea 0x10(%rsi),%r8
0.14 : 8e7c67: cmp %r8,%rcx
2.02 : 8e7c6a: jb 8e7c86 <AllocSetAlloc+0x76>
: chunk = (AllocChunk) (block->freeptr);
:
: /* Prepare to initialize the chunk header. */
: VALGRIND_MAKE_MEM_UNDEFINED(chunk, ALLOC_CHUNKHDRSZ);
:
: chunk->size = chunk_size;
2.04 : 8e7c6c: mov %rsi,(%rax)
:
: block->freeptr += (chunk_size + ALLOC_CHUNKHDRSZ);
20.16 : 8e7c6f: add %r8,0x18(%rdx)
0.28 : 8e7c73: mov %rdi,0x8(%rax)
4.70 : 8e7c77: add $0x10,%rax
: Assert(block->freeptr <= block->endptr);
:
: return AllocSetAllocReturnChunk(set, size, chunk, chunk_size);
: }
0.00 : 8e7c7b: ret
: return AllocSetAllocFromNewBlock(set, size, chunk_size);
0.00 : 8e7c7c: jmp 8e8470 <AllocSetAllocFromNewBlock>
: return AllocSetAllocLarge(set, size, flags);
0.00 : 8e7c81: jmp 8e8330 <AllocSetAllocLarge>
: return AllocSetAllocCarveOldAndAlloc(set, size, chunk_size,
0.00 : 8e7c86: jmp 8e83e0 <AllocSetAllocCarveOldAndAlloc>
[text/plain] palloc_clang.txt (1.3K, ../../CAApHDvpmhKQTcbAT3ryr+_b8v9B53C1AXkV=EeQeUJ+mOQCH9Q@mail.gmail.com/3-palloc_clang.txt)
download | inline:
Percent | Source code & Disassembly of postgres for cycles (123 samples, percent: local period)
-----------------------------------------------------------------------------------------------------
:
:
:
: Disassembly of section .text:
:
: 00000000008ee3a0 <palloc>:
: palloc():
: MemoryContextStatsDetail(TopMemoryContext, 100, false);
: }
:
: void *
: palloc(Size size)
: {
9.12 : 8ee3a0: mov %rdi,%rsi
: /* duplicates MemoryContextAlloc to avoid increased overhead */
: void *ret;
: MemoryContext context = CurrentMemoryContext;
2.47 : 8ee3a3: mov 0x295a86(%rip),%rdi # b83e30 <CurrentMemoryContext>
:
: AssertArg(MemoryContextIsValid(context));
: AssertNotInCriticalSection(context);
: context->isReset = false;
22.83 : 8ee3aa: movb $0x0,0x4(%rdi)
:
: ret = context->methods->alloc(context, size, 0);
34.25 : 8ee3ae: mov 0x10(%rdi),%rax
7.68 : 8ee3b2: mov (%rax),%rax
23.66 : 8ee3b5: xor %edx,%edx
0.00 : 8ee3b7: jmp *%rax
[text/plain] percent_clang.txt (642B, ../../CAApHDvpmhKQTcbAT3ryr+_b8v9B53C1AXkV=EeQeUJ+mOQCH9Q@mail.gmail.com/4-percent_clang.txt)
download | inline:
2.27% postgres postgres [.] AllocSetAlloc
0.59% postgres postgres [.] pfree
0.44% postgres postgres [.] MemoryContextAllocZero
0.39% postgres postgres [.] palloc
0.34% postgres postgres [.] MemoryContextAllocZeroAligned
0.27% postgres postgres [.] palloc0
0.17% postgres postgres [.] AllocSetAllocCarveOldAndAlloc
0.09% postgres postgres [.] MemoryContextAlloc
0.02% postgres postgres [.] AllocSetAllocLarge
0.01% postgres postgres [.] AllocSetAllocFromNewBlock
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Avoid stack frame setup in performance critical routines using tail calls
@ 2021-07-20 15:57 Andres Freund <[email protected]>
parent: David Rowley <[email protected]>
0 siblings, 0 replies; 7+ messages in thread
From: Andres Freund @ 2021-07-20 15:57 UTC (permalink / raw)
To: David Rowley <[email protected]>; +Cc: pgsql-hackers; Robert Haas <[email protected]>; Michael Paquier <[email protected]>; Tomas Vondra <[email protected]>
Hi,
On 2021-07-20 19:37:46 +1200, David Rowley wrote:
> On Tue, 20 Jul 2021 at 19:04, Andres Freund <[email protected]> wrote:
> > > * AllocateSetAlloc.txt
> > > * palloc.txt
> > > * percent.txt
> >
> > Huh, that's interesting. You have some control flow enforcement stuff turned on (the endbr64). And it looks like it has a non zero cost (or maybe it's just skid). Did you enable that intentionally? If not, what compiler/version/distro is it? I think at least on GCC that's -fcf-protection=...
>
> It's ubuntu 21.04 with gcc 10.3 (specifically gcc version 10.3.0
> (Ubuntu 10.3.0-1ubuntu1)
>
> I've attached the same results from compiling with clang 12
> (12.0.0-3ubuntu1~21.04.1)
It looks like the ubuntu folks have changed the default for CET to on.
andres@ubuntu2020:~$ echo 'int foo(void) { return 17;}' > test.c && gcc -O2 -c -o test.o test.c && objdump -S test.o
test.o: file format elf64-x86-64
Disassembly of section .text:
0000000000000000 <foo>:
0: f3 0f 1e fa endbr64
4: b8 11 00 00 00 mov $0x11,%eax
9: c3 retq
andres@ubuntu2020:~$ echo 'int foo(void) { return 17;}' > test.c && gcc -O2 -fcf-protection=none -c -o test.o test.c && objdump -S test.o
test.o: file format elf64-x86-64
Disassembly of section .text:
0000000000000000 <foo>:
0: b8 11 00 00 00 mov $0x11,%eax
5: c3 retq
Independent of this patch, it might be worth running a benchmark with
the default options, and one with -fcf-protection=none. None of my
machines support it...
$ cpuid -1|grep CET
CET_SS: CET shadow stack = false
CET_IBT: CET indirect branch tracking = false
XCR0 supported: CET_U state = false
XCR0 supported: CET_S state = false
Here it adds about 40kB of .text, but I can't measure the CET
overhead...
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2021-07-20 15:57 UTC | newest]
Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-07-19 19:59 Avoid stack frame setup in performance critical routines using tail calls Andres Freund <[email protected]>
2021-07-20 04:50 ` David Rowley <[email protected]>
2021-07-20 06:16 ` Andres Freund <[email protected]>
2021-07-20 06:53 ` David Rowley <[email protected]>
2021-07-20 07:03 ` Andres Freund <[email protected]>
2021-07-20 07:37 ` David Rowley <[email protected]>
2021-07-20 15:57 ` Andres Freund <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox