pg.ddx.io pgsql-performance@postgresql.org mailing list archive help / color / mirror / Atom feed From: Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
To: Andres Freund <andres@2ndquadrant.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: 60 core performance with 9.3
Date: Thu, 17 Jul 2014 11:58:46 +1200
Message-ID: <53C711B6.5040802@catalyst.net.nz> (raw )
In-Reply-To: <20140711082207.GI17261@awork2.anarazel.de >
References: <53ACB190.2030206@catalyst.net.nz >
<CAOR=d=3=Y6yNtwQenwVBGe6ZgjAeA8PwgYb3=aQEKn+vYWhBJA@mail.gmail.com >
<53ACD6C4.9050605@catalyst.net.nz >
<20140627091951.GA16680@awork2.anarazel.de >
<53B283F3.7020005@catalyst.net.nz >
<20140701101307.GO21422@awork2.anarazel.de >
<53BF326F.2070903@catalyst.net.nz >
<20140711082207.GI17261@awork2.anarazel.de >
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-performance>
On 11/07/14 20:22, Andres Freund wrote:
> On 2014-07-11 12:40:15 +1200, Mark Kirkwood wrote:
>> Full report http://paste.ubuntu.com/7777886/
>
>> #
>> 8.82% postgres [kernel.kallsyms] [k]
>> _raw_spin_lock_irqsave
>> |
>> --- _raw_spin_lock_irqsave
>> |
>> |--75.69%-- pagevec_lru_move_fn
>> | __lru_cache_add
>> | lru_cache_add
>> | putback_lru_page
>> | migrate_pages
>> | migrate_misplaced_page
>> | do_numa_page
>> | handle_mm_fault
>> | __do_page_fault
>> | do_page_fault
>> | page_fault
>
> So, the majority of the time is spent in numa page migration. Can you
> disable numa_balancing? I'm not sure if your kernel version does that at
> runtime or whether you need to reboot.
> The kernel.numa_balancing sysctl might work. Otherwise you probably need
> to boot with numa_balancing=0.
>
> It'd also be worthwhile to test this with numactl --interleave.
>
Trying out with numa_balancing=0 seemed to get essentially the same
performance. Similarly wrapping postgres startup with --interleave.
All this made me want to try with numa *really* disabled. So rebooted
the box with "numa=off" appended to the kernel cmdline. Somewhat
surprisingly (to me anyway), the numbers were essentially identical. The
profile, however is quite different:
Full report at http://paste.ubuntu.com/7806285/
4.56% postgres [kernel.kallsyms] [k]
_raw_spin_lock_irqsave
|
--- _raw_spin_lock_irqsave
|
|--41.89%-- try_to_wake_up
| |
| |--96.12%-- default_wake_function
| | |
| | |--99.96%-- pollwake
| | | __wake_up_common
| | | __wake_up_sync_key
| | | sock_def_readable
| | | |
| | | |--99.94%--
unix_stream_sendmsg
| | | |
sock_sendmsg
| | | |
SYSC_sendto
| | | |
sys_sendto
| | | | tracesys
| | | |
__libc_send
| | | | pq_flush
| | | |
ReadyForQuery
| | | |
PostgresMain
| | | |
ServerLoop
| | | |
PostmasterMain
| | | | main
| | | |
__libc_start_main
| | | --0.06%-- [...]
| | --0.04%-- [...]
| |
| |--2.87%-- wake_up_process
| | |
| | |--95.71%--
wake_up_sem_queue_do
| | | SYSC_semtimedop
| | | sys_semop
| | | tracesys
| | | __GI___semop
| | | |
| | | |--99.75%--
LWLockRelease
| | | | |
| | | |
|--25.09%-- RecordTransactionCommit
| | | | |
CommitTransaction
| | | | |
CommitTransactionCommand
| | | | |
finish_xact_command.part.4
| | | | |
PostgresMain
| | | | |
ServerLoop
| | | | |
PostmasterMain
| | | | |
main
| | | | |
__libc_start_main
regards
Mark
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
view thread (26+ messages) latest in thread
Message-ID: <53C711B6.5040802@catalyst.net.nz>
Permalink: ../53C711B6.5040802@catalyst.net.nz/
Also on: postgresql.org/message-id/53C711B6.5040802@catalyst.net.nz
copy link · copy postgr.es
reply Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: mark.kirkwood@catalyst.net.nz, andres@2ndquadrant.com
Subject: Re: 60 core performance with 9.3
In-Reply-To: <53C711B6.5040802@catalyst.net.nz>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox