References: <53ACB190.2030206@catalyst.net.nz>
 <CAOR=d=3=Y6yNtwQenwVBGe6ZgjAeA8PwgYb3=aQEKn+vYWhBJA@mail.gmail.com>
 <53ACD6C4.9050605@catalyst.net.nz>
 <20140627091951.GA16680@awork2.anarazel.de>
 <53B283F3.7020005@catalyst.net.nz>
 <20140701101307.GO21422@awork2.anarazel.de>
 <53BF326F.2070903@catalyst.net.nz>
 <20140711082207.GI17261@awork2.anarazel.de> <53BFA63C.60909@catalyst.net.nz>
Message-ID: <1405084749.26978.YahooMailNeo@web122306.mail.ne1.yahoo.com>
Date: Fri, 11 Jul 2014 06:19:09 -0700
From: Kevin Grittner <kgrittn@ymail.com>
Reply-To: Kevin Grittner <kgrittn@ymail.com>
Subject: Re: 60 core performance with 9.3
To: Mark Kirkwood <mark.kirkwood@catalyst.net.nz>,
 Andres Freund <andres@2ndquadrant.com>
Cc: "pgsql-performance@postgresql.org" <pgsql-performance@postgresql.org>
In-Reply-To: <53BFA63C.60909@catalyst.net.nz>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Precedence: bulk
Sender: pgsql-performance-owner@postgresql.org

Mark Kirkwood <mark.kirkwood@catalyst.net.nz> wrote:
> On 11/07/14 20:22, Andres Freund wrote:

>> So, the majority of the time is spent in numa page migration.
>> Can you disable numa_balancing? I'm not sure if your kernel
>> version does that at runtime or whether you need to reboot.
>> The kernel.numa_balancing sysctl might work. Otherwise you
>> probably need to boot with numa_balancing=0.
>>
>> It'd also be worthwhile to test this with numactl --interleave.
>
> That was my feeling too - but I had no idea what the magic switch
> was to tame it (appears to be in 3.13 kernels), will experiment
> and report back. Thanks again!

It might be worth a test using a cpuset to interleave OS cache and
the NUMA patch I submitted to the current CF to see whether this is
getting into territory where the patch makes a bigger difference. 
I would expect it to do much better than using numactl --interleave
because work_mem and other process-local memory would be allocated
in "near" memory for each process.

http://www.postgresql.org/message-id/1402267501.41111.YahooMailNeo@web122304.mail.ne1.yahoo.com

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance