References: 
 <CAP_rwwnds6tgUT33uei4C0bn+VChWXVb1Z9Rgi+7M-CP4SSEEA@mail.gmail.com>
Message-ID: <1415981309.90631.YahooMailNeo@web133205.mail.ir2.yahoo.com>
Date: Fri, 14 Nov 2014 16:08:29 +0000
From: Glyn Astill <glynastill@yahoo.co.uk>
Reply-To: Glyn Astill <glynastill@yahoo.co.uk>
Subject: Re: 9.0 performance degradation with kernel 3.11
To: =?utf-8?B?RmlsaXAgUmVtYmlhxYJrb3dza2k=?= <filip.rembialkowski@gmail.com>,
 "pgsql-performance@postgresql.org" <pgsql-performance@postgresql.org>
In-Reply-To: 
 <CAP_rwwnds6tgUT33uei4C0bn+VChWXVb1Z9Rgi+7M-CP4SSEEA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Precedence: bulk
Sender: pgsql-performance-owner@postgresql.org

> From: Filip Rembia=C5=82kowski <filip.rembialkowski@gmail.com>
>To: pgsql-performance@postgresql.org=20
>Sent: Thursday, 13 November 2014, 8:10
>Subject: [PERFORM] 9.0 performance degradation with kernel 3.11
>=20
>
>Hi
>
>After upgrading our 9.0 database server
>
>from:
>openSUSE 11.4, kernel 2.6.37.6-24-default, Pg 9.0.13
>
>to:
>openSUSE 13.1, kernel v 3.11.10-21-default, Pg 9.0.15
>
>... and  overall server load is +1 after that.
>
>We did not add any new services/daemons.
>
>It's hard to track down to individual queries - when I tested most
>individual query times are same as before the migration.
>
>
>Any - ANY - hints will be much appreciated.
>
>Thanks
>Filip
>

It's hard to say much going on the little information, but assuming everyth=
ing was rosy for you with your 2.6 version, and you've kept the basics like=
 hardware, filesystem, io scheduler etc the same, there are a few kernel tu=
nables to tweak on later kernels.

Usually defragmentation of transparent huge pages causes an issue and it's =
best to turn off the defrag option:


    echo always > /sys/kernel/mm/transparent_hugepage/enabled=20
    echo madvise > /sys/kernel/mm/transparent_hugepage/defrag


It's also recommended to increase the value of sched_migration_cost (I thin=
k now called sched_migration_cost_ns in 3.11+) and disable sched_autogroup_=
enabled.


    kernel.sched_migration_cost=3D5000000=20
    kernel.sched_autogroup_enabled=3D0

Also disable vm.zone_reclaim_mode

    vm.zone_reclaim_mode=3D0


On some of our systems I also saw marked improvements increasing the values=
 of kernel.sched_min_granularity_ns and kernel.sched_wakeup_granularity_ns =
too, on some other systems this had no effect.  So you may want to try to s=
ee if some larger values there help.

A lot of the earlier 3.x kernels aren't great with PostgreSQL, one of the n=
oted issues being a "stable pages" feature that blocks processes modifying =
pages that are currently being written back until the write completes.  I t=
hink people have noted this gets better in 3.9 onwards, but I personally di=
dn't see much of a marked improvement until 3.16.


>Thanks
>Filip
>


--=20
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance