Message-ID: <53977376.3040406@optionshouse.com>
Date: Tue, 10 Jun 2014 16:07:02 -0500
From: Shaun Thomas <sthomas@optionshouse.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:24.0) Gecko/20100101 Thunderbird/24.5.0
MIME-Version: 1.0
To: Brio <brianoraas@gmail.com>, <pgsql-performance@postgresql.org>
Subject: Re: postgres files in use not staying in linux file cache
References: 
 <CAM+G8pRg9-Dvfr3dam9qG-etQGNGUQEO-cDPyhwR-_HTvWQABA@mail.gmail.com>
In-Reply-To: 
 <CAM+G8pRg9-Dvfr3dam9qG-etQGNGUQEO-cDPyhwR-_HTvWQABA@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
content-transfer-encoding: 7bit
Precedence: bulk
Sender: pgsql-performance-owner@postgresql.org

On 06/05/2014 04:32 PM, Brio wrote:

> So here's where I'm stuck. How can reading a file not leave it in the
> Linux cache? I'd expect it to enter the inactive list (which is about
> 80GB), so I'd expect another 80GB would need to be read before it would
> be its turn to be evicted.... which should take a long time if my
> maximum read speed is 100MB/s.

So here's the thing. The Linux page reclamation code is *extremely 
broken* in everything before 3.11. Take a look at this, then realize 
that this is *only one patch* from several that target the memory 
manager weightings:

http://linux-kernel.2935.n7.nabble.com/patch-v2-0-3-mm-improve-page-aging-fairness-between-zones-nodes-td696105.html

This is especially true of the 3.2 kernel you're using. It's extremely 
aggressive about ageing pages out of memory when there's high memory 
pressure from frequent disk reads. Chances of promotion into the active 
set is dismal, so you end up with a constant churn between inactive and 
disk. Worse, if you kick it hard enough by having too many PostgreSQL 
backends using memory, it'll actively purge the active set while still 
failing to promote the inactive set.

The dev that linked me to this patch said he tested it against 3.10, 
meaning it probably went into 3.11 or 3.12. So I personally wouldn't 
trust anything before 3.13. :p

Since you're using Ubuntu 12.04, I strongly suggest upgrading your core 
to 12.04.4 and apply the linux-generic-lts-saucy pseudo-package to at 
least get onto the 3.11 instead. The 3.2 kernel is pants-on-head 
retarded; we've had a lot more luck with 3.8 and above.

Cheers!

-- 
Shaun Thomas
OptionsHouse, LLC | 141 W. Jackson Blvd. | Suite 800 | Chicago IL, 60604
312-676-8870
sthomas@optionshouse.com

______________________________________________

See http://www.peak6.com/email_disclaimer/ for terms and conditions related to this email


-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance