To: "Tom Lane" <tgl@sss.pgh.pa.us>
Cc: "Decibel!" <decibel@decibel.org>,  <pgsql-performance@postgresql.org>
Subject: Re: Linux mis-reporting memory
In-Reply-To: <16213.1190329934@sss.pgh.pa.us> (Tom Lane's message of "Thu\,
	20 Sep 2007 19\:12\:14 -0400")
User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.1 (gnu/linux)
References: <20070920230400.GM95343@decibel.org>
	<16213.1190329934@sss.pgh.pa.us>
From: Gregory Stark <stark@enterprisedb.com>
Organization: EnterpriseDB
Date: Fri, 21 Sep 2007 09:03:04 +0100
Message-ID: <87r6ksa1jb.fsf@oxford.xeocode.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

"Tom Lane" <tgl@sss.pgh.pa.us> writes:

> Decibel! <decibel@decibel.org> writes:
>> I'm finding this rather interesting report from top on a Debian box...
>
>> Mem:  32945280k total, 32871832k used,    73448k free,   247432k buffers
>> Swap:  1951888k total,    42308k used,  1909580k free, 30294300k cached
>
>> So how is it that linux thinks that 30G is cached?
>
> Why would you think that a number reported by the operating system has
> something to do with Postgres' shared memory?

I think his question is how can the kernel be using 30G for kernel buffers if
it only has 32G total and 8G of that is taken up by Postgres's shared buffers.

It seems to imply Linux is paging out sysV shared memory. In fact some of
Heikki's tests here showed that Linux would do precisely that.

If your working set really is smaller than shared buffers then that's not so
bad. Those buffers really would be completely idle anyways.

But if your working set is larger than shared buffers and you're just not
thrashing it hard enough to keep it in RAM then it's really bad. The buffer
Linux will choose to page out are precisely those that Postgres will likely
choose shortly as victim buffers, forcing Linux to page them back in just so
Postgres can overwrite them.


-- 
  Gregory Stark
  EnterpriseDB          http://www.enterprisedb.com