Date: Thu, 28 Aug 2008 09:51:09 -0700 (PDT)
From: david@lang.hm
To: Matthew Wakeling <matthew@flymine.org>
cc: pgsql-performance@postgresql.org
Subject: Re: select on 22 GB table causes "An I/O error occured
	while sending to the backend." exception
In-Reply-To: <alpine.DEB.1.10.0808281411220.4454@aragorn.flymine.org>
Message-ID: <alpine.DEB.1.10.0808280949060.2713@asgard.lang.hm>
References: <BAY124-W57D3654AF1159CAF8EAC19F5660@phx.gbl>
	<82bpze3ou8.fsf@mid.bfk.de>
	<alpine.DEB.1.10.0808271436210.907@asgard.lang.hm>
	<alpine.DEB.1.10.0808281411220.4454@aragorn.flymine.org>
User-Agent: Alpine 1.10 (DEB 962 2008-03-14)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed

On Thu, 28 Aug 2008, Matthew Wakeling wrote:

> On Wed, 27 Aug 2008, david@lang.hm wrote:
>> if memory overcommit is disabled, the kernel checks to see if you have an 
>> extra 1G of ram available, if you do it allows the process to continue, if 
>> you don't it tries to free memory (by throwing away cache, swapping to 
>> disk, etc), and if it can't free the memory will return a memroy allocation 
>> error (which I believe will cause firefox to exit).
>
> Remember that the memory overcommit check is checking against the amount of 
> RAM + swap you have - not just the amount of RAM. When a fork occurs, hardly 
> any extra actual RAM is used (due to copy on write), but the potential is 
> there for the process to use it. If overcommit is switched off, then you just 
> need to make sure there is *plenty* of swap to convince the kernel that it 
> can actually fulfil all of the memory requests if all the processes behave 
> badly and all shared pages become unshared. Then the consequences of 
> processes actually using that memory are that the machine will swap, rather 
> than the OOM killer having to act.

if you are correct that it just checks against memory+swap then it's not a 
big deal, but I don't think it does that. I think it actually allocates 
the memory, and if it does that it will push things out of ram to do the 
allocation, I don't believe that it will allocate swap space directly.

David Lang

> Of course, it's generally bad to run a machine with more going on than will 
> fit in RAM.
>
> Neither swapping nor OOM killing are particularly good - it's just a 
> consequence of the amount of memory needed being unpredictable.
>
> Probably the best solution is to just tell the kernel somehow to never kill 
> the postmaster.
>
> Matthew
>
>