Message-ID: <48B78662.5010208@mansionfamily.plus.com>
Date: Fri, 29 Aug 2008 06:17:22 +0100
From: James Mansion <james@mansionfamily.plus.com>
Organization: MsgWare
User-Agent: Thunderbird 2.0.0.16 (Windows/20080708)
MIME-Version: 1.0
To: david@lang.hm
CC: Florian Weimer <fweimer@bfk.de>, henk de wit <henk53602@hotmail.com>,
	pgsql-performance@postgresql.org
Subject: Re: select on 22 GB table causes "An I/O error occured
	while sending to the backend." exception
References: <BAY124-W57D3654AF1159CAF8EAC19F5660@phx.gbl>
	<82bpze3ou8.fsf@mid.bfk.de>
	<alpine.DEB.1.10.0808271436210.907@asgard.lang.hm>
In-Reply-To: <alpine.DEB.1.10.0808271436210.907@asgard.lang.hm>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

david@lang.hm wrote:
> for example if you have a process that uses 1G of ram (say firefox) 
> and it needs to start a new process (say acroread to handle a pdf 
> file), what it does is it forks the firefox process (each of which 
> have 1G of ram allocated), and then does an exec of the acroread 
> process (releasing the 1G of ram previously held by that copy of the 
> firefox process)
>
Indeed, which is why we have vfork.  And, OK, vfork is busted if you 
have a threaded environment, so we have posix_spawn and posix_spawnp.

It is also worth noting that the copy isn't really a full copy on any 
decent modern UNIX - it is a reservation against the total swap space 
available.  Most pages will be happilly shared copy-on-write and never 
fully copied to the child before the exec.

I can't see how an OS can lie to processes about memory being allocated 
to them and not be ridiculed as a toy, but there you go.  I don't think 
Linux is the only perpetrator - doesn't AIX do this too?

The 'bests trategy' for the OOM killer is not to have one, and accept 
that you need some swap space available (it doesn't have to be fast 
since it won't actually be touched) to help out when fork/exec happens 
in big process images.

James