From: James Mansion <james@mansionfamily.plus.com>
To: david@lang.hm
Cc: Florian Weimer <fweimer@bfk.de>
Cc: henk de wit <henk53602@hotmail.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: select on 22 GB table causes "An I/O error occured while sending to the backend." exception
Date: Fri, 29 Aug 2008 06:17:22 +0100
Message-ID: <48B78662.5010208@mansionfamily.plus.com> (raw)
In-Reply-To: <alpine.DEB.1.10.0808271436210.907@asgard.lang.hm>
References: <BAY124-W57D3654AF1159CAF8EAC19F5660@phx.gbl>
<82bpze3ou8.fsf@mid.bfk.de>
<alpine.DEB.1.10.0808271436210.907@asgard.lang.hm>
david@lang.hm wrote:
> for example if you have a process that uses 1G of ram (say firefox)
> and it needs to start a new process (say acroread to handle a pdf
> file), what it does is it forks the firefox process (each of which
> have 1G of ram allocated), and then does an exec of the acroread
> process (releasing the 1G of ram previously held by that copy of the
> firefox process)
>
Indeed, which is why we have vfork. And, OK, vfork is busted if you
have a threaded environment, so we have posix_spawn and posix_spawnp.
It is also worth noting that the copy isn't really a full copy on any
decent modern UNIX - it is a reservation against the total swap space
available. Most pages will be happilly shared copy-on-write and never
fully copied to the child before the exec.
I can't see how an OS can lie to processes about memory being allocated
to them and not be ridiculed as a toy, but there you go. I don't think
Linux is the only perpetrator - doesn't AIX do this too?
The 'bests trategy' for the OOM killer is not to have one, and accept
that you need some swap space available (it doesn't have to be fast
since it won't actually be touched) to help out when fork/exec happens
in big process images.
James
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: james@mansionfamily.plus.com, david@lang.hm, fweimer@bfk.de, henk53602@hotmail.com
Subject: Re: select on 22 GB table causes "An I/O error occured while sending to the backend." exception
In-Reply-To: <48B78662.5010208@mansionfamily.plus.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox