From: david@lang.hm
To: Matthew Wakeling <matthew@flymine.org>
Cc: pgsql-performance@postgresql.org
Subject: Re: select on 22 GB table causes "An I/O error occured while sending to the backend." exception
Date: Thu, 28 Aug 2008 09:51:09 -0700 (PDT)
Message-ID: <alpine.DEB.1.10.0808280949060.2713@asgard.lang.hm> (raw)
In-Reply-To: <alpine.DEB.1.10.0808281411220.4454@aragorn.flymine.org>
References: <BAY124-W57D3654AF1159CAF8EAC19F5660@phx.gbl>
<82bpze3ou8.fsf@mid.bfk.de>
<alpine.DEB.1.10.0808271436210.907@asgard.lang.hm>
<alpine.DEB.1.10.0808281411220.4454@aragorn.flymine.org>
On Thu, 28 Aug 2008, Matthew Wakeling wrote:
> On Wed, 27 Aug 2008, david@lang.hm wrote:
>> if memory overcommit is disabled, the kernel checks to see if you have an
>> extra 1G of ram available, if you do it allows the process to continue, if
>> you don't it tries to free memory (by throwing away cache, swapping to
>> disk, etc), and if it can't free the memory will return a memroy allocation
>> error (which I believe will cause firefox to exit).
>
> Remember that the memory overcommit check is checking against the amount of
> RAM + swap you have - not just the amount of RAM. When a fork occurs, hardly
> any extra actual RAM is used (due to copy on write), but the potential is
> there for the process to use it. If overcommit is switched off, then you just
> need to make sure there is *plenty* of swap to convince the kernel that it
> can actually fulfil all of the memory requests if all the processes behave
> badly and all shared pages become unshared. Then the consequences of
> processes actually using that memory are that the machine will swap, rather
> than the OOM killer having to act.
if you are correct that it just checks against memory+swap then it's not a
big deal, but I don't think it does that. I think it actually allocates
the memory, and if it does that it will push things out of ram to do the
allocation, I don't believe that it will allocate swap space directly.
David Lang
> Of course, it's generally bad to run a machine with more going on than will
> fit in RAM.
>
> Neither swapping nor OOM killing are particularly good - it's just a
> consequence of the amount of memory needed being unpredictable.
>
> Probably the best solution is to just tell the kernel somehow to never kill
> the postmaster.
>
> Matthew
>
>
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: david@lang.hm, matthew@flymine.org
Subject: Re: select on 22 GB table causes "An I/O error occured while sending to the backend." exception
In-Reply-To: <alpine.DEB.1.10.0808280949060.2713@asgard.lang.hm>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox