From: Jerry Champlin <jchamplin@absolute-performance.com>
To: 'Steve Atkins' <steve@blighty.com>
To: 'PostgreSQL Performance' <pgsql-performance@postgresql.org>
Subject: Re: select on 22 GB table causes "An I/O error occured while sending to the backend." exception
Date: Thu, 28 Aug 2008 09:47:19 -0600
Message-ID: <00a101c90925$5ac15770$10440650$@com> (raw)
In-Reply-To: <EADBB336-A14E-492C-A8CA-105C8F77A43A@blighty.com>
References: <BAY124-W57D3654AF1159CAF8EAC19F5660@phx.gbl>
<82bpze3ou8.fsf@mid.bfk.de>
<alpine.DEB.1.10.0808271436210.907@asgard.lang.hm>
<alpine.DEB.1.10.0808281411220.4454@aragorn.flymine.org>
<EADBB336-A14E-492C-A8CA-105C8F77A43A@blighty.com>
Another approach we used successfully for a similar problem -- (we had lots
of free high memory but were running out of low memory; oom killer wiped out
MQ a couple times and postmaster a couple times) -- was to change the
settings for how aggressively the virtual memory system protected low memory
by changing /proc/sys/vm/lowmem_reserve_ratio (2.6.18?+ Kernel). I don't
remember all of the details, but we looked at
Documentation/filesystems/proc.txt for the 2.6.25 kernel (it wasn't
documented for earlier kernel releases) to figure out how it worked and set
it appropriate to our system memory configuration.
-Jerry
-----Original Message-----
From: pgsql-performance-owner@postgresql.org
[mailto:pgsql-performance-owner@postgresql.org] On Behalf Of Steve Atkins
Sent: Thursday, August 28, 2008 9:06 AM
To: PostgreSQL Performance
Subject: Re: [PERFORM] select on 22 GB table causes "An I/O error occured
while sending to the backend." exception
On Aug 28, 2008, at 6:26 AM, Matthew Wakeling wrote:
> On Wed, 27 Aug 2008, david@lang.hm wrote:
>> if memory overcommit is disabled, the kernel checks to see if you
>> have an extra 1G of ram available, if you do it allows the process
>> to continue, if you don't it tries to free memory (by throwing away
>> cache, swapping to disk, etc), and if it can't free the memory will
>> return a memroy allocation error (which I believe will cause
>> firefox to exit).
>
> Remember that the memory overcommit check is checking against the
> amount of RAM + swap you have - not just the amount of RAM. When a
> fork occurs, hardly any extra actual RAM is used (due to copy on
> write), but the potential is there for the process to use it. If
> overcommit is switched off, then you just need to make sure there is
> *plenty* of swap to convince the kernel that it can actually fulfil
> all of the memory requests if all the processes behave badly and all
> shared pages become unshared. Then the consequences of processes
> actually using that memory are that the machine will swap, rather
> than the OOM killer having to act.
>
> Of course, it's generally bad to run a machine with more going on
> than will fit in RAM.
>
> Neither swapping nor OOM killing are particularly good - it's just a
> consequence of the amount of memory needed being unpredictable.
>
> Probably the best solution is to just tell the kernel somehow to
> never kill the postmaster.
Or configure adequate swap space?
Cheers,
Steve
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: jchamplin@absolute-performance.com, steve@blighty.com
Subject: Re: select on 22 GB table causes "An I/O error occured while sending to the backend." exception
In-Reply-To: <00a101c90925$5ac15770$10440650$@com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox