Received: from localhost (unknown [200.46.204.183]) by postgresql.org (Postfix) with ESMTP id 8BA0C64FEF9 for ; Thu, 28 Aug 2008 12:47:32 -0300 (ADT) Received: from postgresql.org ([200.46.204.86]) by localhost (mx1.hub.org [200.46.204.183]) (amavisd-maia, port 10024) with ESMTP id 75978-07 for ; Thu, 28 Aug 2008 12:47:19 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from fort-mx02.den.api.local (unknown [66.35.32.228]) by postgresql.org (Postfix) with ESMTP id 90DE264FFAA for ; Thu, 28 Aug 2008 12:47:17 -0300 (ADT) Received: from jgcd430 ([192.168.10.138]) by fort-mx02.den.api.local (8.13.1/8.13.1) with ESMTP id m7SFlD72003998; Thu, 28 Aug 2008 09:47:13 -0600 From: "Jerry Champlin" To: "'Steve Atkins'" , "'PostgreSQL Performance'" References: <82bpze3ou8.fsf@mid.bfk.de> In-Reply-To: Subject: Re: select on 22 GB table causes "An I/O error occured while sending to the backend." exception Date: Thu, 28 Aug 2008 09:47:19 -0600 Message-ID: <00a101c90925$5ac15770$10440650$@com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AckJH8m8qD+mkOwgQM6+RQHxpMLXRgABGEQQ Content-Language: en-us X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=0.1 tagged_above=0 required=5 tests=RDNS_NONE=0.1 X-Spam-Level: X-Archive-Number: 200808/373 X-Sequence-Number: 31254 Another approach we used successfully for a similar problem -- (we had lots of free high memory but were running out of low memory; oom killer wiped out MQ a couple times and postmaster a couple times) -- was to change the settings for how aggressively the virtual memory system protected low memory by changing /proc/sys/vm/lowmem_reserve_ratio (2.6.18?+ Kernel). I don't remember all of the details, but we looked at Documentation/filesystems/proc.txt for the 2.6.25 kernel (it wasn't documented for earlier kernel releases) to figure out how it worked and set it appropriate to our system memory configuration. -Jerry -----Original Message----- From: pgsql-performance-owner@postgresql.org [mailto:pgsql-performance-owner@postgresql.org] On Behalf Of Steve Atkins Sent: Thursday, August 28, 2008 9:06 AM To: PostgreSQL Performance Subject: Re: [PERFORM] select on 22 GB table causes "An I/O error occured while sending to the backend." exception On Aug 28, 2008, at 6:26 AM, Matthew Wakeling wrote: > On Wed, 27 Aug 2008, david@lang.hm wrote: >> if memory overcommit is disabled, the kernel checks to see if you >> have an extra 1G of ram available, if you do it allows the process >> to continue, if you don't it tries to free memory (by throwing away >> cache, swapping to disk, etc), and if it can't free the memory will >> return a memroy allocation error (which I believe will cause >> firefox to exit). > > Remember that the memory overcommit check is checking against the > amount of RAM + swap you have - not just the amount of RAM. When a > fork occurs, hardly any extra actual RAM is used (due to copy on > write), but the potential is there for the process to use it. If > overcommit is switched off, then you just need to make sure there is > *plenty* of swap to convince the kernel that it can actually fulfil > all of the memory requests if all the processes behave badly and all > shared pages become unshared. Then the consequences of processes > actually using that memory are that the machine will swap, rather > than the OOM killer having to act. > > Of course, it's generally bad to run a machine with more going on > than will fit in RAM. > > Neither swapping nor OOM killing are particularly good - it's just a > consequence of the amount of memory needed being unpredictable. > > Probably the best solution is to just tell the kernel somehow to > never kill the postmaster. Or configure adequate swap space? Cheers, Steve -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance