From: Craig James <craig_james@emolecules.com>
To: James Mansion <james@mansionfamily.plus.com>
Cc: david@lang.hm, Florian Weimer <fweimer@bfk.de>
Cc: henk de wit <henk53602@hotmail.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: select on 22 GB table causes "An I/O error occured while sending to the backend." exception
Date: Fri, 29 Aug 2008 08:25:51 -0700
Message-ID: <48B814FF.3000600@emolecules.com> (raw)
In-Reply-To: <48B78662.5010208@mansionfamily.plus.com>
References: <BAY124-W57D3654AF1159CAF8EAC19F5660@phx.gbl>
<82bpze3ou8.fsf@mid.bfk.de>
<alpine.DEB.1.10.0808271436210.907@asgard.lang.hm>
<48B78662.5010208@mansionfamily.plus.com>
James Mansion wrote:
> I can't see how an OS can lie to processes about memory being allocated
> to them and not be ridiculed as a toy, but there you go. I don't think
> Linux is the only perpetrator - doesn't AIX do this too?
This is a leftover from the days of massive physical modeling (chemistry, physics, astronomy, ...) programs written in FORTRAN. Since FORTRAN didn't have pointers, scientists would allocate massive three-dimensional arrays, and their code might only access a tiny fraction of the memory. The operating-system vendors, particularly SGI, added features to the various flavors of UNIX, including the ability to overcommit memory, to support these FORTRAN programs, which at the time were some of the most important applications driving computer science and computer architectures of workstation-class computers.
When these workstation-class computers evolved enough to rival mainframes, companies started shifting apps like Oracle onto the cheaper workstation-class computers. Unfortunately, the legacy of the days of these FORTRAN programs is still with us, and every few years we have to go through this discussion again.
Disable overcommitted memory. There is NO REASON to use it on any modern server-class computer, and MANY REASONS WHY IT IS A BAD IDEA.
Craig
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: craig_james@emolecules.com, james@mansionfamily.plus.com, fweimer@bfk.de, henk53602@hotmail.com
Subject: Re: select on 22 GB table causes "An I/O error occured while sending to the backend." exception
In-Reply-To: <48B814FF.3000600@emolecules.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox