From: Alvaro Herrera <alvherre@commandprompt.com>
To: david@lang.hm
Cc: Scott Marlowe <scott.marlowe@gmail.com>
Cc: Matthew Dennis <mdennis@merfer.net>
Cc: Matthew Wakeling <matthew@flymine.org>
Cc: PostgreSQL Performance <pgsql-performance@postgresql.org>
Subject: Re: select on 22 GB table causes "An I/O error occured while sending to the backend." exception
Date: Fri, 29 Aug 2008 09:48:06 -0400
Message-ID: <20080829134806.GE3983@alvh.no-ip.org> (raw)
In-Reply-To: <alpine.DEB.1.10.0808281955500.2713@asgard.lang.hm>
References: <48B6EA7E.2090707@emolecules.com>
<alpine.DEB.1.10.0808282110140.4454@aragorn.flymine.org>
<dcc563d10808281542u383808bci4874713a31bd1cc6@mail.gmail.com>
<alpine.DEB.1.10.0808281602460.2713@asgard.lang.hm>
<dcc563d10808281811t5c98dcaexd98c5cfe679f428d@mail.gmail.com>
<e94d85500808281853g22f67274pc9b4c5e3b30bb05a@mail.gmail.com>
<dcc563d10808281901w6d449f52t50e5da7938f1fb25@mail.gmail.com>
<alpine.DEB.1.10.0808281936090.2713@asgard.lang.hm>
<20080829024939.GN8424@alvh.no-ip.org>
<alpine.DEB.1.10.0808281955500.2713@asgard.lang.hm>
david@lang.hm escribió:
> On Thu, 28 Aug 2008, Alvaro Herrera wrote:
>
>> david@lang.hm escribi?:
>>> On Thu, 28 Aug 2008, Scott Marlowe wrote:
>>
>>>> scenario 1: There's a postmaster, it owns all the child processes.
>>>> It gets killed. The Postmaster gets restarted. Since there isn't one
>>>
>>> when the postmaster gets killed doesn't that kill all it's children as
>>> well?
>>
>> Of course not. The postmaster gets a SIGKILL, which is instant death.
>> There's no way to signal the children. If they were killed too then
>> this wouldn't be much of a problem.
>
> I'm not saying that it would signal it's children, I thought that the OS
> killed children (unless steps were taken to allow them to re-parent)
Oh, you were mistaken then.
>>> well, if you aren't going through the postmaster, what process is
>>> recieving network messages? it can't be a group of processes, only one
>>> can be listening to a socket at one time.
>>
>> Huh? Each backend has its own socket.
>
> we must be talking about different things. I'm talking about the socket
> that would be used for clients to talk to postgres, this is either a TCP
> socket or a unix socket. in either case only one process can listen on
> it.
Obviously only one process (the postmaster) can call listen() on a given
TCP address/port. Once connected, the socket is passed to the
backend, and the postmaster is no longer involved in the communication
between backend and client. Each backend has its own socket. If the
postmaster dies, the established communication is still alive.
>>> and if the postmaster isn't needed for the child processes to write to
>>> the datastore, how are multiple child processes prevented from writing to
>>> the datastore normally? and why doesn't that mechanism continue to work?
>>
>> They use locks. Those locks are implemented using shared memory. If a
>> new postmaster starts, it gets a new shared memory, and a new set of
>> locks, that do not conflict with the ones already held by the first gang
>> of backends. This is what causes the corruption.
>
> so the new postmaster needs to detect that there is a shared memory
> segment out that used by backends for this database.
> this doesn't sound that hard,
You're welcome to suggest actual improvements to our interlocking
system, after you've read the current code and understood its rationale.
>> Any other signal gives it the chance to signal the children before
>> dying.
>
> are you sure that it's not going to die from a memory allocation error?
> or any other similar type of error without _always_ killing the children?
I am sure. There are no memory allocations in that code. It is
carefully written with that one purpose.
There may be bugs, but that's another matter. This code was written
eons ago and has proven very healthy.
--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-performance@postgresql.org
Cc: alvherre@commandprompt.com, david@lang.hm, scott.marlowe@gmail.com, mdennis@merfer.net, matthew@flymine.org
Subject: Re: select on 22 GB table causes "An I/O error occured while sending to the backend." exception
In-Reply-To: <20080829134806.GE3983@alvh.no-ip.org>
References: <48B6EA7E.2090707@emolecules.com> <alpine.DEB.1.10.0808282110140.4454@aragorn.flymine.org> <dcc563d10808281542u383808bci4874713a31bd1cc6@mail.gmail.com> <alpine.DEB.1.10.0808281602460.2713@asgard.lang.hm> <dcc563d10808281811t5c98dcaexd98c5cfe679f428d@mail.gmail.com> <e94d85500808281853g22f67274pc9b4c5e3b30bb05a@mail.gmail.com> <dcc563d10808281901w6d449f52t50e5da7938f1fb25@mail.gmail.com> <alpine.DEB.1.10.0808281936090.2713@asgard.lang.hm> <20080829024939.GN8424@alvh.no-ip.org> <alpine.DEB.1.10.0808281955500.2713@asgard.lang.hm> <20080829134806.GE3983@alvh.no-ip.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox