To: David Schnur <dnschnur@gmail.com>
cc: pgsql-admin@postgresql.org
Subject: Re: Shutdown fails with both 'fast' and 'immediate' 
In-reply-to: <AANLkTilzcY_VGBiYdTJ2bomtECxJxipSyW4AcpLMsO76@mail.gmail.com> 
References: <AANLkTindCeLyRmRbgV4ICxEBGRLwyBbm5yO2oLBHUnSc@mail.gmail.com>
	<4652.1273674753@sss.pgh.pa.us>
	<AANLkTilzcY_VGBiYdTJ2bomtECxJxipSyW4AcpLMsO76@mail.gmail.com>
Comments: In-reply-to David Schnur <dnschnur@gmail.com>
	message dated "Wed, 12 May 2010 13:03:57 -0400"
Date: Wed, 12 May 2010 13:16:13 -0400
Message-ID: <22516.1273684573@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

David Schnur <dnschnur@gmail.com> writes:
> I'm less concerned with the particular query than with the general question
> of when a shutdown could hang like this.  I expected this to be possible
> when using -m fast, but my understanding was that -m immediate really forced
> termination.

Yeah, it's supposed to.  The sequence is pg_ctl -m immediate sends
SIGQUIT to the postmaster, which in turn sends SIGQUIT to all its child
processes, and their SIGQUIT interrupt handlers just immediately exit().
I was thinking earlier that there might be a bug in the postmaster state
machine that prevented it from sending SIGQUIT if it had already
received SIGTERM (-m fast), but a look at the sources doesn't support
that theory.  The only obvious theory at this point is that the backend
is stuck in some uninterruptable kernel call, but it's hard to imagine
what.

Is the postmaster still there after -m immediate, or does it quit?
If it's still there, maybe there's some problem in the earlier part
of the sequence.

A gdb stack trace from whichever processes are still there after -m
immediate could be informative.  Another thing you could try is a
manual "kill -QUIT pid" on the uncooperative backend(s).

			regards, tom lane