To: "Burgholzer, Robert (DEQ)" <Robert.Burgholzer@deq.virginia.gov>
cc: pgsql-admin@postgresql.org
Subject: Re: diagnosing a db crash - server exit code 2 
In-reply-to: 
 <B6C40D17104BEF47B1CB85623CDFAC6357BE26@COVMSGCES-EMB13.cov.virginia.gov>
References: 
 <B6C40D17104BEF47B1CB85623CDFAC6357BE26@COVMSGCES-EMB13.cov.virginia.gov>
Comments: In-reply-to "Burgholzer,
	Robert (DEQ)" <Robert.Burgholzer@deq.virginia.gov>
	message dated "Fri, 23 Sep 2011 15:02:49 -0400"
Date: Sun, 25 Sep 2011 19:54:21 -0400
Message-ID: <10606.1316994861@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

"Burgholzer, Robert (DEQ)" <Robert.Burgholzer@deq.virginia.gov> writes:
> I am trying to get my head around why I keep getting crashes to my PG
> 8.3.7 database on CentOS - Linux version 2.6.18-164.el5.
> LOG:  server process (PID 5978) exited with exit code 2

Just like it says, this implies that a backend process exited with
"exit(2)".  A quick grep through the 8.3 sources says that the only such
call in the backend code is in the SIGQUIT signal handler.  Now it's
possible that what you're seeing is an exit(2) somewhere in the R
library rather than in Postgres code.  But I think it's more likely that
some external source is SIGQUIT'ing the backend.  Joe suggested an OOM
kill, but I've never heard of the OOM code using anything but SIGKILL
(signal 9).  Have you got any other software that runs as root and
thinks it's licensed to kill processes in the name of
something-or-other?

Anyway, I concur with the plan of getting a stack trace to determine
where the exit call is, so that we can positively eliminate (or not)
the R library.

			regards, tom lane