From: Andrew Biagioni <andrew.biagioni@e-greek.net>
To: scott.marlowe <scott.marlowe@ihs.com>
Cc: pgsql-admin@postgresql.org
Subject: Re: Spontaneous PostgreSQL Server Reboot?
Date: Wed, 31 Mar 2004 23:19:26 -0500
Message-ID: <406B984E.7030109@e-greek.net> (raw)
In-Reply-To: <Pine.LNX.4.33.0403310854320.26243-100000@css120.ihs.com>
References: <Pine.LNX.4.33.0403310854320.26243-100000@css120.ihs.com>
scott.marlowe wrote:
> On Tue, 30 Mar 2004, Andrew Biagioni wrote:
>
>
>>Alex,
>>
>>the answer is "no" to all of these. We are a tiny start-up (2 guys, and
>>we do our own cleaning); ambient temperature varies significantly but
>>is not related to the failure, and one machine starts beeping when it
>>gets too hot (then we added an extra case fan); no fancy watchdogs
>>(maybe someday... One can only dream :-> ); three different cases,
>>power supplies, motherboards, etc., etc. (one power supply is
>>extra-large, and that's the machine that started failing first!).
>>
>>We originally blamed the problem on hardware failure (first machine);
>>then on OS version/configuration (second machine); now we're out of
>>things to blame, except maybe unusually bad luck...
>
>
> What did memtest86 say?
>
> Did the same person build all the machines? I've seen plenty of folks
> build machines and zap the memory when installing it. >95% of all ESD
> failures are partial / delayed failures, so just because a computer boots
> up doesn't mean proper ESD procedures were followed, and if not, and if
> you're in a dry environment like I am (I live in Denver) then it's quite
> possible all three have bad CPU/mobo/memory or something like that.
Two different people built the machines; we're both electrical
engineers with plenty of familiarity and experience with static issues,
so that particular issue is not likely.
As for memtest86 - I haven't been able to run it on two of the machines
yet (they are in production), and I have to restart the third one (it
was "retired" after the third time it died on us).
Meanwhile I found out some more details:
- the first machine had a software raid system that may have been unreliable
- the second machine had a much older kernel and sloppily-updated
modules, and it would hang -- not reboot
- the last machine to reboot MAY have been a line power issue (the whole
building lost power a few hours later, so I lost some info on other
machines' restarting -- I'll dig more).
So -- it's memtest86 and badblocks for all three (as soon as I can),
better UPS-ing, updated kernel(s), and checking more machines' logs;
then we'll see...
Thanks to you all for the suggestions -- keep them coming!
Andrew
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-admin@postgresql.org
Cc: andrew.biagioni@e-greek.net, scott.marlowe@ihs.com
Subject: Re: Spontaneous PostgreSQL Server Reboot?
In-Reply-To: <406B984E.7030109@e-greek.net>
References: <Pine.LNX.4.33.0403310854320.26243-100000@css120.ihs.com> <406B984E.7030109@e-greek.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox