pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Alex Turner <armtuk@gmail.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-admin@postgresql.org, Alex Turner <aturner@neteconomist.com>
Subject: Re: IO Timeout
Date: Thu, 10 Mar 2005 21:19:19 -0500
Message-ID: <33c6269f05031018197ae6206b@mail.gmail.com> (raw)
In-Reply-To: <27055.1110499469@sss.pgh.pa.us>
References: <33c6269f050310111892ae175@mail.gmail.com>
	<27055.1110499469@sss.pgh.pa.us>

Well - I am sort of trying to piece together exactly what happened.

Here's what I know.

Around 02:52 I get messages in my syslog stating that there were
problems writing to a controler channel:
Mar 10 02:52:29 tsunami kernel: 3w-9xxx: scsi1: WARNING:
(0x06:0x002C): Unit #1: Command (0x28) timed out, resetting card.
Mar 10 02:52:29 tsunami kernel: 3w-9xxx: scsi1: ERROR: (0x06:0x001F):
Microcontroller not ready during reset sequence.
Mar 10 02:52:29 tsunami kernel: 3w-9xxx: scsi1: AEN: INFO
(0x04:0x005E): <NULL>:unit=0.
...
Mar 10 02:58:41 tsunami kernel: 3w-9xxx: scsi1: ERROR: (0x03:0x0107):
Duplicate request ID:RequestID=23.
Mar 10 02:58:41 tsunami kernel: end_request: I/O error, dev sdd,
sector 282528903
Mar 10 02:58:41 tsunami kernel: 3w-9xxx: scsi1: ERROR: (0x03:0x0107):
Duplicate request ID:RequestID=23.
Mar 10 02:58:41 tsunami kernel: end_request: I/O error, dev sdd,
sector 282528903
Mar 10 02:58:41 tsunami kernel: 3w-9xxx: scsi1: ERROR: (0x03:0x0107):
Duplicate request ID:RequestID=23.
Mar 10 02:58:41 tsunami kernel: end_request: I/O error, dev sdc, sector 28837057


At around 07:30 all connections were failing giving the error:
InternalError: FATAL:  the database system is in recovery mode
(pygresql - similar error in PHP also)

I reboot the server, and one of the discs comes up as innaccesible
(it's part of a RAID 10), but other than that, everything restarts as
normal.

Nothing significant in /var/log/pg_log which is where I have it
logging to (my log leve is pretty low though).

Alex Turner
netEconomist


On Thu, 10 Mar 2005 19:04:29 -0500, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Alex Turner <armtuk@gmail.com> writes:
> > I have a question about IO timeouts:
> > We are using the 3ware escalade 9500S series of cards, and we had a
> > drive failure this morning.  Apparnetly the card waits 30 seconds for
> > the drive to respond, and if it doesn't, it put's the drive in a fail
> > state.  Postgres it seems didn't wait 30 seconds before it decided
> > that the system was upset, and put the database in maintainence mode.
> 
> > Is there a way to increase to IO wait timeout so this doesn't happen?
> 
> Postgres hasn't got any "IO timeouts".  Your concern would be better
> directed to whatever kernel you're using; any sort of timeout on a disk
> operation would be happening at the kernel level.
> 
> For that matter, Postgres hasn't got any concept of "putting the
> database in maintainence mode", so you haven't described what happened
> very accurately at all.
> 
>                         regards, tom lane
>



view thread (7+ messages)  latest in thread

Message-ID: <33c6269f05031018197ae6206b@mail.gmail.com>
Permalink:  ../33c6269f05031018197ae6206b@mail.gmail.com/
Also on:    postgresql.org/message-id/33c6269f05031018197ae6206b@mail.gmail.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-admin@postgresql.org
  Cc: armtuk@gmail.com, tgl@sss.pgh.pa.us, aturner@neteconomist.com
  Subject: Re: IO Timeout
  In-Reply-To: <33c6269f05031018197ae6206b@mail.gmail.com>
  References: <33c6269f050310111892ae175@mail.gmail.com> <27055.1110499469@sss.pgh.pa.us> <33c6269f05031018197ae6206b@mail.gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox