pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: warren little <warren.little@meridiascapital.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-admin@postgresql.org
Subject: Re: Emergency - Need assistance
Date: Mon, 02 Jan 2006 15:26:12 -0700
Message-ID: <1136240773.16094.43.camel@sharp> (raw)
In-Reply-To: <1136240683.16094.40.camel@sharp>
References: <1136228730.16094.12.camel@sharp>
	<11864.1136232604@sss.pgh.pa.us>
	<1136233928.16094.33.camel@sharp>
	<12399.1136237657@sss.pgh.pa.us>
	<1136240683.16094.40.camel@sharp>

Sorry,
forget the attachment.

On Mon, 2006-01-02 at 15:24 -0700, warren little wrote:
> The dump/restore failed even with the zero_damaged_pages=true.
> The the logfile (postgresql-2006-01-02_130023.log)
> did not have much in the way of useful info. I've attached the section
> of the logfile around the time of the crash.  I cannot find any sign of
> a core file.  Where might the core dump have landed?
> 
> Regarding your comments about losing the evidence, the data I'm trying
> to load is in another database in the same cluster which I have no
> intention of purging until a can get the table moved to the new
> database.
> 
> thanks
> 
> 
> 
> 
> On Mon, 2006-01-02 at 16:34 -0500, Tom Lane wrote:
> > warren little <warren.little@meridiascapital.com> writes:
> > >  pg_dump: SQL command failed
> > > pg_dump: Error message from server: server closed the connection
> > > unexpectedly
> > >         This probably means the server terminated abnormally
> > >         before or while processing the request.
> > > pg_dump: The command was: FETCH 100 FROM _pg_dump_cursor
> > 
> > Hmm.  This could mean corrupted data files, but it's hard to be sure
> > without more info.
> > 
> > > I had removed all the files in pg_log prior to getting this error and no
> > > new logfile was created.  I'm guessing I screwed up the logger when
> > > removing all the files, but I assumed that when writing to the error
> > > logs the backend would create a file if one did not exist.
> > 
> > The file *does* exist, there's just no directory link to it anymore :-(
> > You need to force a logfile rotation, which might be most easily done by
> > stopping and restarting the postmaster.
> > 
> > What you need to do is see the postmaster log entry about the backend
> > crash.  If it's dying on a signal (likely sig11 = SEGV) then inspecting
> > the core file might yield useful information.
> > 
> > > I currently attempt to run the dump/restore with the zero_damaged_pages
> > > turned on to see if the results yield something more useful.  
> > 
> > That really ought to be the last resort not the first one, because it
> > will destroy not only data but most of the evidence about what went
> > wrong...
> > 
> > 			regards, tom lane
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 1: if posting/reading through Usenet, please send an appropriate
>        subscribe-nomail command to majordomo@postgresql.org so that your
>        message can get through to the mailing list cleanly

Attachments:

errlog.txt (text/plain, 3.2K) download @ 2006-01-02 15:02:02 MST:LOG: autovacuum: processing database "tigris" @ 2006-01-02 15:03:01 MST:LOG: server process (PID 28772) was terminated by signal 11 @ 2006-01-02 15:03:01 MST:LOG: terminating any other active server processes [local]@[local] 2006-01-02 15:03:01 MST:WARNING: terminating connection because of crash of another server process [local]@[local] 2006-01-02 15:03:01 MST:DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. [local]@[local] 2006-01-02 15:03:01 MST:HINT: In a moment you should be able to reconnect to the database and repeat your command. 192.168.19.129(50732)@192.168.19.129 2006-01-02 15:03:01 MST:WARNING: terminating connection because of crash of another server process 192.168.19.129(50732)@192.168.19.129 2006-01-02 15:03:01 MST:DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. 192.168.19.129(50732)@192.168.19.129 2006-01-02 15:03:01 MST:HINT: In a moment you should be able to reconnect to the database and repeat your command. 192.168.19.129(50730)@192.168.19.129 2006-01-02 15:03:01 MST:WARNING: terminating connection because of crash of another server process 192.168.19.129(50730)@192.168.19.129 2006-01-02 15:03:01 MST:DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. 192.168.19.129(50730)@192.168.19.129 2006-01-02 15:03:01 MST:HINT: In a moment you should be able to reconnect to the database and repeat your command. 192.168.19.129(50731)@192.168.19.129 2006-01-02 15:03:01 MST:WARNING: terminating connection because of crash of another server process 192.168.19.129(50731)@192.168.19.129 2006-01-02 15:03:01 MST:DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. 192.168.19.129(50731)@192.168.19.129 2006-01-02 15:03:01 MST:HINT: In a moment you should be able to reconnect to the database and repeat your command. @ 2006-01-02 15:03:01 MST:LOG: all server processes terminated; reinitializing @ 2006-01-02 15:03:01 MST:LOG: database system was interrupted at 2006-01-02 15:02:47 MST @ 2006-01-02 15:03:01 MST:LOG: checkpoint record is at 37/D60F93A8 @ 2006-01-02 15:03:01 MST:LOG: redo record is at 37/D6008018; undo record is at 0/0; shutdown FALSE @ 2006-01-02 15:03:01 MST:LOG: next transaction ID: 32196280; next OID: 102041945 @ 2006-01-02 15:03:01 MST:LOG: next MultiXactId: 41; next MultiXactOffset: 93 @ 2006-01-02 15:03:01 MST:LOG: database system was not properly shut down; automatic recovery in progress @ 2006-01-02 15:03:01 MST:LOG: redo starts at 37/D6008018 @ 2006-01-02 15:03:01 MST:LOG: record with zero length at 37/D60F93F8 @ 2006-01-02 15:03:01 MST:LOG: redo done at 37/D60F93A8 @ 2006-01-02 15:03:02 MST:LOG: database system is ready @ 2006-01-02 15:03:02 MST:LOG: transaction ID wrap limit is 1087118600, limited by database "cert"

view thread (7+ messages)  latest in thread

Message-ID: <1136240773.16094.43.camel@sharp>
Permalink:  ../1136240773.16094.43.camel@sharp/
Also on:    postgresql.org/message-id/1136240773.16094.43.camel@sharp

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-admin@postgresql.org
  Cc: warren.little@meridiascapital.com, tgl@sss.pgh.pa.us
  Subject: Re: Emergency - Need assistance
  In-Reply-To: <1136240773.16094.43.camel@sharp>
  References: <1136228730.16094.12.camel@sharp> <11864.1136232604@sss.pgh.pa.us> <1136233928.16094.33.camel@sharp> <12399.1136237657@sss.pgh.pa.us> <1136240683.16094.40.camel@sharp> <1136240773.16094.43.camel@sharp>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox