X-Original-To: pgsql-admin-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by postgresql.org (Postfix) with ESMTP id DBCE19DC805 for ; Mon, 2 Jan 2006 18:26:13 -0400 (AST) Received: from postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 76334-09 for ; Mon, 2 Jan 2006 18:26:15 -0400 (AST) X-Greylist: from auto-whitelisted by SQLgrey- Received: from exchange.meridiascapital.com (exchange.meridiascapital.com [65.90.55.78]) by postgresql.org (Postfix) with ESMTP id 57AFD9DC868 for ; Mon, 2 Jan 2006 18:26:11 -0400 (AST) Received: from 192.168.19.129 ([192.168.19.129]) by atherton-ex-01.meridias.local ([192.168.8.78]) with Microsoft Exchange Server HTTP-DAV ; Mon, 2 Jan 2006 22:26:13 +0000 Received: from sharp by ATHERTON-EX-01; 02 Jan 2006 15:26:13 -0700 Subject: Re: Emergency - Need assistance From: warren little To: Tom Lane Cc: pgsql-admin@postgresql.org In-Reply-To: <1136240683.16094.40.camel@sharp> References: <1136228730.16094.12.camel@sharp> <11864.1136232604@sss.pgh.pa.us> <1136233928.16094.33.camel@sharp> <12399.1136237657@sss.pgh.pa.us> <1136240683.16094.40.camel@sharp> Content-Type: multipart/mixed; boundary="=-PLs+GL3jMplm6JOF/iwc" Date: Mon, 02 Jan 2006 15:26:12 -0700 Message-Id: <1136240773.16094.43.camel@sharp> Mime-Version: 1.0 X-Mailer: Evolution 2.2.2 (2.2.2-5) X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, score=0.121 required=5 tests=[AWL=0.120, UNPARSEABLE_RELAY=0.001] X-Spam-Score: 0.121 X-Spam-Level: X-Archive-Number: 200601/25 X-Sequence-Number: 20113 --=-PLs+GL3jMplm6JOF/iwc Content-Type: text/plain Content-Transfer-Encoding: 7bit Sorry, forget the attachment. On Mon, 2006-01-02 at 15:24 -0700, warren little wrote: > The dump/restore failed even with the zero_damaged_pages=true. > The the logfile (postgresql-2006-01-02_130023.log) > did not have much in the way of useful info. I've attached the section > of the logfile around the time of the crash. I cannot find any sign of > a core file. Where might the core dump have landed? > > Regarding your comments about losing the evidence, the data I'm trying > to load is in another database in the same cluster which I have no > intention of purging until a can get the table moved to the new > database. > > thanks > > > > > On Mon, 2006-01-02 at 16:34 -0500, Tom Lane wrote: > > warren little writes: > > > pg_dump: SQL command failed > > > pg_dump: Error message from server: server closed the connection > > > unexpectedly > > > This probably means the server terminated abnormally > > > before or while processing the request. > > > pg_dump: The command was: FETCH 100 FROM _pg_dump_cursor > > > > Hmm. This could mean corrupted data files, but it's hard to be sure > > without more info. > > > > > I had removed all the files in pg_log prior to getting this error and no > > > new logfile was created. I'm guessing I screwed up the logger when > > > removing all the files, but I assumed that when writing to the error > > > logs the backend would create a file if one did not exist. > > > > The file *does* exist, there's just no directory link to it anymore :-( > > You need to force a logfile rotation, which might be most easily done by > > stopping and restarting the postmaster. > > > > What you need to do is see the postmaster log entry about the backend > > crash. If it's dying on a signal (likely sig11 = SEGV) then inspecting > > the core file might yield useful information. > > > > > I currently attempt to run the dump/restore with the zero_damaged_pages > > > turned on to see if the results yield something more useful. > > > > That really ought to be the last resort not the first one, because it > > will destroy not only data but most of the evidence about what went > > wrong... > > > > regards, tom lane > > ---------------------------(end of broadcast)--------------------------- > TIP 1: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to majordomo@postgresql.org so that your > message can get through to the mailing list cleanly --=-PLs+GL3jMplm6JOF/iwc Content-Disposition: attachment; filename=errlog.txt Content-Transfer-Encoding: base64 Content-Type: text/plain; name=errlog.txt; charset=utf-8 QCAyMDA2LTAxLTAyIDE1OjAyOjAyIE1TVDpMT0c6ICBhdXRvdmFjdXVtOiBwcm9jZXNzaW5nIGRh dGFiYXNlICJ0aWdyaXMiDQpAIDIwMDYtMDEtMDIgMTU6MDM6MDEgTVNUOkxPRzogIHNlcnZlciBw cm9jZXNzIChQSUQgMjg3NzIpIHdhcyB0ZXJtaW5hdGVkIGJ5IHNpZ25hbCAxMQ0KQCAyMDA2LTAx LTAyIDE1OjAzOjAxIE1TVDpMT0c6ICB0ZXJtaW5hdGluZyBhbnkgb3RoZXIgYWN0aXZlIHNlcnZl ciBwcm9jZXNzZXMNCltsb2NhbF1AW2xvY2FsXSAyMDA2LTAxLTAyIDE1OjAzOjAxIE1TVDpXQVJO SU5HOiAgdGVybWluYXRpbmcgY29ubmVjdGlvbiBiZWNhdXNlIG9mIGNyYXNoIG9mIGFub3RoZXIg c2VydmVyIHByb2Nlc3MNCltsb2NhbF1AW2xvY2FsXSAyMDA2LTAxLTAyIDE1OjAzOjAxIE1TVDpE RVRBSUw6ICBUaGUgcG9zdG1hc3RlciBoYXMgY29tbWFuZGVkIHRoaXMgc2VydmVyIHByb2Nlc3Mg dG8gcm9sbCBiYWNrIHRoZSBjdXJyZW50IHRyYW5zYWN0aW9uIGFuZCBleGl0LCBiZWNhdXNlIGFu b3RoZXIgc2VydmVyIHByb2Nlc3MgZXhpdGVkIGFibm9ybWFsbHkgYW5kIHBvc3NpYmx5IGNvcnJ1 cHRlZCBzaGFyZWQgbWVtb3J5Lg0KW2xvY2FsXUBbbG9jYWxdIDIwMDYtMDEtMDIgMTU6MDM6MDEg TVNUOkhJTlQ6ICBJbiBhIG1vbWVudCB5b3Ugc2hvdWxkIGJlIGFibGUgdG8gcmVjb25uZWN0IHRv IHRoZSBkYXRhYmFzZSBhbmQgcmVwZWF0IHlvdXIgY29tbWFuZC4NCjE5Mi4xNjguMTkuMTI5KDUw NzMyKUAxOTIuMTY4LjE5LjEyOSAyMDA2LTAxLTAyIDE1OjAzOjAxIE1TVDpXQVJOSU5HOiAgdGVy bWluYXRpbmcgY29ubmVjdGlvbiBiZWNhdXNlIG9mIGNyYXNoIG9mIGFub3RoZXIgc2VydmVyIHBy b2Nlc3MNCjE5Mi4xNjguMTkuMTI5KDUwNzMyKUAxOTIuMTY4LjE5LjEyOSAyMDA2LTAxLTAyIDE1 OjAzOjAxIE1TVDpERVRBSUw6ICBUaGUgcG9zdG1hc3RlciBoYXMgY29tbWFuZGVkIHRoaXMgc2Vy dmVyIHByb2Nlc3MgdG8gcm9sbCBiYWNrIHRoZSBjdXJyZW50IHRyYW5zYWN0aW9uIGFuZCBleGl0 LCBiZWNhdXNlIGFub3RoZXIgc2VydmVyIHByb2Nlc3MgZXhpdGVkIGFibm9ybWFsbHkgYW5kIHBv c3NpYmx5IGNvcnJ1cHRlZCBzaGFyZWQgbWVtb3J5Lg0KMTkyLjE2OC4xOS4xMjkoNTA3MzIpQDE5 Mi4xNjguMTkuMTI5IDIwMDYtMDEtMDIgMTU6MDM6MDEgTVNUOkhJTlQ6ICBJbiBhIG1vbWVudCB5 b3Ugc2hvdWxkIGJlIGFibGUgdG8gcmVjb25uZWN0IHRvIHRoZSBkYXRhYmFzZSBhbmQgcmVwZWF0 IHlvdXIgY29tbWFuZC4NCjE5Mi4xNjguMTkuMTI5KDUwNzMwKUAxOTIuMTY4LjE5LjEyOSAyMDA2 LTAxLTAyIDE1OjAzOjAxIE1TVDpXQVJOSU5HOiAgdGVybWluYXRpbmcgY29ubmVjdGlvbiBiZWNh dXNlIG9mIGNyYXNoIG9mIGFub3RoZXIgc2VydmVyIHByb2Nlc3MNCjE5Mi4xNjguMTkuMTI5KDUw NzMwKUAxOTIuMTY4LjE5LjEyOSAyMDA2LTAxLTAyIDE1OjAzOjAxIE1TVDpERVRBSUw6ICBUaGUg cG9zdG1hc3RlciBoYXMgY29tbWFuZGVkIHRoaXMgc2VydmVyIHByb2Nlc3MgdG8gcm9sbCBiYWNr IHRoZSBjdXJyZW50IHRyYW5zYWN0aW9uIGFuZCBleGl0LCBiZWNhdXNlIGFub3RoZXIgc2VydmVy IHByb2Nlc3MgZXhpdGVkIGFibm9ybWFsbHkgYW5kIHBvc3NpYmx5IGNvcnJ1cHRlZCBzaGFyZWQg bWVtb3J5Lg0KMTkyLjE2OC4xOS4xMjkoNTA3MzApQDE5Mi4xNjguMTkuMTI5IDIwMDYtMDEtMDIg MTU6MDM6MDEgTVNUOkhJTlQ6ICBJbiBhIG1vbWVudCB5b3Ugc2hvdWxkIGJlIGFibGUgdG8gcmVj b25uZWN0IHRvIHRoZSBkYXRhYmFzZSBhbmQgcmVwZWF0IHlvdXIgY29tbWFuZC4NCjE5Mi4xNjgu MTkuMTI5KDUwNzMxKUAxOTIuMTY4LjE5LjEyOSAyMDA2LTAxLTAyIDE1OjAzOjAxIE1TVDpXQVJO SU5HOiAgdGVybWluYXRpbmcgY29ubmVjdGlvbiBiZWNhdXNlIG9mIGNyYXNoIG9mIGFub3RoZXIg c2VydmVyIHByb2Nlc3MNCjE5Mi4xNjguMTkuMTI5KDUwNzMxKUAxOTIuMTY4LjE5LjEyOSAyMDA2 LTAxLTAyIDE1OjAzOjAxIE1TVDpERVRBSUw6ICBUaGUgcG9zdG1hc3RlciBoYXMgY29tbWFuZGVk IHRoaXMgc2VydmVyIHByb2Nlc3MgdG8gcm9sbCBiYWNrIHRoZSBjdXJyZW50IHRyYW5zYWN0aW9u IGFuZCBleGl0LCBiZWNhdXNlIGFub3RoZXIgc2VydmVyIHByb2Nlc3MgZXhpdGVkIGFibm9ybWFs bHkgYW5kIHBvc3NpYmx5IGNvcnJ1cHRlZCBzaGFyZWQgbWVtb3J5Lg0KMTkyLjE2OC4xOS4xMjko NTA3MzEpQDE5Mi4xNjguMTkuMTI5IDIwMDYtMDEtMDIgMTU6MDM6MDEgTVNUOkhJTlQ6ICBJbiBh IG1vbWVudCB5b3Ugc2hvdWxkIGJlIGFibGUgdG8gcmVjb25uZWN0IHRvIHRoZSBkYXRhYmFzZSBh bmQgcmVwZWF0IHlvdXIgY29tbWFuZC4NCkAgMjAwNi0wMS0wMiAxNTowMzowMSBNU1Q6TE9HOiAg YWxsIHNlcnZlciBwcm9jZXNzZXMgdGVybWluYXRlZDsgcmVpbml0aWFsaXppbmcNCkAgMjAwNi0w MS0wMiAxNTowMzowMSBNU1Q6TE9HOiAgZGF0YWJhc2Ugc3lzdGVtIHdhcyBpbnRlcnJ1cHRlZCBh dCAyMDA2LTAxLTAyIDE1OjAyOjQ3IE1TVA0KQCAyMDA2LTAxLTAyIDE1OjAzOjAxIE1TVDpMT0c6 ICBjaGVja3BvaW50IHJlY29yZCBpcyBhdCAzNy9ENjBGOTNBOA0KQCAyMDA2LTAxLTAyIDE1OjAz OjAxIE1TVDpMT0c6ICByZWRvIHJlY29yZCBpcyBhdCAzNy9ENjAwODAxODsgdW5kbyByZWNvcmQg aXMgYXQgMC8wOyBzaHV0ZG93biBGQUxTRQ0KQCAyMDA2LTAxLTAyIDE1OjAzOjAxIE1TVDpMT0c6 ICBuZXh0IHRyYW5zYWN0aW9uIElEOiAzMjE5NjI4MDsgbmV4dCBPSUQ6IDEwMjA0MTk0NQ0KQCAy MDA2LTAxLTAyIDE1OjAzOjAxIE1TVDpMT0c6ICBuZXh0IE11bHRpWGFjdElkOiA0MTsgbmV4dCBN dWx0aVhhY3RPZmZzZXQ6IDkzDQpAIDIwMDYtMDEtMDIgMTU6MDM6MDEgTVNUOkxPRzogIGRhdGFi YXNlIHN5c3RlbSB3YXMgbm90IHByb3Blcmx5IHNodXQgZG93bjsgYXV0b21hdGljIHJlY292ZXJ5 IGluIHByb2dyZXNzDQpAIDIwMDYtMDEtMDIgMTU6MDM6MDEgTVNUOkxPRzogIHJlZG8gc3RhcnRz IGF0IDM3L0Q2MDA4MDE4DQpAIDIwMDYtMDEtMDIgMTU6MDM6MDEgTVNUOkxPRzogIHJlY29yZCB3 aXRoIHplcm8gbGVuZ3RoIGF0IDM3L0Q2MEY5M0Y4DQpAIDIwMDYtMDEtMDIgMTU6MDM6MDEgTVNU OkxPRzogIHJlZG8gZG9uZSBhdCAzNy9ENjBGOTNBOA0KQCAyMDA2LTAxLTAyIDE1OjAzOjAyIE1T VDpMT0c6ICBkYXRhYmFzZSBzeXN0ZW0gaXMgcmVhZHkNCkAgMjAwNi0wMS0wMiAxNTowMzowMiBN U1Q6TE9HOiAgdHJhbnNhY3Rpb24gSUQgd3JhcCBsaW1pdCBpcyAxMDg3MTE4NjAwLCBsaW1pdGVk IGJ5IGRhdGFiYXNlICJjZXJ0Ig0K --=-PLs+GL3jMplm6JOF/iwc--