pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedStartup proc 30595 exited with status 512 - abort and FATAL 2: XLogFlush
8+ messages / 3 participants
[nested] [flat]
* Startup proc 30595 exited with status 512 - abort and FATAL 2: XLogFlush
@ 2010-02-02 16:55 Wang, Mary Y <mary.y.wang@boeing.com>
0 siblings, 1 reply; 8+ messages in thread
From: Wang, Mary Y @ 2010-02-02 16:55 UTC (permalink / raw)
To: pgsql-admin
Hi,
I'm having a bad day. My Postgresql has this error "FATAL 2: XLogFlush: request is not satisfied". I tried to follow the instructions from a thread about looking for a core dump, but when I tried to start the postmaster, I got "/usr/bin/postmaster: Startup proc 30595 exited with status 512 - abort".
How do I find the corrupted record? If that is the case.
I'm pg version is postgresql-7.1.3-2. What are my options? I read some where that I can pg_resetxlog, but I can't even find it in the path.
This is from the log file:
********************************************************************************************************************************
DEBUG: database system was shut down at 2010-02-01 19:24:40 PST
DEBUG: CheckPoint record at (2, 4173828852)
DEBUG: Redo record at (2, 4173828852); Undo record at (0, 0); Shutdown TRUE
DEBUG: NextTransactionId: 79259759; NextOid: 4395896
DEBUG: database system is in production state
Smart Shutdown request at Mon Feb 1 20:08:59 2010
DEBUG: shutting down
DEBUG: database system is shut down
DEBUG: database system shutdown was interrupted at 2010-02-02 07:54:51 PST
DEBUG: CheckPoint record at (2, 4174741948)
DEBUG: Redo record at (2, 4174741948); Undo record at (0, 0); Shutdown FALSE
DEBUG: NextTransactionId: 79276624; NextOid: 4412280
DEBUG: database system was not properly shut down; automatic recovery in progress...
DEBUG: redo starts at (2, 4174742012)
DEBUG: ReadRecord: record with zero len at (2, 4174768976)
DEBUG: redo done at (2, 4174768916)
FATAL 2: XLogFlush: request is not satisfied
/usr/bin/postmaster: Startup proc 30708 exited with status 512 - abort
DEBUG: database system shutdown was interrupted at 2010-02-02 08:11:50 PST
DEBUG: CheckPoint record at (2, 4174741948)
DEBUG: Redo record at (2, 4174741948); Undo record at (0, 0); Shutdown FALSE
DEBUG: NextTransactionId: 79276624; NextOid: 4412280
DEBUG: database system was not properly shut down; automatic recovery in progress...
DEBUG: redo starts at (2, 4174742012)
DEBUG: ReadRecord: record with zero len at (2, 4174768976)
DEBUG: redo done at (2, 4174768916)
FATAL 2: XLogFlush: request is not satisfied
/usr/bin/postmaster: Startup proc 30722 exited with status 512 - abort
DEBUG: database system shutdown was interrupted at 2010-02-02 08:18:59 PST
DEBUG: CheckPoint record at (2, 4174741948)
DEBUG: Redo record at (2, 4174741948); Undo record at (0, 0); Shutdown FALSE
DEBUG: NextTransactionId: 79276624; NextOid: 4412280
DEBUG: database system was not properly shut down; automatic recovery in progress...
DEBUG: redo starts at (2, 4174742012)
DEBUG: ReadRecord: record with zero len at (2, 4174768976)
DEBUG: redo done at (2, 4174768916)
FATAL 2: XLogFlush: request is not satisfied
/usr/bin/postmaster: Startup proc 30788 exited with status 512 - abort
******************************************************************************************
Thanks in advance.
Mary
------------------------------------------------
Mary Y Wang
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Startup proc 30595 exited with status 512 - abort and FATAL 2: XLogFlush
@ 2010-02-02 19:52 Tom Lane <tgl@sss.pgh.pa.us>
parent: Wang, Mary Y <mary.y.wang@boeing.com>
0 siblings, 1 reply; 8+ messages in thread
From: Tom Lane @ 2010-02-02 19:52 UTC (permalink / raw)
To: Wang, Mary Y <mary.y.wang@boeing.com>; +Cc: pgsql-admin
"Wang, Mary Y" <mary.y.wang@boeing.com> writes:
> I'm having a bad day. My Postgresql has this error "FATAL 2: XLogFlush: request is not satisfied". I tried to follow the instructions from a thread about looking for a core dump, but when I tried to start the postmaster, I got "/usr/bin/postmaster: Startup proc 30595 exited with status 512 - abort".
You've got a corrupted page that is affected by a WAL replay operation,
so things are pretty much a mess.
> I'm pg version is postgresql-7.1.3-2. What are my options?
[ blanches... ] You do realize that that version has been obsolete
since 2002?
pg_resetxlog was a contrib module in 7.1, so if you can find the
software repository you got postgresql from, you should be able to
install postgresql-contrib. However, I am betting this thing is so
old that you don't even have the chance at doing that.
Most likely you're going to have to go back to your last backup.
After which, you should make it a priority to get onto a less
antique version of Postgres (and the underlying OS too, no doubt).
regards, tom lane
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Startup proc 30595 exited with status 512 - abort and FATAL 2: XLogFlush
@ 2010-02-02 20:05 Wang, Mary Y <mary.y.wang@boeing.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 8+ messages in thread
From: Wang, Mary Y @ 2010-02-02 20:05 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-admin
Tom,
Thanks for the help
I was able to find pg_resetxlog in the path. Are there any precautions that I need to be aware of? Or I just don't have any choice?
I'd like to do pg_dump or pg_dumpall.
Should I do a pg_resetxlog $PGDATA?
Here is my pgcontrol info:
------------------------------------------------------------------
_control version number: 71
Catalog version number: 200101061
Database state: SHUTDOWNING
pg_control last modified: Tue Feb 2 08:19:14 2010
Current log file id: 2
Next log file segment: 249
Latest checkpoint location: 2/F8D581BC
Prior checkpoint location: 2/F8D49A34
Latest checkpoint's REDO location: 2/F8D581BC
Latest checkpoint's UNDO location: 0/0
Latest checkpoint's StartUpID: 210
Latest checkpoint's NextXID: 79276624
Latest checkpoint's NextOID: 4412280
Time of latest checkpoint: Tue Feb 2 06:50:48 2010
Database block size: 8192
Blocks per segment of large relation: 131072
LC_COLLATE: en_US
LC_CTYPE: en_US
---------------------------------------------------
In my pg_xlog directory, I've:
total 32808
-rw------- 1 postgres postgres 16777216 Feb 2 06:54 00000002000000F8
-rw------- 1 postgres postgres 16777216 Feb 1 10:51 00000002000000F9
------------------------------------------------
Thanks for any help.
Mary Y Wang
-----Original Message-----
From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
Sent: Tuesday, February 02, 2010 11:53 AM
To: Wang, Mary Y
Cc: pgsql-admin@postgresql.org
Subject: Re: [ADMIN] Startup proc 30595 exited with status 512 - abort and FATAL 2: XLogFlush
"Wang, Mary Y" <mary.y.wang@boeing.com> writes:
> I'm having a bad day. My Postgresql has this error "FATAL 2: XLogFlush: request is not satisfied". I tried to follow the instructions from a thread about looking for a core dump, but when I tried to start the postmaster, I got "/usr/bin/postmaster: Startup proc 30595 exited with status 512 - abort".
You've got a corrupted page that is affected by a WAL replay operation, so things are pretty much a mess.
> I'm pg version is postgresql-7.1.3-2. What are my options?
[ blanches... ] You do realize that that version has been obsolete since 2002?
pg_resetxlog was a contrib module in 7.1, so if you can find the software repository you got postgresql from, you should be able to install postgresql-contrib. However, I am betting this thing is so old that you don't even have the chance at doing that.
Most likely you're going to have to go back to your last backup.
After which, you should make it a priority to get onto a less antique version of Postgres (and the underlying OS too, no doubt).
regards, tom lane
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Startup proc 30595 exited with status 512 - abort and FATAL 2: XLogFlush
@ 2010-02-02 20:14 Tom Lane <tgl@sss.pgh.pa.us>
parent: Wang, Mary Y <mary.y.wang@boeing.com>
0 siblings, 1 reply; 8+ messages in thread
From: Tom Lane @ 2010-02-02 20:14 UTC (permalink / raw)
To: Wang, Mary Y <mary.y.wang@boeing.com>; +Cc: pgsql-admin
"Wang, Mary Y" <mary.y.wang@boeing.com> writes:
> Thanks for the help
> I was able to find pg_resetxlog in the path. Are there any precautions that I need to be aware of? Or I just don't have any choice?
I'd suggest taking a tarball copy of the $PGDATA tree, so you can at
least get back to where you were if it doesn't work.
Actually ... is this RHEL 2.1 as I suspect? If so, can you find the
last RHEL 2.1 update, which was postgresql-7.1.3-7.rhel2.1AS?
There was a fix in that that might well address your issue:
* Wed Feb 23 2005 Tom Lane <tgl@redhat.com> 7.1.3-7.rhel2.1AS
- Back-patch community 7.2 change in error recovery behavior of XLogFlush;
this allows successful restart of the database even in the presence of
dubious values in LSN fields of database pages.
Alternatively, if you have the ability to rebuild the version you've
got, you're welcome to try adding the patch, which is attached.
regards, tom lane
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Startup proc 30595 exited with status 512 - abort and FATAL 2: XLogFlush
@ 2010-02-02 22:29 Wang, Mary Y <mary.y.wang@boeing.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 8+ messages in thread
From: Wang, Mary Y @ 2010-02-02 22:29 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-admin
Thanks Tom.
After talking to my co-worker, we decided to go to the last backup (we used the pg_dumpall -c command).
However, when I did enter "psql -f /usr/pgsql/backups/31.bak template1" to restore the database, I got "
psql: connectDBStart() -- connect() failed: No such file or directory
Is the postmaster running locally
and accepting connections on Unix socket '/tmp/.s.PGSQL.5432'?
".
I can't start my postmaster. So how would I restore my last good backup?
Thanks in advance
Mary
-----Original Message-----
From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
Sent: Tuesday, February 02, 2010 12:14 PM
To: Wang, Mary Y
Cc: pgsql-admin@postgresql.org
Subject: Re: [ADMIN] Startup proc 30595 exited with status 512 - abort and FATAL 2: XLogFlush
"Wang, Mary Y" <mary.y.wang@boeing.com> writes:
> Thanks for the help
> I was able to find pg_resetxlog in the path. Are there any precautions that I need to be aware of? Or I just don't have any choice?
I'd suggest taking a tarball copy of the $PGDATA tree, so you can at least get back to where you were if it doesn't work.
Actually ... is this RHEL 2.1 as I suspect? If so, can you find the last RHEL 2.1 update, which was postgresql-7.1.3-7.rhel2.1AS?
There was a fix in that that might well address your issue:
* Wed Feb 23 2005 Tom Lane <tgl@redhat.com> 7.1.3-7.rhel2.1AS
- Back-patch community 7.2 change in error recovery behavior of XLogFlush;
this allows successful restart of the database even in the presence of
dubious values in LSN fields of database pages.
Alternatively, if you have the ability to rebuild the version you've got, you're welcome to try adding the patch, which is attached.
regards, tom lane
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Startup proc 30595 exited with status 512 - abort and FATAL 2: XLogFlush
@ 2010-02-02 23:17 Scott Marlowe <scott.marlowe@gmail.com>
parent: Wang, Mary Y <mary.y.wang@boeing.com>
0 siblings, 1 reply; 8+ messages in thread
From: Scott Marlowe @ 2010-02-02 23:17 UTC (permalink / raw)
To: Wang, Mary Y <mary.y.wang@boeing.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-admin
On Tue, Feb 2, 2010 at 3:29 PM, Wang, Mary Y <mary.y.wang@boeing.com> wrote:
> Thanks Tom.
>
> After talking to my co-worker, we decided to go to the last backup (we used the pg_dumpall -c command).
> However, when I did enter "psql -f /usr/pgsql/backups/31.bak template1" to restore the database, I got "
> psql: connectDBStart() -- connect() failed: No such file or directory
> Is the postmaster running locally
> and accepting connections on Unix socket '/tmp/.s.PGSQL.5432'?
> ".
> I can't start my postmaster. So how would I restore my last good backup?
The normal way is to drop the old cluster and create a new one. I'm
not entirely sure how to do that on something as old as RHEL 2.1. The
normal way would be to mv or rm -rf the /var/lib/pgsql/data dir and
run initdb again. something like:
sudo /etc/init.d/pgsql stop
sudo rm -rf /var/lib/pgsql/data/*
sudo -u postgres initdb -D /var/lib/pgsql/data
sudo /etc/init.d/pgsql start
or something like that.
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Startup proc 30595 exited with status 512 - abort and FATAL 2: XLogFlush
@ 2010-02-03 00:32 Wang, Mary Y <mary.y.wang@boeing.com>
parent: Scott Marlowe <scott.marlowe@gmail.com>
0 siblings, 1 reply; 8+ messages in thread
From: Wang, Mary Y @ 2010-02-03 00:32 UTC (permalink / raw)
To: Scott Marlowe <scott.marlowe@gmail.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-admin
Thanks Scott.
We used a command like this '/usr/bin/pg_dumpall -cC' postgres to do the pg_dumpall.
I started my postmaster with this command "/usr/bin/postmaster -D /var/lib/pgsql/data -i"
I got my tables back in the database, but I don't see any data.
What could have went wrong when I did the "psql -f /usr/pgsql/backups/31.bak template1"?
Did I miss a step or something?
Now, I'm really worried.
Maybe I should have capture the log file when I did the restore?
Please advise.
Mary
------------------------------------------------
Mary Y Wang
-----Original Message-----
From: Scott Marlowe [mailto:scott.marlowe@gmail.com]
Sent: Tuesday, February 02, 2010 3:17 PM
To: Wang, Mary Y
Cc: Tom Lane; pgsql-admin@postgresql.org
Subject: Re: [ADMIN] Startup proc 30595 exited with status 512 - abort and FATAL 2: XLogFlush
On Tue, Feb 2, 2010 at 3:29 PM, Wang, Mary Y <mary.y.wang@boeing.com> wrote:
> Thanks Tom.
>
> After talking to my co-worker, we decided to go to the last backup (we used the pg_dumpall -c command).
> However, when I did enter "psql -f /usr/pgsql/backups/31.bak template1" to restore the database, I got "
> psql: connectDBStart() -- connect() failed: No such file or directory
> Is the postmaster running locally
> and accepting connections on Unix socket '/tmp/.s.PGSQL.5432'?
> ".
> I can't start my postmaster. So how would I restore my last good backup?
The normal way is to drop the old cluster and create a new one. I'm not entirely sure how to do that on something as old as RHEL 2.1. The normal way would be to mv or rm -rf the /var/lib/pgsql/data dir and run initdb again. something like:
sudo /etc/init.d/pgsql stop
sudo rm -rf /var/lib/pgsql/data/*
sudo -u postgres initdb -D /var/lib/pgsql/data sudo /etc/init.d/pgsql start
or something like that.
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Startup proc 30595 exited with status 512 - abort and FATAL 2: XLogFlush
@ 2010-02-03 01:34 Wang, Mary Y <mary.y.wang@boeing.com>
parent: Wang, Mary Y <mary.y.wang@boeing.com>
0 siblings, 0 replies; 8+ messages in thread
From: Wang, Mary Y @ 2010-02-03 01:34 UTC (permalink / raw)
To: Scott Marlowe <scott.marlowe@gmail.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-admin
Ok. A little relief . It looks like only one table has no data. I guess I need to know what cause that table has no data....Going back to the drawing board.
Mary
-----Original Message-----
From: pgsql-admin-owner@postgresql.org [mailto:pgsql-admin-owner@postgresql.org] On Behalf Of Wang, Mary Y
Sent: Tuesday, February 02, 2010 4:32 PM
To: Scott Marlowe
Cc: Tom Lane; pgsql-admin@postgresql.org
Subject: Re: [ADMIN] Startup proc 30595 exited with status 512 - abort and FATAL 2: XLogFlush
Thanks Scott.
We used a command like this '/usr/bin/pg_dumpall -cC' postgres to do the pg_dumpall.
I started my postmaster with this command "/usr/bin/postmaster -D /var/lib/pgsql/data -i"
I got my tables back in the database, but I don't see any data.
What could have went wrong when I did the "psql -f /usr/pgsql/backups/31.bak template1"?
Did I miss a step or something?
Now, I'm really worried.
Maybe I should have capture the log file when I did the restore?
Please advise.
Mary
------------------------------------------------
Mary Y Wang
-----Original Message-----
From: Scott Marlowe [mailto:scott.marlowe@gmail.com]
Sent: Tuesday, February 02, 2010 3:17 PM
To: Wang, Mary Y
Cc: Tom Lane; pgsql-admin@postgresql.org
Subject: Re: [ADMIN] Startup proc 30595 exited with status 512 - abort and FATAL 2: XLogFlush
On Tue, Feb 2, 2010 at 3:29 PM, Wang, Mary Y <mary.y.wang@boeing.com> wrote:
> Thanks Tom.
>
> After talking to my co-worker, we decided to go to the last backup (we used the pg_dumpall -c command).
> However, when I did enter "psql -f /usr/pgsql/backups/31.bak template1" to restore the database, I got "
> psql: connectDBStart() -- connect() failed: No such file or directory
> Is the postmaster running locally
> and accepting connections on Unix socket '/tmp/.s.PGSQL.5432'?
> ".
> I can't start my postmaster. So how would I restore my last good backup?
The normal way is to drop the old cluster and create a new one. I'm not entirely sure how to do that on something as old as RHEL 2.1. The normal way would be to mv or rm -rf the /var/lib/pgsql/data dir and run initdb again. something like:
sudo /etc/init.d/pgsql stop
sudo rm -rf /var/lib/pgsql/data/*
sudo -u postgres initdb -D /var/lib/pgsql/data sudo /etc/init.d/pgsql start
or something like that.
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 8+ messages in thread
end of thread, other threads:[~2010-02-03 01:34 UTC | newest]
Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2010-02-02 16:55 Startup proc 30595 exited with status 512 - abort and FATAL 2: XLogFlush Wang, Mary Y <mary.y.wang@boeing.com>
2010-02-02 19:52 ` Tom Lane <tgl@sss.pgh.pa.us>
2010-02-02 20:05 ` Wang, Mary Y <mary.y.wang@boeing.com>
2010-02-02 20:14 ` Tom Lane <tgl@sss.pgh.pa.us>
2010-02-02 22:29 ` Wang, Mary Y <mary.y.wang@boeing.com>
2010-02-02 23:17 ` Scott Marlowe <scott.marlowe@gmail.com>
2010-02-03 00:32 ` Wang, Mary Y <mary.y.wang@boeing.com>
2010-02-03 01:34 ` Re: Startup proc 30595 exited with status 512 - abort and FATAL 2: XLogFlush Wang, Mary Y <mary.y.wang@boeing.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox