pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
pg_clog corrupt, can't start postgres
4+ messages / 2 participants
[nested] [flat]

* pg_clog corrupt, can't start postgres
@ 2005-02-21 15:58 Anjan Dave <adave@vantage.com>
  2005-02-21 16:42 ` Re: pg_clog corrupt, can't start postgres Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 1 reply; 4+ messages in thread

From: Anjan Dave @ 2005-02-21 15:58 UTC (permalink / raw)
  To: pgsql-admin

Hi,
 
I need some help in bringing back this db please.
 
The partition ran out of space from an import process. I cleared up the space and attempted to start the postgres service again, but it doesn't start and i get following in the message log.
 
HINT:  This probably means that some data is corrupted and you will have to use the last backup for recovery.
LOG:  checkpoint record is at 1B/27F23A6C
LOG:  redo record is at 1B/27751714; undo record is at 0/0; shutdown FALSE
LOG:  next transaction ID: 45279762; next OID: 43062083
LOG:  database system was not properly shut down; automatic recovery in progress
LOG:  redo starts at 1B/27751714
PANIC:  could not access status of transaction 45514755
DETAIL:  could not read from file "/var/lib/pgsql/data/pg_clog/002B" at offset 106496: Success
LOG:  startup process (PID 23991) was terminated by signal 6
LOG:  aborting startup due to startup process failure

Postgres is 7.4.1 on this machine.
 
I saw some previous posts on this subject and so far the solution seems to be initialize and restore databases from the dumps.
 
I can live with some aborted transactions, if it's possible to recover somehow.
 
$ psql
psql: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
 
# ls -l pg_xlog/
total 131232
-rw-------    1 postgres postgres 16777216 Feb 19 13:30 0000001B00000026
-rw-------    1 postgres postgres 16777216 Feb 19 13:34 0000001B00000027
-rw-------    1 postgres postgres 16777216 Feb 19 13:44 0000001B00000028
-rw-------    1 postgres postgres 16777216 Feb 19 13:15 0000001B00000029
-rw-------    1 postgres postgres 16777216 Feb 19 13:12 0000001B0000002A
-rw-------    1 postgres postgres 16777216 Feb 19 13:18 0000001B0000002B
-rw-------    1 postgres postgres 16777216 Feb 19 13:26 0000001B0000002C
-rw-------    1 postgres postgres 16777216 Feb 19 13:22 0000001B0000002D
# ls -l pg_clog/
total 628
-rw-------    1 postgres postgres   262144 Feb 19 04:31 0029
-rw-------    1 postgres postgres   262144 Feb 19 11:55 002A
-rw-------    1 postgres postgres   106496 Feb 20 22:34 002B

Thanks,
Anjan


^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* Re: pg_clog corrupt, can't start postgres
  2005-02-21 15:58 pg_clog corrupt, can't start postgres Anjan Dave <adave@vantage.com>
@ 2005-02-21 16:42 ` Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 4+ messages in thread

From: Tom Lane @ 2005-02-21 16:42 UTC (permalink / raw)
  To: Anjan Dave <adave@vantage.com>; +Cc: pgsql-admin

"Anjan Dave" <adave@vantage.com> writes:
> The partition ran out of space from an import process. I cleared up the space and attempted to start the postgres service again, but it doesn't start and i get following in the message log.
> PANIC:  could not access status of transaction 45514755
> DETAIL:  could not read from file "/var/lib/pgsql/data/pg_clog/002B" at offset 106496: Success
> LOG:  startup process (PID 23991) was terminated by signal 6
> LOG:  aborting startup due to startup process failure

> Postgres is 7.4.1 on this machine.
 
> I saw some previous posts on this subject and so far the solution seems to be initialize and restore databases from the dumps.

Before that, try updating to 7.4.7 (or at least 7.4.2) --- this looks
like the same bug fixed here:

2004-01-26 14:16  tgl

	* src/backend/access/transam/varsup.c (REL7_4_STABLE): Repair
	incorrect order of operations in GetNewTransactionId().  We must
	complete ExtendCLOG() before advancing nextXid, so that if that
	routine fails, the next incoming transaction will try it again. 
	Per trouble report from Christopher Kings-Lynne.

You might also go back to the mail list archives from that time and see
what advice was given to Chris about getting out of the problem he found
himself in.  I *think* that something along the line of forcibly
appending a page of zeroes to that clog file might be the best solution,
but this was more than a year ago and I don't recall for sure.

			regards, tom lane



^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* Re: pg_clog corrupt, can't start postgres
@ 2005-02-21 17:44 Anjan Dave <adave@vantage.com>
  2005-02-21 20:19 ` Re: pg_clog corrupt, can't start postgres Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 1 reply; 4+ messages in thread

From: Anjan Dave @ 2005-02-21 17:44 UTC (permalink / raw)
  To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-admin

Tom,
 
You're the man! 
 
I zeroed out the troubled pg_clog file and the db started up fine! Here's the link to the discussion, and a detailed explanation of the issue by Tom:
http://groups-beta.google.com/group/comp.databases.postgresql.hackers/browse_thread/thread/c97c853f6...
 
Tom, is the issue resolved after 7.4.1?
 
Thanks,
Anjan
 
 
-----Original Message----- 
From: Tom Lane [mailto:tgl@sss.pgh.pa.us] 
Sent: Mon 2/21/2005 11:42 AM 
To: Anjan Dave 
Cc: pgsql-admin@postgresql.org 
Subject: Re: [ADMIN] pg_clog corrupt, can't start postgres 



	"Anjan Dave" <adave@vantage.com> writes: 
	> The partition ran out of space from an import process. I cleared up the space and attempted to start the postgres service again, but it doesn't start and i get following in the message log.

	> PANIC:  could not access status of transaction 45514755 
	> DETAIL:  could not read from file "/var/lib/pgsql/data/pg_clog/002B" at offset 106496: Success 
	> LOG:  startup process (PID 23991) was terminated by signal 6 
	> LOG:  aborting startup due to startup process failure 

	> Postgres is 7.4.1 on this machine. 
	  
	> I saw some previous posts on this subject and so far the solution seems to be initialize and restore databases from the dumps.

	Before that, try updating to 7.4.7 (or at least 7.4.2) --- this looks 
	like the same bug fixed here: 

	2004-01-26 14:16  tgl 

	        * src/backend/access/transam/varsup.c (REL7_4_STABLE): Repair 
	        incorrect order of operations in GetNewTransactionId().  We must 
	        complete ExtendCLOG() before advancing nextXid, so that if that 
	        routine fails, the next incoming transaction will try it again. 
	        Per trouble report from Christopher Kings-Lynne. 

	You might also go back to the mail list archives from that time and see 
	what advice was given to Chris about getting out of the problem he found 
	himself in.  I *think* that something along the line of forcibly 
	appending a page of zeroes to that clog file might be the best solution, 
	but this was more than a year ago and I don't recall for sure. 

	                        regards, tom lane 



^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* Re: pg_clog corrupt, can't start postgres
  2005-02-21 17:44 Re: pg_clog corrupt, can't start postgres Anjan Dave <adave@vantage.com>
@ 2005-02-21 20:19 ` Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 4+ messages in thread

From: Tom Lane @ 2005-02-21 20:19 UTC (permalink / raw)
  To: Anjan Dave <adave@vantage.com>; +Cc: pgsql-admin

"Anjan Dave" <adave@vantage.com> writes:
> Tom, is the issue resolved after 7.4.1?

Yes, that's why I told you to update.

			regards, tom lane



^ permalink  raw  reply  [nested|flat] 4+ messages in thread


end of thread, other threads:[~2005-02-21 20:19 UTC | newest]

Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2005-02-21 15:58 pg_clog corrupt, can't start postgres Anjan Dave <adave@vantage.com>
2005-02-21 16:42 ` Tom Lane <tgl@sss.pgh.pa.us>
2005-02-21 17:44 Re: pg_clog corrupt, can't start postgres Anjan Dave <adave@vantage.com>
2005-02-21 20:19 ` Tom Lane <tgl@sss.pgh.pa.us>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox