X-Original-To: pgsql-admin-postgresql.org@postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id CFBC29FB20C for ; Wed, 23 Aug 2006 11:32:03 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 94113-06 for ; Wed, 23 Aug 2006 11:31:51 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey- Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130]) by postgresql.org (Postfix) with ESMTP id 85F179FB20B for ; Wed, 23 Aug 2006 11:31:51 -0300 (ADT) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.13.6/8.13.6) with ESMTP id k7NEVoZc019003; Wed, 23 Aug 2006 10:31:50 -0400 (EDT) To: Volker cc: pgsql-admin@postgresql.org Subject: Re: Server Down problem In-reply-to: References: Comments: In-reply-to Volker message dated "Wed, 23 Aug 2006 10:44:05 +0200" Date: Wed, 23 Aug 2006 10:31:50 -0400 Message-ID: <19002.1156343510@sss.pgh.pa.us> From: Tom Lane X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=0.155 tagged_above=0 required=5 tests=AWL, SPF_HELO_PASS, SPF_PASS X-Spam-Level: X-Archive-Number: 200608/246 X-Sequence-Number: 22789 Volker writes: > currently we have a server down problem. The server is a IBM xSeries > 306, 1GB RAM, HD 36GB RAID1. Operating System ist Debian Sarge 3.1. > Postgresql version is 7.4.7. > The reason for this ist not knwon. The postmaster won't come up any > more. Here is the relevant content of the postgres.log file: It looks to me like pg_control is quite out of sync with the files in pg_xlog. pg_control claims to have checkpointed as recently as 16-Aug but there is no file newer than 7-Aug in pg_xlog. The other thing that is strange is that the filename numbers in pg_xlog correspond to WAL locations much higher than what pg_control claims is the end of WAL. Is $PGDATA kept on a dismountable volume (ie, not the root disk)? If so it might be a good idea to unmount the volume and look to see if there's anything under the mount point. I recall having seen corruption that came from trying to start Postgres before the $PGDATA disk had been mounted --- the initscript happily initdb'd a new database under the mount-point directory, and then when the main disk did come up things were badly hosed because the server was working with a pg_control in memory that was completely out of sync with everything else. This looks a bit like that might have happened here. As far as running pg_resetxlog goes, there is advice about setting the parameters in recent releases' documentation; try http://developer.postgresql.org/docs/postgres/app-pgresetxlog.html for the latest. (Some of the parameters mentioned don't exist in 7.4; just ignore 'em.) I'm afraid though that you may have actual database corruption. If so pg_resetxlog won't fix it. regards, tom lane