Received: from localhost (unknown [200.46.204.183]) by developer.postgresql.org (Postfix) with ESMTP id 030D22E00AE for ; Wed, 28 May 2008 22:37:30 -0300 (ADT) Received: from developer.postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.183]) (amavisd-maia, port 10024) with ESMTP id 68516-05 for ; Wed, 28 May 2008 22:37:02 -0300 (ADT) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from wr-out-0506.google.com (wr-out-0506.google.com [64.233.184.231]) by developer.postgresql.org (Postfix) with ESMTP id 235632E0088 for ; Wed, 28 May 2008 22:37:08 -0300 (ADT) Received: by wr-out-0506.google.com with SMTP id 76so2015931wra.10 for ; Wed, 28 May 2008 18:37:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type; bh=RKkFbvvuyOeH3XpSuUA5IvEbsbxuCEY1jO6E3ZQPm90=; b=JUReaw25b54upJ2p/LOrf/A2y31Pwq9vJu3QSiEQM60Qc6LqNTO+gH5xK2kBb+ESaRLp1E1UwXAjrs/OJfrOJPV//dS5OB3Z7gH2Ei0IaVGpT0U6bA8yaCzAUCtXCnsWekrS5Vp+TXIxaU8GQ9OoLm9jVk6qfdKCQZ0HO3cbcUA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type; b=ECAdfnbURhY8ELx+RVBtRxIKCg8RZS7ALbQZ5ag7d09aEyxKPVgfC4fhgufDpDOYmJ6ZaPWF5Im1URDp+NqLjvxQE0P7ipBmRAjiAMP5cE4NlpOb3j5bJfNsHm4vEalg+2GJPJvN0hcsJ0gN334fBKhiUpKSaHCtmxDDxYVf29c= Received: by 10.101.69.6 with SMTP id w6mr5126390ank.63.1212025026666; Wed, 28 May 2008 18:37:06 -0700 (PDT) Received: by 10.100.250.18 with HTTP; Wed, 28 May 2008 18:37:06 -0700 (PDT) Message-ID: <8078a1730805281837y1ed331bey4e7db16a63423cfa@mail.gmail.com> Date: Wed, 28 May 2008 18:37:06 -0700 From: "Medi Montaseri" To: postgresql-admin Subject: what happend to my database MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_9074_7918691.1212025026631" X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200805/233 X-Sequence-Number: 29317 ------=_Part_9074_7918691.1212025026631 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline Hi, I am faced with a database disapperance and seeking some explanations outside of gremlins. I had a database running at cat /etc/sysconfig/pgsl/postmaster PGDATA=/qmsvol/pg_8.1.9/data PGLOG=/var/log/pgsql/pgstartup.log Where /qmsvol is an iSCSI block device A couple of days ago, my server was rebooted and by the time I got to it my database was deleted, gone, zapped, not there any more. I looked at my pgstartup.log where I see the following.... postmaster cannot access the server configuration file "/qmsvol/pg_8.1.9/postgresql.conf": Permission denied over 17 times and then following by... The database cluster will be initialized with locale en_US.UTF-8. I think the following happend... Since my PGDATA was on an iSCSI device, by the time /etc/rc3.d/S64postgresql was executed, the device below it was not available.....question...why the error says "permission denied" vs "file not found". In the meantime, pg_ctl kept trying and finally concluded that the data directory is blank, and hence this must be a out-of-box case and he is good to initdb the PGDATA and as it called initdb to do the job... the iSCSI volume below it came online and by then the bomb had already been dropped. Now I need to find some facts to support this... Where else can I look for forensics Thanks Medi ------=_Part_9074_7918691.1212025026631 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline Hi,

I am faced with a database disapperance and seeking some explanations outside of gremlins.
I had a database running at

cat /etc/sysconfig/pgsl/postmaster
PGDATA=/qmsvol/pg_8.1.9/data
PGLOG=/var/log/pgsql/pgstartup.log

Where /qmsvol is an iSCSI block device
A couple of days ago, my server was rebooted and by the time I got to it my database was deleted, gone, zapped, not there any more.

I looked at my pgstartup.log where I see the following....

postmaster cannot access the server configuration file "/qmsvol/pg_8.1.9/postgresql.conf": Permission denied
over 17 times and then following by...
The database cluster will be initialized with locale en_US.UTF-8.

I think the following happend...
Since my PGDATA was on an iSCSI device, by the time /etc/rc3.d/S64postgresql was executed, the device below it was not available.....question...why the error says "permission denied" vs "file not found".  In the meantime, pg_ctl kept trying and finally concluded that the data directory is blank, and hence this must be a out-of-box case and he is good to initdb the PGDATA and as it called initdb to do the job... the iSCSI volume below it came online and by then the bomb had already been dropped.

Now I need to find some facts to support this...
Where else can I look for forensics

Thanks
Medi




------=_Part_9074_7918691.1212025026631--