pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedError: pid file is invalid, please manually kill the stale server process.
5+ messages / 3 participants
[nested] [flat]
* Error: pid file is invalid, please manually kill the stale server process.
@ 2014-02-12 05:59 Theodotos Andreou <theo@theo-andreou.org>
0 siblings, 2 replies; 5+ messages in thread
From: Theodotos Andreou @ 2014-02-12 05:59 UTC (permalink / raw)
To: pgsql-admin
Hi all,
I am using postgres on Ubuntu 12.04 64 bit server. The version is 9.1.10
and I used the stock postgres from Ubuntu repos.
I have a weird problem. Sometimes when the machine is restarted, there
is no $PGDATA/postmaster.pid file. So when I try to restart postgres
(service postgresql restart) I get the following error:
Error: pid file is invalid, please manually kill the stale server process.
An even scarier side-effect of this is that the data, when this happens,
are older than the most recent data in the database.
It may be important to know that postgres runs under monit non sysv init
or upstart.
Also postgres is installed on a customer appliance that may experience
frequent reboots and even abrupt poweroffs! Is postgres in general ideal
for this situation? Is there an optimal configuration for this scenario?
Regards
theo
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Error: pid file is invalid, please manually kill the stale server process.
@ 2014-02-13 04:37 Lou Picciano <loupicciano@comcast.net>
parent: Theodotos Andreou <theo@theo-andreou.org>
1 sibling, 0 replies; 5+ messages in thread
From: Lou Picciano @ 2014-02-13 04:37 UTC (permalink / raw)
To: pgsql-admin; +Cc: Theodotos Andreou <theo@theo-andreou.org>
Theo,
Don't know much about the tools you may have installed on Ubuntu (don't use Ubuntu), but sounds like you have something 'snapshot-ing' your dataset... And that, on restart-after-inelegant-shutdown, the PostgreSQL service is trying to attach to the 'wrong' data.
Generally, yes; PostgreSQL is extremely reliable. We've found it to recover quite nicely, even from bad (human, sometimes testing-intentional!) behavior. Also doesn't sound like a version-specific problem.
Lou Picciano
----- Original Message -----
From: "Theodotos Andreou" <theo@theo-andreou.org>
To: pgsql-admin@postgresql.org
Sent: Wednesday, February 12, 2014 12:59:22 AM
Subject: [ADMIN] Error: pid file is invalid, please manually kill the stale server process.
Hi all,
I am using postgres on Ubuntu 12.04 64 bit server. The version is 9.1.10
and I used the stock postgres from Ubuntu repos.
I have a weird problem. Sometimes when the machine is restarted, there
is no $PGDATA/postmaster.pid file. So when I try to restart postgres
(service postgresql restart) I get the following error:
Error: pid file is invalid, please manually kill the stale server process.
An even scarier side-effect of this is that the data, when this happens,
are older than the most recent data in the database.
It may be important to know that postgres runs under monit non sysv init
or upstart.
Also postgres is installed on a customer appliance that may experience
frequent reboots and even abrupt poweroffs! Is postgres in general ideal
for this situation? Is there an optimal configuration for this scenario?
Regards
theo
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Error: pid file is invalid, please manually kill the stale server process.
@ 2014-02-13 08:39 Luca Ferrari <fluca1978@infinito.it>
parent: Theodotos Andreou <theo@theo-andreou.org>
1 sibling, 1 reply; 5+ messages in thread
From: Luca Ferrari @ 2014-02-13 08:39 UTC (permalink / raw)
To: Theodotos Andreou <theo@theo-andreou.org>; +Cc: pgsql-admin
On Wed, Feb 12, 2014 at 6:59 AM, Theodotos Andreou
<theo@theo-andreou.org> wrote:
> I have a weird problem. Sometimes when the machine is restarted, there is no
> $PGDATA/postmaster.pid file. So when I try to restart postgres (service
> postgresql restart) I get the following error:
>
It is not clear what the real problem is. After a machine reboot it is
normal that there is no pid file, meaning a clean shutdown happened
(from the PostgreSQL perspective). After a full start, if the
PostgreSQL process is configured for automatic start, a pid file
should exist. I suspect the problem is related to some ubuntu booting
script that will place the pid file somewhere else from where the
pg_ctl expects to find it.
> An even scarier side-effect of this is that the data, when this happens, are
> older than the most recent data in the database.
>
This does not make sense at all. How do you know that data is older?
And most notably: is data stable? I mean, could it be some kind of
snapshotting of the database filesystem (as other suggested) or sounds
like a data corruption?
> Also postgres is installed on a customer appliance that may experience
> frequent reboots and even abrupt poweroffs! Is postgres in general ideal for
> this situation? Is there an optimal configuration for this scenario?
PostgreSQL is reliable, but even a reliable system can crash when
running on a totally unreliable piece of hardware. I suggest investing
in some good platform to avoid stressing the database with resumes.
However, in such scenario, I will choose a sync of all commits and
short checkpoints, but this will lead to bad performances.
Luca
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Error: pid file is invalid, please manually kill the stale server process.
@ 2014-02-13 10:09 Theodotos Andreou <theo@theo-andreou.org>
parent: Luca Ferrari <fluca1978@infinito.it>
0 siblings, 1 reply; 5+ messages in thread
From: Theodotos Andreou @ 2014-02-13 10:09 UTC (permalink / raw)
To: pgsql-admin; +Cc: Luca Ferrari <fluca1978@infinito.it>; Lou Picciano <loupicciano@comcast.net>
Guy thanks for your replies.
You are right about the strange
I found the problem and is related to the peculiar setup we have on this
appliance. PGDATA exists on a mounted partition but we somehow managed
to create an older version of PGDATA under the mount point.
So sometimes postgres would load the data from the unmounted PGDATA and
sometimes from the mounted partition.
It' s a pretty unlikely scenario and I burned lots of braincells trying
to figure it out.
So not really a postgres problem but rather our peculiar setup.
On 02/13/2014 10:39 AM, Luca Ferrari wrote:
> On Wed, Feb 12, 2014 at 6:59 AM, Theodotos Andreou
> <theo@theo-andreou.org> wrote:
>> I have a weird problem. Sometimes when the machine is restarted, there is no
>> $PGDATA/postmaster.pid file. So when I try to restart postgres (service
>> postgresql restart) I get the following error:
>>
> It is not clear what the real problem is. After a machine reboot it is
> normal that there is no pid file, meaning a clean shutdown happened
> (from the PostgreSQL perspective). After a full start, if the
> PostgreSQL process is configured for automatic start, a pid file
> should exist. I suspect the problem is related to some ubuntu booting
> script that will place the pid file somewhere else from where the
> pg_ctl expects to find it.
>
>> An even scarier side-effect of this is that the data, when this happens, are
>> older than the most recent data in the database.
>>
> This does not make sense at all. How do you know that data is older?
> And most notably: is data stable? I mean, could it be some kind of
> snapshotting of the database filesystem (as other suggested) or sounds
> like a data corruption?
>
>
>> Also postgres is installed on a customer appliance that may experience
>> frequent reboots and even abrupt poweroffs! Is postgres in general ideal for
>> this situation? Is there an optimal configuration for this scenario?
> PostgreSQL is reliable, but even a reliable system can crash when
> running on a totally unreliable piece of hardware. I suggest investing
> in some good platform to avoid stressing the database with resumes.
> However, in such scenario, I will choose a sync of all commits and
> short checkpoints, but this will lead to bad performances.
>
> Luca
>
>
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Error: pid file is invalid, please manually kill the stale server process.
@ 2014-02-13 17:05 Luca Ferrari <fluca1978@infinito.it>
parent: Theodotos Andreou <theo@theo-andreou.org>
0 siblings, 0 replies; 5+ messages in thread
From: Luca Ferrari @ 2014-02-13 17:05 UTC (permalink / raw)
To: Theodotos Andreou <theo@theo-andreou.org>; +Cc: pgsql-admin
On Thu, Feb 13, 2014 at 11:09 AM, Theodotos Andreou
<theo@theo-andreou.org> wrote:
> So sometimes postgres would load the data from the unmounted PGDATA and
> sometimes from the mounted partition.
Stackable mount points....I had to fight against them for a while!
It happened to me to have the same problem a lot of times, even if not
related to database.
Good luck.
Luca
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2014-02-13 17:05 UTC | newest]
Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2014-02-12 05:59 Error: pid file is invalid, please manually kill the stale server process. Theodotos Andreou <theo@theo-andreou.org>
2014-02-13 04:37 ` Lou Picciano <loupicciano@comcast.net>
2014-02-13 08:39 ` Luca Ferrari <fluca1978@infinito.it>
2014-02-13 10:09 ` Theodotos Andreou <theo@theo-andreou.org>
2014-02-13 17:05 ` Luca Ferrari <fluca1978@infinito.it>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox