pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
Postgres process terminating
5+ messages / 3 participants
[nested] [flat]

* Postgres process terminating
@ 2006-03-14 14:54 Cory Vokey <cvokey@gmail.com>
  2006-03-14 20:05 ` Re: Postgres process terminating Guido Barosio <gbarosio@gmail.com>
  0 siblings, 1 reply; 5+ messages in thread

From: Cory Vokey @ 2006-03-14 14:54 UTC (permalink / raw)
  To: pgsql-admin

Hi all, I'm currently running postgres-7.4.12 with slony-1.0.5 and am seeing
the below postgres errors:

Mar 14 00:00:25 master postgres[27357]: [12-1] FATAL:  57P03: the database
system is starting up
Mar 14 00:00:25 master postgres[27357]: [12-2] LOCATION:
ProcessStartupPacket, postmaster.c:1437
Mar 14 00:00:25 master postgres[27356]: [17-1] LOG:  00000: unexpected
pageaddr 1/55F4A000 in log file 1, segment 183, offset 16031744
Mar 14 00:00:25 master postgres[27356]: [17-2] LOCATION:  ValidXLOGHeader,
xlog.c:2096
Mar 14 00:00:25 master postgres[27356]: [18-1] LOG:  00000: redo done at
1/B7F49220
Mar 14 00:00:25 master postgres[27356]: [18-2] LOCATION:  StartupXLOG,
xlog.c:2796
Mar 14 00:00:26 master postgres[27386]: [11-1] LOG:  00000: connection
received: host=172.20.42.19 port=54294
Mar 14 00:00:26 master postgres[27386]: [11-2] LOCATION:  BackendFork,
postmaster.c:2395
Mar 14 00:00:26 master postgres[27386]: [12-1] FATAL:  57P03: the database
system is starting up
Mar 14 00:00:26 master postgres[27386]: [12-2] LOCATION:
ProcessStartupPacket, postmaster.c:1437
Mar 14 00:00:26 master postgres[27387]: [11-1] LOG:  00000: connection
received: host=172.20.42.19 port=54295
Mar 14 00:00:26 master postgres[27387]: [11-2] LOCATION:  BackendFork,
postmaster.c:2395
Mar 14 00:00:26 master postgres[27387]: [12-1] FATAL:  57P03: the database
system is starting up
Mar 14 00:00:26 master postgres[27387]: [12-2] LOCATION:
ProcessStartupPacket, postmaster.c:1437
Mar 14 00:00:26 master postgres[27388]: [11-1] LOG:  00000: connection
received: host=172.20.42.20 port=38111
Mar 14 00:00:26 master postgres[27388]: [11-2] LOCATION:  BackendFork,
postmaster.c:2395
Mar 14 00:00:26 master postgres[27388]: [12-1] FATAL:  57P03: the database
system is starting up
Mar 14 00:00:26 master postgres[27388]: [12-2] LOCATION:
ProcessStartupPacket, postmaster.c:1437
Mar 14 00:00:26 master postgres[27389]: [11-1] LOG:  00000: connection
received: host=172.20.42.20 port=38112
Mar 14 00:00:26 master postgres[27389]: [11-2] LOCATION:  BackendFork,
postmaster.c:2395
Mar 14 00:00:26 master postgres[27389]: [12-1] FATAL:  57P03: the database
system is starting up
Mar 14 00:00:26 master postgres[27389]: [12-2] LOCATION:
ProcessStartupPacket, postmaster.c:1437
Mar 14 00:00:26 master postgres[27392]: [11-1] LOG:  00000: connection
received: host=172.20.42.19 port=54297
Mar 14 00:00:26 master postgres[27392]: [11-2] LOCATION:  BackendFork,
postmaster.c:2395
Mar 14 00:00:26 master postgres[27392]: [12-1] FATAL:  57P03: the database
system is starting up
Mar 14 00:00:26 master postgres[27392]: [12-2] LOCATION:
ProcessStartupPacket, postmaster.c:1437
Mar 14 00:00:26 master postgres[27393]: [11-1] LOG:  00000: connection
received: host=172.20.42.19 port=54298
Mar 14 00:00:26 master postgres[27393]: [11-2] LOCATION:  BackendFork,
postmaster.c:2395
Mar 14 00:00:26 master postgres[27393]: [12-1] FATAL:  57P03: the database
system is starting up

This causes slony to die on all clusters as well as our web applications.

In google'ing for "FATAL:  57P03: the database system is starting up" I
noticed other people having similar issues but I not been able to locate a
fix.

This morning I dropped all slony schema's  from the database and am
continuing to re-create my clusters one by one and have not seeing these
errors thus far.

What I am interested in is a description of the above error log.

Any help is appreciated!

Thanks.


--
Cory Vokey

^ permalink  raw  reply  [nested|flat] 5+ messages in thread

* Re: Postgres process terminating
  2006-03-14 14:54 Postgres process terminating Cory Vokey <cvokey@gmail.com>
@ 2006-03-14 20:05 ` Guido Barosio <gbarosio@gmail.com>
  2006-03-14 20:12   ` Re: Postgres process terminating Scott Marlowe <smarlowe@g2switchworks.com>
  0 siblings, 1 reply; 5+ messages in thread

From: Guido Barosio @ 2006-03-14 20:05 UTC (permalink / raw)
  To: Cory Vokey <cvokey@gmail.com>; +Cc: pgsql-admin

Hi Cory,

Actually it seems that the database wasn't shut down properly, and
while it was doing the recovery, it refused connections. Expected and
what needs to happen, indeed.

That's why if you are a client about to connect, you'll recieve the "
FATAL ...database is starting up " message.

Ideally, you (the dba) should let the server end this task (the
recovery). AFAIK, the recovery started as a consecuence of dirty data
somewere, probably a big transaction not comitted when the server went
off.

Are you aware of a shutdown prior to this messages?

Nothing to do with your current issue, but you will receive comments
related with your pgsql version if you are not using a  pgsql 8 :)

Regards
Guido Barosio.
( Shall I confess that I still have some 7.4.2 instances running in production)

On 3/14/06, Cory Vokey <cvokey@gmail.com> wrote:
> Hi all, I'm currently running postgres-7.4.12 with slony-1.0.5 and am seeing
> the below postgres errors:
>
>  Mar 14 00:00:25 master postgres[27357]: [12-1] FATAL:  57P03: the database
> system is starting up
>  Mar 14 00:00:25 master postgres[27357]: [12-2] LOCATION:
> ProcessStartupPacket, postmaster.c:1437
>  Mar 14 00:00:25 master postgres[27356]: [17-1] LOG:  00000: unexpected
> pageaddr 1/55F4A000 in log file 1, segment 183, offset 16031744
>  Mar 14 00:00:25 master postgres[27356]: [17-2] LOCATION:  ValidXLOGHeader,
> xlog.c:2096
>  Mar 14 00:00:25 master postgres[27356]: [18-1] LOG:  00000: redo done at
> 1/B7F49220
>  Mar 14 00:00:25 master postgres[27356]: [18-2] LOCATION:  StartupXLOG,
> xlog.c:2796
>  Mar 14 00:00:26 master postgres[27386]: [11-1] LOG:  00000: connection
> received: host=172.20.42.19 port=54294
>  Mar 14 00:00:26 master postgres[27386]: [11-2] LOCATION:  BackendFork,
> postmaster.c:2395
>  Mar 14 00:00:26 master postgres[27386]: [12-1] FATAL:  57P03: the database
> system is starting up
>  Mar 14 00:00:26 master postgres[27386]: [12-2] LOCATION:
> ProcessStartupPacket, postmaster.c:1437
>  Mar 14 00:00:26 master postgres[27387]: [11-1] LOG:  00000: connection
> received: host=172.20.42.19 port=54295
>  Mar 14 00:00:26 master postgres[27387]: [11-2] LOCATION:  BackendFork,
> postmaster.c:2395
>  Mar 14 00:00:26 master postgres[27387]: [12-1] FATAL:  57P03: the database
> system is starting up
>  Mar 14 00:00:26 master postgres[27387]: [12-2] LOCATION:
> ProcessStartupPacket, postmaster.c:1437
>  Mar 14 00:00:26 master postgres[27388]: [11-1] LOG:  00000: connection
> received: host=172.20.42.20 port=38111
>  Mar 14 00:00:26 master postgres[27388]: [11-2] LOCATION:  BackendFork,
> postmaster.c:2395
>  Mar 14 00:00:26 master postgres[27388]: [12-1] FATAL:  57P03: the database
> system is starting up
>  Mar 14 00:00:26 master postgres[27388]: [12-2] LOCATION:
> ProcessStartupPacket, postmaster.c:1437
>  Mar 14 00:00:26 master postgres[27389]: [11-1] LOG:  00000: connection
> received: host=172.20.42.20 port=38112
>  Mar 14 00:00:26 master postgres[27389]: [11-2] LOCATION:  BackendFork,
> postmaster.c:2395
>  Mar 14 00:00:26 master postgres[27389]: [12-1] FATAL:  57P03: the database
> system is starting up
>  Mar 14 00:00:26 master postgres[27389]: [12-2] LOCATION:
> ProcessStartupPacket, postmaster.c:1437
>  Mar 14 00:00:26 master postgres[27392]: [11-1] LOG:  00000: connection
> received: host=172.20.42.19 port=54297
>  Mar 14 00:00:26 master postgres[27392]: [11-2] LOCATION:  BackendFork,
> postmaster.c:2395
>  Mar 14 00:00:26 master postgres[27392]: [12-1] FATAL:  57P03: the database
> system is starting up
>  Mar 14 00:00:26 master postgres[27392]: [12-2] LOCATION:
> ProcessStartupPacket, postmaster.c:1437
>  Mar 14 00:00:26 master postgres[27393]: [11-1] LOG:  00000: connection
> received: host=172.20.42.19 port=54298
>  Mar 14 00:00:26 master postgres[27393]: [11-2] LOCATION:  BackendFork,
> postmaster.c:2395
>  Mar 14 00:00:26 master postgres[27393]: [12-1] FATAL:  57P03: the database
> system is starting up
>
>  This causes slony to die on all clusters as well as our web applications.
>
>  In google'ing for "FATAL:  57P03: the database system is starting up" I
> noticed other people having similar issues but I not been able to locate a
> fix.
>
>  This morning I dropped all slony schema's  from the database and am
> continuing to re-create my clusters one by one and have not seeing these
> errors thus far.
>
>  What I am interested in is a description of the above error log.
>
>  Any help is appreciated!
>
>  Thanks.
>
>
> --
> Cory Vokey


--
/"\   ASCII Ribbon Campaign  .
\ / - NO HTML/RTF in e-mail  .
 X  - NO Word docs in e-mail .
/ \ -----------------------------------------------------------------



^ permalink  raw  reply  [nested|flat] 5+ messages in thread

* Re: Postgres process terminating
  2006-03-14 14:54 Postgres process terminating Cory Vokey <cvokey@gmail.com>
  2006-03-14 20:05 ` Re: Postgres process terminating Guido Barosio <gbarosio@gmail.com>
@ 2006-03-14 20:12   ` Scott Marlowe <smarlowe@g2switchworks.com>
  2006-03-14 20:17     ` Re: Postgres process terminating Guido Barosio <gbarosio@gmail.com>
  0 siblings, 1 reply; 5+ messages in thread

From: Scott Marlowe @ 2006-03-14 20:12 UTC (permalink / raw)
  To: Guido Barosio <gbarosio@gmail.com>; +Cc: Cory Vokey <cvokey@gmail.com>; pgsql-admin

On Tue, 2006-03-14 at 14:05, Guido Barosio wrote:
> Hi Cory,
> 
> Actually it seems that the database wasn't shut down properly, and
> while it was doing the recovery, it refused connections. Expected and
> what needs to happen, indeed.
> 
> That's why if you are a client about to connect, you'll recieve the "
> FATAL ...database is starting up " message.
> 
> Ideally, you (the dba) should let the server end this task (the
> recovery). AFAIK, the recovery started as a consecuence of dirty data
> somewere, probably a big transaction not comitted when the server went
> off.
> 
> Are you aware of a shutdown prior to this messages?
> 
> Nothing to do with your current issue, but you will receive comments
> related with your pgsql version if you are not using a  pgsql 8 :)

As mentioned, this is normal operation.

Note that the command:

pg_ctl -m immediate stop

will cause a recovery.  It's pretty much a functional equivalent of
senting kill -9 to all backends and the postmaster at once.  Brutal but
effective.



^ permalink  raw  reply  [nested|flat] 5+ messages in thread

* Re: Postgres process terminating
  2006-03-14 14:54 Postgres process terminating Cory Vokey <cvokey@gmail.com>
  2006-03-14 20:05 ` Re: Postgres process terminating Guido Barosio <gbarosio@gmail.com>
  2006-03-14 20:12   ` Re: Postgres process terminating Scott Marlowe <smarlowe@g2switchworks.com>
@ 2006-03-14 20:17     ` Guido Barosio <gbarosio@gmail.com>
  2006-03-15 02:31       ` Re: Postgres process terminating Cory Vokey <cvokey@gmail.com>
  0 siblings, 1 reply; 5+ messages in thread

From: Guido Barosio @ 2006-03-14 20:17 UTC (permalink / raw)
  To: Scott Marlowe <smarlowe@g2switchworks.com>; +Cc: Cory Vokey <cvokey@gmail.com>; pgsql-admin

Indeed, from the pg_ctl man pages:

``Immediate'' mode will abort all server processes without a clean
shutdown. This will lead to a recovery run on restart.

Regards,
Guido Barosio

On 3/14/06, Scott Marlowe <smarlowe@g2switchworks.com> wrote:
> On Tue, 2006-03-14 at 14:05, Guido Barosio wrote:
> > Hi Cory,
> >
> > Actually it seems that the database wasn't shut down properly, and
> > while it was doing the recovery, it refused connections. Expected and
> > what needs to happen, indeed.
> >
> > That's why if you are a client about to connect, you'll recieve the "
> > FATAL ...database is starting up " message.
> >
> > Ideally, you (the dba) should let the server end this task (the
> > recovery). AFAIK, the recovery started as a consecuence of dirty data
> > somewere, probably a big transaction not comitted when the server went
> > off.
> >
> > Are you aware of a shutdown prior to this messages?
> >
> > Nothing to do with your current issue, but you will receive comments
> > related with your pgsql version if you are not using a  pgsql 8 :)
>
> As mentioned, this is normal operation.
>
> Note that the command:
>
> pg_ctl -m immediate stop
>
> will cause a recovery.  It's pretty much a functional equivalent of
> senting kill -9 to all backends and the postmaster at once.  Brutal but
> effective.
>


--
/"\   ASCII Ribbon Campaign  .
\ / - NO HTML/RTF in e-mail  .
 X  - NO Word docs in e-mail .
/ \ -----------------------------------------------------------------



^ permalink  raw  reply  [nested|flat] 5+ messages in thread

* Re: Postgres process terminating
  2006-03-14 14:54 Postgres process terminating Cory Vokey <cvokey@gmail.com>
  2006-03-14 20:05 ` Re: Postgres process terminating Guido Barosio <gbarosio@gmail.com>
  2006-03-14 20:12   ` Re: Postgres process terminating Scott Marlowe <smarlowe@g2switchworks.com>
  2006-03-14 20:17     ` Re: Postgres process terminating Guido Barosio <gbarosio@gmail.com>
@ 2006-03-15 02:31       ` Cory Vokey <cvokey@gmail.com>
  0 siblings, 0 replies; 5+ messages in thread

From: Cory Vokey @ 2006-03-15 02:31 UTC (permalink / raw)
  To: Guido Barosio <gbarosio@gmail.com>; +Cc: Scott Marlowe <smarlowe@g2switchworks.com>; pgsql-admin

On 3/14/06, Guido Barosio <gbarosio@gmail.com> wrote:
>
> Indeed, from the pg_ctl man pages:
>
> ``Immediate'' mode will abort all server processes without a clean
> shutdown. This will lead to a recovery run on restart.
>
> Regards,
> Guido Barosio
>
> On 3/14/06, Scott Marlowe <smarlowe@g2switchworks.com> wrote:
> > On Tue, 2006-03-14 at 14:05, Guido Barosio wrote:
> > > Hi Cory,
> > >
> > > Actually it seems that the database wasn't shut down properly, and
> > > while it was doing the recovery, it refused connections. Expected and
> > > what needs to happen, indeed.
> > >
> > > That's why if you are a client about to connect, you'll recieve the "
> > > FATAL ...database is starting up " message.
> > >
> > > Ideally, you (the dba) should let the server end this task (the
> > > recovery). AFAIK, the recovery started as a consecuence of dirty data
> > > somewere, probably a big transaction not comitted when the server went
> > > off.
> > >
> > > Are you aware of a shutdown prior to this messages?
> > >
> > > Nothing to do with your current issue, but you will receive comments
> > > related with your pgsql version if you are not using a  pgsql 8 :)
> >
> > As mentioned, this is normal operation.
> >
> > Note that the command:
> >
> > pg_ctl -m immediate stop
> >
> > will cause a recovery.  It's pretty much a functional equivalent of
> > senting kill -9 to all backends and the postmaster at once.  Brutal but
> > effective.
> >
>
>
> --
> /"\   ASCII Ribbon Campaign  .
> \ / - NO HTML/RTF in e-mail  .
> X  - NO Word docs in e-mail .
> / \ -----------------------------------------------------------------
>

Ok, thanks a lot guys.

It seems that I should have brought the database down to "local access" mode
via pg_hba.conf and allow all transactions to fully recover before starting
up replication or allowing other applications to access the db.

Have a good night all!

--
Cory Vokey

^ permalink  raw  reply  [nested|flat] 5+ messages in thread


end of thread, other threads:[~2006-03-15 02:31 UTC | newest]

Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2006-03-14 14:54 Postgres process terminating Cory Vokey <cvokey@gmail.com>
2006-03-14 20:05 ` Guido Barosio <gbarosio@gmail.com>
2006-03-14 20:12   ` Scott Marlowe <smarlowe@g2switchworks.com>
2006-03-14 20:17     ` Guido Barosio <gbarosio@gmail.com>
2006-03-15 02:31       ` Cory Vokey <cvokey@gmail.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox