pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
postmaster hangs
7+ messages / 4 participants
[nested] [flat]

* postmaster hangs
@ 2005-04-19 14:38 Rolf Staege <Rolf.Staege@med.uni-rostock.de>
  2005-04-19 14:57 ` Re: postmaster hangs Tom Lane <tgl@sss.pgh.pa.us>
  2005-04-19 15:58 ` Re: postmaster hangs Scott Marlowe <smarlowe@g2switchworks.com>
  0 siblings, 2 replies; 7+ messages in thread

From: Rolf Staege @ 2005-04-19 14:38 UTC (permalink / raw)
  To: pgsql-admin

hi,
i'm using postgres version 7.4.1 and it run very stable until, may be,  
4 weeks ago.
now the postmaster hangs sometimes for a while (actually since 2 hours) 
and doesn't accept connections but comes back anytime. there is no 
possibility to stop the postmaster. even a kill -9 (it's only the 
developement system) doesn't work. i have to reboot the system to get 
the database running again.
actually ps brings:
postgres  2148     1  0  Mar 11  ?         0:28 
/usr/local/pgsql/bin/postmaster -i -d 2

top shows the postmaster process as follows:
 ?     2148 postgres 154 20   210M   244K run      0:28  0.02  0.02 
postmaster

the output of netstat is (grep for 5432):
tcp        0      0  192.168.143.116.5432    192.168.143.116.61145    
ESTABLISHED
tcp        0      0  192.168.143.116.61145   192.168.143.116.5432     
ESTABLISHED
tcp        0      0  *.5432                 *.*                     LISTEN
tcp        0      0  192.168.143.116.5432    192.168.143.114.2550     
ESTABLISHED
        45d91540 stream     38      0                0                
0                0                0 /tmp/.s.PGSQL.5432
        45d916c0 stream     39      0                0                
0                0                0 /tmp/.s.PGSQL.5432
        40ed2680 stream     38      0                0         
43191e80                0                0 /tmp/.s.PGSQL.5432
        40ed2b00 stream     38      0                0         
4458a400                0                0 /tmp/.s.PGSQL.5432
        440c3b80 stream     37      0                0                
0                0                0 /tmp/.s.PGSQL.5432
        440c3d00 stream     37      0                0                
0                0                0 /tmp/.s.PGSQL.5432
        449e5100 stream     37      0                0                
0                0                0 /tmp/.s.PGSQL.5432
        43c65240 stream     37      0                0         
4458a280                0                0 /tmp/.s.PGSQL.5432
        43a15200 stream      0      0         43b48c40                
0                0                0 /tmp/.s.PGSQL.5432
        437db7c0 stream     37      0                0                
0                0                0 /tmp/.s.PGSQL.5432
        437dbc40 stream     37      0                0                
0                0                0 /tmp/.s.PGSQL.5432

the server is a pa-risc system with a HP-UX 11.11, 64Bit
does anybody has an idea, what could be the reason for it?
thanks
rolf



^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* Re: postmaster hangs
  2005-04-19 14:38 postmaster hangs Rolf Staege <Rolf.Staege@med.uni-rostock.de>
@ 2005-04-19 14:57 ` Tom Lane <tgl@sss.pgh.pa.us>
  1 sibling, 0 replies; 7+ messages in thread

From: Tom Lane @ 2005-04-19 14:57 UTC (permalink / raw)
  To: Rolf Staege <Rolf.Staege@med.uni-rostock.de>; +Cc: pgsql-admin

Rolf Staege <Rolf.Staege@med.uni-rostock.de> writes:
> i'm using postgres version 7.4.1 and it run very stable until, may be,  
> 4 weeks ago.
> now the postmaster hangs sometimes for a while (actually since 2 hours) 
> and doesn't accept connections but comes back anytime. there is no 
> possibility to stop the postmaster. even a kill -9 (it's only the 
> developement system) doesn't work. i have to reboot the system to get 
> the database running again.

Hardware problems?  Most of the cases I've heard of where kill -9 didn't
work, it was because the process was partway through a disk operation
and the kernel was holding off the signal until the I/O completed
... but there was something broken about the disk and so the operation
never completed.

In any case, there is *no* way for an unprivileged process like Postgres
to resist kill -9, so your problem is certainly at either the hardware
or kernel level.  I'd suggest talking to HP ...

			regards, tom lane



^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* Re: postmaster hangs
  2005-04-19 14:38 postmaster hangs Rolf Staege <Rolf.Staege@med.uni-rostock.de>
@ 2005-04-19 15:58 ` Scott Marlowe <smarlowe@g2switchworks.com>
  1 sibling, 0 replies; 7+ messages in thread

From: Scott Marlowe @ 2005-04-19 15:58 UTC (permalink / raw)
  To: Rolf Staege <Rolf.Staege@med.uni-rostock.de>; +Cc: pgsql-admin

On Tue, 2005-04-19 at 09:38, Rolf Staege wrote:
> hi,
> i'm using postgres version 7.4.1 and it run very stable until, may be,  
> 4 weeks ago.
> now the postmaster hangs sometimes for a while (actually since 2 hours) 
> and doesn't accept connections but comes back anytime. there is no 
> possibility to stop the postmaster. even a kill -9 (it's only the 
> developement system) doesn't work. i have to reboot the system to get 
> the database running again.

In addition to what Tom said (i.e. look at hardware problems) I'd also
recommend updating the latest update of your versions of pgsql, 7.4.7.



^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* postmaster hangs
@ 2006-01-04 11:33 Tsirkin Evgeny <tsurkin@mail.jct.ac.il>
  2006-01-04 11:38 ` Re: postmaster hangs Tsirkin Evgeny <tsurkin@mail.jct.ac.il>
  2006-01-04 15:25 ` Re: postmaster hangs Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 2 replies; 7+ messages in thread

From: Tsirkin Evgeny @ 2006-01-04 11:33 UTC (permalink / raw)
  To: pgsql-admin

Today i was trying to run an old ver. of an application on test server .
After a while i have got the pg hangs :
 9899 ?        S      0:00 postgres:  [local] SELECT
 9932 ?        S      0:00 postgres: [local] SELECT
 9933 ?        S      0:00 postgres: [local] startup
 9936 ?        S      0:00 postgres: [local] startup
 9941 ?        S      0:00 postgres:
 9974 ?        S      0:00 postgres: [local] startup
 9976 ?        S      0:00 postgres: [local] startup
 9978 ?        S      0:00 postgres: [local] startup
 9983 ?        S      0:00 postgres: [local] startup
 9989 ?        S      0:00 postgres:  template1 [local] startup
This a web application with apache connecting to post and doing things.
I have tryed to connect to the db with psql - nosuccess.
This the backtrace of 9989.
(gdb)
backtrace
#0  0xffffe410 in ?? ()
#1  0xbfffcc28 in ?? ()
#2  0x00000001 in ?? ()
#3  0x01488005 in ?? ()
#4  0x402ffffb in semop () from /lib/tls/libc.so.6
#5  0x08154d07 in PGSemaphoreLock ()
#6  0x08177ade in LWLockAcquire ()
#7  0x0816c7f2 in ReadBufferInternal ()
#8  0x08086eb0 in _bt_getbuf ()
#9  0x080882b6 in _bt_getroot ()
#10 0x0808ab62 in _bt_search ()
#11 0x0808aeca in _bt_first ()
#12 0x080894e0 in btgettuple ()
#13 0x081eba7b in FunctionCall2 ()
#14 0x08083f86 in index_getnext ()
#15 0x080836e4 in systable_getnext ()
#16 0x080f87ed in RelationBuildTriggers ()
#17 0x081e44ab in RelationCacheInitializePhase2 ()
#18 0x081f0ce5 in InitPostgres ()
#19 0x0817db30 in PostgresMain ()
#20 0x08157754 in ServerLoop ()
#21 0x08158d1e in PostmasterMain ()
#22 0x08128c21 in main ()

Version 7.4 something (does not remember and can't get it because can't
connect to the db).Trying to connect to the template also hangs up.
Please help.
Evgeny.



^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* Re: postmaster hangs
  2006-01-04 11:33 postmaster hangs Tsirkin Evgeny <tsurkin@mail.jct.ac.il>
@ 2006-01-04 11:38 ` Tsirkin Evgeny <tsurkin@mail.jct.ac.il>
  1 sibling, 0 replies; 7+ messages in thread

From: Tsirkin Evgeny @ 2006-01-04 11:38 UTC (permalink / raw)
  To: Tsirkin Evgeny <tsurkin@jct.ac.il>; +Cc: pgsql-admin

Forgot to mention this SUSE 9 with the default 2.6.5 kernel.
Logs does not show anything strange and post refuses to shutdown
gracefully or receive connections.Processes just hangs out there ... .

Evgeny.

On Wed, 4 Jan 2006, Tsirkin Evgeny wrote:

> Today i was trying to run an old ver. of an application on test server .
> After a while i have got the pg hangs :
>  9899 ?      S      0:00 postgres:  [local] SELECT
>  9932 ?      S      0:00 postgres: [local] SELECT
>  9933 ?      S      0:00 postgres: [local] startup
>  9936 ?      S      0:00 postgres: [local] startup
>  9941 ?      S      0:00 postgres:
>  9974 ?      S      0:00 postgres: [local] startup
>  9976 ?      S      0:00 postgres: [local] startup
>  9978 ?      S      0:00 postgres: [local] startup
>  9983 ?      S      0:00 postgres: [local] startup
>  9989 ?      S      0:00 postgres:  template1 [local] startup
> This a web application with apache connecting to post and doing things.
> I have tryed to connect to the db with psql - nosuccess.
> This the backtrace of 9989.
> (gdb)
> backtrace
> #00xffffe410 in ?? ()
> #10xbfffcc28 in ?? ()
> #20x00000001 in ?? ()
> #30x01488005 in ?? ()
> #40x402ffffb in semop () from /lib/tls/libc.so.6
> #50x08154d07 in PGSemaphoreLock ()
> #6 0x08177ade in LWLockAcquire ()
> #70x0816c7f2 in ReadBufferInternal ()
> #80x08086eb0 in _bt_getbuf ()
> #90x080882b6 in _bt_getroot ()
> #10 0x0808ab62 in _bt_search ()
> #11 0x0808aeca in _bt_first ()
> #12 0x080894e0 in btgettuple ()
> #13 0x081eba7b in FunctionCall2 ()
> #14 0x08083f86 in index_getnext ()
> #15 0x080836e4 in systable_getnext ()
> #16 0x080f87ed in RelationBuildTriggers ()
> #17 0x081e44ab in RelationCacheInitializePhase2 ()
> #18 0x081f0ce5 in InitPostgres ()
> #19 0x0817db30 in PostgresMain ()
> #20 0x08157754 in ServerLoop ()
> #21 0x08158d1e in PostmasterMain ()
> #22 0x08128c21 in main ()
>
> Version 7.4 something (does not remember and can't get it because can't
> connect to the db).Trying to connect to the template also hangs up.
> Please help.
> Evgeny.
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: if posting/reading through Usenet, please send an appropriate
>      subscribe-nomail command to majordomo@postgresql.org so that your
>      message can get through to the mailing list cleanly
>




^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* Re: postmaster hangs
  2006-01-04 11:33 postmaster hangs Tsirkin Evgeny <tsurkin@mail.jct.ac.il>
@ 2006-01-04 15:25 ` Tom Lane <tgl@sss.pgh.pa.us>
  2006-01-04 15:50   ` Re: postmaster hangs Tsirkin Evgeny <tsurkin@mail.jct.ac.il>
  1 sibling, 1 reply; 7+ messages in thread

From: Tom Lane @ 2006-01-04 15:25 UTC (permalink / raw)
  To: Tsirkin Evgeny <tsurkin@mail.jct.ac.il>; +Cc: pgsql-admin

Tsirkin Evgeny <tsurkin@mail.jct.ac.il> writes:
> Today i was trying to run an old ver. of an application on test server .
> After a while i have got the pg hangs :

It looks like someone is holding a critical LWLock --- hard to tell if
it's the BufMgrLock or a per-buffer lock for a system index's metapage,
but must be one or the other.  Try backtracing the processes that
*aren't* hung in this particular way.  I'm not sure you'll be able to
learn anything much given that the build doesn't have debug info, but
you might as well try.

			regards, tom lane



^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* Re: postmaster hangs
  2006-01-04 11:33 postmaster hangs Tsirkin Evgeny <tsurkin@mail.jct.ac.il>
  2006-01-04 15:25 ` Re: postmaster hangs Tom Lane <tgl@sss.pgh.pa.us>
@ 2006-01-04 15:50   ` Tsirkin Evgeny <tsurkin@mail.jct.ac.il>
  0 siblings, 0 replies; 7+ messages in thread

From: Tsirkin Evgeny @ 2006-01-04 15:50 UTC (permalink / raw)
  To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Tsirkin Evgeny <tsurkin@jct.ac.il>; pgsql-admin


sorry ,but i had to go with my job so i have killed one of the backends 
and this made the post restart.
However befor this i have done another backtrace ,the one that was hang 
in the middle of SELECT (not startup)
and got this:
#0  0xffffe410 in ?? ()
#1  0xbfffcf08 in ?? ()
#2  0x00000001 in ?? ()
#3  0x01490006 in ?? ()
#4  0x402ffffb in semop () from /lib/tls/libc.so.6
#5  0x08154d07 in PGSemaphoreLock ()
#6  0x08177ade in LWLockAcquire ()
#7  0x0816c7f2 in ReadBufferInternal ()
#8  0x08086eb0 in _bt_getbuf ()
#9  0x080882b6 in _bt_getroot ()
#10 0x0808ab62 in _bt_search ()
#11 0x0808aeca in _bt_first ()
#12 0x080894e0 in btgettuple ()
#13 0x081eba7b in FunctionCall2 ()
#14 0x08083f86 in index_getnext ()
#15 0x081174a0 in IndexNext ()
#16 0x0811139f in ExecScan ()
#17 0x081173c8 in ExecIndexScan ()
#18 0x0810d34d in ExecProcNode ()
#19 0x0810b9ef in ExecutorRun ()
#20 0x0817f7f3 in PortalRunSelect ()
#21 0x08180507 in PortalRun ()
#22 0x0817c5b4 in exec_simple_query ()
#23 0x0817e4d3 in PostgresMain ()
#24 0x08157754 in ServerLoop ()
#25 0x08158d1e in PostmasterMain ()
#26 0x08128c21 in main ()
I did not understand from your message wheither this is a problem in pg 
or in application.
The exact version of pg is postgres (PostgreSQL) 7.4.8.
Thanks.
Tom Lane wrote:

>Tsirkin Evgeny <tsurkin@mail.jct.ac.il> writes:
>  
>
>>Today i was trying to run an old ver. of an application on test server .
>>After a while i have got the pg hangs :
>>    
>>
>
>It looks like someone is holding a critical LWLock --- hard to tell if
>it's the BufMgrLock or a per-buffer lock for a system index's metapage,
>but must be one or the other.  Try backtracing the processes that
>*aren't* hung in this particular way.  I'm not sure you'll be able to
>learn anything much given that the build doesn't have debug info, but
>you might as well try.
>
>			regards, tom lane
>  
>




^ permalink  raw  reply  [nested|flat] 7+ messages in thread


end of thread, other threads:[~2006-01-04 15:50 UTC | newest]

Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2005-04-19 14:38 postmaster hangs Rolf Staege <Rolf.Staege@med.uni-rostock.de>
2005-04-19 14:57 ` Tom Lane <tgl@sss.pgh.pa.us>
2005-04-19 15:58 ` Scott Marlowe <smarlowe@g2switchworks.com>
2006-01-04 11:33 postmaster hangs Tsirkin Evgeny <tsurkin@mail.jct.ac.il>
2006-01-04 11:38 ` Tsirkin Evgeny <tsurkin@mail.jct.ac.il>
2006-01-04 15:25 ` Tom Lane <tgl@sss.pgh.pa.us>
2006-01-04 15:50   ` Tsirkin Evgeny <tsurkin@mail.jct.ac.il>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox