pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedsemop hanging - Postgres 9.4.4
9+ messages / 4 participants
[nested] [flat]
* semop hanging - Postgres 9.4.4
@ 2015-12-15 19:34 Michał Nowak <minowack@wp.pl>
0 siblings, 2 replies; 9+ messages in thread
From: Michał Nowak @ 2015-12-15 19:34 UTC (permalink / raw)
To: pgsql-admin
Hello,
Since couple of days on my Postgres cluster i've seen few times a hanged process. It looks like one of postgres workers stuck on simple INSERT query.
PID USERNAME PRI NICE SIZE RES STATE TIME WCPU CPU COMMAND
12173 postgres 20 0 1372M 345M run 166:18 24.47% 99.90% postgres: db01 db01 10.0.0.101(58713) INSERT
It utilizes 100% of one CPU and after 1-2 hours it starts to utilize memory. After another 1-2 hours OOM-Killer have to take action. At the same time other postgres workers are properly processing queries.
There's no error on syslog/dmesg. Strace:
semop(58261738, {{12, -1, 0}}, 1) = 0
semop(58261738, {{12, -1, 0}}, 1) = 0
semop(58261738, {{12, -1, 0}}, 1) = 0
semop(58851580, {{0, 1, 0}}, 1) = 0
semop(58261738, {{12, -1, 0}}, 1) = 0
semop(58261738, {{12, -1, 0}}, 1) = 0
semop(58261738, {{12, -1, 0}}, 1) = 0
semop(57802972, {{1, 1, 0}}, 1) = 0
select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
semop(58261738, {{12, -1, 0}}, 1) = 0
semop(58261738, {{12, -1, 0}}, 1) = 0
semop(58261738, {{12, -1, 0}}, 1) = 0
Is this related to incorrect semaphores configuration?
Here's my current configuration:
sql01:~# ipcs -l
------ Shared Memory Limits --------
max number of segments = 4096
max seg size (kbytes) = 18014398509465599
max total shared memory (kbytes) = 18446744073642442748
min seg size (bytes) = 1
------ Semaphore Limits --------
max number of arrays = 2048
max semaphores per array = 250
max semaphores system wide = 512000
max ops per semop call = 100
semaphore max value = 32767
------ Messages Limits --------
max queues system wide = 32000
max size of message (bytes) = 8192
default max size of queue (bytes) = 16384
sql01:~# ipcs -u
------ Shared Memory Status --------
segments allocated 1
pages allocated 1
pages resident 1
pages swapped 0
Swap performance: 0 attempts 0 successes
------ Semaphore Status --------
used arrays = 257
allocated semaphores = 4369
------ Messages Status --------
allocated queues = 0
used headers = 0
used space = 0 bytes
Some values from postgresql.conf:
max_connections = 4096
effective_cache_size = 12GB
work_mem = 16MB
shared_buffers = 1GB
postgres=# select version();
version
------------------------------------------------------------------------------------------------------
PostgreSQL 9.4.4 on x86_64-unknown-linux-gnu, compiled by gcc (Ubuntu 4.8.2-19ubuntu1) 4.8.2, 64-bit
I would appreciate any help.
Best Regards,
Marcin
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: semop hanging - Postgres 9.4.4
@ 2015-12-16 10:44 Shreeyansh Dba <shreeyansh2014@gmail.com>
parent: Michał Nowak <minowack@wp.pl>
1 sibling, 0 replies; 9+ messages in thread
From: Shreeyansh Dba @ 2015-12-16 10:44 UTC (permalink / raw)
To: Michał Nowak <minowack@wp.pl>; +Cc: pgsql-admin
On Wed, Dec 16, 2015 at 1:04 AM, Michał Nowak <minowack@wp.pl> wrote:
> Hello,
>
> Since couple of days on my Postgres cluster i've seen few times a hanged
> process. It looks like one of postgres workers stuck on simple INSERT query.
>
> PID USERNAME PRI NICE SIZE RES STATE TIME WCPU CPU COMMAND
> 12173 postgres 20 0 1372M 345M run 166:18 24.47% 99.90% postgres:
> db01 db01 10.0.0.101(58713) INSERT
>
>
> It utilizes 100% of one CPU and after 1-2 hours it starts to utilize
> memory. After another 1-2 hours OOM-Killer have to take action. At the same
> time other postgres workers are properly processing queries.
>
> There's no error on syslog/dmesg. Strace:
>
>
> semop(58261738, {{12, -1, 0}}, 1) = 0
> semop(58261738, {{12, -1, 0}}, 1) = 0
> semop(58261738, {{12, -1, 0}}, 1) = 0
> semop(58851580, {{0, 1, 0}}, 1) = 0
> semop(58261738, {{12, -1, 0}}, 1) = 0
> semop(58261738, {{12, -1, 0}}, 1) = 0
> semop(58261738, {{12, -1, 0}}, 1) = 0
> semop(57802972, {{1, 1, 0}}, 1) = 0
> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> semop(58261738, {{12, -1, 0}}, 1) = 0
> semop(58261738, {{12, -1, 0}}, 1) = 0
> semop(58261738, {{12, -1, 0}}, 1) = 0
>
>
> Is this related to incorrect semaphores configuration?
>
>
>
> Here's my current configuration:
>
> sql01:~# ipcs -l
>
> ------ Shared Memory Limits --------
> max number of segments = 4096
> max seg size (kbytes) = 18014398509465599
> max total shared memory (kbytes) = 18446744073642442748
> min seg size (bytes) = 1
>
> ------ Semaphore Limits --------
> max number of arrays = 2048
> max semaphores per array = 250
> max semaphores system wide = 512000
> max ops per semop call = 100
> semaphore max value = 32767
>
> ------ Messages Limits --------
> max queues system wide = 32000
> max size of message (bytes) = 8192
> default max size of queue (bytes) = 16384
>
>
> sql01:~# ipcs -u
>
> ------ Shared Memory Status --------
> segments allocated 1
> pages allocated 1
> pages resident 1
> pages swapped 0
> Swap performance: 0 attempts 0 successes
>
> ------ Semaphore Status --------
> used arrays = 257
> allocated semaphores = 4369
>
> ------ Messages Status --------
> allocated queues = 0
> used headers = 0
> used space = 0 bytes
>
>
> Some values from postgresql.conf:
> max_connections = 4096
> effective_cache_size = 12GB
> work_mem = 16MB
> shared_buffers = 1GB
>
>
> postgres=# select version();
> version
>
> ------------------------------------------------------------------------------------------------------
> PostgreSQL 9.4.4 on x86_64-unknown-linux-gnu, compiled by gcc (Ubuntu
> 4.8.2-19ubuntu1) 4.8.2, 64-bit
>
>
> I would appreciate any help.
>
> Best Regards,
>
> Marcin
>
>
>
>
>
> --
> Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-admin
>
Hi Miachal,
May I know what is the RAM of server and how much shmmax is configured ?
You can check if any Zombi or defunct process are running on the server.
I want to know if any maintenance activity performed same time?
Regards,
Sagar Jadhav
Database Administrator
www.shreeyansh.com
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: semop hanging - Postgres 9.4.4
@ 2015-12-16 11:31 Michał Nowak <minowack@wp.pl>
0 siblings, 0 replies; 9+ messages in thread
From: Michał Nowak @ 2015-12-16 11:31 UTC (permalink / raw)
To: pgsql-admin
Hello,
> There is no zombi nor defunct process. Now i have 3 stucked processes.
When i try to pg_terminate_backend nothing happens:
strace:
semop(57802972, {{9, -1, 0}}, 1) = 0
semop(57802972, {{9, -1, 0}}, 1) = 0
select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
semop(57802972, {{9, -1, 0}}, 1) = 0
--- SIGTERM {si_signo=SIGTERM, si_code=SI_USER, si_pid=4431, si_uid=105} ---
rt_sigreturn() = 139718030872480
select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
semop(57802972, {{9, -1, 0}}, 1) = 0
semop(57802972, {{9, -1, 0}}, 1) = 0
select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
Best Regards,
Michal
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: semop hanging - Postgres 9.4.4
@ 2015-12-16 11:50 Shreeyansh Dba <shreeyansh2014@gmail.com>
1 sibling, 1 reply; 9+ messages in thread
From: Shreeyansh Dba @ 2015-12-16 11:50 UTC (permalink / raw)
To: Michał Nowak <minowack@wp.pl>; +Cc: pgsql-admin
Hi Michal,
Sometimes terminating a query via normal methods of pg_cancel_backend() and
pg_terminate_backend() fail and additional steps need to be taken.
ps -eflyCpostgres |grep PID(stuck pid)
OS level kill is not recommended but we don't have alternate solution in
some scenarios we need to from OS
kill -11 'stuck pid'
You could decide shmmax depending upon the size of DB. We recommend to set
shmmax to 6 GB and increase shared_buffers size to 2 GB.
Best regards,
Sagar(DBA)
Shreeyansh Technologies.
On Wed, Dec 16, 2015 at 4:44 PM, Michał Nowak <minowack@wp.pl> wrote:
> Hi Sagar,
>
> > May I know what is the RAM of server and how much shmmax is configured ?
> > You can check if any Zombi or defunct process are running on the server.
> > I want to know if any maintenance activity performed same time?
>
> Thanks for you reply.
>
> Server has 16GB of RAM. Kernel shmmax has strange value, i think it's
> default on Ubuntu - i haven't changed that.
>
> kernel.shmall = 18446744073692774399
> kernel.shmmax = 18446744073692774399
> kernel.shmmni = 4096
>
> I changed kermel.sem from defaults to: 250 512000 100 2048
>
> There is no zombi nor defunct process. Now i have 3 stucked processes.
>
> First:
>
> psql01:~# strace -ffp 14135
> Process 14135 attached
> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> semop(58261738, {{12, -1, 0}}, 1) = 0
> semop(58261738, {{12, -1, 0}}, 1) = 0
> semop(58261738, {{12, -1, 0}}, 1) = 0
> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> semop(58261738, {{12, -1, 0}}, 1) = 0
> semop(58261738, {{12, -1, 0}}, 1) = 0
> semop(58261738, {{12, -1, 0}}, 1) = 0
> semop(58261738, {{12, -1, 0}}, 1) = 0
> semop(58261738, {{12, -1, 0}}, 1) = 0
> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>
>
> Second:
>
> psql01:~# strace -ffp 12712
> Process 12712 attached
> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> semop(57737434, {{6, 1, 0}}, 1) = 0
> semop(57934048, {{2, -1, 0}}, 1) = 0
> semop(57802972, {{7, 1, 0}}, 1) = 0
> semop(57934048, {{2, -1, 0}}, 1) = 0
> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> semop(57934048, {{2, -1, 0}}, 1) = 0
> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> semop(57934048, {{2, -1, 0}}, 1) = 0
> semop(57934048, {{2, -1, 0}}, 1) = 0
>
>
> Third:
>
> psql02:~# strace -ffp 18283
> Process 18283 attached
> semop(58523890, {{11, 1, 0}}, 1) = 0
> semop(57802972, {{9, -1, 0}}, 1) = 0
> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> semop(57802972, {{9, -1, 0}}, 1) = 0
> semop(58818811, {{13, 1, 0}}, 1) = 0
> semop(57802972, {{9, -1, 0}}, 1) = 0
> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> semop(57802972, {{9, -1, 0}}, 1) = 0
> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>
>
> perf top:
>
> 17.04% postgres [.] _bt_moveright
> 13.39% postgres [.] LWLockAcquire
> 8.82% postgres [.] LWLockRelease
> 8.11% postgres [.] _bt_checkpage
> 6.10% postgres [.] hash_search_with_hash_value
> 4.35% libc-2.19.so [.] __strcoll_l
> 3.16% postgres [.] 0x00000000002989ec
> 2.06% postgres [.] s_lock
> 1.99% postgres [.] _bt_compare
> 1.49% postgres [.] hash_any
> 1.41% postgres [.] 0x0000000000298a10
> 1.14% postgres [.] varstr_cmp
> 1.07% [kernel] [k] _raw_spin_unlock_irqrestore
> 1.00% libc-2.19.so [.] strlen
> 0.99% postgres [.] 0x0000000000298fdb
> 0.96% postgres [.] 0x00000000002989fc
> 0.80% postgres [.] LockBuffer
> 0.73% libc-2.19.so [.] 0x000000000009478e
> 0.60% postgres [.] ReadBufferExtended
> 0.49% postgres [.] bttextcmp
> 0.47% libc-2.19.so [.] 0x0000000000094787
> 0.44% [kernel] [k] finish_task_switch
> 0.40% postgres [.] _bt_relandgetbuf
> 0.40% postgres [.] ResourceOwnerForgetBuffer
> 0.40% libc-2.19.so [.] 0x0000000000094782
> 0.39% postgres [.] FunctionCall2Coll
> 0.38% postgres [.] ReleaseAndReadBuffer
> 0.31% postgres [.] pg_detoast_datum_packed
>
>
> Best regards,
> Michal
>
>
>
>
>
> --
> Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-admin
>
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: semop hanging - Postgres 9.4.4
@ 2015-12-16 11:59 Shreeyansh Dba <shreeyansh2014@gmail.com>
parent: Shreeyansh Dba <shreeyansh2014@gmail.com>
0 siblings, 0 replies; 9+ messages in thread
From: Shreeyansh Dba @ 2015-12-16 11:59 UTC (permalink / raw)
To: Michał Nowak <minowack@wp.pl>; +Cc: pgsql-admin
Hi Michal,
You could try using pg_ctl option as given below
pg_ctl kill TERM <stuck pid>
Regards,
Sagar(DBA)
Shreeyansh Technologies.
On Wed, Dec 16, 2015 at 5:20 PM, Shreeyansh Dba <shreeyansh2014@gmail.com>
wrote:
> Hi Michal,
>
> Sometimes terminating a query via normal methods of pg_cancel_backend()
> and pg_terminate_backend() fail and additional steps need to be taken.
>
> ps -eflyCpostgres |grep PID(stuck pid)
>
> OS level kill is not recommended but we don't have alternate solution in
> some scenarios we need to from OS
> kill -11 'stuck pid'
>
> You could decide shmmax depending upon the size of DB. We recommend to set
> shmmax to 6 GB and increase shared_buffers size to 2 GB.
>
>
> Best regards,
> Sagar(DBA)
> Shreeyansh Technologies.
>
> On Wed, Dec 16, 2015 at 4:44 PM, Michał Nowak <minowack@wp.pl> wrote:
>
>> Hi Sagar,
>>
>> > May I know what is the RAM of server and how much shmmax is configured ?
>> > You can check if any Zombi or defunct process are running on the
>> server.
>> > I want to know if any maintenance activity performed same time?
>>
>> Thanks for you reply.
>>
>> Server has 16GB of RAM. Kernel shmmax has strange value, i think it's
>> default on Ubuntu - i haven't changed that.
>>
>> kernel.shmall = 18446744073692774399
>> kernel.shmmax = 18446744073692774399
>> kernel.shmmni = 4096
>>
>> I changed kermel.sem from defaults to: 250 512000 100 2048
>>
>> There is no zombi nor defunct process. Now i have 3 stucked processes.
>>
>> First:
>>
>> psql01:~# strace -ffp 14135
>> Process 14135 attached
>> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>> semop(58261738, {{12, -1, 0}}, 1) = 0
>> semop(58261738, {{12, -1, 0}}, 1) = 0
>> semop(58261738, {{12, -1, 0}}, 1) = 0
>> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>> semop(58261738, {{12, -1, 0}}, 1) = 0
>> semop(58261738, {{12, -1, 0}}, 1) = 0
>> semop(58261738, {{12, -1, 0}}, 1) = 0
>> semop(58261738, {{12, -1, 0}}, 1) = 0
>> semop(58261738, {{12, -1, 0}}, 1) = 0
>> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>>
>>
>> Second:
>>
>> psql01:~# strace -ffp 12712
>> Process 12712 attached
>> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>> semop(57737434, {{6, 1, 0}}, 1) = 0
>> semop(57934048, {{2, -1, 0}}, 1) = 0
>> semop(57802972, {{7, 1, 0}}, 1) = 0
>> semop(57934048, {{2, -1, 0}}, 1) = 0
>> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>> semop(57934048, {{2, -1, 0}}, 1) = 0
>> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>> semop(57934048, {{2, -1, 0}}, 1) = 0
>> semop(57934048, {{2, -1, 0}}, 1) = 0
>>
>>
>> Third:
>>
>> psql02:~# strace -ffp 18283
>> Process 18283 attached
>> semop(58523890, {{11, 1, 0}}, 1) = 0
>> semop(57802972, {{9, -1, 0}}, 1) = 0
>> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>> semop(57802972, {{9, -1, 0}}, 1) = 0
>> semop(58818811, {{13, 1, 0}}, 1) = 0
>> semop(57802972, {{9, -1, 0}}, 1) = 0
>> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>> semop(57802972, {{9, -1, 0}}, 1) = 0
>> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>>
>>
>> perf top:
>>
>> 17.04% postgres [.] _bt_moveright
>> 13.39% postgres [.] LWLockAcquire
>> 8.82% postgres [.] LWLockRelease
>> 8.11% postgres [.] _bt_checkpage
>> 6.10% postgres [.] hash_search_with_hash_value
>> 4.35% libc-2.19.so [.] __strcoll_l
>> 3.16% postgres [.] 0x00000000002989ec
>> 2.06% postgres [.] s_lock
>> 1.99% postgres [.] _bt_compare
>> 1.49% postgres [.] hash_any
>> 1.41% postgres [.] 0x0000000000298a10
>> 1.14% postgres [.] varstr_cmp
>> 1.07% [kernel] [k] _raw_spin_unlock_irqrestore
>> 1.00% libc-2.19.so [.] strlen
>> 0.99% postgres [.] 0x0000000000298fdb
>> 0.96% postgres [.] 0x00000000002989fc
>> 0.80% postgres [.] LockBuffer
>> 0.73% libc-2.19.so [.] 0x000000000009478e
>> 0.60% postgres [.] ReadBufferExtended
>> 0.49% postgres [.] bttextcmp
>> 0.47% libc-2.19.so [.] 0x0000000000094787
>> 0.44% [kernel] [k] finish_task_switch
>> 0.40% postgres [.] _bt_relandgetbuf
>> 0.40% postgres [.] ResourceOwnerForgetBuffer
>> 0.40% libc-2.19.so [.] 0x0000000000094782
>> 0.39% postgres [.] FunctionCall2Coll
>> 0.38% postgres [.] ReleaseAndReadBuffer
>> 0.31% postgres [.] pg_detoast_datum_packed
>>
>>
>> Best regards,
>> Michal
>>
>>
>>
>>
>>
>> --
>> Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgsql-admin
>>
>
>
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: semop hanging - Postgres 9.4.4
@ 2015-12-16 12:26 Albe Laurenz <laurenz.albe@wien.gv.at>
1 sibling, 0 replies; 9+ messages in thread
From: Albe Laurenz @ 2015-12-16 12:26 UTC (permalink / raw)
To: 'Michal Nowak *EXTERN*' <minowack@wp.pl>; pgsql-admin
Michal Nowak wrote:
> There is no zombi nor defunct process. Now i have 3 stucked processes.
Can you take a stack trace of the stuck processes?
https://wiki.postgresql.org/wiki/Getting_a_stack_trace_of_a_running_PostgreSQL_backend_on_Linux/BSD
Yours,
Laurenz Albe
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: semop hanging - Postgres 9.4.4
@ 2015-12-16 13:41 Kevin Grittner <kgrittn@gmail.com>
parent: Michał Nowak <minowack@wp.pl>
1 sibling, 0 replies; 9+ messages in thread
From: Kevin Grittner @ 2015-12-16 13:41 UTC (permalink / raw)
To: Michał Nowak <minowack@wp.pl>; +Cc: pgsql-admin
On Tue, Dec 15, 2015 at 1:34 PM, Michał Nowak <minowack@wp.pl> wrote:
> postgres=# select version();
> version
> ------------------------------------------------------------------------------------------------------
> PostgreSQL 9.4.4 on x86_64-unknown-linux-gnu, compiled by gcc (Ubuntu 4.8.2-19ubuntu1) 4.8.2, 64-bit
There is a bug fix in 9.4.5 which can cause a deadlock on
lightweight locking.
http://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=bab959906911c97437f410a03b0346e6dd28d5...
http://www.postgresql.org/support/versioning/
From a subsequent post:
> perf top:
>
> 17.04% postgres [.] _bt_moveright
> 13.39% postgres [.] LWLockAcquire
> 8.82% postgres [.] LWLockRelease
> 8.11% postgres [.] _bt_checkpage
The perf top output suggests the possibility of a corrupted index,
causing an endless loop.
Can you get a few stack traces from "stuck" backends?
--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: semop hanging - Postgres 9.4.4
@ 2015-12-17 17:21 Michał Nowak <minowack@wp.pl>
0 siblings, 0 replies; 9+ messages in thread
From: Michał Nowak @ 2015-12-17 17:21 UTC (permalink / raw)
To: pgsql-admin
Hello,
Thanks for all replies.
> The perf top output suggests the possibility of a corrupted index,
> causing an endless loop.
Yes! Rebuilding indexes solves problem for this table. :)
But after 20h problem is back on another table. First table has about 2m records, second has only 200k.
> There is a bug fix in 9.4.5 which can cause a deadlock on
> lightweight locking.
I've just upgraded to 9.4.5 and hope that helps. I'll keep you posted.
Best Regards,
Michal
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: semop hanging - Postgres 9.4.4
@ 2015-12-21 13:18 Michał Nowak <minowack@wp.pl>
0 siblings, 0 replies; 9+ messages in thread
From: Michał Nowak @ 2015-12-21 13:18 UTC (permalink / raw)
To: pgsql-admin
Hello,
> I've just upgraded to 9.4.5 and hope that helps. I'll keep you posted.
After upgrade everything works fine for almost 4 days.
Thank you all for help.
Best Regards,
Michal
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 9+ messages in thread
end of thread, other threads:[~2015-12-21 13:18 UTC | newest]
Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2015-12-15 19:34 semop hanging - Postgres 9.4.4 Michał Nowak <minowack@wp.pl>
2015-12-16 10:44 ` Shreeyansh Dba <shreeyansh2014@gmail.com>
2015-12-16 13:41 ` Kevin Grittner <kgrittn@gmail.com>
2015-12-16 11:31 Re: semop hanging - Postgres 9.4.4 Michał Nowak <minowack@wp.pl>
2015-12-16 11:50 ` Shreeyansh Dba <shreeyansh2014@gmail.com>
2015-12-16 11:59 ` Shreeyansh Dba <shreeyansh2014@gmail.com>
2015-12-16 12:26 ` Albe Laurenz <laurenz.albe@wien.gv.at>
2015-12-17 17:21 Re: semop hanging - Postgres 9.4.4 Michał Nowak <minowack@wp.pl>
2015-12-21 13:18 Re: semop hanging - Postgres 9.4.4 Michał Nowak <minowack@wp.pl>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox