pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedWARNING: terminating connection because of crash of another server process
4+ messages / 4 participants
[nested] [flat]
* WARNING: terminating connection because of crash of another server process
@ 2017-09-18 12:32 Saeed Ahmed (DB) <saeed.ahmed1@telenor.com.pk>
2017-09-19 14:51 ` Re: WARNING: terminating connection because of crash of another server process Scott Whitney <scott@journyx.com>
0 siblings, 1 reply; 4+ messages in thread
From: Saeed Ahmed (DB) @ 2017-09-18 12:32 UTC (permalink / raw)
To: pgsql-admin
Dear Team,
We are having POSGRESQL version 9.6 installed on OS version Linux REDHAT 7.2. While running our Decoder (for larger load) OR Audit execution, we are seeing connection errors as service to DB goes down when application tries to connect to postgres.
Log file is attached.
2017-09-14 00:35:16 ICT [5769]: [1-1] user=,db=,app=,client= LOG: checkpoint starting: time
2017-09-14 00:35:16 ICT [5769]: [2-1] user=,db=,app=,client= LOG: checkpoint complete: wrote 3 buffers (0.0%); 0 transaction log file(s) added, 0 removed, 0 recycled; write=0.486 s, sync=0.000 s, total=0.491 s; sync files=3, longest=0.000 s, average=0.000 s; distance=23 kB, estimate=23 kB
2017-09-14 00:39:01 ICT [5803]: [1-1] user=[unknown],db=[unknown],app=[unknown],client=[local] LOG: connection received: host=[local]
2017-09-14 00:39:01 ICT [5803]: [2-1] user=postgres,db=postgres,app=[unknown],client=[local] LOG: connection authorized: user=postgres database=postgres
2017-09-14 00:39:01 ICT [5803]: [3-1] user=postgres,db=postgres,app=psql,client=[local] LOG: disconnection: session time: 0:00:00.008 user=postgres database=postgres host=[local]
2017-09-14 00:39:16 ICT [5812]: [1-1] user=,db=,app=,client= FATAL: semctl(1693253819, 10, SETVAL, 0) failed: Invalid argument
2017-09-14 00:39:16 ICT [1617]: [13584-1] user=,db=,app=,client= LOG: server process (PID 5812) exited with exit code 1
2017-09-14 00:39:16 ICT [1617]: [13585-1] user=,db=,app=,client= LOG: terminating any other active server processes
2017-09-14 00:39:16 ICT [5782]: [3-1] user=raref,db=radb1,app=[unknown],client=10.68.128.7 WARNING: terminating connection because of crash of another server process
2017-09-14 00:39:16 ICT [5782]: [4-1] user=raref,db=radb1,app=[unknown],client=10.68.128.7 DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2017-09-14 00:39:16 ICT [5782]: [5-1] user=raref,db=radb1,app=[unknown],client=10.68.128.7 HINT: In a moment you should be able to reconnect to the database and repeat your command.
2017-09-14 00:39:16 ICT [5783]: [3-1] user=raref,db=radb1,app=[unknown],client=10.68.128.1 WARNING: terminating connection because of crash of another server process
2017-09-14 00:39:16 ICT [5783]: [4-1] user=raref,db=radb1,app=[unknown],client=10.68.128.1 DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2017-09-14 00:39:16 ICT [5783]: [5-1] user=raref,db=radb1,app=[unknown],client=10.68.128.1 HINT: In a moment you should be able to reconnect to the database and repeat your command.
2017-09-14 00:39:16 ICT [5779]: [3-1] user=raref,db=radb1,app=[unknown],client=10.68.128.1 WARNING: terminating connection because of crash of another server process
2017-09-14 00:39:16 ICT [5779]: [4-1] user=raref,db=radb1,app=[unknown],client=10.68.128.1 DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2017-09-14 00:39:16 ICT [5779]: [5-1] user=raref,db=radb1,app=[unknown],client=10.68.128.1 HINT: In a moment you should be able to reconnect to the database and repeat your command.
2017-09-14 00:39:16 ICT [5780]: [3-1] user=raref,db=radb1,app=[unknown],client=10.68.128.1 WARNING: terminating connection because of crash of another server process
2017-09-14 00:39:16 ICT [5780]: [4-1] user=raref,db=radb1,app=[unknown],client=10.68.128.1 DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2017-09-14 00:39:16 ICT [5780]: [5-1] user=raref,db=radb1,app=[unknown],client=10.68.128.1 HINT: In a moment you should be able to reconnect to the database and repeat your command.
2017-09-14 00:39:16 ICT [5781]: [3-1] user=raref,db=radb1,app=[unknown],client=10.68.128.7 WARNING: terminating connection because of crash of another server process
2017-09-14 00:39:16 ICT [5781]: [4-1] user=raref,db=radb1,app=[unknown],client=10.68.128.7 DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2017-09-14 00:39:16 ICT [5781]: [5-1] user=raref,db=radb1,app=[unknown],client=10.68.128.7 HINT: In a moment you should be able to reconnect to the database and repeat your command.
2017-09-14 00:39:16 ICT [5777]: [3-1] user=raref,db=radb1,app=[unknown],client=10.68.128.1 WARNING: terminating connection because of crash of another server process
2017-09-14 00:39:16 ICT [5777]: [4-1] user=raref,db=radb1,app=[unknown],client=10.68.128.1 DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2017-09-14 00:39:16 ICT [5777]: [5-1] user=raref,db=radb1,app=[unknown],client=10.68.128.1 HINT: In a moment you should be able to reconnect to the database and repeat your command.
2017-09-14 00:39:16 ICT [5778]: [3-1] user=raref,db=radb1,app=[unknown],client=10.68.128.7 WARNING: terminating connection because of crash of another server process
2017-09-14 00:39:16 ICT [5778]: [4-1] user=raref,db=radb1,app=[unknown],client=10.68.128.7 DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
Regards,
Saeed Ahmed
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
Attachments:
postgresql-2017-09-14.rar (application/octet-stream, 549.7K) download
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: WARNING: terminating connection because of crash of another server process
2017-09-18 12:32 WARNING: terminating connection because of crash of another server process Saeed Ahmed (DB) <saeed.ahmed1@telenor.com.pk>
@ 2017-09-19 14:51 ` Scott Whitney <scott@journyx.com>
2017-09-19 15:18 ` Re: Re: WARNING: terminating connection because of crash of another server process Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 4+ messages in thread
From: Scott Whitney @ 2017-09-19 14:51 UTC (permalink / raw)
To: Saeed Ahmed (DB) <saeed.ahmed1@telenor.com.pk>; pgsql-admin
It looks like maybe your semaphores are getting deleted out from under PG as a guess.
________________________________
From: pgsql-admin-owner@postgresql.org <pgsql-admin-owner@postgresql.org> on behalf of Saeed Ahmed (DB) <saeed.ahmed1@telenor.com.pk>
Sent: Monday, September 18, 2017 7:32 AM
To: pgsql-admin@postgresql.org
Subject: [ADMIN] WARNING: terminating connection because of crash of another server process
Dear Team,
We are having POSGRESQL version 9.6 installed on OS version Linux REDHAT 7.2. While running our Decoder (for larger load) OR Audit execution, we are seeing connection errors as service to DB goes down when application tries to connect to postgres.
Log file is attached.
2017-09-14 00:35:16 ICT [5769]: [1-1] user=,db=,app=,client= LOG: checkpoint starting: time
2017-09-14 00:35:16 ICT [5769]: [2-1] user=,db=,app=,client= LOG: checkpoint complete: wrote 3 buffers (0.0%); 0 transaction log file(s) added, 0 removed, 0 recycled; write=0.486 s, sync=0.000 s, total=0.491 s; sync files=3, longest=0.000 s, average=0.000 s; distance=23 kB, estimate=23 kB
2017-09-14 00:39:01 ICT [5803]: [1-1] user=[unknown],db=[unknown],app=[unknown],client=[local] LOG: connection received: host=[local]
2017-09-14 00:39:01 ICT [5803]: [2-1] user=postgres,db=postgres,app=[unknown],client=[local] LOG: connection authorized: user=postgres database=postgres
2017-09-14 00:39:01 ICT [5803]: [3-1] user=postgres,db=postgres,app=psql,client=[local] LOG: disconnection: session time: 0:00:00.008 user=postgres database=postgres host=[local]
2017-09-14 00:39:16 ICT [5812]: [1-1] user=,db=,app=,client= FATAL: semctl(1693253819, 10, SETVAL, 0) failed: Invalid argument
2017-09-14 00:39:16 ICT [1617]: [13584-1] user=,db=,app=,client= LOG: server process (PID 5812) exited with exit code 1
2017-09-14 00:39:16 ICT [1617]: [13585-1] user=,db=,app=,client= LOG: terminating any other active server processes
2017-09-14 00:39:16 ICT [5782]: [3-1] user=raref,db=radb1,app=[unknown],client=10.68.128.7 WARNING: terminating connection because of crash of another server process
2017-09-14 00:39:16 ICT [5782]: [4-1] user=raref,db=radb1,app=[unknown],client=10.68.128.7 DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2017-09-14 00:39:16 ICT [5782]: [5-1] user=raref,db=radb1,app=[unknown],client=10.68.128.7 HINT: In a moment you should be able to reconnect to the database and repeat your command.
2017-09-14 00:39:16 ICT [5783]: [3-1] user=raref,db=radb1,app=[unknown],client=10.68.128.1 WARNING: terminating connection because of crash of another server process
2017-09-14 00:39:16 ICT [5783]: [4-1] user=raref,db=radb1,app=[unknown],client=10.68.128.1 DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2017-09-14 00:39:16 ICT [5783]: [5-1] user=raref,db=radb1,app=[unknown],client=10.68.128.1 HINT: In a moment you should be able to reconnect to the database and repeat your command.
2017-09-14 00:39:16 ICT [5779]: [3-1] user=raref,db=radb1,app=[unknown],client=10.68.128.1 WARNING: terminating connection because of crash of another server process
2017-09-14 00:39:16 ICT [5779]: [4-1] user=raref,db=radb1,app=[unknown],client=10.68.128.1 DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2017-09-14 00:39:16 ICT [5779]: [5-1] user=raref,db=radb1,app=[unknown],client=10.68.128.1 HINT: In a moment you should be able to reconnect to the database and repeat your command.
2017-09-14 00:39:16 ICT [5780]: [3-1] user=raref,db=radb1,app=[unknown],client=10.68.128.1 WARNING: terminating connection because of crash of another server process
2017-09-14 00:39:16 ICT [5780]: [4-1] user=raref,db=radb1,app=[unknown],client=10.68.128.1 DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2017-09-14 00:39:16 ICT [5780]: [5-1] user=raref,db=radb1,app=[unknown],client=10.68.128.1 HINT: In a moment you should be able to reconnect to the database and repeat your command.
2017-09-14 00:39:16 ICT [5781]: [3-1] user=raref,db=radb1,app=[unknown],client=10.68.128.7 WARNING: terminating connection because of crash of another server process
2017-09-14 00:39:16 ICT [5781]: [4-1] user=raref,db=radb1,app=[unknown],client=10.68.128.7 DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2017-09-14 00:39:16 ICT [5781]: [5-1] user=raref,db=radb1,app=[unknown],client=10.68.128.7 HINT: In a moment you should be able to reconnect to the database and repeat your command.
2017-09-14 00:39:16 ICT [5777]: [3-1] user=raref,db=radb1,app=[unknown],client=10.68.128.1 WARNING: terminating connection because of crash of another server process
2017-09-14 00:39:16 ICT [5777]: [4-1] user=raref,db=radb1,app=[unknown],client=10.68.128.1 DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
2017-09-14 00:39:16 ICT [5777]: [5-1] user=raref,db=radb1,app=[unknown],client=10.68.128.1 HINT: In a moment you should be able to reconnect to the database and repeat your command.
2017-09-14 00:39:16 ICT [5778]: [3-1] user=raref,db=radb1,app=[unknown],client=10.68.128.7 WARNING: terminating connection because of crash of another server process
2017-09-14 00:39:16 ICT [5778]: [4-1] user=raref,db=radb1,app=[unknown],client=10.68.128.7 DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
Regards,
Saeed Ahmed
Journyx, Inc.
7600 Burnet Road #300
Austin, TX 78757
www.journyx.com
p 512.834.8888
f 512-834-8858
Do you receive our promotional emails? Click here <http://journyx.com/communication-preferences; or visit http://journyx.com/communication-preferences to unsubscribe.
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Re: WARNING: terminating connection because of crash of another server process
2017-09-18 12:32 WARNING: terminating connection because of crash of another server process Saeed Ahmed (DB) <saeed.ahmed1@telenor.com.pk>
2017-09-19 14:51 ` Re: WARNING: terminating connection because of crash of another server process Scott Whitney <scott@journyx.com>
@ 2017-09-19 15:18 ` Tom Lane <tgl@sss.pgh.pa.us>
2017-12-08 16:00 ` Re: [ADMIN] Re: WARNING: terminating connection because of crash of another server process Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
0 siblings, 1 reply; 4+ messages in thread
From: Tom Lane @ 2017-09-19 15:18 UTC (permalink / raw)
To: Scott Whitney <scott@journyx.com>; +Cc: Saeed Ahmed (DB) <saeed.ahmed1@telenor.com.pk>; pgsql-admin; Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Scott Whitney <scott@journyx.com> writes:
> It looks like maybe your semaphores are getting deleted out from under PG as a guess.
Yeah. See
https://www.postgresql.org/docs/devel/static/kernel-resources.html#systemd-removeipc
(Probably we should back-patch that section into older versions of the
docs, as it's not at all PG-version-specific.)
regards, tom lane
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: [ADMIN] Re: WARNING: terminating connection because of crash of another server process
2017-09-18 12:32 WARNING: terminating connection because of crash of another server process Saeed Ahmed (DB) <saeed.ahmed1@telenor.com.pk>
2017-09-19 14:51 ` Re: WARNING: terminating connection because of crash of another server process Scott Whitney <scott@journyx.com>
2017-09-19 15:18 ` Re: Re: WARNING: terminating connection because of crash of another server process Tom Lane <tgl@sss.pgh.pa.us>
@ 2017-12-08 16:00 ` Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
0 siblings, 0 replies; 4+ messages in thread
From: Peter Eisentraut @ 2017-12-08 16:00 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; Scott Whitney <scott@journyx.com>; +Cc: Saeed Ahmed (DB) <saeed.ahmed1@telenor.com.pk>; pgsql-admin
On 9/19/17 11:18, Tom Lane wrote:
> Scott Whitney <scott@journyx.com> writes:
>> It looks like maybe your semaphores are getting deleted out from under PG as a guess.
>
> Yeah. See
>
> https://www.postgresql.org/docs/devel/static/kernel-resources.html#systemd-removeipc
>
> (Probably we should back-patch that section into older versions of the
> docs, as it's not at all PG-version-specific.)
done
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2017-12-08 16:00 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2017-09-18 12:32 WARNING: terminating connection because of crash of another server process Saeed Ahmed (DB) <saeed.ahmed1@telenor.com.pk>
2017-09-19 14:51 ` Scott Whitney <scott@journyx.com>
2017-09-19 15:18 ` Re: Re: WARNING: terminating connection because of crash of another server process Tom Lane <tgl@sss.pgh.pa.us>
2017-12-08 16:00 ` Re: [ADMIN] Re: WARNING: terminating connection because of crash of another server process Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox