Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1v9VyL-001lCY-OO for pgsql-admin@arkaria.postgresql.org; Thu, 16 Oct 2025 21:57:21 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1v9VyK-00Flrr-Nc for pgsql-admin@arkaria.postgresql.org; Thu, 16 Oct 2025 21:57:19 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1v9Lpl-00C4PN-4b for pgsql-admin@lists.postgresql.org; Thu, 16 Oct 2025 11:07:48 +0000 Received: from efreedom.netcracker.com ([198.49.86.140]) by makus.postgresql.org with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1v9Lpi-00259z-1A for pgsql-admin@postgresql.org; Thu, 16 Oct 2025 11:07:47 +0000 DKIM-Signature: v=1; a=rsa-sha256; d=netcracker.com; s=i; c=simple/simple; t=1760612865; h=from:subject:to:date:message-id; bh=dbIlk2WN43bSye05Z4yvG4rBtUncWoMJ2rdn6xPHsIw=; b=FlfBlylLinIm7kal55awqnfjiVIKN0K9EQJ0zowFG5CnpHBLBYxJvPcbmDfRKmpBuhZ3DEdFAZc 8xUuw8IDOTlRREbHSsklN72szDeAZYCUqNA3EGTlDuowrUoMcuJtV/NOuTmycS4tkhscBo9cwmmKD zDD9V2slyyD23PG78rVt42cXZoCeT7XfNMuODyYE9rjnrhGOFEJOlPLoPoS9+6esgX0YVCvXgJi+r cHIp/Cqxj7YilVQ0xljXrUv8OwuWLHYq0dyPZrv7ZcjlktTTL0quHRiYgw3JeTDZp/f1TmGe4/7lo thZ5rvIR+wKzTTRqlQOPE/yg/4QnkV9d6Pjw== From: Ishan Arunkumar Joshi To: "pgsql-admin@postgresql.org" Subject: postgres server crash with "Segmentation fault" Thread-Topic: postgres server crash with "Segmentation fault" Thread-Index: Adw+iCqmAtBIs9TZTSSuQmfQ/2bfKg== Date: Thu, 16 Oct 2025 11:07:44 +0000 Message-ID: <1bef483a9e134bb595d3b25a35ac0e16@netcracker.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: multipart/alternative; boundary="_000_1bef483a9e134bb595d3b25a35ac0e16netcrackercom_" MIME-Version: 1.0 X-TM-AS-Product-Ver: SMEX-14.0.0.3223-9.1.2019-29516.007 X-TM-AS-Result: No-10--33.830500-4.000000 X-TMASE-MatchedRID: +g4YE93QWXfdU3YKncE6eiQptwyVJHc8BJ+fPezc4sNjlyQtJbCZRoik TlcySjEdOioJOIKSEjjTnbgeeeYmQqHtzDekdzh/U4vLutFdoGbSNGSme9Cpu+SxnfO8XDh3CM2 UFD1tdJRwyhCvvjl7gV7TNCWlTyqxaFtBXjg8UFxxWCN0QH9j+IYTx7IjYnvT6nVE/KTaADSuB1 4ADbID6gODDIGAlA5O5wRmFetIKdLdcVqaZky34koTJyYTYhwtGOSh0CwDuOOMnmW32UtJ6YlVU UIgnPcdwtghHqCPJpGu/AApmC81Did2THclb8I18WAg/+Nfkucq8IyudVpZ92UX6HMaaim43QO/ 1xl/AIj+e23e355lsghUNgliyN+HAlQHBQURCB7U0ho7mxaK+AhFWKVFTFrrhcXcyywpCI7ZHRg EJqVu0jwPsWzrY2Y0Is0yhPd2wLmejBMe+7AMaGE7EKfp+9g6jQmHD3Vo9Y6IcOmd382XO1b//x dcOFsBV4xUzO1C9WlT7PQhFbK2uXHRzjcS4ENUaS5ourTIFp8S9JCfUkBxvJGeN+9dVIq3iixUG JhD4/ofP6dEVJ4XcgjGmJm5YMZPvjPJRHkNhUx3InXM2YFWY9Z83hD4SDtCb/Mf4HpAx4MxEMH2 +NC7Vth5c4TN8zbZgcmRaCwD9EklTScbe2l74JsI6K9TYesLZLQpoie3cD6l8ia45E7uPzR3N7O +I2B8d05DzqdnDlm7zryBwIKH7Ej/cM0lG/fvNkUSDDq742k= X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No X-TMASE-Result: 10--33.830500-4.000000 X-TMASE-Version: SMEX-14.0.0.3223-9.1.2019-29516.007 X-TM-SNTS-SMTP: CBE55D660EA64EC44AEAC24C5061AB603A2ADBAF3C1EBF5C9F5749C513540D0D2000:9 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --_000_1bef483a9e134bb595d3b25a35ac0e16netcrackercom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi team, We are using PG16.9 in Patroni Postgres setup in production. Last night we = have face an issue where postgres server got crash with "Segmentation fault= " on table with auto vacuum task. Interestingly in standby node while we pe= rform vacuum on same table standby node was also got crash. The table was n= ot able to get query on the table as while executing select statement also= crash the database. we have observed few error prior to crash for same table. (Table name and f= unction details change for purpose) "ERROR : Error occurred at function get_details page 117 of relation ""impa= cted_table"" should be empty but is not" During the same time the other table also getting below error. However once= the database restart, we were not getting any issue for table oid=3D 11080= 29 "ERROR : Error occurred at function get_details unexpected data beyond EOF = in block 16276 of relation base/33195/1108029" At last it got failed as follows 2025-10-15 02:50:52.428 [432443]LOG: terminating any other active server p= rocesses" 2025-10-15 02:50:52.428 [432443]DETAIL: Failed process was running: autova= cuum: VACUUM ANALYZE schema.impacted_table" 2025-10-15 02:50:52.428 [432443]LOG: server process (PID 390906) was termi= nated by signal 11: Segmentation fault" 2025-10-15 02:50:55.475 [432443]LOG: all server processes terminated; rein= itializing" 2025-10-15 02:51:32.575 [432443]LOG: received immediate shutdown request" 2025-10-15 02:51:32.629 [432443]LOG: database system is shut down" The function which was executing having truncate table and insert/update st= atement executing on this table. As this is normally functionality but we = are suspecting it during the execution runtime it corrupt the shared memory= and data inflight which got replicated to Replica node/DR site and corrupt= the same table. We had to drop the table "impacted_table" from database once the database i= s up. however we are not able to identify exact root cause behind "segmentation f= ault" error for this table and need expert advice to find the root case and= also need suggestions to prevention steps. Thanks & Regards, ------------------------- Ishan Joshi ________________________________ The information transmitted herein is intended only for the person or entit= y to which it is addressed and may contain confidential, proprietary and/or= privileged material. Any review, retransmission, dissemination or other us= e of, or taking of any action in reliance upon, this information by persons= or entities other than the intended recipient is prohibited. If you receiv= ed this in error, please contact the sender and delete the material from an= y computer. --_000_1bef483a9e134bb595d3b25a35ac0e16netcrackercom_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi team,

 

We are using PG16.9 in Patroni Postgres setup in pro= duction. Last night we have face an issue where postgres server got crash w= ith “Segmentation fault” on table with auto vacuum task. Intere= stingly in standby node while we perform vacuum on same table standby node was also got crash. The table was not able to g= et query  on the table as while executing select statement also crash = the database.

we have observed few error prior to crash for same table. (Table name and f= unction details change for purpose)  


"ERROR : Error occurred at function get_details page 117 of rel= ation ""impacted_table"" should be empty but is not&quo= t;

 

During the same time the other table also getting be= low error. However once the database restart, we were not getting any issue= for table oid=3D 1108029


“ERROR : Error occurred at function get_details unexpected data beyon= d EOF in block 16276 of relation base/33195/1108029"

 

At last it got failed as follows

2025-10-15 02:50:52.428 [432443]LOG:  terminating any other active ser= ver processes"

2025-10-15 02:50:52.428 [432443]DETAIL:  Failed= process was running: autovacuum: VACUUM ANALYZE schema.impacted_table"= ;

2025-10-15 02:50:52.428 [432443]LOG:  server pr= ocess (PID 390906) was terminated by signal 11: Segmentation fault"

2025-10-15 02:50:55.475 [432443]LOG:  all serve= r processes terminated; reinitializing"

2025-10-15 02:51:32.575 [432443]LOG:  received = immediate shutdown request"

2025-10-15 02:51:32.629 [432443]LOG:  database = system is shut down"

 

The function which was executing having truncate tab= le and insert/update statement executing on this table.  As this is no= rmally functionality but we are suspecting it during the execution runtime = it corrupt the shared memory and data inflight which got replicated to Replica node/DR site and corrupt the same table.

 

We had to drop the table “impacted_table”= ; from database once the database is up.

however we are not able to identify exact root cause behind “segmenta= tion fault” error for this table and need expert advice to find the r= oot case and also need suggestions to prevention steps.

 

Thanks & Regards,

-----------= --------------
Ishan Joshi

 

=  

=

The information transmitted herein is intended only for the person or entit= y to which it is addressed and may contain confidential, proprietary and/or= privileged material. Any review, retransmission, dissemination or other us= e of, or taking of any action in reliance upon, this information by persons or entities other than the inte= nded recipient is prohibited. If you received this in error, please contact= the sender and delete the material from any computer.  

--_000_1bef483a9e134bb595d3b25a35ac0e16netcrackercom_--