Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rnifn-00Ehdv-N9 for pgsql-general@arkaria.postgresql.org; Fri, 22 Mar 2024 17:27:20 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1rnifm-005u4Z-6P for pgsql-general@arkaria.postgresql.org; Fri, 22 Mar 2024 17:27:18 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rnifl-005u4R-SR for pgsql-general@lists.postgresql.org; Fri, 22 Mar 2024 17:27:18 +0000 Received: from sss.pgh.pa.us ([68.162.161.243]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rnifi-005t7r-RZ for pgsql-general@lists.postgresql.org; Fri, 22 Mar 2024 17:27:17 +0000 Received: from sss1.sss.pgh.pa.us (localhost [127.0.0.1]) by sss.pgh.pa.us (8.15.2/8.15.2) with ESMTP id 42MHRF01944274; Fri, 22 Mar 2024 13:27:15 -0400 From: Tom Lane To: Matthias Apitz cc: pgsql-general@lists.postgresql.org Subject: Re: soft lockup - CPU#16 stuck for 3124s! [postmaster:2273] In-reply-to: References: Comments: In-reply-to Matthias Apitz message dated "Fri, 22 Mar 2024 18:12:28 +0100" MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <944272.1711128435.1@sss.pgh.pa.us> Content-Transfer-Encoding: quoted-printable Date: Fri, 22 Mar 2024 13:27:15 -0400 Message-ID: <944273.1711128435@sss.pgh.pa.us> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk Matthias Apitz writes: > We have a PostgreSQL 15.1 server in production at a customer for some > weeks (migrated from an older version) on SuSE SLES 15. > The customer is facing machine locks and before the Linux server does > not respond any more (not even on SSH, only power-cycle reset helps to > get it back), short before the fault a lot of messages are in > /var/log/messages of the content: > # grep watchdog: /var/log/messages > ... > 2024-03-22T13:11:32.056154+01:00 sunrise kernel: [327844.313048][ C25]= watchdog: BUG: soft lockup - CPU#25 stuck for 3069s! [migration/25:166] > 2024-03-22T13:12:28.056244+01:00 sunrise kernel: [327900.310267][ C16]= watchdog: BUG: soft lockup - CPU#16 stuck for 3124s! [postmaster:2273] > 2024-03-22T13:12:28.056340+01:00 sunrise kernel: [327900.311052][ C25]= watchdog: BUG: soft lockup - CPU#25 stuck for 3121s! [migration/25:166] Sounds like failing hardware to me :-( regards, tom lane