Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1vM5fL-000l4N-39 for pgsql-admin@arkaria.postgresql.org; Thu, 20 Nov 2025 14:29:45 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.96) (envelope-from ) id 1vM5fJ-0026g0-0R for pgsql-admin@arkaria.postgresql.org; Thu, 20 Nov 2025 14:29:41 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1vM5fI-0026fs-2L for pgsql-admin@lists.postgresql.org; Thu, 20 Nov 2025 14:29:41 +0000 Received: from ironport2.embl.de ([194.94.44.76]) by makus.postgresql.org with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1vM5fF-000XSV-19 for pgsql-admin@postgresql.org; Thu, 20 Nov 2025 14:29:40 +0000 Received: from hl-outgoing01.ebi.ac.uk ([193.62.192.137]) by ironport2.embl.de with ESMTP; 20 Nov 2025 15:29:35 +0100 Received: from [172.23.2.51] (unknown [172.23.2.51]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: sri) by hl-outgoing01.ebi.ac.uk (Postfix) with ESMTPSA id 9643036CE for ; Thu, 20 Nov 2025 14:29:34 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ebi.ac.uk; s=ebinew; t=1763648974; bh=JdEmmVzoZutoQP9ENzWphhsYyydMEIXN2lQLNkBSTfw=; h=Date:Subject:References:To:From:In-Reply-To:From; b=dH9tTJnpb4LhNmioiRPjAB48+ApYvGiHtTdkXJVXrN6G7xLRINQyz5tr4WEew+HQN VoDjv8GsFZcBeqSXQr8E9fSTC53Yij67XWvb1tl65kjSbq8lwk/BvAFvGwkK9sWLyE mm+s8H9PtCZJnPjbfdFjIACThtu8fzNMQXrR6G3k= Content-Type: multipart/alternative; boundary="------------2fptLz7uq6PazmuGRQFgFtdg" Message-ID: Date: Thu, 20 Nov 2025 14:29:34 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Barman was hung after a Transport endpoint error Content-Language: en-US References: To: pgsql-admin@postgresql.org From: Sri Mrudula Attili In-Reply-To: X-Forwarded-Message-Id: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk This is a multi-part message in MIME format. --------------2fptLz7uq6PazmuGRQFgFtdg Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Hello Team,  This is regarding our Barman backup host, which seems like hung for 5 days with no backup, streaming, cron etc happening.  We have checkmk monitoring that notifies us if any one of the parameters of Barman check fail, but we havent hot any events from checkmk as barman check was showing all OK and no FAILED. What do you explain this situation in Barman. *To provide further details about the issue:* * We user Data Domain as barman backup home for storage. * On 5th November data domain endpoints is not connect but the mount was still up on the host. We still dont know from the data domain side what caused this issue. * Barman log has errors*: [2800226] barman.cli ERROR: [Errno 107] Transport endpoint is not connected: '/nfs/DDbackup/pgsql-****-****/.active-model.auto'* * If the end point was missing barman should fail completely and the checks should fail as well, Which in turn would have helped us to identify the issue immediately. * The only way we figured out that there was an issue, on November 11th one of barman backup maximum age parameter failed saying that weeks backup dint happen. * Upon checking why the backups failed etc, I went to barman logs and found out that all this time until the endpoint issue resolved barman was not doing the barman checks or barman check was hung at a state *Below are the configuration details:* * Barman version : 3.10.0 Barman by EnterpriseDB (www.enterprisedb.com) * Postgresql version: (PostgreSQL) 16.9 * Number of hosts configured: 26 * Red Hat Enterprise Linux release 8.10 Thanks, Sri --------------2fptLz7uq6PazmuGRQFgFtdg Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit

Hello Team,


 This is regarding our Barman backup host, which seems like hung for 5 days with no backup, streaming, cron etc happening.

 We have checkmk monitoring that notifies us if any one of the parameters of Barman check fail, but we havent hot any events from checkmk as barman check was showing all OK and no FAILED.

What do you explain this situation in Barman.


To provide further details about the issue:

  • We user Data Domain as barman backup home for storage.
  • On 5th November data domain endpoints is not connect but the mount was still up on the host. We still dont know from the data domain side what caused this issue.
  • Barman log has errors : [2800226] barman.cli ERROR: [Errno 107] Transport endpoint is not connected: '/nfs/DDbackup/pgsql-****-****/.active-model.auto'
  • If the end point was missing barman should fail completely and the checks should fail as well, Which in turn would have helped us to identify the issue immediately.
  • The only way we figured out that there was an issue, on November 11th one of barman backup maximum age parameter failed saying that weeks backup dint happen.
  • Upon checking why the backups failed etc, I went to barman logs and found out that all this time until the endpoint issue resolved barman was not doing the barman checks or barman check was hung at a state

Below are the configuration details:

  • Barman version : 3.10.0 Barman by EnterpriseDB (www.enterprisedb.com)
  • Postgresql version: (PostgreSQL) 16.9
  • Number of hosts configured: 26
  • Red Hat Enterprise Linux release 8.10 

Thanks,

Sri



--------------2fptLz7uq6PazmuGRQFgFtdg--