MIME-Version: 1.0
References: <a50e9505-b696-4717-b2c3-3a382456046d@ebi.ac.uk>
 <fa694cbb-fc32-46ee-8759-ac6d8084eab5@ebi.ac.uk>
In-Reply-To: <fa694cbb-fc32-46ee-8759-ac6d8084eab5@ebi.ac.uk>
From: Ron Johnson <ronljohnsonjr@gmail.com>
Date: Thu, 20 Nov 2025 12:15:59 -0500
Message-ID: 
 <CANzqJaABT+jvSBqc4Jv-ok9v7BT_UeQpnQpRmv1Y4vueJdfNzg@mail.gmail.com>
Subject: Re: Barman was hung after a Transport endpoint error
To: pgsql-admin <pgsql-admin@postgresql.org>
Content-Type: multipart/alternative; boundary="0000000000006db664064409da4b"
Archived-At: 
 <https://www.postgresql.org/message-id/CANzqJaABT%2BjvSBqc4Jv-ok9v7BT_UeQpnQpRmv1Y4vueJdfNzg%40mail.gmail.com>
Precedence: bulk

--0000000000006db664064409da4b
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Nov 20, 2025 at 9:29=E2=80=AFAM Sri Mrudula Attili <sri@ebi.ac.uk> =
wrote:

> Hello Team,
>
>
>  This is regarding our Barman backup host, which seems like hung for 5
> days with no backup, streaming, cron etc happening.
>
>  We have checkmk monitoring that notifies us if any one of the parameters
> of Barman check fail, but we havent hot any events from checkmk as barman
> check was showing all OK and no FAILED.
>
> What do you explain this situation in Barman.
>
>
> *To provide further details about the issue:*
>
>    - We user Data Domain as barman backup home for storage.
>    - On 5th November data domain endpoints is not connect but the mount
>    was still up on the host. We still dont know from the data domain side=
 what
>    caused this issue.
>    - Barman log has errors* : [2800226] barman.cli ERROR: [Errno 107]
>    Transport endpoint is not connected:
>    '/nfs/DDbackup/pgsql-****-****/.active-model.auto'*
>    - If the end point was missing barman should fail completely and the
>    checks should fail as well, Which in turn would have helped us to iden=
tify
>    the issue immediately.
>    - The only way we figured out that there was an issue, on November
>    11th one of barman backup maximum age parameter failed saying that wee=
ks
>    backup dint happen.
>    - Upon checking why the backups failed etc, I went to barman logs and
>    found out that all this time until the endpoint issue resolved barman =
was
>    not doing the barman checks or barman check was hung at a state
>
> *Below are the configuration details:*
>
>    - Barman version : 3.10.0 Barman by EnterpriseDB (www.enterprisedb.com=
)
>    - Postgresql version: (PostgreSQL) 16.9
>    - Number of hosts configured: 26
>    - Red Hat Enterprise Linux release 8.10
>
>
Is NFS hung?

--=20
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!

--0000000000006db664064409da4b
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">On Thu, Nov 20, 2025 at 9:29=E2=80=AFAM S=
ri Mrudula Attili &lt;<a href=3D"mailto:sri@ebi.ac.uk">sri@ebi.ac.uk</a>&gt=
; wrote:</div><div class=3D"gmail_quote gmail_quote_container"><blockquote =
class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px sol=
id rgb(204,204,204);padding-left:1ex"><u></u>

 =20

   =20
 =20
  <div>
    <div>
      <div lang=3D"x-unicode">
        <p>Hello Team,</p>
        <p><br>
        </p>
        <p>=C2=A0This is regarding our Barman backup host, which seems like
          hung for 5 days with no backup, streaming, cron etc happening.</p=
>
        <p>=C2=A0We have checkmk monitoring that notifies us if any one of
          the parameters of Barman check fail, but we havent hot any
          events from checkmk as barman check was showing all OK and no
          FAILED.</p>
        <p>What do you explain this situation in Barman.</p>
        <p><br>
        </p>
        <p><b>To provide further details about the issue:</b></p>
        <ul>
          <li>We user Data Domain as barman backup home for storage.</li>
          <li>On 5th November data domain endpoints is not connect but
            the mount was still up on the host. We still dont know from
            the data domain side what caused this issue.</li>
          <li>Barman log has errors<b> : [2800226] barman.cli ERROR:
              [Errno 107] Transport endpoint is not connected:
              &#39;/nfs/DDbackup/pgsql-****-****/.active-model.auto&#39;</b=
></li>
          <li>If the end point was missing barman should fail completely
            and the checks should fail as well, Which in turn would have
            helped us to identify the issue immediately.</li>
          <li>The only way we figured out that there was an issue, on
            November 11th one of barman backup maximum age parameter
            failed saying that weeks backup dint happen.</li>
          <li>Upon checking why the backups failed etc, I went to barman
            logs and found out that all this time until the endpoint
            issue resolved barman was not doing the barman checks or
            barman check was hung at a state</li>
        </ul>
        <p><b>Below are the configuration details:</b></p>
        <ul>
          <li>Barman version : 3.10.0 Barman by EnterpriseDB (<a href=3D"ht=
tp://www.enterprisedb.com" target=3D"_blank">www.enterprisedb.com</a>)</li>
          <li>Postgresql version: (PostgreSQL) 16.9</li>
          <li>Number of hosts configured: 26</li>
          <li>Red Hat Enterprise Linux release 8.10=C2=A0</li></ul></div></=
div></div></blockquote><div><br></div><div>Is NFS hung?</div></div><div><br=
></div><span class=3D"gmail_signature_prefix">-- </span><br><div dir=3D"ltr=
" class=3D"gmail_signature"><div dir=3D"ltr">Death to &lt;Redacted&gt;, and=
 butter sauce.<div>Don&#39;t boil me, I&#39;m still alive.<br><div><div>&lt=
;Redacted&gt; lobster!</div></div></div></div></div></div>

--0000000000006db664064409da4b--