MIME-Version: 1.0
References: 
 <CA+fnDAa8Kbbx3KkqpfxutTx41RchJsT3Lykqwcedm0CVr5Tw=A@mail.gmail.com>
 <CAFC+b6pQXYMks5f4XJ-QX4QpxnYnrNz9Zrkg8EiJ5-O+PxKxBQ@mail.gmail.com>
In-Reply-To: 
 <CAFC+b6pQXYMks5f4XJ-QX4QpxnYnrNz9Zrkg8EiJ5-O+PxKxBQ@mail.gmail.com>
From: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Date: Mon, 6 Jul 2026 08:30:06 +0200
Message-ID: 
 <CAO6_Xqpzj_V1gF0ikPCVkA3Q2K5YqRnAdHiV5TqpLNAZsdwWbQ@mail.gmail.com>
Subject: Re: hang during shutdown
To: Srinath Reddy Sadipiralla <srinath2133@gmail.com>
Cc: Jeremy Schneider <schneider@ardentperf.com>,
	PostgreSQL mailing lists <pgsql-hackers@postgresql.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: 
 <https://www.postgresql.org/message-id/CAO6_Xqpzj_V1gF0ikPCVkA3Q2K5YqRnAdHiV5TqpLNAZsdwWbQ%40mail.gmail.com>
Precedence: bulk

Hi,

On Sun, Jul 5, 2026 at 11:55=E2=80=AFPM Jeremy Schneider
<schneider@ardentperf.com> wrote:
> so far, it all started with this postgres hang during a shutdown. we didn=
t capture enough diagnostics to root-cause the domino before the hang, but =
the data i do have about this postgres hang is very peculiar. i'm working o=
n some automation to gather more diagnostics if there is a recurrence. in t=
he meantime i wanted to share what i do know, in case anyone else has seen =
something similar.
>
> postgres version 16.13
> Ubuntu 22.04.5 LTS
> kernel 5.15.0-1110-azure
>
> between log messages and wal contents, it appears PostmasterStateMachine =
transitioned into PM_WAIT_BACKENDS but never progressed beyond this for a f=
ull 30 minutes until external automation finally killed the processes.

Did you have an active logical replication stream at the time of the
shutdown? If so, it's very likely due to a bug in the logical
walsender that can be stuck in an infinite loop when the WAL is in a
specific state (last record is a rollback crossing the WAL page
boundary).

A fix for that was released in the latest 16.14 minor release[0].

[0]: https://git.postgresql.org/gitweb/?p=3Dpostgresql.git;a=3Dcommitdiff;h=
=3D82935467a

Regards,
Anthonin Bonnefoy