From: Tom Lane <tgl@sss.pgh.pa.us>
To: Achilleas Mantzios <achill@matrix.gatewaynet.com>
cc: pgsql-admin@lists.postgresql.org
Subject: Re: PostgreSQL 10.5 : Strange pg_wal fill-up,
 solved with the shutdown checkpoint
In-reply-to: <6031b670-52d1-b823-6b41-35d2fa1873c6@matrix.gatewaynet.com>
References: <0d25bb8c-9cc7-10bc-71ba-6a8a8c33ec49@matrix.gatewaynet.com>
 <46D70340-2172-406A-A346-B2D4C6E947E4@crazybean.net>
 <6031b670-52d1-b823-6b41-35d2fa1873c6@matrix.gatewaynet.com>
Comments: In-reply-to Achilleas Mantzios <achill@matrix.gatewaynet.com>
	message dated "Tue, 06 Nov 2018 10:37:37 +0200"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <23168.1541516303.1@sss.pgh.pa.us>
Content-Transfer-Encoding: quoted-printable
Date: Tue, 06 Nov 2018 09:58:23 -0500
Message-ID: <23169.1541516303@sss.pgh.pa.us>
Precedence: bulk

Achilleas Mantzios <achill@matrix.gatewaynet.com> writes:
> Remember : postgresql checkpointer decided to remove 5000+ files before =
shutdown. If any conditions were keeping those files afloat should also ho=
ld at this point, right.
> The question is why didn't Postgresql removed them earlier.

WAL files get removed/recycled after completion of a checkpoint.  So
apparently, checkpoints were not finishing during normal operation,
but the shutdown checkpoint managed to terminate normally.  That
eliminates a lot of the usual theories about why checkpoints might
not be succeeding (like a dirty buffer that always fails to be
written, say as a result of broken permissions on its file).

The only theory that comes to mind is that the checkpointer process
was stuck somehow, but just "soft" stuck, in a way that allowed the
postmaster's time-to-shut-down-please signal to unstick it.  No,
I have no idea how that could happen exactly.  If it happens again,
it'd be really interesting to attach to the checkpointer with a
debugger and collect a stack trace.

			regards, tom lane