Date: Wed, 22 Aug 2018 17:11:26 +0900
From: Michael Paquier <michael@paquier.xyz>
To: cyberdemn@gmail.com, pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #15346: Replica fails to start after the crash
Message-ID: <20180822081126.GE4333@paquier.xyz>
References: <153492341830.1368.3936905691758473953@wrigleys.postgresql.org>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="n+lFg1Zro7sl44OB"
Content-Disposition: inline
In-Reply-To: <153492341830.1368.3936905691758473953@wrigleys.postgresql.org>
Precedence: bulk


--n+lFg1Zro7sl44OB
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

On Wed, Aug 22, 2018 at 07:36:58AM +0000, PG Bug reporting form wrote:
> 2018-08-22 06:22:23.633 UTC,,,54,,5b7d0114.36,23,,2018-08-22 06:22:12
> UTC,1/0,0,WARNING,01000,"page 179503104 of relation base/18055/212875 does
> not exist",,,,,"xlog redo at AB3/50323E78 for Btree/DELETE: 182
> items",,,,""
> 2018-08-22 06:22:23.634 UTC,,,54,,5b7d0114.36,24,,2018-08-22 06:22:12
> UTC,1/0,0,PANIC,XX000,"WAL contains references to invalid pages",,,,,"xlog
> redo at AB3/50323E78 for Btree/DELETE: 182 items",,,,""

Once recovery has reached a consistent state, the startup process would
look at if there are any invalid pages tracked in a given hash table and
complains loudly about them.  It is not the last record or its
surroundings which matter in case, but if this page has been found in
one of the records replayed during recovery up to the consistent point.
Do you have in any records from the WAL segments fetched a reference to
this page?  A page is 8kB, and the page number is 179503104, which is
definitely weird as that would cause a relation file to be more than
1000GB.  If the record itself is in bad shape, this may be a corrupted
segment.  As far as I can see you only have one incorrect page reference
(see XLogCheckInvalidPages in xlog.c).

> I will keep this instance around for further investigation and would be
> happy to provide some more details if you need.

That would be nice!
--
Michael

--n+lFg1Zro7sl44OB
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAlt9Gq4ACgkQnvQgOdby
QH2pSg/+Ilvhm+YRTDDEQoLzw7fvS7AkbfcUe7QLbxQbtFCFcIqKcs8Msvk8szpG
WMDLIbq8pbHIGsQ9X1rZOMoSwkYPFoTAyDGx90nF+NsxqhjQiYVtxFA+BomA8/3h
JmlTACLTAzCaj+KVDGAdPKA3UrAi0A316HveJYwW177WlJlZeHy1A2rxUrgCzQuS
r8k/rpuNskBpSiaj4s6Q0YYBHL6wdyUIZu/C0T2euOVqmZIASx7xSfk60RN7eCsw
leV1R8CSecXNxJpHS0lJj/h4/Y5Qk+bmNk8csL+gta8n6xfLP3cIrxJY8PyTHXDm
y5XLoy4xZry49skhTGmsmCBXVKSAn968xFxMYewfLUSzSpS+75ixDr8R+iiI9mmH
w/PchifysG/KpzdkDDr0AKHPEuTLtGxDds81+vvFelvxEXmBqEbGZhJ2gnOf+4di
3IpLjwfsAE0vCmo9EoMOdN/mFZ1FycmbVa3iJmN/bS1Xvmk943gUjeK9h848y0Hh
S/lh8Gt7a0dlub6YVMDZ0wfo+gwq49LV/UMVH/bN9Pgv124ZzumwM3gLlGy8Fn5H
G++qfo5JQnhnE2G6hN1sC2zAR9TAEgtQ0tPQIvablKz3ouW7i+uiiR4h/agM9Z/2
Og9YXTpr/+0NaTpZD1S74Ee+wOy+h5rQcR9AbwmhQVkUQf1aIl8=
=5H6Y
-----END PGP SIGNATURE-----

--n+lFg1Zro7sl44OB--