Date: Wed, 22 Aug 2018 12:08:38 -0300
From: Alvaro Herrera <alvherre@2ndquadrant.com>
To: Alexander Kukushkin <cyberdemn@gmail.com>
Cc: Michael Paquier <michael@paquier.xyz>, pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #15346: Replica fails to start after the crash
Message-ID: <20180822150838.ffdszn4wz57owd3c@alvherre.pgsql>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: 
 <CAFh8B=kvY2BYgikf=tpjUGD2h0rsm8GosOeU4TeYR+Q6YK-mjw@mail.gmail.com>
User-Agent: NeoMutt/20170306-137-4415bd-dirty (1.8.0)
Precedence: bulk

On 2018-Aug-22, Alexander Kukushkin wrote:

> 2018-08-22 16:44 GMT+02:00 Alvaro Herrera <alvherre@2ndquadrant.com>:
> 
> >
> > Sounds likely.  I suggest to have a look at what's going on inside the
> > postmaster process when it gets stuck.
> 
> Well, it doesn't get stuck, it aborts start with the message:
> 2018-08-22 14:26:42.073 UTC,,,28485,,5b7d7282.6f45,23,,2018-08-22
> 14:26:10 UTC,1/0,0,WARNING,01000,"page 179503104 of relation
> base/18055/212875 does not exist",,,,,"xlog redo at AB3/50323E78 for
> Btree/DELETE: 182 items",,,,""
> 2018-08-22 14:26:42.073 UTC,,,28485,,5b7d7282.6f45,24,,2018-08-22
> 14:26:10 UTC,1/0,0,PANIC,XX000,"WAL contains references to invalid
> pages",,,,,"xlog redo at AB3/50323E78 for Btree/DELETE: 182
> items",,,,""
> 2018-08-22 14:26:42.214 UTC,,,28483,,5b7d7282.6f43,3,,2018-08-22
> 14:26:10 UTC,,0,LOG,00000,"startup process (PID 28485) was terminated
> by signal 6: Aborted",,,,,,,,,""

Oh, that's weird ... sounds like the fact that the bgworker starts
somehow manages to corrupt the list of invalid pages in the startup
process.  That doesn't make any sense ...

ENOTIME for a closer look ATM, though, sorry.  Maybe you could try
running under valgrind?

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services