From: Tom Lane <tgl@sss.pgh.pa.us>
To: Thomas Munro <thomas.munro@gmail.com>
cc: Tomas Vondra <tomas.vondra@enterprisedb.com>,
        Andres Freund <andres@anarazel.de>,
 Stephen Frost <sfrost@snowman.net>,
        Alvaro Herrera <alvherre@2ndquadrant.com>,
        David Steele <david@pgmasters.net>,
        Dmitry Dolgov <9erthalion6@gmail.com>,
        Jakub Wartak <Jakub.Wartak@tomtom.com>,
        pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: WIP: WAL prefetch (another approach)
In-reply-to: <3895573.1620135972@sss.pgh.pa.us>
References: <8dd2dcc2-0b6b-a4ab-c1b3-032c0a5ab1c3@enterprisedb.com>
 <CA+hUKGKmkKfH1h9XG5ctYCgqeOHGdN5bHMoQMZLR=+kLY0PQzA@mail.gmail.com>
 <CA+hUKGLsF8sEE+n3qoaV6+XjmYEwYoFVDbLPbYTGz7oGkL=iZA@mail.gmail.com>
 <CA+hUKG+a4wn7B-qBu9SLc1HAbZRSKbp1Ep=rDeveM-UQz84+jg@mail.gmail.com>
 <773932.1619022622@sss.pgh.pa.us>
 <b971373e-5f05-e691-d83c-c72a4f67167b@enterprisedb.com>
 <CA+hUKGJKDibdw5HzOdgh+tFsA7=wwJ3O6p+qYwbgAD_xkLWA3Q@mail.gmail.com>
 <CAOuzzgrn7iKnFRsB4MHp3UisEQAGgZMbk_ViTN4HV4-Ksq8zCg@mail.gmail.com>
 <892344.1619054465@sss.pgh.pa.us>
 <CA+hUKGJKKh8TbA+6977zK5TKUSsTspBHsr0s4S42q7ynkxcPGQ@mail.gmail.com>
 <20210428163245.ff67pb2ii5irr5zr@alap3.anarazel.de>
 <2808045.1619628335@sss.pgh.pa.us>
 <CA+hUKGJ03Q-TmE8-JzAm04_1maYus6_kX7XB--JBh1K63agSbQ@mail.gmail.com>
 <3502526.1619925367@sss.pgh.pa.us>
 <CA+hUKGLGKnzTDhW9rnzyYb7yvUiCoGK5pB74qQf3YYLZYSX4OA@mail.gmail.com>
 <f2be6caa-5a7a-990b-c56e-a29454ae1cee@enterprisedb.com>
 <3895573.1620135972@sss.pgh.pa.us>
Comments: In-reply-to Tom Lane <tgl@sss.pgh.pa.us>
	message dated "Tue, 04 May 2021 09:46:12 -0400"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <3967043.1620157661.1@sss.pgh.pa.us>
Content-Transfer-Encoding: quoted-printable
Date: Tue, 04 May 2021 15:47:41 -0400
Message-ID: <3967044.1620157661@sss.pgh.pa.us>
Archived-At: 
 <https://www.postgresql.org/message-id/3967044.1620157661%40sss.pgh.pa.us>
Precedence: bulk

I wrote:
> I suppose that if we're unable to reproduce it on at least one other box=
,
> we have to write it off as hardware flakiness.

BTW, that conclusion shouldn't distract us from the very real bug
that Andres identified.  I was just scraping the buildfarm logs
concerning recent failures, and I found several recent cases
that match the symptom he reported:

https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=3Dchipmunk&dt=3D20=
21-04-23%2022%3A27%3A41
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=3Dhornet&dt=3D2021=
-04-21%2005%3A15%3A24
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=3Dmandrill&dt=3D20=
21-04-20%2002%3A03%3A08
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=3Dtern&dt=3D2021-0=
5-04%2004%3A07%3A41
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=3Dwrasse&dt=3D2021=
-04-20%2021%3A08%3A59

They all show the standby in recovery/019_replslot_limit.pl failing
with symptoms like

2021-05-04 07:42:00.968 UTC [24707406:1] LOG:  database system was shut do=
wn in recovery at 2021-05-04 07:41:39 UTC
2021-05-04 07:42:00.968 UTC [24707406:2] LOG:  entering standby mode
2021-05-04 07:42:01.050 UTC [24707406:3] LOG:  redo starts at 0/1C000D8
2021-05-04 07:42:01.079 UTC [24707406:4] LOG:  consistent recovery state r=
eached at 0/1D00000
2021-05-04 07:42:01.079 UTC [24707406:5] FATAL:  invalid memory alloc requ=
est size 1476397045
2021-05-04 07:42:01.080 UTC [13238274:3] LOG:  database system is ready to=
 accept read only connections
2021-05-04 07:42:01.082 UTC [13238274:4] LOG:  startup process (PID 247074=
06) exited with exit code 1

(BTW, the behavior seen here where the failure occurs *immediately*
after reporting "consistent recovery state reached" is seen in the
other reports as well, including Andres' version.  I wonder if that
means anything.)

			regards, tom lane