From: Tom Lane <tgl@sss.pgh.pa.us>
To: Thomas Munro <thomas.munro@gmail.com>
cc: Andres Freund <andres@anarazel.de>,
        Andrew Dunstan <andrew.dunstan@2ndquadrant.com>,
        Mark Wong <mark@2ndquadrant.com>,
 Justin Pryzby <pryzby@telsasoft.com>,
        pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: stress test for parallel workers
In-reply-to: <4113.1570982833@sss.pgh.pa.us>
References: <20032.1570808731@sss.pgh.pa.us> <14878.1570820201@sss.pgh.pa.us>
 <CA+hUKGKNgufn12Uh4iEh5y=JkEnUBWnLLmi8L4zwzMunFeKwSA@mail.gmail.com>
 <18266.1570824826@sss.pgh.pa.us> <21051.1570828384@sss.pgh.pa.us>
 <17884.1570915552@sss.pgh.pa.us> <25033.1570925192@sss.pgh.pa.us>
 <CA+hUKGLZh8_Js04nd1-z4_awZyTT-2nzHPxuztn1ApuTS_r=Ow@mail.gmail.com>
 <20191013133154.fyxrkcbf4zpwrmtt@alap3.anarazel.de>
 <32038.1570976985@sss.pgh.pa.us>
 <20191013145053.z5w4gyew5dcgft7g@alap3.anarazel.de>
 <942.1570978653@sss.pgh.pa.us> <4113.1570982833@sss.pgh.pa.us>
Comments: In-reply-to Tom Lane <tgl@sss.pgh.pa.us>
	message dated "Sun, 13 Oct 2019 12:07:13 -0400"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <27923.1571068231.1@sss.pgh.pa.us>
Content-Transfer-Encoding: quoted-printable
Date: Mon, 14 Oct 2019 11:50:31 -0400
Message-ID: <27924.1571068231@sss.pgh.pa.us>
Precedence: bulk

I wrote:
> Filed at
> https://bugzilla.kernel.org/show_bug.cgi?id=3D205183
> We'll see what happens ...

Further to this --- I went back and looked at the outlier events
where we saw an infinite_recurse failure on a non-Linux-PPC64
platform.  There were only three:

 mereswine    | ARMv7            | Linux debian-armhf | Clarence Ho     | =
REL_11_STABLE | 2019-08-11 02:10:12 | InstallCheck-C  | 2019-08-11 02:36:1=
0.159 PDT [5004:4] DETAIL:  Failed process was running: select infinite_re=
curse();
 mereswine    | ARMv7            | Linux debian-armhf | Clarence Ho     | =
REL_12_STABLE | 2019-08-11 09:52:46 | pg_upgradeCheck | 2019-08-11 04:21:1=
6.756 PDT [6804:5] DETAIL:  Failed process was running: select infinite_re=
curse();
 mereswine    | ARMv7            | Linux debian-armhf | Clarence Ho     | =
HEAD          | 2019-08-11 11:29:27 | pg_upgradeCheck | 2019-08-11 07:15:2=
8.454 PDT [9954:76] DETAIL:  Failed process was running: select infinite_r=
ecurse();
 =

Looking closer at these, though, they were *not* SIGSEGV failures,
but SIGKILLs.  Seeing that they were all on the same machine on the
same day, I'm thinking we can write them off as a transiently
misconfigured OOM killer.

So, pending some other theory emerging from the kernel hackers, we're
down to it's-a-PPC64-kernel-bug.  That leaves me wondering what if
anything we want to do about it.  Even if it's fixed reasonably promptly
in Linux upstream, and then we successfully nag assorted vendors to
incorporate the fix quickly, that's still going to leave us with frequent
buildfarm failures on Mark's flotilla of not-the-very-shiniest Linux
versions.

Should we move the infinite_recurse test to happen alone in a parallel
group just to stop these failures?  That's annoying from a parallelism
standpoint, but I don't see any other way to avoid these failures.

			regards, tom lane