Date: Fri, 11 Oct 2019 13:28:53 -0700
From: Mark Wong <mark@2ndQuadrant.com>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>,
	Andrew Dunstan <andrew.dunstan@2ndquadrant.com>,
	Justin Pryzby <pryzby@telsasoft.com>,
	Andres Freund <andres@anarazel.de>,
	pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: stress test for parallel workers
Message-ID: <20191011202853.GA23809@2ndQuadrant.com>
References: 
 <CA+hUKGL6cDyb2maq2P60cEsjFK=3saBCAj7sDzE3jysL-PRwqg@mail.gmail.com>
 <31032.1565152159@sss.pgh.pa.us>
 <CA+hUKGJNS6gZ=w=+wxjqhmck-voJ7FBuejHnw8mQgAp2yt0A9g@mail.gmail.com>
 <18537.1570421268@sss.pgh.pa.us>
 <5350.1570743291@sss.pgh.pa.us>
 <84813a30-8c34-9c32-7ad5-90d9eefba468@2ndQuadrant.com>
 <06d6be5a-7af2-ef78-0abc-6bfe500ead87@2ndQuadrant.com>
 <20032.1570808731@sss.pgh.pa.us>
 <14878.1570820201@sss.pgh.pa.us>
 <CA+hUKGKNgufn12Uh4iEh5y=JkEnUBWnLLmi8L4zwzMunFeKwSA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
 <CA+hUKGKNgufn12Uh4iEh5y=JkEnUBWnLLmi8L4zwzMunFeKwSA@mail.gmail.com>
User-Agent: Mutt/1.12.2 (2019-09-21)
Precedence: bulk

On Sat, Oct 12, 2019 at 08:41:12AM +1300, Thomas Munro wrote:
> On Sat, Oct 12, 2019 at 7:56 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > This matches up with the intermittent infinite_recurse failures
> > we've been seeing in the buildfarm.  Those are happening across
> > a range of systems, but they're (almost) all Linux-based ppc64,
> > suggesting that there's a longstanding arch-specific kernel bug
> > involved.  For reference, I scraped the attached list of such
> > failures in the last three months.  I wonder whether we can get
> > the attention of any kernel hackers about that.
> 
> Yeah, I don't know anything about this stuff, but I was also beginning
> to wonder if something is busted in the arch-specific fault.c code
> that checks if stack expansion is valid[1], in a way that fails with a
> rapidly growing stack, well timed incoming signals, and perhaps
> Docker/LXC (that's on Mark's systems IIUC, not sure about the ARM
> boxes that failed or if it could be relevant here).  Perhaps the
> arbitrary tolerances mentioned in that comment are relevant.

This specific one (wobbegon) is OpenStack/KVM[2], for what it's worth...

"... cluster is an OpenStack based cluster offering POWER8 & POWER9 LE
instances running on KVM ..."

But to keep you on your toes, some of my ppc animals are Docker within
other OpenStack/KVM instance...

Regards,
Mark

[1] https://github.com/torvalds/linux/blob/master/arch/powerpc/mm/fault.c#L244
[2] https://osuosl.org/services/powerdev/

-- 
Mark Wong
2ndQuadrant - PostgreSQL Solutions for the Enterprise
https://www.2ndQuadrant.com/