From: Tom Lane <tgl@sss.pgh.pa.us>
To: Andres Freund <andres@anarazel.de>
cc: Andrew Dunstan <andrew.dunstan@2ndquadrant.com>, mark@2ndquadrant.com,
        Thomas Munro <thomas.munro@gmail.com>,
        Justin Pryzby <pryzby@telsasoft.com>,
        pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: stress test for parallel workers
In-reply-to: <20191011203141.xjhgln2vwhemzvra@alap3.anarazel.de>
References: <17389.1563945314@sss.pgh.pa.us>
 <CA+hUKGL6cDyb2maq2P60cEsjFK=3saBCAj7sDzE3jysL-PRwqg@mail.gmail.com>
 <31032.1565152159@sss.pgh.pa.us>
 <CA+hUKGJNS6gZ=w=+wxjqhmck-voJ7FBuejHnw8mQgAp2yt0A9g@mail.gmail.com>
 <18537.1570421268@sss.pgh.pa.us> <5350.1570743291@sss.pgh.pa.us>
 <84813a30-8c34-9c32-7ad5-90d9eefba468@2ndQuadrant.com>
 <06d6be5a-7af2-ef78-0abc-6bfe500ead87@2ndQuadrant.com>
 <20032.1570808731@sss.pgh.pa.us> <14878.1570820201@sss.pgh.pa.us>
 <20191011203141.xjhgln2vwhemzvra@alap3.anarazel.de>
Comments: In-reply-to Andres Freund <andres@anarazel.de>
	message dated "Fri, 11 Oct 2019 13:31:41 -0700"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <19524.1570826441.1@sss.pgh.pa.us>
Content-Transfer-Encoding: quoted-printable
Date: Fri, 11 Oct 2019 16:40:41 -0400
Message-ID: <19525.1570826441@sss.pgh.pa.us>
Precedence: bulk

Andres Freund <andres@anarazel.de> writes:
> On 2019-10-11 14:56:41 -0400, Tom Lane wrote:
>> ... So it's really hard to explain
>> that as anything except a kernel bug: sometimes, the kernel
>> doesn't give us as much stack as it promised it would.  And the
>> machine is not loaded enough for there to be any rational
>> resource-exhaustion excuse for that.

> Linux expands stack space only on demand, thus it's possible to run out
> of stack space while there ought to be stack space. Unfortunately that
> during a stack expansion, which means there's no easy place to report
> that.  I've seen this be hit in production on busy machines.

As I said, this machine doesn't seem busy enough for that to be a
tenable excuse; there's nobody but me logged in, and the buildfarm
critter isn't running.

> I wonder if the machine is configured with overcommit_memory=3D2,
> i.e. don't overcommit.  cat /proc/sys/vm/overcommit_memory would tell.

$ cat /proc/sys/vm/overcommit_memory
0

> What does grep -E '^(Mem|Commit)' /proc/meminfo show while it's
> happening?

idle:

$ grep -E '^(Mem|Commit)' /proc/meminfo =

MemTotal:        2074816 kB
MemFree:           36864 kB
MemAvailable:    1779584 kB
CommitLimit:     1037376 kB
Committed_AS:     412480 kB

a few captures while regression tests are running:

$ grep -E '^(Mem|Commit)' /proc/meminfo =

MemTotal:        2074816 kB
MemFree:            8512 kB
MemAvailable:    1819264 kB
CommitLimit:     1037376 kB
Committed_AS:     371904 kB
$ grep -E '^(Mem|Commit)' /proc/meminfo =

MemTotal:        2074816 kB
MemFree:           32640 kB
MemAvailable:    1753792 kB
CommitLimit:     1037376 kB
Committed_AS:     585984 kB
$ grep -E '^(Mem|Commit)' /proc/meminfo =

MemTotal:        2074816 kB
MemFree:           56640 kB
MemAvailable:    1695744 kB
CommitLimit:     1037376 kB
Committed_AS:     568768 kB


> What does the signal information say? You can see it with
> p $_siginfo
> after receiving the signal. A SIGSEGV here, I assume.

(gdb) p $_siginfo
$1 =3D {si_signo =3D 11, si_errno =3D 0, si_code =3D 128, _sifields =3D {_=
pad =3D {0 <repeats 28 times>}, _kill =3D {si_pid =3D 0, si_uid =3D 0}, =

    _timer =3D {si_tid =3D 0, si_overrun =3D 0, si_sigval =3D {sival_int =3D=
 0, sival_ptr =3D 0x0}}, _rt =3D {si_pid =3D 0, si_uid =3D 0, si_sigval =3D=
 {
        sival_int =3D 0, sival_ptr =3D 0x0}}, _sigchld =3D {si_pid =3D 0, =
si_uid =3D 0, si_status =3D 0, si_utime =3D 0, si_stime =3D 0}, _sigfault =
=3D {
      si_addr =3D 0x0}, _sigpoll =3D {si_band =3D 0, si_fd =3D 0}}}

> Yea, that seems like it might be good. But we have to be careful too, as
> there's some thing were do want to be interruptable from within a signal
> handler. We start some processes from within one after all...

The proposed patch has zero effect on what the signal mask will be inside
a signal handler, only on the transient state during handler entry/exit.

			regards, tom lane