From: Noah Misch <noah@leadboat.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Heath Lord <heath.lord@crunchydata.com>
Cc: Stephen Frost <sfrost@snowman.net>
Cc: Andres Freund <andres@anarazel.de>
Cc: Thomas Munro <thomas.munro@enterprisedb.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Cc: joseph.ayers@crunchydata.com
Subject: Re: "could not reattach to shared memory" on buildfarm member dory
Date: Sat, 6 Apr 2019 21:32:21 -0700
Message-ID: <20190407043221.GA1914190@rfd.leadboat.com> (raw)
In-Reply-To: <18088.1554214140@sss.pgh.pa.us>
References: <29823.1525132900@sss.pgh.pa.us>
<20180501002419.onmikldoe2ssjauq@alap3.anarazel.de>
<18056.1525148010@sss.pgh.pa.us>
<22096.1525188710@sss.pgh.pa.us>
<20180819020007.GD3795674@rfd.leadboat.com>
<14554.1537811585@sss.pgh.pa.us>
<20180925150512.GA3990982@rfd.leadboat.com>
<20181203053506.GB2860387@rfd.leadboat.com>
<20190402135442.GA1173872@rfd.leadboat.com>
<18088.1554214140@sss.pgh.pa.us>
On Tue, Apr 02, 2019 at 10:09:00AM -0400, Tom Lane wrote:
> Noah Misch <noah@leadboat.com> writes:
> > I can reproduce the 4 MiB allocations described
> > in https://postgr.es/m/29823.1525132900@sss.pgh.pa.us; a few times per
> > "vcregress check", they emerge in the middle of PGSharedMemoryReAttach().
> > The 4 MiB allocations are stacks for new threads of the default thread
> > pool[1].
>
> Hah! It is great to finally have an understanding of what is happening
> here.
>
> I worry that your proposed fix is unstable, in particular this assumption
> seems shaky:
>
> > + * ... The idea is that, if the allocator handed out
> > + * REGION1 pages before REGION2 pages at one occasion, it will do so whenever
> > + * both regions are free.
True. If Windows changes to prefer allocating from the most recently freed
region, shmem-protective-region-v1.patch would cease to help. It's not
impossible.
> I wonder whether it's possible to address this by configuring the "default
> thread pool" to have only one thread? It seems like the extra threads are
> just adding backend startup overhead to no benefit, since we won't use 'em.
I didn't find a way to configure the pool's size.
Another option is to reattach shared memory earlier, before the default thread
pool starts. A Windows application using only the unavoidable DLLs
(kernel32.dll, ntdll.dll, kernelbase.dll) doesn't get a default thread pool;
the pool starts when one loads ws2_32.dll, ucrtbased.dll, etc. Hence, the
DllMain() function of a DLL that loads early could avoid the problem. (Cygwin
fork() uses that route to remap shared memory, though it also retries failed
forks.) If we proceed that way, we'd add a tiny pg_shmem.dll that appears
early in the load order, just after the unavoidable DLLs. It would extract
applicable parameters from the command line and reattach shared memory. When
it fails, it would set a flag so the usual code can raise an error. Does that
sound more or less attractive than shmem-protective-region-v1.patch?
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: noah@leadboat.com, tgl@sss.pgh.pa.us, heath.lord@crunchydata.com, sfrost@snowman.net, andres@anarazel.de, thomas.munro@enterprisedb.com, pgsql-hackers@lists.postgresql.org, joseph.ayers@crunchydata.com
Subject: Re: "could not reattach to shared memory" on buildfarm member dory
In-Reply-To: <20190407043221.GA1914190@rfd.leadboat.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox