Date: Thu, 17 Jan 2019 00:27:07 -0800
From: Noah Misch <noah@leadboat.com>
To: Heath Lord <heath.lord@crunchydata.com>,
	Stephen Frost <sfrost@snowman.net>
Cc: Andres Freund <andres@anarazel.de>,
	Thomas Munro <thomas.munro@enterprisedb.com>,
	PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
	Tom Lane <tgl@sss.pgh.pa.us>
Subject: Re: "could not reattach to shared memory" on buildfarm member dory
Message-ID: <20190117082707.GA3908397@rfd.leadboat.com>
References: <20180424070646.GA2693533@rfd.leadboat.com>
 <17897.1525108069@sss.pgh.pa.us>
 <29823.1525132900@sss.pgh.pa.us>
 <20180501002419.onmikldoe2ssjauq@alap3.anarazel.de>
 <18056.1525148010@sss.pgh.pa.us>
 <22096.1525188710@sss.pgh.pa.us>
 <20180819020007.GD3795674@rfd.leadboat.com>
 <14554.1537811585@sss.pgh.pa.us>
 <20180925150512.GA3990982@rfd.leadboat.com>
 <20181203053506.GB2860387@rfd.leadboat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20181203053506.GB2860387@rfd.leadboat.com>
User-Agent: Mutt/1.5.24 (2015-08-30)
Precedence: bulk

On Sun, Dec 02, 2018 at 09:35:06PM -0800, Noah Misch wrote:
> On Tue, Sep 25, 2018 at 08:05:12AM -0700, Noah Misch wrote:
> > On Mon, Sep 24, 2018 at 01:53:05PM -0400, Tom Lane wrote:
> > > Overall, I agree that neither of these approaches are exactly attractive.
> > > We're paying a heck of a lot of performance or complexity to solve a
> > > problem that shouldn't even be there, and that we don't understand well.
> > > In particular, the theory that some privileged code is injecting a thread
> > > into every new process doesn't square with my results at
> > > https://www.postgresql.org/message-id/15345.1525145612%40sss.pgh.pa.us
> > > 
> > > I think our best course of action at this point is to do nothing until
> > > we have a clearer understanding of what's actually happening on dory.
> > > Perhaps such understanding will yield an idea for a less painful fix.
> > 
> > I see.
> 
> Could one of you having a dory login use
> https://live.sysinternals.com/Procmon.exe to capture process events during
> backend startup?

Ping.

> The ideal would be one capture where startup failed reattach
> and another where it succeeded, but having the successful run alone would be a
> good start.  The procedure is roughly this:
> 
> - Install PostgreSQL w/ debug symbols.
> - Start a postmaster.
> - procmon /nomonitor
> - procmon "Filter" menu -> Enable Advanced Output
> - Ctrl-l, add filter for "Process Name" is "postgres.exe"
> - Ctrl-e (starts collecting data)
> - psql (leave it running)
> - After ~60s, Ctrl-e again in procmon (stops collecting data)
> - File -> Save -> PML
> - File -> Save -> XML, include stack traces, resolve stack symbols
> - Compress the PML and XML files, and mail them here