pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Andres Freund <andres@anarazel.de>
Cc: pgsql-committers@lists.postgresql.org
Subject: Re: pgsql: pgstat: Bring up pgstat in BaseInit() to fix uninitialized use o
Date: Sat, 07 Aug 2021 13:06:47 -0400
Message-ID: <2771326.1628356007@sss.pgh.pa.us> (raw)
In-Reply-To: <20210807164850.52atvo2qkmtfnzgt@alap3.anarazel.de>
References: <E1mCC6U-0004Ik-Fs@gemulon.postgresql.org>
	<2638683.1628304247@sss.pgh.pa.us>
	<20210807030652.qf5wsbhxk7dpsmtn@alap3.anarazel.de>
	<2640064.1628306008@sss.pgh.pa.us>
	<2640490.1628306556@sss.pgh.pa.us>
	<20210807044952.orwuf7hua2wytx4g@alap3.anarazel.de>
	<20210807051149.x2j3n64ejcu23hm4@alap3.anarazel.de>
	<2675112.1628350987@sss.pgh.pa.us>
	<20210807164850.52atvo2qkmtfnzgt@alap3.anarazel.de>

Andres Freund <andres@anarazel.de> writes:
> On 2021-08-07 11:43:07 -0400, Tom Lane wrote:
>> Intuitively it seems like temp file management should be a low-level,
>> backend-local function and therefore should be okay to run after
>> shmem disconnection.  I do not have a warm feeling about reversing that
>> module layering --- what's to stop someone from breaking things by
>> trying to use a temp file in their on_proc_exit or on_shmem_exit hook?

> We could just add an assert preventing that from happening. It's hard to
> believe that there could be a good reason to use temp files in those hook
> points.

The bigger picture here is that anyplace anybody ever wants to add stats
collection in suddenly becomes a "must run before shmem disconnection"
module.  I think that way madness lies --- we can't have the entire
backend shut down before shmem disconnection.  So I feel like there's
a serious design problem here, and it's not confined to temp files,
or at least it won't stay confined to temp files.  (There may indeed
be other problems already, that we just haven't had the good luck to
have buildfarm timing vagaries expose to us.)

Maybe the solution is to acknowledge that we might lose some events
during backend shutdown, and redefine the behavior as "we ignore
event reports after pgstat shutdown", not "we assert that there never
can be any such reports".

> I'm somewhat inclined to split InitFileAccess() into two by separating out
> InitTemporaryFileAccess() or such. InitFileAccess() would continue to happen
> early and register a proc exit hook that errors out when there's a temp file
> (as a backstop for non cassert builds). The new InitTemporaryFileAccess()
> would happen a bit later and schedule CleanupTempFiles() to happen via
> before_shmem_access(). And we add a Assert(!proc_exit_inprogress) to the
> routines for opening a temp file.

Maybe that would work, but after you multiply it by a bunch of different
scenarios in different modules, it's going to get less and less attractive.

> I've bitten by all this often enough to be motivated to propose
> something. However I want to get the basics of the shared memory stats stuff
> in first - it's a pain to keep it upated, and we'll need to find and solve all
> of the issues it has anyway, even if we go for a redesign of module / startup
> / shutdown layering.

Fair.  But I suggest that the first cut should look more like what
I suggest above, ie just be willing to lose events during shutdown.
The downsides of that are not so enormous that we should be willing
to undertake major klugery to avoid it before we've even got a
semi-working system.

			regards, tom lane





view thread (53+ messages)  latest in thread

Message-ID: <2771326.1628356007@sss.pgh.pa.us>
Permalink:  ../2771326.1628356007@sss.pgh.pa.us/
Also on:    postgresql.org/message-id/2771326.1628356007@sss.pgh.pa.us

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-committers@postgresql.org
  Cc: tgl@sss.pgh.pa.us, andres@anarazel.de
  Subject: Re: pgsql: pgstat: Bring up pgstat in BaseInit() to fix uninitialized use o
  In-Reply-To: <2771326.1628356007@sss.pgh.pa.us>
  References: <E1mCC6U-0004Ik-Fs@gemulon.postgresql.org> <2638683.1628304247@sss.pgh.pa.us> <20210807030652.qf5wsbhxk7dpsmtn@alap3.anarazel.de> <2640064.1628306008@sss.pgh.pa.us> <2640490.1628306556@sss.pgh.pa.us> <20210807044952.orwuf7hua2wytx4g@alap3.anarazel.de> <20210807051149.x2j3n64ejcu23hm4@alap3.anarazel.de> <2675112.1628350987@sss.pgh.pa.us> <20210807164850.52atvo2qkmtfnzgt@alap3.anarazel.de> <2771326.1628356007@sss.pgh.pa.us>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox