From: Andres Freund <andres@anarazel.de>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Cc: Dilip Kumar <dilipbalaut@gmail.com>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-committers <pgsql-committers@lists.postgresql.org>
Subject: Re: pgsql: pgstat: Bring up pgstat in BaseInit() to fix uninitialized use o
Date: Thu, 12 Aug 2021 05:48:19 -0700
Message-ID: <20210812124819.5mrurb6ueizmqgru@alap3.anarazel.de> (raw)
In-Reply-To: <CAA4eK1L=fepiRg8PCx7_0z2Xgcu3bohzs=zMS6qutq0oD07wSQ@mail.gmail.com>
References: <20210807164850.52atvo2qkmtfnzgt@alap3.anarazel.de>
<20210807190131.2bm24acbebl4wl6i@alap3.anarazel.de>
<20210808022435.svp4spxlrfoi3w4p@alap3.anarazel.de>
<CAD21AoCCAa+J1-udHRo5-Hbtv=D38WdZDAaXZGDbQQ_Vg_d3bQ@mail.gmail.com>
<CAA4eK1LYw4J+BmYza+1VWF3QGo5NaV7goXFJ8g5gfA3WZRo6Xw@mail.gmail.com>
<CAD21AoAjrg5EuWM9qNApKRLC2LP+G18=u8xE=qbm2smmMzGC-A@mail.gmail.com>
<CAFiTN-uzNeU+cO-JZW6WCNAXu0qZgOoqzGrsP8y+k2BKu4SHdQ@mail.gmail.com>
<CAA4eK1JHUoE997NdUXbS9ikHk1oOtPupnNpGGnN_P3pn+jb6LQ@mail.gmail.com>
<20210812082237.in7l5x3rc4gxhgjh@alap3.anarazel.de>
<CAA4eK1L=fepiRg8PCx7_0z2Xgcu3bohzs=zMS6qutq0oD07wSQ@mail.gmail.com>
Hi,
On 2021-08-12 15:06:23 +0530, Amit Kapila wrote:
> On Thu, Aug 12, 2021 at 1:52 PM Andres Freund <andres@anarazel.de> wrote:
> > I'm not so sure. Why does sharedfileset have its own proc exit hook in the
> > first place? ISTM that this should be dealt with using resowners, rathers than
> > a sharedfileset specific mechanism?
> >
> The underlying temporary files need to be closed at xact end but need
> to survive across transactions.
Why do they need to be closed at xact end? To avoid wasting memory due to too
many buffered files?
> These are registered with the resource owner via
> PathNameOpenTemporaryFile/PathNameCreateTemporaryFile and then closed
> at xact end. So, we need a way to remove the files used by the process
> (apply worker in this particular case) before process exit and used
> this proc_exit hook (possibly on the lines of AtProcExit_Files).
What I'm wondering is why it is a good idea to have a SharedFileSet specific
cleanup mechanism. One that only operates on process lifetime level, rather
than something more granular. I get that the of the files here needs to be
longer than a transaction, but that can easily be addressed by having a longer
lived resource owner.
Process lifetime may work well for the current worker.c, but even there it
doesn't seem optimal. One e.g. could easily imagine that we'd want to handle
connection errors or configuration changes without restarting the worker, in
which case process lifetime obviously isn't a good idea anymore.
I think SharedFileSetInit() needs a comment explaining that it needs to be
called in a process-lifetime memory context if used without dsm
segments. Because otherwise SharedFileSetDeleteOnProcExit() will access
already freed memory (both for filesetlist and the SharedFileSet itself).
Greetings,
Andres Freund
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-committers@postgresql.org
Cc: andres@anarazel.de, amit.kapila16@gmail.com, pgsql-hackers@postgresql.org, dilipbalaut@gmail.com, sawada.mshk@gmail.com, tgl@sss.pgh.pa.us
Subject: Re: pgsql: pgstat: Bring up pgstat in BaseInit() to fix uninitialized use o
In-Reply-To: <20210812124819.5mrurb6ueizmqgru@alap3.anarazel.de>
References: <20210807164850.52atvo2qkmtfnzgt@alap3.anarazel.de> <20210807190131.2bm24acbebl4wl6i@alap3.anarazel.de> <20210808022435.svp4spxlrfoi3w4p@alap3.anarazel.de> <CAD21AoCCAa+J1-udHRo5-Hbtv=D38WdZDAaXZGDbQQ_Vg_d3bQ@mail.gmail.com> <CAA4eK1LYw4J+BmYza+1VWF3QGo5NaV7goXFJ8g5gfA3WZRo6Xw@mail.gmail.com> <CAD21AoAjrg5EuWM9qNApKRLC2LP+G18=u8xE=qbm2smmMzGC-A@mail.gmail.com> <CAFiTN-uzNeU+cO-JZW6WCNAXu0qZgOoqzGrsP8y+k2BKu4SHdQ@mail.gmail.com> <CAA4eK1JHUoE997NdUXbS9ikHk1oOtPupnNpGGnN_P3pn+jb6LQ@mail.gmail.com> <20210812082237.in7l5x3rc4gxhgjh@alap3.anarazel.de> <CAA4eK1L=fepiRg8PCx7_0z2Xgcu3bohzs=zMS6qutq0oD07wSQ@mail.gmail.com> <20210812124819.5mrurb6ueizmqgru@alap3.anarazel.de>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox