public inbox for [email protected]
help / color / mirror / Atom feedFrom: Bossart, Nathan <[email protected]>
To: Bharath Rupireddy <[email protected]>
Cc: [email protected] <[email protected]>
Subject: Re: make MaxBackends available in _PG_init
Date: Mon, 2 Aug 2021 18:18:25 +0000
Message-ID: <[email protected]> (raw)
In-Reply-To: <CALj2ACUHruScDf2dvnVso03bhSdr7MVvLie5C8ND1dR+hJrFEg@mail.gmail.com>
References: <4f20d57b2aeb447b8eb1495319940c5f@G08CNEXMBPEKD06.g08.fujitsu.local>
<CALj2ACUHruScDf2dvnVso03bhSdr7MVvLie5C8ND1dR+hJrFEg@mail.gmail.com>
On 9/21/20, 4:52 AM, "Bharath Rupireddy" <[email protected]> wrote:
> On Mon, Sep 21, 2020 at 9:14 AM Wang, Shenhao
> <[email protected]> wrote:
>>
>> In source, I find that the postmaster will first load library, and then calculate the value of MaxBackends.
>>
>> In the old version, the MaxBackends was calculated by:
>> MaxBackends = MaxConnections + autovacuum_max_workers + 1 +
>> GetNumShmemAttachedBgworkers();
>> Because any extension can register workers which will affect the return value of GetNumShmemAttachedBgworkers.
>> InitializeMaxBackends must be called after shared_preload_libraries. This is also mentioned in comments.
>>
>> Now, function GetNumShmemAttachedBgworkers was deleted and replaced by guc max_worker_processes,
>> so if we changed the calling order like:
>> Step1: calling InitializeMaxBackends.
>> Step2: calling process_shared_preload_libraries
>>
>
> Yes, the GetNumShmemAttachedBgworkers() was removed by commit #
> dfbba2c86cc8f09cf3ffca3d305b4ce54a7fb49a. ASAICS, changing the order
> of InitializeMaxBackends() and process_shared_preload_libraries() has
> no problem, as InitializeMaxBackends() doesn't calculate the
> MaxBackends based on bgworker infra code, it does calculate based on
> GUCs.
>
> Having said that, I'm not quite sure whether any of the bgworker
> registration code, for that matter process_shared_preload_libraries()
> code path will somewhere use MaxBackends?
>
>>
>> In this order extension can get the correct value of MaxBackends in _PG_init.
>>
>
> Is there any specific use case that any of the _PG_init will use MaxBackends?
I just encountered the same thing, so I am bumping this thread. I was
trying to use MaxBackends in a call to RequestAddinShmemSpace() in a
_PG_init() function for a module, but since MaxBackends is not yet
initialized, you essentially need to open-code InitializeMaxBackends()
instead.
I think the comments about needing to register background workers
before initializing MaxBackends have been incorrect since the addition
of max_worker_processes in v9.4 (6bc8ef0b). Furthermore, I think the
suggested reordering is a good idea because it is not obvious that
MaxBackends will be uninitialized in _PG_init(), and use-cases like
the RequestAddinShmemSpace() one are not guaranteed to fail when
MaxBackends is used incorrectly (presumably due to the 100 KB buffer
added in CreateSharedMemoryAndSemaphores()).
I've attached a new version of the proposed patch with some slight
adjustments and an attempt at a commit message.
Nathan
Attachments:
[application/octet-stream] v1-0001-Calculate-MaxBackends-earlier-in-PostmasterMain.patch (2.9K, ../[email protected]/2-v1-0001-Calculate-MaxBackends-earlier-in-PostmasterMain.patch)
download | inline diff:
From 09f678653820b62d0823ea6c951adb3ff2f470ea Mon Sep 17 00:00:00 2001
From: Nathan Bossart <[email protected]>
Date: Mon, 2 Aug 2021 17:42:25 +0000
Subject: [PATCH v1 1/1] Calculate MaxBackends earlier in PostmasterMain().
Presently, InitializeMaxBackends() is called after processing
shared_preload_libraries because it used to tally up the number of
registered background workers requested by the libraries. Since
6bc8ef0b, InitializeMaxBackends() has simply used the
max_worker_processes GUC instead, so all the comments about needing
to register background workers before initializing MaxBackends are
no longer correct.
In addition to revising the comments, this patch reorders
InitializeMaxBackends() to before shared_preload_libraries is
processed so that modules can make use of MaxBackends in their
_PG_init() functions.
---
src/backend/postmaster/postmaster.c | 19 +++++++++----------
src/backend/utils/init/postinit.c | 4 +---
2 files changed, 10 insertions(+), 13 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 00d051d520..5eff4610fd 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -990,10 +990,15 @@ PostmasterMain(int argc, char *argv[])
LocalProcessControlFile(false);
/*
- * Register the apply launcher. Since it registers a background worker,
- * it needs to be called before InitializeMaxBackends(), and it's probably
- * a good idea to call it before any modules had chance to take the
- * background worker slots.
+ * Calculate MaxBackends. This is done before processing
+ * shared_preload_libraries so that such libraries can make use of it in
+ * _PG_init().
+ */
+ InitializeMaxBackends();
+
+ /*
+ * Register the apply launcher. It's probably a good idea to call it before
+ * any modules had chance to take the background worker slots.
*/
ApplyLauncherRegister();
@@ -1013,12 +1018,6 @@ PostmasterMain(int argc, char *argv[])
}
#endif
- /*
- * Now that loadable modules have had their chance to register background
- * workers, calculate MaxBackends.
- */
- InitializeMaxBackends();
-
/*
* Set up shared memory and semaphores.
*/
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 51d1bbef30..f8136dfc6f 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -502,9 +502,7 @@ pg_split_opts(char **argv, int *argcp, const char *optstr)
/*
* Initialize MaxBackends value from config options.
*
- * This must be called after modules have had the chance to register background
- * workers in shared_preload_libraries, and before shared memory size is
- * determined.
+ * This must be called before shared memory size is determined.
*
* Note that in EXEC_BACKEND environment, the value is passed down from
* postmaster to subprocesses via BackendParameters in SubPostmasterMain; only
--
2.16.6
view thread (67+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected]
Subject: Re: make MaxBackends available in _PG_init
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox