pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feedAuxiliary Processes and MyAuxProc
38+ messages / 9 participants
[nested] [flat]
* Auxiliary Processes and MyAuxProc
@ 2019-02-25 17:58 Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 38+ messages in thread
From: Mike Palmiotto @ 2019-02-25 17:58 UTC (permalink / raw)
To: pgsql-hackers
Greetings,
I've been looking through the startup code for postmaster forks and
saw a couple of mechanisms used. Most forks seem to be using
StartChildProcess with a MyAuxProc emumerator, but then some
(autovacuum launcher/worker, syslogger, bgworker, archiver) are forked
through their own start functions.
I noticed some implication in the pgsql-hackers archives [1] that
non-AuxProcs are as such because they don't need access to shared
memory. Is this an accurate explanation?
For some context, I'm trying to come up with a patch to set the
process identifier (MyAuxProc/am_autovacuumworker/launcher,
am_archiver, etc.) immediately after forking, so we can provide
process context info to a hook in early startup. I wanted to make sure
to do things as cleanly as possible.
With the current startup architecture, we have to touch multiple
entrypoints to achieve the desired effect. Is there any particular
reason we couldn't fold all of the startup processes into the
StartChildProcess code and assign MyAuxProc for processes that don't
currently have one, or is this a non-starter?
Thank you for your time.
--
Mike Palmiotto
Software Engineer
Crunchy Data Solutions
https://crunchydata.com
[1] https://www.postgresql.org/message-id/20181127.175949.06807946.horiguchi.kyotaro%40lab.ntt.co.jp
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
@ 2019-02-25 18:29 ` Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
0 siblings, 1 reply; 38+ messages in thread
From: Tom Lane @ 2019-02-25 18:29 UTC (permalink / raw)
To: Mike Palmiotto <mike.palmiotto@crunchydata.com>; +Cc: pgsql-hackers
Mike Palmiotto <mike.palmiotto@crunchydata.com> writes:
> I've been looking through the startup code for postmaster forks and
> saw a couple of mechanisms used. Most forks seem to be using
> StartChildProcess with a MyAuxProc emumerator, but then some
> (autovacuum launcher/worker, syslogger, bgworker, archiver) are forked
> through their own start functions.
> I noticed some implication in the pgsql-hackers archives [1] that
> non-AuxProcs are as such because they don't need access to shared
> memory. Is this an accurate explanation?
That was the original idea, but it hasn't stood the test of time very
well. In particular, the AuxProcType enum only works for child
processes that there's supposed to be exactly one of, so we haven't
used it for autovac workers or bgworkers, although those certainly
have to be connected to shmem (hm, maybe that's not true for all
types of bgworker, not sure). The autovac launcher is kind of a
weird special case --- I think it could have been included in AuxProcType,
but it wasn't, possibly to minimize the cosmetic difference between it
and autovac workers.
> For some context, I'm trying to come up with a patch to set the
> process identifier (MyAuxProc/am_autovacuumworker/launcher,
> am_archiver, etc.) immediately after forking,
Don't we do that already?
> With the current startup architecture, we have to touch multiple
> entrypoints to achieve the desired effect. Is there any particular
> reason we couldn't fold all of the startup processes into the
> StartChildProcess code and assign MyAuxProc for processes that don't
> currently have one, or is this a non-starter?
If memory serves, StartChildProcess already was an attempt to unify
the treatment of postmaster children. It's possible that another
round of unification would be productive, but I think you'll find
that there are random small differences in requirements that'd
make it messy.
regards, tom lane
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
@ 2019-02-25 18:41 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
0 siblings, 1 reply; 38+ messages in thread
From: Mike Palmiotto @ 2019-02-25 18:41 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-hackers
On Mon, Feb 25, 2019 at 1:29 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> Mike Palmiotto <mike.palmiotto@crunchydata.com> writes:
> <snip>
>
> > For some context, I'm trying to come up with a patch to set the
> > process identifier (MyAuxProc/am_autovacuumworker/launcher,
> > am_archiver, etc.) immediately after forking,
>
> Don't we do that already?
Kind of. The indicators are set in the Main functions after
InitPostmasterChild and some other setup. In my little bit of digging
I found InitPostmasterChild to be the best place for a centralized
"early start-up hook." Is there a better place you can think of
off-hand?
> <snip>
>
> If memory serves, StartChildProcess already was an attempt to unify
> the treatment of postmaster children. It's possible that another
> round of unification would be productive, but I think you'll find
> that there are random small differences in requirements that'd
> make it messy.
It kind of seemed like it, but I noticed the small differences in
requirements, which made me a bit hesitant. I'll go ahead and see what
I can do and submit the patch for consideration.
--
Mike Palmiotto
Software Engineer
Crunchy Data Solutions
https://crunchydata.com
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
@ 2019-02-25 22:25 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
0 siblings, 1 reply; 38+ messages in thread
From: Mike Palmiotto @ 2019-02-25 22:25 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-hackers
On Mon, Feb 25, 2019 at 1:41 PM Mike Palmiotto
<mike.palmiotto@crunchydata.com> wrote:
>
> <snip>
> >
> > If memory serves, StartChildProcess already was an attempt to unify
> > the treatment of postmaster children. It's possible that another
> > round of unification would be productive, but I think you'll find
> > that there are random small differences in requirements that'd
> > make it messy.
>
> It kind of seemed like it, but I noticed the small differences in
> requirements, which made me a bit hesitant. I'll go ahead and see what
> I can do and submit the patch for consideration.
I'm considering changing StartChildProcess to take a struct with data
for forking/execing each different process. Each different backend
type would build up the struct and then pass it on to
StartChildProcess, which would handle each separately. This would
ensure that the fork type is set prior to InitPostmasterChild and
would provide us with the information necessary to do what we need in
the InitPostmasterChild_hook.
Attached is a patch to fork_process.h which shows roughly what I'm
thinking. Does this seem somewhat sane as a first step?
--
Mike Palmiotto
Software Engineer
Crunchy Data Solutions
https://crunchydata.com
Attachments:
fork-proc-type.patch (text/x-patch, 768B) download
diff --git a/src/include/postmaster/fork_process.h b/src/include/postmaster/fork_process.h
index eb3c8ec974..6cf0bfa2fe 100644
--- a/src/include/postmaster/fork_process.h
+++ b/src/include/postmaster/fork_process.h
@@ -12,6 +12,29 @@
#ifndef FORK_PROCESS_H
#define FORK_PROCESS_H
+typedef enum
+{
+ CheckerProcess = 0,
+ BootstrapProcess,
+ StartupProcess,
+ BgWriterProcess,
+ CheckpointerProcess,
+ WalWriterProcess,
+ WalReceiverProcess,
+ AutoVacuumLauncherProcess,
+ AutoVacuumWorkerProcess,
+ SysloggerProcess,
+ BgWorkerProcess
+
+ NUMFORKPROCTYPES /* Must be last! */
+} ForkProcType;
+
+typedef struct ForkProcData
+{
+ ForkProcType type;
+ char *fork_av[10];
+} ForkProcData
+
extern pid_t fork_process(void);
#endif /* FORK_PROCESS_H */
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
@ 2019-08-14 15:01 ` Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
0 siblings, 1 reply; 38+ messages in thread
From: Yuli Khodorkovskiy @ 2019-08-14 15:01 UTC (permalink / raw)
To: Mike Palmiotto <mike.palmiotto@crunchydata.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Mon, Feb 25, 2019 at 5:25 PM Mike Palmiotto
<mike.palmiotto@crunchydata.com> wrote:
>
> On Mon, Feb 25, 2019 at 1:41 PM Mike Palmiotto
> <mike.palmiotto@crunchydata.com> wrote:
> >
> > <snip>
> > >
> > > If memory serves, StartChildProcess already was an attempt to unify
> > > the treatment of postmaster children. It's possible that another
> > > round of unification would be productive, but I think you'll find
> > > that there are random small differences in requirements that'd
> > > make it messy.
> >
> > It kind of seemed like it, but I noticed the small differences in
> > requirements, which made me a bit hesitant. I'll go ahead and see what
> > I can do and submit the patch for consideration.
>
> I'm considering changing StartChildProcess to take a struct with data
> for forking/execing each different process. Each different backend
> type would build up the struct and then pass it on to
> StartChildProcess, which would handle each separately. This would
> ensure that the fork type is set prior to InitPostmasterChild and
> would provide us with the information necessary to do what we need in
> the InitPostmasterChild_hook.
>
> Attached is a patch to fork_process.h which shows roughly what I'm
> thinking. Does this seem somewhat sane as a first step?
>
All,
Mike and I have written two patches to solve the issues
discussed in this thread.
The first patch centralizes the startup of workers and extends
worker identification that was introduced by AuxProcType. The worker
id can then be leveraged by extensions for identification of each
process.
The second patch adds a new hook that allows extensions to modify a worker
process' metadata in backends.
These patches should make future worker implementation more
straightforward as there is now one function to call that sets up
each worker. There is also code cleanup and removal of startup
redundancies.
Please let me know your thoughts. I look forward to your feedback.
Thanks,
Yuli
Attachments: download all (2 files, .tar.gz)
Add-a-hook-to-allow-extensions-to-set-worker-metadat.patch (application/octet-stream, 2.0K) download
From e907e8c0c0e5bc476d8eb400c0db0848ff863f36 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 19 Feb 2019 15:29:33 +0000
Subject: [PATCH 1/2] Add a hook to allow extensions to set worker metadata
This patch implements a hook that allows extensions to modify a worker
process' metadata in backends.
Additional work done by Yuli Khodorkovskiy <yuli@crunchydata.com>
Original discussion here: https://www.postgresql.org/message-id/flat/CAMN686FE0OdZKp9YPO%3DhtC6LnA6aW4r-%2Bjq%3D3Q5RAoFQgW8EtA%40mail.gmail.com
---
src/backend/postmaster/fork_process.c | 11 +++++++++++
src/include/postmaster/fork_process.h | 4 ++++
2 files changed, 15 insertions(+)
diff --git a/src/backend/postmaster/fork_process.c b/src/backend/postmaster/fork_process.c
index a9138f589e..964c04d846 100644
--- a/src/backend/postmaster/fork_process.c
+++ b/src/backend/postmaster/fork_process.c
@@ -21,6 +21,14 @@
#include <openssl/rand.h>
#endif
+/*
+ * This hook allows plugins to get control of the child process following
+ * a successful fork_process(). It can be used to perform some action in
+ * extensions to set metadata in backends (including special backends) upon
+ * setting of the session user.
+ */
+ForkProcess_post_hook_type ForkProcess_post_hook = NULL;
+
#ifndef WIN32
/*
* Wrapper for fork(). Return values are the same as those for fork():
@@ -113,6 +121,9 @@ fork_process(void)
#ifdef USE_OPENSSL
RAND_cleanup();
#endif
+
+ if (ForkProcess_post_hook)
+ (*ForkProcess_post_hook) ();
}
return result;
diff --git a/src/include/postmaster/fork_process.h b/src/include/postmaster/fork_process.h
index eb3c8ec974..30be2faa7c 100644
--- a/src/include/postmaster/fork_process.h
+++ b/src/include/postmaster/fork_process.h
@@ -14,4 +14,8 @@
extern pid_t fork_process(void);
+/* Hook for plugins to get control after a successful fork_process() */
+typedef void (*ForkProcess_post_hook_type) ();
+extern PGDLLIMPORT ForkProcess_post_hook_type ForkProcess_post_hook;
+
#endif /* FORK_PROCESS_H */
--
2.19.0
Refactor-child-process-startup.patch (application/octet-stream, 70.1K) download
From c70168db4cea2e86a421f1cf38c13b33f8e0a882 Mon Sep 17 00:00:00 2001
From: Yuli Khodorkovskiy <yuli@crunchydata.com>
Date: Wed, 27 Feb 2019 09:21:13 -0500
Subject: [PATCH 2/2] Refactor child process startup
This patch centralizes child process startup. StartChildProcess() now
gains ownership of all worker initialization and forking, rather
than only workers of AuxProcType. An added benefit of using the centralized
startup is to make implementation of future workers easier as well as removal
of redundant code.
StartChildProcess is responsible for prepping ForkProcData for each
worker type by setting fork or fork/exec arguments. Additionally, the
the callback functions for the worker are set up using child_main
(child), postmaster_main (parent), and fail_main (startup failure).
This patch also extends the concept of AuxProcType to all workers using
ForkProcType. ForkProcType along with ForkProcess_post_hook allows for
extensions to identify and gain control of worker processes.
Initial work done by Mike Palmiotto <mike.palmiotto@crunchydata.com>
Original discussion here: https://www.postgresql.org/message-id/flat/CAMN686FE0OdZKp9YPO%3DhtC6LnA6aW4r-%2Bjq%3D3Q5RAoFQgW8EtA%40mail.gmail.com
---
src/backend/bootstrap/bootstrap.c | 1 +
src/backend/main/main.c | 7 +
src/backend/postmaster/autovacuum.c | 157 +---
src/backend/postmaster/bgworker.c | 138 ++-
src/backend/postmaster/pgarch.c | 89 +-
src/backend/postmaster/pgstat.c | 88 +-
src/backend/postmaster/postmaster.c | 888 +++++++++-----------
src/backend/postmaster/syslogger.c | 152 ++--
src/backend/utils/init/globals.c | 3 +-
src/backend/utils/init/postinit.c | 2 +-
src/include/bootstrap/bootstrap.h | 2 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 4 +-
src/include/postmaster/autovacuum.h | 9 +-
src/include/postmaster/bgworker.h | 4 +
src/include/postmaster/bgworker_internals.h | 3 +-
src/include/postmaster/fork_process.h | 45 +
src/include/postmaster/pgarch.h | 5 +-
src/include/postmaster/postmaster.h | 54 +-
src/include/postmaster/syslogger.h | 3 +
20 files changed, 776 insertions(+), 879 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 9238fbe98d..9f3dad1c6d 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -70,6 +70,7 @@ static void cleanup(void);
*/
AuxProcType MyAuxProcType = NotAnAuxProcess; /* declared in miscadmin.h */
+ForkProcType MyForkProcType = NoForkProcess; /* declared in fork_process.h */
Relation boot_reldesc; /* current relation descriptor */
diff --git a/src/backend/main/main.c b/src/backend/main/main.c
index 7b18f8c758..e566858f7a 100644
--- a/src/backend/main/main.c
+++ b/src/backend/main/main.c
@@ -217,7 +217,14 @@ main(int argc, char *argv[])
#endif
if (argc > 1 && strcmp(argv[1], "--boot") == 0)
+ {
+ MemoryContext old_ctx;
+
+ old_ctx = MemoryContextSwitchTo(TopMemoryContext);
+
AuxiliaryProcessMain(argc, argv); /* does not return */
+ MemoryContextSwitchTo(old_ctx);
+ }
else if (argc > 1 && strcmp(argv[1], "--describe-config") == 0)
GucInfoMain(); /* does not return */
else if (argc > 1 && strcmp(argv[1], "--single") == 0)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 073f313337..5c3ac1568f 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -134,8 +134,8 @@ int Log_autovacuum_min_duration = -1;
#define MAX_AUTOVAC_SLEEPTIME 300 /* seconds */
/* Flags to tell if we are in an autovacuum process */
-static bool am_autovacuum_launcher = false;
-static bool am_autovacuum_worker = false;
+bool am_autovacuum_launcher = false;
+bool am_autovacuum_worker = false;
/* Flags set by signal handlers */
static volatile sig_atomic_t got_SIGHUP = false;
@@ -303,10 +303,6 @@ static WorkerInfo MyWorkerInfo = NULL;
/* PID of launcher, valid only in worker while shutting down */
int AutovacuumLauncherPid = 0;
-#ifdef EXEC_BACKEND
-static pid_t avlauncher_forkexec(void);
-static pid_t avworker_forkexec(void);
-#endif
NON_EXEC_STATIC void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
NON_EXEC_STATIC void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
@@ -347,88 +343,53 @@ static void avl_sigusr2_handler(SIGNAL_ARGS);
static void avl_sigterm_handler(SIGNAL_ARGS);
static void autovac_refresh_stats(void);
-
-
/********************************************************************
* AUTOVACUUM LAUNCHER CODE
********************************************************************/
-#ifdef EXEC_BACKEND
/*
- * forkexec routine for the autovacuum launcher process.
+ * PrepAutoVacProcessFork
*
* Format up the arglist, then fork and exec.
*/
-static pid_t
-avlauncher_forkexec(void)
+void
+PrepAutoVacProcessFork(ForkProcData *autovac_fork)
{
- char *av[10];
int ac = 0;
- av[ac++] = "postgres";
- av[ac++] = "--forkavlauncher";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
/*
- * We need this set from the outside, before InitProcess is called
+ * Set up command-line arguments for subprocess
*/
-void
-AutovacuumLauncherIAm(void)
-{
- am_autovacuum_launcher = true;
-}
-#endif
+ autovac_fork->av[ac++] = pstrdup("postgres");
-/*
- * Main entry point for autovacuum launcher process, to be called from the
- * postmaster.
- */
-int
-StartAutoVacLauncher(void)
+ if (MyForkProcType == AutoVacLauncherFork)
{
- pid_t AutoVacPID;
-
+ autovac_fork->type_desc = pstrdup("autovacuum launcher");
+ autovac_fork->child_main = AutoVacLauncherMain;
#ifdef EXEC_BACKEND
- switch ((AutoVacPID = avlauncher_forkexec()))
-#else
- switch ((AutoVacPID = fork_process()))
+ autovac_fork->av[ac++] = pstrdup("--forkavlauncher");
+ autovac_fork->av[ac++] = NULL; /* filled in by postmaster_forkexec */
#endif
+ }
+ else if (MyForkProcType == AutoVacWorkerFork)
{
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum launcher process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacLauncherMain(0, NULL);
- break;
+ autovac_fork->type_desc = pstrdup("autovacuum worker");
+ autovac_fork->child_main = AutoVacWorkerMain;
+#ifdef EXEC_BACKEND
+ autovac_fork->av[ac++] = pstrdup("--forkavworker");
+ autovac_fork->av[ac++] = NULL; /* filled in by postmaster_forkexec */
#endif
- default:
- return (int) AutoVacPID;
}
- /* shouldn't get here */
- return 0;
+ autovac_fork->ac = ac;
+ Assert(autovac_fork->ac < lengthof(*autovac_fork->av));
}
/*
* Main loop for the autovacuum launcher process.
*/
NON_EXEC_STATIC void
-AutoVacLauncherMain(int argc, char *argv[])
+AutoVacLauncherMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
@@ -1428,83 +1389,11 @@ avl_sigterm_handler(SIGNAL_ARGS)
* AUTOVACUUM WORKER CODE
********************************************************************/
-#ifdef EXEC_BACKEND
-/*
- * forkexec routines for the autovacuum worker.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avworker_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavworker";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
-/*
- * We need this set from the outside, before InitProcess is called
- */
-void
-AutovacuumWorkerIAm(void)
-{
- am_autovacuum_worker = true;
-}
-#endif
-
-/*
- * Main entry point for autovacuum worker process.
- *
- * This code is heavily based on pgarch.c, q.v.
- */
-int
-StartAutoVacWorker(void)
-{
- pid_t worker_pid;
-
-#ifdef EXEC_BACKEND
- switch ((worker_pid = avworker_forkexec()))
-#else
- switch ((worker_pid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum worker process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacWorkerMain(0, NULL);
- break;
-#endif
- default:
- return (int) worker_pid;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
/*
* AutoVacWorkerMain
*/
NON_EXEC_STATIC void
-AutoVacWorkerMain(int argc, char *argv[])
+AutoVacWorkerMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
Oid dbid;
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index b66b517aca..8c60a13cd1 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -36,6 +36,7 @@
#include "utils/ascii.h"
#include "utils/ps_status.h"
#include "utils/timeout.h"
+#include "utils/timestamp.h"
/*
* The postmaster's list of registered background workers, in private memory.
@@ -134,7 +135,110 @@ static const struct
/* Private functions. */
static bgworker_main_type LookupBackgroundWorkerFunction(const char *libraryname, const char *funcname);
+static bool assign_backendlist_entry(RegisteredBgWorker *rw);
+NON_EXEC_STATIC void BackgroundWorkerFailFork(int argc, char *argv[]);
+NON_EXEC_STATIC void BackgroundWorkerPostmasterMain(int argc, char *argv[]);
+/*
+ * Allocate the Backend struct for a connected background worker, but don't
+ * add it to the list of backends just yet.
+ *
+ * On failure, return false without changing any worker state.
+ *
+ * Some info from the Backend is copied into the passed rw.
+ */
+static bool
+assign_backendlist_entry(RegisteredBgWorker *rw)
+{
+ Backend *bn;
+
+ /*
+ * Compute the cancel key that will be assigned to this session. We
+ * probably don't need cancel keys for background workers, but we'd better
+ * have something random in the field to prevent unfriendly people from
+ * sending cancels to them.
+ */
+ if (!RandomCancelKey(&MyCancelKey))
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random cancel key")));
+ return false;
+ }
+
+ bn = malloc(sizeof(Backend));
+ if (bn == NULL)
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+ return false;
+ }
+
+ bn->cancel_key = MyCancelKey;
+ bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
+ bn->bkend_type = BACKEND_TYPE_BGWORKER;
+ bn->dead_end = false;
+ bn->bgworker_notify = false;
+
+ rw->rw_backend = bn;
+ rw->rw_child_slot = bn->child_slot;
+
+ return true;
+}
+
+/*
+ * PrepBgWorkerFork
+ *
+ * Postmaster subroutine to prepare a bgworker subprocess.
+ *
+ * Returns 0 on success or -1 on failure.
+ *
+ * Note: if fail, we will be called again from the postmaster main loop.
+ */
+int
+PrepBgWorkerFork(ForkProcData *bgworker_fork)
+{
+ int ac = 0;
+ RegisteredBgWorker *rw = CurrentBgWorker;
+
+ Assert(CurrentBgWorker);
+ Assert(rw->rw_pid == 0);
+
+ /*
+ * Allocate and assign the Backend element. Note we must do this before
+ * forking, so that we can handle out of memory properly.
+ *
+ * Treat failure as though the worker had crashed. That way, the
+ * postmaster will wait a bit before attempting to start it again; if it
+ * tried again right away, most likely it'd find itself repeating the
+ * out-of-memory or fork failure condition.
+ */
+ if (!assign_backendlist_entry(rw))
+ {
+ rw->rw_crashed_at = GetCurrentTimestamp();
+ return -1;
+ }
+
+ ereport(DEBUG1,
+ (errmsg("starting background worker process \"%s\"",
+ rw->rw_worker.bgw_name)));
+
+#ifdef EXEC_BACKEND
+ bgworker_fork->av[ac++] = pstrdup("postgres");
+ bgworker_fork->av[ac++] = psprintf("--forkbgworker=%d", rw->rw_shmem_slot);
+ bgworker_fork->av[ac++] = NULL; /* filled in by postmaster_forkexec */
+ bgworker_fork->av[ac] = NULL;
+#endif
+ Assert(bgworker_fork->ac < lengthof(*bgworker_fork->av));
+ bgworker_fork->ac = ac;
+ bgworker_fork->type_desc = pstrdup("background worker");
+ bgworker_fork->child_main = BackgroundWorkerMain;
+ bgworker_fork->fail_main = BackgroundWorkerFailFork;
+ bgworker_fork->postmaster_main = BackgroundWorkerPostmasterMain;
+
+ return 0;
+}
/*
* Calculate shared memory needed.
@@ -698,7 +802,7 @@ bgworker_sigusr1_handler(SIGNAL_ARGS)
* postmaster.
*/
void
-StartBackgroundWorker(void)
+BackgroundWorkerMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
BackgroundWorker *worker = MyBgworkerEntry;
@@ -837,6 +941,38 @@ StartBackgroundWorker(void)
proc_exit(0);
}
+NON_EXEC_STATIC void
+BackgroundWorkerFailFork(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ RegisteredBgWorker *rw = CurrentBgWorker;
+
+ /* in postmaster, fork failed ... */
+ ereport(LOG,
+ (errmsg("could not fork worker process: %m")));
+
+ /* undo what assign_backendlist_entry did */
+ ReleasePostmasterChildSlot(rw->rw_child_slot);
+ rw->rw_child_slot = 0;
+ free(rw->rw_backend);
+ rw->rw_backend = NULL;
+ /* mark entry as crashed, so we'll try again later */
+ rw->rw_crashed_at = GetCurrentTimestamp();
+}
+
+NON_EXEC_STATIC void
+BackgroundWorkerPostmasterMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ RegisteredBgWorker *rw = CurrentBgWorker;
+
+ rw->rw_pid = MyChildProcPid;
+ rw->rw_backend->pid = rw->rw_pid;
+ ReportBackgroundWorkerPID(rw);
+ /* add new worker to lists of backends */
+ dlist_push_head(&BackendList, &(rw->rw_backend->elem));
+#ifdef EXEC_BACKEND
+ ShmemBackendArrayAdd(rw->rw_backend);
+#endif
+}
/*
* Register a new static background worker.
*
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index f84f882c4c..53661004fd 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -92,10 +92,6 @@ static volatile sig_atomic_t ready_to_stop = false;
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgarch_forkexec(void);
-#endif
-
NON_EXEC_STATIC void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgarch_exit(SIGNAL_ARGS);
static void ArchSigHupHandler(SIGNAL_ARGS);
@@ -115,7 +111,7 @@ static void pgarch_archiveDone(char *xlog);
*/
/*
- * pgarch_start
+ * PrepPgArchiverFork
*
* Called from postmaster at startup or after an existing archiver
* died. Attempt to fire up a fresh archiver process.
@@ -124,17 +120,17 @@ static void pgarch_archiveDone(char *xlog);
*
* Note: if fail, we will be called again from the postmaster main loop.
*/
-int
-pgarch_start(void)
+void
+PrepPgArchiverFork(ForkProcData *pgarch_fork)
{
+ int ac = 0;
time_t curtime;
- pid_t pgArchPid;
/*
* Do nothing if no archiver needed
*/
if (!XLogArchivingActive())
- return 0;
+ return;
/*
* Do nothing if too soon since last archiver start. This is a safety
@@ -145,42 +141,20 @@ pgarch_start(void)
curtime = time(NULL);
if ((unsigned int) (curtime - last_pgarch_start_time) <
(unsigned int) PGARCH_RESTART_INTERVAL)
- return 0;
+ return;
last_pgarch_start_time = curtime;
-#ifdef EXEC_BACKEND
- switch ((pgArchPid = pgarch_forkexec()))
-#else
- switch ((pgArchPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork archiver: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
+ pgarch_fork->av[ac++] = pstrdup("postgres");
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgArchiverMain(0, NULL);
- break;
+#ifdef EXEC_BACKEND
+ pgarch_fork->av[ac++] = pstrdup("--forkarch");
+ pgarch_fork->av[ac++] = NULL; /* filled in by postmaster_forkexec */
#endif
+ pgarch_fork->ac = ac;
- default:
- return (int) pgArchPid;
- }
-
- /* shouldn't get here */
- return 0;
+ Assert(pgarch_fork->ac < lengthof(*pgarch_fork->av));
+ pgarch_fork->child_main = PgArchiverMain;
+ pgarch_fork->type_desc = pstrdup("archiver");
}
/* ------------------------------------------------------------
@@ -189,42 +163,19 @@ pgarch_start(void)
*/
-#ifdef EXEC_BACKEND
-
-/*
- * pgarch_forkexec() -
- *
- * Format up the arglist for, then fork and exec, archive process
- */
-static pid_t
-pgarch_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
-
- av[ac++] = "--forkarch";
-
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
/*
* PgArchiverMain
*
* The argc/argv parameters are valid only in EXEC_BACKEND case. However,
* since we don't use 'em, it hardly matters...
*/
-NON_EXEC_STATIC void
-PgArchiverMain(int argc, char *argv[])
+void
+PgArchiverMain(pg_attribute_unused() int argc, pg_attribute_unused() char *arg[])
{
+ /* Drop our connection to postmaster's shared memory, as well */
+ dsm_detach_all();
+ PGSharedMemoryDetach();
+
/*
* Ignore all signals usually bound to some action in the postmaster,
* except for SIGHUP, SIGTERM, SIGUSR1, SIGUSR2, and SIGQUIT.
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 2bb14cdd02..0e1da8f377 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -280,10 +280,6 @@ static instr_time total_func_time;
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgstat_forkexec(void);
-#endif
-
NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgstat_exit(SIGNAL_ARGS);
static void pgstat_beshutdown_hook(int code, Datum arg);
@@ -689,53 +685,26 @@ pgstat_reset_all(void)
pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
}
-#ifdef EXEC_BACKEND
/*
- * pgstat_forkexec() -
- *
- * Format up the arglist for, then fork and exec, statistics collector process
- */
-static pid_t
-pgstat_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkcol";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
-/*
- * pgstat_start() -
+ * PrepPgstatCollectorFork
*
* Called from postmaster at startup or after an existing collector
* died. Attempt to fire up a fresh statistics collector.
*
- * Returns PID of child process, or 0 if fail.
- *
- * Note: if fail, we will be called again from the postmaster main loop.
*/
-int
-pgstat_start(void)
+void
+PrepPgstatCollectorFork(ForkProcData *pgstat_fork)
{
+ int ac = 0;
time_t curtime;
- pid_t pgStatPid;
/*
* Check that the socket is there, else pgstat_init failed and we can do
* nothing useful.
*/
if (pgStatSock == PGINVALID_SOCKET)
- return 0;
+ return;
/*
* Do nothing if too soon since last collector start. This is a safety
@@ -746,45 +715,20 @@ pgstat_start(void)
curtime = time(NULL);
if ((unsigned int) (curtime - last_pgstat_start_time) <
(unsigned int) PGSTAT_RESTART_INTERVAL)
- return 0;
+ return;
last_pgstat_start_time = curtime;
- /*
- * Okay, fork off the collector.
- */
#ifdef EXEC_BACKEND
- switch ((pgStatPid = pgstat_forkexec()))
-#else
- switch ((pgStatPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork statistics collector: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgstatCollectorMain(0, NULL);
- break;
+ pgstat_fork->av[ac++] = pstrdup("postgres");
+ pgstat_fork->av[ac++] = pstrdup("--forkcol");
+ pgstat_fork->av[ac++] = NULL; /* filled in by postmaster_forkexec */
#endif
- default:
- return (int) pgStatPid;
- }
+ pgstat_fork->ac = ac;
+ Assert(pgstat_fork->ac < lengthof(*pgstat_fork->av));
- /* shouldn't get here */
- return 0;
+ pgstat_fork->type_desc = pstrdup("statistics collector");
+ pgstat_fork->child_main = PgstatCollectorMain;
}
void
@@ -4425,12 +4369,16 @@ pgstat_send_bgwriter(void)
* ----------
*/
NON_EXEC_STATIC void
-PgstatCollectorMain(int argc, char *argv[])
+PgstatCollectorMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
int len;
PgStat_Msg msg;
int wr;
+ /* Drop our connection to postmaster's shared memory, as well */
+ dsm_detach_all();
+ PGSharedMemoryDetach();
+
/*
* Ignore all signals usually bound to some action in the postmaster,
* except SIGHUP and SIGQUIT. Note we don't need a SIGUSR1 handler to
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 3339804be9..e5492b8e42 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -136,62 +136,16 @@
#include "storage/spin.h"
#endif
-
-/*
- * Possible types of a backend. Beyond being the possible bkend_type values in
- * struct bkend, these are OR-able request flag bits for SignalSomeChildren()
- * and CountChildren().
- */
-#define BACKEND_TYPE_NORMAL 0x0001 /* normal backend */
-#define BACKEND_TYPE_AUTOVAC 0x0002 /* autovacuum worker process */
-#define BACKEND_TYPE_WALSND 0x0004 /* walsender process */
-#define BACKEND_TYPE_BGWORKER 0x0008 /* bgworker process */
-#define BACKEND_TYPE_ALL 0x000F /* OR of all the above */
-
-#define BACKEND_TYPE_WORKER (BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER)
-
-/*
- * List of active backends (or child processes anyway; we don't actually
- * know whether a given child has become a backend or is still in the
- * authorization phase). This is used mainly to keep track of how many
- * children we have and send them appropriate signals when necessary.
- *
- * "Special" children such as the startup, bgwriter and autovacuum launcher
- * tasks are not in this list. Autovacuum worker and walsender are in it.
- * Also, "dead_end" children are in it: these are children launched just for
- * the purpose of sending a friendly rejection message to a would-be client.
- * We must track them because they are attached to shared memory, but we know
- * they will never become live backends. dead_end children are not assigned a
- * PMChildSlot.
- *
- * Background workers are in this list, too.
- */
-typedef struct bkend
-{
- pid_t pid; /* process id of backend */
- int32 cancel_key; /* cancel key for cancels for this backend */
- int child_slot; /* PMChildSlot for this backend, if any */
-
- /*
- * Flavor of backend or auxiliary process. Note that BACKEND_TYPE_WALSND
- * backends initially announce themselves as BACKEND_TYPE_NORMAL, so if
- * bkend_type is normal, you should check for a recent transition.
- */
- int bkend_type;
- bool dead_end; /* is it going to send an error and quit? */
- bool bgworker_notify; /* gets bgworker start/stop notifications */
- dlist_node elem; /* list link in BackendList */
-} Backend;
-
-static dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
+dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
#ifdef EXEC_BACKEND
static Backend *ShmemBackendArray;
#endif
BackgroundWorker *MyBgworkerEntry = NULL;
-
-
+RegisteredBgWorker *CurrentBgWorker = NULL;
+static Backend *MyBackend;
+static int child_errno;
/* The socket number we are listening for connections on */
int PostPortNumber;
@@ -403,25 +357,25 @@ static void BackendInitialize(Port *port);
static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
-static int BackendStartup(Port *port);
static int ProcessStartupPacket(Port *port, bool secure_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
static int initMasks(fd_set *rmask);
static void report_fork_failure_to_client(Port *port, int errnum);
static CAC_state canAcceptConnections(void);
-static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+#ifdef EXEC_BACKEND
+static void shmemSetup(bool aux_process);
+#endif
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
-static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
-static pid_t StartChildProcess(AuxProcType type);
+static pid_t StartChildProcess(ForkProcType type);
static void StartAutovacuumWorker(void);
static void MaybeStartWalReceiver(void);
static void InitPostmasterDeathWatchHandle(void);
@@ -455,8 +409,7 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(Port *port);
-static pid_t internal_forkexec(int argc, char *argv[], Port *port);
+static pid_t internal_forkexec(int argc, char *argv[]);
/* Type for a socket that can be inherited to a client process */
#ifdef WIN32
@@ -522,6 +475,7 @@ typedef struct
char my_exec_path[MAXPGPATH];
char pkglib_path[MAXPGPATH];
char ExtraOptions[MAXPGPATH];
+ ForkProcType proc_type;
} BackendParameters;
static void read_backend_variables(char *id, Port *port);
@@ -534,15 +488,23 @@ static bool save_backend_variables(BackendParameters *param, Port *port,
HANDLE childProcess, pid_t childPid);
#endif
-static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
-#define StartupDataBase() StartChildProcess(StartupProcess)
-#define StartBackgroundWriter() StartChildProcess(BgWriterProcess)
-#define StartCheckpointer() StartChildProcess(CheckpointerProcess)
-#define StartWalWriter() StartChildProcess(WalWriterProcess)
-#define StartWalReceiver() StartChildProcess(WalReceiverProcess)
+#define BGWORKER_LEN 15
+
+#define StartupDataBase() StartChildProcess(StartupFork)
+#define StartBackgroundWriter() StartChildProcess(BgWriterFork)
+#define StartCheckpointer() StartChildProcess(CheckpointerFork)
+#define StartWalWriter() StartChildProcess(WalWriterFork)
+#define StartWalReceiver() StartChildProcess(WalReceiverFork)
+#define StartAutoVacLauncher() StartChildProcess(AutoVacLauncherFork)
+#define StartAutoVacWorker() StartChildProcess(AutoVacWorkerFork)
+#define pgstat_start() StartChildProcess(PgstatCollectorFork)
+#define pgarch_start() StartChildProcess(PgArchiverFork)
+#define SysLogger_Start() StartChildProcess(SysLoggerFork)
+#define do_start_bgworker() StartChildProcess(BgWorkerFork)
+#define BackendStartup() StartChildProcess(BackendFork)
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -560,6 +522,195 @@ int postmaster_alive_fds[2] = {-1, -1};
HANDLE PostmasterHandle;
#endif
+static Port *ConnProcPort = NULL;
+
+/*
+ * BackendMain
+ *
+ * Child code when forking a Backend.
+ */
+static void BackendMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ /*
+ * Perform additional initialization and collect startup packet.
+ *
+ * We want to do this before InitProcess() for a couple of reasons: 1.
+ * so that we aren't eating up a PGPROC slot while waiting on the
+ * client. 2. so that if InitProcess() fails due to being out of
+ * PGPROC slots, we have already initialized libpq and are able to
+ * report the error to the client.
+ */
+ BackendInitialize(ConnProcPort);
+
+#ifdef EXEC_BACKEND
+ shmemSetup(false);
+#endif
+
+ /* And run the backend */
+ BackendRun(ConnProcPort); /* does not return */
+}
+
+/*
+ * BackendPostmasterMain
+ *
+ * Parent code when forking a Backend.
+ */
+static void BackendPostmasterMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ /* in parent, successful fork */
+ ereport(DEBUG2,
+ (errmsg_internal("forked new backend, pid=%d socket=%d",
+ (int) MyChildProcPid, (int) MyProcPort->sock)));
+
+ /*
+ * Everything's been successful, it's safe to add this backend to our list
+ * of backends.
+ */
+ MyBackend->pid = MyChildProcPid;
+ MyBackend->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
+ dlist_push_head(&BackendList, &MyBackend->elem);
+
+#ifdef EXEC_BACKEND
+ if (!MyBackend->dead_end)
+ ShmemBackendArrayAdd(MyBackend);
+#endif
+}
+
+/*
+ * BackendFailFork
+ *
+ * Backend cleanup in case a failure occurs forking a new Backend.
+ */
+static void BackendFailFork(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ if (!MyBackend->dead_end)
+ (void) ReleasePostmasterChildSlot(MyBackend->child_slot);
+ free(MyBackend);
+
+ report_fork_failure_to_client(MyProcPort, child_errno);
+}
+
+/*
+ * PrepBackendFork
+ *
+ * Prepare a ForkProcType struct for starting a Backend.
+ * This does all prep related to av parameters and error messages.
+*/
+static void
+PrepBackendFork(ForkProcData *backend_fork)
+{
+ int ac = 0;
+
+ /*
+ * Create backend data structure. Better before the fork() so we can
+ * handle failure cleanly.
+ */
+ MyBackend = (Backend *) malloc(sizeof(Backend));
+ if (!MyBackend)
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+ }
+
+#ifdef EXEC_BACKEND
+ backend_fork->av[ac++] = pstrdup("postgres");
+ backend_fork->av[ac++] = pstrdup("--forkbackend");
+ backend_fork->av[ac++] = NULL;
+#endif
+ backend_fork->av[ac] = NULL;
+ backend_fork->ac = ac;
+
+ Assert(ac < lengthof(*backend_fork->av));
+
+ /*
+ * Compute the cancel key that will be assigned to this backend. The
+ * backend will have its own copy in the forked-off process' value of
+ * MyCancelKey, so that it can transmit the key to the frontend.
+ */
+ if (!RandomCancelKey(&MyCancelKey))
+ {
+ free(MyBackend);
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random cancel key")));
+ }
+
+ MyBackend->cancel_key = MyCancelKey;
+
+ /* Pass down canAcceptConnections state */
+ ConnProcPort->canAcceptConnections = canAcceptConnections();
+ MyBackend->dead_end = (ConnProcPort->canAcceptConnections != CAC_OK &&
+ ConnProcPort->canAcceptConnections != CAC_WAITBACKUP);
+
+ /*
+ * Unless it's a dead_end child, assign it a child slot number
+ */
+ if (!MyBackend->dead_end)
+ MyBackend->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
+ else
+ MyBackend->child_slot = 0;
+
+ /* Hasn't asked to be notified about any bgworkers yet */
+ MyBackend->bgworker_notify = false;
+
+ backend_fork->postmaster_main = BackendPostmasterMain;
+ backend_fork->child_main = BackendMain;
+ backend_fork->fail_main = BackendFailFork;
+}
+
+/*
+ * PrepAuxProcessFork
+ *
+ * Prepare a ForkProcType struct for the auxiliary process specified by
+ * AuxProcType. This does all prep related to av parameters and error messages.
+ */
+static void
+PrepAuxProcessFork(ForkProcData *aux_fork)
+{
+ int ac = 0;
+
+ /*
+ * Set up command-line arguments for subprocess
+ */
+ aux_fork->av[ac++] = pstrdup("postgres");
+
+#ifdef EXEC_BACKEND
+ aux_fork->av[ac++] = pstrdup("--forkboot");
+ aux_fork->av[ac++] = NULL; /* filled in by postmaster_forkexec */
+#endif
+
+ aux_fork->av[ac++] = psprintf("-x%d", MyForkProcType);
+
+ aux_fork->av[ac] = NULL;
+ Assert(ac < lengthof(*aux_fork->av));
+
+ aux_fork->ac = ac;
+ switch (MyForkProcType)
+ {
+ case StartupProcess:
+ aux_fork->type_desc = pstrdup("startup");
+ break;
+ case BgWriterProcess:
+ aux_fork->type_desc = pstrdup("background writer");
+ break;
+ case CheckpointerProcess:
+ aux_fork->type_desc = pstrdup("checkpointer");
+ break;
+ case WalWriterProcess:
+ aux_fork->type_desc = pstrdup("WAL writer");
+ break;
+ case WalReceiverProcess:
+ aux_fork->type_desc = pstrdup("WAL receiver");
+ break;
+ default:
+ aux_fork->type_desc = pstrdup("child");
+ break;
+ }
+
+ aux_fork->child_main = AuxiliaryProcessMain;
+}
+
/*
* Postmaster main entry point
*/
@@ -1696,19 +1847,17 @@ ServerLoop(void)
break;
if (FD_ISSET(ListenSocket[i], &rmask))
{
- Port *port;
-
- port = ConnCreate(ListenSocket[i]);
- if (port)
+ ConnProcPort = ConnCreate(ListenSocket[i]);
+ if (ConnProcPort)
{
- BackendStartup(port);
+ BackendStartup();
/*
* We no longer need the open socket or port structure
* in this process
*/
- StreamClose(port->sock);
- ConnFree(port);
+ StreamClose(ConnProcPort->sock);
+ ConnFree(ConnProcPort);
}
}
}
@@ -4043,122 +4192,6 @@ TerminateChildren(int signal)
signal_child(PgStatPID, signal);
}
-/*
- * BackendStartup -- start backend process
- *
- * returns: STATUS_ERROR if the fork failed, STATUS_OK otherwise.
- *
- * Note: if you change this code, also consider StartAutovacuumWorker.
- */
-static int
-BackendStartup(Port *port)
-{
- Backend *bn; /* for backend cleanup */
- pid_t pid;
-
- /*
- * Create backend data structure. Better before the fork() so we can
- * handle failure cleanly.
- */
- bn = (Backend *) malloc(sizeof(Backend));
- if (!bn)
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- return STATUS_ERROR;
- }
-
- /*
- * Compute the cancel key that will be assigned to this backend. The
- * backend will have its own copy in the forked-off process' value of
- * MyCancelKey, so that it can transmit the key to the frontend.
- */
- if (!RandomCancelKey(&MyCancelKey))
- {
- free(bn);
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("could not generate random cancel key")));
- return STATUS_ERROR;
- }
-
- bn->cancel_key = MyCancelKey;
-
- /* Pass down canAcceptConnections state */
- port->canAcceptConnections = canAcceptConnections();
- bn->dead_end = (port->canAcceptConnections != CAC_OK &&
- port->canAcceptConnections != CAC_WAITBACKUP);
-
- /*
- * Unless it's a dead_end child, assign it a child slot number
- */
- if (!bn->dead_end)
- bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
- else
- bn->child_slot = 0;
-
- /* Hasn't asked to be notified about any bgworkers yet */
- bn->bgworker_notify = false;
-
-#ifdef EXEC_BACKEND
- pid = backend_forkexec(port);
-#else /* !EXEC_BACKEND */
- pid = fork_process();
- if (pid == 0) /* child */
- {
- free(bn);
-
- /* Detangle from postmaster */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Perform additional initialization and collect startup packet */
- BackendInitialize(port);
-
- /* And run the backend */
- BackendRun(port);
- }
-#endif /* EXEC_BACKEND */
-
- if (pid < 0)
- {
- /* in parent, fork failed */
- int save_errno = errno;
-
- if (!bn->dead_end)
- (void) ReleasePostmasterChildSlot(bn->child_slot);
- free(bn);
- errno = save_errno;
- ereport(LOG,
- (errmsg("could not fork new process for connection: %m")));
- report_fork_failure_to_client(port, save_errno);
- return STATUS_ERROR;
- }
-
- /* in parent, successful fork */
- ereport(DEBUG2,
- (errmsg_internal("forked new backend, pid=%d socket=%d",
- (int) pid, (int) port->sock)));
-
- /*
- * Everything's been successful, it's safe to add this backend to our list
- * of backends.
- */
- bn->pid = pid;
- bn->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
- dlist_push_head(&BackendList, &bn->elem);
-
-#ifdef EXEC_BACKEND
- if (!bn->dead_end)
- ShmemBackendArrayAdd(bn);
-#endif
-
- return STATUS_OK;
-}
-
/*
* Try to report backend fork() failure to client before we close the
* connection. Since we do not care to risk blocking the postmaster on
@@ -4398,7 +4431,7 @@ BackendRun(Port *port)
maxac * sizeof(char *));
ac = 0;
- av[ac++] = "postgres";
+ av[ac++] = pstrdup("postgres");
/*
* Pass any backend switches specified with -o on the postmaster's own
@@ -4447,38 +4480,13 @@ BackendRun(Port *port)
* child process.
*/
pid_t
-postmaster_forkexec(int argc, char *argv[])
+postmaster_forkexec(ForkProcData *fork_data)
{
- Port port;
-
+ if (!ConnProcPort)
/* This entry point passes dummy values for the Port variables */
- memset(&port, 0, sizeof(port));
- return internal_forkexec(argc, argv, &port);
-}
-
-/*
- * backend_forkexec -- fork/exec off a backend process
- *
- * Some operating systems (WIN32) don't have fork() so we have to simulate
- * it by storing parameters that need to be passed to the child and
- * then create a new child process.
- *
- * returns the pid of the fork/exec'd process, or -1 on failure
- */
-static pid_t
-backend_forkexec(Port *port)
-{
- char *av[4];
- int ac = 0;
+ ConnProcPort = palloc0(sizeof(*ConnProcPort));
- av[ac++] = "postgres";
- av[ac++] = "--forkbackend";
- av[ac++] = NULL; /* filled in by internal_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return internal_forkexec(ac, av, port);
+ return internal_forkexec(fork_data->ac, fork_data->av);
}
#ifndef WIN32
@@ -4490,7 +4498,7 @@ backend_forkexec(Port *port)
* - fork():s, and then exec():s the child process
*/
static pid_t
-internal_forkexec(int argc, char *argv[], Port *port)
+internal_forkexec(int argc, char *argv[])
{
static unsigned long tmpBackendFileNum = 0;
pid_t pid;
@@ -4498,7 +4506,7 @@ internal_forkexec(int argc, char *argv[], Port *port)
BackendParameters param;
FILE *fp;
- if (!save_backend_variables(¶m, port))
+ if (!save_backend_variables(¶m, ConnProcPort))
return -1; /* log made by save_backend_variables */
/* Calculate name for temp file */
@@ -4777,6 +4785,29 @@ retry:
}
#endif /* WIN32 */
+/*
+ * shmemSetup
+ *
+ * Helper function for a child to set up shmem before
+ * executing.
+ *
+ * aux_process - set to true if an auxiliary process.
+ */
+static void
+shmemSetup(bool aux_process)
+{
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
+
+ /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ if (aux_process)
+ InitAuxiliaryProcess();
+ else
+ InitProcess();
+
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores(0);
+}
/*
* SubPostmasterMain -- Get the fork/exec'd process into a state equivalent
@@ -4808,9 +4839,12 @@ SubPostmasterMain(int argc, char *argv[])
if (argc < 3)
elog(FATAL, "invalid subpostmaster invocation");
+ Assert(!ConnProcPort);
+
/* Read in the variables file */
memset(&port, 0, sizeof(Port));
read_backend_variables(argv[2], &port);
+ ConnProcPort = &port;
/* Close the postmaster's sockets (as soon as we know them) */
ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
@@ -4852,16 +4886,16 @@ SubPostmasterMain(int argc, char *argv[])
strcmp(argv[1], "--forkavlauncher") == 0 ||
strcmp(argv[1], "--forkavworker") == 0 ||
strcmp(argv[1], "--forkboot") == 0 ||
- strncmp(argv[1], "--forkbgworker=", 15) == 0)
+ strncmp(argv[1], "--forkbgworker=", BGWORKER_LEN) == 0)
PGSharedMemoryReAttach();
else
PGSharedMemoryNoReAttach();
/* autovacuum needs this set before calling InitProcess */
if (strcmp(argv[1], "--forkavlauncher") == 0)
- AutovacuumLauncherIAm();
+ am_autovacuum_launcher = true;
if (strcmp(argv[1], "--forkavworker") == 0)
- AutovacuumWorkerIAm();
+ am_autovacuum_worker = true;
/*
* Start our win32 signal implementation. This has to be done after we
@@ -4928,90 +4962,38 @@ SubPostmasterMain(int argc, char *argv[])
(errmsg("SSL configuration could not be loaded in child process")));
}
#endif
-
- /*
- * Perform additional initialization and collect startup packet.
- *
- * We want to do this before InitProcess() for a couple of reasons: 1.
- * so that we aren't eating up a PGPROC slot while waiting on the
- * client. 2. so that if InitProcess() fails due to being out of
- * PGPROC slots, we have already initialized libpq and are able to
- * report the error to the client.
- */
- BackendInitialize(&port);
-
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(0);
-
/* And run the backend */
- BackendRun(&port); /* does not return */
+ BackendMain(argc, argv);
}
if (strcmp(argv[1], "--forkboot") == 0)
{
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitAuxiliaryProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(0);
-
+ shmemSetup(true);
AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
}
if (strcmp(argv[1], "--forkavlauncher") == 0)
{
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(0);
-
+ shmemSetup(false);
AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
}
if (strcmp(argv[1], "--forkavworker") == 0)
{
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(0);
-
+ shmemSetup(false);
AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
}
- if (strncmp(argv[1], "--forkbgworker=", 15) == 0)
+ if (strncmp(argv[1], "--forkbgworker=", BGWORKER_LEN) == 0)
{
int shmem_slot;
/* do this as early as possible; in particular, before InitProcess() */
IsBackgroundWorker = true;
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores(0);
+ shmemSetup(false);
/* Fetch MyBgworkerEntry from shared memory */
- shmem_slot = atoi(argv[1] + 15);
+ shmem_slot = atoi(argv[1] + BGWORKER_LEN);
MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
- StartBackgroundWorker();
+ BackgroundWorkerMain(argc, argv);
}
if (strcmp(argv[1], "--forkarch") == 0)
{
@@ -5286,7 +5268,7 @@ StartupPacketTimeoutHandler(void)
/*
* Generate a random cancel key.
*/
-static bool
+bool
RandomCancelKey(int32 *cancel_key)
{
return pg_strong_random(cancel_key, sizeof(int32));
@@ -5332,7 +5314,6 @@ CountChildren(int target)
return cnt;
}
-
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5343,93 +5324,158 @@ CountChildren(int target)
* to start subprocess.
*/
static pid_t
-StartChildProcess(AuxProcType type)
+StartChildProcess(ForkProcType type)
{
pid_t pid;
- char *av[10];
- int ac = 0;
- char typebuf[32];
-
- /*
- * Set up command-line arguments for subprocess
- */
- av[ac++] = "postgres";
+ ForkProcData *fork_data;
+ MemoryContext old_ctx;
+
+ /* Keep fork data after splitting from postmaster */
+ old_ctx = MemoryContextSwitchTo(TopMemoryContext);
+ fork_data = (ForkProcData *) palloc0(sizeof(ForkProcData));
+ fork_data->av = palloc0(sizeof(char *) * 10);
+ MyForkProcType = type;
+
+ switch(type)
+ {
+ /* Auxiliary Processes */
+ case CheckerFork:
+ case BootstrapFork:
+ case StartupFork:
+ case BgWriterFork:
+ case CheckpointerFork:
+ case WalWriterFork:
+ case WalReceiverFork:
+ PrepAuxProcessFork(fork_data);
+ break;
+ /* Non-Auxiliary Processes */
+ case AutoVacLauncherFork:
+ case AutoVacWorkerFork:
+ PrepAutoVacProcessFork(fork_data);
+ break;
+ case PgstatCollectorFork:
+ PrepPgstatCollectorFork(fork_data);
+ break;
+ case PgArchiverFork:
+ PrepPgArchiverFork(fork_data);
+ break;
+ case SysLoggerFork:
+ if (!Logging_collector)
+ return 0;
-#ifdef EXEC_BACKEND
- av[ac++] = "--forkboot";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-#endif
+ PrepSysLoggerFork(fork_data);
+ break;
+ case BgWorkerFork:
+ if (PrepBgWorkerFork(fork_data))
+ {
+ /* We failed to assign a backendlist entry
+ * so tell postmaster to try again later.
+ */
+ return -1;
+ }
+ break;
+ case BackendFork:
+ PrepBackendFork(fork_data);
+ break;
+ default:
+ break;
+ }
- snprintf(typebuf, sizeof(typebuf), "-x%d", type);
- av[ac++] = typebuf;
+ MemoryContextSwitchTo(old_ctx);
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ /* Bail out if pre-conditions are not met */
+ if (fork_data == NULL)
+ return 0;
#ifdef EXEC_BACKEND
- pid = postmaster_forkexec(ac, av);
-#else /* !EXEC_BACKEND */
+ pid = postmaster_forkexec(fork_data);
+#else
pid = fork_process();
+#endif
+
+ /* some processes like backends and bgworkers need the pid */
+ MyChildProcPid = pid;
if (pid == 0) /* child */
{
+#ifndef EXEC_BACKEND
InitPostmasterChild();
+ /* Release postmaster's working memory context */
+ if (type != SysLoggerFork && type != BackendFork)
+ {
/* Close the postmaster's sockets */
ClosePostmasterPorts(false);
- /* Release postmaster's working memory context */
+ if (type == BgWorkerFork)
+ {
+ /*
+ * Before blowing away PostmasterContext, save this bgworker's
+ * data where it can find it.
+ */
+ MyBgworkerEntry = (BackgroundWorker *)
+ MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
+ memcpy(MyBgworkerEntry, &(CurrentBgWorker->rw_worker), sizeof(BackgroundWorker));
+
+ }
+
MemoryContextSwitchTo(TopMemoryContext);
MemoryContextDelete(PostmasterContext);
+
PostmasterContext = NULL;
+ }
+ else
+ {
+ ClosePostmasterPorts(true);
+ }
+#endif
+ /* Call the process's main subroutine */
+ fork_data->child_main (fork_data->ac, fork_data->av);
- AuxiliaryProcessMain(ac, av);
+#ifndef EXEC_BACKEND
ExitPostmaster(0);
+#endif
}
-#endif /* EXEC_BACKEND */
-
- if (pid < 0)
+ else if (pid < 0)
{
/* in parent, fork failed */
- int save_errno = errno;
+ child_errno = errno;
- errno = save_errno;
- switch (type)
- {
- case StartupProcess:
- ereport(LOG,
- (errmsg("could not fork startup process: %m")));
- break;
- case BgWriterProcess:
- ereport(LOG,
- (errmsg("could not fork background writer process: %m")));
- break;
- case CheckpointerProcess:
ereport(LOG,
- (errmsg("could not fork checkpointer process: %m")));
- break;
- case WalWriterProcess:
- ereport(LOG,
- (errmsg("could not fork WAL writer process: %m")));
- break;
- case WalReceiverProcess:
- ereport(LOG,
- (errmsg("could not fork WAL receiver process: %m")));
- break;
- default:
- ereport(LOG,
- (errmsg("could not fork process: %m")));
- break;
- }
+ (errmsg("could not fork %s process: %m", fork_data->type_desc)));
+ switch (MyForkProcType)
+ {
+ case StartupFork:
/*
* fork failure is fatal during startup, but there's no need to choke
* immediately if starting other child types fails.
*/
- if (type == StartupProcess)
ExitPostmaster(1);
+ break;
+ case BgWorkerFork:
+ case BackendFork:
+ fork_data->fail_main(fork_data->ac, fork_data->av);
+ return -1;
+ default:
+ break;
+ }
+
return 0;
}
+ else /* parent */
+ {
+ switch (MyForkProcType)
+ {
+ case SysLoggerFork:
+ case BackendFork:
+ case BgWorkerFork:
+ fork_data->postmaster_main(fork_data->ac, fork_data->av);
+ break;
+ default:
+ break;
+ }
+ }
/*
* in parent, successful fork
@@ -5664,123 +5710,6 @@ BackgroundWorkerUnblockSignals(void)
PG_SETMASK(&UnBlockSig);
}
-#ifdef EXEC_BACKEND
-static pid_t
-bgworker_forkexec(int shmem_slot)
-{
- char *av[10];
- int ac = 0;
- char forkav[MAXPGPATH];
-
- snprintf(forkav, MAXPGPATH, "--forkbgworker=%d", shmem_slot);
-
- av[ac++] = "postgres";
- av[ac++] = forkav;
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif
-
-/*
- * Start a new bgworker.
- * Starting time conditions must have been checked already.
- *
- * Returns true on success, false on failure.
- * In either case, update the RegisteredBgWorker's state appropriately.
- *
- * This code is heavily based on autovacuum.c, q.v.
- */
-static bool
-do_start_bgworker(RegisteredBgWorker *rw)
-{
- pid_t worker_pid;
-
- Assert(rw->rw_pid == 0);
-
- /*
- * Allocate and assign the Backend element. Note we must do this before
- * forking, so that we can handle out of memory properly.
- *
- * Treat failure as though the worker had crashed. That way, the
- * postmaster will wait a bit before attempting to start it again; if it
- * tried again right away, most likely it'd find itself repeating the
- * out-of-memory or fork failure condition.
- */
- if (!assign_backendlist_entry(rw))
- {
- rw->rw_crashed_at = GetCurrentTimestamp();
- return false;
- }
-
- ereport(DEBUG1,
- (errmsg("starting background worker process \"%s\"",
- rw->rw_worker.bgw_name)));
-
-#ifdef EXEC_BACKEND
- switch ((worker_pid = bgworker_forkexec(rw->rw_shmem_slot)))
-#else
- switch ((worker_pid = fork_process()))
-#endif
- {
- case -1:
- /* in postmaster, fork failed ... */
- ereport(LOG,
- (errmsg("could not fork worker process: %m")));
- /* undo what assign_backendlist_entry did */
- ReleasePostmasterChildSlot(rw->rw_child_slot);
- rw->rw_child_slot = 0;
- free(rw->rw_backend);
- rw->rw_backend = NULL;
- /* mark entry as crashed, so we'll try again later */
- rw->rw_crashed_at = GetCurrentTimestamp();
- break;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /*
- * Before blowing away PostmasterContext, save this bgworker's
- * data where it can find it.
- */
- MyBgworkerEntry = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(MyBgworkerEntry, &rw->rw_worker, sizeof(BackgroundWorker));
-
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
-
- StartBackgroundWorker();
-
- exit(1); /* should not get here */
- break;
-#endif
- default:
- /* in postmaster, fork successful ... */
- rw->rw_pid = worker_pid;
- rw->rw_backend->pid = rw->rw_pid;
- ReportBackgroundWorkerPID(rw);
- /* add new worker to lists of backends */
- dlist_push_head(&BackendList, &rw->rw_backend->elem);
-#ifdef EXEC_BACKEND
- ShmemBackendArrayAdd(rw->rw_backend);
-#endif
- return true;
- }
-
- return false;
-}
-
/*
* Does the current postmaster state require starting a worker with the
* specified start_time?
@@ -5821,54 +5750,6 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
return false;
}
-/*
- * Allocate the Backend struct for a connected background worker, but don't
- * add it to the list of backends just yet.
- *
- * On failure, return false without changing any worker state.
- *
- * Some info from the Backend is copied into the passed rw.
- */
-static bool
-assign_backendlist_entry(RegisteredBgWorker *rw)
-{
- Backend *bn;
-
- /*
- * Compute the cancel key that will be assigned to this session. We
- * probably don't need cancel keys for background workers, but we'd better
- * have something random in the field to prevent unfriendly people from
- * sending cancels to them.
- */
- if (!RandomCancelKey(&MyCancelKey))
- {
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("could not generate random cancel key")));
- return false;
- }
-
- bn = malloc(sizeof(Backend));
- if (bn == NULL)
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- return false;
- }
-
- bn->cancel_key = MyCancelKey;
- bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
- bn->bkend_type = BACKEND_TYPE_BGWORKER;
- bn->dead_end = false;
- bn->bgworker_notify = false;
-
- rw->rw_backend = bn;
- rw->rw_child_slot = bn->child_slot;
-
- return true;
-}
-
/*
* If the time is right, start background worker(s).
*
@@ -5973,7 +5854,8 @@ maybe_start_bgworkers(void)
* crashed, but there's no need because the next run of this
* function will do that.
*/
- if (!do_start_bgworker(rw))
+ CurrentBgWorker = rw;
+ if (do_start_bgworker() <= 0)
{
StartWorkerNeeded = true;
return;
@@ -6096,6 +5978,7 @@ save_backend_variables(BackendParameters *param, Port *port,
param->max_safe_fds = max_safe_fds;
param->MaxBackends = MaxBackends;
+ param->proc_type = MyForkProcType;
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
@@ -6331,6 +6214,7 @@ restore_backend_variables(BackendParameters *param, Port *port)
max_safe_fds = param->max_safe_fds;
MaxBackends = param->MaxBackends;
+ MyForkProcType = param->proc_type;
#ifdef WIN32
PostmasterHandle = param->PostmasterHandle;
@@ -6366,7 +6250,7 @@ ShmemBackendArrayAllocation(void)
memset(ShmemBackendArray, 0, size);
}
-static void
+void
ShmemBackendArrayAdd(Backend *bn)
{
/* The array slot corresponding to my PMChildSlot should be free */
diff --git a/src/backend/postmaster/syslogger.c b/src/backend/postmaster/syslogger.c
index bafd31d22b..6ff8d6b44a 100644
--- a/src/backend/postmaster/syslogger.c
+++ b/src/backend/postmaster/syslogger.c
@@ -48,6 +48,7 @@
#include "storage/pg_shmem.h"
#include "tcop/tcopprot.h"
#include "utils/guc.h"
+#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/timestamp.h"
@@ -134,10 +135,12 @@ static volatile sig_atomic_t rotation_requested = false;
/* Local subroutines */
#ifdef EXEC_BACKEND
-static pid_t syslogger_forkexec(void);
static void syslogger_parseArgs(int argc, char *argv[]);
#endif
+
NON_EXEC_STATIC void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
+NON_EXEC_STATIC void SysLoggerPostmasterMain(int argc, char *argv[]);
+
static void process_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static void flush_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static FILE *logfile_open(const char *filename, const char *mode,
@@ -171,12 +174,16 @@ SysLoggerMain(int argc, char *argv[])
pg_time_t now;
WaitEventSet *wes;
- now = MyStartTime;
-
#ifdef EXEC_BACKEND
syslogger_parseArgs(argc, argv);
+#else
+ /* Drop our connection to postmaster's shared memory, as well */
+ dsm_detach_all();
+ PGSharedMemoryDetach();
#endif /* EXEC_BACKEND */
+ now = MyStartTime;
+
am_syslogger = true;
init_ps_display("logger", "", "", "");
@@ -540,16 +547,42 @@ SysLoggerMain(int argc, char *argv[])
}
/*
- * Postmaster subroutine to start a syslogger subprocess.
+ * Helper function for setting up a file number buffer
+ */
+static char *
+setup_file_buff(FILE *file)
+{
+ char *filenobuf;
+
+ /* static variables (those not passed by write_backend_variables) */
+#ifndef WIN32
+ if (file != NULL)
+ filenobuf = psprintf("%d", fileno(file));
+ else
+ filenobuf = pstrdup("-1");
+#else /* WIN32 */
+ if (file != NULL)
+ filenobuf = psprintf("%ld", (long) _get_osfhandle(_fileno(file)));
+ else
+ filenobuf = pstrdup("0");
+#endif /* WIN32 */
+
+ return filenobuf;
+}
+
+/*
+ * PrepSysLoggerFork
+ *
+ * Postmaster subroutine to prepare a syslogger subprocess.
*/
-int
-SysLogger_Start(void)
+void
+PrepSysLoggerFork(ForkProcData *logger_fork)
{
- pid_t sysloggerPid;
+ int ac = 0;
char *filename;
+ MemoryContext cxt;
- if (!Logging_collector)
- return 0;
+ cxt = MemoryContextSwitchTo(PostmasterContext);
/*
* If first time through, create the pipe which will receive stderr
@@ -626,37 +659,29 @@ SysLogger_Start(void)
pfree(filename);
}
-#ifdef EXEC_BACKEND
- switch ((sysloggerPid = syslogger_forkexec()))
-#else
- switch ((sysloggerPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork system logger: %m")));
- return 0;
+ MemoryContextSwitchTo(cxt);
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
+ logger_fork->av[ac++] = pstrdup("postgres");
+ logger_fork->av[ac++] = pstrdup("--forklog");
+ logger_fork->av[ac++] = NULL; /* filled in by postmaster_forkexec */
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(true);
+ logger_fork->av[ac++] = setup_file_buff(syslogFile);
+ logger_fork->av[ac++] = setup_file_buff(csvlogFile);
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
+ logger_fork->av[ac] = NULL;
- /* do the work */
- SysLoggerMain(0, NULL);
- break;
-#endif
+ logger_fork->ac = ac;
+ Assert(logger_fork->ac < lengthof(*logger_fork->av));
+
+ logger_fork->child_main = SysLoggerMain;
+ logger_fork->postmaster_main = SysLoggerPostmasterMain;
+ logger_fork->type_desc = pstrdup("system logger");
+}
- default:
- /* success, in postmaster */
+NON_EXEC_STATIC void
+SysLoggerPostmasterMain(int argc, char *argv[])
+{
/* now we redirect stderr, if not done already */
if (!redirection_done)
{
@@ -723,70 +748,11 @@ SysLogger_Start(void)
fclose(csvlogFile);
csvlogFile = NULL;
}
- return (int) sysloggerPid;
- }
-
- /* we should never reach here */
- return 0;
}
#ifdef EXEC_BACKEND
-/*
- * syslogger_forkexec() -
- *
- * Format up the arglist for, then fork and exec, a syslogger process
- */
-static pid_t
-syslogger_forkexec(void)
-{
- char *av[10];
- int ac = 0;
- char filenobuf[32];
- char csvfilenobuf[32];
-
- av[ac++] = "postgres";
- av[ac++] = "--forklog";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- /* static variables (those not passed by write_backend_variables) */
-#ifndef WIN32
- if (syslogFile != NULL)
- snprintf(filenobuf, sizeof(filenobuf), "%d",
- fileno(syslogFile));
- else
- strcpy(filenobuf, "-1");
-#else /* WIN32 */
- if (syslogFile != NULL)
- snprintf(filenobuf, sizeof(filenobuf), "%ld",
- (long) _get_osfhandle(_fileno(syslogFile)));
- else
- strcpy(filenobuf, "0");
-#endif /* WIN32 */
- av[ac++] = filenobuf;
-
-#ifndef WIN32
- if (csvlogFile != NULL)
- snprintf(csvfilenobuf, sizeof(csvfilenobuf), "%d",
- fileno(csvlogFile));
- else
- strcpy(csvfilenobuf, "-1");
-#else /* WIN32 */
- if (csvlogFile != NULL)
- snprintf(csvfilenobuf, sizeof(csvfilenobuf), "%ld",
- (long) _get_osfhandle(_fileno(csvlogFile)));
- else
- strcpy(csvfilenobuf, "0");
-#endif /* WIN32 */
- av[ac++] = csvfilenobuf;
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
/*
* syslogger_parseArgs() -
*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 3bf96de256..51657924f1 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -38,9 +38,10 @@ volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
int MyProcPid;
+int MyChildProcPid;
pg_time_t MyStartTime;
TimestampTz MyStartTimestamp;
-struct Port *MyProcPort;
+struct Port *MyProcPort = NULL;
int32 MyCancelKey;
int MyPMChildSlot;
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 43b9f17f72..9114c5eebb 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -1115,7 +1115,7 @@ process_startup_options(Port *port, bool am_superuser)
av = (char **) palloc(maxac * sizeof(char *));
ac = 0;
- av[ac++] = "postgres";
+ av[ac++] = pstrdup("postgres");
pg_split_opts(av, &ac, port->cmdline_options);
diff --git a/src/include/bootstrap/bootstrap.h b/src/include/bootstrap/bootstrap.h
index 85705602cb..3c53454069 100644
--- a/src/include/bootstrap/bootstrap.h
+++ b/src/include/bootstrap/bootstrap.h
@@ -15,7 +15,7 @@
#define BOOTSTRAP_H
#include "nodes/execnodes.h"
-
+#include "postmaster/fork_process.h"
/*
* MAXATTR is the maximum number of attributes in a relation supported
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 61a24c2e3c..06dd5d2792 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -163,6 +163,7 @@ extern PGDLLIMPORT int max_worker_processes;
extern PGDLLIMPORT int max_parallel_workers;
extern PGDLLIMPORT int MyProcPid;
+extern PGDLLIMPORT int MyChildProcPid;
extern PGDLLIMPORT pg_time_t MyStartTime;
extern PGDLLIMPORT TimestampTz MyStartTimestamp;
extern PGDLLIMPORT struct Port *MyProcPort;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 0a3ad3a188..1236da534f 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -16,6 +16,7 @@
#include "libpq/pqcomm.h"
#include "port/atomics.h"
#include "portability/instr_time.h"
+#include "postmaster/fork_process.h"
#include "postmaster/pgarch.h"
#include "storage/proc.h"
#include "utils/hsearch.h"
@@ -1236,10 +1237,11 @@ extern Size BackendStatusShmemSize(void);
extern void CreateSharedBackendStatus(void);
extern void pgstat_init(void);
-extern int pgstat_start(void);
extern void pgstat_reset_all(void);
extern void allow_immediate_pgstat_restart(void);
+extern void PrepPgstatCollectorFork(ForkProcData *pgstat_fork);
+
#ifdef EXEC_BACKEND
extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
#endif
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 8451e5d9e2..518775f528 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -15,6 +15,7 @@
#define AUTOVACUUM_H
#include "storage/block.h"
+#include "postmaster/fork_process.h"
/*
* Other processes can request specific work from autovacuum, identified by
@@ -45,6 +46,10 @@ extern int AutovacuumLauncherPid;
extern int Log_autovacuum_min_duration;
+/* autovacuum identification */
+extern bool am_autovacuum_launcher;
+extern bool am_autovacuum_worker;
+
/* Status inquiry functions */
extern bool AutoVacuumingActive(void);
extern bool IsAutoVacuumLauncherProcess(void);
@@ -58,6 +63,8 @@ extern void autovac_init(void);
extern int StartAutoVacLauncher(void);
extern int StartAutoVacWorker(void);
+extern void PrepAutoVacProcessFork(ForkProcData *autovac_fork);
+
/* called from postmaster when a worker could not be forked */
extern void AutoVacWorkerFailed(void);
@@ -67,8 +74,6 @@ extern void AutoVacuumUpdateDelay(void);
#ifdef EXEC_BACKEND
extern void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern void AutovacuumWorkerIAm(void);
-extern void AutovacuumLauncherIAm(void);
#endif
extern bool AutoVacuumRequestWork(AutoVacuumWorkItemType type,
diff --git a/src/include/postmaster/bgworker.h b/src/include/postmaster/bgworker.h
index a8864946cb..363e03aca4 100644
--- a/src/include/postmaster/bgworker.h
+++ b/src/include/postmaster/bgworker.h
@@ -41,6 +41,8 @@
#ifndef BGWORKER_H
#define BGWORKER_H
+#include "postmaster/fork_process.h"
+
/*---------------------------------------------------------------------
* External module API.
*---------------------------------------------------------------------
@@ -158,4 +160,6 @@ extern void BackgroundWorkerInitializeConnectionByOid(Oid dboid, Oid useroid, ui
extern void BackgroundWorkerBlockSignals(void);
extern void BackgroundWorkerUnblockSignals(void);
+extern int PrepBgWorkerFork(ForkProcData *bgworker_fork);
+
#endif /* BGWORKER_H */
diff --git a/src/include/postmaster/bgworker_internals.h b/src/include/postmaster/bgworker_internals.h
index 0809e0af53..b6d86b503e 100644
--- a/src/include/postmaster/bgworker_internals.h
+++ b/src/include/postmaster/bgworker_internals.h
@@ -43,6 +43,7 @@ typedef struct RegisteredBgWorker
} RegisteredBgWorker;
extern slist_head BackgroundWorkerList;
+extern RegisteredBgWorker *CurrentBgWorker;
extern Size BackgroundWorkerShmemSize(void);
extern void BackgroundWorkerShmemInit(void);
@@ -54,7 +55,7 @@ extern void BackgroundWorkerStopNotifications(pid_t pid);
extern void ResetBackgroundWorkerCrashTimes(void);
/* Function to start a background worker, called from postmaster.c */
-extern void StartBackgroundWorker(void) pg_attribute_noreturn();
+extern void BackgroundWorkerMain(int argc, char *argv[]);
#ifdef EXEC_BACKEND
extern BackgroundWorker *BackgroundWorkerEntry(int slotno);
diff --git a/src/include/postmaster/fork_process.h b/src/include/postmaster/fork_process.h
index 30be2faa7c..b942247c4b 100644
--- a/src/include/postmaster/fork_process.h
+++ b/src/include/postmaster/fork_process.h
@@ -12,8 +12,53 @@
#ifndef FORK_PROCESS_H
#define FORK_PROCESS_H
+#include "postmaster.h"
+#include "libpq/libpq.h"
+
+typedef enum
+{
+ NoForkProcess = -1,
+ CheckerFork = 0,
+ BootstrapFork,
+ StartupFork,
+ BgWriterFork,
+ CheckpointerFork,
+ WalWriterFork,
+ WalReceiverFork, /* end of Auxiliary Process Forks */
+ AutoVacLauncherFork,
+ AutoVacWorkerFork,
+ PgstatCollectorFork,
+ PgArchiverFork,
+ SysLoggerFork,
+ BgWorkerFork,
+ BackendFork,
+
+ NUMFORKPROCTYPES /* Must be last! */
+} ForkProcType;
+
+extern ForkProcType MyForkProcType;
+
+struct ForkProcData;
+
+typedef void (*ChildProcessMain) (int argc, char *argv[]);
+
+typedef struct ForkProcData
+{
+ char **av;
+ int ac;
+ int save_errno; /* for failure */
+ char *type_desc;
+ ChildProcessMain postmaster_main;
+ ChildProcessMain child_main;
+ ChildProcessMain fail_main;
+} ForkProcData;
+
extern pid_t fork_process(void);
+#ifdef EXEC_BACKEND
+extern pid_t postmaster_forkexec(ForkProcData *fork_data);
+#endif
+
/* Hook for plugins to get control after a successful fork_process() */
typedef void (*ForkProcess_post_hook_type) ();
extern PGDLLIMPORT ForkProcess_post_hook_type ForkProcess_post_hook;
diff --git a/src/include/postmaster/pgarch.h b/src/include/postmaster/pgarch.h
index 2474eac26a..ce23e39c8b 100644
--- a/src/include/postmaster/pgarch.h
+++ b/src/include/postmaster/pgarch.h
@@ -13,6 +13,8 @@
#ifndef _PGARCH_H
#define _PGARCH_H
+#include "postmaster/fork_process.h"
+
/* ----------
* Archiver control info.
*
@@ -30,10 +32,9 @@
* Functions called from postmaster
* ----------
*/
-extern int pgarch_start(void);
-
#ifdef EXEC_BACKEND
extern void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
#endif
+extern void PrepPgArchiverFork(ForkProcData *pgarch_fork);
#endif /* _PGARCH_H */
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index b692d8be11..a52952b74e 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -13,6 +13,41 @@
#ifndef _POSTMASTER_H
#define _POSTMASTER_H
+#include "lib/ilist.h"
+
+/*
+ * List of active backends (or child processes anyway; we don't actually
+ * know whether a given child has become a backend or is still in the
+ * authorization phase). This is used mainly to keep track of how many
+ * children we have and send them appropriate signals when necessary.
+ *
+ * "Special" children such as the startup, bgwriter and autovacuum launcher
+ * tasks are not in this list. Autovacuum worker and walsender are in it.
+ * Also, "dead_end" children are in it: these are children launched just for
+ * the purpose of sending a friendly rejection message to a would-be client.
+ * We must track them because they are attached to shared memory, but we know
+ * they will never become live backends. dead_end children are not assigned a
+ * PMChildSlot.
+ *
+ * Background workers are in this list, too.
+ */
+typedef struct bkend
+{
+ pid_t pid; /* process id of backend */
+ int32 cancel_key; /* cancel key for cancels for this backend */
+ int child_slot; /* PMChildSlot for this backend, if any */
+
+ /*
+ * Flavor of backend or auxiliary process. Note that BACKEND_TYPE_WALSND
+ * backends initially announce themselves as BACKEND_TYPE_NORMAL, so if
+ * bkend_type is normal, you should check for a recent transition.
+ */
+ int bkend_type;
+ bool dead_end; /* is it going to send an error and quit? */
+ bool bgworker_notify; /* gets bgworker start/stop notifications */
+ dlist_node elem; /* list link in BackendList */
+} Backend;
+
/* GUC options */
extern bool EnableSSL;
extern int ReservedBackends;
@@ -54,14 +89,18 @@ extern int MaxLivePostmasterChildren(void);
extern bool PostmasterMarkPIDForWorkerNotify(int);
+extern bool RandomCancelKey(int32 *cancel_key);
+
#ifdef EXEC_BACKEND
-extern pid_t postmaster_forkexec(int argc, char *argv[]);
extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
extern Size ShmemBackendArraySize(void);
extern void ShmemBackendArrayAllocation(void);
+extern void ShmemBackendArrayAdd(Backend *bn);
#endif
+extern PGDLLIMPORT dlist_head BackendList;
+
/*
* Note: MAX_BACKENDS is limited to 2^18-1 because that's the width reserved
* for buffer references in buf_internals.h. This limitation could be lifted
@@ -74,4 +113,17 @@ extern void ShmemBackendArrayAllocation(void);
*/
#define MAX_BACKENDS 0x3FFFF
+/*
+ * Possible types of a backend. Beyond being the possible bkend_type values in
+ * struct bkend, these are OR-able request flag bits for SignalSomeChildren()
+ * and CountChildren().
+ */
+#define BACKEND_TYPE_NORMAL 0x0001 /* normal backend */
+#define BACKEND_TYPE_AUTOVAC 0x0002 /* autovacuum worker process */
+#define BACKEND_TYPE_WALSND 0x0004 /* walsender process */
+#define BACKEND_TYPE_BGWORKER 0x0008 /* bgworker process */
+#define BACKEND_TYPE_ALL 0x000F /* OR of all the above */
+
+#define BACKEND_TYPE_WORKER (BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER)
+
#endif /* _POSTMASTER_H */
diff --git a/src/include/postmaster/syslogger.h b/src/include/postmaster/syslogger.h
index 3a61104573..1d0b125141 100644
--- a/src/include/postmaster/syslogger.h
+++ b/src/include/postmaster/syslogger.h
@@ -14,6 +14,7 @@
#include <limits.h> /* for PIPE_BUF */
+#include "postmaster/fork_process.h"
/*
* Primitive protocol structure for writing to syslogger pipe(s). The idea
@@ -83,6 +84,8 @@ extern int SysLogger_Start(void);
extern void write_syslogger_file(const char *buffer, int count, int dest);
+extern void PrepSysLoggerFork(ForkProcData *logger_fork);
+
#ifdef EXEC_BACKEND
extern void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
#endif
--
2.19.0
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
@ 2019-09-26 13:49 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
0 siblings, 1 reply; 38+ messages in thread
From: Alvaro Herrera @ 2019-09-26 13:49 UTC (permalink / raw)
To: Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; +Cc: Mike Palmiotto <mike.palmiotto@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
0002 seems way too large (and it doesn't currently apply). Is there
something we can do to make it more manageable?
I think it would be better to put your 0001 in second place rather than
first, since your other patch doesn't use it AFAICS and it adds
functionality that has not yet received approval [or even been
discussed], while the other is necessary refactoring.
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
@ 2019-09-26 14:11 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
0 siblings, 1 reply; 38+ messages in thread
From: Mike Palmiotto @ 2019-09-26 14:11 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@2ndquadrant.com>; +Cc: Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Thu, Sep 26, 2019 at 9:49 AM Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
>
> 0002 seems way too large (and it doesn't currently apply). Is there
> something we can do to make it more manageable?
Initially we were thinking of submitting one patch for the
centralization work and then separate patches per backend type. We
opted not to go that route, mainly because of the number of resulting
patches (there were somewhere around 13 total, as I remember). If it
makes sense, we can go ahead and split the patches up in that fashion
after rebasing.
>
> I think it would be better to put your 0001 in second place rather than
> first, since your other patch doesn't use it AFAICS and it adds
> functionality that has not yet received approval [or even been
> discussed], while the other is necessary refactoring.
Sounds good.
--
Mike Palmiotto
https://crunchydata.com
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
@ 2019-09-26 14:56 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
0 siblings, 1 reply; 38+ messages in thread
From: Alvaro Herrera @ 2019-09-26 14:56 UTC (permalink / raw)
To: Mike Palmiotto <mike.palmiotto@crunchydata.com>; +Cc: Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On 2019-Sep-26, Mike Palmiotto wrote:
> On Thu, Sep 26, 2019 at 9:49 AM Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
> >
> > 0002 seems way too large (and it doesn't currently apply). Is there
> > something we can do to make it more manageable?
>
> Initially we were thinking of submitting one patch for the
> centralization work and then separate patches per backend type. We
> opted not to go that route, mainly because of the number of resulting
> patches (there were somewhere around 13 total, as I remember). If it
> makes sense, we can go ahead and split the patches up in that fashion
> after rebasing.
Well, I think it would be easier to manage as split patches, yeah.
I think it'd be infrastructure that needs to be carefully reviewed,
while the other ones are mostly boilerplate. If I were the committer
for it, I would push that initial patch first immediately followed by
conversion of some process that's heavily exercised in buildfarm, wait
until lack of trouble is evident, followed by a trickle of pushes to
adapt the other processes.
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
@ 2019-09-26 22:03 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
0 siblings, 1 reply; 38+ messages in thread
From: Mike Palmiotto @ 2019-09-26 22:03 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@2ndquadrant.com>; +Cc: Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Thu, Sep 26, 2019 at 10:56 AM Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:
>
> On 2019-Sep-26, Mike Palmiotto wrote:
>
> > On Thu, Sep 26, 2019 at 9:49 AM Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
> > >
> > > 0002 seems way too large (and it doesn't currently apply). Is there
> > > something we can do to make it more manageable?
> >
> > Initially we were thinking of submitting one patch for the
> > centralization work and then separate patches per backend type. We
> > opted not to go that route, mainly because of the number of resulting
> > patches (there were somewhere around 13 total, as I remember). If it
> > makes sense, we can go ahead and split the patches up in that fashion
> > after rebasing.
>
> Well, I think it would be easier to manage as split patches, yeah.
> I think it'd be infrastructure that needs to be carefully reviewed,
> while the other ones are mostly boilerplate. If I were the committer
> for it, I would push that initial patch first immediately followed by
> conversion of some process that's heavily exercised in buildfarm, wait
> until lack of trouble is evident, followed by a trickle of pushes to
> adapt the other processes.
Thanks for the feedback! I've rebased and tested on my F30 box with
and without EXEC_BACKEND. Just working on splitting out the patches
now and will post the new patchset as soon as that's done (hopefully
sometime tomorrow).
--
Mike Palmiotto
https://crunchydata.com
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
@ 2019-09-30 19:13 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 38+ messages in thread
From: Mike Palmiotto @ 2019-09-30 19:13 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@2ndquadrant.com>; +Cc: Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Thu, Sep 26, 2019 at 6:03 PM Mike Palmiotto
<mike.palmiotto@crunchydata.com> wrote:
>
> On Thu, Sep 26, 2019 at 10:56 AM Alvaro Herrera
> <alvherre@2ndquadrant.com> wrote:
> <snip>
> >
> > Well, I think it would be easier to manage as split patches, yeah.
> > I think it'd be infrastructure that needs to be carefully reviewed,
> > while the other ones are mostly boilerplate. If I were the committer
> > for it, I would push that initial patch first immediately followed by
> > conversion of some process that's heavily exercised in buildfarm, wait
> > until lack of trouble is evident, followed by a trickle of pushes to
> > adapt the other processes.
>
> Thanks for the feedback! I've rebased and tested on my F30 box with
> and without EXEC_BACKEND. Just working on splitting out the patches
> now and will post the new patchset as soon as that's done (hopefully
> sometime tomorrow).
Attached is the reworked and rebased patch set. I put the hook on top
and a separate commit for each process type. Note that avworker and
avlauncher were intentionally left together. Let me know if you think
those should be split out as well.
Tested again with and without EXEC_BACKEND. Note that you'll need to
set randomize_va_space to 0 to disable ASLR in order to avoid
occasional failure in the EXEC_BACKEND case.
Please let me know if anything else jumps out to you.
Thanks,
--
Mike Palmiotto
https://crunchydata.com
Attachments: download all (8 files, .tar.gz)
0007-Add-backends-to-process-centralization.patch (text/x-patch, 13.1K) download
From 205ea86dde898f7edac327d46b2b43b04721aadc Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Mon, 30 Sep 2019 14:42:53 -0400
Subject: [PATCH 7/8] Add backends to process centralization
---
src/backend/postmaster/postmaster.c | 309 +++++++++++++-------------
src/include/postmaster/fork_process.h | 5 +
src/include/postmaster/postmaster.h | 1 -
3 files changed, 160 insertions(+), 155 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 8f862fcd64..b55cc4556d 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -144,6 +144,7 @@ static Backend *ShmemBackendArray;
BackgroundWorker *MyBgworkerEntry = NULL;
RegisteredBgWorker *CurrentBgWorker = NULL;
+static Backend *MyBackend;
static int child_errno;
/* The socket number we are listening for connections on */
@@ -356,7 +357,6 @@ static void BackendInitialize(Port *port);
static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
-static int BackendStartup(Port *port);
static int ProcessStartupPacket(Port *port, bool secure_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
@@ -409,7 +409,6 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(Port *port);
static pid_t internal_forkexec(int argc, char *argv[]);
/* Type for a socket that can be inherited to a client process */
@@ -505,6 +504,7 @@ static void ShmemBackendArrayRemove(Backend *bn);
#define pgarch_start() StartChildProcess(PgArchiverFork)
#define SysLogger_Start() StartChildProcess(SysLoggerFork)
#define do_start_bgworker() StartChildProcess(BgWorkerFork)
+#define BackendStartup() StartChildProcess(BackendFork)
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -524,6 +524,141 @@ HANDLE PostmasterHandle;
static Port *ConnProcPort = NULL;
+/*
+ * BackendMain
+ *
+ * Child code when forking a Backend.
+ */
+static void BackendMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ /*
+ * Perform additional initialization and collect startup packet.
+ *
+ * We want to do this before InitProcess() for a couple of reasons: 1.
+ * so that we aren't eating up a PGPROC slot while waiting on the
+ * client. 2. so that if InitProcess() fails due to being out of
+ * PGPROC slots, we have already initialized libpq and are able to
+ * report the error to the client.
+ */
+ BackendInitialize(ConnProcPort);
+
+#ifdef EXEC_BACKEND
+ shmemSetup(false);
+#endif
+
+ /* And run the backend */
+ BackendRun(ConnProcPort); /* does not return */
+}
+
+/*
+ * BackendPostmasterMain
+ *
+ * Parent code when forking a Backend.
+ */
+static void BackendPostmasterMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ /* in parent, successful fork */
+ ereport(DEBUG2,
+ (errmsg_internal("forked new backend, pid=%d socket=%d",
+ (int) MyChildProcPid, (int) MyProcPort->sock)));
+
+ /*
+ * Everything's been successful, it's safe to add this backend to our list
+ * of backends.
+ */
+ MyBackend->pid = MyChildProcPid;
+ MyBackend->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
+ dlist_push_head(&BackendList, &MyBackend->elem);
+
+#ifdef EXEC_BACKEND
+ if (!MyBackend->dead_end)
+ ShmemBackendArrayAdd(MyBackend);
+#endif
+}
+
+/*
+ * BackendFailFork
+ *
+ * Backend cleanup in case a failure occurs forking a new Backend.
+ */
+static void BackendFailFork(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ if (!MyBackend->dead_end)
+ (void) ReleasePostmasterChildSlot(MyBackend->child_slot);
+ free(MyBackend);
+
+ report_fork_failure_to_client(MyProcPort, child_errno);
+}
+
+/*
+ * PrepBackendFork
+ *
+ * Prepare a ForkProcType struct for starting a Backend.
+ * This does all prep related to av parameters and error messages.
+*/
+static void
+PrepBackendFork(ForkProcData *backend_fork)
+{
+ int ac = 0;
+
+ /*
+ * Create backend data structure. Better before the fork() so we can
+ * handle failure cleanly.
+ */
+ MyBackend = (Backend *) malloc(sizeof(Backend));
+ if (!MyBackend)
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+ }
+
+#ifdef EXEC_BACKEND
+ backend_fork->av[ac++] = pstrdup("postgres");
+ backend_fork->av[ac++] = pstrdup("--forkbackend");
+ backend_fork->av[ac++] = NULL;
+#endif
+ backend_fork->av[ac] = NULL;
+ backend_fork->ac = ac;
+
+ Assert(ac < lengthof(*backend_fork->av));
+
+ /*
+ * Compute the cancel key that will be assigned to this backend. The
+ * backend will have its own copy in the forked-off process' value of
+ * MyCancelKey, so that it can transmit the key to the frontend.
+ */
+ if (!RandomCancelKey(&MyCancelKey))
+ {
+ free(MyBackend);
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random cancel key")));
+ }
+
+ MyBackend->cancel_key = MyCancelKey;
+
+ /* Pass down canAcceptConnections state */
+ ConnProcPort->canAcceptConnections = canAcceptConnections();
+ MyBackend->dead_end = (ConnProcPort->canAcceptConnections != CAC_OK &&
+ ConnProcPort->canAcceptConnections != CAC_WAITBACKUP);
+
+ /*
+ * Unless it's a dead_end child, assign it a child slot number
+ */
+ if (!MyBackend->dead_end)
+ MyBackend->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
+ else
+ MyBackend->child_slot = 0;
+
+ /* Hasn't asked to be notified about any bgworkers yet */
+ MyBackend->bgworker_notify = false;
+
+ backend_fork->postmaster_main = BackendPostmasterMain;
+ backend_fork->child_main = BackendMain;
+ backend_fork->fail_main = BackendFailFork;
+}
+
/*
* PrepAuxProcessFork
*
@@ -1729,7 +1864,7 @@ ServerLoop(void)
ConnProcPort = ConnCreate(ListenSocket[i]);
if (ConnProcPort)
{
- BackendStartup(ConnProcPort);
+ BackendStartup();
/*
* We no longer need the open socket or port structure
@@ -4085,122 +4220,6 @@ TerminateChildren(int signal)
signal_child(PgStatPID, signal);
}
-/*
- * BackendStartup -- start backend process
- *
- * returns: STATUS_ERROR if the fork failed, STATUS_OK otherwise.
- *
- * Note: if you change this code, also consider StartAutovacuumWorker.
- */
-static int
-BackendStartup(Port *port)
-{
- Backend *bn; /* for backend cleanup */
- pid_t pid;
-
- /*
- * Create backend data structure. Better before the fork() so we can
- * handle failure cleanly.
- */
- bn = (Backend *) malloc(sizeof(Backend));
- if (!bn)
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- return STATUS_ERROR;
- }
-
- /*
- * Compute the cancel key that will be assigned to this backend. The
- * backend will have its own copy in the forked-off process' value of
- * MyCancelKey, so that it can transmit the key to the frontend.
- */
- if (!RandomCancelKey(&MyCancelKey))
- {
- free(bn);
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("could not generate random cancel key")));
- return STATUS_ERROR;
- }
-
- bn->cancel_key = MyCancelKey;
-
- /* Pass down canAcceptConnections state */
- port->canAcceptConnections = canAcceptConnections();
- bn->dead_end = (port->canAcceptConnections != CAC_OK &&
- port->canAcceptConnections != CAC_WAITBACKUP);
-
- /*
- * Unless it's a dead_end child, assign it a child slot number
- */
- if (!bn->dead_end)
- bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
- else
- bn->child_slot = 0;
-
- /* Hasn't asked to be notified about any bgworkers yet */
- bn->bgworker_notify = false;
-
-#ifdef EXEC_BACKEND
- pid = backend_forkexec(port);
-#else /* !EXEC_BACKEND */
- pid = fork_process();
- if (pid == 0) /* child */
- {
- free(bn);
-
- /* Detangle from postmaster */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Perform additional initialization and collect startup packet */
- BackendInitialize(port);
-
- /* And run the backend */
- BackendRun(port);
- }
-#endif /* EXEC_BACKEND */
-
- if (pid < 0)
- {
- /* in parent, fork failed */
- int save_errno = errno;
-
- if (!bn->dead_end)
- (void) ReleasePostmasterChildSlot(bn->child_slot);
- free(bn);
- errno = save_errno;
- ereport(LOG,
- (errmsg("could not fork new process for connection: %m")));
- report_fork_failure_to_client(port, save_errno);
- return STATUS_ERROR;
- }
-
- /* in parent, successful fork */
- ereport(DEBUG2,
- (errmsg_internal("forked new backend, pid=%d socket=%d",
- (int) pid, (int) port->sock)));
-
- /*
- * Everything's been successful, it's safe to add this backend to our list
- * of backends.
- */
- bn->pid = pid;
- bn->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
- dlist_push_head(&BackendList, &bn->elem);
-
-#ifdef EXEC_BACKEND
- if (!bn->dead_end)
- ShmemBackendArrayAdd(bn);
-#endif
-
- return STATUS_OK;
-}
-
/*
* Try to report backend fork() failure to client before we close the
* connection. Since we do not care to risk blocking the postmaster on
@@ -4440,7 +4459,7 @@ BackendRun(Port *port)
maxac * sizeof(char *));
ac = 0;
- av[ac++] = "postgres";
+ av[ac++] = pstrdup("postgres");
/*
* Pass any backend switches specified with -o on the postmaster's own
@@ -4489,38 +4508,13 @@ BackendRun(Port *port)
* child process.
*/
pid_t
-postmaster_forkexec(int argc, char *argv[])
+postmaster_forkexec(ForkProcData *fork_data)
{
/* This entry point passes dummy values for the Port variables */
if (!ConnProcPort)
ConnProcPort = palloc0(sizeof(*ConnProcPort));
- return internal_forkexec(argc, argv);
-}
-
-/*
- * backend_forkexec -- fork/exec off a backend process
- *
- * Some operating systems (WIN32) don't have fork() so we have to simulate
- * it by storing parameters that need to be passed to the child and
- * then create a new child process.
- *
- * returns the pid of the fork/exec'd process, or -1 on failure
- */
-static pid_t
-backend_forkexec(Port *port)
-{
- char *av[4];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkbackend";
- av[ac++] = NULL; /* filled in by internal_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return internal_forkexec(ac, av);
+ return internal_forkexec(fork_data->ac, fork_data->av);
}
#ifndef WIN32
@@ -5413,6 +5407,13 @@ StartChildProcess(ForkProcType type)
break;
case PgArchiverFork:
PrepPgArchiverFork(fork_data);
+ break;
+ case SysLoggerFork:
+ if (!Logging_collector)
+ return 0;
+
+ PrepSysLoggerFork(fork_data);
+ break;
case BgWorkerFork:
if (PrepBgWorkerFork(fork_data))
{
@@ -5422,11 +5423,8 @@ StartChildProcess(ForkProcType type)
return -1;
}
break;
- case SysLoggerFork:
- if (!Logging_collector)
- return 0;
-
- PrepSysLoggerFork(fork_data);
+ case BackendFork:
+ PrepBackendFork(fork_data);
break;
default:
break;
@@ -5439,7 +5437,7 @@ StartChildProcess(ForkProcType type)
return 0;
#ifdef EXEC_BACKEND
- pid = postmaster_forkexec(fork_data->ac, fork_data->av);
+ pid = postmaster_forkexec(fork_data);
#else
pid = fork_process();
#endif
@@ -5453,10 +5451,11 @@ StartChildProcess(ForkProcType type)
InitPostmasterChild();
/* Release postmaster's working memory context */
- if (type != SysLoggerFork)
+ if (type != SysLoggerFork && type != BackendFork)
{
/* Close the postmaster's sockets */
ClosePostmasterPorts(false);
+
if (type == BgWorkerFork)
{
/*
@@ -5504,6 +5503,7 @@ StartChildProcess(ForkProcType type)
ExitPostmaster(1);
break;
case BgWorkerFork:
+ case BackendFork:
fork_data->fail_main(fork_data->ac, fork_data->av);
return -1;
default:
@@ -5518,6 +5518,7 @@ StartChildProcess(ForkProcType type)
{
case SysLoggerFork:
case BgWorkerFork:
+ case BackendFork:
fork_data->postmaster_main(fork_data->ac, fork_data->av);
break;
default:
diff --git a/src/include/postmaster/fork_process.h b/src/include/postmaster/fork_process.h
index b5d5457ce7..631a2113b5 100644
--- a/src/include/postmaster/fork_process.h
+++ b/src/include/postmaster/fork_process.h
@@ -30,6 +30,7 @@ typedef enum
PgArchiverFork,
SysLoggerFork,
BgWorkerFork,
+ BackendFork,
NUMFORKPROCTYPES /* Must be last! */
} ForkProcType;
@@ -53,4 +54,8 @@ typedef struct ForkProcData
extern pid_t fork_process(void);
+#ifdef EXEC_BACKEND
+extern pid_t postmaster_forkexec(ForkProcData *fork_data);
+#endif
+
#endif /* FORK_PROCESS_H */
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index 9962bdd4be..a52952b74e 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -92,7 +92,6 @@ extern bool PostmasterMarkPIDForWorkerNotify(int);
extern bool RandomCancelKey(int32 *cancel_key);
#ifdef EXEC_BACKEND
-extern pid_t postmaster_forkexec(int argc, char *argv[]);
extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
extern Size ShmemBackendArraySize(void);
--
2.23.0
0008-Add-a-hook-to-allow-extensions-to-set-worker-metadat.patch (text/x-patch, 2.1K) download
From 718c6598e9e02e1ce9ade99afb3c8948c67ef5ae Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 19 Feb 2019 15:29:33 +0000
Subject: [PATCH 8/8] Add a hook to allow extensions to set worker metadata
This patch implements a hook that allows extensions to modify a worker
process' metadata in backends.
Additional work done by Yuli Khodorkovskiy <yuli@crunchydata.com>
Original discussion here: https://www.postgresql.org/message-id/flat/CAMN686FE0OdZKp9YPO%3DhtC6LnA6aW4r-%2Bjq%3D3Q5RAoFQgW8EtA%40mail.gmail.com
---
src/backend/postmaster/fork_process.c | 11 +++++++++++
src/include/postmaster/fork_process.h | 4 ++++
2 files changed, 15 insertions(+)
diff --git a/src/backend/postmaster/fork_process.c b/src/backend/postmaster/fork_process.c
index a9138f589e..964c04d846 100644
--- a/src/backend/postmaster/fork_process.c
+++ b/src/backend/postmaster/fork_process.c
@@ -21,6 +21,14 @@
#include <openssl/rand.h>
#endif
+/*
+ * This hook allows plugins to get control of the child process following
+ * a successful fork_process(). It can be used to perform some action in
+ * extensions to set metadata in backends (including special backends) upon
+ * setting of the session user.
+ */
+ForkProcess_post_hook_type ForkProcess_post_hook = NULL;
+
#ifndef WIN32
/*
* Wrapper for fork(). Return values are the same as those for fork():
@@ -113,6 +121,9 @@ fork_process(void)
#ifdef USE_OPENSSL
RAND_cleanup();
#endif
+
+ if (ForkProcess_post_hook)
+ (*ForkProcess_post_hook) ();
}
return result;
diff --git a/src/include/postmaster/fork_process.h b/src/include/postmaster/fork_process.h
index 631a2113b5..d756fcead7 100644
--- a/src/include/postmaster/fork_process.h
+++ b/src/include/postmaster/fork_process.h
@@ -58,4 +58,8 @@ extern pid_t fork_process(void);
extern pid_t postmaster_forkexec(ForkProcData *fork_data);
#endif
+/* Hook for plugins to get control after a successful fork_process() */
+typedef void (*ForkProcess_post_hook_type) ();
+extern PGDLLIMPORT ForkProcess_post_hook_type ForkProcess_post_hook;
+
#endif /* FORK_PROCESS_H */
--
2.23.0
0006-Add-bgworker-to-process-centralization.patch (text/x-patch, 25.4K) download
From ad1004aea268d329d4ad03a73fc817f51449aa5b Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Sun, 29 Sep 2019 17:17:16 -0400
Subject: [PATCH 6/8] Add bgworker to process centralization
---
src/backend/postmaster/bgworker.c | 138 ++++++++-
src/backend/postmaster/postmaster.c | 316 +++++---------------
src/include/miscadmin.h | 1 +
src/include/postmaster/bgworker.h | 4 +
src/include/postmaster/bgworker_internals.h | 3 +-
src/include/postmaster/fork_process.h | 1 +
src/include/postmaster/postmaster.h | 53 ++++
7 files changed, 269 insertions(+), 247 deletions(-)
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index b66b517aca..8c60a13cd1 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -36,6 +36,7 @@
#include "utils/ascii.h"
#include "utils/ps_status.h"
#include "utils/timeout.h"
+#include "utils/timestamp.h"
/*
* The postmaster's list of registered background workers, in private memory.
@@ -134,7 +135,110 @@ static const struct
/* Private functions. */
static bgworker_main_type LookupBackgroundWorkerFunction(const char *libraryname, const char *funcname);
+static bool assign_backendlist_entry(RegisteredBgWorker *rw);
+NON_EXEC_STATIC void BackgroundWorkerFailFork(int argc, char *argv[]);
+NON_EXEC_STATIC void BackgroundWorkerPostmasterMain(int argc, char *argv[]);
+/*
+ * Allocate the Backend struct for a connected background worker, but don't
+ * add it to the list of backends just yet.
+ *
+ * On failure, return false without changing any worker state.
+ *
+ * Some info from the Backend is copied into the passed rw.
+ */
+static bool
+assign_backendlist_entry(RegisteredBgWorker *rw)
+{
+ Backend *bn;
+
+ /*
+ * Compute the cancel key that will be assigned to this session. We
+ * probably don't need cancel keys for background workers, but we'd better
+ * have something random in the field to prevent unfriendly people from
+ * sending cancels to them.
+ */
+ if (!RandomCancelKey(&MyCancelKey))
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random cancel key")));
+ return false;
+ }
+
+ bn = malloc(sizeof(Backend));
+ if (bn == NULL)
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+ return false;
+ }
+
+ bn->cancel_key = MyCancelKey;
+ bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
+ bn->bkend_type = BACKEND_TYPE_BGWORKER;
+ bn->dead_end = false;
+ bn->bgworker_notify = false;
+
+ rw->rw_backend = bn;
+ rw->rw_child_slot = bn->child_slot;
+
+ return true;
+}
+
+/*
+ * PrepBgWorkerFork
+ *
+ * Postmaster subroutine to prepare a bgworker subprocess.
+ *
+ * Returns 0 on success or -1 on failure.
+ *
+ * Note: if fail, we will be called again from the postmaster main loop.
+ */
+int
+PrepBgWorkerFork(ForkProcData *bgworker_fork)
+{
+ int ac = 0;
+ RegisteredBgWorker *rw = CurrentBgWorker;
+
+ Assert(CurrentBgWorker);
+ Assert(rw->rw_pid == 0);
+
+ /*
+ * Allocate and assign the Backend element. Note we must do this before
+ * forking, so that we can handle out of memory properly.
+ *
+ * Treat failure as though the worker had crashed. That way, the
+ * postmaster will wait a bit before attempting to start it again; if it
+ * tried again right away, most likely it'd find itself repeating the
+ * out-of-memory or fork failure condition.
+ */
+ if (!assign_backendlist_entry(rw))
+ {
+ rw->rw_crashed_at = GetCurrentTimestamp();
+ return -1;
+ }
+
+ ereport(DEBUG1,
+ (errmsg("starting background worker process \"%s\"",
+ rw->rw_worker.bgw_name)));
+
+#ifdef EXEC_BACKEND
+ bgworker_fork->av[ac++] = pstrdup("postgres");
+ bgworker_fork->av[ac++] = psprintf("--forkbgworker=%d", rw->rw_shmem_slot);
+ bgworker_fork->av[ac++] = NULL; /* filled in by postmaster_forkexec */
+ bgworker_fork->av[ac] = NULL;
+#endif
+ Assert(bgworker_fork->ac < lengthof(*bgworker_fork->av));
+ bgworker_fork->ac = ac;
+ bgworker_fork->type_desc = pstrdup("background worker");
+ bgworker_fork->child_main = BackgroundWorkerMain;
+ bgworker_fork->fail_main = BackgroundWorkerFailFork;
+ bgworker_fork->postmaster_main = BackgroundWorkerPostmasterMain;
+
+ return 0;
+}
/*
* Calculate shared memory needed.
@@ -698,7 +802,7 @@ bgworker_sigusr1_handler(SIGNAL_ARGS)
* postmaster.
*/
void
-StartBackgroundWorker(void)
+BackgroundWorkerMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
BackgroundWorker *worker = MyBgworkerEntry;
@@ -837,6 +941,38 @@ StartBackgroundWorker(void)
proc_exit(0);
}
+NON_EXEC_STATIC void
+BackgroundWorkerFailFork(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ RegisteredBgWorker *rw = CurrentBgWorker;
+
+ /* in postmaster, fork failed ... */
+ ereport(LOG,
+ (errmsg("could not fork worker process: %m")));
+
+ /* undo what assign_backendlist_entry did */
+ ReleasePostmasterChildSlot(rw->rw_child_slot);
+ rw->rw_child_slot = 0;
+ free(rw->rw_backend);
+ rw->rw_backend = NULL;
+ /* mark entry as crashed, so we'll try again later */
+ rw->rw_crashed_at = GetCurrentTimestamp();
+}
+
+NON_EXEC_STATIC void
+BackgroundWorkerPostmasterMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ RegisteredBgWorker *rw = CurrentBgWorker;
+
+ rw->rw_pid = MyChildProcPid;
+ rw->rw_backend->pid = rw->rw_pid;
+ ReportBackgroundWorkerPID(rw);
+ /* add new worker to lists of backends */
+ dlist_push_head(&BackendList, &(rw->rw_backend->elem));
+#ifdef EXEC_BACKEND
+ ShmemBackendArrayAdd(rw->rw_backend);
+#endif
+}
/*
* Register a new static background worker.
*
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 5d78db067f..8f862fcd64 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -136,62 +136,15 @@
#include "storage/spin.h"
#endif
-
-/*
- * Possible types of a backend. Beyond being the possible bkend_type values in
- * struct bkend, these are OR-able request flag bits for SignalSomeChildren()
- * and CountChildren().
- */
-#define BACKEND_TYPE_NORMAL 0x0001 /* normal backend */
-#define BACKEND_TYPE_AUTOVAC 0x0002 /* autovacuum worker process */
-#define BACKEND_TYPE_WALSND 0x0004 /* walsender process */
-#define BACKEND_TYPE_BGWORKER 0x0008 /* bgworker process */
-#define BACKEND_TYPE_ALL 0x000F /* OR of all the above */
-
-#define BACKEND_TYPE_WORKER (BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER)
-
-/*
- * List of active backends (or child processes anyway; we don't actually
- * know whether a given child has become a backend or is still in the
- * authorization phase). This is used mainly to keep track of how many
- * children we have and send them appropriate signals when necessary.
- *
- * "Special" children such as the startup, bgwriter and autovacuum launcher
- * tasks are not in this list. Autovacuum worker and walsender are in it.
- * Also, "dead_end" children are in it: these are children launched just for
- * the purpose of sending a friendly rejection message to a would-be client.
- * We must track them because they are attached to shared memory, but we know
- * they will never become live backends. dead_end children are not assigned a
- * PMChildSlot.
- *
- * Background workers are in this list, too.
- */
-typedef struct bkend
-{
- pid_t pid; /* process id of backend */
- int32 cancel_key; /* cancel key for cancels for this backend */
- int child_slot; /* PMChildSlot for this backend, if any */
-
- /*
- * Flavor of backend or auxiliary process. Note that BACKEND_TYPE_WALSND
- * backends initially announce themselves as BACKEND_TYPE_NORMAL, so if
- * bkend_type is normal, you should check for a recent transition.
- */
- int bkend_type;
- bool dead_end; /* is it going to send an error and quit? */
- bool bgworker_notify; /* gets bgworker start/stop notifications */
- dlist_node elem; /* list link in BackendList */
-} Backend;
-
-static dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
+dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
#ifdef EXEC_BACKEND
static Backend *ShmemBackendArray;
#endif
BackgroundWorker *MyBgworkerEntry = NULL;
-
-
+RegisteredBgWorker *CurrentBgWorker = NULL;
+static int child_errno;
/* The socket number we are listening for connections on */
int PostPortNumber;
@@ -410,7 +363,6 @@ static void processCancelRequest(Port *port, void *pkt);
static int initMasks(fd_set *rmask);
static void report_fork_failure_to_client(Port *port, int errnum);
static CAC_state canAcceptConnections(void);
-static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
@@ -421,7 +373,6 @@ static void shmemSetup(bool aux_process);
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
-static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
static pid_t StartChildProcess(ForkProcType type);
@@ -538,10 +489,11 @@ static bool save_backend_variables(BackendParameters *param, Port *port,
HANDLE childProcess, pid_t childPid);
#endif
-static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
+#define BGWORKER_LEN 15
+
#define StartupDataBase() StartChildProcess(StartupFork)
#define StartBackgroundWriter() StartChildProcess(BgWriterFork)
#define StartCheckpointer() StartChildProcess(CheckpointerFork)
@@ -552,6 +504,7 @@ static void ShmemBackendArrayRemove(Backend *bn);
#define pgstat_start() StartChildProcess(PgstatCollectorFork)
#define pgarch_start() StartChildProcess(PgArchiverFork)
#define SysLogger_Start() StartChildProcess(SysLoggerFork)
+#define do_start_bgworker() StartChildProcess(BgWorkerFork)
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -4967,7 +4920,7 @@ SubPostmasterMain(int argc, char *argv[])
strcmp(argv[1], "--forkavlauncher") == 0 ||
strcmp(argv[1], "--forkavworker") == 0 ||
strcmp(argv[1], "--forkboot") == 0 ||
- strncmp(argv[1], "--forkbgworker=", 15) == 0)
+ strncmp(argv[1], "--forkbgworker=", BGWORKER_LEN) == 0)
PGSharedMemoryReAttach();
else
PGSharedMemoryNoReAttach();
@@ -5082,27 +5035,20 @@ SubPostmasterMain(int argc, char *argv[])
shmemSetup(true);
AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
}
- if (strncmp(argv[1], "--forkbgworker=", 15) == 0)
+ if (strncmp(argv[1], "--forkbgworker=", BGWORKER_LEN) == 0)
{
int shmem_slot;
/* do this as early as possible; in particular, before InitProcess() */
IsBackgroundWorker = true;
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
+ shmemSetup(false);
/* Fetch MyBgworkerEntry from shared memory */
- shmem_slot = atoi(argv[1] + 15);
+ shmem_slot = atoi(argv[1] + BGWORKER_LEN);
MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
- StartBackgroundWorker();
+ BackgroundWorkerMain(argc, argv);
}
if (strcmp(argv[1], "--forkarch") == 0)
{
@@ -5377,7 +5323,7 @@ StartupPacketTimeoutHandler(void)
/*
* Generate a random cancel key.
*/
-static bool
+bool
RandomCancelKey(int32 *cancel_key)
{
return pg_strong_random(cancel_key, sizeof(int32));
@@ -5467,6 +5413,14 @@ StartChildProcess(ForkProcType type)
break;
case PgArchiverFork:
PrepPgArchiverFork(fork_data);
+ case BgWorkerFork:
+ if (PrepBgWorkerFork(fork_data))
+ {
+ /* We failed to assign a backendlist entry
+ * so tell postmaster to try again later.
+ */
+ return -1;
+ }
break;
case SysLoggerFork:
if (!Logging_collector)
@@ -5490,6 +5444,9 @@ StartChildProcess(ForkProcType type)
pid = fork_process();
#endif
+ /* some processes like backends and bgworkers need the pid */
+ MyChildProcPid = pid;
+
if (pid == 0) /* child */
{
#ifndef EXEC_BACKEND
@@ -5498,8 +5455,19 @@ StartChildProcess(ForkProcType type)
/* Release postmaster's working memory context */
if (type != SysLoggerFork)
{
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
+ /* Close the postmaster's sockets */
+ ClosePostmasterPorts(false);
+ if (type == BgWorkerFork)
+ {
+ /*
+ * Before blowing away PostmasterContext, save this bgworker's
+ * data where it can find it.
+ */
+ MyBgworkerEntry = (BackgroundWorker *)
+ MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
+ memcpy(MyBgworkerEntry, &(CurrentBgWorker->rw_worker), sizeof(BackgroundWorker));
+
+ }
MemoryContextSwitchTo(TopMemoryContext);
MemoryContextDelete(PostmasterContext);
@@ -5521,21 +5489,41 @@ StartChildProcess(ForkProcType type)
else if (pid < 0)
{
/* in parent, fork failed */
- int save_errno = errno;
+ child_errno = errno;
- errno = save_errno;
- ereport(LOG,
- (errmsg("could not fork %s process: %m", fork_data->type_desc)));
+ ereport(LOG,
+ (errmsg("could not fork %s process: %m", fork_data->type_desc)));
- /*
- * fork failure is fatal during startup, but there's no need to choke
- * immediately if starting other child types fails.
- */
- if (MyForkProcType == StartupFork)
- ExitPostmaster(1);
+ switch (MyForkProcType)
+ {
+ /*
+ * fork failure is fatal during startup, but there's no need to choke
+ * immediately if starting other child types fails.
+ */
+ case StartupFork:
+ ExitPostmaster(1);
+ break;
+ case BgWorkerFork:
+ fork_data->fail_main(fork_data->ac, fork_data->av);
+ return -1;
+ default:
+ break;
+ }
return 0;
}
+ else /* parent */
+ {
+ switch (MyForkProcType)
+ {
+ case SysLoggerFork:
+ case BgWorkerFork:
+ fork_data->postmaster_main(fork_data->ac, fork_data->av);
+ break;
+ default:
+ break;
+ }
+ }
/*
* in parent, successful fork
@@ -5770,123 +5758,6 @@ BackgroundWorkerUnblockSignals(void)
PG_SETMASK(&UnBlockSig);
}
-#ifdef EXEC_BACKEND
-static pid_t
-bgworker_forkexec(int shmem_slot)
-{
- char *av[10];
- int ac = 0;
- char forkav[MAXPGPATH];
-
- snprintf(forkav, MAXPGPATH, "--forkbgworker=%d", shmem_slot);
-
- av[ac++] = "postgres";
- av[ac++] = forkav;
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif
-
-/*
- * Start a new bgworker.
- * Starting time conditions must have been checked already.
- *
- * Returns true on success, false on failure.
- * In either case, update the RegisteredBgWorker's state appropriately.
- *
- * This code is heavily based on autovacuum.c, q.v.
- */
-static bool
-do_start_bgworker(RegisteredBgWorker *rw)
-{
- pid_t worker_pid;
-
- Assert(rw->rw_pid == 0);
-
- /*
- * Allocate and assign the Backend element. Note we must do this before
- * forking, so that we can handle out of memory properly.
- *
- * Treat failure as though the worker had crashed. That way, the
- * postmaster will wait a bit before attempting to start it again; if it
- * tried again right away, most likely it'd find itself repeating the
- * out-of-memory or fork failure condition.
- */
- if (!assign_backendlist_entry(rw))
- {
- rw->rw_crashed_at = GetCurrentTimestamp();
- return false;
- }
-
- ereport(DEBUG1,
- (errmsg("starting background worker process \"%s\"",
- rw->rw_worker.bgw_name)));
-
-#ifdef EXEC_BACKEND
- switch ((worker_pid = bgworker_forkexec(rw->rw_shmem_slot)))
-#else
- switch ((worker_pid = fork_process()))
-#endif
- {
- case -1:
- /* in postmaster, fork failed ... */
- ereport(LOG,
- (errmsg("could not fork worker process: %m")));
- /* undo what assign_backendlist_entry did */
- ReleasePostmasterChildSlot(rw->rw_child_slot);
- rw->rw_child_slot = 0;
- free(rw->rw_backend);
- rw->rw_backend = NULL;
- /* mark entry as crashed, so we'll try again later */
- rw->rw_crashed_at = GetCurrentTimestamp();
- break;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /*
- * Before blowing away PostmasterContext, save this bgworker's
- * data where it can find it.
- */
- MyBgworkerEntry = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(MyBgworkerEntry, &rw->rw_worker, sizeof(BackgroundWorker));
-
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
-
- StartBackgroundWorker();
-
- exit(1); /* should not get here */
- break;
-#endif
- default:
- /* in postmaster, fork successful ... */
- rw->rw_pid = worker_pid;
- rw->rw_backend->pid = rw->rw_pid;
- ReportBackgroundWorkerPID(rw);
- /* add new worker to lists of backends */
- dlist_push_head(&BackendList, &rw->rw_backend->elem);
-#ifdef EXEC_BACKEND
- ShmemBackendArrayAdd(rw->rw_backend);
-#endif
- return true;
- }
-
- return false;
-}
-
/*
* Does the current postmaster state require starting a worker with the
* specified start_time?
@@ -5927,54 +5798,6 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
return false;
}
-/*
- * Allocate the Backend struct for a connected background worker, but don't
- * add it to the list of backends just yet.
- *
- * On failure, return false without changing any worker state.
- *
- * Some info from the Backend is copied into the passed rw.
- */
-static bool
-assign_backendlist_entry(RegisteredBgWorker *rw)
-{
- Backend *bn;
-
- /*
- * Compute the cancel key that will be assigned to this session. We
- * probably don't need cancel keys for background workers, but we'd better
- * have something random in the field to prevent unfriendly people from
- * sending cancels to them.
- */
- if (!RandomCancelKey(&MyCancelKey))
- {
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("could not generate random cancel key")));
- return false;
- }
-
- bn = malloc(sizeof(Backend));
- if (bn == NULL)
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- return false;
- }
-
- bn->cancel_key = MyCancelKey;
- bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
- bn->bkend_type = BACKEND_TYPE_BGWORKER;
- bn->dead_end = false;
- bn->bgworker_notify = false;
-
- rw->rw_backend = bn;
- rw->rw_child_slot = bn->child_slot;
-
- return true;
-}
-
/*
* If the time is right, start background worker(s).
*
@@ -6079,7 +5902,8 @@ maybe_start_bgworkers(void)
* crashed, but there's no need because the next run of this
* function will do that.
*/
- if (!do_start_bgworker(rw))
+ CurrentBgWorker = rw;
+ if (do_start_bgworker() <= 0)
{
StartWorkerNeeded = true;
return;
@@ -6202,6 +6026,7 @@ save_backend_variables(BackendParameters *param, Port *port,
param->max_safe_fds = max_safe_fds;
param->MaxBackends = MaxBackends;
+ param->proc_type = MyForkProcType;
#ifdef WIN32
param->PostmasterHandle = PostmasterHandle;
@@ -6437,6 +6262,7 @@ restore_backend_variables(BackendParameters *param, Port *port)
max_safe_fds = param->max_safe_fds;
MaxBackends = param->MaxBackends;
+ MyForkProcType = param->proc_type;
#ifdef WIN32
PostmasterHandle = param->PostmasterHandle;
@@ -6472,7 +6298,7 @@ ShmemBackendArrayAllocation(void)
memset(ShmemBackendArray, 0, size);
}
-static void
+void
ShmemBackendArrayAdd(Backend *bn)
{
/* The array slot corresponding to my PMChildSlot should be free */
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index bc6e03fbc7..da9c4319b9 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -163,6 +163,7 @@ extern PGDLLIMPORT int max_worker_processes;
extern PGDLLIMPORT int max_parallel_workers;
extern PGDLLIMPORT int MyProcPid;
+extern PGDLLIMPORT int MyChildProcPid;
extern PGDLLIMPORT pg_time_t MyStartTime;
extern PGDLLIMPORT TimestampTz MyStartTimestamp;
extern PGDLLIMPORT struct Port *MyProcPort;
diff --git a/src/include/postmaster/bgworker.h b/src/include/postmaster/bgworker.h
index a8864946cb..363e03aca4 100644
--- a/src/include/postmaster/bgworker.h
+++ b/src/include/postmaster/bgworker.h
@@ -41,6 +41,8 @@
#ifndef BGWORKER_H
#define BGWORKER_H
+#include "postmaster/fork_process.h"
+
/*---------------------------------------------------------------------
* External module API.
*---------------------------------------------------------------------
@@ -158,4 +160,6 @@ extern void BackgroundWorkerInitializeConnectionByOid(Oid dboid, Oid useroid, ui
extern void BackgroundWorkerBlockSignals(void);
extern void BackgroundWorkerUnblockSignals(void);
+extern int PrepBgWorkerFork(ForkProcData *bgworker_fork);
+
#endif /* BGWORKER_H */
diff --git a/src/include/postmaster/bgworker_internals.h b/src/include/postmaster/bgworker_internals.h
index 0809e0af53..b6d86b503e 100644
--- a/src/include/postmaster/bgworker_internals.h
+++ b/src/include/postmaster/bgworker_internals.h
@@ -43,6 +43,7 @@ typedef struct RegisteredBgWorker
} RegisteredBgWorker;
extern slist_head BackgroundWorkerList;
+extern RegisteredBgWorker *CurrentBgWorker;
extern Size BackgroundWorkerShmemSize(void);
extern void BackgroundWorkerShmemInit(void);
@@ -54,7 +55,7 @@ extern void BackgroundWorkerStopNotifications(pid_t pid);
extern void ResetBackgroundWorkerCrashTimes(void);
/* Function to start a background worker, called from postmaster.c */
-extern void StartBackgroundWorker(void) pg_attribute_noreturn();
+extern void BackgroundWorkerMain(int argc, char *argv[]);
#ifdef EXEC_BACKEND
extern BackgroundWorker *BackgroundWorkerEntry(int slotno);
diff --git a/src/include/postmaster/fork_process.h b/src/include/postmaster/fork_process.h
index 32e10e3629..b5d5457ce7 100644
--- a/src/include/postmaster/fork_process.h
+++ b/src/include/postmaster/fork_process.h
@@ -29,6 +29,7 @@ typedef enum
PgstatCollectorFork,
PgArchiverFork,
SysLoggerFork,
+ BgWorkerFork,
NUMFORKPROCTYPES /* Must be last! */
} ForkProcType;
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index b692d8be11..9962bdd4be 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -13,6 +13,41 @@
#ifndef _POSTMASTER_H
#define _POSTMASTER_H
+#include "lib/ilist.h"
+
+/*
+ * List of active backends (or child processes anyway; we don't actually
+ * know whether a given child has become a backend or is still in the
+ * authorization phase). This is used mainly to keep track of how many
+ * children we have and send them appropriate signals when necessary.
+ *
+ * "Special" children such as the startup, bgwriter and autovacuum launcher
+ * tasks are not in this list. Autovacuum worker and walsender are in it.
+ * Also, "dead_end" children are in it: these are children launched just for
+ * the purpose of sending a friendly rejection message to a would-be client.
+ * We must track them because they are attached to shared memory, but we know
+ * they will never become live backends. dead_end children are not assigned a
+ * PMChildSlot.
+ *
+ * Background workers are in this list, too.
+ */
+typedef struct bkend
+{
+ pid_t pid; /* process id of backend */
+ int32 cancel_key; /* cancel key for cancels for this backend */
+ int child_slot; /* PMChildSlot for this backend, if any */
+
+ /*
+ * Flavor of backend or auxiliary process. Note that BACKEND_TYPE_WALSND
+ * backends initially announce themselves as BACKEND_TYPE_NORMAL, so if
+ * bkend_type is normal, you should check for a recent transition.
+ */
+ int bkend_type;
+ bool dead_end; /* is it going to send an error and quit? */
+ bool bgworker_notify; /* gets bgworker start/stop notifications */
+ dlist_node elem; /* list link in BackendList */
+} Backend;
+
/* GUC options */
extern bool EnableSSL;
extern int ReservedBackends;
@@ -54,14 +89,19 @@ extern int MaxLivePostmasterChildren(void);
extern bool PostmasterMarkPIDForWorkerNotify(int);
+extern bool RandomCancelKey(int32 *cancel_key);
+
#ifdef EXEC_BACKEND
extern pid_t postmaster_forkexec(int argc, char *argv[]);
extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
extern Size ShmemBackendArraySize(void);
extern void ShmemBackendArrayAllocation(void);
+extern void ShmemBackendArrayAdd(Backend *bn);
#endif
+extern PGDLLIMPORT dlist_head BackendList;
+
/*
* Note: MAX_BACKENDS is limited to 2^18-1 because that's the width reserved
* for buffer references in buf_internals.h. This limitation could be lifted
@@ -74,4 +114,17 @@ extern void ShmemBackendArrayAllocation(void);
*/
#define MAX_BACKENDS 0x3FFFF
+/*
+ * Possible types of a backend. Beyond being the possible bkend_type values in
+ * struct bkend, these are OR-able request flag bits for SignalSomeChildren()
+ * and CountChildren().
+ */
+#define BACKEND_TYPE_NORMAL 0x0001 /* normal backend */
+#define BACKEND_TYPE_AUTOVAC 0x0002 /* autovacuum worker process */
+#define BACKEND_TYPE_WALSND 0x0004 /* walsender process */
+#define BACKEND_TYPE_BGWORKER 0x0008 /* bgworker process */
+#define BACKEND_TYPE_ALL 0x000F /* OR of all the above */
+
+#define BACKEND_TYPE_WORKER (BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER)
+
#endif /* _POSTMASTER_H */
--
2.23.0
0005-Add-syslogger-to-process-centralization.patch (text/x-patch, 8.4K) download
From 85cac3f12ff70df613859d01dca8d971701442f6 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Fri, 27 Sep 2019 20:34:17 -0400
Subject: [PATCH 5/8] Add syslogger to process centralization
---
src/backend/postmaster/postmaster.c | 17 ++-
src/backend/postmaster/syslogger.c | 152 ++++++++++----------------
src/include/postmaster/fork_process.h | 1 +
src/include/postmaster/syslogger.h | 3 +
4 files changed, 79 insertions(+), 94 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 6bdd57d1e2..5d78db067f 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -551,6 +551,7 @@ static void ShmemBackendArrayRemove(Backend *bn);
#define StartAutoVacWorker() StartChildProcess(AutoVacWorkerFork)
#define pgstat_start() StartChildProcess(PgstatCollectorFork)
#define pgarch_start() StartChildProcess(PgArchiverFork)
+#define SysLogger_Start() StartChildProcess(SysLoggerFork)
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -5467,6 +5468,12 @@ StartChildProcess(ForkProcType type)
case PgArchiverFork:
PrepPgArchiverFork(fork_data);
break;
+ case SysLoggerFork:
+ if (!Logging_collector)
+ return 0;
+
+ PrepSysLoggerFork(fork_data);
+ break;
default:
break;
}
@@ -5488,13 +5495,21 @@ StartChildProcess(ForkProcType type)
#ifndef EXEC_BACKEND
InitPostmasterChild();
+ /* Release postmaster's working memory context */
+ if (type != SysLoggerFork)
+ {
/* Close the postmaster's sockets */
ClosePostmasterPorts(false);
- /* Release postmaster's working memory context */
MemoryContextSwitchTo(TopMemoryContext);
MemoryContextDelete(PostmasterContext);
+
PostmasterContext = NULL;
+ }
+ else
+ {
+ ClosePostmasterPorts(true);
+ }
#endif
/* Call the process's main subroutine */
fork_data->child_main (fork_data->ac, fork_data->av);
diff --git a/src/backend/postmaster/syslogger.c b/src/backend/postmaster/syslogger.c
index bb2baff763..37baad6450 100644
--- a/src/backend/postmaster/syslogger.c
+++ b/src/backend/postmaster/syslogger.c
@@ -48,6 +48,7 @@
#include "storage/pg_shmem.h"
#include "tcop/tcopprot.h"
#include "utils/guc.h"
+#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/timestamp.h"
@@ -134,10 +135,12 @@ static volatile sig_atomic_t rotation_requested = false;
/* Local subroutines */
#ifdef EXEC_BACKEND
-static pid_t syslogger_forkexec(void);
static void syslogger_parseArgs(int argc, char *argv[]);
#endif
+
NON_EXEC_STATIC void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
+NON_EXEC_STATIC void SysLoggerPostmasterMain(int argc, char *argv[]);
+
static void process_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static void flush_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static FILE *logfile_open(const char *filename, const char *mode,
@@ -171,12 +174,16 @@ SysLoggerMain(int argc, char *argv[])
pg_time_t now;
WaitEventSet *wes;
- now = MyStartTime;
-
#ifdef EXEC_BACKEND
syslogger_parseArgs(argc, argv);
+#else
+ /* Drop our connection to postmaster's shared memory, as well */
+ dsm_detach_all();
+ PGSharedMemoryDetach();
#endif /* EXEC_BACKEND */
+ now = MyStartTime;
+
am_syslogger = true;
init_ps_display("logger", "", "", "");
@@ -540,16 +547,42 @@ SysLoggerMain(int argc, char *argv[])
}
/*
- * Postmaster subroutine to start a syslogger subprocess.
+ * Helper function for setting up a file number buffer
+ */
+static char *
+setup_file_buff(FILE *file)
+{
+ char *filenobuf;
+
+ /* static variables (those not passed by write_backend_variables) */
+#ifndef WIN32
+ if (file != NULL)
+ filenobuf = psprintf("%d", fileno(file));
+ else
+ filenobuf = pstrdup("-1");
+#else /* WIN32 */
+ if (file != NULL)
+ filenobuf = psprintf("%ld", (long) _get_osfhandle(_fileno(file)));
+ else
+ filenobuf = pstrdup("0");
+#endif /* WIN32 */
+
+ return filenobuf;
+}
+
+/*
+ * PrepSysLoggerFork
+ *
+ * Postmaster subroutine to prepare a syslogger subprocess.
*/
-int
-SysLogger_Start(void)
+void
+PrepSysLoggerFork(ForkProcData *logger_fork)
{
- pid_t sysloggerPid;
+ int ac = 0;
char *filename;
+ MemoryContext cxt;
- if (!Logging_collector)
- return 0;
+ cxt = MemoryContextSwitchTo(PostmasterContext);
/*
* If first time through, create the pipe which will receive stderr
@@ -626,37 +659,29 @@ SysLogger_Start(void)
pfree(filename);
}
-#ifdef EXEC_BACKEND
- switch ((sysloggerPid = syslogger_forkexec()))
-#else
- switch ((sysloggerPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork system logger: %m")));
- return 0;
+ MemoryContextSwitchTo(cxt);
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
+ logger_fork->av[ac++] = pstrdup("postgres");
+ logger_fork->av[ac++] = pstrdup("--forklog");
+ logger_fork->av[ac++] = NULL; /* filled in by postmaster_forkexec */
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(true);
+ logger_fork->av[ac++] = setup_file_buff(syslogFile);
+ logger_fork->av[ac++] = setup_file_buff(csvlogFile);
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
+ logger_fork->av[ac] = NULL;
- /* do the work */
- SysLoggerMain(0, NULL);
- break;
-#endif
+ logger_fork->ac = ac;
+ Assert(logger_fork->ac < lengthof(*logger_fork->av));
+
+ logger_fork->child_main = SysLoggerMain;
+ logger_fork->postmaster_main = SysLoggerPostmasterMain;
+ logger_fork->type_desc = pstrdup("system logger");
+}
- default:
- /* success, in postmaster */
+NON_EXEC_STATIC void
+SysLoggerPostmasterMain(int argc, char *argv[])
+{
/* now we redirect stderr, if not done already */
if (!redirection_done)
{
@@ -723,70 +748,11 @@ SysLogger_Start(void)
fclose(csvlogFile);
csvlogFile = NULL;
}
- return (int) sysloggerPid;
- }
-
- /* we should never reach here */
- return 0;
}
#ifdef EXEC_BACKEND
-/*
- * syslogger_forkexec() -
- *
- * Format up the arglist for, then fork and exec, a syslogger process
- */
-static pid_t
-syslogger_forkexec(void)
-{
- char *av[10];
- int ac = 0;
- char filenobuf[32];
- char csvfilenobuf[32];
-
- av[ac++] = "postgres";
- av[ac++] = "--forklog";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- /* static variables (those not passed by write_backend_variables) */
-#ifndef WIN32
- if (syslogFile != NULL)
- snprintf(filenobuf, sizeof(filenobuf), "%d",
- fileno(syslogFile));
- else
- strcpy(filenobuf, "-1");
-#else /* WIN32 */
- if (syslogFile != NULL)
- snprintf(filenobuf, sizeof(filenobuf), "%ld",
- (long) _get_osfhandle(_fileno(syslogFile)));
- else
- strcpy(filenobuf, "0");
-#endif /* WIN32 */
- av[ac++] = filenobuf;
-
-#ifndef WIN32
- if (csvlogFile != NULL)
- snprintf(csvfilenobuf, sizeof(csvfilenobuf), "%d",
- fileno(csvlogFile));
- else
- strcpy(csvfilenobuf, "-1");
-#else /* WIN32 */
- if (csvlogFile != NULL)
- snprintf(csvfilenobuf, sizeof(csvfilenobuf), "%ld",
- (long) _get_osfhandle(_fileno(csvlogFile)));
- else
- strcpy(csvfilenobuf, "0");
-#endif /* WIN32 */
- av[ac++] = csvfilenobuf;
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
/*
* syslogger_parseArgs() -
*
diff --git a/src/include/postmaster/fork_process.h b/src/include/postmaster/fork_process.h
index 86bad60cea..32e10e3629 100644
--- a/src/include/postmaster/fork_process.h
+++ b/src/include/postmaster/fork_process.h
@@ -28,6 +28,7 @@ typedef enum
AutoVacWorkerFork,
PgstatCollectorFork,
PgArchiverFork,
+ SysLoggerFork,
NUMFORKPROCTYPES /* Must be last! */
} ForkProcType;
diff --git a/src/include/postmaster/syslogger.h b/src/include/postmaster/syslogger.h
index 3a61104573..1d0b125141 100644
--- a/src/include/postmaster/syslogger.h
+++ b/src/include/postmaster/syslogger.h
@@ -14,6 +14,7 @@
#include <limits.h> /* for PIPE_BUF */
+#include "postmaster/fork_process.h"
/*
* Primitive protocol structure for writing to syslogger pipe(s). The idea
@@ -83,6 +84,8 @@ extern int SysLogger_Start(void);
extern void write_syslogger_file(const char *buffer, int count, int dest);
+extern void PrepSysLoggerFork(ForkProcData *logger_fork);
+
#ifdef EXEC_BACKEND
extern void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
#endif
--
2.23.0
0004-Add-archiver-to-process-centralization.patch (text/x-patch, 5.9K) download
From 367a6a98bd2b8ea5c810bd081027806c6045dfd8 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Fri, 27 Sep 2019 19:33:15 -0400
Subject: [PATCH 4/8] Add archiver to process centralization
---
src/backend/postmaster/pgarch.c | 89 ++++++---------------------
src/backend/postmaster/postmaster.c | 4 ++
src/include/postmaster/fork_process.h | 1 +
src/include/postmaster/pgarch.h | 5 +-
4 files changed, 28 insertions(+), 71 deletions(-)
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index f84f882c4c..53661004fd 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -92,10 +92,6 @@ static volatile sig_atomic_t ready_to_stop = false;
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgarch_forkexec(void);
-#endif
-
NON_EXEC_STATIC void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgarch_exit(SIGNAL_ARGS);
static void ArchSigHupHandler(SIGNAL_ARGS);
@@ -115,7 +111,7 @@ static void pgarch_archiveDone(char *xlog);
*/
/*
- * pgarch_start
+ * PrepPgArchiverFork
*
* Called from postmaster at startup or after an existing archiver
* died. Attempt to fire up a fresh archiver process.
@@ -124,17 +120,17 @@ static void pgarch_archiveDone(char *xlog);
*
* Note: if fail, we will be called again from the postmaster main loop.
*/
-int
-pgarch_start(void)
+void
+PrepPgArchiverFork(ForkProcData *pgarch_fork)
{
+ int ac = 0;
time_t curtime;
- pid_t pgArchPid;
/*
* Do nothing if no archiver needed
*/
if (!XLogArchivingActive())
- return 0;
+ return;
/*
* Do nothing if too soon since last archiver start. This is a safety
@@ -145,42 +141,20 @@ pgarch_start(void)
curtime = time(NULL);
if ((unsigned int) (curtime - last_pgarch_start_time) <
(unsigned int) PGARCH_RESTART_INTERVAL)
- return 0;
+ return;
last_pgarch_start_time = curtime;
-#ifdef EXEC_BACKEND
- switch ((pgArchPid = pgarch_forkexec()))
-#else
- switch ((pgArchPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork archiver: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
+ pgarch_fork->av[ac++] = pstrdup("postgres");
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgArchiverMain(0, NULL);
- break;
+#ifdef EXEC_BACKEND
+ pgarch_fork->av[ac++] = pstrdup("--forkarch");
+ pgarch_fork->av[ac++] = NULL; /* filled in by postmaster_forkexec */
#endif
+ pgarch_fork->ac = ac;
- default:
- return (int) pgArchPid;
- }
-
- /* shouldn't get here */
- return 0;
+ Assert(pgarch_fork->ac < lengthof(*pgarch_fork->av));
+ pgarch_fork->child_main = PgArchiverMain;
+ pgarch_fork->type_desc = pstrdup("archiver");
}
/* ------------------------------------------------------------
@@ -189,42 +163,19 @@ pgarch_start(void)
*/
-#ifdef EXEC_BACKEND
-
-/*
- * pgarch_forkexec() -
- *
- * Format up the arglist for, then fork and exec, archive process
- */
-static pid_t
-pgarch_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
-
- av[ac++] = "--forkarch";
-
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
/*
* PgArchiverMain
*
* The argc/argv parameters are valid only in EXEC_BACKEND case. However,
* since we don't use 'em, it hardly matters...
*/
-NON_EXEC_STATIC void
-PgArchiverMain(int argc, char *argv[])
+void
+PgArchiverMain(pg_attribute_unused() int argc, pg_attribute_unused() char *arg[])
{
+ /* Drop our connection to postmaster's shared memory, as well */
+ dsm_detach_all();
+ PGSharedMemoryDetach();
+
/*
* Ignore all signals usually bound to some action in the postmaster,
* except for SIGHUP, SIGTERM, SIGUSR1, SIGUSR2, and SIGQUIT.
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 37a36387a3..6bdd57d1e2 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -550,6 +550,7 @@ static void ShmemBackendArrayRemove(Backend *bn);
#define StartAutoVacLauncher() StartChildProcess(AutoVacLauncherFork)
#define StartAutoVacWorker() StartChildProcess(AutoVacWorkerFork)
#define pgstat_start() StartChildProcess(PgstatCollectorFork)
+#define pgarch_start() StartChildProcess(PgArchiverFork)
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -5463,6 +5464,9 @@ StartChildProcess(ForkProcType type)
case PgstatCollectorFork:
PrepPgstatCollectorFork(fork_data);
break;
+ case PgArchiverFork:
+ PrepPgArchiverFork(fork_data);
+ break;
default:
break;
}
diff --git a/src/include/postmaster/fork_process.h b/src/include/postmaster/fork_process.h
index 16ca8be968..86bad60cea 100644
--- a/src/include/postmaster/fork_process.h
+++ b/src/include/postmaster/fork_process.h
@@ -27,6 +27,7 @@ typedef enum
AutoVacLauncherFork,
AutoVacWorkerFork,
PgstatCollectorFork,
+ PgArchiverFork,
NUMFORKPROCTYPES /* Must be last! */
} ForkProcType;
diff --git a/src/include/postmaster/pgarch.h b/src/include/postmaster/pgarch.h
index 2474eac26a..ce23e39c8b 100644
--- a/src/include/postmaster/pgarch.h
+++ b/src/include/postmaster/pgarch.h
@@ -13,6 +13,8 @@
#ifndef _PGARCH_H
#define _PGARCH_H
+#include "postmaster/fork_process.h"
+
/* ----------
* Archiver control info.
*
@@ -30,10 +32,9 @@
* Functions called from postmaster
* ----------
*/
-extern int pgarch_start(void);
-
#ifdef EXEC_BACKEND
extern void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
#endif
+extern void PrepPgArchiverFork(ForkProcData *pgarch_fork);
#endif /* _PGARCH_H */
--
2.23.0
0002-Add-centralized-autovac-launcher-worker.patch (text/x-patch, 11.4K) download
From a6f005d9ea81e1a989353f320149e12281e9fb04 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Fri, 27 Sep 2019 16:45:57 -0400
Subject: [PATCH 2/8] Add centralized autovac launcher/worker
---
src/backend/postmaster/autovacuum.c | 157 ++++----------------------
src/backend/postmaster/postmaster.c | 67 ++++++-----
src/include/postmaster/autovacuum.h | 9 +-
src/include/postmaster/fork_process.h | 2 +
4 files changed, 70 insertions(+), 165 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 073f313337..5c3ac1568f 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -134,8 +134,8 @@ int Log_autovacuum_min_duration = -1;
#define MAX_AUTOVAC_SLEEPTIME 300 /* seconds */
/* Flags to tell if we are in an autovacuum process */
-static bool am_autovacuum_launcher = false;
-static bool am_autovacuum_worker = false;
+bool am_autovacuum_launcher = false;
+bool am_autovacuum_worker = false;
/* Flags set by signal handlers */
static volatile sig_atomic_t got_SIGHUP = false;
@@ -303,10 +303,6 @@ static WorkerInfo MyWorkerInfo = NULL;
/* PID of launcher, valid only in worker while shutting down */
int AutovacuumLauncherPid = 0;
-#ifdef EXEC_BACKEND
-static pid_t avlauncher_forkexec(void);
-static pid_t avworker_forkexec(void);
-#endif
NON_EXEC_STATIC void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
NON_EXEC_STATIC void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
@@ -347,88 +343,53 @@ static void avl_sigusr2_handler(SIGNAL_ARGS);
static void avl_sigterm_handler(SIGNAL_ARGS);
static void autovac_refresh_stats(void);
-
-
/********************************************************************
* AUTOVACUUM LAUNCHER CODE
********************************************************************/
-#ifdef EXEC_BACKEND
/*
- * forkexec routine for the autovacuum launcher process.
+ * PrepAutoVacProcessFork
*
* Format up the arglist, then fork and exec.
*/
-static pid_t
-avlauncher_forkexec(void)
+void
+PrepAutoVacProcessFork(ForkProcData *autovac_fork)
{
- char *av[10];
int ac = 0;
- av[ac++] = "postgres";
- av[ac++] = "--forkavlauncher";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
/*
- * We need this set from the outside, before InitProcess is called
+ * Set up command-line arguments for subprocess
*/
-void
-AutovacuumLauncherIAm(void)
-{
- am_autovacuum_launcher = true;
-}
-#endif
+ autovac_fork->av[ac++] = pstrdup("postgres");
-/*
- * Main entry point for autovacuum launcher process, to be called from the
- * postmaster.
- */
-int
-StartAutoVacLauncher(void)
+ if (MyForkProcType == AutoVacLauncherFork)
{
- pid_t AutoVacPID;
-
+ autovac_fork->type_desc = pstrdup("autovacuum launcher");
+ autovac_fork->child_main = AutoVacLauncherMain;
#ifdef EXEC_BACKEND
- switch ((AutoVacPID = avlauncher_forkexec()))
-#else
- switch ((AutoVacPID = fork_process()))
+ autovac_fork->av[ac++] = pstrdup("--forkavlauncher");
+ autovac_fork->av[ac++] = NULL; /* filled in by postmaster_forkexec */
#endif
+ }
+ else if (MyForkProcType == AutoVacWorkerFork)
{
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum launcher process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacLauncherMain(0, NULL);
- break;
+ autovac_fork->type_desc = pstrdup("autovacuum worker");
+ autovac_fork->child_main = AutoVacWorkerMain;
+#ifdef EXEC_BACKEND
+ autovac_fork->av[ac++] = pstrdup("--forkavworker");
+ autovac_fork->av[ac++] = NULL; /* filled in by postmaster_forkexec */
#endif
- default:
- return (int) AutoVacPID;
}
- /* shouldn't get here */
- return 0;
+ autovac_fork->ac = ac;
+ Assert(autovac_fork->ac < lengthof(*autovac_fork->av));
}
/*
* Main loop for the autovacuum launcher process.
*/
NON_EXEC_STATIC void
-AutoVacLauncherMain(int argc, char *argv[])
+AutoVacLauncherMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
@@ -1428,83 +1389,11 @@ avl_sigterm_handler(SIGNAL_ARGS)
* AUTOVACUUM WORKER CODE
********************************************************************/
-#ifdef EXEC_BACKEND
-/*
- * forkexec routines for the autovacuum worker.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avworker_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavworker";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
-/*
- * We need this set from the outside, before InitProcess is called
- */
-void
-AutovacuumWorkerIAm(void)
-{
- am_autovacuum_worker = true;
-}
-#endif
-
-/*
- * Main entry point for autovacuum worker process.
- *
- * This code is heavily based on pgarch.c, q.v.
- */
-int
-StartAutoVacWorker(void)
-{
- pid_t worker_pid;
-
-#ifdef EXEC_BACKEND
- switch ((worker_pid = avworker_forkexec()))
-#else
- switch ((worker_pid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum worker process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacWorkerMain(0, NULL);
- break;
-#endif
- default:
- return (int) worker_pid;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
/*
* AutoVacWorkerMain
*/
NON_EXEC_STATIC void
-AutoVacWorkerMain(int argc, char *argv[])
+AutoVacWorkerMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
Oid dbid;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index e0d11cc6ab..35cd1479b9 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -414,6 +414,9 @@ static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+#ifdef EXEC_BACKEND
+static void shmemSetup(bool aux_process);
+#endif
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
@@ -544,6 +547,8 @@ static void ShmemBackendArrayRemove(Backend *bn);
#define StartCheckpointer() StartChildProcess(CheckpointerFork)
#define StartWalWriter() StartChildProcess(WalWriterFork)
#define StartWalReceiver() StartChildProcess(WalReceiverFork)
+#define StartAutoVacLauncher() StartChildProcess(AutoVacLauncherFork)
+#define StartAutoVacWorker() StartChildProcess(AutoVacWorkerFork)
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -4858,6 +4863,29 @@ retry:
}
#endif /* WIN32 */
+/*
+ * shmemSetup
+ *
+ * Helper function for a child to set up shmem before
+ * executing.
+ *
+ * aux_process - set to true if an auxiliary process.
+ */
+static void
+shmemSetup(bool aux_process)
+{
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
+
+ /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ if (aux_process)
+ InitAuxiliaryProcess();
+ else
+ InitProcess();
+
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores();
+}
/*
* SubPostmasterMain -- Get the fork/exec'd process into a state equivalent
@@ -4943,9 +4971,9 @@ SubPostmasterMain(int argc, char *argv[])
/* autovacuum needs this set before calling InitProcess */
if (strcmp(argv[1], "--forkavlauncher") == 0)
- AutovacuumLauncherIAm();
+ am_autovacuum_launcher = true;
if (strcmp(argv[1], "--forkavworker") == 0)
- AutovacuumWorkerIAm();
+ am_autovacuum_worker = true;
/*
* Start our win32 signal implementation. This has to be done after we
@@ -5038,41 +5066,17 @@ SubPostmasterMain(int argc, char *argv[])
}
if (strcmp(argv[1], "--forkboot") == 0)
{
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitAuxiliaryProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
+ shmemSetup(true);
AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
}
if (strcmp(argv[1], "--forkavlauncher") == 0)
{
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
+ shmemSetup(true);
AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
}
if (strcmp(argv[1], "--forkavworker") == 0)
{
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
+ shmemSetup(true);
AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
}
if (strncmp(argv[1], "--forkbgworker=", 15) == 0)
@@ -5450,6 +5454,11 @@ StartChildProcess(ForkProcType type)
case WalReceiverFork:
PrepAuxProcessFork(fork_data);
break;
+ /* Non-Auxiliary Processes */
+ case AutoVacLauncherFork:
+ case AutoVacWorkerFork:
+ PrepAutoVacProcessFork(fork_data);
+ break;
default:
break;
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 8451e5d9e2..518775f528 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -15,6 +15,7 @@
#define AUTOVACUUM_H
#include "storage/block.h"
+#include "postmaster/fork_process.h"
/*
* Other processes can request specific work from autovacuum, identified by
@@ -45,6 +46,10 @@ extern int AutovacuumLauncherPid;
extern int Log_autovacuum_min_duration;
+/* autovacuum identification */
+extern bool am_autovacuum_launcher;
+extern bool am_autovacuum_worker;
+
/* Status inquiry functions */
extern bool AutoVacuumingActive(void);
extern bool IsAutoVacuumLauncherProcess(void);
@@ -58,6 +63,8 @@ extern void autovac_init(void);
extern int StartAutoVacLauncher(void);
extern int StartAutoVacWorker(void);
+extern void PrepAutoVacProcessFork(ForkProcData *autovac_fork);
+
/* called from postmaster when a worker could not be forked */
extern void AutoVacWorkerFailed(void);
@@ -67,8 +74,6 @@ extern void AutoVacuumUpdateDelay(void);
#ifdef EXEC_BACKEND
extern void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern void AutovacuumWorkerIAm(void);
-extern void AutovacuumLauncherIAm(void);
#endif
extern bool AutoVacuumRequestWork(AutoVacuumWorkItemType type,
diff --git a/src/include/postmaster/fork_process.h b/src/include/postmaster/fork_process.h
index d912229055..1f319fc98f 100644
--- a/src/include/postmaster/fork_process.h
+++ b/src/include/postmaster/fork_process.h
@@ -24,6 +24,8 @@ typedef enum
CheckpointerFork,
WalWriterFork,
WalReceiverFork, /* end of Auxiliary Process Forks */
+ AutoVacLauncherFork,
+ AutoVacWorkerFork,
NUMFORKPROCTYPES /* Must be last! */
} ForkProcType;
--
2.23.0
0003-Add-centralized-stat-collector.patch (text/x-patch, 6.2K) download
From efc6f0531b71847c6500062b5a0fe43331a6ea7e Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Fri, 27 Sep 2019 19:13:53 -0400
Subject: [PATCH 3/8] Add centralized stat collector
---
src/backend/postmaster/pgstat.c | 88 ++++++---------------------
src/backend/postmaster/postmaster.c | 5 +-
src/include/pgstat.h | 4 +-
src/include/postmaster/fork_process.h | 1 +
4 files changed, 26 insertions(+), 72 deletions(-)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 011076c3e3..73d953aedf 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -280,10 +280,6 @@ static instr_time total_func_time;
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgstat_forkexec(void);
-#endif
-
NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgstat_exit(SIGNAL_ARGS);
static void pgstat_beshutdown_hook(int code, Datum arg);
@@ -689,53 +685,26 @@ pgstat_reset_all(void)
pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
}
-#ifdef EXEC_BACKEND
/*
- * pgstat_forkexec() -
- *
- * Format up the arglist for, then fork and exec, statistics collector process
- */
-static pid_t
-pgstat_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkcol";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
-/*
- * pgstat_start() -
+ * PrepPgstatCollectorFork
*
* Called from postmaster at startup or after an existing collector
* died. Attempt to fire up a fresh statistics collector.
*
- * Returns PID of child process, or 0 if fail.
- *
- * Note: if fail, we will be called again from the postmaster main loop.
*/
-int
-pgstat_start(void)
+void
+PrepPgstatCollectorFork(ForkProcData *pgstat_fork)
{
+ int ac = 0;
time_t curtime;
- pid_t pgStatPid;
/*
* Check that the socket is there, else pgstat_init failed and we can do
* nothing useful.
*/
if (pgStatSock == PGINVALID_SOCKET)
- return 0;
+ return;
/*
* Do nothing if too soon since last collector start. This is a safety
@@ -746,45 +715,20 @@ pgstat_start(void)
curtime = time(NULL);
if ((unsigned int) (curtime - last_pgstat_start_time) <
(unsigned int) PGSTAT_RESTART_INTERVAL)
- return 0;
+ return;
last_pgstat_start_time = curtime;
- /*
- * Okay, fork off the collector.
- */
#ifdef EXEC_BACKEND
- switch ((pgStatPid = pgstat_forkexec()))
-#else
- switch ((pgStatPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork statistics collector: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgstatCollectorMain(0, NULL);
- break;
+ pgstat_fork->av[ac++] = pstrdup("postgres");
+ pgstat_fork->av[ac++] = pstrdup("--forkcol");
+ pgstat_fork->av[ac++] = NULL; /* filled in by postmaster_forkexec */
#endif
- default:
- return (int) pgStatPid;
- }
+ pgstat_fork->ac = ac;
+ Assert(pgstat_fork->ac < lengthof(*pgstat_fork->av));
- /* shouldn't get here */
- return 0;
+ pgstat_fork->type_desc = pstrdup("statistics collector");
+ pgstat_fork->child_main = PgstatCollectorMain;
}
void
@@ -4425,12 +4369,16 @@ pgstat_send_bgwriter(void)
* ----------
*/
NON_EXEC_STATIC void
-PgstatCollectorMain(int argc, char *argv[])
+PgstatCollectorMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
int len;
PgStat_Msg msg;
int wr;
+ /* Drop our connection to postmaster's shared memory, as well */
+ dsm_detach_all();
+ PGSharedMemoryDetach();
+
/*
* Ignore all signals usually bound to some action in the postmaster,
* except SIGHUP and SIGQUIT. Note we don't need a SIGUSR1 handler to
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 35cd1479b9..37a36387a3 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -549,6 +549,7 @@ static void ShmemBackendArrayRemove(Backend *bn);
#define StartWalReceiver() StartChildProcess(WalReceiverFork)
#define StartAutoVacLauncher() StartChildProcess(AutoVacLauncherFork)
#define StartAutoVacWorker() StartChildProcess(AutoVacWorkerFork)
+#define pgstat_start() StartChildProcess(PgstatCollectorFork)
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -5459,7 +5460,9 @@ StartChildProcess(ForkProcType type)
case AutoVacWorkerFork:
PrepAutoVacProcessFork(fork_data);
break;
-
+ case PgstatCollectorFork:
+ PrepPgstatCollectorFork(fork_data);
+ break;
default:
break;
}
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index fe076d823d..00e95caa60 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -15,6 +15,7 @@
#include "libpq/pqcomm.h"
#include "port/atomics.h"
#include "portability/instr_time.h"
+#include "postmaster/fork_process.h"
#include "postmaster/pgarch.h"
#include "storage/proc.h"
#include "utils/hsearch.h"
@@ -1235,10 +1236,11 @@ extern Size BackendStatusShmemSize(void);
extern void CreateSharedBackendStatus(void);
extern void pgstat_init(void);
-extern int pgstat_start(void);
extern void pgstat_reset_all(void);
extern void allow_immediate_pgstat_restart(void);
+extern void PrepPgstatCollectorFork(ForkProcData *pgstat_fork);
+
#ifdef EXEC_BACKEND
extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
#endif
diff --git a/src/include/postmaster/fork_process.h b/src/include/postmaster/fork_process.h
index 1f319fc98f..16ca8be968 100644
--- a/src/include/postmaster/fork_process.h
+++ b/src/include/postmaster/fork_process.h
@@ -26,6 +26,7 @@ typedef enum
WalReceiverFork, /* end of Auxiliary Process Forks */
AutoVacLauncherFork,
AutoVacWorkerFork,
+ PgstatCollectorFork,
NUMFORKPROCTYPES /* Must be last! */
} ForkProcType;
--
2.23.0
0001-Add-ForkProcType-infrastructure.patch (text/x-patch, 13.0K) download
From 2a3a35f2e80cb2badcb0efbce1bad2484e364b7b Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Fri, 27 Sep 2019 12:28:19 -0400
Subject: [PATCH 1/8] Add ForkProcType infrastructure
---
src/backend/bootstrap/bootstrap.c | 1 +
src/backend/main/main.c | 7 +
src/backend/postmaster/postmaster.c | 197 ++++++++++++++++----------
src/backend/utils/init/globals.c | 3 +-
src/backend/utils/init/postinit.c | 2 +-
src/include/bootstrap/bootstrap.h | 2 +-
src/include/postmaster/fork_process.h | 33 +++++
7 files changed, 167 insertions(+), 78 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 9238fbe98d..9f3dad1c6d 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -70,6 +70,7 @@ static void cleanup(void);
*/
AuxProcType MyAuxProcType = NotAnAuxProcess; /* declared in miscadmin.h */
+ForkProcType MyForkProcType = NoForkProcess; /* declared in fork_process.h */
Relation boot_reldesc; /* current relation descriptor */
diff --git a/src/backend/main/main.c b/src/backend/main/main.c
index a9edbfd4a4..79942deb7d 100644
--- a/src/backend/main/main.c
+++ b/src/backend/main/main.c
@@ -199,7 +199,14 @@ main(int argc, char *argv[])
#endif
if (argc > 1 && strcmp(argv[1], "--boot") == 0)
+ {
+ MemoryContext old_ctx;
+
+ old_ctx = MemoryContextSwitchTo(TopMemoryContext);
+
AuxiliaryProcessMain(argc, argv); /* does not return */
+ MemoryContextSwitchTo(old_ctx);
+ }
else if (argc > 1 && strcmp(argv[1], "--describe-config") == 0)
GucInfoMain(); /* does not return */
else if (argc > 1 && strcmp(argv[1], "--single") == 0)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index eb9e0221f8..e0d11cc6ab 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -421,7 +421,7 @@ static int CountChildren(int target);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
-static pid_t StartChildProcess(AuxProcType type);
+static pid_t StartChildProcess(ForkProcType type);
static void StartAutovacuumWorker(void);
static void MaybeStartWalReceiver(void);
static void InitPostmasterDeathWatchHandle(void);
@@ -456,7 +456,7 @@ typedef struct
#endif /* WIN32 */
static pid_t backend_forkexec(Port *port);
-static pid_t internal_forkexec(int argc, char *argv[], Port *port);
+static pid_t internal_forkexec(int argc, char *argv[]);
/* Type for a socket that can be inherited to a client process */
#ifdef WIN32
@@ -522,6 +522,7 @@ typedef struct
char my_exec_path[MAXPGPATH];
char pkglib_path[MAXPGPATH];
char ExtraOptions[MAXPGPATH];
+ ForkProcType proc_type;
} BackendParameters;
static void read_backend_variables(char *id, Port *port);
@@ -538,11 +539,11 @@ static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
-#define StartupDataBase() StartChildProcess(StartupProcess)
-#define StartBackgroundWriter() StartChildProcess(BgWriterProcess)
-#define StartCheckpointer() StartChildProcess(CheckpointerProcess)
-#define StartWalWriter() StartChildProcess(WalWriterProcess)
-#define StartWalReceiver() StartChildProcess(WalReceiverProcess)
+#define StartupDataBase() StartChildProcess(StartupFork)
+#define StartBackgroundWriter() StartChildProcess(BgWriterFork)
+#define StartCheckpointer() StartChildProcess(CheckpointerFork)
+#define StartWalWriter() StartChildProcess(WalWriterFork)
+#define StartWalReceiver() StartChildProcess(WalReceiverFork)
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -560,6 +561,60 @@ int postmaster_alive_fds[2] = {-1, -1};
HANDLE PostmasterHandle;
#endif
+static Port *ConnProcPort = NULL;
+
+/*
+ * PrepAuxProcessFork
+ *
+ * Prepare a ForkProcType struct for the auxiliary process specified by
+ * AuxProcType. This does all prep related to av parameters and error messages.
+ */
+static void
+PrepAuxProcessFork(ForkProcData *aux_fork)
+{
+ int ac = 0;
+
+ /*
+ * Set up command-line arguments for subprocess
+ */
+ aux_fork->av[ac++] = pstrdup("postgres");
+
+#ifdef EXEC_BACKEND
+ aux_fork->av[ac++] = pstrdup("--forkboot");
+ aux_fork->av[ac++] = NULL; /* filled in by postmaster_forkexec */
+#endif
+
+ aux_fork->av[ac++] = psprintf("-x%d", MyForkProcType);
+
+ aux_fork->av[ac] = NULL;
+ Assert(ac < lengthof(*aux_fork->av));
+
+ aux_fork->ac = ac;
+ switch (MyForkProcType)
+ {
+ case StartupProcess:
+ aux_fork->type_desc = pstrdup("startup");
+ break;
+ case BgWriterProcess:
+ aux_fork->type_desc = pstrdup("background writer");
+ break;
+ case CheckpointerProcess:
+ aux_fork->type_desc = pstrdup("checkpointer");
+ break;
+ case WalWriterProcess:
+ aux_fork->type_desc = pstrdup("WAL writer");
+ break;
+ case WalReceiverProcess:
+ aux_fork->type_desc = pstrdup("WAL receiver");
+ break;
+ default:
+ aux_fork->type_desc = pstrdup("child");
+ break;
+ }
+
+ aux_fork->child_main = AuxiliaryProcessMain;
+}
+
/*
* Postmaster main entry point
*/
@@ -1710,19 +1765,17 @@ ServerLoop(void)
break;
if (FD_ISSET(ListenSocket[i], &rmask))
{
- Port *port;
-
- port = ConnCreate(ListenSocket[i]);
- if (port)
+ ConnProcPort = ConnCreate(ListenSocket[i]);
+ if (ConnProcPort)
{
- BackendStartup(port);
+ BackendStartup(ConnProcPort);
/*
* We no longer need the open socket or port structure
* in this process
*/
- StreamClose(port->sock);
- ConnFree(port);
+ StreamClose(ConnProcPort->sock);
+ ConnFree(ConnProcPort);
}
}
}
@@ -4477,11 +4530,11 @@ BackendRun(Port *port)
pid_t
postmaster_forkexec(int argc, char *argv[])
{
- Port port;
-
/* This entry point passes dummy values for the Port variables */
- memset(&port, 0, sizeof(port));
- return internal_forkexec(argc, argv, &port);
+ if (!ConnProcPort)
+ ConnProcPort = palloc0(sizeof(*ConnProcPort));
+
+ return internal_forkexec(argc, argv);
}
/*
@@ -4506,7 +4559,7 @@ backend_forkexec(Port *port)
av[ac] = NULL;
Assert(ac < lengthof(av));
- return internal_forkexec(ac, av, port);
+ return internal_forkexec(ac, av);
}
#ifndef WIN32
@@ -4518,7 +4571,7 @@ backend_forkexec(Port *port)
* - fork():s, and then exec():s the child process
*/
static pid_t
-internal_forkexec(int argc, char *argv[], Port *port)
+internal_forkexec(int argc, char *argv[])
{
static unsigned long tmpBackendFileNum = 0;
pid_t pid;
@@ -4526,7 +4579,7 @@ internal_forkexec(int argc, char *argv[], Port *port)
BackendParameters param;
FILE *fp;
- if (!save_backend_variables(¶m, port))
+ if (!save_backend_variables(¶m, ConnProcPort))
return -1; /* log made by save_backend_variables */
/* Calculate name for temp file */
@@ -4836,9 +4889,12 @@ SubPostmasterMain(int argc, char *argv[])
if (argc < 3)
elog(FATAL, "invalid subpostmaster invocation");
+ Assert(!ConnProcPort);
+
/* Read in the variables file */
memset(&port, 0, sizeof(Port));
read_backend_variables(argv[2], &port);
+ ConnProcPort = &port;
/* Close the postmaster's sockets (as soon as we know them) */
ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
@@ -5360,7 +5416,6 @@ CountChildren(int target)
return cnt;
}
-
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5371,36 +5426,50 @@ CountChildren(int target)
* to start subprocess.
*/
static pid_t
-StartChildProcess(AuxProcType type)
+StartChildProcess(ForkProcType type)
{
pid_t pid;
- char *av[10];
- int ac = 0;
- char typebuf[32];
-
- /*
- * Set up command-line arguments for subprocess
- */
- av[ac++] = "postgres";
+ ForkProcData *fork_data;
+ MemoryContext old_ctx;
+
+ /* Keep fork data after splitting from postmaster */
+ old_ctx = MemoryContextSwitchTo(TopMemoryContext);
+ fork_data = (ForkProcData *) palloc0(sizeof(ForkProcData));
+ fork_data->av = palloc0(sizeof(char *) * 10);
+ MyForkProcType = type;
+
+ switch(type)
+ {
+ /* Auxiliary Processes */
+ case CheckerFork:
+ case BootstrapFork:
+ case StartupFork:
+ case BgWriterFork:
+ case CheckpointerFork:
+ case WalWriterFork:
+ case WalReceiverFork:
+ PrepAuxProcessFork(fork_data);
+ break;
-#ifdef EXEC_BACKEND
- av[ac++] = "--forkboot";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-#endif
+ default:
+ break;
+ }
- snprintf(typebuf, sizeof(typebuf), "-x%d", type);
- av[ac++] = typebuf;
+ MemoryContextSwitchTo(old_ctx);
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ /* Bail out if pre-conditions are not met */
+ if (fork_data == NULL)
+ return 0;
#ifdef EXEC_BACKEND
- pid = postmaster_forkexec(ac, av);
-#else /* !EXEC_BACKEND */
+ pid = postmaster_forkexec(fork_data->ac, fork_data->av);
+#else
pid = fork_process();
+#endif
if (pid == 0) /* child */
{
+#ifndef EXEC_BACKEND
InitPostmasterChild();
/* Close the postmaster's sockets */
@@ -5410,52 +5479,30 @@ StartChildProcess(AuxProcType type)
MemoryContextSwitchTo(TopMemoryContext);
MemoryContextDelete(PostmasterContext);
PostmasterContext = NULL;
+#endif
+ /* Call the process's main subroutine */
+ fork_data->child_main (fork_data->ac, fork_data->av);
- AuxiliaryProcessMain(ac, av);
+#ifndef EXEC_BACKEND
ExitPostmaster(0);
+#endif
}
-#endif /* EXEC_BACKEND */
-
- if (pid < 0)
+ else if (pid < 0)
{
/* in parent, fork failed */
- int save_errno = errno;
+ int save_errno = errno;
- errno = save_errno;
- switch (type)
- {
- case StartupProcess:
- ereport(LOG,
- (errmsg("could not fork startup process: %m")));
- break;
- case BgWriterProcess:
- ereport(LOG,
- (errmsg("could not fork background writer process: %m")));
- break;
- case CheckpointerProcess:
- ereport(LOG,
- (errmsg("could not fork checkpointer process: %m")));
- break;
- case WalWriterProcess:
- ereport(LOG,
- (errmsg("could not fork WAL writer process: %m")));
- break;
- case WalReceiverProcess:
- ereport(LOG,
- (errmsg("could not fork WAL receiver process: %m")));
- break;
- default:
- ereport(LOG,
- (errmsg("could not fork process: %m")));
- break;
- }
+ errno = save_errno;
+ ereport(LOG,
+ (errmsg("could not fork %s process: %m", fork_data->type_desc)));
/*
* fork failure is fatal during startup, but there's no need to choke
* immediately if starting other child types fails.
*/
- if (type == StartupProcess)
+ if (MyForkProcType == StartupFork)
ExitPostmaster(1);
+
return 0;
}
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 3bf96de256..51657924f1 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -38,9 +38,10 @@ volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
int MyProcPid;
+int MyChildProcPid;
pg_time_t MyStartTime;
TimestampTz MyStartTimestamp;
-struct Port *MyProcPort;
+struct Port *MyProcPort = NULL;
int32 MyCancelKey;
int MyPMChildSlot;
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 29c5ec7b58..c0840c33a7 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -1113,7 +1113,7 @@ process_startup_options(Port *port, bool am_superuser)
av = (char **) palloc(maxac * sizeof(char *));
ac = 0;
- av[ac++] = "postgres";
+ av[ac++] = pstrdup("postgres");
pg_split_opts(av, &ac, port->cmdline_options);
diff --git a/src/include/bootstrap/bootstrap.h b/src/include/bootstrap/bootstrap.h
index 85705602cb..3c53454069 100644
--- a/src/include/bootstrap/bootstrap.h
+++ b/src/include/bootstrap/bootstrap.h
@@ -15,7 +15,7 @@
#define BOOTSTRAP_H
#include "nodes/execnodes.h"
-
+#include "postmaster/fork_process.h"
/*
* MAXATTR is the maximum number of attributes in a relation supported
diff --git a/src/include/postmaster/fork_process.h b/src/include/postmaster/fork_process.h
index eb3c8ec974..d912229055 100644
--- a/src/include/postmaster/fork_process.h
+++ b/src/include/postmaster/fork_process.h
@@ -12,6 +12,39 @@
#ifndef FORK_PROCESS_H
#define FORK_PROCESS_H
+#include "postmaster.h"
+
+typedef enum
+{
+ NoForkProcess = -1,
+ CheckerFork = 0,
+ BootstrapFork,
+ StartupFork,
+ BgWriterFork,
+ CheckpointerFork,
+ WalWriterFork,
+ WalReceiverFork, /* end of Auxiliary Process Forks */
+
+ NUMFORKPROCTYPES /* Must be last! */
+} ForkProcType;
+
+extern ForkProcType MyForkProcType;
+
+struct ForkProcData;
+
+typedef void (*ChildProcessMain) (int argc, char *argv[]);
+
+typedef struct ForkProcData
+{
+ char **av;
+ int ac;
+ int save_errno; /* for failure */
+ char *type_desc;
+ ChildProcessMain postmaster_main;
+ ChildProcessMain child_main;
+ ChildProcessMain fail_main;
+} ForkProcData;
+
extern pid_t fork_process(void);
#endif /* FORK_PROCESS_H */
--
2.23.0
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
@ 2019-10-03 17:31 ` Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
0 siblings, 1 reply; 38+ messages in thread
From: Andres Freund @ 2019-10-03 17:31 UTC (permalink / raw)
To: Mike Palmiotto <mike.palmiotto@crunchydata.com>; +Cc: Alvaro Herrera <alvherre@2ndquadrant.com>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
Hi,
On 2019-09-30 15:13:18 -0400, Mike Palmiotto wrote:
> Attached is the reworked and rebased patch set. I put the hook on top
> and a separate commit for each process type. Note that avworker and
> avlauncher were intentionally left together. Let me know if you think
> those should be split out as well.
I've not looked at this before, nor the thread. Just jumping here
because of the reference to this patch you made in another thread...
In short, I think this is far from ready, nor am I sure this is going in
the right direction. There's a bit more detail about where I think this
ought to go interspersed below (in particular search for PgSubProcess,
EXEC_BACKEND).
> From 205ea86dde898f7edac327d46b2b43b04721aadc Mon Sep 17 00:00:00 2001
> From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
> Date: Mon, 30 Sep 2019 14:42:53 -0400
> Subject: [PATCH 7/8] Add backends to process centralization
.oO(out of order attached patches, how fun). 7, 8, 6, 5, 4, 2, 3, 1...
> From 2a3a35f2e80cb2badcb0efbce1bad2484e364b7b Mon Sep 17 00:00:00 2001
> From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
> Date: Fri, 27 Sep 2019 12:28:19 -0400
> Subject: [PATCH 1/8] Add ForkProcType infrastructure
A better explanation would be nice. It's hard to review non-trivial
patches that have little explanation as to what they're
achieving. Especially when there's some unrelated seeming changes.
> diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
> index 9238fbe98d..9f3dad1c6d 100644
> --- a/src/backend/bootstrap/bootstrap.c
> +++ b/src/backend/bootstrap/bootstrap.c
> @@ -70,6 +70,7 @@ static void cleanup(void);
> */
>
> AuxProcType MyAuxProcType = NotAnAuxProcess; /* declared in miscadmin.h */
> +ForkProcType MyForkProcType = NoForkProcess; /* declared in fork_process.h */
>
> Relation boot_reldesc; /* current relation descriptor
> */
IMO, before this stuff gets massaged meaningfully, we ought to move code
like this (including AuxiliaryProcessMain etc) somewhere where it makes
sense. If we're going to rejigger things, it ought to be towards a more
understandable architecture, rather than keeping weird quirks around.
I'm inclined to think that this better also include properly
centralizing a good portion of the EXEC_BACKEND code. Building
infrastructure that's supposed to be long-lived with that being left as
is just seems like recipe for yet another odd framework.
> @@ -538,11 +539,11 @@ static void ShmemBackendArrayAdd(Backend *bn);
> static void ShmemBackendArrayRemove(Backend *bn);
> #endif /* EXEC_BACKEND */
>
> -#define StartupDataBase() StartChildProcess(StartupProcess)
> -#define StartBackgroundWriter() StartChildProcess(BgWriterProcess)
> -#define StartCheckpointer() StartChildProcess(CheckpointerProcess)
> -#define StartWalWriter() StartChildProcess(WalWriterProcess)
> -#define StartWalReceiver() StartChildProcess(WalReceiverProcess)
> +#define StartupDataBase() StartChildProcess(StartupFork)
> +#define StartBackgroundWriter() StartChildProcess(BgWriterFork)
> +#define StartCheckpointer() StartChildProcess(CheckpointerFork)
> +#define StartWalWriter() StartChildProcess(WalWriterFork)
> +#define StartWalReceiver() StartChildProcess(WalReceiverFork)
FWIW, I'd just rip this out, it adds exactly nothing but one more place
to change.
> +/*
> + * PrepAuxProcessFork
> + *
> + * Prepare a ForkProcType struct for the auxiliary process specified by
ForkProcData, not ForkProcType
> + * AuxProcType. This does all prep related to av parameters and error messages.
> + */
> +static void
> +PrepAuxProcessFork(ForkProcData *aux_fork)
> +{
> + int ac = 0;
> +
> + /*
> + * Set up command-line arguments for subprocess
> + */
> + aux_fork->av[ac++] = pstrdup("postgres");
There's no documentation as to what 'av' is actually supposed to mean. I
assume auxvector or such, but if so that's neither obvious, nor
necessarily good.
> +#ifdef EXEC_BACKEND
> + aux_fork->av[ac++] = pstrdup("--forkboot");
> + aux_fork->av[ac++] = NULL; /* filled in by postmaster_forkexec */
> +#endif
What's the point of pstrdup()ing constant strings here? Afaict you're
not freeing them ever?
> + aux_fork->av[ac++] = psprintf("-x%d", MyForkProcType);
> +
> + aux_fork->av[ac] = NULL;
> + Assert(ac < lengthof(*aux_fork->av));
Unless I miss something, the lengthof here doesn't do anythign
useful. ForkProcData->av is defined as:
> +typedef struct ForkProcData
> +{
> + char **av;
which I think means you're computing sizeof(char*)/sizeof(char)?
> + aux_fork->ac = ac;
> + switch (MyForkProcType)
> + {
> + case StartupProcess:
> + aux_fork->type_desc = pstrdup("startup");
> + break;
> + case BgWriterProcess:
> + aux_fork->type_desc = pstrdup("background writer");
> + break;
> + case CheckpointerProcess:
> + aux_fork->type_desc = pstrdup("checkpointer");
> + break;
> + case WalWriterProcess:
> + aux_fork->type_desc = pstrdup("WAL writer");
> + break;
> + case WalReceiverProcess:
> + aux_fork->type_desc = pstrdup("WAL receiver");
> + break;
> + default:
> + aux_fork->type_desc = pstrdup("child");
> + break;
> + }
> +
> + aux_fork->child_main = AuxiliaryProcessMain;
I'm really not in love in having details like the descriptors of
processes, the types of processes, the mapping from MyForkProcType to
AuxProcType in multiple places. I feel this is adding even more
confusion to this, rather than reducing it.
I feel like we should define the properties of the different types of
processes in *one* place, rather multiple. Define one array of
structs that contains information about the different types of
processes, rather than distributing that knowledge all over.
> @@ -4477,11 +4530,11 @@ BackendRun(Port *port)
> pid_t
> postmaster_forkexec(int argc, char *argv[])
> {
> - Port port;
> -
> /* This entry point passes dummy values for the Port variables */
> - memset(&port, 0, sizeof(port));
> - return internal_forkexec(argc, argv, &port);
> + if (!ConnProcPort)
> + ConnProcPort = palloc0(sizeof(*ConnProcPort));
> +
> + return internal_forkexec(argc, argv);
> }
What's the point of creating a dummy ConnProcPort? And when is this
being pfreed? I mean previously this at least used a stack variable that
was automatically being released by the time postmaster_forkexec
returned, now it sure looks like a permanent leak to me.
Also, why is it a good idea to rely *more* on data being implicitly
passed by stuffing things into global variables? That strikes me as
> /* in parent, fork failed */
> - int save_errno = errno;
> + int save_errno = errno;
>
> - errno = save_errno;
> - switch (type)
> - {
> - case StartupProcess:
> - ereport(LOG,
> - (errmsg("could not fork startup process: %m")));
> - break;
> - case BgWriterProcess:
> - ereport(LOG,
> - (errmsg("could not fork background writer process: %m")));
> - break;
> - case CheckpointerProcess:
> - ereport(LOG,
> - (errmsg("could not fork checkpointer process: %m")));
> - break;
> - case WalWriterProcess:
> - ereport(LOG,
> - (errmsg("could not fork WAL writer process: %m")));
> - break;
> - case WalReceiverProcess:
> - ereport(LOG,
> - (errmsg("could not fork WAL receiver process: %m")));
> - break;
> - default:
> - ereport(LOG,
> - (errmsg("could not fork process: %m")));
> - break;
> - }
> + errno = save_errno;
> + ereport(LOG,
> + (errmsg("could not fork %s process: %m", fork_data->type_desc)));
Previously the process type was translatable, now it is not
anymore. You'd probably have to solve that somehow, perhaps by
translating the type_desc once, when setting it.
> @@ -1113,7 +1113,7 @@ process_startup_options(Port *port, bool am_superuser)
> av = (char **) palloc(maxac * sizeof(char *));
> ac = 0;
>
> - av[ac++] = "postgres";
> + av[ac++] = pstrdup("postgres");
>
> pg_split_opts(av, &ac, port->cmdline_options);
Again, what's with all these conversions to dynamic memory allocations?
> /* Flags to tell if we are in an autovacuum process */
> -static bool am_autovacuum_launcher = false;
> -static bool am_autovacuum_worker = false;
> +bool am_autovacuum_launcher = false;
> +bool am_autovacuum_worker = false;
I'm against exposing a multitude of am_* variables for each process
type, especially globally. If we're building proper infrastructure
here, why do we need multiple variables, rather than exactly one
indicating the process type?
> @@ -347,88 +343,53 @@ static void avl_sigusr2_handler(SIGNAL_ARGS);
> static void avl_sigterm_handler(SIGNAL_ARGS);
> static void autovac_refresh_stats(void);
>
> -
> -
> /********************************************************************
> * AUTOVACUUM LAUNCHER CODE
> ********************************************************************/
Spurious whitespace changes (in a few other places too).
> +void
> +PrepAutoVacProcessFork(ForkProcData *autovac_fork)
> {
I'm really not in love with keeping "fork" in all these, given that
we're not actually going to fork in some cases. Why do we not just call
it subprocess and get rid of the "fork" and "forkexec" suffixes?
> +/*
> + * shmemSetup
> + *
> + * Helper function for a child to set up shmem before
> + * executing.
> + *
> + * aux_process - set to true if an auxiliary process.
> + */
> +static void
> +shmemSetup(bool aux_process)
> +{
> + /* Restore basic shared memory pointers */
> + InitShmemAccess(UsedShmemSegAddr);
> +
> + /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
> + if (aux_process)
> + InitAuxiliaryProcess();
> + else
> + InitProcess();
> +
> + /* Attach process to shared data structures */
> + CreateSharedMemoryAndSemaphores();
> +}
Not quite convinced this is a useful abstraction. I'm inclined to think
that this again should use something more like one array of
per-subprocess properties.
Imagine if we had an array like
typedef struct PgSubProcess
{
const char *name;
const char *translated_name;
bool needs_shmem;
bool needs_aux_proc;
bool needs_full_proc;
SubProcessEntryPoint entrypoint;
} PgSubProcess;
PgSubProcess process_types[] = {
{.name = "startup process", .needs_shmem = true, .needs_aux_proc = true, .entrypoint = StartupProcessMain},
{.name = "background writer", .needs_shmem = true, .needs_aux_proc = true, .entrypoint = BackgroundWriterMain},
{.name = "autovacuum launcher", .needs_shmem = true, .needs_aux_proc = true, .entrypoint = AutoVacLauncherMain},
{.name = "backend", .needs_shmem = true, .needs_full_proc = true, , .entrypoint = PostgresMain},
> From efc6f0531b71847c6500062b5a0fe43331a6ea7e Mon Sep 17 00:00:00 2001
> From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
> Date: Fri, 27 Sep 2019 19:13:53 -0400
> Subject: [PATCH 3/8] Add centralized stat collector
>
> ---
> src/backend/postmaster/pgstat.c | 88 ++++++---------------------
> src/backend/postmaster/postmaster.c | 5 +-
> src/include/pgstat.h | 4 +-
> src/include/postmaster/fork_process.h | 1 +
> 4 files changed, 26 insertions(+), 72 deletions(-)
>
> diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
> index 011076c3e3..73d953aedf 100644
> --- a/src/backend/postmaster/pgstat.c
> +++ b/src/backend/postmaster/pgstat.c
> @@ -280,10 +280,6 @@ static instr_time total_func_time;
> * Local function forward declarations
> * ----------
> */
> -#ifdef EXEC_BACKEND
> -static pid_t pgstat_forkexec(void);
> -#endif
> -
> NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
> static void pgstat_exit(SIGNAL_ARGS);
> static void pgstat_beshutdown_hook(int code, Datum arg);
> @@ -689,53 +685,26 @@ pgstat_reset_all(void)
> pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
> }
>
> -#ifdef EXEC_BACKEND
>
> /*
> - * pgstat_forkexec() -
> - *
> - * Format up the arglist for, then fork and exec, statistics collector process
> - */
> -static pid_t
> -pgstat_forkexec(void)
> -{
> - char *av[10];
> - int ac = 0;
> -
> - av[ac++] = "postgres";
> - av[ac++] = "--forkcol";
> - av[ac++] = NULL; /* filled in by postmaster_forkexec */
> -
> - av[ac] = NULL;
> - Assert(ac < lengthof(av));
> -
> - return postmaster_forkexec(ac, av);
> -}
> -#endif /* EXEC_BACKEND */
> -
> -
> -/*
> - * pgstat_start() -
> + * PrepPgstatCollectorFork
> *
> * Called from postmaster at startup or after an existing collector
> * died. Attempt to fire up a fresh statistics collector.
> *
> - * Returns PID of child process, or 0 if fail.
> - *
> - * Note: if fail, we will be called again from the postmaster main loop.
> */
> -int
> -pgstat_start(void)
> +void
> +PrepPgstatCollectorFork(ForkProcData *pgstat_fork)
> {
> + int ac = 0;
> time_t curtime;
> - pid_t pgStatPid;
>
> /*
> * Check that the socket is there, else pgstat_init failed and we can do
> * nothing useful.
> */
> if (pgStatSock == PGINVALID_SOCKET)
> - return 0;
> + return;
>
> /*
> * Do nothing if too soon since last collector start. This is a safety
> @@ -746,45 +715,20 @@ pgstat_start(void)
> curtime = time(NULL);
> if ((unsigned int) (curtime - last_pgstat_start_time) <
> (unsigned int) PGSTAT_RESTART_INTERVAL)
> - return 0;
> + return;
> last_pgstat_start_time = curtime;
>
> - /*
> - * Okay, fork off the collector.
> - */
> #ifdef EXEC_BACKEND
> - switch ((pgStatPid = pgstat_forkexec()))
> -#else
> - switch ((pgStatPid = fork_process()))
> -#endif
> - {
> - case -1:
> - ereport(LOG,
> - (errmsg("could not fork statistics collector: %m")));
> - return 0;
> -
> -#ifndef EXEC_BACKEND
> - case 0:
> - /* in postmaster child ... */
> - InitPostmasterChild();
> -
> - /* Close the postmaster's sockets */
> - ClosePostmasterPorts(false);
> -
> - /* Drop our connection to postmaster's shared memory, as well */
> - dsm_detach_all();
> - PGSharedMemoryDetach();
> -
> - PgstatCollectorMain(0, NULL);
> - break;
> + pgstat_fork->av[ac++] = pstrdup("postgres");
> + pgstat_fork->av[ac++] = pstrdup("--forkcol");
> + pgstat_fork->av[ac++] = NULL; /* filled in by postmaster_forkexec */
> #endif
>
> - default:
> - return (int) pgStatPid;
> - }
> + pgstat_fork->ac = ac;
> + Assert(pgstat_fork->ac < lengthof(*pgstat_fork->av));
>
> - /* shouldn't get here */
> - return 0;
> + pgstat_fork->type_desc = pstrdup("statistics collector");
> + pgstat_fork->child_main = PgstatCollectorMain;
> }
>
> void
> @@ -4425,12 +4369,16 @@ pgstat_send_bgwriter(void)
> * ----------
> */
> NON_EXEC_STATIC void
> -PgstatCollectorMain(int argc, char *argv[])
> +PgstatCollectorMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
> {
> int len;
> PgStat_Msg msg;
> int wr;
>
> + /* Drop our connection to postmaster's shared memory, as well */
> + dsm_detach_all();
> + PGSharedMemoryDetach();
> +
> /*
> * Ignore all signals usually bound to some action in the postmaster,
> * except SIGHUP and SIGQUIT. Note we don't need a SIGUSR1 handler to
> diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
> index 35cd1479b9..37a36387a3 100644
> --- a/src/backend/postmaster/postmaster.c
> +++ b/src/backend/postmaster/postmaster.c
> @@ -549,6 +549,7 @@ static void ShmemBackendArrayRemove(Backend *bn);
> #define StartWalReceiver() StartChildProcess(WalReceiverFork)
> #define StartAutoVacLauncher() StartChildProcess(AutoVacLauncherFork)
> #define StartAutoVacWorker() StartChildProcess(AutoVacWorkerFork)
> +#define pgstat_start() StartChildProcess(PgstatCollectorFork)
>
> /* Macros to check exit status of a child process */
> #define EXIT_STATUS_0(st) ((st) == 0)
> @@ -5459,7 +5460,9 @@ StartChildProcess(ForkProcType type)
> case AutoVacWorkerFork:
> PrepAutoVacProcessFork(fork_data);
> break;
> -
> + case PgstatCollectorFork:
> + PrepPgstatCollectorFork(fork_data);
> + break;
> default:
> break;
> }
> diff --git a/src/include/pgstat.h b/src/include/pgstat.h
> index fe076d823d..00e95caa60 100644
> --- a/src/include/pgstat.h
> +++ b/src/include/pgstat.h
> @@ -15,6 +15,7 @@
> #include "libpq/pqcomm.h"
> #include "port/atomics.h"
> #include "portability/instr_time.h"
> +#include "postmaster/fork_process.h"
> #include "postmaster/pgarch.h"
> #include "storage/proc.h"
> #include "utils/hsearch.h"
> @@ -1235,10 +1236,11 @@ extern Size BackendStatusShmemSize(void);
> extern void CreateSharedBackendStatus(void);
>
> extern void pgstat_init(void);
> -extern int pgstat_start(void);
> extern void pgstat_reset_all(void);
> extern void allow_immediate_pgstat_restart(void);
>
> +extern void PrepPgstatCollectorFork(ForkProcData *pgstat_fork);
> +
> #ifdef EXEC_BACKEND
> extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
> #endif
> diff --git a/src/include/postmaster/fork_process.h b/src/include/postmaster/fork_process.h
> index 1f319fc98f..16ca8be968 100644
> --- a/src/include/postmaster/fork_process.h
> +++ b/src/include/postmaster/fork_process.h
> @@ -26,6 +26,7 @@ typedef enum
> WalReceiverFork, /* end of Auxiliary Process Forks */
> AutoVacLauncherFork,
> AutoVacWorkerFork,
> + PgstatCollectorFork,
>
> NUMFORKPROCTYPES /* Must be last! */
> } ForkProcType;
> ---
> src/backend/postmaster/postmaster.c | 309 +++++++++++++-------------
> src/include/postmaster/fork_process.h | 5 +
> src/include/postmaster/postmaster.h | 1 -
> 3 files changed, 160 insertions(+), 155 deletions(-)
>
> diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
> index 8f862fcd64..b55cc4556d 100644
> --- a/src/backend/postmaster/postmaster.c
> +++ b/src/backend/postmaster/postmaster.c
> @@ -144,6 +144,7 @@ static Backend *ShmemBackendArray;
>
> BackgroundWorker *MyBgworkerEntry = NULL;
> RegisteredBgWorker *CurrentBgWorker = NULL;
> +static Backend *MyBackend;
> static int child_errno;
This is another global variable that strikes me as a bad idea. Whereas
MyBgworkerEntry is only set in the bgworker itself, you appear to set
MyBackend *in postmaster*:
> +/*
> + * PrepBackendFork
> + *
> + * Prepare a ForkProcType struct for starting a Backend.
> + * This does all prep related to av parameters and error messages.
> +*/
> +static void
> +PrepBackendFork(ForkProcData *backend_fork)
> +{
> + int ac = 0;
> +
> + /*
> + * Create backend data structure. Better before the fork() so we can
> + * handle failure cleanly.
> + */
> + MyBackend = (Backend *) malloc(sizeof(Backend));
> + if (!MyBackend)
> + {
> + ereport(LOG,
> + (errcode(ERRCODE_OUT_OF_MEMORY),
> + errmsg("out of memory")));
> + }
That's totally not ok, we'll constantly overwrite the variable set by
another process, and otherwise have now obsolete values in there. And
even worse, you leave it dangling in some cases:
> + /*
> + * Compute the cancel key that will be assigned to this backend. The
> + * backend will have its own copy in the forked-off process' value of
> + * MyCancelKey, so that it can transmit the key to the frontend.
> + */
> + if (!RandomCancelKey(&MyCancelKey))
> + {
> + free(MyBackend);
> + ereport(LOG,
> + (errcode(ERRCODE_INTERNAL_ERROR),
> + errmsg("could not generate random cancel key")));
> + }
Nor do I see where you're actually freeing that memory in postmaster, in
the successful case. Previously it was freed in postmaster in the
success case
> -#ifdef EXEC_BACKEND
> - pid = backend_forkexec(port);
> -#else /* !EXEC_BACKEND */
> - pid = fork_process();
> - if (pid == 0) /* child */
> - {
> - free(bn);
> -
but now it appears it's only freed in the failure case:
> +/*
> + * BackendFailFork
> + *
> + * Backend cleanup in case a failure occurs forking a new Backend.
> + */
> +static void BackendFailFork(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
> +{
> + if (!MyBackend->dead_end)
> + (void) ReleasePostmasterChildSlot(MyBackend->child_slot);
> + free(MyBackend);
> +
> + report_fork_failure_to_client(MyProcPort, child_errno);
> +}
> From 718c6598e9e02e1ce9ade99afb3c8948c67ef5ae Mon Sep 17 00:00:00 2001
> From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
> Date: Tue, 19 Feb 2019 15:29:33 +0000
> Subject: [PATCH 8/8] Add a hook to allow extensions to set worker metadata
>
> This patch implements a hook that allows extensions to modify a worker
> process' metadata in backends.
>
> Additional work done by Yuli Khodorkovskiy <yuli@crunchydata.com>
> Original discussion here: https://www.postgresql.org/message-id/flat/CAMN686FE0OdZKp9YPO%3DhtC6LnA6aW4r-%2Bjq%3D3Q5RAoFQgW8EtA...
> ---
> src/backend/postmaster/fork_process.c | 11 +++++++++++
> src/include/postmaster/fork_process.h | 4 ++++
> 2 files changed, 15 insertions(+)
>
> diff --git a/src/backend/postmaster/fork_process.c b/src/backend/postmaster/fork_process.c
> index a9138f589e..964c04d846 100644
> --- a/src/backend/postmaster/fork_process.c
> +++ b/src/backend/postmaster/fork_process.c
> @@ -21,6 +21,14 @@
> #include <openssl/rand.h>
> #endif
>
> +/*
> + * This hook allows plugins to get control of the child process following
> + * a successful fork_process(). It can be used to perform some action in
> + * extensions to set metadata in backends (including special backends) upon
> + * setting of the session user.
> + */
> +ForkProcess_post_hook_type ForkProcess_post_hook = NULL;
> +
> #ifndef WIN32
> /*
> * Wrapper for fork(). Return values are the same as those for fork():
> @@ -113,6 +121,9 @@ fork_process(void)
> #ifdef USE_OPENSSL
> RAND_cleanup();
> #endif
> +
> + if (ForkProcess_post_hook)
> + (*ForkProcess_post_hook) ();
> }
> return result;
> diff --git a/src/include/postmaster/fork_process.h b/src/include/postmaster/fork_process.h
> index 631a2113b5..d756fcead7 100644
> --- a/src/include/postmaster/fork_process.h
> +++ b/src/include/postmaster/fork_process.h
> @@ -58,4 +58,8 @@ extern pid_t fork_process(void);
> extern pid_t postmaster_forkexec(ForkProcData *fork_data);
> #endif
>
> +/* Hook for plugins to get control after a successful fork_process() */
> +typedef void (*ForkProcess_post_hook_type) ();
> +extern PGDLLIMPORT ForkProcess_post_hook_type ForkProcess_post_hook;
> +
> #endif /* FORK_PROCESS_H */
> --
> 2.23.0
>
Why do we want libraries to allow to hook into processes like the
startup process etc?
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
@ 2019-10-03 18:33 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 38+ messages in thread
From: Mike Palmiotto @ 2019-10-03 18:33 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Alvaro Herrera <alvherre@2ndquadrant.com>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Thu, Oct 3, 2019 at 1:31 PM Andres Freund <andres@anarazel.de> wrote:
>
> Hi,
>
> On 2019-09-30 15:13:18 -0400, Mike Palmiotto wrote:
> > Attached is the reworked and rebased patch set. I put the hook on top
> > and a separate commit for each process type. Note that avworker and
> > avlauncher were intentionally left together. Let me know if you think
> > those should be split out as well.
>
> I've not looked at this before, nor the thread. Just jumping here
> because of the reference to this patch you made in another thread...
>
> In short, I think this is far from ready, nor am I sure this is going in
> the right direction. There's a bit more detail about where I think this
> ought to go interspersed below (in particular search for PgSubProcess,
> EXEC_BACKEND).
Great, thanks for the review!
>
>
>
> > From 205ea86dde898f7edac327d46b2b43b04721aadc Mon Sep 17 00:00:00 2001
> > From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
> > Date: Mon, 30 Sep 2019 14:42:53 -0400
> > Subject: [PATCH 7/8] Add backends to process centralization
>
> .oO(out of order attached patches, how fun). 7, 8, 6, 5, 4, 2, 3, 1...
Yeah, hm. Not really sure why/how that happened.
>
>
>
> > From 2a3a35f2e80cb2badcb0efbce1bad2484e364b7b Mon Sep 17 00:00:00 2001
> > From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
> > Date: Fri, 27 Sep 2019 12:28:19 -0400
> > Subject: [PATCH 1/8] Add ForkProcType infrastructure
>
> A better explanation would be nice. It's hard to review non-trivial
> patches that have little explanation as to what they're
> achieving. Especially when there's some unrelated seeming changes.
I should have maintained the initial patch description from the last
cut -- there is a lot more detail there. I will pull that back in
after addressing the other concerns. :)
>
>
> > diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
> > index 9238fbe98d..9f3dad1c6d 100644
> > --- a/src/backend/bootstrap/bootstrap.c
> > +++ b/src/backend/bootstrap/bootstrap.c
> > @@ -70,6 +70,7 @@ static void cleanup(void);
> > */
> >
> > AuxProcType MyAuxProcType = NotAnAuxProcess; /* declared in miscadmin.h */
> > +ForkProcType MyForkProcType = NoForkProcess; /* declared in fork_process.h */
> >
> > Relation boot_reldesc; /* current relation descriptor
> > */
>
> IMO, before this stuff gets massaged meaningfully, we ought to move code
> like this (including AuxiliaryProcessMain etc) somewhere where it makes
> sense. If we're going to rejigger things, it ought to be towards a more
> understandable architecture, rather than keeping weird quirks around.
Sounds good. I'll address this in the next round.
>
> I'm inclined to think that this better also include properly
> centralizing a good portion of the EXEC_BACKEND code. Building
> infrastructure that's supposed to be long-lived with that being left as
> is just seems like recipe for yet another odd framework.
+1
>
>
> > @@ -538,11 +539,11 @@ static void ShmemBackendArrayAdd(Backend *bn);
> > static void ShmemBackendArrayRemove(Backend *bn);
> > #endif /* EXEC_BACKEND */
> >
> > -#define StartupDataBase() StartChildProcess(StartupProcess)
> > -#define StartBackgroundWriter() StartChildProcess(BgWriterProcess)
> > -#define StartCheckpointer() StartChildProcess(CheckpointerProcess)
> > -#define StartWalWriter() StartChildProcess(WalWriterProcess)
> > -#define StartWalReceiver() StartChildProcess(WalReceiverProcess)
> > +#define StartupDataBase() StartChildProcess(StartupFork)
> > +#define StartBackgroundWriter() StartChildProcess(BgWriterFork)
> > +#define StartCheckpointer() StartChildProcess(CheckpointerFork)
> > +#define StartWalWriter() StartChildProcess(WalWriterFork)
> > +#define StartWalReceiver() StartChildProcess(WalReceiverFork)
>
> FWIW, I'd just rip this out, it adds exactly nothing but one more place
> to change.
+1
>
>
> > +/*
> > + * PrepAuxProcessFork
> > + *
> > + * Prepare a ForkProcType struct for the auxiliary process specified by
>
> ForkProcData, not ForkProcType
>
>
> > + * AuxProcType. This does all prep related to av parameters and error messages.
> > + */
> > +static void
> > +PrepAuxProcessFork(ForkProcData *aux_fork)
> > +{
> > + int ac = 0;
> > +
> > + /*
> > + * Set up command-line arguments for subprocess
> > + */
> > + aux_fork->av[ac++] = pstrdup("postgres");
>
> There's no documentation as to what 'av' is actually supposed to mean. I
> assume auxvector or such, but if so that's neither obvious, nor
> necessarily good.
Got it, better variable names here.
>
>
> > +#ifdef EXEC_BACKEND
> > + aux_fork->av[ac++] = pstrdup("--forkboot");
> > + aux_fork->av[ac++] = NULL; /* filled in by postmaster_forkexec */
> > +#endif
>
> What's the point of pstrdup()ing constant strings here? Afaict you're
> not freeing them ever?
The idea was supposed to be that the prep work lives up until the fork
happens and then is freed en masse with the rest of the MemoryContext.
Perhaps there was some oversight here. I'll revisit this and others in
the next pass.
>
>
> > + aux_fork->av[ac++] = psprintf("-x%d", MyForkProcType);
> > +
> > + aux_fork->av[ac] = NULL;
> > + Assert(ac < lengthof(*aux_fork->av));
>
> Unless I miss something, the lengthof here doesn't do anythign
> useful. ForkProcData->av is defined as:
>
> > +typedef struct ForkProcData
> > +{
> > + char **av;
>
> which I think means you're computing sizeof(char*)/sizeof(char)?
I think this might be a leftover assertion from the previous
infrastructure. Agreed that this is probably useless.
>
>
>
> > + aux_fork->ac = ac;
> > + switch (MyForkProcType)
> > + {
> > + case StartupProcess:
> > + aux_fork->type_desc = pstrdup("startup");
> > + break;
> > + case BgWriterProcess:
> > + aux_fork->type_desc = pstrdup("background writer");
> > + break;
> > + case CheckpointerProcess:
> > + aux_fork->type_desc = pstrdup("checkpointer");
> > + break;
> > + case WalWriterProcess:
> > + aux_fork->type_desc = pstrdup("WAL writer");
> > + break;
> > + case WalReceiverProcess:
> > + aux_fork->type_desc = pstrdup("WAL receiver");
> > + break;
> > + default:
> > + aux_fork->type_desc = pstrdup("child");
> > + break;
> > + }
> > +
> > + aux_fork->child_main = AuxiliaryProcessMain;
>
> I'm really not in love in having details like the descriptors of
> processes, the types of processes, the mapping from MyForkProcType to
> AuxProcType in multiple places. I feel this is adding even more
> confusion to this, rather than reducing it.
>
> I feel like we should define the properties of the different types of
> processes in *one* place, rather multiple. Define one array of
> structs that contains information about the different types of
> processes, rather than distributing that knowledge all over.
Agreed here. Thanks for the suggestion.
>
>
> > @@ -4477,11 +4530,11 @@ BackendRun(Port *port)
> > pid_t
> > postmaster_forkexec(int argc, char *argv[])
> > {
> > - Port port;
> > -
> > /* This entry point passes dummy values for the Port variables */
> > - memset(&port, 0, sizeof(port));
> > - return internal_forkexec(argc, argv, &port);
> > + if (!ConnProcPort)
> > + ConnProcPort = palloc0(sizeof(*ConnProcPort));
> > +
> > + return internal_forkexec(argc, argv);
> > }
>
> What's the point of creating a dummy ConnProcPort? And when is this
> being pfreed? I mean previously this at least used a stack variable that
> was automatically being released by the time postmaster_forkexec
> returned, now it sure looks like a permanent leak to me.
Will address this in the next pass.
>
> Also, why is it a good idea to rely *more* on data being implicitly
> passed by stuffing things into global variables? That strikes me as
It is certainly not. This will be addressed in the next pass, thank you!
>
>
> > /* in parent, fork failed */
> > - int save_errno = errno;
> > + int save_errno = errno;
> >
> > - errno = save_errno;
> > - switch (type)
> > - {
> > - case StartupProcess:
> > - ereport(LOG,
> > - (errmsg("could not fork startup process: %m")));
> > - break;
> > - case BgWriterProcess:
> > - ereport(LOG,
> > - (errmsg("could not fork background writer process: %m")));
> > - break;
> > - case CheckpointerProcess:
> > - ereport(LOG,
> > - (errmsg("could not fork checkpointer process: %m")));
> > - break;
> > - case WalWriterProcess:
> > - ereport(LOG,
> > - (errmsg("could not fork WAL writer process: %m")));
> > - break;
> > - case WalReceiverProcess:
> > - ereport(LOG,
> > - (errmsg("could not fork WAL receiver process: %m")));
> > - break;
> > - default:
> > - ereport(LOG,
> > - (errmsg("could not fork process: %m")));
> > - break;
> > - }
> > + errno = save_errno;
> > + ereport(LOG,
> > + (errmsg("could not fork %s process: %m", fork_data->type_desc)));
>
> Previously the process type was translatable, now it is not
> anymore. You'd probably have to solve that somehow, perhaps by
> translating the type_desc once, when setting it.
+1
>
>
> > @@ -1113,7 +1113,7 @@ process_startup_options(Port *port, bool am_superuser)
> > av = (char **) palloc(maxac * sizeof(char *));
> > ac = 0;
> >
> > - av[ac++] = "postgres";
> > + av[ac++] = pstrdup("postgres");
> >
> > pg_split_opts(av, &ac, port->cmdline_options);
>
> Again, what's with all these conversions to dynamic memory allocations?
>
>
>
> > /* Flags to tell if we are in an autovacuum process */
> > -static bool am_autovacuum_launcher = false;
> > -static bool am_autovacuum_worker = false;
> > +bool am_autovacuum_launcher = false;
> > +bool am_autovacuum_worker = false;
>
> I'm against exposing a multitude of am_* variables for each process
> type, especially globally. If we're building proper infrastructure
> here, why do we need multiple variables, rather than exactly one
> indicating the process type?
Noted.
>
>
>
> > @@ -347,88 +343,53 @@ static void avl_sigusr2_handler(SIGNAL_ARGS);
> > static void avl_sigterm_handler(SIGNAL_ARGS);
> > static void autovac_refresh_stats(void);
> >
> > -
> > -
> > /********************************************************************
> > * AUTOVACUUM LAUNCHER CODE
> > ********************************************************************/
>
> Spurious whitespace changes (in a few other places too).
Noted.
>
>
> > +void
> > +PrepAutoVacProcessFork(ForkProcData *autovac_fork)
> > {
>
> I'm really not in love with keeping "fork" in all these, given that
> we're not actually going to fork in some cases. Why do we not just call
> it subprocess and get rid of the "fork" and "forkexec" suffixes?
Sounds good to me.
>
>
>
>
> > +/*
> > + * shmemSetup
> > + *
> > + * Helper function for a child to set up shmem before
> > + * executing.
> > + *
> > + * aux_process - set to true if an auxiliary process.
> > + */
> > +static void
> > +shmemSetup(bool aux_process)
> > +{
> > + /* Restore basic shared memory pointers */
> > + InitShmemAccess(UsedShmemSegAddr);
> > +
> > + /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
> > + if (aux_process)
> > + InitAuxiliaryProcess();
> > + else
> > + InitProcess();
> > +
> > + /* Attach process to shared data structures */
> > + CreateSharedMemoryAndSemaphores();
> > +}
>
> Not quite convinced this is a useful abstraction. I'm inclined to think
> that this again should use something more like one array of
> per-subprocess properties.
>
>
> Imagine if we had an array like
>
> typedef struct PgSubProcess
> {
> const char *name;
> const char *translated_name;
> bool needs_shmem;
> bool needs_aux_proc;
> bool needs_full_proc;
> SubProcessEntryPoint entrypoint;
> } PgSubProcess;
>
> PgSubProcess process_types[] = {
> {.name = "startup process", .needs_shmem = true, .needs_aux_proc = true, .entrypoint = StartupProcessMain},
> {.name = "background writer", .needs_shmem = true, .needs_aux_proc = true, .entrypoint = BackgroundWriterMain},
> {.name = "autovacuum launcher", .needs_shmem = true, .needs_aux_proc = true, .entrypoint = AutoVacLauncherMain},
> {.name = "backend", .needs_shmem = true, .needs_full_proc = true, , .entrypoint = PostgresMain},
Great idea, thanks!
>
>
> > From efc6f0531b71847c6500062b5a0fe43331a6ea7e Mon Sep 17 00:00:00 2001
> > From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
> > Date: Fri, 27 Sep 2019 19:13:53 -0400
> > Subject: [PATCH 3/8] Add centralized stat collector
> >
> > ---
> > src/backend/postmaster/pgstat.c | 88 ++++++---------------------
> > src/backend/postmaster/postmaster.c | 5 +-
> > src/include/pgstat.h | 4 +-
> > src/include/postmaster/fork_process.h | 1 +
> > 4 files changed, 26 insertions(+), 72 deletions(-)
> >
> > diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
> > index 011076c3e3..73d953aedf 100644
> > --- a/src/backend/postmaster/pgstat.c
> > +++ b/src/backend/postmaster/pgstat.c
> > @@ -280,10 +280,6 @@ static instr_time total_func_time;
> > * Local function forward declarations
> > * ----------
> > */
> > -#ifdef EXEC_BACKEND
> > -static pid_t pgstat_forkexec(void);
> > -#endif
> > -
> > NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
> > static void pgstat_exit(SIGNAL_ARGS);
> > static void pgstat_beshutdown_hook(int code, Datum arg);
> > @@ -689,53 +685,26 @@ pgstat_reset_all(void)
> > pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
> > }
> >
> > -#ifdef EXEC_BACKEND
> >
> > /*
> > - * pgstat_forkexec() -
> > - *
> > - * Format up the arglist for, then fork and exec, statistics collector process
> > - */
> > -static pid_t
> > -pgstat_forkexec(void)
> > -{
> > - char *av[10];
> > - int ac = 0;
> > -
> > - av[ac++] = "postgres";
> > - av[ac++] = "--forkcol";
> > - av[ac++] = NULL; /* filled in by postmaster_forkexec */
> > -
> > - av[ac] = NULL;
> > - Assert(ac < lengthof(av));
> > -
> > - return postmaster_forkexec(ac, av);
> > -}
> > -#endif /* EXEC_BACKEND */
> > -
> > -
> > -/*
> > - * pgstat_start() -
> > + * PrepPgstatCollectorFork
> > *
> > * Called from postmaster at startup or after an existing collector
> > * died. Attempt to fire up a fresh statistics collector.
> > *
> > - * Returns PID of child process, or 0 if fail.
> > - *
> > - * Note: if fail, we will be called again from the postmaster main loop.
> > */
> > -int
> > -pgstat_start(void)
> > +void
> > +PrepPgstatCollectorFork(ForkProcData *pgstat_fork)
> > {
> > + int ac = 0;
> > time_t curtime;
> > - pid_t pgStatPid;
> >
> > /*
> > * Check that the socket is there, else pgstat_init failed and we can do
> > * nothing useful.
> > */
> > if (pgStatSock == PGINVALID_SOCKET)
> > - return 0;
> > + return;
> >
> > /*
> > * Do nothing if too soon since last collector start. This is a safety
> > @@ -746,45 +715,20 @@ pgstat_start(void)
> > curtime = time(NULL);
> > if ((unsigned int) (curtime - last_pgstat_start_time) <
> > (unsigned int) PGSTAT_RESTART_INTERVAL)
> > - return 0;
> > + return;
> > last_pgstat_start_time = curtime;
> >
> > - /*
> > - * Okay, fork off the collector.
> > - */
> > #ifdef EXEC_BACKEND
> > - switch ((pgStatPid = pgstat_forkexec()))
> > -#else
> > - switch ((pgStatPid = fork_process()))
> > -#endif
> > - {
> > - case -1:
> > - ereport(LOG,
> > - (errmsg("could not fork statistics collector: %m")));
> > - return 0;
> > -
> > -#ifndef EXEC_BACKEND
> > - case 0:
> > - /* in postmaster child ... */
> > - InitPostmasterChild();
> > -
> > - /* Close the postmaster's sockets */
> > - ClosePostmasterPorts(false);
> > -
> > - /* Drop our connection to postmaster's shared memory, as well */
> > - dsm_detach_all();
> > - PGSharedMemoryDetach();
> > -
> > - PgstatCollectorMain(0, NULL);
> > - break;
> > + pgstat_fork->av[ac++] = pstrdup("postgres");
> > + pgstat_fork->av[ac++] = pstrdup("--forkcol");
> > + pgstat_fork->av[ac++] = NULL; /* filled in by postmaster_forkexec */
> > #endif
> >
> > - default:
> > - return (int) pgStatPid;
> > - }
> > + pgstat_fork->ac = ac;
> > + Assert(pgstat_fork->ac < lengthof(*pgstat_fork->av));
> >
> > - /* shouldn't get here */
> > - return 0;
> > + pgstat_fork->type_desc = pstrdup("statistics collector");
> > + pgstat_fork->child_main = PgstatCollectorMain;
> > }
> >
> > void
> > @@ -4425,12 +4369,16 @@ pgstat_send_bgwriter(void)
> > * ----------
> > */
> > NON_EXEC_STATIC void
> > -PgstatCollectorMain(int argc, char *argv[])
> > +PgstatCollectorMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
> > {
> > int len;
> > PgStat_Msg msg;
> > int wr;
> >
> > + /* Drop our connection to postmaster's shared memory, as well */
> > + dsm_detach_all();
> > + PGSharedMemoryDetach();
> > +
> > /*
> > * Ignore all signals usually bound to some action in the postmaster,
> > * except SIGHUP and SIGQUIT. Note we don't need a SIGUSR1 handler to
> > diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
> > index 35cd1479b9..37a36387a3 100644
> > --- a/src/backend/postmaster/postmaster.c
> > +++ b/src/backend/postmaster/postmaster.c
> > @@ -549,6 +549,7 @@ static void ShmemBackendArrayRemove(Backend *bn);
> > #define StartWalReceiver() StartChildProcess(WalReceiverFork)
> > #define StartAutoVacLauncher() StartChildProcess(AutoVacLauncherFork)
> > #define StartAutoVacWorker() StartChildProcess(AutoVacWorkerFork)
> > +#define pgstat_start() StartChildProcess(PgstatCollectorFork)
> >
> > /* Macros to check exit status of a child process */
> > #define EXIT_STATUS_0(st) ((st) == 0)
> > @@ -5459,7 +5460,9 @@ StartChildProcess(ForkProcType type)
> > case AutoVacWorkerFork:
> > PrepAutoVacProcessFork(fork_data);
> > break;
> > -
> > + case PgstatCollectorFork:
> > + PrepPgstatCollectorFork(fork_data);
> > + break;
> > default:
> > break;
> > }
> > diff --git a/src/include/pgstat.h b/src/include/pgstat.h
> > index fe076d823d..00e95caa60 100644
> > --- a/src/include/pgstat.h
> > +++ b/src/include/pgstat.h
> > @@ -15,6 +15,7 @@
> > #include "libpq/pqcomm.h"
> > #include "port/atomics.h"
> > #include "portability/instr_time.h"
> > +#include "postmaster/fork_process.h"
> > #include "postmaster/pgarch.h"
> > #include "storage/proc.h"
> > #include "utils/hsearch.h"
> > @@ -1235,10 +1236,11 @@ extern Size BackendStatusShmemSize(void);
> > extern void CreateSharedBackendStatus(void);
> >
> > extern void pgstat_init(void);
> > -extern int pgstat_start(void);
> > extern void pgstat_reset_all(void);
> > extern void allow_immediate_pgstat_restart(void);
> >
> > +extern void PrepPgstatCollectorFork(ForkProcData *pgstat_fork);
> > +
> > #ifdef EXEC_BACKEND
> > extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
> > #endif
> > diff --git a/src/include/postmaster/fork_process.h b/src/include/postmaster/fork_process.h
> > index 1f319fc98f..16ca8be968 100644
> > --- a/src/include/postmaster/fork_process.h
> > +++ b/src/include/postmaster/fork_process.h
> > @@ -26,6 +26,7 @@ typedef enum
> > WalReceiverFork, /* end of Auxiliary Process Forks */
> > AutoVacLauncherFork,
> > AutoVacWorkerFork,
> > + PgstatCollectorFork,
> >
> > NUMFORKPROCTYPES /* Must be last! */
> > } ForkProcType;
>
>
>
>
> > ---
> > src/backend/postmaster/postmaster.c | 309 +++++++++++++-------------
> > src/include/postmaster/fork_process.h | 5 +
> > src/include/postmaster/postmaster.h | 1 -
> > 3 files changed, 160 insertions(+), 155 deletions(-)
> >
> > diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
> > index 8f862fcd64..b55cc4556d 100644
> > --- a/src/backend/postmaster/postmaster.c
> > +++ b/src/backend/postmaster/postmaster.c
> > @@ -144,6 +144,7 @@ static Backend *ShmemBackendArray;
> >
> > BackgroundWorker *MyBgworkerEntry = NULL;
> > RegisteredBgWorker *CurrentBgWorker = NULL;
> > +static Backend *MyBackend;
> > static int child_errno;
>
> This is another global variable that strikes me as a bad idea. Whereas
> MyBgworkerEntry is only set in the bgworker itself, you appear to set
> MyBackend *in postmaster*:
Okay, will address this in the next pass.
>
> > +/*
> > + * PrepBackendFork
> > + *
> > + * Prepare a ForkProcType struct for starting a Backend.
> > + * This does all prep related to av parameters and error messages.
> > +*/
> > +static void
> > +PrepBackendFork(ForkProcData *backend_fork)
> > +{
> > + int ac = 0;
> > +
> > + /*
> > + * Create backend data structure. Better before the fork() so we can
> > + * handle failure cleanly.
> > + */
> > + MyBackend = (Backend *) malloc(sizeof(Backend));
> > + if (!MyBackend)
> > + {
> > + ereport(LOG,
> > + (errcode(ERRCODE_OUT_OF_MEMORY),
> > + errmsg("out of memory")));
> > + }
>
> That's totally not ok, we'll constantly overwrite the variable set by
> another process, and otherwise have now obsolete values in there. And
> even worse, you leave it dangling in some cases:
Got it. Will address in the next pass.
>
> > + /*
> > + * Compute the cancel key that will be assigned to this backend. The
> > + * backend will have its own copy in the forked-off process' value of
> > + * MyCancelKey, so that it can transmit the key to the frontend.
> > + */
> > + if (!RandomCancelKey(&MyCancelKey))
> > + {
> > + free(MyBackend);
> > + ereport(LOG,
> > + (errcode(ERRCODE_INTERNAL_ERROR),
> > + errmsg("could not generate random cancel key")));
> > + }
>
>
> Nor do I see where you're actually freeing that memory in postmaster, in
> the successful case. Previously it was freed in postmaster in the
> success case
>
> > -#ifdef EXEC_BACKEND
> > - pid = backend_forkexec(port);
> > -#else /* !EXEC_BACKEND */
> > - pid = fork_process();
> > - if (pid == 0) /* child */
> > - {
> > - free(bn);
> > -
>
> but now it appears it's only freed in the failure case:
Good catch. Will fix.
>
> > +/*
> > + * BackendFailFork
> > + *
> > + * Backend cleanup in case a failure occurs forking a new Backend.
> > + */
> > +static void BackendFailFork(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
> > +{
> > + if (!MyBackend->dead_end)
> > + (void) ReleasePostmasterChildSlot(MyBackend->child_slot);
> > + free(MyBackend);
> > +
> > + report_fork_failure_to_client(MyProcPort, child_errno);
> > +}
>
>
>
> > From 718c6598e9e02e1ce9ade99afb3c8948c67ef5ae Mon Sep 17 00:00:00 2001
> > From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
> > Date: Tue, 19 Feb 2019 15:29:33 +0000
> > Subject: [PATCH 8/8] Add a hook to allow extensions to set worker metadata
> >
> > This patch implements a hook that allows extensions to modify a worker
> > process' metadata in backends.
> >
> > Additional work done by Yuli Khodorkovskiy <yuli@crunchydata.com>
> > Original discussion here: https://www.postgresql.org/message-id/flat/CAMN686FE0OdZKp9YPO%3DhtC6LnA6aW4r-%2Bjq%3D3Q5RAoFQgW8EtA...
> > ---
> > src/backend/postmaster/fork_process.c | 11 +++++++++++
> > src/include/postmaster/fork_process.h | 4 ++++
> > 2 files changed, 15 insertions(+)
> >
> > diff --git a/src/backend/postmaster/fork_process.c b/src/backend/postmaster/fork_process.c
> > index a9138f589e..964c04d846 100644
> > --- a/src/backend/postmaster/fork_process.c
> > +++ b/src/backend/postmaster/fork_process.c
> > @@ -21,6 +21,14 @@
> > #include <openssl/rand.h>
> > #endif
> >
> > +/*
> > + * This hook allows plugins to get control of the child process following
> > + * a successful fork_process(). It can be used to perform some action in
> > + * extensions to set metadata in backends (including special backends) upon
> > + * setting of the session user.
> > + */
> > +ForkProcess_post_hook_type ForkProcess_post_hook = NULL;
> > +
> > #ifndef WIN32
> > /*
> > * Wrapper for fork(). Return values are the same as those for fork():
> > @@ -113,6 +121,9 @@ fork_process(void)
> > #ifdef USE_OPENSSL
> > RAND_cleanup();
> > #endif
> > +
> > + if (ForkProcess_post_hook)
> > + (*ForkProcess_post_hook) ();
> > }
>
>
> > return result;
> > diff --git a/src/include/postmaster/fork_process.h b/src/include/postmaster/fork_process.h
> > index 631a2113b5..d756fcead7 100644
> > --- a/src/include/postmaster/fork_process.h
> > +++ b/src/include/postmaster/fork_process.h
> > @@ -58,4 +58,8 @@ extern pid_t fork_process(void);
> > extern pid_t postmaster_forkexec(ForkProcData *fork_data);
> > #endif
> >
> > +/* Hook for plugins to get control after a successful fork_process() */
> > +typedef void (*ForkProcess_post_hook_type) ();
> > +extern PGDLLIMPORT ForkProcess_post_hook_type ForkProcess_post_hook;
> > +
> > #endif /* FORK_PROCESS_H */
> > --
> > 2.23.0
> >
>
> Why do we want libraries to allow to hook into processes like the
> startup process etc?
There are a number of OS-level process manipulations that this could
afford you as an extension developer. For instance, you could roll
your own seccomp implementation (to limit syscalls per-process-type,
perhaps?), perform some setcap magic, or some other security-related
magic.
Thanks for your patience with this patchset. I understand it's still
very immature. Hopefully all of the concerns raised will be adjusted
in the follow-up patches.
--
Mike Palmiotto
https://crunchydata.com
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
@ 2019-10-03 18:39 ` Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 38+ messages in thread
From: Andres Freund @ 2019-10-03 18:39 UTC (permalink / raw)
To: Mike Palmiotto <mike.palmiotto@crunchydata.com>; +Cc: Alvaro Herrera <alvherre@2ndquadrant.com>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
Hi,
On 2019-10-03 14:33:26 -0400, Mike Palmiotto wrote:
> > > +#ifdef EXEC_BACKEND
> > > + aux_fork->av[ac++] = pstrdup("--forkboot");
> > > + aux_fork->av[ac++] = NULL; /* filled in by postmaster_forkexec */
> > > +#endif
> >
> > What's the point of pstrdup()ing constant strings here? Afaict you're
> > not freeing them ever?
>
> The idea was supposed to be that the prep work lives up until the fork
> happens and then is freed en masse with the rest of the MemoryContext.
> Perhaps there was some oversight here. I'll revisit this and others in
> the next pass.
Well, two things:
1) That'd *NEVER* be more efficient that just referencing constant
strings. Unless you pfree() the variable, and there sometimes can be
constant, and sometimes non-constant strings, there is simply no
reason to ever pstrdup a constant string.
2) Which context are you talking about? Are you thinking of? A forked
process doing MemoryContextDelete(PostmasterContext); doesn't free
that memory in postmaster.
> > > @@ -58,4 +58,8 @@ extern pid_t fork_process(void);
> > > extern pid_t postmaster_forkexec(ForkProcData *fork_data);
> > > #endif
> > >
> > > +/* Hook for plugins to get control after a successful fork_process() */
> > > +typedef void (*ForkProcess_post_hook_type) ();
> > > +extern PGDLLIMPORT ForkProcess_post_hook_type ForkProcess_post_hook;
> > > +
> > > #endif /* FORK_PROCESS_H */
> > > --
> > > 2.23.0
> > >
> >
> > Why do we want libraries to allow to hook into processes like the
> > startup process etc?
>
> There are a number of OS-level process manipulations that this could
> afford you as an extension developer. For instance, you could roll
> your own seccomp implementation (to limit syscalls per-process-type,
> perhaps?), perform some setcap magic, or some other security-related
> magic.
Color me unconvinced.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
@ 2019-12-01 02:15 ` Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
0 siblings, 1 reply; 38+ messages in thread
From: Michael Paquier @ 2019-12-01 02:15 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Mike Palmiotto <mike.palmiotto@crunchydata.com>; Alvaro Herrera <alvherre@2ndquadrant.com>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Thu, Oct 03, 2019 at 11:39:37AM -0700, Andres Freund wrote:
> Color me unconvinced.
The latest comments of the thread have not been addressed yet. so I am
marking the patch as returned with feedback. If you think that's not
correct, please feel free to update the status of the patch. If you
do so, please provide at the same time a rebased version of the patch,
as it is failing to apply on HEAD.
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
@ 2020-03-01 02:51 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-02 14:53 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
0 siblings, 1 reply; 38+ messages in thread
From: Mike Palmiotto @ 2020-03-01 02:51 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: Andres Freund <andres@anarazel.de>; Alvaro Herrera <alvherre@2ndquadrant.com>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Sat, Nov 30, 2019 at 9:15 PM Michael Paquier <michael@paquier.xyz> wrote:
>
> On Thu, Oct 03, 2019 at 11:39:37AM -0700, Andres Freund wrote:
> > Color me unconvinced.
>
> The latest comments of the thread have not been addressed yet. so I am
> marking the patch as returned with feedback. If you think that's not
> correct, please feel free to update the status of the patch. If you
> do so, please provide at the same time a rebased version of the patch,
> as it is failing to apply on HEAD.
I've finally had some time to update the patchset with an attempt at
addressing Andres' concerns. Note that the current spin has a bug
which does not allow syslogger to run properly. Additionally, it is
squashed into one patch again. I intend to split the patch out into
separate functional patches and fix the syslogger issue, but wanted to
get this on the ticket for the open CommitFest before it closes. I'll
go ahead and re-add it and will be pushing updates as I have them.
I intend to give this patchset the time it deserves, so will be much
more responsive this go-around.
Thanks,
--
Mike Palmiotto
https://crunchydata.com
Attachments: download all (2 files, .tar.gz)
0002-Add-a-hook-to-allow-extensions-to-set-worker-metadat.patch (text/x-patch, 2.1K) download
From 34feb7399fe218eab0da851032299f81e7ad6689 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 19 Feb 2019 15:29:33 +0000
Subject: [PATCH 2/2] Add a hook to allow extensions to set worker metadata
This patch implements a hook that allows extensions to modify a worker
process' metadata in backends.
Additional work done by Yuli Khodorkovskiy <yuli@crunchydata.com>
Original discussion here: https://www.postgresql.org/message-id/flat/CAMN686FE0OdZKp9YPO%3DhtC6LnA6aW4r-%2Bjq%3D3Q5RAoFQgW8EtA%40mail.gmail.com
---
src/backend/postmaster/fork_process.c | 11 +++++++++++
src/include/postmaster/fork_process.h | 4 ++++
2 files changed, 15 insertions(+)
diff --git a/src/backend/postmaster/fork_process.c b/src/backend/postmaster/fork_process.c
index def3cee37e..e392f5207e 100644
--- a/src/backend/postmaster/fork_process.c
+++ b/src/backend/postmaster/fork_process.c
@@ -22,6 +22,14 @@
#include "postmaster/fork_process.h"
+/*
+ * This hook allows plugins to get control of the child process following
+ * a successful fork_process(). It can be used to perform some action in
+ * extensions to set metadata in backends (including special backends) upon
+ * setting of the session user.
+ */
+ForkProcess_post_hook_type ForkProcess_post_hook = NULL;
+
#ifndef WIN32
/*
* Wrapper for fork(). Return values are the same as those for fork():
@@ -114,6 +122,9 @@ fork_process(void)
#ifdef USE_OPENSSL
RAND_cleanup();
#endif
+
+ if (ForkProcess_post_hook)
+ (*ForkProcess_post_hook) ();
}
return result;
diff --git a/src/include/postmaster/fork_process.h b/src/include/postmaster/fork_process.h
index a42f859a08..b4d1560b72 100644
--- a/src/include/postmaster/fork_process.h
+++ b/src/include/postmaster/fork_process.h
@@ -20,4 +20,8 @@ extern pid_t fork_process(void);
extern pid_t postmaster_forkexec(int argc, char *argvp[]);
#endif
+/* Hook for plugins to get control after a successful fork_process() */
+typedef void (*ForkProcess_post_hook_type) ();
+extern PGDLLIMPORT ForkProcess_post_hook_type ForkProcess_post_hook;
+
#endif /* FORK_PROCESS_H */
--
2.21.0
0001-Introduce-subprocess-infrastructure.patch (text/x-patch, 86.6K) download
From 6fb91ba335153882d16d2dff28d584c3c0bc83a2 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Fri, 27 Sep 2019 12:28:19 -0400
Subject: [PATCH 1/2] Introduce subprocess infrastructure
This is an initial attempt at coalescing all of the postmaster
subprocess startups into one central framework. The patchset introduces
a MySubprocess global, which contains an PgSubprocess entry from the
process_types array. Each entry has defined entrypoints, cleanup
functions, prep functions, and a few other control-oriented variables.
This is a rework of https://commitfest.postgresql.org/25/2259/, which
attempts to address the feedback from Andres Freund.
---
src/backend/bootstrap/bootstrap.c | 78 +-
src/backend/main/main.c | 3 +
src/backend/postmaster/Makefile | 1 +
src/backend/postmaster/autovacuum.c | 165 +---
src/backend/postmaster/bgworker.c | 151 +++-
src/backend/postmaster/bgwriter.c | 2 +-
src/backend/postmaster/checkpointer.c | 2 +-
src/backend/postmaster/pgarch.c | 83 +--
src/backend/postmaster/pgstat.c | 89 +--
src/backend/postmaster/postmaster.c | 786 +++++++-------------
src/backend/postmaster/startup.c | 8 +-
src/backend/postmaster/subprocess.c | 206 +++++
src/backend/postmaster/syslogger.c | 278 +++----
src/backend/postmaster/walwriter.c | 2 +-
src/backend/replication/walreceiver.c | 2 +-
src/backend/utils/init/globals.c | 3 +-
src/backend/utils/init/postinit.c | 2 +-
src/include/bootstrap/bootstrap.h | 4 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 5 +-
src/include/postmaster/autovacuum.h | 13 +-
src/include/postmaster/bgworker_internals.h | 7 +-
src/include/postmaster/bgwriter.h | 4 +-
src/include/postmaster/fork_process.h | 6 +
src/include/postmaster/pgarch.h | 9 +-
src/include/postmaster/postmaster.h | 62 +-
src/include/postmaster/startup.h | 5 +-
src/include/postmaster/subprocess.h | 62 ++
src/include/postmaster/syslogger.h | 6 +-
src/include/postmaster/walwriter.h | 2 +-
src/include/replication/walreceiver.h | 2 +-
31 files changed, 989 insertions(+), 1060 deletions(-)
create mode 100644 src/backend/postmaster/subprocess.c
create mode 100644 src/include/postmaster/subprocess.h
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index bfc629c753..ed1066efd0 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -35,6 +35,7 @@
#include "pgstat.h"
#include "postmaster/bgwriter.h"
#include "postmaster/startup.h"
+#include "postmaster/subprocess.h"
#include "postmaster/walwriter.h"
#include "replication/walreceiver.h"
#include "storage/bufmgr.h"
@@ -56,8 +57,6 @@ uint32 bootstrap_data_checksum_version = 0; /* No checksum */
#define ALLOC(t, c) \
((t *) MemoryContextAllocZero(TopMemoryContext, (unsigned)(c) * sizeof(t)))
-static void CheckerModeMain(void);
-static void BootstrapModeMain(void);
static void bootstrap_signals(void);
static void ShutdownAuxiliaryProcess(int code, Datum arg);
static Form_pg_attribute AllocateAttribute(void);
@@ -426,56 +425,18 @@ AuxiliaryProcessMain(int argc, char *argv[])
*/
SetProcessingMode(NormalProcessing);
- switch (MyAuxProcType)
- {
- case CheckerProcess:
- /* don't set signals, they're useless here */
- CheckerModeMain();
- proc_exit(1); /* should never return */
-
- case BootstrapProcess:
-
- /*
- * There was a brief instant during which mode was Normal; this is
- * okay. We need to be in bootstrap mode during BootStrapXLOG for
- * the sake of multixact initialization.
- */
- SetProcessingMode(BootstrapProcessing);
- bootstrap_signals();
- BootStrapXLOG();
- BootstrapModeMain();
- proc_exit(1); /* should never return */
-
- case StartupProcess:
- /* don't set signals, startup process has its own agenda */
- StartupProcessMain();
- proc_exit(1); /* should never return */
-
- case BgWriterProcess:
- /* don't set signals, bgwriter has its own agenda */
- BackgroundWriterMain();
- proc_exit(1); /* should never return */
-
- case CheckpointerProcess:
- /* don't set signals, checkpointer has its own agenda */
- CheckpointerMain();
- proc_exit(1); /* should never return */
-
- case WalWriterProcess:
- /* don't set signals, walwriter has its own agenda */
- InitXLOGAccess();
- WalWriterMain();
- proc_exit(1); /* should never return */
-
- case WalReceiverProcess:
- /* don't set signals, walreceiver has its own agenda */
- WalReceiverMain();
- proc_exit(1); /* should never return */
-
- default:
- elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
- proc_exit(1);
- }
+ /*
+ * At this point, we should know what kind of process we are. If the
+ * MySubprocess global has not been initialized, just cast and pass along the
+ * MyAuxProcType enum
+ */
+ if (MySubprocess == NULL)
+ InitializeMySubprocess((SubprocessType)MyAuxProcType);
+
+ /* Now jump into the subprocess main function and never look back! */
+ MySubprocess->entrypoint(argc, argv);
+
+ proc_exit(1); /* should never return */
}
/*
@@ -484,8 +445,8 @@ AuxiliaryProcessMain(int argc, char *argv[])
* settings). Since, in fact, that was already done by BaseInit(),
* we have nothing more to do here.
*/
-static void
-CheckerModeMain(void)
+void
+CheckerModeMain(int argc, char *argv[])
{
proc_exit(0);
}
@@ -497,11 +458,15 @@ CheckerModeMain(void)
* The bootstrap backend doesn't speak SQL, but instead expects
* commands in a special bootstrap language.
*/
-static void
-BootstrapModeMain(void)
+void
+BootstrapModeMain(int argc, char *argv[])
{
int i;
+ SetProcessingMode(BootstrapProcessing);
+ bootstrap_signals();
+ BootStrapXLOG();
+
Assert(!IsUnderPostmaster);
Assert(IsBootstrapProcessingMode());
@@ -516,7 +481,6 @@ BootstrapModeMain(void)
* Do backend-like initialization for bootstrap mode
*/
InitProcess();
-
InitPostgres(NULL, InvalidOid, NULL, InvalidOid, NULL, false);
/* Initialize stuff for bootstrap-file processing */
diff --git a/src/backend/main/main.c b/src/backend/main/main.c
index da3dae9e25..d1e8089098 100644
--- a/src/backend/main/main.c
+++ b/src/backend/main/main.c
@@ -35,6 +35,7 @@
#include "common/username.h"
#include "port/atomics.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "storage/s_lock.h"
#include "storage/spin.h"
#include "tcop/tcopprot.h"
@@ -199,7 +200,9 @@ main(int argc, char *argv[])
#endif
if (argc > 1 && strcmp(argv[1], "--boot") == 0)
+ {
AuxiliaryProcessMain(argc, argv); /* does not return */
+ }
else if (argc > 1 && strcmp(argv[1], "--describe-config") == 0)
GucInfoMain(); /* does not return */
else if (argc > 1 && strcmp(argv[1], "--single") == 0)
diff --git a/src/backend/postmaster/Makefile b/src/backend/postmaster/Makefile
index bfdf6a833d..52b4c2142c 100644
--- a/src/backend/postmaster/Makefile
+++ b/src/backend/postmaster/Makefile
@@ -23,6 +23,7 @@ OBJS = \
pgstat.o \
postmaster.o \
startup.o \
+ subprocess.o \
syslogger.o \
walwriter.o
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index e3a43d3296..cb55fb9019 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -83,7 +83,6 @@
#include "nodes/makefuncs.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "storage/bufmgr.h"
@@ -135,8 +134,8 @@ int Log_autovacuum_min_duration = -1;
#define MAX_AUTOVAC_SLEEPTIME 300 /* seconds */
/* Flags to tell if we are in an autovacuum process */
-static bool am_autovacuum_launcher = false;
-static bool am_autovacuum_worker = false;
+bool am_autovacuum_launcher = false;
+bool am_autovacuum_worker = false;
/* Flags set by signal handlers */
static volatile sig_atomic_t got_SIGUSR2 = false;
@@ -302,13 +301,6 @@ static WorkerInfo MyWorkerInfo = NULL;
/* PID of launcher, valid only in worker while shutting down */
int AutovacuumLauncherPid = 0;
-#ifdef EXEC_BACKEND
-static pid_t avlauncher_forkexec(void);
-static pid_t avworker_forkexec(void);
-#endif
-NON_EXEC_STATIC void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-NON_EXEC_STATIC void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
-
static Oid do_start_worker(void);
static void HandleAutoVacLauncherInterrupts(void);
static void AutoVacLauncherShutdown(void) pg_attribute_noreturn();
@@ -346,88 +338,15 @@ static void autovac_report_workitem(AutoVacuumWorkItem *workitem,
static void avl_sigusr2_handler(SIGNAL_ARGS);
static void autovac_refresh_stats(void);
-
-
/********************************************************************
* AUTOVACUUM LAUNCHER CODE
********************************************************************/
-#ifdef EXEC_BACKEND
-/*
- * forkexec routine for the autovacuum launcher process.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avlauncher_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavlauncher";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
-/*
- * We need this set from the outside, before InitProcess is called
- */
-void
-AutovacuumLauncherIAm(void)
-{
- am_autovacuum_launcher = true;
-}
-#endif
-
-/*
- * Main entry point for autovacuum launcher process, to be called from the
- * postmaster.
- */
-int
-StartAutoVacLauncher(void)
-{
- pid_t AutoVacPID;
-
-#ifdef EXEC_BACKEND
- switch ((AutoVacPID = avlauncher_forkexec()))
-#else
- switch ((AutoVacPID = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum launcher process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacLauncherMain(0, NULL);
- break;
-#endif
- default:
- return (int) AutoVacPID;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
/*
* Main loop for the autovacuum launcher process.
*/
-NON_EXEC_STATIC void
-AutoVacLauncherMain(int argc, char *argv[])
+void
+AutoVacLauncherMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
@@ -1423,83 +1342,11 @@ avl_sigusr2_handler(SIGNAL_ARGS)
* AUTOVACUUM WORKER CODE
********************************************************************/
-#ifdef EXEC_BACKEND
-/*
- * forkexec routines for the autovacuum worker.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avworker_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavworker";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
-/*
- * We need this set from the outside, before InitProcess is called
- */
-void
-AutovacuumWorkerIAm(void)
-{
- am_autovacuum_worker = true;
-}
-#endif
-
-/*
- * Main entry point for autovacuum worker process.
- *
- * This code is heavily based on pgarch.c, q.v.
- */
-int
-StartAutoVacWorker(void)
-{
- pid_t worker_pid;
-
-#ifdef EXEC_BACKEND
- switch ((worker_pid = avworker_forkexec()))
-#else
- switch ((worker_pid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum worker process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacWorkerMain(0, NULL);
- break;
-#endif
- default:
- return (int) worker_pid;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
/*
* AutoVacWorkerMain
*/
-NON_EXEC_STATIC void
-AutoVacWorkerMain(int argc, char *argv[])
+void
+AutoVacWorkerMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
Oid dbid;
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index 75fc0d5d33..acda30be40 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -33,8 +33,10 @@
#include "storage/shmem.h"
#include "tcop/tcopprot.h"
#include "utils/ascii.h"
+#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/timeout.h"
+#include "utils/timestamp.h"
/*
* The postmaster's list of registered background workers, in private memory.
@@ -133,7 +135,97 @@ static const struct
/* Private functions. */
static bgworker_main_type LookupBackgroundWorkerFunction(const char *libraryname, const char *funcname);
+static bool assign_backendlist_entry(RegisteredBgWorker *rw);
+bool BackgroundWorkerCleanup(int argc, char *argv[]);
+void BackgroundWorkerParentMain(int argc, char *argv[]);
+
+/*
+ * Allocate the Backend struct for a connected background worker, but don't
+ * add it to the list of backends just yet.
+ *
+ * On failure, return false without changing any worker state.
+ *
+ * Some info from the Backend is copied into the passed rw.
+ */
+static bool
+assign_backendlist_entry(RegisteredBgWorker *rw)
+{
+ Backend *bn;
+
+ /*
+ * Compute the cancel key that will be assigned to this session. We
+ * probably don't need cancel keys for background workers, but we'd better
+ * have something random in the field to prevent unfriendly people from
+ * sending cancels to them.
+ */
+ if (!RandomCancelKey(&MyCancelKey))
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random cancel key")));
+ return false;
+ }
+
+ bn = malloc(sizeof(Backend));
+ if (bn == NULL)
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+ return false;
+ }
+
+ bn->cancel_key = MyCancelKey;
+ bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
+ bn->bkend_type = BACKEND_TYPE_BGWORKER;
+ bn->dead_end = false;
+ bn->bgworker_notify = false;
+
+ rw->rw_backend = bn;
+ rw->rw_child_slot = bn->child_slot;
+
+ return true;
+}
+
+/*
+ * BgWorkerPrep
+ *
+ * Postmaster subroutine to prepare a bgworker subprocess.
+ *
+ * Returns 0 on success or -1 on failure.
+ *
+ * Note: if fail, we will be called again from the postmaster main loop.
+ */
+int
+BgWorkerPrep(int argc, char *argv[])
+{
+ RegisteredBgWorker *rw = CurrentBgWorker;
+
+ Assert(CurrentBgWorker);
+ Assert(rw->rw_pid == 0);
+
+ /*
+ * Allocate and assign the Backend element. Note we must do this before
+ * forking, so that we can handle out of memory properly.
+ *
+ * Treat failure as though the worker had crashed. That way, the
+ * postmaster will wait a bit before attempting to start it again; if it
+ * tried again right away, most likely it'd find itself repeating the
+ * out-of-memory or fork failure condition.
+ */
+ if (!assign_backendlist_entry(rw))
+ {
+ rw->rw_crashed_at = GetCurrentTimestamp();
+ return -1;
+ }
+
+ ereport(DEBUG1,
+ (errmsg("starting background worker process \"%s\"",
+ rw->rw_worker.bgw_name)));
+
+ return 0;
+}
/*
* Calculate shared memory needed.
@@ -677,12 +769,31 @@ bgworker_sigusr1_handler(SIGNAL_ARGS)
* postmaster.
*/
void
-StartBackgroundWorker(void)
+BackgroundWorkerMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
- BackgroundWorker *worker = MyBgworkerEntry;
+ BackgroundWorker *worker;
bgworker_main_type entrypt;
+ /* Close the postmaster's sockets */
+ ClosePostmasterPorts();
+
+ /*
+ * Before blowing away PostmasterContext, save this bgworker's
+ * data where it can find it.
+ */
+ MyBgworkerEntry = (BackgroundWorker *)
+ MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
+ memcpy(MyBgworkerEntry, &(CurrentBgWorker->rw_worker), sizeof(BackgroundWorker));
+
+ worker = (BackgroundWorker *) MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
+ MyBgworkerEntry = worker;
+
+ /* Now we can release postmaster's working memory context */
+ MemoryContextSwitchTo(TopMemoryContext);
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+
if (worker == NULL)
elog(FATAL, "unable to find bgworker entry");
@@ -816,6 +927,42 @@ StartBackgroundWorker(void)
proc_exit(0);
}
+bool
+BackgroundWorkerCleanup(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ RegisteredBgWorker *rw = CurrentBgWorker;
+
+ /* in postmaster, fork failed ... */
+ ereport(LOG,
+ (errmsg("could not fork worker process: %m")));
+
+ /* undo what assign_backendlist_entry did */
+ ReleasePostmasterChildSlot(rw->rw_child_slot);
+ rw->rw_child_slot = 0;
+ free(rw->rw_backend);
+ rw->rw_backend = NULL;
+ /* mark entry as crashed, so we'll try again later */
+ rw->rw_crashed_at = GetCurrentTimestamp();
+
+ /* Don't panic. */
+ return false;
+}
+
+void
+BackgroundWorkerParentMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ RegisteredBgWorker *rw = CurrentBgWorker;
+
+ rw->rw_pid = MyChildProcPid;
+ rw->rw_backend->pid = rw->rw_pid;
+ ReportBackgroundWorkerPID(rw);
+ /* add new worker to lists of backends */
+ dlist_push_head(&BackendList, &(rw->rw_backend->elem));
+#ifdef EXEC_BACKEND
+ ShmemBackendArrayAdd(rw->rw_backend);
+#endif
+}
+
/*
* Register a new static background worker.
*
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 069e27e427..b2170938f3 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -91,7 +91,7 @@ static XLogRecPtr last_snapshot_lsn = InvalidXLogRecPtr;
* basic execution environment, but not enabled signals yet.
*/
void
-BackgroundWriterMain(void)
+BackgroundWriterMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
MemoryContext bgwriter_context;
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index e354a78725..83f7be1b1e 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -180,7 +180,7 @@ static void ReqCheckpointHandler(SIGNAL_ARGS);
* basic execution environment, but not enabled signals yet.
*/
void
-CheckpointerMain(void)
+CheckpointerMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
MemoryContext checkpointer_context;
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 3ca30badb2..02c98d5d9f 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -91,11 +91,7 @@ static volatile sig_atomic_t ready_to_stop = false;
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgarch_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
+void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgarch_exit(SIGNAL_ARGS);
static void pgarch_waken(SIGNAL_ARGS);
static void pgarch_waken_stop(SIGNAL_ARGS);
@@ -112,7 +108,7 @@ static void pgarch_archiveDone(char *xlog);
*/
/*
- * pgarch_start
+ * PgArchiverPrep
*
* Called from postmaster at startup or after an existing archiver
* died. Attempt to fire up a fresh archiver process.
@@ -122,10 +118,9 @@ static void pgarch_archiveDone(char *xlog);
* Note: if fail, we will be called again from the postmaster main loop.
*/
int
-pgarch_start(void)
+PgArchiverPrep(int argc, char *argv[])
{
time_t curtime;
- pid_t pgArchPid;
/*
* Do nothing if no archiver needed
@@ -140,43 +135,10 @@ pgarch_start(void)
* the postmaster main loop, we will get another chance later.
*/
curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgarch_start_time) <
+ if ((unsigned int) (curtime - last_pgarch_start_time) >=
(unsigned int) PGARCH_RESTART_INTERVAL)
- return 0;
- last_pgarch_start_time = curtime;
-
-#ifdef EXEC_BACKEND
- switch ((pgArchPid = pgarch_forkexec()))
-#else
- switch ((pgArchPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork archiver: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgArchiverMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgArchPid;
- }
+ last_pgarch_start_time = curtime;
- /* shouldn't get here */
return 0;
}
@@ -186,42 +148,19 @@ pgarch_start(void)
*/
-#ifdef EXEC_BACKEND
-
-/*
- * pgarch_forkexec() -
- *
- * Format up the arglist for, then fork and exec, archive process
- */
-static pid_t
-pgarch_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
-
- av[ac++] = "--forkarch";
-
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
/*
* PgArchiverMain
*
* The argc/argv parameters are valid only in EXEC_BACKEND case. However,
* since we don't use 'em, it hardly matters...
*/
-NON_EXEC_STATIC void
-PgArchiverMain(int argc, char *argv[])
+void
+PgArchiverMain(pg_attribute_unused() int argc, pg_attribute_unused() char *arg[])
{
+ /* Drop our connection to postmaster's shared memory, as well */
+ dsm_detach_all();
+ PGSharedMemoryDetach();
+
/*
* Ignore all signals usually bound to some action in the postmaster,
* except for SIGHUP, SIGTERM, SIGUSR1, SIGUSR2, and SIGQUIT.
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 462b4d7e06..27af4fc1ed 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -275,11 +275,7 @@ static instr_time total_func_time;
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgstat_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
+void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgstat_beshutdown_hook(int code, Datum arg);
static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
@@ -685,46 +681,18 @@ pgstat_reset_all(void)
pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
}
-#ifdef EXEC_BACKEND
-
-/*
- * pgstat_forkexec() -
- *
- * Format up the arglist for, then fork and exec, statistics collector process
- */
-static pid_t
-pgstat_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkcol";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
/*
- * pgstat_start() -
+ * PgstatCollectorPrep
*
* Called from postmaster at startup or after an existing collector
* died. Attempt to fire up a fresh statistics collector.
*
- * Returns PID of child process, or 0 if fail.
- *
- * Note: if fail, we will be called again from the postmaster main loop.
*/
int
-pgstat_start(void)
+PgstatCollectorPrep(int argc, char *argv[])
{
time_t curtime;
- pid_t pgStatPid;
/*
* Check that the socket is there, else pgstat_init failed and we can do
@@ -733,53 +701,18 @@ pgstat_start(void)
if (pgStatSock == PGINVALID_SOCKET)
return 0;
+ curtime = time(NULL);
+
/*
* Do nothing if too soon since last collector start. This is a safety
* valve to protect against continuous respawn attempts if the collector
* is dying immediately at launch. Note that since we will be re-called
* from the postmaster main loop, we will get another chance later.
*/
- curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgstat_start_time) <
+ if ((unsigned int) (curtime - last_pgstat_start_time) >=
(unsigned int) PGSTAT_RESTART_INTERVAL)
- return 0;
- last_pgstat_start_time = curtime;
+ last_pgstat_start_time = curtime;
- /*
- * Okay, fork off the collector.
- */
-#ifdef EXEC_BACKEND
- switch ((pgStatPid = pgstat_forkexec()))
-#else
- switch ((pgStatPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork statistics collector: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgstatCollectorMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgStatPid;
- }
-
- /* shouldn't get here */
return 0;
}
@@ -4423,13 +4356,17 @@ pgstat_send_bgwriter(void)
* The argc/argv parameters are valid only in EXEC_BACKEND case.
* ----------
*/
-NON_EXEC_STATIC void
-PgstatCollectorMain(int argc, char *argv[])
+void
+PgstatCollectorMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
int len;
PgStat_Msg msg;
int wr;
+ /* Drop our connection to postmaster's shared memory, as well */
+ dsm_detach_all();
+ PGSharedMemoryDetach();
+
/*
* Ignore all signals usually bound to some action in the postmaster,
* except SIGHUP and SIGQUIT. Note we don't need a SIGUSR1 handler to
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 55187eb910..45723c1cd3 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -114,6 +114,7 @@
#include "postmaster/fork_process.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "postmaster/syslogger.h"
#include "replication/logicallauncher.h"
#include "replication/walsender.h"
@@ -136,62 +137,16 @@
#include "storage/spin.h"
#endif
-
-/*
- * Possible types of a backend. Beyond being the possible bkend_type values in
- * struct bkend, these are OR-able request flag bits for SignalSomeChildren()
- * and CountChildren().
- */
-#define BACKEND_TYPE_NORMAL 0x0001 /* normal backend */
-#define BACKEND_TYPE_AUTOVAC 0x0002 /* autovacuum worker process */
-#define BACKEND_TYPE_WALSND 0x0004 /* walsender process */
-#define BACKEND_TYPE_BGWORKER 0x0008 /* bgworker process */
-#define BACKEND_TYPE_ALL 0x000F /* OR of all the above */
-
-#define BACKEND_TYPE_WORKER (BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER)
-
-/*
- * List of active backends (or child processes anyway; we don't actually
- * know whether a given child has become a backend or is still in the
- * authorization phase). This is used mainly to keep track of how many
- * children we have and send them appropriate signals when necessary.
- *
- * "Special" children such as the startup, bgwriter and autovacuum launcher
- * tasks are not in this list. Autovacuum worker and walsender are in it.
- * Also, "dead_end" children are in it: these are children launched just for
- * the purpose of sending a friendly rejection message to a would-be client.
- * We must track them because they are attached to shared memory, but we know
- * they will never become live backends. dead_end children are not assigned a
- * PMChildSlot.
- *
- * Background workers are in this list, too.
- */
-typedef struct bkend
-{
- pid_t pid; /* process id of backend */
- int32 cancel_key; /* cancel key for cancels for this backend */
- int child_slot; /* PMChildSlot for this backend, if any */
-
- /*
- * Flavor of backend or auxiliary process. Note that BACKEND_TYPE_WALSND
- * backends initially announce themselves as BACKEND_TYPE_NORMAL, so if
- * bkend_type is normal, you should check for a recent transition.
- */
- int bkend_type;
- bool dead_end; /* is it going to send an error and quit? */
- bool bgworker_notify; /* gets bgworker start/stop notifications */
- dlist_node elem; /* list link in BackendList */
-} Backend;
-
-static dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
+dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
#ifdef EXEC_BACKEND
static Backend *ShmemBackendArray;
#endif
BackgroundWorker *MyBgworkerEntry = NULL;
-
-
+RegisteredBgWorker *CurrentBgWorker = NULL;
+static Backend *MyBackend;
+static int child_errno;
/* The socket number we are listening for connections on */
int PostPortNumber;
@@ -403,25 +358,25 @@ static void BackendInitialize(Port *port);
static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
-static int BackendStartup(Port *port);
static int ProcessStartupPacket(Port *port, bool secure_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
static int initMasks(fd_set *rmask);
static void report_fork_failure_to_client(Port *port, int errnum);
static CAC_state canAcceptConnections(int backend_type);
-static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+#ifdef EXEC_BACKEND
+static void shmemSetup(bool aux_process);
+#endif
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
-static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
-static pid_t StartChildProcess(AuxProcType type);
+static pid_t StartSubprocess(SubprocessType type);
static void StartAutovacuumWorker(void);
static void MaybeStartWalReceiver(void);
static void InitPostmasterDeathWatchHandle(void);
@@ -455,8 +410,7 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(Port *port);
-static pid_t internal_forkexec(int argc, char *argv[], Port *port);
+static pid_t internal_forkexec(int argc, char *argv[]);
/* Type for a socket that can be inherited to a client process */
#ifdef WIN32
@@ -507,6 +461,7 @@ typedef struct
TimestampTz PgStartTime;
TimestampTz PgReloadTime;
pg_time_t first_syslogger_file_time;
+
bool redirection_done;
bool IsBinaryUpgrade;
int max_safe_fds;
@@ -534,15 +489,23 @@ static bool save_backend_variables(BackendParameters *param, Port *port,
HANDLE childProcess, pid_t childPid);
#endif
-static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
-#define StartupDataBase() StartChildProcess(StartupProcess)
-#define StartBackgroundWriter() StartChildProcess(BgWriterProcess)
-#define StartCheckpointer() StartChildProcess(CheckpointerProcess)
-#define StartWalWriter() StartChildProcess(WalWriterProcess)
-#define StartWalReceiver() StartChildProcess(WalReceiverProcess)
+#define BGWORKER_LEN 15
+
+#define StartupDataBase() StartSubprocess(StartupType)
+#define StartBackgroundWriter() StartSubprocess(BgWriterType)
+#define StartCheckpointer() StartSubprocess(CheckpointerType)
+#define StartWalWriter() StartSubprocess(WalWriterType)
+#define StartWalReceiver() StartSubprocess(WalReceiverType)
+#define StartAutoVacLauncher() StartSubprocess(AutoVacLauncherType)
+#define StartAutoVacWorker() StartSubprocess(AutoVacWorkerType)
+#define pgstat_start() StartSubprocess(PgstatCollectorType)
+#define pgarch_start() StartSubprocess(PgArchiverType)
+#define SysLogger_Start() StartSubprocess(SysLoggerType)
+#define do_start_bgworker() StartSubprocess(BgWorkerType)
+#define BackendStartup() StartSubprocess(ClientBackendType)
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -560,6 +523,133 @@ int postmaster_alive_fds[2] = {-1, -1};
HANDLE PostmasterHandle;
#endif
+static Port *ConnProcPort = NULL;
+
+/*
+ * BackendMain
+ *
+ * Child code when forking a Backend.
+ */
+void BackendMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ /*
+ * Perform additional initialization and collect startup packet.
+ *
+ * We want to do this before InitProcess() for a couple of reasons: 1.
+ * so that we aren't eating up a PGPROC slot while waiting on the
+ * client. 2. so that if InitProcess() fails due to being out of
+ * PGPROC slots, we have already initialized libpq and are able to
+ * report the error to the client.
+ */
+ BackendInitialize(ConnProcPort);
+
+#ifdef EXEC_BACKEND
+ shmemSetup(false);
+#endif
+
+ /* And run the backend */
+ BackendRun(ConnProcPort); /* does not return */
+}
+
+/*
+ * BackendPostmasterMain
+ *
+ * Parent code when forking a Backend.
+ */
+void BackendParentMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ /* in parent, successful fork */
+ ereport(DEBUG2,
+ (errmsg_internal("forked new backend, pid=%d socket=%d",
+ (int) MyChildProcPid, (int) MyProcPort->sock)));
+
+ /*
+ * Everything's been successful, it's safe to add this backend to our list
+ * of backends.
+ */
+ MyBackend->pid = MyChildProcPid;
+ MyBackend->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
+ dlist_push_head(&BackendList, &MyBackend->elem);
+
+#ifdef EXEC_BACKEND
+ if (!MyBackend->dead_end)
+ ShmemBackendArrayAdd(MyBackend);
+#endif
+}
+
+/*
+ * BackendCleanup
+ *
+ * Backend cleanup in case a failure occurs forking a new Backend.
+ */
+bool
+BackendCleanup(int argc, char *argv[])
+{
+ if (!MyBackend->dead_end)
+ (void) ReleasePostmasterChildSlot(MyBackend->child_slot);
+ free(MyBackend);
+
+ report_fork_failure_to_client(MyProcPort, child_errno);
+
+ /* Don't panic */
+ return false;
+}
+
+/*
+ * BackendPrep
+ *
+ * Prepare a ForkProcType struct for starting a Backend.
+ * This does all prep related to av parameters and error messages.
+*/
+int
+BackendPrep(int argc, char *argv[])
+{
+ /*
+ * Create backend data structure. Better before the fork() so we can
+ * handle failure cleanly.
+ */
+ MyBackend = (Backend *) malloc(sizeof(Backend));
+ if (!MyBackend)
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+ }
+
+ /*
+ * Compute the cancel key that will be assigned to this backend. The
+ * backend will have its own copy in the forked-off process' value of
+ * MyCancelKey, so that it can transmit the key to the frontend.
+ */
+ if (!RandomCancelKey(&MyCancelKey))
+ {
+ free(MyBackend);
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random cancel key")));
+ }
+
+ MyBackend->cancel_key = MyCancelKey;
+
+ /* Pass down canAcceptConnections state */
+ ConnProcPort->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
+ MyBackend->dead_end = (ConnProcPort->canAcceptConnections != CAC_OK &&
+ ConnProcPort->canAcceptConnections != CAC_WAITBACKUP);
+
+ /*
+ * Unless it's a dead_end child, assign it a child slot number
+ */
+ if (!MyBackend->dead_end)
+ MyBackend->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
+ else
+ MyBackend->child_slot = 0;
+
+ /* Hasn't asked to be notified about any bgworkers yet */
+ MyBackend->bgworker_notify = false;
+
+ return 0;
+}
+
/*
* Postmaster main entry point
*/
@@ -1719,19 +1809,17 @@ ServerLoop(void)
break;
if (FD_ISSET(ListenSocket[i], &rmask))
{
- Port *port;
-
- port = ConnCreate(ListenSocket[i]);
- if (port)
+ ConnProcPort = ConnCreate(ListenSocket[i]);
+ if (ConnProcPort)
{
- BackendStartup(port);
+ BackendStartup();
/*
* We no longer need the open socket or port structure
* in this process
*/
- StreamClose(port->sock);
- ConnFree(port);
+ StreamClose(ConnProcPort->sock);
+ ConnFree(ConnProcPort);
}
}
}
@@ -2533,12 +2621,9 @@ ConnFree(Port *conn)
* This is called during child process startup to release file descriptors
* that are not needed by that child process. The postmaster still has
* them open, of course.
- *
- * Note: we pass am_syslogger as a boolean because we don't want to set
- * the global variable yet when this is called.
*/
void
-ClosePostmasterPorts(bool am_syslogger)
+ClosePostmasterPorts(void)
{
int i;
@@ -2575,7 +2660,8 @@ ClosePostmasterPorts(bool am_syslogger)
* If using syslogger, close the read side of the pipe. We don't bother
* tracking this in fd.c, either.
*/
- if (!am_syslogger)
+ if (strcmp(MySubprocess->name, "logger") ||
+ strcmp(MySubprocess->name, "backend"))
{
#ifndef WIN32
if (syslogPipe[0] >= 0)
@@ -4118,122 +4204,6 @@ TerminateChildren(int signal)
signal_child(PgStatPID, signal);
}
-/*
- * BackendStartup -- start backend process
- *
- * returns: STATUS_ERROR if the fork failed, STATUS_OK otherwise.
- *
- * Note: if you change this code, also consider StartAutovacuumWorker.
- */
-static int
-BackendStartup(Port *port)
-{
- Backend *bn; /* for backend cleanup */
- pid_t pid;
-
- /*
- * Create backend data structure. Better before the fork() so we can
- * handle failure cleanly.
- */
- bn = (Backend *) malloc(sizeof(Backend));
- if (!bn)
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- return STATUS_ERROR;
- }
-
- /*
- * Compute the cancel key that will be assigned to this backend. The
- * backend will have its own copy in the forked-off process' value of
- * MyCancelKey, so that it can transmit the key to the frontend.
- */
- if (!RandomCancelKey(&MyCancelKey))
- {
- free(bn);
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("could not generate random cancel key")));
- return STATUS_ERROR;
- }
-
- bn->cancel_key = MyCancelKey;
-
- /* Pass down canAcceptConnections state */
- port->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
- bn->dead_end = (port->canAcceptConnections != CAC_OK &&
- port->canAcceptConnections != CAC_WAITBACKUP);
-
- /*
- * Unless it's a dead_end child, assign it a child slot number
- */
- if (!bn->dead_end)
- bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
- else
- bn->child_slot = 0;
-
- /* Hasn't asked to be notified about any bgworkers yet */
- bn->bgworker_notify = false;
-
-#ifdef EXEC_BACKEND
- pid = backend_forkexec(port);
-#else /* !EXEC_BACKEND */
- pid = fork_process();
- if (pid == 0) /* child */
- {
- free(bn);
-
- /* Detangle from postmaster */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Perform additional initialization and collect startup packet */
- BackendInitialize(port);
-
- /* And run the backend */
- BackendRun(port);
- }
-#endif /* EXEC_BACKEND */
-
- if (pid < 0)
- {
- /* in parent, fork failed */
- int save_errno = errno;
-
- if (!bn->dead_end)
- (void) ReleasePostmasterChildSlot(bn->child_slot);
- free(bn);
- errno = save_errno;
- ereport(LOG,
- (errmsg("could not fork new process for connection: %m")));
- report_fork_failure_to_client(port, save_errno);
- return STATUS_ERROR;
- }
-
- /* in parent, successful fork */
- ereport(DEBUG2,
- (errmsg_internal("forked new backend, pid=%d socket=%d",
- (int) pid, (int) port->sock)));
-
- /*
- * Everything's been successful, it's safe to add this backend to our list
- * of backends.
- */
- bn->pid = pid;
- bn->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
- dlist_push_head(&BackendList, &bn->elem);
-
-#ifdef EXEC_BACKEND
- if (!bn->dead_end)
- ShmemBackendArrayAdd(bn);
-#endif
-
- return STATUS_OK;
-}
-
/*
* Try to report backend fork() failure to client before we close the
* connection. Since we do not care to risk blocking the postmaster on
@@ -4476,7 +4446,7 @@ BackendRun(Port *port)
maxac * sizeof(char *));
ac = 0;
- av[ac++] = "postgres";
+ av[ac++] = pstrdup("postgres");
/*
* Pass any backend switches specified with -o on the postmaster's own
@@ -4527,36 +4497,7 @@ BackendRun(Port *port)
pid_t
postmaster_forkexec(int argc, char *argv[])
{
- Port port;
-
- /* This entry point passes dummy values for the Port variables */
- memset(&port, 0, sizeof(port));
- return internal_forkexec(argc, argv, &port);
-}
-
-/*
- * backend_forkexec -- fork/exec off a backend process
- *
- * Some operating systems (WIN32) don't have fork() so we have to simulate
- * it by storing parameters that need to be passed to the child and
- * then create a new child process.
- *
- * returns the pid of the fork/exec'd process, or -1 on failure
- */
-static pid_t
-backend_forkexec(Port *port)
-{
- char *av[4];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkbackend";
- av[ac++] = NULL; /* filled in by internal_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return internal_forkexec(ac, av, port);
+ return internal_forkexec(argc, argv);
}
#ifndef WIN32
@@ -4568,7 +4509,7 @@ backend_forkexec(Port *port)
* - fork():s, and then exec():s the child process
*/
static pid_t
-internal_forkexec(int argc, char *argv[], Port *port)
+internal_forkexec(int argc, char *argv[])
{
static unsigned long tmpBackendFileNum = 0;
pid_t pid;
@@ -4576,7 +4517,7 @@ internal_forkexec(int argc, char *argv[], Port *port)
BackendParameters param;
FILE *fp;
- if (!save_backend_variables(¶m, port))
+ if (!save_backend_variables(¶m, ConnProcPort))
return -1; /* log made by save_backend_variables */
/* Calculate name for temp file */
@@ -4861,6 +4802,29 @@ retry:
}
#endif /* WIN32 */
+/*
+ * shmemSetup
+ *
+ * Helper function for a child to set up shmem before
+ * executing.
+ *
+ * aux_process - set to true if an auxiliary process.
+ */
+static void
+shmemSetup(bool aux_process)
+{
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
+
+ /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ if (aux_process)
+ InitAuxiliaryProcess();
+ else
+ InitProcess();
+
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores();
+}
/*
* SubPostmasterMain -- Get the fork/exec'd process into a state equivalent
@@ -4892,12 +4856,15 @@ SubPostmasterMain(int argc, char *argv[])
if (argc < 3)
elog(FATAL, "invalid subpostmaster invocation");
+ Assert(!ConnProcPort);
+
/* Read in the variables file */
memset(&port, 0, sizeof(Port));
read_backend_variables(argv[2], &port);
+ ConnProcPort = &port;
/* Close the postmaster's sockets (as soon as we know them) */
- ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
+ ClosePostmasterPorts();
/*
* Set reference point for stack-depth checking
@@ -4936,16 +4903,16 @@ SubPostmasterMain(int argc, char *argv[])
strcmp(argv[1], "--forkavlauncher") == 0 ||
strcmp(argv[1], "--forkavworker") == 0 ||
strcmp(argv[1], "--forkboot") == 0 ||
- strncmp(argv[1], "--forkbgworker=", 15) == 0)
+ strncmp(argv[1], "--forkbgworker=", BGWORKER_LEN) == 0)
PGSharedMemoryReAttach();
else
PGSharedMemoryNoReAttach();
/* autovacuum needs this set before calling InitProcess */
if (strcmp(argv[1], "--forkavlauncher") == 0)
- AutovacuumLauncherIAm();
+ am_autovacuum_launcher = true;
if (strcmp(argv[1], "--forkavworker") == 0)
- AutovacuumWorkerIAm();
+ am_autovacuum_worker = true;
/*
* Start our win32 signal implementation. This has to be done after we
@@ -5038,64 +5005,33 @@ SubPostmasterMain(int argc, char *argv[])
}
if (strcmp(argv[1], "--forkboot") == 0)
{
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitAuxiliaryProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
+ shmemSetup(true);
AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
}
if (strcmp(argv[1], "--forkavlauncher") == 0)
{
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
+ shmemSetup(true);
AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
}
if (strcmp(argv[1], "--forkavworker") == 0)
{
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
+ shmemSetup(true);
AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
}
- if (strncmp(argv[1], "--forkbgworker=", 15) == 0)
+ if (strncmp(argv[1], "--forkbgworker=", BGWORKER_LEN) == 0)
{
int shmem_slot;
/* do this as early as possible; in particular, before InitProcess() */
IsBackgroundWorker = true;
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
+ shmemSetup(false);
/* Fetch MyBgworkerEntry from shared memory */
- shmem_slot = atoi(argv[1] + 15);
+ shmem_slot = atoi(argv[1] + BGWORKER_LEN);
MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
- StartBackgroundWorker();
+ BackgroundWorkerMain(argc, argv);
}
if (strcmp(argv[1], "--forkarch") == 0)
{
@@ -5378,7 +5314,7 @@ StartupPacketTimeoutHandler(void)
/*
* Generate a random cancel key.
*/
-static bool
+bool
RandomCancelKey(int32 *cancel_key)
{
return pg_strong_random(cancel_key, sizeof(int32));
@@ -5424,7 +5360,6 @@ CountChildren(int target)
return cnt;
}
-
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5435,93 +5370,118 @@ CountChildren(int target)
* to start subprocess.
*/
static pid_t
-StartChildProcess(AuxProcType type)
+StartSubprocess(SubprocessType type)
{
pid_t pid;
- char *av[10];
- int ac = 0;
+ char *argv[10];
+ int argc = 0;
char typebuf[32];
- /*
- * Set up command-line arguments for subprocess
- */
- av[ac++] = "postgres";
+ /* This should be our first time in this function */
+ //Assert(!MySubprocess);
+ InitializeMySubprocess(type);
+
+ argv[argc++] = "postgres";
#ifdef EXEC_BACKEND
- av[ac++] = "--forkboot";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
+ argv[argc++] = psprintf("--%s", MySubprocess->progname);
+ argv[argc++] = NULL;
#endif
snprintf(typebuf, sizeof(typebuf), "-x%d", type);
- av[ac++] = typebuf;
+ argv[argc++] = typebuf;
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ argv[argc] = NULL;
+ Assert(argc < lengthof(argv));
+
+ /* Prep the subprocesses for a fork */
+ if (MySubprocess->init)
+ {
+ int rc = 0;
+ rc = MySubprocess->init(argc, argv);
+
+ if (rc != 0)
+ {
+ ereport(WARNING,
+ (errmsg("subprocess %s returned with %d: %m",
+ MySubprocess->desc, rc)));
+ }
+ }
#ifdef EXEC_BACKEND
- pid = postmaster_forkexec(ac, av);
-#else /* !EXEC_BACKEND */
+ pid = postmaster_forkexec(argc, argv);
+#else
pid = fork_process();
+#endif
+
+ /* some processes like backends and bgworkers need the pid */
+ MyChildProcPid = pid;
if (pid == 0) /* child */
{
+
+#ifdef EXEC_BACKEND
+ /* Call the process's main subroutine */
+ if (MySubprocess->needs_aux_proc)
+ AuxiliaryProcessMain(argc, argv);
+ else
+ MySubprocess->entrypoint(argc, argv);
+#else
InitPostmasterChild();
+ ClosePostmasterPorts();
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
+ /*
+ * Release postmaster's working memory context unless we're still using
+ * it.
+ */
+ if (!MySubprocess->keep_postmaster_memcontext)
+ {
+ MemoryContextSwitchTo(TopMemoryContext);
+ MemoryContextDelete(PostmasterContext);
+
+ PostmasterContext = NULL;
+ }
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
+ /* Call the process's main subroutine */
+ if (MySubprocess->needs_aux_proc)
+ AuxiliaryProcessMain(argc, argv);
+ else
+ MySubprocess->entrypoint(argc, argv);
- AuxiliaryProcessMain(ac, av);
ExitPostmaster(0);
+#endif
}
-#endif /* EXEC_BACKEND */
-
if (pid < 0)
{
/* in parent, fork failed */
- int save_errno = errno;
+ child_errno = errno;
- errno = save_errno;
- switch (type)
- {
- case StartupProcess:
- ereport(LOG,
- (errmsg("could not fork startup process: %m")));
- break;
- case BgWriterProcess:
- ereport(LOG,
- (errmsg("could not fork background writer process: %m")));
- break;
- case CheckpointerProcess:
- ereport(LOG,
- (errmsg("could not fork checkpointer process: %m")));
- break;
- case WalWriterProcess:
- ereport(LOG,
- (errmsg("could not fork WAL writer process: %m")));
- break;
- case WalReceiverProcess:
- ereport(LOG,
- (errmsg("could not fork WAL receiver process: %m")));
- break;
- default:
- ereport(LOG,
- (errmsg("could not fork process: %m")));
- break;
- }
+ ereport(LOG,
+ (errmsg("could not fork %s: %m", MySubprocess->desc)));
/*
* fork failure is fatal during startup, but there's no need to choke
* immediately if starting other child types fails.
*/
- if (type == StartupProcess)
- ExitPostmaster(1);
+ if (MySubprocess->cleanup)
+ {
+ /*
+ * Panic if the cleanup function tells us to. Otherwise, just bail
+ * with -1.
+ */
+ if (MySubprocess->cleanup(argc, argv))
+ ExitPostmaster(1);
+ else
+ return -1;
+ }
+
return 0;
}
+ else
+ {
+ if (MySubprocess->parent_main)
+ MySubprocess->parent_main(argc, argv);
+ }
/*
* in parent, successful fork
@@ -5756,124 +5716,6 @@ BackgroundWorkerUnblockSignals(void)
PG_SETMASK(&UnBlockSig);
}
-#ifdef EXEC_BACKEND
-static pid_t
-bgworker_forkexec(int shmem_slot)
-{
- char *av[10];
- int ac = 0;
- char forkav[MAXPGPATH];
-
- snprintf(forkav, MAXPGPATH, "--forkbgworker=%d", shmem_slot);
-
- av[ac++] = "postgres";
- av[ac++] = forkav;
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif
-
-/*
- * Start a new bgworker.
- * Starting time conditions must have been checked already.
- *
- * Returns true on success, false on failure.
- * In either case, update the RegisteredBgWorker's state appropriately.
- *
- * This code is heavily based on autovacuum.c, q.v.
- */
-static bool
-do_start_bgworker(RegisteredBgWorker *rw)
-{
- pid_t worker_pid;
-
- Assert(rw->rw_pid == 0);
-
- /*
- * Allocate and assign the Backend element. Note we must do this before
- * forking, so that we can handle failures (out of memory or child-process
- * slots) cleanly.
- *
- * Treat failure as though the worker had crashed. That way, the
- * postmaster will wait a bit before attempting to start it again; if we
- * tried again right away, most likely we'd find ourselves hitting the
- * same resource-exhaustion condition.
- */
- if (!assign_backendlist_entry(rw))
- {
- rw->rw_crashed_at = GetCurrentTimestamp();
- return false;
- }
-
- ereport(DEBUG1,
- (errmsg("starting background worker process \"%s\"",
- rw->rw_worker.bgw_name)));
-
-#ifdef EXEC_BACKEND
- switch ((worker_pid = bgworker_forkexec(rw->rw_shmem_slot)))
-#else
- switch ((worker_pid = fork_process()))
-#endif
- {
- case -1:
- /* in postmaster, fork failed ... */
- ereport(LOG,
- (errmsg("could not fork worker process: %m")));
- /* undo what assign_backendlist_entry did */
- ReleasePostmasterChildSlot(rw->rw_child_slot);
- rw->rw_child_slot = 0;
- free(rw->rw_backend);
- rw->rw_backend = NULL;
- /* mark entry as crashed, so we'll try again later */
- rw->rw_crashed_at = GetCurrentTimestamp();
- break;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /*
- * Before blowing away PostmasterContext, save this bgworker's
- * data where it can find it.
- */
- MyBgworkerEntry = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(MyBgworkerEntry, &rw->rw_worker, sizeof(BackgroundWorker));
-
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
-
- StartBackgroundWorker();
-
- exit(1); /* should not get here */
- break;
-#endif
- default:
- /* in postmaster, fork successful ... */
- rw->rw_pid = worker_pid;
- rw->rw_backend->pid = rw->rw_pid;
- ReportBackgroundWorkerPID(rw);
- /* add new worker to lists of backends */
- dlist_push_head(&BackendList, &rw->rw_backend->elem);
-#ifdef EXEC_BACKEND
- ShmemBackendArrayAdd(rw->rw_backend);
-#endif
- return true;
- }
-
- return false;
-}
-
/*
* Does the current postmaster state require starting a worker with the
* specified start_time?
@@ -5914,67 +5756,6 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
return false;
}
-/*
- * Allocate the Backend struct for a connected background worker, but don't
- * add it to the list of backends just yet.
- *
- * On failure, return false without changing any worker state.
- *
- * Some info from the Backend is copied into the passed rw.
- */
-static bool
-assign_backendlist_entry(RegisteredBgWorker *rw)
-{
- Backend *bn;
-
- /*
- * Check that database state allows another connection. Currently the
- * only possible failure is CAC_TOOMANY, so we just log an error message
- * based on that rather than checking the error code precisely.
- */
- if (canAcceptConnections(BACKEND_TYPE_BGWORKER) != CAC_OK)
- {
- ereport(LOG,
- (errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("no slot available for new worker process")));
- return false;
- }
-
- /*
- * Compute the cancel key that will be assigned to this session. We
- * probably don't need cancel keys for background workers, but we'd better
- * have something random in the field to prevent unfriendly people from
- * sending cancels to them.
- */
- if (!RandomCancelKey(&MyCancelKey))
- {
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("could not generate random cancel key")));
- return false;
- }
-
- bn = malloc(sizeof(Backend));
- if (bn == NULL)
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- return false;
- }
-
- bn->cancel_key = MyCancelKey;
- bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
- bn->bkend_type = BACKEND_TYPE_BGWORKER;
- bn->dead_end = false;
- bn->bgworker_notify = false;
-
- rw->rw_backend = bn;
- rw->rw_child_slot = bn->child_slot;
-
- return true;
-}
-
/*
* If the time is right, start background worker(s).
*
@@ -6079,7 +5860,8 @@ maybe_start_bgworkers(void)
* crashed, but there's no need because the next run of this
* function will do that.
*/
- if (!do_start_bgworker(rw))
+ CurrentBgWorker = rw;
+ if (do_start_bgworker() <= 0)
{
StartWorkerNeeded = true;
return;
@@ -6486,7 +6268,7 @@ ShmemBackendArrayAllocation(void)
memset(ShmemBackendArray, 0, size);
}
-static void
+void
ShmemBackendArrayAdd(Backend *bn)
{
/* The array slot corresponding to my PMChildSlot should be free */
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index c2250d7d4e..8957b341b9 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -127,13 +127,19 @@ HandleStartupProcInterrupts(void)
ProcessProcSignalBarrier();
}
+bool
+StartupCleanup(int argc, char *argv[])
+{
+ /* Panic! */
+ return true;
+}
/* ----------------------------------
* Startup Process main entry point
* ----------------------------------
*/
void
-StartupProcessMain(void)
+StartupProcessMain(int argc, char *argv[])
{
/*
* Properly accept or ignore signals the postmaster might send us.
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
new file mode 100644
index 0000000000..acd7da9213
--- /dev/null
+++ b/src/backend/postmaster/subprocess.c
@@ -0,0 +1,206 @@
+/*-------------------------------------------------------------------------
+ *
+ * subprocess.c
+ *
+ * Copyright (c) 2004-2020, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/postmaster/syslogger.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "postmaster/subprocess.h"
+#include "bootstrap/bootstrap.h"
+#include "postmaster/bgwriter.h"
+#include "postmaster/walwriter.h"
+#include "postmaster/syslogger.h"
+#include "postmaster/autovacuum.h"
+#include "postmaster/bgworker_internals.h"
+#include "postmaster/bgworker.h"
+#include "postmaster/postmaster.h"
+#include "postmaster/startup.h"
+#include "pgstat.h"
+#include "replication/walreceiver.h"
+
+PgSubprocess *MySubprocess = NULL;
+
+PgSubprocess process_types[] = {
+ {
+ .progname = "boot",
+ .name = "checker",
+ .desc = "checker process",
+ .needs_shmem = true,
+ .needs_aux_proc = true,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = CheckerModeMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "boot",
+ .name = "bootstrap",
+ .desc = "bootstrap process",
+ .needs_shmem = true,
+ .needs_aux_proc = true,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = BootstrapModeMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "boot",
+ .name = "startup",
+ .desc = "startup process",
+ .needs_shmem = true,
+ .needs_aux_proc = true,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = StartupProcessMain,
+ .cleanup = StartupCleanup,
+ .parent_main = NULL
+ },
+ {
+ .progname = "boot",
+ .name = "bgwriter",
+ .desc = "background writer process",
+ .needs_shmem = true,
+ .needs_aux_proc = true,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = BackgroundWriterMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "boot",
+ .name = "checkpointer",
+ .desc = "checkpointer process",
+ .needs_shmem = true,
+ .needs_aux_proc = true,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = CheckpointerMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "boot",
+ .name = "walwriter",
+ .desc = "WAL writer process",
+ .needs_shmem = true,
+ .needs_aux_proc = true,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = WalWriterMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "boot",
+ .name = "walreceiver",
+ .desc = "WAL receiver process",
+ .needs_shmem = true,
+ .needs_aux_proc = true,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = WalReceiverMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forkavlauncher",
+ .name = "autovacuum launcher",
+ .desc = "autovacuum launcher process",
+ .needs_shmem = true,
+ .needs_aux_proc = false,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = AutoVacLauncherMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forkavworker",
+ .name = "autovacuum worker",
+ .desc = "autovacuum worker process",
+ .needs_shmem = true,
+ .needs_aux_proc = false,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = AutoVacWorkerMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forkcol",
+ .name = "pgstat collector",
+ .desc = "statistics collector process",
+ .needs_shmem = true,
+ .needs_aux_proc = false,
+ .keep_postmaster_memcontext = false,
+ .init = PgstatCollectorPrep,
+ .entrypoint = PgstatCollectorMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forkarch",
+ .name = "archiver",
+ .desc = "archiver process",
+ .needs_shmem = true,
+ .needs_aux_proc = false,
+ .keep_postmaster_memcontext = false,
+ .init = PgArchiverPrep,
+ .entrypoint = PgArchiverMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forklog",
+ .name = "logger",
+ .desc = "system logger",
+ .needs_shmem = true,
+ .needs_aux_proc = false,
+ .keep_postmaster_memcontext = true,
+ .init = SysLoggerPrep,
+ .entrypoint = SysLoggerMain,
+ .cleanup = NULL,
+ .parent_main = SysLoggerParentMain
+ },
+ {
+ .progname = "forkbgworker",
+ .name = "bgworker",
+ .desc = "background worker",
+ .needs_shmem = true,
+ .needs_aux_proc = false,
+ .keep_postmaster_memcontext = false,
+ .init = BgWorkerPrep,
+ .entrypoint = BackgroundWorkerMain,
+ .cleanup = BackgroundWorkerCleanup,
+ .parent_main = BackgroundWorkerParentMain
+ },
+ {
+ .progname = "forkbackend",
+ .name = "backend",
+ .desc = "client backend",
+ .needs_shmem = true,
+ .needs_aux_proc = false,
+ .keep_postmaster_memcontext = false,
+ .init = BackendPrep,
+ .entrypoint = BackendMain,
+ .cleanup = BackendCleanup,
+ .parent_main = BackendParentMain
+ }
+};
+
+void
+InitializeMySubprocess(SubprocessType type)
+{
+ /* Globally set subprocess information */
+ MySubprocess = (PgSubprocess *)malloc(sizeof(PgSubprocess));
+ memcpy(MySubprocess, &process_types[type], sizeof(PgSubprocess));
+}
diff --git a/src/backend/postmaster/syslogger.c b/src/backend/postmaster/syslogger.c
index cf7b535e4e..6cb35c9165 100644
--- a/src/backend/postmaster/syslogger.c
+++ b/src/backend/postmaster/syslogger.c
@@ -48,6 +48,7 @@
#include "storage/pg_shmem.h"
#include "tcop/tcopprot.h"
#include "utils/guc.h"
+#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/timestamp.h"
@@ -93,6 +94,7 @@ NON_EXEC_STATIC pg_time_t first_syslogger_file_time = 0;
static char *last_file_name = NULL;
static char *last_csv_file_name = NULL;
+
/*
* Buffers for saving partial messages from different backends.
*
@@ -131,13 +133,15 @@ static CRITICAL_SECTION sysloggerSection;
static volatile sig_atomic_t got_SIGHUP = false;
static volatile sig_atomic_t rotation_requested = false;
-
/* Local subroutines */
#ifdef EXEC_BACKEND
-static pid_t syslogger_forkexec(void);
static void syslogger_parseArgs(int argc, char *argv[]);
#endif
-NON_EXEC_STATIC void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
+
+/* Subprocess start routines */
+void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
+void SysLoggerPostmasterMain(int argc, char *argv[]);
+
static void process_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static void flush_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static FILE *logfile_open(const char *filename, const char *mode,
@@ -158,7 +162,7 @@ static void update_metainfo_datafile(void);
* Main entry point for syslogger process
* argc/argv parameters are valid only in EXEC_BACKEND case.
*/
-NON_EXEC_STATIC void
+void
SysLoggerMain(int argc, char *argv[])
{
#ifndef WIN32
@@ -171,12 +175,16 @@ SysLoggerMain(int argc, char *argv[])
pg_time_t now;
WaitEventSet *wes;
- now = MyStartTime;
-
#ifdef EXEC_BACKEND
syslogger_parseArgs(argc, argv);
+#else
+ /* Drop our connection to postmaster's shared memory, as well */
+ dsm_detach_all();
+ PGSharedMemoryDetach();
#endif /* EXEC_BACKEND */
+ now = MyStartTime;
+
am_syslogger = true;
init_ps_display("logger", "", "", "");
@@ -540,17 +548,47 @@ SysLoggerMain(int argc, char *argv[])
}
/*
- * Postmaster subroutine to start a syslogger subprocess.
+ * Helper function for setting up a file number buffer
+ */
+static char *
+setup_file_buff(FILE *file)
+{
+ char *filenobuf;
+
+ /* static variables (those not passed by write_backend_variables) */
+#ifndef WIN32
+ if (file != NULL)
+ filenobuf = psprintf("%d", fileno(file));
+ else
+ filenobuf = pstrdup("-1");
+#else /* WIN32 */
+ if (file != NULL)
+ filenobuf = psprintf("%ld", (long) _get_osfhandle(_fileno(file)));
+ else
+ filenobuf = pstrdup("0");
+#endif /* WIN32 */
+
+ return filenobuf;
+}
+
+/*
+ * SysLoggerPrep
+ *
+ * Postmaster subroutine to prepare a syslogger subprocess.
*/
int
-SysLogger_Start(void)
+SysLoggerPrep(int argc, char *argv[])
{
- pid_t sysloggerPid;
char *filename;
+ MemoryContext cxt;
if (!Logging_collector)
return 0;
+ cxt = MemoryContextSwitchTo(PostmasterContext);
+
+ elog(LOG, "setting up the syslogger");
+
/*
* If first time through, create the pipe which will receive stderr
* output.
@@ -631,167 +669,94 @@ SysLogger_Start(void)
pfree(filename);
}
-#ifdef EXEC_BACKEND
- switch ((sysloggerPid = syslogger_forkexec()))
-#else
- switch ((sysloggerPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork system logger: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(true);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- /* do the work */
- SysLoggerMain(0, NULL);
- break;
-#endif
-
- default:
- /* success, in postmaster */
-
- /* now we redirect stderr, if not done already */
- if (!redirection_done)
- {
-#ifdef WIN32
- int fd;
-#endif
-
- /*
- * Leave a breadcrumb trail when redirecting, in case the user
- * forgets that redirection is active and looks only at the
- * original stderr target file.
- */
- ereport(LOG,
- (errmsg("redirecting log output to logging collector process"),
- errhint("Future log output will appear in directory \"%s\".",
- Log_directory)));
+ argv[argc++] = setup_file_buff(syslogFile);
+ argv[argc++] = setup_file_buff(csvlogFile);
+ argv[argc] = NULL;
-#ifndef WIN32
- fflush(stdout);
- if (dup2(syslogPipe[1], fileno(stdout)) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stdout: %m")));
- fflush(stderr);
- if (dup2(syslogPipe[1], fileno(stderr)) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- /* Now we are done with the write end of the pipe. */
- close(syslogPipe[1]);
- syslogPipe[1] = -1;
-#else
-
- /*
- * open the pipe in binary mode and make sure stderr is binary
- * after it's been dup'ed into, to avoid disturbing the pipe
- * chunking protocol.
- */
- fflush(stderr);
- fd = _open_osfhandle((intptr_t) syslogPipe[1],
- _O_APPEND | _O_BINARY);
- if (dup2(fd, _fileno(stderr)) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- close(fd);
- _setmode(_fileno(stderr), _O_BINARY);
-
- /*
- * Now we are done with the write end of the pipe.
- * CloseHandle() must not be called because the preceding
- * close() closes the underlying handle.
- */
- syslogPipe[1] = 0;
-#endif
- redirection_done = true;
- }
+ Assert(argc < lengthof(argv));
- /* postmaster will never write the file(s); close 'em */
- fclose(syslogFile);
- syslogFile = NULL;
- if (csvlogFile != NULL)
- {
- fclose(csvlogFile);
- csvlogFile = NULL;
- }
- return (int) sysloggerPid;
- }
+ MemoryContextSwitchTo(cxt);
- /* we should never reach here */
return 0;
}
-
-#ifdef EXEC_BACKEND
-
/*
- * syslogger_forkexec() -
+ * SysLoggerParentMain
*
- * Format up the arglist for, then fork and exec, a syslogger process
+ * Fallthrough subroutine for parent running syslogger.
*/
-static pid_t
-syslogger_forkexec(void)
+void
+SysLoggerParentMain(int argc, char *argv[])
{
- char *av[10];
- int ac = 0;
- char filenobuf[32];
- char csvfilenobuf[32];
+ /* now we redirect stderr, if not done already */
+ if (!redirection_done)
+ {
+#ifdef WIN32
+ int fd;
+#endif
- av[ac++] = "postgres";
- av[ac++] = "--forklog";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
+ /*
+ * Leave a breadcrumb trail when redirecting, in case the user
+ * forgets that redirection is active and looks only at the
+ * original stderr target file.
+ */
+ ereport(LOG,
+ (errmsg("redirecting log output to logging collector process"),
+ errhint("Future log output will appear in directory \"%s\".",
+ Log_directory)));
- /* static variables (those not passed by write_backend_variables) */
#ifndef WIN32
- if (syslogFile != NULL)
- snprintf(filenobuf, sizeof(filenobuf), "%d",
- fileno(syslogFile));
- else
- strcpy(filenobuf, "-1");
-#else /* WIN32 */
- if (syslogFile != NULL)
- snprintf(filenobuf, sizeof(filenobuf), "%ld",
- (long) _get_osfhandle(_fileno(syslogFile)));
- else
- strcpy(filenobuf, "0");
-#endif /* WIN32 */
- av[ac++] = filenobuf;
+ fflush(stdout);
+ if (dup2(syslogPipe[1], fileno(stdout)) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stdout: %m")));
+ fflush(stderr);
+ if (dup2(syslogPipe[1], fileno(stderr)) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ /* Now we are done with the write end of the pipe. */
+ close(syslogPipe[1]);
+ syslogPipe[1] = -1;
+#else
-#ifndef WIN32
- if (csvlogFile != NULL)
- snprintf(csvfilenobuf, sizeof(csvfilenobuf), "%d",
- fileno(csvlogFile));
- else
- strcpy(csvfilenobuf, "-1");
-#else /* WIN32 */
- if (csvlogFile != NULL)
- snprintf(csvfilenobuf, sizeof(csvfilenobuf), "%ld",
- (long) _get_osfhandle(_fileno(csvlogFile)));
- else
- strcpy(csvfilenobuf, "0");
-#endif /* WIN32 */
- av[ac++] = csvfilenobuf;
+ /*
+ * open the pipe in binary mode and make sure stderr is binary
+ * after it's been dup'ed into, to avoid disturbing the pipe
+ * chunking protocol.
+ */
+ fflush(stderr);
+ fd = _open_osfhandle((intptr_t) syslogPipe[1],
+ _O_APPEND | _O_BINARY);
+ if (dup2(fd, _fileno(stderr)) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ close(fd);
+ _setmode(_fileno(stderr), _O_BINARY);
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ /*
+ * Now we are done with the write end of the pipe.
+ * CloseHandle() must not be called because the preceding
+ * close() closes the underlying handle.
+ */
+ syslogPipe[1] = 0;
+#endif
+ redirection_done = true;
+ }
- return postmaster_forkexec(ac, av);
+ /* postmaster will never write the file(s); close 'em */
+ fclose(syslogFile);
+ syslogFile = NULL;
+ if (csvlogFile != NULL)
+ {
+ fclose(csvlogFile);
+ csvlogFile = NULL;
+ }
}
+#ifdef EXEC_BACKEND
/*
* syslogger_parseArgs() -
*
@@ -800,7 +765,7 @@ syslogger_forkexec(void)
static void
syslogger_parseArgs(int argc, char *argv[])
{
- int fd;
+ int fd;
Assert(argc == 5);
argv += 3;
@@ -825,7 +790,7 @@ syslogger_parseArgs(int argc, char *argv[])
csvlogFile = fdopen(fd, "a");
setvbuf(csvlogFile, NULL, PG_IOLBF, 0);
}
-#else /* WIN32 */
+#else /* WIN32 */
fd = atoi(*argv++);
if (fd != 0)
{
@@ -846,10 +811,9 @@ syslogger_parseArgs(int argc, char *argv[])
setvbuf(csvlogFile, NULL, PG_IOLBF, 0);
}
}
-#endif /* WIN32 */
+#endif /* WIN32 */
}
-#endif /* EXEC_BACKEND */
-
+#endif
/* --------------------------------
* pipe protocol handling
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index 45a2757969..3a0cec3371 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -85,7 +85,7 @@ int WalWriterFlushAfter = 128;
* basic execution environment, but not enabled signals yet.
*/
void
-WalWriterMain(void)
+WalWriterMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
MemoryContext walwriter_context;
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index 2ab15c3cbb..725023f0f6 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -164,7 +164,7 @@ ProcessWalRcvInterrupts(void)
/* Main entry point for walreceiver process */
void
-WalReceiverMain(void)
+WalReceiverMain(int argc, char *argv[])
{
char conninfo[MAXCONNINFO];
char *tmp_conninfo;
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index eb19644419..3e7be0bfcd 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -38,9 +38,10 @@ volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
int MyProcPid;
+int MyChildProcPid;
pg_time_t MyStartTime;
TimestampTz MyStartTimestamp;
-struct Port *MyProcPort;
+struct Port *MyProcPort = NULL;
int32 MyCancelKey;
int MyPMChildSlot;
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 8a47dcdcb1..91e43382c0 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -1112,7 +1112,7 @@ process_startup_options(Port *port, bool am_superuser)
av = (char **) palloc(maxac * sizeof(char *));
ac = 0;
- av[ac++] = "postgres";
+ av[ac++] = pstrdup("postgres");
pg_split_opts(av, &ac, port->cmdline_options);
diff --git a/src/include/bootstrap/bootstrap.h b/src/include/bootstrap/bootstrap.h
index b67fa9acc1..5d851e0081 100644
--- a/src/include/bootstrap/bootstrap.h
+++ b/src/include/bootstrap/bootstrap.h
@@ -15,7 +15,7 @@
#define BOOTSTRAP_H
#include "nodes/execnodes.h"
-
+#include "postmaster/fork_process.h"
/*
* MAXATTR is the maximum number of attributes in a relation supported
@@ -31,6 +31,8 @@ extern Relation boot_reldesc;
extern Form_pg_attribute attrtypes[MAXATTR];
extern int numattr;
+extern void CheckerModeMain(int argc, char *argv[]);
+extern void BootstrapModeMain(int argc, char *argv[]);
extern void AuxiliaryProcessMain(int argc, char *argv[]) pg_attribute_noreturn();
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index f985453ec3..823f5258d8 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -163,6 +163,7 @@ extern PGDLLIMPORT int max_worker_processes;
extern PGDLLIMPORT int max_parallel_workers;
extern PGDLLIMPORT int MyProcPid;
+extern PGDLLIMPORT int MyChildProcPid;
extern PGDLLIMPORT pg_time_t MyStartTime;
extern PGDLLIMPORT TimestampTz MyStartTimestamp;
extern PGDLLIMPORT struct Port *MyProcPort;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 3a65a51696..901b709fb8 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1237,13 +1237,12 @@ extern Size BackendStatusShmemSize(void);
extern void CreateSharedBackendStatus(void);
extern void pgstat_init(void);
-extern int pgstat_start(void);
extern void pgstat_reset_all(void);
extern void allow_immediate_pgstat_restart(void);
-#ifdef EXEC_BACKEND
+/* Stat collector subprocess startup functions */
+extern int PgstatCollectorPrep(int argc, char *argv[]);
extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
/* ----------
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index d40ed55531..dc02de745e 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -45,6 +45,10 @@ extern int AutovacuumLauncherPid;
extern int Log_autovacuum_min_duration;
+/* autovacuum identification */
+extern bool am_autovacuum_launcher;
+extern bool am_autovacuum_worker;
+
/* Status inquiry functions */
extern bool AutoVacuumingActive(void);
extern bool IsAutoVacuumLauncherProcess(void);
@@ -58,18 +62,15 @@ extern void autovac_init(void);
extern int StartAutoVacLauncher(void);
extern int StartAutoVacWorker(void);
+extern void AutoVacWorkerMain(int argc, char *argv[]);
+extern void AutoVacLauncherMain(int argc, char *argv[]);
+
/* called from postmaster when a worker could not be forked */
extern void AutoVacWorkerFailed(void);
/* autovacuum cost-delay balancer */
extern void AutoVacuumUpdateDelay(void);
-#ifdef EXEC_BACKEND
-extern void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern void AutovacuumWorkerIAm(void);
-extern void AutovacuumLauncherIAm(void);
-#endif
extern bool AutoVacuumRequestWork(AutoVacuumWorkItemType type,
Oid relationId, BlockNumber blkno);
diff --git a/src/include/postmaster/bgworker_internals.h b/src/include/postmaster/bgworker_internals.h
index f7e24664d5..366bbcf1e3 100644
--- a/src/include/postmaster/bgworker_internals.h
+++ b/src/include/postmaster/bgworker_internals.h
@@ -43,6 +43,7 @@ typedef struct RegisteredBgWorker
} RegisteredBgWorker;
extern slist_head BackgroundWorkerList;
+extern RegisteredBgWorker *CurrentBgWorker;
extern Size BackgroundWorkerShmemSize(void);
extern void BackgroundWorkerShmemInit(void);
@@ -54,7 +55,11 @@ extern void BackgroundWorkerStopNotifications(pid_t pid);
extern void ResetBackgroundWorkerCrashTimes(void);
/* Function to start a background worker, called from postmaster.c */
-extern void StartBackgroundWorker(void) pg_attribute_noreturn();
+extern int BgWorkerPrep(int argc, char *argv[]);
+extern void BackgroundWorkerMain(int argc, char *argv[]);
+extern void BackgroundWorkerParentMain(int argc, char *argv[]);
+extern bool BackgroundWorkerCleanup(int argc, char *argv[]);
+
#ifdef EXEC_BACKEND
extern BackgroundWorker *BackgroundWorkerEntry(int slotno);
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..6fb1ee1d6d 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -27,8 +27,8 @@ extern int CheckPointTimeout;
extern int CheckPointWarning;
extern double CheckPointCompletionTarget;
-extern void BackgroundWriterMain(void) pg_attribute_noreturn();
-extern void CheckpointerMain(void) pg_attribute_noreturn();
+extern void BackgroundWriterMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern void CheckpointerMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void RequestCheckpoint(int flags);
extern void CheckpointWriteDelay(int flags, double progress);
diff --git a/src/include/postmaster/fork_process.h b/src/include/postmaster/fork_process.h
index 8cb568fc63..a42f859a08 100644
--- a/src/include/postmaster/fork_process.h
+++ b/src/include/postmaster/fork_process.h
@@ -12,6 +12,12 @@
#ifndef FORK_PROCESS_H
#define FORK_PROCESS_H
+#include "postmaster.h"
+
extern pid_t fork_process(void);
+#ifdef EXEC_BACKEND
+extern pid_t postmaster_forkexec(int argc, char *argvp[]);
+#endif
+
#endif /* FORK_PROCESS_H */
diff --git a/src/include/postmaster/pgarch.h b/src/include/postmaster/pgarch.h
index b3200874ca..ba5b012839 100644
--- a/src/include/postmaster/pgarch.h
+++ b/src/include/postmaster/pgarch.h
@@ -26,14 +26,9 @@
#define MAX_XFN_CHARS 40
#define VALID_XFN_CHARS "0123456789ABCDEF.history.backup.partial"
-/* ----------
- * Functions called from postmaster
- * ----------
- */
-extern int pgarch_start(void);
-#ifdef EXEC_BACKEND
+/* Archiver subprocess startup functions */
+extern int PgArchiverPrep(int argc, char *argv[]);
extern void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
#endif /* _PGARCH_H */
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index babc87dfc9..d4c0f46220 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -13,6 +13,41 @@
#ifndef _POSTMASTER_H
#define _POSTMASTER_H
+#include "lib/ilist.h"
+
+/*
+ * List of active backends (or child processes anyway; we don't actually
+ * know whether a given child has become a backend or is still in the
+ * authorization phase). This is used mainly to keep track of how many
+ * children we have and send them appropriate signals when necessary.
+ *
+ * "Special" children such as the startup, bgwriter and autovacuum launcher
+ * tasks are not in this list. Autovacuum worker and walsender are in it.
+ * Also, "dead_end" children are in it: these are children launched just for
+ * the purpose of sending a friendly rejection message to a would-be client.
+ * We must track them because they are attached to shared memory, but we know
+ * they will never become live backends. dead_end children are not assigned a
+ * PMChildSlot.
+ *
+ * Background workers are in this list, too.
+ */
+typedef struct bkend
+{
+ pid_t pid; /* process id of backend */
+ int32 cancel_key; /* cancel key for cancels for this backend */
+ int child_slot; /* PMChildSlot for this backend, if any */
+
+ /*
+ * Flavor of backend or auxiliary process. Note that BACKEND_TYPE_WALSND
+ * backends initially announce themselves as BACKEND_TYPE_NORMAL, so if
+ * bkend_type is normal, you should check for a recent transition.
+ */
+ int bkend_type;
+ bool dead_end; /* is it going to send an error and quit? */
+ bool bgworker_notify; /* gets bgworker start/stop notifications */
+ dlist_node elem; /* list link in BackendList */
+} Backend;
+
/* GUC options */
extern bool EnableSSL;
extern int ReservedBackends;
@@ -47,21 +82,31 @@ extern int postmaster_alive_fds[2];
extern PGDLLIMPORT const char *progname;
extern void PostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern void ClosePostmasterPorts(bool am_syslogger);
+extern void ClosePostmasterPorts(void);
extern void InitProcessGlobals(void);
extern int MaxLivePostmasterChildren(void);
extern bool PostmasterMarkPIDForWorkerNotify(int);
+extern bool RandomCancelKey(int32 *cancel_key);
+
+extern int BackendPrep(int argc, char *argv[]);
+extern void BackendMain(int argc, char *argv[]);
+extern void BackendParentMain(int argc, char *argv[]);
+extern bool BackendCleanup(int argc, char *argv[]);
+extern void BackgroundWorkerMain(int argc, char *argv[]);
+
#ifdef EXEC_BACKEND
-extern pid_t postmaster_forkexec(int argc, char *argv[]);
extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
extern Size ShmemBackendArraySize(void);
extern void ShmemBackendArrayAllocation(void);
+extern void ShmemBackendArrayAdd(Backend *bn);
#endif
+extern PGDLLIMPORT dlist_head BackendList;
+
/*
* Note: MAX_BACKENDS is limited to 2^18-1 because that's the width reserved
* for buffer references in buf_internals.h. This limitation could be lifted
@@ -74,4 +119,17 @@ extern void ShmemBackendArrayAllocation(void);
*/
#define MAX_BACKENDS 0x3FFFF
+/*
+ * Possible types of a backend. Beyond being the possible bkend_type values in
+ * struct bkend, these are OR-able request flag bits for SignalSomeChildren()
+ * and CountChildren().
+ */
+#define BACKEND_TYPE_NORMAL 0x0001 /* normal backend */
+#define BACKEND_TYPE_AUTOVAC 0x0002 /* autovacuum worker process */
+#define BACKEND_TYPE_WALSND 0x0004 /* walsender process */
+#define BACKEND_TYPE_BGWORKER 0x0008 /* bgworker process */
+#define BACKEND_TYPE_ALL 0x000F /* OR of all the above */
+
+#define BACKEND_TYPE_WORKER (BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER)
+
#endif /* _POSTMASTER_H */
diff --git a/src/include/postmaster/startup.h b/src/include/postmaster/startup.h
index 9f59c1ffa3..14a646136f 100644
--- a/src/include/postmaster/startup.h
+++ b/src/include/postmaster/startup.h
@@ -13,10 +13,13 @@
#define _STARTUP_H
extern void HandleStartupProcInterrupts(void);
-extern void StartupProcessMain(void) pg_attribute_noreturn();
extern void PreRestoreCommand(void);
extern void PostRestoreCommand(void);
extern bool IsPromoteTriggered(void);
extern void ResetPromoteTriggered(void);
+/* Startup subprocess functions */
+extern void StartupProcessMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern bool StartupCleanup(int argc, char *argv[]);
+
#endif /* _STARTUP_H */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
new file mode 100644
index 0000000000..001f8455a9
--- /dev/null
+++ b/src/include/postmaster/subprocess.h
@@ -0,0 +1,62 @@
+/*-------------------------------------------------------------------------
+ *
+ * subprocess.h
+ *
+ * Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ *
+ * src/include/postmaster/subprocess.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef SUBPROCESS_H
+#define SUBPROCESS_H
+
+#include "postmaster.h"
+
+typedef enum
+{
+ NoProcessType = -1,
+ CheckerType = 0,
+ BootstrapType,
+ StartupType,
+ BgWriterType,
+ CheckpointerType,
+ WalWriterType,
+ WalReceiverType, /* end of Auxiliary Process Forks */
+ AutoVacLauncherType,
+ AutoVacWorkerType,
+ PgstatCollectorType,
+ PgArchiverType,
+ SysLoggerType,
+ BgWorkerType,
+ ClientBackendType,
+
+ NUMSUBPROCESSTYPES /* Must be last! */
+} SubprocessType;
+
+typedef int (*SubprocessInit) (int argc, char *argv[]);
+typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
+typedef bool (*SubprocessCleanup) (int argc, char *argv[]);
+typedef void (*SubprocessParent) (int argc, char *argv[]);
+
+/* Current subprocess initializer */
+extern void InitializeMySubprocess(SubprocessType type);
+
+typedef struct PgSubprocess
+{
+ const char *progname;
+ const char *name;
+ const char *desc;
+ bool needs_shmem;
+ bool needs_aux_proc;
+ bool needs_full_proc;
+ bool keep_postmaster_memcontext;
+ SubprocessInit init;
+ SubprocessEntryPoint entrypoint;
+ SubprocessCleanup cleanup;
+ SubprocessParent parent_main;
+} PgSubprocess;
+
+extern PGDLLIMPORT PgSubprocess *MySubprocess;
+
+#endif /* SUBPROCESS_H */
diff --git a/src/include/postmaster/syslogger.h b/src/include/postmaster/syslogger.h
index 9b7a386dbd..4cbed2ec40 100644
--- a/src/include/postmaster/syslogger.h
+++ b/src/include/postmaster/syslogger.h
@@ -14,7 +14,6 @@
#include <limits.h> /* for PIPE_BUF */
-
/*
* Primitive protocol structure for writing to syslogger pipe(s). The idea
* here is to divide long messages into chunks that are not more than
@@ -83,9 +82,10 @@ extern int SysLogger_Start(void);
extern void write_syslogger_file(const char *buffer, int count, int dest);
-#ifdef EXEC_BACKEND
+/* Subprocess startup functions */
+extern int SysLoggerPrep(int argc, char *argv[]);
+extern void SysLoggerParentMain(int argc, char *argv[]);
extern void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
extern bool CheckLogrotateSignal(void);
extern void RemoveLogrotateSignalFiles(void);
diff --git a/src/include/postmaster/walwriter.h b/src/include/postmaster/walwriter.h
index 011dc26a6e..596c437c54 100644
--- a/src/include/postmaster/walwriter.h
+++ b/src/include/postmaster/walwriter.h
@@ -16,6 +16,6 @@
extern int WalWriterDelay;
extern int WalWriterFlushAfter;
-extern void WalWriterMain(void) pg_attribute_noreturn();
+extern void WalWriterMain(int argc, char *argv[]) pg_attribute_noreturn();
#endif /* _WALWRITER_H */
diff --git a/src/include/replication/walreceiver.h b/src/include/replication/walreceiver.h
index e08afc6548..2025f19ab3 100644
--- a/src/include/replication/walreceiver.h
+++ b/src/include/replication/walreceiver.h
@@ -311,7 +311,7 @@ walrcv_clear_result(WalRcvExecResult *walres)
}
/* prototypes for functions in walreceiver.c */
-extern void WalReceiverMain(void) pg_attribute_noreturn();
+extern void WalReceiverMain(int argc, char *argv[]);
extern void ProcessWalRcvInterrupts(void);
/* prototypes for functions in walreceiverfuncs.c */
--
2.21.0
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
@ 2020-03-02 14:53 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-04 04:56 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
0 siblings, 1 reply; 38+ messages in thread
From: Mike Palmiotto @ 2020-03-02 14:53 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: Andres Freund <andres@anarazel.de>; Alvaro Herrera <alvherre@2ndquadrant.com>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Sat, Feb 29, 2020 at 9:51 PM Mike Palmiotto
<mike.palmiotto@crunchydata.com> wrote:
>
> On Sat, Nov 30, 2019 at 9:15 PM Michael Paquier <michael@paquier.xyz> wrote:
> >
> > On Thu, Oct 03, 2019 at 11:39:37AM -0700, Andres Freund wrote:
> > > Color me unconvinced.
> >
> > The latest comments of the thread have not been addressed yet. so I am
> > marking the patch as returned with feedback. If you think that's not
> > correct, please feel free to update the status of the patch. If you
> > do so, please provide at the same time a rebased version of the patch,
> > as it is failing to apply on HEAD.
>
> I've finally had some time to update the patchset with an attempt at
> addressing Andres' concerns. Note that the current spin has a bug
> which does not allow syslogger to run properly. Additionally, it is
> squashed into one patch again. I intend to split the patch out into
> separate functional patches and fix the syslogger issue, but wanted to
> get this on the ticket for the open CommitFest before it closes. I'll
> go ahead and re-add it and will be pushing updates as I have them.
Okay, here is an updated and rebased patch that passes all regression
tests with and without EXEC_BACKEND. This also treats more of the
issues raised by Andres.
I still need to do the following:
- split giant patch out into separate functional commits
- add translated process descriptions
- fix some variable names here and there (notably the PgSubprocess
struct ".progname" member -- that's a misnomer.
- address some valgrind findings
I should be able to split this out into smaller commits sometime today
and will continue iterating to scratch the other items off the list.
Thanks,
--
Mike Palmiotto
https://crunchydata.com
Attachments: download all (2 files, .tar.gz)
0002-Add-a-hook-to-allow-extensions-to-set-worker-metadat.patch (text/x-patch, 2.1K) download
From a497a8905fa7f25d41f831e83174548315fa658f Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 19 Feb 2019 15:29:33 +0000
Subject: [PATCH 2/2] Add a hook to allow extensions to set worker metadata
This patch implements a hook that allows extensions to modify a worker
process' metadata in backends.
Additional work done by Yuli Khodorkovskiy <yuli@crunchydata.com>
Original discussion here: https://www.postgresql.org/message-id/flat/CAMN686FE0OdZKp9YPO%3DhtC6LnA6aW4r-%2Bjq%3D3Q5RAoFQgW8EtA%40mail.gmail.com
---
src/backend/postmaster/fork_process.c | 11 +++++++++++
src/include/postmaster/fork_process.h | 4 ++++
2 files changed, 15 insertions(+)
diff --git a/src/backend/postmaster/fork_process.c b/src/backend/postmaster/fork_process.c
index def3cee37e..e392f5207e 100644
--- a/src/backend/postmaster/fork_process.c
+++ b/src/backend/postmaster/fork_process.c
@@ -22,6 +22,14 @@
#include "postmaster/fork_process.h"
+/*
+ * This hook allows plugins to get control of the child process following
+ * a successful fork_process(). It can be used to perform some action in
+ * extensions to set metadata in backends (including special backends) upon
+ * setting of the session user.
+ */
+ForkProcess_post_hook_type ForkProcess_post_hook = NULL;
+
#ifndef WIN32
/*
* Wrapper for fork(). Return values are the same as those for fork():
@@ -114,6 +122,9 @@ fork_process(void)
#ifdef USE_OPENSSL
RAND_cleanup();
#endif
+
+ if (ForkProcess_post_hook)
+ (*ForkProcess_post_hook) ();
}
return result;
diff --git a/src/include/postmaster/fork_process.h b/src/include/postmaster/fork_process.h
index a42f859a08..b4d1560b72 100644
--- a/src/include/postmaster/fork_process.h
+++ b/src/include/postmaster/fork_process.h
@@ -20,4 +20,8 @@ extern pid_t fork_process(void);
extern pid_t postmaster_forkexec(int argc, char *argvp[]);
#endif
+/* Hook for plugins to get control after a successful fork_process() */
+typedef void (*ForkProcess_post_hook_type) ();
+extern PGDLLIMPORT ForkProcess_post_hook_type ForkProcess_post_hook;
+
#endif /* FORK_PROCESS_H */
--
2.21.0
0001-Introduce-subprocess-infrastructure.patch (text/x-patch, 103.1K) download
From 682aaa548249be06a6693839f1877946de8ca927 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Fri, 27 Sep 2019 12:28:19 -0400
Subject: [PATCH 1/2] Introduce subprocess infrastructure
This is an initial attempt at coalescing all of the postmaster
subprocess startups into one central framework. The patchset introduces
a MySubprocess global, which contains an PgSubprocess entry from the
process_types array. Each entry has defined entrypoints, cleanup
functions, prep functions, and a few other control-oriented variables.
This is a rework of https://commitfest.postgresql.org/25/2259/, which
attempts to address the feedback from Andres Freund.
---
src/backend/bootstrap/bootstrap.c | 103 +--
src/backend/main/main.c | 1 +
src/backend/postmaster/Makefile | 1 +
src/backend/postmaster/autovacuum.c | 170 +---
src/backend/postmaster/bgworker.c | 157 +++-
src/backend/postmaster/bgwriter.c | 2 +-
src/backend/postmaster/checkpointer.c | 2 +-
src/backend/postmaster/pgarch.c | 83 +-
src/backend/postmaster/pgstat.c | 195 +----
src/backend/postmaster/postmaster.c | 890 +++++++-------------
src/backend/postmaster/startup.c | 8 +-
src/backend/postmaster/subprocess.c | 208 +++++
src/backend/postmaster/syslogger.c | 277 +++---
src/backend/postmaster/walwriter.c | 2 +-
src/backend/replication/walreceiver.c | 2 +-
src/backend/utils/adt/pgstatfuncs.c | 9 +-
src/backend/utils/init/globals.c | 3 +-
src/backend/utils/init/postinit.c | 2 +-
src/include/bootstrap/bootstrap.h | 4 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 8 +-
src/include/postmaster/autovacuum.h | 13 +-
src/include/postmaster/bgworker_internals.h | 7 +-
src/include/postmaster/bgwriter.h | 4 +-
src/include/postmaster/fork_process.h | 6 +
src/include/postmaster/pgarch.h | 9 +-
src/include/postmaster/postmaster.h | 62 +-
src/include/postmaster/startup.h | 5 +-
src/include/postmaster/subprocess.h | 66 ++
src/include/postmaster/syslogger.h | 8 +-
src/include/postmaster/walwriter.h | 2 +-
src/include/replication/walreceiver.h | 2 +-
32 files changed, 1041 insertions(+), 1271 deletions(-)
create mode 100644 src/backend/postmaster/subprocess.c
create mode 100644 src/include/postmaster/subprocess.h
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index bfc629c753..76a84a15f5 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -35,6 +35,7 @@
#include "pgstat.h"
#include "postmaster/bgwriter.h"
#include "postmaster/startup.h"
+#include "postmaster/subprocess.h"
#include "postmaster/walwriter.h"
#include "replication/walreceiver.h"
#include "storage/bufmgr.h"
@@ -56,8 +57,6 @@ uint32 bootstrap_data_checksum_version = 0; /* No checksum */
#define ALLOC(t, c) \
((t *) MemoryContextAllocZero(TopMemoryContext, (unsigned)(c) * sizeof(t)))
-static void CheckerModeMain(void);
-static void BootstrapModeMain(void);
static void bootstrap_signals(void);
static void ShutdownAuxiliaryProcess(int code, Datum arg);
static Form_pg_attribute AllocateAttribute(void);
@@ -314,35 +313,20 @@ AuxiliaryProcessMain(int argc, char *argv[])
proc_exit(1);
}
+ /*
+ * At this point, we should know what kind of process we are. If the
+ * MySubprocess global has not been initialized, just cast and pass along the
+ * MyAuxProcType enum
+ */
+ if (MySubprocess == NULL)
+ InitializeMySubprocess((SubprocessType)MyAuxProcType);
+
/*
* Identify myself via ps
*/
if (IsUnderPostmaster)
{
- const char *statmsg;
-
- switch (MyAuxProcType)
- {
- case StartupProcess:
- statmsg = pgstat_get_backend_desc(B_STARTUP);
- break;
- case BgWriterProcess:
- statmsg = pgstat_get_backend_desc(B_BG_WRITER);
- break;
- case CheckpointerProcess:
- statmsg = pgstat_get_backend_desc(B_CHECKPOINTER);
- break;
- case WalWriterProcess:
- statmsg = pgstat_get_backend_desc(B_WAL_WRITER);
- break;
- case WalReceiverProcess:
- statmsg = pgstat_get_backend_desc(B_WAL_RECEIVER);
- break;
- default:
- statmsg = "??? process";
- break;
- }
- init_ps_display(statmsg, "", "", "");
+ init_ps_display(MySubprocess->desc, "", "", "");
}
/* Acquire configuration parameters, unless inherited from postmaster */
@@ -426,56 +410,10 @@ AuxiliaryProcessMain(int argc, char *argv[])
*/
SetProcessingMode(NormalProcessing);
- switch (MyAuxProcType)
- {
- case CheckerProcess:
- /* don't set signals, they're useless here */
- CheckerModeMain();
- proc_exit(1); /* should never return */
-
- case BootstrapProcess:
-
- /*
- * There was a brief instant during which mode was Normal; this is
- * okay. We need to be in bootstrap mode during BootStrapXLOG for
- * the sake of multixact initialization.
- */
- SetProcessingMode(BootstrapProcessing);
- bootstrap_signals();
- BootStrapXLOG();
- BootstrapModeMain();
- proc_exit(1); /* should never return */
-
- case StartupProcess:
- /* don't set signals, startup process has its own agenda */
- StartupProcessMain();
- proc_exit(1); /* should never return */
-
- case BgWriterProcess:
- /* don't set signals, bgwriter has its own agenda */
- BackgroundWriterMain();
- proc_exit(1); /* should never return */
-
- case CheckpointerProcess:
- /* don't set signals, checkpointer has its own agenda */
- CheckpointerMain();
- proc_exit(1); /* should never return */
-
- case WalWriterProcess:
- /* don't set signals, walwriter has its own agenda */
- InitXLOGAccess();
- WalWriterMain();
- proc_exit(1); /* should never return */
-
- case WalReceiverProcess:
- /* don't set signals, walreceiver has its own agenda */
- WalReceiverMain();
- proc_exit(1); /* should never return */
-
- default:
- elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
- proc_exit(1);
- }
+ /* Now jump into the subprocess main function and never look back! */
+ MySubprocess->entrypoint(argc, argv);
+
+ proc_exit(1); /* should never return */
}
/*
@@ -484,8 +422,8 @@ AuxiliaryProcessMain(int argc, char *argv[])
* settings). Since, in fact, that was already done by BaseInit(),
* we have nothing more to do here.
*/
-static void
-CheckerModeMain(void)
+void
+CheckerModeMain(int argc, char *argv[])
{
proc_exit(0);
}
@@ -497,11 +435,15 @@ CheckerModeMain(void)
* The bootstrap backend doesn't speak SQL, but instead expects
* commands in a special bootstrap language.
*/
-static void
-BootstrapModeMain(void)
+void
+BootstrapModeMain(int argc, char *argv[])
{
int i;
+ SetProcessingMode(BootstrapProcessing);
+ bootstrap_signals();
+ BootStrapXLOG();
+
Assert(!IsUnderPostmaster);
Assert(IsBootstrapProcessingMode());
@@ -516,7 +458,6 @@ BootstrapModeMain(void)
* Do backend-like initialization for bootstrap mode
*/
InitProcess();
-
InitPostgres(NULL, InvalidOid, NULL, InvalidOid, NULL, false);
/* Initialize stuff for bootstrap-file processing */
diff --git a/src/backend/main/main.c b/src/backend/main/main.c
index da3dae9e25..68b8b3a918 100644
--- a/src/backend/main/main.c
+++ b/src/backend/main/main.c
@@ -35,6 +35,7 @@
#include "common/username.h"
#include "port/atomics.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "storage/s_lock.h"
#include "storage/spin.h"
#include "tcop/tcopprot.h"
diff --git a/src/backend/postmaster/Makefile b/src/backend/postmaster/Makefile
index bfdf6a833d..52b4c2142c 100644
--- a/src/backend/postmaster/Makefile
+++ b/src/backend/postmaster/Makefile
@@ -23,6 +23,7 @@ OBJS = \
pgstat.o \
postmaster.o \
startup.o \
+ subprocess.o \
syslogger.o \
walwriter.o
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index e3a43d3296..78a4031b40 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -83,7 +83,6 @@
#include "nodes/makefuncs.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "storage/bufmgr.h"
@@ -95,6 +94,7 @@
#include "storage/procsignal.h"
#include "storage/sinvaladt.h"
#include "storage/smgr.h"
+#include "postmaster/subprocess.h"
#include "tcop/tcopprot.h"
#include "utils/fmgroids.h"
#include "utils/fmgrprotos.h"
@@ -135,8 +135,8 @@ int Log_autovacuum_min_duration = -1;
#define MAX_AUTOVAC_SLEEPTIME 300 /* seconds */
/* Flags to tell if we are in an autovacuum process */
-static bool am_autovacuum_launcher = false;
-static bool am_autovacuum_worker = false;
+bool am_autovacuum_launcher = false;
+bool am_autovacuum_worker = false;
/* Flags set by signal handlers */
static volatile sig_atomic_t got_SIGUSR2 = false;
@@ -302,13 +302,6 @@ static WorkerInfo MyWorkerInfo = NULL;
/* PID of launcher, valid only in worker while shutting down */
int AutovacuumLauncherPid = 0;
-#ifdef EXEC_BACKEND
-static pid_t avlauncher_forkexec(void);
-static pid_t avworker_forkexec(void);
-#endif
-NON_EXEC_STATIC void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-NON_EXEC_STATIC void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
-
static Oid do_start_worker(void);
static void HandleAutoVacLauncherInterrupts(void);
static void AutoVacLauncherShutdown(void) pg_attribute_noreturn();
@@ -346,95 +339,22 @@ static void autovac_report_workitem(AutoVacuumWorkItem *workitem,
static void avl_sigusr2_handler(SIGNAL_ARGS);
static void autovac_refresh_stats(void);
-
-
/********************************************************************
* AUTOVACUUM LAUNCHER CODE
********************************************************************/
-#ifdef EXEC_BACKEND
-/*
- * forkexec routine for the autovacuum launcher process.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avlauncher_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavlauncher";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
-/*
- * We need this set from the outside, before InitProcess is called
- */
-void
-AutovacuumLauncherIAm(void)
-{
- am_autovacuum_launcher = true;
-}
-#endif
-
-/*
- * Main entry point for autovacuum launcher process, to be called from the
- * postmaster.
- */
-int
-StartAutoVacLauncher(void)
-{
- pid_t AutoVacPID;
-
-#ifdef EXEC_BACKEND
- switch ((AutoVacPID = avlauncher_forkexec()))
-#else
- switch ((AutoVacPID = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum launcher process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacLauncherMain(0, NULL);
- break;
-#endif
- default:
- return (int) AutoVacPID;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
/*
* Main loop for the autovacuum launcher process.
*/
-NON_EXEC_STATIC void
-AutoVacLauncherMain(int argc, char *argv[])
+void
+AutoVacLauncherMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
am_autovacuum_launcher = true;
/* Identify myself via ps */
- init_ps_display(pgstat_get_backend_desc(B_AUTOVAC_LAUNCHER), "", "", "");
+ init_ps_display(MySubprocess->desc, "", "", "");
ereport(DEBUG1,
(errmsg("autovacuum launcher started")));
@@ -1423,83 +1343,11 @@ avl_sigusr2_handler(SIGNAL_ARGS)
* AUTOVACUUM WORKER CODE
********************************************************************/
-#ifdef EXEC_BACKEND
-/*
- * forkexec routines for the autovacuum worker.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avworker_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavworker";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
-/*
- * We need this set from the outside, before InitProcess is called
- */
-void
-AutovacuumWorkerIAm(void)
-{
- am_autovacuum_worker = true;
-}
-#endif
-
-/*
- * Main entry point for autovacuum worker process.
- *
- * This code is heavily based on pgarch.c, q.v.
- */
-int
-StartAutoVacWorker(void)
-{
- pid_t worker_pid;
-
-#ifdef EXEC_BACKEND
- switch ((worker_pid = avworker_forkexec()))
-#else
- switch ((worker_pid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum worker process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacWorkerMain(0, NULL);
- break;
-#endif
- default:
- return (int) worker_pid;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
/*
* AutoVacWorkerMain
*/
-NON_EXEC_STATIC void
-AutoVacWorkerMain(int argc, char *argv[])
+void
+AutoVacWorkerMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
Oid dbid;
@@ -1507,7 +1355,7 @@ AutoVacWorkerMain(int argc, char *argv[])
am_autovacuum_worker = true;
/* Identify myself via ps */
- init_ps_display(pgstat_get_backend_desc(B_AUTOVAC_WORKER), "", "", "");
+ init_ps_display(MySubprocess->desc, "", "", "");
SetProcessingMode(InitProcessing);
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index 75fc0d5d33..7993a0e360 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -33,8 +33,10 @@
#include "storage/shmem.h"
#include "tcop/tcopprot.h"
#include "utils/ascii.h"
+#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/timeout.h"
+#include "utils/timestamp.h"
/*
* The postmaster's list of registered background workers, in private memory.
@@ -133,7 +135,106 @@ static const struct
/* Private functions. */
static bgworker_main_type LookupBackgroundWorkerFunction(const char *libraryname, const char *funcname);
+static bool assign_backendlist_entry(RegisteredBgWorker *rw);
+bool BackgroundWorkerCleanup(int argc, char *argv[]);
+void BackgroundWorkerParentMain(int argc, char *argv[]);
+
+/*
+ * Allocate the Backend struct for a connected background worker, but don't
+ * add it to the list of backends just yet.
+ *
+ * On failure, return false without changing any worker state.
+ *
+ * Some info from the Backend is copied into the passed rw.
+ */
+static bool
+assign_backendlist_entry(RegisteredBgWorker *rw)
+{
+ Backend *bn;
+
+ /*
+ * Compute the cancel key that will be assigned to this session. We
+ * probably don't need cancel keys for background workers, but we'd better
+ * have something random in the field to prevent unfriendly people from
+ * sending cancels to them.
+ */
+ if (!RandomCancelKey(&MyCancelKey))
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random cancel key")));
+ return false;
+ }
+
+ bn = malloc(sizeof(Backend));
+ if (bn == NULL)
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+ return false;
+ }
+
+ bn->cancel_key = MyCancelKey;
+ bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
+ bn->bkend_type = BACKEND_TYPE_BGWORKER;
+ bn->dead_end = false;
+ bn->bgworker_notify = false;
+
+ rw->rw_backend = bn;
+ rw->rw_child_slot = bn->child_slot;
+
+ return true;
+}
+
+/*
+ * BgWorkerPrep
+ *
+ * Postmaster subroutine to prepare a bgworker subprocess.
+ *
+ * Returns 0 on success or -1 on failure.
+ *
+ * Note: if fail, we will be called again from the postmaster main loop.
+ */
+int
+BgWorkerPrep(int argc, char *argv[])
+{
+ RegisteredBgWorker *rw = CurrentBgWorker;
+#ifdef EXEC_BACKEND
+ char forkav[MAXPGPATH];
+#endif
+
+ Assert(CurrentBgWorker);
+ Assert(rw->rw_pid == 0);
+
+ /*
+ * Allocate and assign the Backend element. Note we must do this before
+ * forking, so that we can handle out of memory properly.
+ *
+ * Treat failure as though the worker had crashed. That way, the
+ * postmaster will wait a bit before attempting to start it again; if it
+ * tried again right away, most likely it'd find itself repeating the
+ * out-of-memory or fork failure condition.
+ */
+ if (!assign_backendlist_entry(rw))
+ {
+ rw->rw_crashed_at = GetCurrentTimestamp();
+ return -1;
+ }
+
+ ereport(DEBUG1,
+ (errmsg("starting background worker process \"%s\"",
+ rw->rw_worker.bgw_name)));
+
+#ifdef EXEC_BACKEND
+ /* Rewrite bgworker fork parameter with shmem_slot */
+ snprintf(forkav, MAXPGPATH, "%s=%d", argv[1], rw->rw_shmem_slot);
+ argv[1] = forkav;
+#endif
+
+ return 0;
+}
/*
* Calculate shared memory needed.
@@ -677,12 +778,28 @@ bgworker_sigusr1_handler(SIGNAL_ARGS)
* postmaster.
*/
void
-StartBackgroundWorker(void)
+BackgroundWorkerMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
- BackgroundWorker *worker = MyBgworkerEntry;
+ BackgroundWorker *worker;
bgworker_main_type entrypt;
+#ifndef EXEC_BACKEND
+ /*
+ * Before blowing away PostmasterContext, save this bgworker's
+ * data where it can find it.
+ */
+ MyBgworkerEntry = (BackgroundWorker *)
+ MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
+ memcpy(MyBgworkerEntry, &(CurrentBgWorker->rw_worker), sizeof(BackgroundWorker));
+
+ /* Now we can release postmaster's working memory context */
+ MemoryContextSwitchTo(TopMemoryContext);
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+#endif
+
+ worker = MyBgworkerEntry;
if (worker == NULL)
elog(FATAL, "unable to find bgworker entry");
@@ -816,6 +933,42 @@ StartBackgroundWorker(void)
proc_exit(0);
}
+bool
+BackgroundWorkerCleanup(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ RegisteredBgWorker *rw = CurrentBgWorker;
+
+ /* in postmaster, fork failed ... */
+ ereport(LOG,
+ (errmsg("could not fork worker process: %m")));
+
+ /* undo what assign_backendlist_entry did */
+ ReleasePostmasterChildSlot(rw->rw_child_slot);
+ rw->rw_child_slot = 0;
+ free(rw->rw_backend);
+ rw->rw_backend = NULL;
+ /* mark entry as crashed, so we'll try again later */
+ rw->rw_crashed_at = GetCurrentTimestamp();
+
+ /* Don't panic. */
+ return false;
+}
+
+void
+BackgroundWorkerParentMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ RegisteredBgWorker *rw = CurrentBgWorker;
+
+ rw->rw_pid = MyChildProcPid;
+ rw->rw_backend->pid = rw->rw_pid;
+ ReportBackgroundWorkerPID(rw);
+ /* add new worker to lists of backends */
+ dlist_push_head(&BackendList, &(rw->rw_backend->elem));
+#ifdef EXEC_BACKEND
+ ShmemBackendArrayAdd(rw->rw_backend);
+#endif
+}
+
/*
* Register a new static background worker.
*
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 069e27e427..b2170938f3 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -91,7 +91,7 @@ static XLogRecPtr last_snapshot_lsn = InvalidXLogRecPtr;
* basic execution environment, but not enabled signals yet.
*/
void
-BackgroundWriterMain(void)
+BackgroundWriterMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
MemoryContext bgwriter_context;
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index e354a78725..83f7be1b1e 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -180,7 +180,7 @@ static void ReqCheckpointHandler(SIGNAL_ARGS);
* basic execution environment, but not enabled signals yet.
*/
void
-CheckpointerMain(void)
+CheckpointerMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
MemoryContext checkpointer_context;
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 3ca30badb2..02c98d5d9f 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -91,11 +91,7 @@ static volatile sig_atomic_t ready_to_stop = false;
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgarch_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
+void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgarch_exit(SIGNAL_ARGS);
static void pgarch_waken(SIGNAL_ARGS);
static void pgarch_waken_stop(SIGNAL_ARGS);
@@ -112,7 +108,7 @@ static void pgarch_archiveDone(char *xlog);
*/
/*
- * pgarch_start
+ * PgArchiverPrep
*
* Called from postmaster at startup or after an existing archiver
* died. Attempt to fire up a fresh archiver process.
@@ -122,10 +118,9 @@ static void pgarch_archiveDone(char *xlog);
* Note: if fail, we will be called again from the postmaster main loop.
*/
int
-pgarch_start(void)
+PgArchiverPrep(int argc, char *argv[])
{
time_t curtime;
- pid_t pgArchPid;
/*
* Do nothing if no archiver needed
@@ -140,43 +135,10 @@ pgarch_start(void)
* the postmaster main loop, we will get another chance later.
*/
curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgarch_start_time) <
+ if ((unsigned int) (curtime - last_pgarch_start_time) >=
(unsigned int) PGARCH_RESTART_INTERVAL)
- return 0;
- last_pgarch_start_time = curtime;
-
-#ifdef EXEC_BACKEND
- switch ((pgArchPid = pgarch_forkexec()))
-#else
- switch ((pgArchPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork archiver: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgArchiverMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgArchPid;
- }
+ last_pgarch_start_time = curtime;
- /* shouldn't get here */
return 0;
}
@@ -186,42 +148,19 @@ pgarch_start(void)
*/
-#ifdef EXEC_BACKEND
-
-/*
- * pgarch_forkexec() -
- *
- * Format up the arglist for, then fork and exec, archive process
- */
-static pid_t
-pgarch_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
-
- av[ac++] = "--forkarch";
-
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
/*
* PgArchiverMain
*
* The argc/argv parameters are valid only in EXEC_BACKEND case. However,
* since we don't use 'em, it hardly matters...
*/
-NON_EXEC_STATIC void
-PgArchiverMain(int argc, char *argv[])
+void
+PgArchiverMain(pg_attribute_unused() int argc, pg_attribute_unused() char *arg[])
{
+ /* Drop our connection to postmaster's shared memory, as well */
+ dsm_detach_all();
+ PGSharedMemoryDetach();
+
/*
* Ignore all signals usually bound to some action in the postmaster,
* except for SIGHUP, SIGTERM, SIGUSR1, SIGUSR2, and SIGQUIT.
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 462b4d7e06..1df3d83f3f 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -61,6 +61,7 @@
#include "storage/pg_shmem.h"
#include "storage/procsignal.h"
#include "storage/sinvaladt.h"
+#include "postmaster/subprocess.h"
#include "utils/ascii.h"
#include "utils/guc.h"
#include "utils/memutils.h"
@@ -275,11 +276,7 @@ static instr_time total_func_time;
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgstat_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
+void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgstat_beshutdown_hook(int code, Datum arg);
static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
@@ -685,46 +682,18 @@ pgstat_reset_all(void)
pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
}
-#ifdef EXEC_BACKEND
/*
- * pgstat_forkexec() -
- *
- * Format up the arglist for, then fork and exec, statistics collector process
- */
-static pid_t
-pgstat_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkcol";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
-/*
- * pgstat_start() -
+ * PgstatCollectorPrep
*
* Called from postmaster at startup or after an existing collector
* died. Attempt to fire up a fresh statistics collector.
*
- * Returns PID of child process, or 0 if fail.
- *
- * Note: if fail, we will be called again from the postmaster main loop.
*/
int
-pgstat_start(void)
+PgstatCollectorPrep(int argc, char *argv[])
{
time_t curtime;
- pid_t pgStatPid;
/*
* Check that the socket is there, else pgstat_init failed and we can do
@@ -733,53 +702,18 @@ pgstat_start(void)
if (pgStatSock == PGINVALID_SOCKET)
return 0;
+ curtime = time(NULL);
+
/*
* Do nothing if too soon since last collector start. This is a safety
* valve to protect against continuous respawn attempts if the collector
* is dying immediately at launch. Note that since we will be re-called
* from the postmaster main loop, we will get another chance later.
*/
- curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgstat_start_time) <
+ if ((unsigned int) (curtime - last_pgstat_start_time) >=
(unsigned int) PGSTAT_RESTART_INTERVAL)
- return 0;
- last_pgstat_start_time = curtime;
-
- /*
- * Okay, fork off the collector.
- */
-#ifdef EXEC_BACKEND
- switch ((pgStatPid = pgstat_forkexec()))
-#else
- switch ((pgStatPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork statistics collector: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
+ last_pgstat_start_time = curtime;
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgstatCollectorMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgStatPid;
- }
-
- /* shouldn't get here */
return 0;
}
@@ -2893,61 +2827,6 @@ pgstat_bestart(void)
*/
lbeentry.st_procpid = MyProcPid;
- if (MyBackendId != InvalidBackendId)
- {
- if (IsAutoVacuumLauncherProcess())
- {
- /* Autovacuum Launcher */
- lbeentry.st_backendType = B_AUTOVAC_LAUNCHER;
- }
- else if (IsAutoVacuumWorkerProcess())
- {
- /* Autovacuum Worker */
- lbeentry.st_backendType = B_AUTOVAC_WORKER;
- }
- else if (am_walsender)
- {
- /* Wal sender */
- lbeentry.st_backendType = B_WAL_SENDER;
- }
- else if (IsBackgroundWorker)
- {
- /* bgworker */
- lbeentry.st_backendType = B_BG_WORKER;
- }
- else
- {
- /* client-backend */
- lbeentry.st_backendType = B_BACKEND;
- }
- }
- else
- {
- /* Must be an auxiliary process */
- Assert(MyAuxProcType != NotAnAuxProcess);
- switch (MyAuxProcType)
- {
- case StartupProcess:
- lbeentry.st_backendType = B_STARTUP;
- break;
- case BgWriterProcess:
- lbeentry.st_backendType = B_BG_WRITER;
- break;
- case CheckpointerProcess:
- lbeentry.st_backendType = B_CHECKPOINTER;
- break;
- case WalWriterProcess:
- lbeentry.st_backendType = B_WAL_WRITER;
- break;
- case WalReceiverProcess:
- lbeentry.st_backendType = B_WAL_RECEIVER;
- break;
- default:
- elog(FATAL, "unrecognized process type: %d",
- (int) MyAuxProcType);
- }
- }
-
lbeentry.st_proc_start_timestamp = MyStartTimestamp;
lbeentry.st_activity_start_timestamp = 0;
lbeentry.st_state_start_timestamp = 0;
@@ -2955,9 +2834,9 @@ pgstat_bestart(void)
lbeentry.st_databaseid = MyDatabaseId;
/* We have userid for client-backends, wal-sender and bgworker processes */
- if (lbeentry.st_backendType == B_BACKEND
- || lbeentry.st_backendType == B_WAL_SENDER
- || lbeentry.st_backendType == B_BG_WORKER)
+ if (MySubprocessType == ClientBackendType
+ || MySubprocessType == WalSenderType
+ || MySubprocessType == BgWorkerType)
lbeentry.st_userid = GetSessionUserId();
else
lbeentry.st_userid = InvalidOid;
@@ -4269,48 +4148,6 @@ pgstat_get_crashed_backend_activity(int pid, char *buffer, int buflen)
return NULL;
}
-const char *
-pgstat_get_backend_desc(BackendType backendType)
-{
- const char *backendDesc = "unknown process type";
-
- switch (backendType)
- {
- case B_AUTOVAC_LAUNCHER:
- backendDesc = "autovacuum launcher";
- break;
- case B_AUTOVAC_WORKER:
- backendDesc = "autovacuum worker";
- break;
- case B_BACKEND:
- backendDesc = "client backend";
- break;
- case B_BG_WORKER:
- backendDesc = "background worker";
- break;
- case B_BG_WRITER:
- backendDesc = "background writer";
- break;
- case B_CHECKPOINTER:
- backendDesc = "checkpointer";
- break;
- case B_STARTUP:
- backendDesc = "startup";
- break;
- case B_WAL_RECEIVER:
- backendDesc = "walreceiver";
- break;
- case B_WAL_SENDER:
- backendDesc = "walsender";
- break;
- case B_WAL_WRITER:
- backendDesc = "walwriter";
- break;
- }
-
- return backendDesc;
-}
-
/* ------------------------------------------------------------
* Local support functions follow
* ------------------------------------------------------------
@@ -4423,13 +4260,19 @@ pgstat_send_bgwriter(void)
* The argc/argv parameters are valid only in EXEC_BACKEND case.
* ----------
*/
-NON_EXEC_STATIC void
-PgstatCollectorMain(int argc, char *argv[])
+void
+PgstatCollectorMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
int len;
PgStat_Msg msg;
int wr;
+#ifndef EXEC_BACKEND
+ /* Drop our connection to postmaster's shared memory, as well */
+ dsm_detach_all();
+ PGSharedMemoryDetach();
+#endif
+
/*
* Ignore all signals usually bound to some action in the postmaster,
* except SIGHUP and SIGQUIT. Note we don't need a SIGUSR1 handler to
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 55187eb910..dcfc6990c1 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -114,6 +114,7 @@
#include "postmaster/fork_process.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "postmaster/syslogger.h"
#include "replication/logicallauncher.h"
#include "replication/walsender.h"
@@ -136,62 +137,19 @@
#include "storage/spin.h"
#endif
-
-/*
- * Possible types of a backend. Beyond being the possible bkend_type values in
- * struct bkend, these are OR-able request flag bits for SignalSomeChildren()
- * and CountChildren().
- */
-#define BACKEND_TYPE_NORMAL 0x0001 /* normal backend */
-#define BACKEND_TYPE_AUTOVAC 0x0002 /* autovacuum worker process */
-#define BACKEND_TYPE_WALSND 0x0004 /* walsender process */
-#define BACKEND_TYPE_BGWORKER 0x0008 /* bgworker process */
-#define BACKEND_TYPE_ALL 0x000F /* OR of all the above */
-
-#define BACKEND_TYPE_WORKER (BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER)
-
-/*
- * List of active backends (or child processes anyway; we don't actually
- * know whether a given child has become a backend or is still in the
- * authorization phase). This is used mainly to keep track of how many
- * children we have and send them appropriate signals when necessary.
- *
- * "Special" children such as the startup, bgwriter and autovacuum launcher
- * tasks are not in this list. Autovacuum worker and walsender are in it.
- * Also, "dead_end" children are in it: these are children launched just for
- * the purpose of sending a friendly rejection message to a would-be client.
- * We must track them because they are attached to shared memory, but we know
- * they will never become live backends. dead_end children are not assigned a
- * PMChildSlot.
- *
- * Background workers are in this list, too.
- */
-typedef struct bkend
-{
- pid_t pid; /* process id of backend */
- int32 cancel_key; /* cancel key for cancels for this backend */
- int child_slot; /* PMChildSlot for this backend, if any */
-
- /*
- * Flavor of backend or auxiliary process. Note that BACKEND_TYPE_WALSND
- * backends initially announce themselves as BACKEND_TYPE_NORMAL, so if
- * bkend_type is normal, you should check for a recent transition.
- */
- int bkend_type;
- bool dead_end; /* is it going to send an error and quit? */
- bool bgworker_notify; /* gets bgworker start/stop notifications */
- dlist_node elem; /* list link in BackendList */
-} Backend;
-
-static dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
+dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
#ifdef EXEC_BACKEND
static Backend *ShmemBackendArray;
#endif
BackgroundWorker *MyBgworkerEntry = NULL;
+RegisteredBgWorker *CurrentBgWorker = NULL;
+static Backend *MyBackend;
+static int child_errno;
-
+/* Struct containing postmaster subprocess control info */
+SubprocessType MySubprocessType;
/* The socket number we are listening for connections on */
int PostPortNumber;
@@ -403,25 +361,25 @@ static void BackendInitialize(Port *port);
static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
-static int BackendStartup(Port *port);
static int ProcessStartupPacket(Port *port, bool secure_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
static int initMasks(fd_set *rmask);
static void report_fork_failure_to_client(Port *port, int errnum);
static CAC_state canAcceptConnections(int backend_type);
-static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+#ifdef EXEC_BACKEND
+static void shmemSetup(void);
+#endif
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
-static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
-static pid_t StartChildProcess(AuxProcType type);
+static pid_t StartSubprocess(SubprocessType type);
static void StartAutovacuumWorker(void);
static void MaybeStartWalReceiver(void);
static void InitPostmasterDeathWatchHandle(void);
@@ -455,8 +413,7 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(Port *port);
-static pid_t internal_forkexec(int argc, char *argv[], Port *port);
+static pid_t internal_forkexec(int argc, char *argv[]);
/* Type for a socket that can be inherited to a client process */
#ifdef WIN32
@@ -507,6 +464,9 @@ typedef struct
TimestampTz PgStartTime;
TimestampTz PgReloadTime;
pg_time_t first_syslogger_file_time;
+
+ SubprocessType MySubprocessType;
+
bool redirection_done;
bool IsBinaryUpgrade;
int max_safe_fds;
@@ -534,15 +494,10 @@ static bool save_backend_variables(BackendParameters *param, Port *port,
HANDLE childProcess, pid_t childPid);
#endif
-static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
-#define StartupDataBase() StartChildProcess(StartupProcess)
-#define StartBackgroundWriter() StartChildProcess(BgWriterProcess)
-#define StartCheckpointer() StartChildProcess(CheckpointerProcess)
-#define StartWalWriter() StartChildProcess(WalWriterProcess)
-#define StartWalReceiver() StartChildProcess(WalReceiverProcess)
+#define BGWORKER_LEN 15
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -560,6 +515,129 @@ int postmaster_alive_fds[2] = {-1, -1};
HANDLE PostmasterHandle;
#endif
+static Port *ConnProcPort = NULL;
+
+/*
+ * BackendMain
+ *
+ * Child code when forking a Backend.
+ */
+void BackendMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ /*
+ * Perform additional initialization and collect startup packet.
+ *
+ * We want to do this before InitProcess() for a couple of reasons: 1.
+ * so that we aren't eating up a PGPROC slot while waiting on the
+ * client. 2. so that if InitProcess() fails due to being out of
+ * PGPROC slots, we have already initialized libpq and are able to
+ * report the error to the client.
+ */
+ BackendInitialize(ConnProcPort);
+
+ /* And run the backend */
+ BackendRun(ConnProcPort); /* does not return */
+}
+
+/*
+ * BackendPostmasterMain
+ *
+ * Parent code when forking a Backend.
+ */
+void BackendParentMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ /* in parent, successful fork */
+ ereport(DEBUG2,
+ (errmsg_internal("forked new backend, pid=%d socket=%d",
+ (int) MyChildProcPid, (int) MyProcPort->sock)));
+
+ /*
+ * Everything's been successful, it's safe to add this backend to our list
+ * of backends.
+ */
+ MyBackend->pid = MyChildProcPid;
+ MyBackend->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
+ dlist_push_head(&BackendList, &MyBackend->elem);
+
+#ifdef EXEC_BACKEND
+ if (!MyBackend->dead_end)
+ ShmemBackendArrayAdd(MyBackend);
+#endif
+}
+
+/*
+ * BackendCleanup
+ *
+ * Backend cleanup in case a failure occurs forking a new Backend.
+ */
+bool
+BackendCleanup(int argc, char *argv[])
+{
+ if (!MyBackend->dead_end)
+ (void) ReleasePostmasterChildSlot(MyBackend->child_slot);
+ free(MyBackend);
+
+ report_fork_failure_to_client(MyProcPort, child_errno);
+
+ /* Don't panic */
+ return false;
+}
+
+/*
+ * BackendPrep
+ *
+ * Prepare a ForkProcType struct for starting a Backend.
+ * This does all prep related to av parameters and error messages.
+*/
+int
+BackendPrep(int argc, char *argv[])
+{
+ /*
+ * Create backend data structure. Better before the fork() so we can
+ * handle failure cleanly.
+ */
+ MyBackend = (Backend *) malloc(sizeof(Backend));
+ if (!MyBackend)
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+ }
+
+ /*
+ * Compute the cancel key that will be assigned to this backend. The
+ * backend will have its own copy in the forked-off process' value of
+ * MyCancelKey, so that it can transmit the key to the frontend.
+ */
+ if (!RandomCancelKey(&MyCancelKey))
+ {
+ free(MyBackend);
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random cancel key")));
+ }
+
+ MyBackend->cancel_key = MyCancelKey;
+
+ /* Pass down canAcceptConnections state */
+ ConnProcPort->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
+ MyBackend->dead_end = (ConnProcPort->canAcceptConnections != CAC_OK &&
+ ConnProcPort->canAcceptConnections != CAC_WAITBACKUP);
+
+ /*
+ * Unless it's a dead_end child, assign it a child slot number
+ */
+ if (!MyBackend->dead_end)
+ MyBackend->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
+ else
+ MyBackend->child_slot = 0;
+
+ /* Hasn't asked to be notified about any bgworkers yet */
+ MyBackend->bgworker_notify = false;
+
+ return 0;
+}
+
/*
* Postmaster main entry point
*/
@@ -1083,7 +1161,7 @@ PostmasterMain(int argc, char *argv[])
/*
* If enabled, start up syslogger collection subprocess
*/
- SysLoggerPID = SysLogger_Start();
+ SysLoggerPID = StartSubprocess(SysLoggerType);
/*
* Reset whereToSendOutput from DestDebug (its starting state) to
@@ -1389,7 +1467,7 @@ PostmasterMain(int argc, char *argv[])
/*
* We're ready to rock and roll...
*/
- StartupPID = StartupDataBase();
+ StartupPID = StartSubprocess(StartupType);
Assert(StartupPID != 0);
StartupStatus = STARTUP_RUNNING;
pmState = PM_STARTUP;
@@ -1719,19 +1797,17 @@ ServerLoop(void)
break;
if (FD_ISSET(ListenSocket[i], &rmask))
{
- Port *port;
-
- port = ConnCreate(ListenSocket[i]);
- if (port)
+ ConnProcPort = ConnCreate(ListenSocket[i]);
+ if (ConnProcPort)
{
- BackendStartup(port);
+ StartSubprocess(ClientBackendType);
/*
* We no longer need the open socket or port structure
* in this process
*/
- StreamClose(port->sock);
- ConnFree(port);
+ StreamClose(ConnProcPort->sock);
+ ConnFree(ConnProcPort);
}
}
}
@@ -1739,7 +1815,7 @@ ServerLoop(void)
/* If we have lost the log collector, try to start a new one */
if (SysLoggerPID == 0 && Logging_collector)
- SysLoggerPID = SysLogger_Start();
+ SysLoggerPID = StartSubprocess(SysLoggerType);
/*
* If no background writer process is running, and we are not in a
@@ -1750,9 +1826,9 @@ ServerLoop(void)
pmState == PM_HOT_STANDBY)
{
if (CheckpointerPID == 0)
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
if (BgWriterPID == 0)
- BgWriterPID = StartBackgroundWriter();
+ BgWriterPID = StartSubprocess(BgWriterType);
}
/*
@@ -1761,7 +1837,7 @@ ServerLoop(void)
* be writing any new WAL).
*/
if (WalWriterPID == 0 && pmState == PM_RUN)
- WalWriterPID = StartWalWriter();
+ WalWriterPID = StartSubprocess(WalWriterType);
/*
* If we have lost the autovacuum launcher, try to start a new one. We
@@ -1773,7 +1849,7 @@ ServerLoop(void)
(AutoVacuumingActive() || start_autovac_launcher) &&
pmState == PM_RUN)
{
- AutoVacPID = StartAutoVacLauncher();
+ AutoVacPID = StartSubprocess(AutoVacLauncherType);
if (AutoVacPID != 0)
start_autovac_launcher = false; /* signal processed */
}
@@ -1781,11 +1857,11 @@ ServerLoop(void)
/* If we have lost the stats collector, try to start a new one */
if (PgStatPID == 0 &&
(pmState == PM_RUN || pmState == PM_HOT_STANDBY))
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
/* If we need to signal the autovacuum launcher, do so now */
if (avlauncher_needs_signal)
@@ -2533,12 +2609,9 @@ ConnFree(Port *conn)
* This is called during child process startup to release file descriptors
* that are not needed by that child process. The postmaster still has
* them open, of course.
- *
- * Note: we pass am_syslogger as a boolean because we don't want to set
- * the global variable yet when this is called.
*/
void
-ClosePostmasterPorts(bool am_syslogger)
+ClosePostmasterPorts(void)
{
int i;
@@ -2575,7 +2648,8 @@ ClosePostmasterPorts(bool am_syslogger)
* If using syslogger, close the read side of the pipe. We don't bother
* tracking this in fd.c, either.
*/
- if (!am_syslogger)
+ if (MySubprocessType == SysLoggerType ||
+ MySubprocessType == ClientBackendType)
{
#ifndef WIN32
if (syslogPipe[0] >= 0)
@@ -3037,22 +3111,22 @@ reaper(SIGNAL_ARGS)
* if this fails, we'll just try again later.
*/
if (CheckpointerPID == 0)
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
if (BgWriterPID == 0)
- BgWriterPID = StartBackgroundWriter();
+ BgWriterPID = StartSubprocess(BgWriterType);
if (WalWriterPID == 0)
- WalWriterPID = StartWalWriter();
+ WalWriterPID = StartSubprocess(WalWriterType);
/*
* Likewise, start other special children as needed. In a restart
* situation, some of them may be alive already.
*/
if (!IsBinaryUpgrade && AutoVacuumingActive() && AutoVacPID == 0)
- AutoVacPID = StartAutoVacLauncher();
+ AutoVacPID = StartSubprocess(AutoVacLauncherType);
if (PgArchStartupAllowed() && PgArchPID == 0)
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
if (PgStatPID == 0)
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
/* workers may be scheduled to start now */
maybe_start_bgworkers();
@@ -3198,7 +3272,7 @@ reaper(SIGNAL_ARGS)
LogChildExit(LOG, _("archiver process"),
pid, exitstatus);
if (PgArchStartupAllowed())
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
continue;
}
@@ -3214,7 +3288,7 @@ reaper(SIGNAL_ARGS)
LogChildExit(LOG, _("statistics collector process"),
pid, exitstatus);
if (pmState == PM_RUN || pmState == PM_HOT_STANDBY)
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
continue;
}
@@ -3223,7 +3297,7 @@ reaper(SIGNAL_ARGS)
{
SysLoggerPID = 0;
/* for safety's sake, launch new logger *first* */
- SysLoggerPID = SysLogger_Start();
+ SysLoggerPID = StartSubprocess(SysLoggerType);
if (!EXIT_STATUS_0(exitstatus))
LogChildExit(LOG, _("system logger process"),
pid, exitstatus);
@@ -3855,7 +3929,7 @@ PostmasterStateMachine(void)
Assert(Shutdown > NoShutdown);
/* Start the checkpointer if not running */
if (CheckpointerPID == 0)
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
/* And tell it to shut down */
if (CheckpointerPID != 0)
{
@@ -3996,7 +4070,7 @@ PostmasterStateMachine(void)
reset_shared();
- StartupPID = StartupDataBase();
+ StartupPID = StartSubprocess(StartupType);
Assert(StartupPID != 0);
StartupStatus = STARTUP_RUNNING;
pmState = PM_STARTUP;
@@ -4118,122 +4192,6 @@ TerminateChildren(int signal)
signal_child(PgStatPID, signal);
}
-/*
- * BackendStartup -- start backend process
- *
- * returns: STATUS_ERROR if the fork failed, STATUS_OK otherwise.
- *
- * Note: if you change this code, also consider StartAutovacuumWorker.
- */
-static int
-BackendStartup(Port *port)
-{
- Backend *bn; /* for backend cleanup */
- pid_t pid;
-
- /*
- * Create backend data structure. Better before the fork() so we can
- * handle failure cleanly.
- */
- bn = (Backend *) malloc(sizeof(Backend));
- if (!bn)
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- return STATUS_ERROR;
- }
-
- /*
- * Compute the cancel key that will be assigned to this backend. The
- * backend will have its own copy in the forked-off process' value of
- * MyCancelKey, so that it can transmit the key to the frontend.
- */
- if (!RandomCancelKey(&MyCancelKey))
- {
- free(bn);
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("could not generate random cancel key")));
- return STATUS_ERROR;
- }
-
- bn->cancel_key = MyCancelKey;
-
- /* Pass down canAcceptConnections state */
- port->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
- bn->dead_end = (port->canAcceptConnections != CAC_OK &&
- port->canAcceptConnections != CAC_WAITBACKUP);
-
- /*
- * Unless it's a dead_end child, assign it a child slot number
- */
- if (!bn->dead_end)
- bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
- else
- bn->child_slot = 0;
-
- /* Hasn't asked to be notified about any bgworkers yet */
- bn->bgworker_notify = false;
-
-#ifdef EXEC_BACKEND
- pid = backend_forkexec(port);
-#else /* !EXEC_BACKEND */
- pid = fork_process();
- if (pid == 0) /* child */
- {
- free(bn);
-
- /* Detangle from postmaster */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Perform additional initialization and collect startup packet */
- BackendInitialize(port);
-
- /* And run the backend */
- BackendRun(port);
- }
-#endif /* EXEC_BACKEND */
-
- if (pid < 0)
- {
- /* in parent, fork failed */
- int save_errno = errno;
-
- if (!bn->dead_end)
- (void) ReleasePostmasterChildSlot(bn->child_slot);
- free(bn);
- errno = save_errno;
- ereport(LOG,
- (errmsg("could not fork new process for connection: %m")));
- report_fork_failure_to_client(port, save_errno);
- return STATUS_ERROR;
- }
-
- /* in parent, successful fork */
- ereport(DEBUG2,
- (errmsg_internal("forked new backend, pid=%d socket=%d",
- (int) pid, (int) port->sock)));
-
- /*
- * Everything's been successful, it's safe to add this backend to our list
- * of backends.
- */
- bn->pid = pid;
- bn->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
- dlist_push_head(&BackendList, &bn->elem);
-
-#ifdef EXEC_BACKEND
- if (!bn->dead_end)
- ShmemBackendArrayAdd(bn);
-#endif
-
- return STATUS_OK;
-}
-
/*
* Try to report backend fork() failure to client before we close the
* connection. Since we do not care to risk blocking the postmaster on
@@ -4433,7 +4391,7 @@ BackendInitialize(Port *port)
* init_ps_display() to avoid abusing the parameters like this.
*/
if (am_walsender)
- init_ps_display(pgstat_get_backend_desc(B_WAL_SENDER), port->user_name, remote_ps_data,
+ init_ps_display(MySubprocess->desc, port->user_name, remote_ps_data,
update_process_title ? "authentication" : "");
else
init_ps_display(port->user_name, port->database_name, remote_ps_data,
@@ -4476,7 +4434,7 @@ BackendRun(Port *port)
maxac * sizeof(char *));
ac = 0;
- av[ac++] = "postgres";
+ av[ac++] = pstrdup("postgres");
/*
* Pass any backend switches specified with -o on the postmaster's own
@@ -4527,36 +4485,11 @@ BackendRun(Port *port)
pid_t
postmaster_forkexec(int argc, char *argv[])
{
- Port port;
-
/* This entry point passes dummy values for the Port variables */
- memset(&port, 0, sizeof(port));
- return internal_forkexec(argc, argv, &port);
-}
+ if (!ConnProcPort)
+ ConnProcPort = palloc0(sizeof(*ConnProcPort));
-/*
- * backend_forkexec -- fork/exec off a backend process
- *
- * Some operating systems (WIN32) don't have fork() so we have to simulate
- * it by storing parameters that need to be passed to the child and
- * then create a new child process.
- *
- * returns the pid of the fork/exec'd process, or -1 on failure
- */
-static pid_t
-backend_forkexec(Port *port)
-{
- char *av[4];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkbackend";
- av[ac++] = NULL; /* filled in by internal_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return internal_forkexec(ac, av, port);
+ return internal_forkexec(argc, argv);
}
#ifndef WIN32
@@ -4568,7 +4501,7 @@ backend_forkexec(Port *port)
* - fork():s, and then exec():s the child process
*/
static pid_t
-internal_forkexec(int argc, char *argv[], Port *port)
+internal_forkexec(int argc, char *argv[])
{
static unsigned long tmpBackendFileNum = 0;
pid_t pid;
@@ -4576,7 +4509,7 @@ internal_forkexec(int argc, char *argv[], Port *port)
BackendParameters param;
FILE *fp;
- if (!save_backend_variables(¶m, port))
+ if (!save_backend_variables(¶m, ConnProcPort))
return -1; /* log made by save_backend_variables */
/* Calculate name for temp file */
@@ -4861,6 +4794,28 @@ retry:
}
#endif /* WIN32 */
+/*
+ * shmemSetup
+ *
+ * Helper function for a child to set up shmem before
+ * executing.
+ *
+ * aux_process - set to true if an auxiliary process.
+ */
+static void
+shmemSetup(void)
+{
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
+
+ if (MySubprocess->needs_aux_proc)
+ InitAuxiliaryProcess();
+ else
+ InitProcess();
+
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores();
+}
/*
* SubPostmasterMain -- Get the fork/exec'd process into a state equivalent
@@ -4877,6 +4832,7 @@ void
SubPostmasterMain(int argc, char *argv[])
{
Port port;
+ int shmem_slot;
/* In EXEC_BACKEND case we will not have inherited these settings */
IsPostmasterEnvironment = true;
@@ -4892,12 +4848,15 @@ SubPostmasterMain(int argc, char *argv[])
if (argc < 3)
elog(FATAL, "invalid subpostmaster invocation");
+ Assert(!ConnProcPort);
+
/* Read in the variables file */
memset(&port, 0, sizeof(Port));
read_backend_variables(argv[2], &port);
+ ConnProcPort = &port;
/* Close the postmaster's sockets (as soon as we know them) */
- ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
+ ClosePostmasterPorts();
/*
* Set reference point for stack-depth checking
@@ -4916,6 +4875,9 @@ SubPostmasterMain(int argc, char *argv[])
errmsg("out of memory")));
#endif
+ if (!MySubprocess)
+ InitializeMySubprocess(MySubprocessType);
+
/*
* If appropriate, physically re-attach to shared memory segment. We want
* to do this before going any further to ensure that we can attach at the
@@ -4932,20 +4894,16 @@ SubPostmasterMain(int argc, char *argv[])
* sometimes impossible to attach to shared memory at the desired address.
* Return the setting to its old value (usually '1' or '2') when finished.
*/
- if (strcmp(argv[1], "--forkbackend") == 0 ||
- strcmp(argv[1], "--forkavlauncher") == 0 ||
- strcmp(argv[1], "--forkavworker") == 0 ||
- strcmp(argv[1], "--forkboot") == 0 ||
- strncmp(argv[1], "--forkbgworker=", 15) == 0)
+ if (MySubprocess->needs_shmem)
PGSharedMemoryReAttach();
else
PGSharedMemoryNoReAttach();
/* autovacuum needs this set before calling InitProcess */
if (strcmp(argv[1], "--forkavlauncher") == 0)
- AutovacuumLauncherIAm();
+ am_autovacuum_launcher = true;
if (strcmp(argv[1], "--forkavworker") == 0)
- AutovacuumWorkerIAm();
+ am_autovacuum_worker = true;
/*
* Start our win32 signal implementation. This has to be done after we
@@ -4985,8 +4943,7 @@ SubPostmasterMain(int argc, char *argv[])
*/
process_shared_preload_libraries();
- /* Run backend or appropriate child */
- if (strcmp(argv[1], "--forkbackend") == 0)
+ if (MySubprocessType == ClientBackendType)
{
Assert(argc == 3); /* shouldn't be any more args */
@@ -5024,96 +4981,43 @@ SubPostmasterMain(int argc, char *argv[])
*/
BackendInitialize(&port);
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
+ shmemSetup();
/* And run the backend */
BackendRun(&port); /* does not return */
}
- if (strcmp(argv[1], "--forkboot") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitAuxiliaryProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
- }
- if (strcmp(argv[1], "--forkavlauncher") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
- }
- if (strcmp(argv[1], "--forkavworker") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
- }
- if (strncmp(argv[1], "--forkbgworker=", 15) == 0)
- {
- int shmem_slot;
- /* do this as early as possible; in particular, before InitProcess() */
+ /* do this as early as possible; in particular, before InitProcess() */
+ if (MySubprocessType == BgWorkerType)
IsBackgroundWorker = true;
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
+ shmemSetup();
- /* Fetch MyBgworkerEntry from shared memory */
- shmem_slot = atoi(argv[1] + 15);
- MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
-
- StartBackgroundWorker();
- }
- if (strcmp(argv[1], "--forkarch") == 0)
+ if (MySubprocess->needs_aux_proc)
{
- /* Do not want to attach to shared memory */
-
- PgArchiverMain(argc, argv); /* does not return */
+ AuxiliaryProcessMain(argc-2, argv+2);
}
- if (strcmp(argv[1], "--forkcol") == 0)
- {
- /* Do not want to attach to shared memory */
- PgstatCollectorMain(argc, argv); /* does not return */
- }
- if (strcmp(argv[1], "--forklog") == 0)
+ switch(MySubprocessType)
{
- /* Do not want to attach to shared memory */
-
- SysLoggerMain(argc, argv); /* does not return */
+ case BgWorkerType:
+ /* Fetch MyBgworkerEntry from shared memory */
+ shmem_slot = atoi(argv[1] + 15);
+ MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
+ /* fallthrough */
+ case PgArchiverType:
+ case PgstatCollectorType:
+ case SysLoggerType:
+ MySubprocess->entrypoint(argc, argv); /* no return */
+ break;
+ case AutoVacLauncherType:
+ case AutoVacWorkerType:
+ MySubprocess->entrypoint(argc-2, argv+2);
+ break;
+ default:
+ ereport(LOG,
+ (errmsg("could not start unknown process type (%d) under postmaster",
+ MySubprocessType)));
}
abort(); /* shouldn't get here */
@@ -5197,9 +5101,9 @@ sigusr1_handler(SIGNAL_ARGS)
* we'll just try again later.
*/
Assert(CheckpointerPID == 0);
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
Assert(BgWriterPID == 0);
- BgWriterPID = StartBackgroundWriter();
+ BgWriterPID = StartSubprocess(BgWriterType);
/*
* Start the archiver if we're responsible for (re-)archiving received
@@ -5207,7 +5111,7 @@ sigusr1_handler(SIGNAL_ARGS)
*/
Assert(PgArchPID == 0);
if (XLogArchivingAlways())
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
/*
* If we aren't planning to enter hot standby mode later, treat
@@ -5231,7 +5135,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Likewise, start other special children as needed.
*/
Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5378,7 +5282,7 @@ StartupPacketTimeoutHandler(void)
/*
* Generate a random cancel key.
*/
-static bool
+bool
RandomCancelKey(int32 *cancel_key)
{
return pg_strong_random(cancel_key, sizeof(int32));
@@ -5424,7 +5328,6 @@ CountChildren(int target)
return cnt;
}
-
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5435,93 +5338,110 @@ CountChildren(int target)
* to start subprocess.
*/
static pid_t
-StartChildProcess(AuxProcType type)
+StartSubprocess(SubprocessType type)
{
pid_t pid;
- char *av[10];
- int ac = 0;
+ char *argv[10];
+ int argc = 0;
char typebuf[32];
+#ifdef EXEC_BACKEND
+ char forkname[32];
+#endif
- /*
- * Set up command-line arguments for subprocess
- */
- av[ac++] = "postgres";
+ /* This should be our first time in this function */
+ InitializeMySubprocess(type);
+
+ argv[argc++] = "postgres";
#ifdef EXEC_BACKEND
- av[ac++] = "--forkboot";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
+ snprintf(forkname, sizeof(forkname), "--%s", MySubprocess->progname);
+ argv[argc++] = psprintf("--%s", MySubprocess->progname);
+ argv[argc++] = NULL;
#endif
snprintf(typebuf, sizeof(typebuf), "-x%d", type);
- av[ac++] = typebuf;
+ argv[argc++] = typebuf;
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ argv[argc] = NULL;
+ Assert(argc < lengthof(argv));
+
+ /* Prep the subprocesses for a fork */
+ if (MySubprocess->init)
+ {
+ int rc = 0;
+ rc = MySubprocess->init(argc, argv);
+
+ if (rc != 0)
+ return 0;
+ }
#ifdef EXEC_BACKEND
- pid = postmaster_forkexec(ac, av);
-#else /* !EXEC_BACKEND */
+ pid = postmaster_forkexec(argc, argv);
+#else
pid = fork_process();
+#endif
+ /* some processes like backends and bgworkers need the pid */
+ MyChildProcPid = pid;
+
+#ifndef EXEC_BACKEND
if (pid == 0) /* child */
{
InitPostmasterChild();
+ ClosePostmasterPorts();
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
+ /*
+ * Release postmaster's working memory context unless we're still using
+ * it.
+ */
+ if (!MySubprocess->keep_postmaster_memcontext)
+ {
+ MemoryContextSwitchTo(TopMemoryContext);
+ MemoryContextDelete(PostmasterContext);
+
+ PostmasterContext = NULL;
+ }
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
+ /* Call the process's main subroutine */
+ if (MySubprocess->needs_aux_proc)
+ AuxiliaryProcessMain(argc, argv);
+ else
+ MySubprocess->entrypoint(argc, argv);
- AuxiliaryProcessMain(ac, av);
ExitPostmaster(0);
}
-#endif /* EXEC_BACKEND */
-
+#endif
if (pid < 0)
{
/* in parent, fork failed */
- int save_errno = errno;
+ child_errno = errno;
- errno = save_errno;
- switch (type)
- {
- case StartupProcess:
- ereport(LOG,
- (errmsg("could not fork startup process: %m")));
- break;
- case BgWriterProcess:
- ereport(LOG,
- (errmsg("could not fork background writer process: %m")));
- break;
- case CheckpointerProcess:
- ereport(LOG,
- (errmsg("could not fork checkpointer process: %m")));
- break;
- case WalWriterProcess:
- ereport(LOG,
- (errmsg("could not fork WAL writer process: %m")));
- break;
- case WalReceiverProcess:
- ereport(LOG,
- (errmsg("could not fork WAL receiver process: %m")));
- break;
- default:
- ereport(LOG,
- (errmsg("could not fork process: %m")));
- break;
- }
+ ereport(LOG,
+ (errmsg("could not fork %s: %m", MySubprocess->desc)));
/*
* fork failure is fatal during startup, but there's no need to choke
* immediately if starting other child types fails.
*/
- if (type == StartupProcess)
- ExitPostmaster(1);
+ if (MySubprocess->cleanup)
+ {
+ /*
+ * Panic if the cleanup function tells us to. Otherwise, just bail
+ * with -1.
+ */
+ if (MySubprocess->cleanup(argc, argv))
+ ExitPostmaster(1);
+ else
+ return -1;
+ }
+
return 0;
}
+ else
+ {
+ if (MySubprocess->parent_main)
+ MySubprocess->parent_main(argc, argv);
+ }
/*
* in parent, successful fork
@@ -5576,7 +5496,7 @@ StartAutovacuumWorker(void)
bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
bn->bgworker_notify = false;
- bn->pid = StartAutoVacWorker();
+ bn->pid = StartSubprocess(AutoVacWorkerType);
if (bn->pid > 0)
{
bn->bkend_type = BACKEND_TYPE_AUTOVAC;
@@ -5637,7 +5557,7 @@ MaybeStartWalReceiver(void)
pmState == PM_HOT_STANDBY || pmState == PM_WAIT_READONLY) &&
Shutdown == NoShutdown)
{
- WalReceiverPID = StartWalReceiver();
+ WalReceiverPID = StartSubprocess(WalReceiverType);
if (WalReceiverPID != 0)
WalReceiverRequested = false;
/* else leave the flag set, so we'll try again later */
@@ -5756,124 +5676,6 @@ BackgroundWorkerUnblockSignals(void)
PG_SETMASK(&UnBlockSig);
}
-#ifdef EXEC_BACKEND
-static pid_t
-bgworker_forkexec(int shmem_slot)
-{
- char *av[10];
- int ac = 0;
- char forkav[MAXPGPATH];
-
- snprintf(forkav, MAXPGPATH, "--forkbgworker=%d", shmem_slot);
-
- av[ac++] = "postgres";
- av[ac++] = forkav;
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif
-
-/*
- * Start a new bgworker.
- * Starting time conditions must have been checked already.
- *
- * Returns true on success, false on failure.
- * In either case, update the RegisteredBgWorker's state appropriately.
- *
- * This code is heavily based on autovacuum.c, q.v.
- */
-static bool
-do_start_bgworker(RegisteredBgWorker *rw)
-{
- pid_t worker_pid;
-
- Assert(rw->rw_pid == 0);
-
- /*
- * Allocate and assign the Backend element. Note we must do this before
- * forking, so that we can handle failures (out of memory or child-process
- * slots) cleanly.
- *
- * Treat failure as though the worker had crashed. That way, the
- * postmaster will wait a bit before attempting to start it again; if we
- * tried again right away, most likely we'd find ourselves hitting the
- * same resource-exhaustion condition.
- */
- if (!assign_backendlist_entry(rw))
- {
- rw->rw_crashed_at = GetCurrentTimestamp();
- return false;
- }
-
- ereport(DEBUG1,
- (errmsg("starting background worker process \"%s\"",
- rw->rw_worker.bgw_name)));
-
-#ifdef EXEC_BACKEND
- switch ((worker_pid = bgworker_forkexec(rw->rw_shmem_slot)))
-#else
- switch ((worker_pid = fork_process()))
-#endif
- {
- case -1:
- /* in postmaster, fork failed ... */
- ereport(LOG,
- (errmsg("could not fork worker process: %m")));
- /* undo what assign_backendlist_entry did */
- ReleasePostmasterChildSlot(rw->rw_child_slot);
- rw->rw_child_slot = 0;
- free(rw->rw_backend);
- rw->rw_backend = NULL;
- /* mark entry as crashed, so we'll try again later */
- rw->rw_crashed_at = GetCurrentTimestamp();
- break;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /*
- * Before blowing away PostmasterContext, save this bgworker's
- * data where it can find it.
- */
- MyBgworkerEntry = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(MyBgworkerEntry, &rw->rw_worker, sizeof(BackgroundWorker));
-
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
-
- StartBackgroundWorker();
-
- exit(1); /* should not get here */
- break;
-#endif
- default:
- /* in postmaster, fork successful ... */
- rw->rw_pid = worker_pid;
- rw->rw_backend->pid = rw->rw_pid;
- ReportBackgroundWorkerPID(rw);
- /* add new worker to lists of backends */
- dlist_push_head(&BackendList, &rw->rw_backend->elem);
-#ifdef EXEC_BACKEND
- ShmemBackendArrayAdd(rw->rw_backend);
-#endif
- return true;
- }
-
- return false;
-}
-
/*
* Does the current postmaster state require starting a worker with the
* specified start_time?
@@ -5914,67 +5716,6 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
return false;
}
-/*
- * Allocate the Backend struct for a connected background worker, but don't
- * add it to the list of backends just yet.
- *
- * On failure, return false without changing any worker state.
- *
- * Some info from the Backend is copied into the passed rw.
- */
-static bool
-assign_backendlist_entry(RegisteredBgWorker *rw)
-{
- Backend *bn;
-
- /*
- * Check that database state allows another connection. Currently the
- * only possible failure is CAC_TOOMANY, so we just log an error message
- * based on that rather than checking the error code precisely.
- */
- if (canAcceptConnections(BACKEND_TYPE_BGWORKER) != CAC_OK)
- {
- ereport(LOG,
- (errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("no slot available for new worker process")));
- return false;
- }
-
- /*
- * Compute the cancel key that will be assigned to this session. We
- * probably don't need cancel keys for background workers, but we'd better
- * have something random in the field to prevent unfriendly people from
- * sending cancels to them.
- */
- if (!RandomCancelKey(&MyCancelKey))
- {
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("could not generate random cancel key")));
- return false;
- }
-
- bn = malloc(sizeof(Backend));
- if (bn == NULL)
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- return false;
- }
-
- bn->cancel_key = MyCancelKey;
- bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
- bn->bkend_type = BACKEND_TYPE_BGWORKER;
- bn->dead_end = false;
- bn->bgworker_notify = false;
-
- rw->rw_backend = bn;
- rw->rw_child_slot = bn->child_slot;
-
- return true;
-}
-
/*
* If the time is right, start background worker(s).
*
@@ -6079,7 +5820,8 @@ maybe_start_bgworkers(void)
* crashed, but there's no need because the next run of this
* function will do that.
*/
- if (!do_start_bgworker(rw))
+ CurrentBgWorker = rw;
+ if (StartSubprocess(BgWorkerType) <= 0)
{
StartWorkerNeeded = true;
return;
@@ -6197,6 +5939,8 @@ save_backend_variables(BackendParameters *param, Port *port,
param->PgReloadTime = PgReloadTime;
param->first_syslogger_file_time = first_syslogger_file_time;
+ param->MySubprocessType = MySubprocessType;
+
param->redirection_done = redirection_done;
param->IsBinaryUpgrade = IsBinaryUpgrade;
param->max_safe_fds = max_safe_fds;
@@ -6432,6 +6176,8 @@ restore_backend_variables(BackendParameters *param, Port *port)
PgReloadTime = param->PgReloadTime;
first_syslogger_file_time = param->first_syslogger_file_time;
+ MySubprocessType = param->MySubprocessType;
+
redirection_done = param->redirection_done;
IsBinaryUpgrade = param->IsBinaryUpgrade;
max_safe_fds = param->max_safe_fds;
@@ -6486,7 +6232,7 @@ ShmemBackendArrayAllocation(void)
memset(ShmemBackendArray, 0, size);
}
-static void
+void
ShmemBackendArrayAdd(Backend *bn)
{
/* The array slot corresponding to my PMChildSlot should be free */
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index c2250d7d4e..8957b341b9 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -127,13 +127,19 @@ HandleStartupProcInterrupts(void)
ProcessProcSignalBarrier();
}
+bool
+StartupCleanup(int argc, char *argv[])
+{
+ /* Panic! */
+ return true;
+}
/* ----------------------------------
* Startup Process main entry point
* ----------------------------------
*/
void
-StartupProcessMain(void)
+StartupProcessMain(int argc, char *argv[])
{
/*
* Properly accept or ignore signals the postmaster might send us.
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
new file mode 100644
index 0000000000..1c91c5f0c3
--- /dev/null
+++ b/src/backend/postmaster/subprocess.c
@@ -0,0 +1,208 @@
+/*-------------------------------------------------------------------------
+ *
+ * subprocess.c
+ *
+ * Copyright (c) 2004-2020, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/postmaster/syslogger.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "postmaster/subprocess.h"
+#include "bootstrap/bootstrap.h"
+#include "postmaster/bgwriter.h"
+#include "postmaster/walwriter.h"
+#include "postmaster/syslogger.h"
+#include "postmaster/autovacuum.h"
+#include "postmaster/bgworker_internals.h"
+#include "postmaster/bgworker.h"
+#include "postmaster/postmaster.h"
+#include "postmaster/startup.h"
+#include "pgstat.h"
+#include "replication/walreceiver.h"
+
+PgSubprocess *MySubprocess;
+
+PgSubprocess process_types[] = {
+ {
+ .progname = "boot",
+ .name = "checker",
+ .desc = "checker process",
+ .needs_shmem = true,
+ .needs_aux_proc = true,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = CheckerModeMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "boot",
+ .name = "bootstrap",
+ .desc = "bootstrap process",
+ .needs_shmem = true,
+ .needs_aux_proc = true,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = BootstrapModeMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forkboot",
+ .name = "startup",
+ .desc = "startup process",
+ .needs_shmem = true,
+ .needs_aux_proc = true,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = StartupProcessMain,
+ .cleanup = StartupCleanup,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forkboot",
+ .name = "bgwriter",
+ .desc = "background writer process",
+ .needs_shmem = true,
+ .needs_aux_proc = true,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = BackgroundWriterMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forkboot",
+ .name = "checkpointer",
+ .desc = "checkpointer process",
+ .needs_shmem = true,
+ .needs_aux_proc = true,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = CheckpointerMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forkboot",
+ .name = "walwriter",
+ .desc = "WAL writer process",
+ .needs_shmem = true,
+ .needs_aux_proc = true,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = WalWriterMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forkboot",
+ .name = "walreceiver",
+ .desc = "WAL receiver process",
+ .needs_shmem = true,
+ .needs_aux_proc = true,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = WalReceiverMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forkavlauncher",
+ .name = "autovacuum launcher",
+ .desc = "autovacuum launcher process",
+ .needs_shmem = true,
+ .needs_aux_proc = false,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = AutoVacLauncherMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forkavworker",
+ .name = "autovacuum worker",
+ .desc = "autovacuum worker process",
+ .needs_shmem = true,
+ .needs_aux_proc = false,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = AutoVacWorkerMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forkcol",
+ .name = "pgstat collector",
+ .desc = "statistics collector process",
+ .needs_shmem = true,
+ .needs_aux_proc = false,
+ .keep_postmaster_memcontext = false,
+ .init = PgstatCollectorPrep,
+ .entrypoint = PgstatCollectorMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forkarch",
+ .name = "archiver",
+ .desc = "archiver process",
+ .needs_shmem = false,
+ .needs_aux_proc = false,
+ .keep_postmaster_memcontext = false,
+ .init = PgArchiverPrep,
+ .entrypoint = PgArchiverMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forklog",
+ .name = "logger",
+ .desc = "system logger",
+ .needs_shmem = false,
+ .needs_aux_proc = false,
+ .keep_postmaster_memcontext = true,
+ .init = SysLoggerPrep,
+ .entrypoint = SysLoggerMain,
+ .cleanup = NULL,
+ .parent_main = SysLoggerParentMain
+ },
+ {
+ .progname = "forkbgworker",
+ .name = "bgworker",
+ .desc = "background worker",
+ .needs_shmem = true,
+ .needs_aux_proc = false,
+ .keep_postmaster_memcontext = true,
+ .init = BgWorkerPrep,
+ .entrypoint = BackgroundWorkerMain,
+ .cleanup = BackgroundWorkerCleanup,
+ .parent_main = BackgroundWorkerParentMain
+ },
+ {
+ .progname = "forkbackend",
+ .name = "backend",
+ .desc = "client backend",
+ .needs_shmem = true,
+ .needs_aux_proc = false,
+ .keep_postmaster_memcontext = true,
+ .init = BackendPrep,
+ .entrypoint = BackendMain,
+ .cleanup = BackendCleanup,
+ .parent_main = BackendParentMain
+ }
+};
+
+void
+InitializeMySubprocess(SubprocessType type)
+{
+ /* Globally set subprocess information */
+ MySubprocessType = type;
+ MySubprocess = (PgSubprocess *)malloc(sizeof(PgSubprocess));
+
+ memcpy(MySubprocess, &process_types[type], sizeof(PgSubprocess));
+}
diff --git a/src/backend/postmaster/syslogger.c b/src/backend/postmaster/syslogger.c
index cf7b535e4e..59497e829b 100644
--- a/src/backend/postmaster/syslogger.c
+++ b/src/backend/postmaster/syslogger.c
@@ -48,6 +48,7 @@
#include "storage/pg_shmem.h"
#include "tcop/tcopprot.h"
#include "utils/guc.h"
+#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/timestamp.h"
@@ -93,6 +94,7 @@ NON_EXEC_STATIC pg_time_t first_syslogger_file_time = 0;
static char *last_file_name = NULL;
static char *last_csv_file_name = NULL;
+
/*
* Buffers for saving partial messages from different backends.
*
@@ -131,13 +133,11 @@ static CRITICAL_SECTION sysloggerSection;
static volatile sig_atomic_t got_SIGHUP = false;
static volatile sig_atomic_t rotation_requested = false;
-
/* Local subroutines */
#ifdef EXEC_BACKEND
-static pid_t syslogger_forkexec(void);
static void syslogger_parseArgs(int argc, char *argv[]);
#endif
-NON_EXEC_STATIC void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
+
static void process_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static void flush_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static FILE *logfile_open(const char *filename, const char *mode,
@@ -158,7 +158,7 @@ static void update_metainfo_datafile(void);
* Main entry point for syslogger process
* argc/argv parameters are valid only in EXEC_BACKEND case.
*/
-NON_EXEC_STATIC void
+void
SysLoggerMain(int argc, char *argv[])
{
#ifndef WIN32
@@ -171,12 +171,16 @@ SysLoggerMain(int argc, char *argv[])
pg_time_t now;
WaitEventSet *wes;
- now = MyStartTime;
-
#ifdef EXEC_BACKEND
syslogger_parseArgs(argc, argv);
+#else
+ /* Drop our connection to postmaster's shared memory, as well */
+ dsm_detach_all();
+ PGSharedMemoryDetach();
#endif /* EXEC_BACKEND */
+ now = MyStartTime;
+
am_syslogger = true;
init_ps_display("logger", "", "", "");
@@ -540,16 +544,40 @@ SysLoggerMain(int argc, char *argv[])
}
/*
- * Postmaster subroutine to start a syslogger subprocess.
+ * Helper function for setting up a file number buffer
+ */
+static void
+write_filenobuf(char *filenobuf, FILE *file)
+{
+ /* static variables (those not passed by write_backend_variables) */
+#ifndef WIN32
+ if (file != NULL)
+ snprintf(filenobuf, sizeof(filenobuf), "%d", fileno(file));
+ else
+ strcpy(filenobuf, "-1");
+#else /* WIN32 */
+ if (file != NULL)
+ snprintf(filenobuf, sizeof(filenobuf), "%ld", (long) _get_osfhandle(_fileno(file)));
+ else
+ strcpy(filenobuf, "-1");
+#endif /* WIN32 */
+}
+
+/*
+ * SysLoggerPrep
+ *
+ * Postmaster subroutine to prepare a syslogger subprocess.
*/
int
-SysLogger_Start(void)
+SysLoggerPrep(int argc, char *argv[])
{
- pid_t sysloggerPid;
char *filename;
+ char filenobuf[32];
+ char csvfilenobuf[32];
+ /* Tell StartupSubprocess not to continue forking */
if (!Logging_collector)
- return 0;
+ return -1;
/*
* If first time through, create the pipe which will receive stderr
@@ -610,11 +638,9 @@ SysLogger_Start(void)
* a time-based rotation.
*/
first_syslogger_file_time = time(NULL);
-
filename = logfile_getname(first_syslogger_file_time, NULL);
-
syslogFile = logfile_open(filename, "a", false);
-
+ write_filenobuf(filenobuf, syslogFile);
pfree(filename);
/*
@@ -625,173 +651,97 @@ SysLogger_Start(void)
if (Log_destination & LOG_DESTINATION_CSVLOG)
{
filename = logfile_getname(first_syslogger_file_time, ".csv");
-
csvlogFile = logfile_open(filename, "a", false);
-
+ write_filenobuf(csvfilenobuf, csvlogFile);
pfree(filename);
}
-#ifdef EXEC_BACKEND
- switch ((sysloggerPid = syslogger_forkexec()))
-#else
- switch ((sysloggerPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork system logger: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(true);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- /* do the work */
- SysLoggerMain(0, NULL);
- break;
-#endif
-
- default:
- /* success, in postmaster */
-
- /* now we redirect stderr, if not done already */
- if (!redirection_done)
- {
-#ifdef WIN32
- int fd;
-#endif
-
- /*
- * Leave a breadcrumb trail when redirecting, in case the user
- * forgets that redirection is active and looks only at the
- * original stderr target file.
- */
- ereport(LOG,
- (errmsg("redirecting log output to logging collector process"),
- errhint("Future log output will appear in directory \"%s\".",
- Log_directory)));
-
-#ifndef WIN32
- fflush(stdout);
- if (dup2(syslogPipe[1], fileno(stdout)) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stdout: %m")));
- fflush(stderr);
- if (dup2(syslogPipe[1], fileno(stderr)) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- /* Now we are done with the write end of the pipe. */
- close(syslogPipe[1]);
- syslogPipe[1] = -1;
-#else
-
- /*
- * open the pipe in binary mode and make sure stderr is binary
- * after it's been dup'ed into, to avoid disturbing the pipe
- * chunking protocol.
- */
- fflush(stderr);
- fd = _open_osfhandle((intptr_t) syslogPipe[1],
- _O_APPEND | _O_BINARY);
- if (dup2(fd, _fileno(stderr)) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- close(fd);
- _setmode(_fileno(stderr), _O_BINARY);
-
- /*
- * Now we are done with the write end of the pipe.
- * CloseHandle() must not be called because the preceding
- * close() closes the underlying handle.
- */
- syslogPipe[1] = 0;
-#endif
- redirection_done = true;
- }
+ argv[argc++] = filenobuf;
+ argv[argc++] = csvfilenobuf;
- /* postmaster will never write the file(s); close 'em */
- fclose(syslogFile);
- syslogFile = NULL;
- if (csvlogFile != NULL)
- {
- fclose(csvlogFile);
- csvlogFile = NULL;
- }
- return (int) sysloggerPid;
- }
+ argv[argc] = NULL;
+ Assert(argc < lengthof(argv));
- /* we should never reach here */
return 0;
}
-
-#ifdef EXEC_BACKEND
-
/*
- * syslogger_forkexec() -
+ * SysLoggerParentMain
*
- * Format up the arglist for, then fork and exec, a syslogger process
+ * Fallthrough subroutine for parent running syslogger.
*/
-static pid_t
-syslogger_forkexec(void)
+void
+SysLoggerParentMain(int argc, char *argv[])
{
- char *av[10];
- int ac = 0;
- char filenobuf[32];
- char csvfilenobuf[32];
+ /* now we redirect stderr, if not done already */
+ if (!redirection_done)
+ {
+#ifdef WIN32
+ int fd;
+#endif
- av[ac++] = "postgres";
- av[ac++] = "--forklog";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
+ /*
+ * Leave a breadcrumb trail when redirecting, in case the user
+ * forgets that redirection is active and looks only at the
+ * original stderr target file.
+ */
+ ereport(LOG,
+ (errmsg("redirecting log output to logging collector process"),
+ errhint("Future log output will appear in directory \"%s\".",
+ Log_directory)));
- /* static variables (those not passed by write_backend_variables) */
#ifndef WIN32
- if (syslogFile != NULL)
- snprintf(filenobuf, sizeof(filenobuf), "%d",
- fileno(syslogFile));
- else
- strcpy(filenobuf, "-1");
-#else /* WIN32 */
- if (syslogFile != NULL)
- snprintf(filenobuf, sizeof(filenobuf), "%ld",
- (long) _get_osfhandle(_fileno(syslogFile)));
- else
- strcpy(filenobuf, "0");
-#endif /* WIN32 */
- av[ac++] = filenobuf;
+ fflush(stdout);
+ if (dup2(syslogPipe[1], fileno(stdout)) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stdout: %m")));
+ fflush(stderr);
+ if (dup2(syslogPipe[1], fileno(stderr)) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ /* Now we are done with the write end of the pipe. */
+ close(syslogPipe[1]);
+ syslogPipe[1] = -1;
+#else
-#ifndef WIN32
- if (csvlogFile != NULL)
- snprintf(csvfilenobuf, sizeof(csvfilenobuf), "%d",
- fileno(csvlogFile));
- else
- strcpy(csvfilenobuf, "-1");
-#else /* WIN32 */
- if (csvlogFile != NULL)
- snprintf(csvfilenobuf, sizeof(csvfilenobuf), "%ld",
- (long) _get_osfhandle(_fileno(csvlogFile)));
- else
- strcpy(csvfilenobuf, "0");
-#endif /* WIN32 */
- av[ac++] = csvfilenobuf;
+ /*
+ * open the pipe in binary mode and make sure stderr is binary
+ * after it's been dup'ed into, to avoid disturbing the pipe
+ * chunking protocol.
+ */
+ fflush(stderr);
+ fd = _open_osfhandle((intptr_t) syslogPipe[1],
+ _O_APPEND | _O_BINARY);
+ if (dup2(fd, _fileno(stderr)) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ close(fd);
+ _setmode(_fileno(stderr), _O_BINARY);
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ /*
+ * Now we are done with the write end of the pipe.
+ * CloseHandle() must not be called because the preceding
+ * close() closes the underlying handle.
+ */
+ syslogPipe[1] = 0;
+#endif
+ redirection_done = true;
+ }
- return postmaster_forkexec(ac, av);
+ /* postmaster will never write the file(s); close 'em */
+ fclose(syslogFile);
+ syslogFile = NULL;
+ if (csvlogFile != NULL)
+ {
+ fclose(csvlogFile);
+ csvlogFile = NULL;
+ }
}
+#ifdef EXEC_BACKEND
/*
* syslogger_parseArgs() -
*
@@ -800,7 +750,7 @@ syslogger_forkexec(void)
static void
syslogger_parseArgs(int argc, char *argv[])
{
- int fd;
+ int fd;
Assert(argc == 5);
argv += 3;
@@ -825,7 +775,7 @@ syslogger_parseArgs(int argc, char *argv[])
csvlogFile = fdopen(fd, "a");
setvbuf(csvlogFile, NULL, PG_IOLBF, 0);
}
-#else /* WIN32 */
+#else /* WIN32 */
fd = atoi(*argv++);
if (fd != 0)
{
@@ -846,10 +796,9 @@ syslogger_parseArgs(int argc, char *argv[])
setvbuf(csvlogFile, NULL, PG_IOLBF, 0);
}
}
-#endif /* WIN32 */
+#endif /* WIN32 */
}
-#endif /* EXEC_BACKEND */
-
+#endif
/* --------------------------------
* pipe protocol handling
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index 45a2757969..3a0cec3371 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -85,7 +85,7 @@ int WalWriterFlushAfter = 128;
* basic execution environment, but not enabled signals yet.
*/
void
-WalWriterMain(void)
+WalWriterMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
MemoryContext walwriter_context;
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index 2ab15c3cbb..725023f0f6 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -164,7 +164,7 @@ ProcessWalRcvInterrupts(void)
/* Main entry point for walreceiver process */
void
-WalReceiverMain(void)
+WalReceiverMain(int argc, char *argv[])
{
char conninfo[MAXCONNINFO];
char *tmp_conninfo;
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7e6a3c1774..ec0f34d582 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -24,6 +24,7 @@
#include "pgstat.h"
#include "postmaster/bgworker_internals.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "storage/proc.h"
#include "storage/procarray.h"
#include "utils/acl.h"
@@ -691,7 +692,7 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
proc = BackendPidGetProc(beentry->st_procpid);
- if (proc == NULL && (beentry->st_backendType != B_BACKEND))
+ if (proc == NULL && (MySubprocessType != ClientBackendType))
{
/*
* For an auxiliary process, retrieve process info from
@@ -739,7 +740,7 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
* date.
*/
if (beentry->st_xact_start_timestamp != 0 &&
- beentry->st_backendType != B_WAL_SENDER)
+ MySubprocessType != WalSenderType)
values[8] = TimestampTzGetDatum(beentry->st_xact_start_timestamp);
else
nulls[8] = true;
@@ -827,7 +828,7 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
}
}
/* Add backend type */
- if (beentry->st_backendType == B_BG_WORKER)
+ if (MySubprocessType == BgWorkerType)
{
const char *bgw_type;
@@ -839,7 +840,7 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
}
else
values[17] =
- CStringGetTextDatum(pgstat_get_backend_desc(beentry->st_backendType));
+ CStringGetTextDatum(MySubprocess->desc);
/* SSL information */
if (beentry->st_ssl)
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index eb19644419..3e7be0bfcd 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -38,9 +38,10 @@ volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
int MyProcPid;
+int MyChildProcPid;
pg_time_t MyStartTime;
TimestampTz MyStartTimestamp;
-struct Port *MyProcPort;
+struct Port *MyProcPort = NULL;
int32 MyCancelKey;
int MyPMChildSlot;
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 8a47dcdcb1..91e43382c0 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -1112,7 +1112,7 @@ process_startup_options(Port *port, bool am_superuser)
av = (char **) palloc(maxac * sizeof(char *));
ac = 0;
- av[ac++] = "postgres";
+ av[ac++] = pstrdup("postgres");
pg_split_opts(av, &ac, port->cmdline_options);
diff --git a/src/include/bootstrap/bootstrap.h b/src/include/bootstrap/bootstrap.h
index b67fa9acc1..5d851e0081 100644
--- a/src/include/bootstrap/bootstrap.h
+++ b/src/include/bootstrap/bootstrap.h
@@ -15,7 +15,7 @@
#define BOOTSTRAP_H
#include "nodes/execnodes.h"
-
+#include "postmaster/fork_process.h"
/*
* MAXATTR is the maximum number of attributes in a relation supported
@@ -31,6 +31,8 @@ extern Relation boot_reldesc;
extern Form_pg_attribute attrtypes[MAXATTR];
extern int numattr;
+extern void CheckerModeMain(int argc, char *argv[]);
+extern void BootstrapModeMain(int argc, char *argv[]);
extern void AuxiliaryProcessMain(int argc, char *argv[]) pg_attribute_noreturn();
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index f985453ec3..823f5258d8 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -163,6 +163,7 @@ extern PGDLLIMPORT int max_worker_processes;
extern PGDLLIMPORT int max_parallel_workers;
extern PGDLLIMPORT int MyProcPid;
+extern PGDLLIMPORT int MyChildProcPid;
extern PGDLLIMPORT pg_time_t MyStartTime;
extern PGDLLIMPORT TimestampTz MyStartTimestamp;
extern PGDLLIMPORT struct Port *MyProcPort;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 3a65a51696..4e77a86943 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1048,9 +1048,6 @@ typedef struct PgBackendStatus
/* The entry is valid iff st_procpid > 0, unused if st_procpid == 0 */
int st_procpid;
- /* Type of backends */
- BackendType st_backendType;
-
/* Times when current backend, transaction, and activity started */
TimestampTz st_proc_start_timestamp;
TimestampTz st_xact_start_timestamp;
@@ -1237,13 +1234,12 @@ extern Size BackendStatusShmemSize(void);
extern void CreateSharedBackendStatus(void);
extern void pgstat_init(void);
-extern int pgstat_start(void);
extern void pgstat_reset_all(void);
extern void allow_immediate_pgstat_restart(void);
-#ifdef EXEC_BACKEND
+/* Stat collector subprocess startup functions */
+extern int PgstatCollectorPrep(int argc, char *argv[]);
extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
/* ----------
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index d40ed55531..dc02de745e 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -45,6 +45,10 @@ extern int AutovacuumLauncherPid;
extern int Log_autovacuum_min_duration;
+/* autovacuum identification */
+extern bool am_autovacuum_launcher;
+extern bool am_autovacuum_worker;
+
/* Status inquiry functions */
extern bool AutoVacuumingActive(void);
extern bool IsAutoVacuumLauncherProcess(void);
@@ -58,18 +62,15 @@ extern void autovac_init(void);
extern int StartAutoVacLauncher(void);
extern int StartAutoVacWorker(void);
+extern void AutoVacWorkerMain(int argc, char *argv[]);
+extern void AutoVacLauncherMain(int argc, char *argv[]);
+
/* called from postmaster when a worker could not be forked */
extern void AutoVacWorkerFailed(void);
/* autovacuum cost-delay balancer */
extern void AutoVacuumUpdateDelay(void);
-#ifdef EXEC_BACKEND
-extern void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern void AutovacuumWorkerIAm(void);
-extern void AutovacuumLauncherIAm(void);
-#endif
extern bool AutoVacuumRequestWork(AutoVacuumWorkItemType type,
Oid relationId, BlockNumber blkno);
diff --git a/src/include/postmaster/bgworker_internals.h b/src/include/postmaster/bgworker_internals.h
index f7e24664d5..366bbcf1e3 100644
--- a/src/include/postmaster/bgworker_internals.h
+++ b/src/include/postmaster/bgworker_internals.h
@@ -43,6 +43,7 @@ typedef struct RegisteredBgWorker
} RegisteredBgWorker;
extern slist_head BackgroundWorkerList;
+extern RegisteredBgWorker *CurrentBgWorker;
extern Size BackgroundWorkerShmemSize(void);
extern void BackgroundWorkerShmemInit(void);
@@ -54,7 +55,11 @@ extern void BackgroundWorkerStopNotifications(pid_t pid);
extern void ResetBackgroundWorkerCrashTimes(void);
/* Function to start a background worker, called from postmaster.c */
-extern void StartBackgroundWorker(void) pg_attribute_noreturn();
+extern int BgWorkerPrep(int argc, char *argv[]);
+extern void BackgroundWorkerMain(int argc, char *argv[]);
+extern void BackgroundWorkerParentMain(int argc, char *argv[]);
+extern bool BackgroundWorkerCleanup(int argc, char *argv[]);
+
#ifdef EXEC_BACKEND
extern BackgroundWorker *BackgroundWorkerEntry(int slotno);
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..6fb1ee1d6d 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -27,8 +27,8 @@ extern int CheckPointTimeout;
extern int CheckPointWarning;
extern double CheckPointCompletionTarget;
-extern void BackgroundWriterMain(void) pg_attribute_noreturn();
-extern void CheckpointerMain(void) pg_attribute_noreturn();
+extern void BackgroundWriterMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern void CheckpointerMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void RequestCheckpoint(int flags);
extern void CheckpointWriteDelay(int flags, double progress);
diff --git a/src/include/postmaster/fork_process.h b/src/include/postmaster/fork_process.h
index 8cb568fc63..a42f859a08 100644
--- a/src/include/postmaster/fork_process.h
+++ b/src/include/postmaster/fork_process.h
@@ -12,6 +12,12 @@
#ifndef FORK_PROCESS_H
#define FORK_PROCESS_H
+#include "postmaster.h"
+
extern pid_t fork_process(void);
+#ifdef EXEC_BACKEND
+extern pid_t postmaster_forkexec(int argc, char *argvp[]);
+#endif
+
#endif /* FORK_PROCESS_H */
diff --git a/src/include/postmaster/pgarch.h b/src/include/postmaster/pgarch.h
index b3200874ca..ba5b012839 100644
--- a/src/include/postmaster/pgarch.h
+++ b/src/include/postmaster/pgarch.h
@@ -26,14 +26,9 @@
#define MAX_XFN_CHARS 40
#define VALID_XFN_CHARS "0123456789ABCDEF.history.backup.partial"
-/* ----------
- * Functions called from postmaster
- * ----------
- */
-extern int pgarch_start(void);
-#ifdef EXEC_BACKEND
+/* Archiver subprocess startup functions */
+extern int PgArchiverPrep(int argc, char *argv[]);
extern void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
#endif /* _PGARCH_H */
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index babc87dfc9..d4c0f46220 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -13,6 +13,41 @@
#ifndef _POSTMASTER_H
#define _POSTMASTER_H
+#include "lib/ilist.h"
+
+/*
+ * List of active backends (or child processes anyway; we don't actually
+ * know whether a given child has become a backend or is still in the
+ * authorization phase). This is used mainly to keep track of how many
+ * children we have and send them appropriate signals when necessary.
+ *
+ * "Special" children such as the startup, bgwriter and autovacuum launcher
+ * tasks are not in this list. Autovacuum worker and walsender are in it.
+ * Also, "dead_end" children are in it: these are children launched just for
+ * the purpose of sending a friendly rejection message to a would-be client.
+ * We must track them because they are attached to shared memory, but we know
+ * they will never become live backends. dead_end children are not assigned a
+ * PMChildSlot.
+ *
+ * Background workers are in this list, too.
+ */
+typedef struct bkend
+{
+ pid_t pid; /* process id of backend */
+ int32 cancel_key; /* cancel key for cancels for this backend */
+ int child_slot; /* PMChildSlot for this backend, if any */
+
+ /*
+ * Flavor of backend or auxiliary process. Note that BACKEND_TYPE_WALSND
+ * backends initially announce themselves as BACKEND_TYPE_NORMAL, so if
+ * bkend_type is normal, you should check for a recent transition.
+ */
+ int bkend_type;
+ bool dead_end; /* is it going to send an error and quit? */
+ bool bgworker_notify; /* gets bgworker start/stop notifications */
+ dlist_node elem; /* list link in BackendList */
+} Backend;
+
/* GUC options */
extern bool EnableSSL;
extern int ReservedBackends;
@@ -47,21 +82,31 @@ extern int postmaster_alive_fds[2];
extern PGDLLIMPORT const char *progname;
extern void PostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern void ClosePostmasterPorts(bool am_syslogger);
+extern void ClosePostmasterPorts(void);
extern void InitProcessGlobals(void);
extern int MaxLivePostmasterChildren(void);
extern bool PostmasterMarkPIDForWorkerNotify(int);
+extern bool RandomCancelKey(int32 *cancel_key);
+
+extern int BackendPrep(int argc, char *argv[]);
+extern void BackendMain(int argc, char *argv[]);
+extern void BackendParentMain(int argc, char *argv[]);
+extern bool BackendCleanup(int argc, char *argv[]);
+extern void BackgroundWorkerMain(int argc, char *argv[]);
+
#ifdef EXEC_BACKEND
-extern pid_t postmaster_forkexec(int argc, char *argv[]);
extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
extern Size ShmemBackendArraySize(void);
extern void ShmemBackendArrayAllocation(void);
+extern void ShmemBackendArrayAdd(Backend *bn);
#endif
+extern PGDLLIMPORT dlist_head BackendList;
+
/*
* Note: MAX_BACKENDS is limited to 2^18-1 because that's the width reserved
* for buffer references in buf_internals.h. This limitation could be lifted
@@ -74,4 +119,17 @@ extern void ShmemBackendArrayAllocation(void);
*/
#define MAX_BACKENDS 0x3FFFF
+/*
+ * Possible types of a backend. Beyond being the possible bkend_type values in
+ * struct bkend, these are OR-able request flag bits for SignalSomeChildren()
+ * and CountChildren().
+ */
+#define BACKEND_TYPE_NORMAL 0x0001 /* normal backend */
+#define BACKEND_TYPE_AUTOVAC 0x0002 /* autovacuum worker process */
+#define BACKEND_TYPE_WALSND 0x0004 /* walsender process */
+#define BACKEND_TYPE_BGWORKER 0x0008 /* bgworker process */
+#define BACKEND_TYPE_ALL 0x000F /* OR of all the above */
+
+#define BACKEND_TYPE_WORKER (BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER)
+
#endif /* _POSTMASTER_H */
diff --git a/src/include/postmaster/startup.h b/src/include/postmaster/startup.h
index 9f59c1ffa3..14a646136f 100644
--- a/src/include/postmaster/startup.h
+++ b/src/include/postmaster/startup.h
@@ -13,10 +13,13 @@
#define _STARTUP_H
extern void HandleStartupProcInterrupts(void);
-extern void StartupProcessMain(void) pg_attribute_noreturn();
extern void PreRestoreCommand(void);
extern void PostRestoreCommand(void);
extern bool IsPromoteTriggered(void);
extern void ResetPromoteTriggered(void);
+/* Startup subprocess functions */
+extern void StartupProcessMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern bool StartupCleanup(int argc, char *argv[]);
+
#endif /* _STARTUP_H */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
new file mode 100644
index 0000000000..b65fc75475
--- /dev/null
+++ b/src/include/postmaster/subprocess.h
@@ -0,0 +1,66 @@
+/*-------------------------------------------------------------------------
+ *
+ * subprocess.h
+ *
+ * Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ *
+ * src/include/postmaster/subprocess.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef SUBPROCESS_H
+#define SUBPROCESS_H
+
+#include "postmaster.h"
+
+typedef enum
+{
+ NoProcessType = -1,
+ CheckerType = 0,
+ BootstrapType,
+ StartupType,
+ BgWriterType,
+ CheckpointerType,
+ WalWriterType,
+ WalReceiverType, /* end of Auxiliary Process Forks */
+ AutoVacLauncherType,
+ AutoVacWorkerType,
+ PgstatCollectorType,
+ PgArchiverType,
+ SysLoggerType,
+ BgWorkerType,
+ ClientBackendType,
+
+ WalSenderType, /* Dummy type for wal sender. No process struct needed as
+ * this process just takes over an existing bgworker */
+
+ NUMSUBPROCESSTYPES /* Must be last! */
+} SubprocessType;
+
+typedef int (*SubprocessInit) (int argc, char *argv[]);
+typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
+typedef bool (*SubprocessCleanup) (int argc, char *argv[]);
+typedef void (*SubprocessParent) (int argc, char *argv[]);
+
+/* Current subprocess initializer */
+extern void InitializeMySubprocess(SubprocessType type);
+
+typedef struct PgSubprocess
+{
+ const char *progname;
+ const char *name;
+ const char *desc;
+ bool needs_shmem;
+ bool needs_aux_proc;
+ bool needs_full_proc;
+ bool keep_postmaster_memcontext;
+ SubprocessInit init;
+ SubprocessEntryPoint entrypoint;
+ SubprocessCleanup cleanup;
+ SubprocessParent parent_main;
+} PgSubprocess;
+
+extern SubprocessType MySubprocessType;
+extern PGDLLIMPORT PgSubprocess *MySubprocess;
+
+#endif /* SUBPROCESS_H */
diff --git a/src/include/postmaster/syslogger.h b/src/include/postmaster/syslogger.h
index 9b7a386dbd..ededfd7c23 100644
--- a/src/include/postmaster/syslogger.h
+++ b/src/include/postmaster/syslogger.h
@@ -14,7 +14,6 @@
#include <limits.h> /* for PIPE_BUF */
-
/*
* Primitive protocol structure for writing to syslogger pipe(s). The idea
* here is to divide long messages into chunks that are not more than
@@ -83,9 +82,10 @@ extern int SysLogger_Start(void);
extern void write_syslogger_file(const char *buffer, int count, int dest);
-#ifdef EXEC_BACKEND
-extern void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+/* Subprocess startup functions */
+extern int SysLoggerPrep(int argc, char *argv[]);
+extern void SysLoggerParentMain(int argc, char *argv[]);
+extern void SysLoggerMain(int argc, char *argv[]);
extern bool CheckLogrotateSignal(void);
extern void RemoveLogrotateSignalFiles(void);
diff --git a/src/include/postmaster/walwriter.h b/src/include/postmaster/walwriter.h
index 011dc26a6e..596c437c54 100644
--- a/src/include/postmaster/walwriter.h
+++ b/src/include/postmaster/walwriter.h
@@ -16,6 +16,6 @@
extern int WalWriterDelay;
extern int WalWriterFlushAfter;
-extern void WalWriterMain(void) pg_attribute_noreturn();
+extern void WalWriterMain(int argc, char *argv[]) pg_attribute_noreturn();
#endif /* _WALWRITER_H */
diff --git a/src/include/replication/walreceiver.h b/src/include/replication/walreceiver.h
index e08afc6548..2025f19ab3 100644
--- a/src/include/replication/walreceiver.h
+++ b/src/include/replication/walreceiver.h
@@ -311,7 +311,7 @@ walrcv_clear_result(WalRcvExecResult *walres)
}
/* prototypes for functions in walreceiver.c */
-extern void WalReceiverMain(void) pg_attribute_noreturn();
+extern void WalReceiverMain(int argc, char *argv[]);
extern void ProcessWalRcvInterrupts(void);
/* prototypes for functions in walreceiverfuncs.c */
--
2.21.0
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-02 14:53 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
@ 2020-03-04 04:56 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 18:50 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
0 siblings, 1 reply; 38+ messages in thread
From: Mike Palmiotto @ 2020-03-04 04:56 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: Andres Freund <andres@anarazel.de>; Alvaro Herrera <alvherre@2ndquadrant.com>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Mon, Mar 2, 2020 at 9:53 AM Mike Palmiotto
<mike.palmiotto@crunchydata.com> wrote:
>
> On Sat, Feb 29, 2020 at 9:51 PM Mike Palmiotto
> <mike.palmiotto@crunchydata.com> wrote:
> >
<snip>
> Okay, here is an updated and rebased patch that passes all regression
> tests with and without EXEC_BACKEND. This also treats more of the
> issues raised by Andres.
>
> I still need to do the following:
> - split giant patch out into separate functional commits
> - add translated process descriptions
> - fix some variable names here and there (notably the PgSubprocess
> struct ".progname" member -- that's a misnomer.
> - address some valgrind findings
>
> I should be able to split this out into smaller commits sometime today
> and will continue iterating to scratch the other items off the list.
I've addressed all of the points above (except splitting the base
patch up) and rebased on master. I didn't squash before generating the
patches so it may be easier to read the progress here, or annoying
that there are so many attachments.
This patchset hits all of Andres' points with the exception of the
reshuffling of functions like AuxiliaryProcessMain, but it does get
rid of a lot of direct argv string comparison. I'll reorganize a bit
when I re-write the patchset and will likely move
AuxiliaryProcessMain, along with the Backend subprocess functions but
just wanted to get this up here in the meantime.
There is quite a lot to be gained from this patchset in the form of
re-organization and removal of redundant code-paths/globals. I see
this as a good first step and am willing to continue to push it
forward with any new suggestions.
Thanks,
--
Mike Palmiotto
https://crunchydata.com
Attachments: download all (11 files, .tar.gz)
0008-Rename-subprocess-init-to-prep.patch (text/x-patch, 5.8K) download
From 77cab84ba36865540aa5f7fd792760d8268a71eb Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 4 Mar 2020 03:53:33 +0000
Subject: [PATCH 08/11] Rename subprocess init to prep
---
src/backend/postmaster/postmaster.c | 4 ++--
src/backend/postmaster/subprocess.c | 28 ++++++++++++++--------------
src/include/postmaster/subprocess.h | 4 ++--
3 files changed, 18 insertions(+), 18 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 8273d53c30..ceb738bf1e 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -5412,10 +5412,10 @@ StartSubprocess(SubprocessType type)
Assert(argc < lengthof(argv));
/* Prep the subprocesses for a fork */
- if (MySubprocess->init)
+ if (MySubprocess->prep)
{
int rc = 0;
- rc = MySubprocess->init(argc, argv);
+ rc = MySubprocess->prep(argc, argv);
if (rc != 0)
return 0;
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 0bb1ba4655..dbb9be5f6b 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -34,7 +34,7 @@ static PgSubprocess process_types[] = {
.needs_shmem = true,
.needs_aux_proc = true,
.keep_postmaster_memcontext = false,
- .init = NULL,
+ .prep = NULL,
.entrypoint = CheckerModeMain,
.cleanup = NULL,
.parent_main = NULL
@@ -45,7 +45,7 @@ static PgSubprocess process_types[] = {
.needs_shmem = true,
.needs_aux_proc = true,
.keep_postmaster_memcontext = false,
- .init = NULL,
+ .prep = NULL,
.entrypoint = BootstrapModeMain,
.cleanup = NULL,
.parent_main = NULL
@@ -56,7 +56,7 @@ static PgSubprocess process_types[] = {
.needs_shmem = true,
.needs_aux_proc = true,
.keep_postmaster_memcontext = false,
- .init = NULL,
+ .prep = NULL,
.entrypoint = StartupProcessMain,
.cleanup = StartupCleanup,
.parent_main = NULL
@@ -67,7 +67,7 @@ static PgSubprocess process_types[] = {
.needs_shmem = true,
.needs_aux_proc = true,
.keep_postmaster_memcontext = false,
- .init = NULL,
+ .prep = NULL,
.entrypoint = BackgroundWriterMain,
.cleanup = NULL,
.parent_main = NULL
@@ -78,7 +78,7 @@ static PgSubprocess process_types[] = {
.needs_shmem = true,
.needs_aux_proc = true,
.keep_postmaster_memcontext = false,
- .init = NULL,
+ .prep = NULL,
.entrypoint = CheckpointerMain,
.cleanup = NULL,
.parent_main = NULL
@@ -89,7 +89,7 @@ static PgSubprocess process_types[] = {
.needs_shmem = true,
.needs_aux_proc = true,
.keep_postmaster_memcontext = false,
- .init = NULL,
+ .prep = NULL,
.entrypoint = WalWriterMain,
.cleanup = NULL,
.parent_main = NULL
@@ -101,7 +101,7 @@ static PgSubprocess process_types[] = {
.needs_shmem = true,
.needs_aux_proc = true,
.keep_postmaster_memcontext = false,
- .init = NULL,
+ .prep = NULL,
.entrypoint = WalReceiverMain,
.cleanup = NULL,
.parent_main = NULL
@@ -112,7 +112,7 @@ static PgSubprocess process_types[] = {
.needs_shmem = true,
.needs_aux_proc = false,
.keep_postmaster_memcontext = false,
- .init = NULL,
+ .prep = NULL,
.entrypoint = AutoVacLauncherMain,
.cleanup = NULL,
.parent_main = NULL
@@ -123,7 +123,7 @@ static PgSubprocess process_types[] = {
.needs_shmem = true,
.needs_aux_proc = false,
.keep_postmaster_memcontext = false,
- .init = NULL,
+ .prep = NULL,
.entrypoint = AutoVacWorkerMain,
.cleanup = NULL,
.parent_main = NULL
@@ -134,7 +134,7 @@ static PgSubprocess process_types[] = {
.needs_shmem = true,
.needs_aux_proc = false,
.keep_postmaster_memcontext = false,
- .init = PgstatCollectorPrep,
+ .prep = PgstatCollectorPrep,
.entrypoint = PgstatCollectorMain,
.cleanup = NULL,
.parent_main = NULL
@@ -145,7 +145,7 @@ static PgSubprocess process_types[] = {
.needs_shmem = false,
.needs_aux_proc = false,
.keep_postmaster_memcontext = false,
- .init = PgArchiverPrep,
+ .prep = PgArchiverPrep,
.entrypoint = PgArchiverMain,
.cleanup = NULL,
.parent_main = NULL
@@ -156,7 +156,7 @@ static PgSubprocess process_types[] = {
.needs_shmem = false,
.needs_aux_proc = false,
.keep_postmaster_memcontext = true,
- .init = SysLoggerPrep,
+ .prep = SysLoggerPrep,
.entrypoint = SysLoggerMain,
.cleanup = NULL,
.parent_main = SysLoggerParentMain
@@ -167,7 +167,7 @@ static PgSubprocess process_types[] = {
.needs_shmem = true,
.needs_aux_proc = false,
.keep_postmaster_memcontext = true,
- .init = BgWorkerPrep,
+ .prep = BgWorkerPrep,
.entrypoint = BackgroundWorkerMain,
.cleanup = BackgroundWorkerCleanup,
.parent_main = BackgroundWorkerParentMain
@@ -178,7 +178,7 @@ static PgSubprocess process_types[] = {
.needs_shmem = true,
.needs_aux_proc = false,
.keep_postmaster_memcontext = true,
- .init = BackendPrep,
+ .prep = BackendPrep,
.entrypoint = BackendMain,
.cleanup = BackendCleanup,
.parent_main = BackendParentMain
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 2634bbd142..70a92647fe 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -37,7 +37,7 @@ typedef enum
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
-typedef int (*SubprocessInit) (int argc, char *argv[]);
+typedef int (*SubprocessPrep) (int argc, char *argv[]);
typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
typedef bool (*SubprocessCleanup) (int child_errno);
typedef void (*SubprocessParent) (int argc, char *argv[]);
@@ -54,7 +54,7 @@ typedef struct PgSubprocess
bool needs_aux_proc;
bool needs_full_proc;
bool keep_postmaster_memcontext;
- SubprocessInit init;
+ SubprocessPrep prep;
SubprocessEntryPoint entrypoint;
SubprocessCleanup cleanup;
SubprocessParent parent_main;
--
2.21.0
0011-Use-MyProcPort-instead-of-adding-ConnProcPort.patch (text/x-patch, 4.9K) download
From 0bce567e279e88e868a2b23f3905fa0de2e26eb9 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 4 Mar 2020 04:41:18 +0000
Subject: [PATCH 11/11] Use MyProcPort instead of adding ConnProcPort
We still need to handle the edge-case of no backend variables being
inherited. In the case where MyProcPort is not obtained from the backend
variables, just create a dummy port so we can proceed with the forkexec.
---
src/backend/postmaster/postmaster.c | 53 ++++++++++++++++-------------
1 file changed, 30 insertions(+), 23 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 06622d3acb..17a5c3d26c 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -412,7 +412,7 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t internal_forkexec(int argc, char *argv[]);
+static pid_t internal_forkexec(int argc, char *argv[], Port *port);
/* Type for a socket that can be inherited to a client process */
#ifdef WIN32
@@ -514,8 +514,6 @@ int postmaster_alive_fds[2] = {-1, -1};
HANDLE PostmasterHandle;
#endif
-static Port *ConnProcPort = NULL;
-
/*
* Postmaster main entry point
*/
@@ -1675,17 +1673,20 @@ ServerLoop(void)
break;
if (FD_ISSET(ListenSocket[i], &rmask))
{
- ConnProcPort = ConnCreate(ListenSocket[i]);
- if (ConnProcPort)
+ Port *port;
+
+ port= ConnCreate(ListenSocket[i]);
+ if (port)
{
+ MyProcPort = port;
StartSubprocess(ClientBackendType);
/*
* We no longer need the open socket or port structure
* in this process
*/
- StreamClose(ConnProcPort->sock);
- ConnFree(ConnProcPort);
+ StreamClose(port->sock);
+ ConnFree(port);
}
}
}
@@ -3248,9 +3249,9 @@ BackendPrep(int argc, char *argv[])
bn->cancel_key = MyCancelKey;
/* Pass down canAcceptConnections state */
- ConnProcPort->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
- bn->dead_end = (ConnProcPort->canAcceptConnections != CAC_OK &&
- ConnProcPort->canAcceptConnections != CAC_WAITBACKUP);
+ MyProcPort->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
+ bn->dead_end = (MyProcPort->canAcceptConnections != CAC_OK &&
+ MyProcPort->canAcceptConnections != CAC_WAITBACKUP);
/*
* Unless it's a dead_end child, assign it a child slot number
@@ -3288,10 +3289,10 @@ void BackendMain(pg_attribute_unused() int argc, pg_attribute_unused() char *arg
* PGPROC slots, we have already initialized libpq and are able to
* report the error to the client.
*/
- BackendInitialize(ConnProcPort);
+ BackendInitialize(MyProcPort);
/* And run the backend */
- BackendRun(ConnProcPort); /* does not return */
+ BackendRun(MyProcPort); /* does not return */
}
/*
@@ -4528,11 +4529,19 @@ BackendRun(Port *port)
pid_t
postmaster_forkexec(int argc, char *argv[])
{
- /* This entry point passes dummy values for the Port variables */
- if (!ConnProcPort)
- ConnProcPort = palloc0(sizeof(*ConnProcPort));
+ Port port;
+
+ /*
+ * When there are no backend variables to inherit (i.e. checker process),
+ * MyProcPort will be empty. Create a dummy variable to pass along to
+ * satisfy this case. Otherwise just use MyProcPort.
+ */
+ if (!MyProcPort)
+ memset(&port, 0, sizeof(port));
+ else
+ port = *MyProcPort;
- return internal_forkexec(argc, argv);
+ return internal_forkexec(argc, argv, &port);
}
#ifndef WIN32
@@ -4544,7 +4553,7 @@ postmaster_forkexec(int argc, char *argv[])
* - fork():s, and then exec():s the child process
*/
static pid_t
-internal_forkexec(int argc, char *argv[])
+internal_forkexec(int argc, char *argv[], Port *port)
{
static unsigned long tmpBackendFileNum = 0;
pid_t pid;
@@ -4552,7 +4561,7 @@ internal_forkexec(int argc, char *argv[])
BackendParameters param;
FILE *fp;
- if (!save_backend_variables(¶m, ConnProcPort))
+ if (!save_backend_variables(¶m, port))
return -1; /* log made by save_backend_variables */
/* Calculate name for temp file */
@@ -4891,12 +4900,10 @@ SubPostmasterMain(int argc, char *argv[])
if (argc < 3)
elog(FATAL, "invalid subpostmaster invocation");
- Assert(!ConnProcPort);
-
/* Read in the variables file */
memset(&port, 0, sizeof(Port));
read_backend_variables(argv[2], &port);
- ConnProcPort = &port;
+ MyProcPort = &port;
/* Close the postmaster's sockets (as soon as we know them) */
ClosePostmasterPorts();
@@ -5023,12 +5030,12 @@ SubPostmasterMain(int argc, char *argv[])
* PGPROC slots, we have already initialized libpq and are able to
* report the error to the client.
*/
- BackendInitialize(&port);
+ BackendInitialize(MyProcPort);
shmemSetup();
/* And run the backend */
- BackendRun(&port); /* does not return */
+ BackendRun(MyProcPort); /* does not return */
}
/* do this as early as possible; in particular, before InitProcess() */
--
2.21.0
0007-Translate-descriptions-and-drop-unused-fields.patch (text/x-patch, 5.9K) download
From 7e7a7048fe85cac14fa328735a528a5d96eb746e Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 4 Mar 2020 03:42:41 +0000
Subject: [PATCH 07/11] Translate descriptions and drop unused fields
---
src/backend/postmaster/postmaster.c | 6 +--
src/backend/postmaster/subprocess.c | 65 ++++++++++++-----------------
2 files changed, 29 insertions(+), 42 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 0589fd27f9..8273d53c30 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -5400,8 +5400,8 @@ StartSubprocess(SubprocessType type)
argv[argc++] = "postgres";
#ifdef EXEC_BACKEND
- snprintf(forkname, sizeof(forkname), "--%s", MySubprocess->progname);
- argv[argc++] = psprintf("--%s", MySubprocess->progname);
+ snprintf(forkname, sizeof(forkname), "--fork%s", MySubprocess->name);
+ argv[argc++] = forkname;
argv[argc++] = NULL;
#endif
@@ -5463,7 +5463,7 @@ StartSubprocess(SubprocessType type)
child_errno = errno;
ereport(LOG,
- (errmsg("could not fork %s: %m", MySubprocess->desc)));
+ (errmsg("could not fork %s process: %m", MySubprocess->desc)));
/*
* fork failure is fatal during startup, but there's no need to choke
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index c4fe641456..0bb1ba4655 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -29,9 +29,8 @@ PgSubprocess *MySubprocess;
static PgSubprocess process_types[] = {
{
- .progname = "boot",
- .name = "checker",
- .desc = "checker process",
+ .name = "boot",
+ .desc = gettext("checker"),
.needs_shmem = true,
.needs_aux_proc = true,
.keep_postmaster_memcontext = false,
@@ -41,9 +40,8 @@ static PgSubprocess process_types[] = {
.parent_main = NULL
},
{
- .progname = "boot",
- .name = "bootstrap",
- .desc = "bootstrap process",
+ .name = "boot",
+ .desc = gettext("bootstrap"),
.needs_shmem = true,
.needs_aux_proc = true,
.keep_postmaster_memcontext = false,
@@ -53,9 +51,8 @@ static PgSubprocess process_types[] = {
.parent_main = NULL
},
{
- .progname = "forkboot",
- .name = "startup",
- .desc = "startup process",
+ .name = "boot",
+ .desc = gettext("startup"),
.needs_shmem = true,
.needs_aux_proc = true,
.keep_postmaster_memcontext = false,
@@ -65,9 +62,8 @@ static PgSubprocess process_types[] = {
.parent_main = NULL
},
{
- .progname = "forkboot",
- .name = "bgwriter",
- .desc = "background writer process",
+ .name = "boot",
+ .desc = gettext("background writer"),
.needs_shmem = true,
.needs_aux_proc = true,
.keep_postmaster_memcontext = false,
@@ -77,9 +73,8 @@ static PgSubprocess process_types[] = {
.parent_main = NULL
},
{
- .progname = "forkboot",
- .name = "checkpointer",
- .desc = "checkpointer process",
+ .name = "boot",
+ .desc = gettext("checkpointer"),
.needs_shmem = true,
.needs_aux_proc = true,
.keep_postmaster_memcontext = false,
@@ -89,9 +84,8 @@ static PgSubprocess process_types[] = {
.parent_main = NULL
},
{
- .progname = "forkboot",
- .name = "walwriter",
- .desc = "WAL writer process",
+ .name = "boot",
+ .desc = gettext("WAL writer"),
.needs_shmem = true,
.needs_aux_proc = true,
.keep_postmaster_memcontext = false,
@@ -101,9 +95,9 @@ static PgSubprocess process_types[] = {
.parent_main = NULL
},
{
- .progname = "forkboot",
+ .name = "boot",
.name = "walreceiver",
- .desc = "WAL receiver process",
+ .desc = gettext("WAL receiver"),
.needs_shmem = true,
.needs_aux_proc = true,
.keep_postmaster_memcontext = false,
@@ -113,9 +107,8 @@ static PgSubprocess process_types[] = {
.parent_main = NULL
},
{
- .progname = "forkavlauncher",
- .name = "autovacuum launcher",
- .desc = "autovacuum launcher process",
+ .name = "avlauncher",
+ .desc = gettext("autovacuum launcher"),
.needs_shmem = true,
.needs_aux_proc = false,
.keep_postmaster_memcontext = false,
@@ -125,9 +118,8 @@ static PgSubprocess process_types[] = {
.parent_main = NULL
},
{
- .progname = "forkavworker",
- .name = "autovacuum worker",
- .desc = "autovacuum worker process",
+ .name = "avworker",
+ .desc = gettext("autovacuum worker"),
.needs_shmem = true,
.needs_aux_proc = false,
.keep_postmaster_memcontext = false,
@@ -137,9 +129,8 @@ static PgSubprocess process_types[] = {
.parent_main = NULL
},
{
- .progname = "forkcol",
- .name = "pgstat collector",
- .desc = "statistics collector process",
+ .name = "col",
+ .desc = gettext("statistics collector"),
.needs_shmem = true,
.needs_aux_proc = false,
.keep_postmaster_memcontext = false,
@@ -149,9 +140,8 @@ static PgSubprocess process_types[] = {
.parent_main = NULL
},
{
- .progname = "forkarch",
- .name = "archiver",
- .desc = "archiver process",
+ .name = "arch",
+ .desc = gettext("archiver"),
.needs_shmem = false,
.needs_aux_proc = false,
.keep_postmaster_memcontext = false,
@@ -161,9 +151,8 @@ static PgSubprocess process_types[] = {
.parent_main = NULL
},
{
- .progname = "forklog",
- .name = "logger",
- .desc = "system logger",
+ .name = "log",
+ .desc = gettext("system logger"),
.needs_shmem = false,
.needs_aux_proc = false,
.keep_postmaster_memcontext = true,
@@ -173,9 +162,8 @@ static PgSubprocess process_types[] = {
.parent_main = SysLoggerParentMain
},
{
- .progname = "forkbgworker",
.name = "bgworker",
- .desc = "background worker",
+ .desc = gettext("background worker"),
.needs_shmem = true,
.needs_aux_proc = false,
.keep_postmaster_memcontext = true,
@@ -185,9 +173,8 @@ static PgSubprocess process_types[] = {
.parent_main = BackgroundWorkerParentMain
},
{
- .progname = "forkbackend",
.name = "backend",
- .desc = "client backend",
+ .desc = gettext("client backend"),
.needs_shmem = true,
.needs_aux_proc = false,
.keep_postmaster_memcontext = true,
--
2.21.0
0010-Update-StartSubprocess-comments.patch (text/x-patch, 1.4K) download
From 084eb3d257f49c4b25df9911e46d649a09399a2e Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 4 Mar 2020 04:00:55 +0000
Subject: [PATCH 10/11] Update StartSubprocess comments
---
src/backend/postmaster/postmaster.c | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index f7a433f3a3..06622d3acb 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -5375,11 +5375,13 @@ CountChildren(int target)
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
- * "type" determines what kind of child will be started. All child types
- * initially go to AuxiliaryProcessMain, which will handle common setup.
+ * "type" determines what kind of child will be started. This code relies
+ * heavily on MySubprocess, which is an entry in the process_types struct in
+ * subprocess.c. All auxiliary processes go through AuxiliaryProcessMain after
+ * forking. Everything else goes through the designated entrypoint.
*
- * Return value of StartChildProcess is subprocess' PID, or 0 if failed
- * to start subprocess.
+ * Return value of StartChildProcess is subprocess' PID on success. On failure,
+ * the subprocess cleanup function decides whether or not to panic.
*/
static pid_t
StartSubprocess(SubprocessType type)
--
2.21.0
0009-Move-Backend-specific-code-down.patch (text/x-patch, 9.1K) download
From be1aaed474578ee5076bf53c939164f507ab62e3 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 4 Mar 2020 03:53:59 +0000
Subject: [PATCH 09/11] Move Backend-specific code down
---
src/backend/postmaster/postmaster.c | 330 ++++++++++++++--------------
1 file changed, 165 insertions(+), 165 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index ceb738bf1e..f7a433f3a3 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -516,171 +516,6 @@ HANDLE PostmasterHandle;
static Port *ConnProcPort = NULL;
-/*
- * BackendPrep
- *
- * Do all of the pre-fork setup for a new Backend.
-*/
-int
-BackendPrep(int argc, char *argv[])
-{
- /*
- * Create backend data structure. Better before the fork() so we can
- * handle failure cleanly.
- */
- Backend *bn;
-
- bn = (Backend *) malloc(sizeof(Backend));
- if (!bn)
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- }
-
- /*
- * Compute the cancel key that will be assigned to this backend. The
- * backend will have its own copy in the forked-off process' value of
- * MyCancelKey, so that it can transmit the key to the frontend.
- */
- if (!RandomCancelKey(&MyCancelKey))
- {
- free(bn);
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("could not generate random cancel key")));
- }
-
- bn->cancel_key = MyCancelKey;
-
- /* Pass down canAcceptConnections state */
- ConnProcPort->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
- bn->dead_end = (ConnProcPort->canAcceptConnections != CAC_OK &&
- ConnProcPort->canAcceptConnections != CAC_WAITBACKUP);
-
- /*
- * Unless it's a dead_end child, assign it a child slot number
- */
- if (!bn->dead_end)
- bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
- else
- bn->child_slot = 0;
-
- /* Hasn't asked to be notified about any bgworkers yet */
- bn->bgworker_notify = false;
-
- /*
- * Push the Backend to the stack as-is so it can be retreived by the parent
- * on the other side of the fork/exec. They will modify this entry in place.
- */
- dlist_push_head(&BackendList, &bn->elem);
-
- return 0;
-}
-
-/*
- * BackendMain
- *
- * Child code when forking a Backend.
- */
-void BackendMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
-{
- /*
- * Perform additional initialization and collect startup packet.
- *
- * We want to do this before InitProcess() for a couple of reasons: 1.
- * so that we aren't eating up a PGPROC slot while waiting on the
- * client. 2. so that if InitProcess() fails due to being out of
- * PGPROC slots, we have already initialized libpq and are able to
- * report the error to the client.
- */
- BackendInitialize(ConnProcPort);
-
- /* And run the backend */
- BackendRun(ConnProcPort); /* does not return */
-}
-
-/*
- * BackendPostmasterMain
- *
- * Parent code when forking a Backend.
- */
-void BackendParentMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
-{
- dlist_mutable_iter iter;
-
- /*
- * Find our backend node by popping the element with a matching child slot
- * off the list. This was pushed to the stack (incomplete) by BackendPrep.
- * We need to fix it up and push back.
- */
- dlist_foreach_modify(iter, &BackendList)
- {
- Backend *bp = dlist_container(Backend, elem, iter.cur);
-
- /* MyPMChildSlot is guaranteed to match the child MyPMChildSlot */
- if (bp->child_slot == MyPMChildSlot)
- {
- /*
- * Everything's been successful, it's safe to add this backend to our list
- * of backends.
- */
- bp->pid = MyChildProcPid;
- bp->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
-
-#ifdef EXEC_BACKEND
- if (!bp->dead_end)
- ShmemBackendArrayAdd(bp);
-#endif
- break; /* There is only one entry and we've found it */
- }
- }
-
- /* in parent, successful fork */
- ereport(DEBUG2,
- (errmsg_internal("forked new backend, pid=%d socket=%d",
- (int) MyChildProcPid, (int) MyProcPort->sock)));
-}
-
-/*
- * BackendCleanup
- *
- * Backend cleanup in case a failure occurs forking a new Backend.
- */
-bool
-BackendCleanup(int child_errno)
-{
- dlist_mutable_iter iter;
-
- dlist_foreach_modify(iter, &BackendList)
- {
- Backend *bp = dlist_container(Backend, elem, iter.cur);
-
- if (bp->child_slot == MyPMChildSlot)
- {
- if (!bp->dead_end)
- {
- if (!ReleasePostmasterChildSlot(bp->child_slot))
- {
- /* Could not release child slot! Panic! */
- return true;
- }
-#ifdef EXEC_BACKEND
- ShmemBackendArrayRemove(bp);
-#endif
- }
- dlist_delete(iter.cur);
- free(bp);
- break;
- }
- }
-
- report_fork_failure_to_client(MyProcPort, child_errno);
-
- /* Don't panic */
- return false;
-}
-
/*
* Postmaster main entry point
*/
@@ -3375,6 +3210,171 @@ reaper(SIGNAL_ARGS)
errno = save_errno;
}
+/*
+ * BackendPrep
+ *
+ * Do all of the pre-fork setup for a new Backend.
+*/
+int
+BackendPrep(int argc, char *argv[])
+{
+ /*
+ * Create backend data structure. Better before the fork() so we can
+ * handle failure cleanly.
+ */
+ Backend *bn;
+
+ bn = (Backend *) malloc(sizeof(Backend));
+ if (!bn)
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+ }
+
+ /*
+ * Compute the cancel key that will be assigned to this backend. The
+ * backend will have its own copy in the forked-off process' value of
+ * MyCancelKey, so that it can transmit the key to the frontend.
+ */
+ if (!RandomCancelKey(&MyCancelKey))
+ {
+ free(bn);
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random cancel key")));
+ }
+
+ bn->cancel_key = MyCancelKey;
+
+ /* Pass down canAcceptConnections state */
+ ConnProcPort->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
+ bn->dead_end = (ConnProcPort->canAcceptConnections != CAC_OK &&
+ ConnProcPort->canAcceptConnections != CAC_WAITBACKUP);
+
+ /*
+ * Unless it's a dead_end child, assign it a child slot number
+ */
+ if (!bn->dead_end)
+ bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
+ else
+ bn->child_slot = 0;
+
+ /* Hasn't asked to be notified about any bgworkers yet */
+ bn->bgworker_notify = false;
+
+ /*
+ * Push the Backend to the stack as-is so it can be retreived by the parent
+ * on the other side of the fork/exec. They will modify this entry in place.
+ */
+ dlist_push_head(&BackendList, &bn->elem);
+
+ return 0;
+}
+
+/*
+ * BackendMain
+ *
+ * Child code when forking a Backend.
+ */
+void BackendMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ /*
+ * Perform additional initialization and collect startup packet.
+ *
+ * We want to do this before InitProcess() for a couple of reasons: 1.
+ * so that we aren't eating up a PGPROC slot while waiting on the
+ * client. 2. so that if InitProcess() fails due to being out of
+ * PGPROC slots, we have already initialized libpq and are able to
+ * report the error to the client.
+ */
+ BackendInitialize(ConnProcPort);
+
+ /* And run the backend */
+ BackendRun(ConnProcPort); /* does not return */
+}
+
+/*
+ * BackendPostmasterMain
+ *
+ * Parent code when forking a Backend.
+ */
+void BackendParentMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ dlist_mutable_iter iter;
+
+ /*
+ * Find our backend node by popping the element with a matching child slot
+ * off the list. This was pushed to the stack (incomplete) by BackendPrep.
+ * We need to fix it up and push back.
+ */
+ dlist_foreach_modify(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ /* MyPMChildSlot is guaranteed to match the child MyPMChildSlot */
+ if (bp->child_slot == MyPMChildSlot)
+ {
+ /*
+ * Everything's been successful, it's safe to add this backend to our list
+ * of backends.
+ */
+ bp->pid = MyChildProcPid;
+ bp->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
+
+#ifdef EXEC_BACKEND
+ if (!bp->dead_end)
+ ShmemBackendArrayAdd(bp);
+#endif
+ break; /* There is only one entry and we've found it */
+ }
+ }
+
+ /* in parent, successful fork */
+ ereport(DEBUG2,
+ (errmsg_internal("forked new backend, pid=%d socket=%d",
+ (int) MyChildProcPid, (int) MyProcPort->sock)));
+}
+
+/*
+ * BackendCleanup
+ *
+ * Backend cleanup in case a failure occurs forking a new Backend.
+ */
+bool
+BackendCleanup(int child_errno)
+{
+ dlist_mutable_iter iter;
+
+ dlist_foreach_modify(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->child_slot == MyPMChildSlot)
+ {
+ if (!bp->dead_end)
+ {
+ if (!ReleasePostmasterChildSlot(bp->child_slot))
+ {
+ /* Could not release child slot! Panic! */
+ return true;
+ }
+#ifdef EXEC_BACKEND
+ ShmemBackendArrayRemove(bp);
+#endif
+ }
+ dlist_delete(iter.cur);
+ free(bp);
+ break;
+ }
+ }
+
+ report_fork_failure_to_client(MyProcPort, child_errno);
+
+ /* Don't panic */
+ return false;
+}
+
/*
* Scan the bgworkers list and see if the given PID (which has just stopped
* or crashed) is in it. Handle its shutdown if so, and return true. If not a
--
2.21.0
0006-Fix-up-subprocess-init-logic-and-docs.patch (text/x-patch, 2.1K) download
From ea90a3ed0c0d346caedcb52f942d662e7c4a90f2 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 4 Mar 2020 02:59:23 +0000
Subject: [PATCH 06/11] Fix up subprocess init logic and docs
---
src/backend/bootstrap/bootstrap.c | 9 +++++----
src/backend/postmaster/postmaster.c | 9 ++++++---
2 files changed, 11 insertions(+), 7 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 76a84a15f5..bd3d80b996 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -314,11 +314,12 @@ AuxiliaryProcessMain(int argc, char *argv[])
}
/*
- * At this point, we should know what kind of process we are. If the
- * MySubprocess global has not been initialized, just cast and pass along the
- * MyAuxProcType enum
+ * We have two callers of this function: 1. StartSubprocess(), which calls
+ * this for auxiliary processes after calling InitPostmasterChild() and 2.
+ * main(), which calls this for bootstrapping postgres. In the latter case,
+ * we need to initialize the process struct.
*/
- if (MySubprocess == NULL)
+ if (!IsUnderPostmaster)
InitializeMySubprocess((SubprocessType)MyAuxProcType);
/*
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 55f4067f79..0589fd27f9 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -4918,8 +4918,9 @@ SubPostmasterMain(int argc, char *argv[])
errmsg("out of memory")));
#endif
- if (!MySubprocess)
- InitializeMySubprocess(MySubprocessType);
+ /* We should only be here once per fork */
+ Assert(!MySubprocess);
+ InitializeMySubprocess(MySubprocessType);
/*
* If appropriate, physically re-attach to shared memory segment. We want
@@ -5391,7 +5392,9 @@ StartSubprocess(SubprocessType type)
char forkname[32];
#endif
- /* This should be our first time in this function */
+ /*
+ * Get new subprocess data every time we start a new subprocess.
+ */
InitializeMySubprocess(type);
argv[argc++] = "postgres";
--
2.21.0
0005-Drop-MyBackend-and-handle-backend-fork-failure.patch (text/x-patch, 10.8K) download
From 585b3e6506e3d183219b72506e69213775d3eb2a Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 4 Mar 2020 02:04:50 +0000
Subject: [PATCH 05/11] Drop MyBackend and handle backend fork failure
---
src/backend/postmaster/bgworker.c | 4 +-
src/backend/postmaster/postmaster.c | 185 ++++++++++++--------
src/backend/postmaster/startup.c | 2 +-
src/include/postmaster/bgworker_internals.h | 3 +-
src/include/postmaster/postmaster.h | 2 +-
src/include/postmaster/startup.h | 2 +-
src/include/postmaster/subprocess.h | 2 +-
7 files changed, 121 insertions(+), 79 deletions(-)
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index 7993a0e360..1ac262fae4 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -137,7 +137,7 @@ static const struct
static bgworker_main_type LookupBackgroundWorkerFunction(const char *libraryname, const char *funcname);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
-bool BackgroundWorkerCleanup(int argc, char *argv[]);
+bool BackgroundWorkerCleanup(int child_errno);
void BackgroundWorkerParentMain(int argc, char *argv[]);
/*
@@ -934,7 +934,7 @@ BackgroundWorkerMain(pg_attribute_unused() int argc, pg_attribute_unused() char
}
bool
-BackgroundWorkerCleanup(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+BackgroundWorkerCleanup(int child_errno)
{
RegisteredBgWorker *rw = CurrentBgWorker;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index ffc3afa9d2..55f4067f79 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -145,7 +145,6 @@ static Backend *ShmemBackendArray;
BackgroundWorker *MyBgworkerEntry = NULL;
RegisteredBgWorker *CurrentBgWorker = NULL;
-static Backend *MyBackend;
static int child_errno;
/* Struct containing postmaster subprocess control info */
@@ -517,6 +516,68 @@ HANDLE PostmasterHandle;
static Port *ConnProcPort = NULL;
+/*
+ * BackendPrep
+ *
+ * Do all of the pre-fork setup for a new Backend.
+*/
+int
+BackendPrep(int argc, char *argv[])
+{
+ /*
+ * Create backend data structure. Better before the fork() so we can
+ * handle failure cleanly.
+ */
+ Backend *bn;
+
+ bn = (Backend *) malloc(sizeof(Backend));
+ if (!bn)
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+ }
+
+ /*
+ * Compute the cancel key that will be assigned to this backend. The
+ * backend will have its own copy in the forked-off process' value of
+ * MyCancelKey, so that it can transmit the key to the frontend.
+ */
+ if (!RandomCancelKey(&MyCancelKey))
+ {
+ free(bn);
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random cancel key")));
+ }
+
+ bn->cancel_key = MyCancelKey;
+
+ /* Pass down canAcceptConnections state */
+ ConnProcPort->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
+ bn->dead_end = (ConnProcPort->canAcceptConnections != CAC_OK &&
+ ConnProcPort->canAcceptConnections != CAC_WAITBACKUP);
+
+ /*
+ * Unless it's a dead_end child, assign it a child slot number
+ */
+ if (!bn->dead_end)
+ bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
+ else
+ bn->child_slot = 0;
+
+ /* Hasn't asked to be notified about any bgworkers yet */
+ bn->bgworker_notify = false;
+
+ /*
+ * Push the Backend to the stack as-is so it can be retreived by the parent
+ * on the other side of the fork/exec. They will modify this entry in place.
+ */
+ dlist_push_head(&BackendList, &bn->elem);
+
+ return 0;
+}
+
/*
* BackendMain
*
@@ -546,23 +607,39 @@ void BackendMain(pg_attribute_unused() int argc, pg_attribute_unused() char *arg
*/
void BackendParentMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
- /* in parent, successful fork */
- ereport(DEBUG2,
- (errmsg_internal("forked new backend, pid=%d socket=%d",
- (int) MyChildProcPid, (int) MyProcPort->sock)));
+ dlist_mutable_iter iter;
/*
- * Everything's been successful, it's safe to add this backend to our list
- * of backends.
+ * Find our backend node by popping the element with a matching child slot
+ * off the list. This was pushed to the stack (incomplete) by BackendPrep.
+ * We need to fix it up and push back.
*/
- MyBackend->pid = MyChildProcPid;
- MyBackend->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
- dlist_push_head(&BackendList, &MyBackend->elem);
+ dlist_foreach_modify(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ /* MyPMChildSlot is guaranteed to match the child MyPMChildSlot */
+ if (bp->child_slot == MyPMChildSlot)
+ {
+ /*
+ * Everything's been successful, it's safe to add this backend to our list
+ * of backends.
+ */
+ bp->pid = MyChildProcPid;
+ bp->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
#ifdef EXEC_BACKEND
- if (!MyBackend->dead_end)
- ShmemBackendArrayAdd(MyBackend);
+ if (!bp->dead_end)
+ ShmemBackendArrayAdd(bp);
#endif
+ break; /* There is only one entry and we've found it */
+ }
+ }
+
+ /* in parent, successful fork */
+ ereport(DEBUG2,
+ (errmsg_internal("forked new backend, pid=%d socket=%d",
+ (int) MyChildProcPid, (int) MyProcPort->sock)));
}
/*
@@ -571,71 +648,37 @@ void BackendParentMain(pg_attribute_unused() int argc, pg_attribute_unused() cha
* Backend cleanup in case a failure occurs forking a new Backend.
*/
bool
-BackendCleanup(int argc, char *argv[])
+BackendCleanup(int child_errno)
{
- if (!MyBackend->dead_end)
- (void) ReleasePostmasterChildSlot(MyBackend->child_slot);
- free(MyBackend);
-
- report_fork_failure_to_client(MyProcPort, child_errno);
-
- /* Don't panic */
- return false;
-}
+ dlist_mutable_iter iter;
-/*
- * BackendPrep
- *
- * Prepare a ForkProcType struct for starting a Backend.
- * This does all prep related to av parameters and error messages.
-*/
-int
-BackendPrep(int argc, char *argv[])
-{
- /*
- * Create backend data structure. Better before the fork() so we can
- * handle failure cleanly.
- */
- MyBackend = (Backend *) malloc(sizeof(Backend));
- if (!MyBackend)
+ dlist_foreach_modify(iter, &BackendList)
{
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- }
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
- /*
- * Compute the cancel key that will be assigned to this backend. The
- * backend will have its own copy in the forked-off process' value of
- * MyCancelKey, so that it can transmit the key to the frontend.
- */
- if (!RandomCancelKey(&MyCancelKey))
- {
- free(MyBackend);
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("could not generate random cancel key")));
+ if (bp->child_slot == MyPMChildSlot)
+ {
+ if (!bp->dead_end)
+ {
+ if (!ReleasePostmasterChildSlot(bp->child_slot))
+ {
+ /* Could not release child slot! Panic! */
+ return true;
+ }
+#ifdef EXEC_BACKEND
+ ShmemBackendArrayRemove(bp);
+#endif
+ }
+ dlist_delete(iter.cur);
+ free(bp);
+ break;
+ }
}
- MyBackend->cancel_key = MyCancelKey;
-
- /* Pass down canAcceptConnections state */
- ConnProcPort->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
- MyBackend->dead_end = (ConnProcPort->canAcceptConnections != CAC_OK &&
- ConnProcPort->canAcceptConnections != CAC_WAITBACKUP);
-
- /*
- * Unless it's a dead_end child, assign it a child slot number
- */
- if (!MyBackend->dead_end)
- MyBackend->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
- else
- MyBackend->child_slot = 0;
-
- /* Hasn't asked to be notified about any bgworkers yet */
- MyBackend->bgworker_notify = false;
+ report_fork_failure_to_client(MyProcPort, child_errno);
- return 0;
+ /* Don't panic */
+ return false;
}
/*
@@ -5429,7 +5472,7 @@ StartSubprocess(SubprocessType type)
* Panic if the cleanup function tells us to. Otherwise, just bail
* with -1.
*/
- if (MySubprocess->cleanup(argc, argv))
+ if (MySubprocess->cleanup(child_errno))
ExitPostmaster(1);
else
return -1;
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index 8957b341b9..975492e674 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -128,7 +128,7 @@ HandleStartupProcInterrupts(void)
}
bool
-StartupCleanup(int argc, char *argv[])
+StartupCleanup(int child_errno)
{
/* Panic! */
return true;
diff --git a/src/include/postmaster/bgworker_internals.h b/src/include/postmaster/bgworker_internals.h
index 366bbcf1e3..aac31980fc 100644
--- a/src/include/postmaster/bgworker_internals.h
+++ b/src/include/postmaster/bgworker_internals.h
@@ -58,8 +58,7 @@ extern void ResetBackgroundWorkerCrashTimes(void);
extern int BgWorkerPrep(int argc, char *argv[]);
extern void BackgroundWorkerMain(int argc, char *argv[]);
extern void BackgroundWorkerParentMain(int argc, char *argv[]);
-extern bool BackgroundWorkerCleanup(int argc, char *argv[]);
-
+extern bool BackgroundWorkerCleanup(int child_errno);
#ifdef EXEC_BACKEND
extern BackgroundWorker *BackgroundWorkerEntry(int slotno);
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index d4c0f46220..f13fcc880a 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -94,7 +94,7 @@ extern bool RandomCancelKey(int32 *cancel_key);
extern int BackendPrep(int argc, char *argv[]);
extern void BackendMain(int argc, char *argv[]);
extern void BackendParentMain(int argc, char *argv[]);
-extern bool BackendCleanup(int argc, char *argv[]);
+extern bool BackendCleanup(int child_errno);
extern void BackgroundWorkerMain(int argc, char *argv[]);
#ifdef EXEC_BACKEND
diff --git a/src/include/postmaster/startup.h b/src/include/postmaster/startup.h
index 14a646136f..6a8108dfe0 100644
--- a/src/include/postmaster/startup.h
+++ b/src/include/postmaster/startup.h
@@ -20,6 +20,6 @@ extern void ResetPromoteTriggered(void);
/* Startup subprocess functions */
extern void StartupProcessMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern bool StartupCleanup(int argc, char *argv[]);
+extern bool StartupCleanup(int child_errno);
#endif /* _STARTUP_H */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index b65fc75475..2634bbd142 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -39,7 +39,7 @@ typedef enum
typedef int (*SubprocessInit) (int argc, char *argv[]);
typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
-typedef bool (*SubprocessCleanup) (int argc, char *argv[]);
+typedef bool (*SubprocessCleanup) (int child_errno);
typedef void (*SubprocessParent) (int argc, char *argv[]);
/* Current subprocess initializer */
--
2.21.0
0003-Rearrange-subprocess.c-headers.patch (text/x-patch, 1.3K) download
From 1295941def76eff9b9a97393165a83e63252158a Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 4 Mar 2020 01:58:02 +0000
Subject: [PATCH 03/11] Rearrange subprocess.c headers
---
src/backend/postmaster/subprocess.c | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 1c91c5f0c3..e127236513 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -11,18 +11,19 @@
*-------------------------------------------------------------------------
*/
#include "postgres.h"
-#include "postmaster/subprocess.h"
#include "bootstrap/bootstrap.h"
-#include "postmaster/bgwriter.h"
-#include "postmaster/walwriter.h"
-#include "postmaster/syslogger.h"
+#include "pgstat.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/bgworker_internals.h"
#include "postmaster/bgworker.h"
+#include "postmaster/bgworker_internals.h"
+#include "postmaster/bgwriter.h"
#include "postmaster/postmaster.h"
#include "postmaster/startup.h"
-#include "pgstat.h"
+#include "postmaster/subprocess.h"
+#include "postmaster/syslogger.h"
+#include "postmaster/walwriter.h"
#include "replication/walreceiver.h"
+#include "storage/ipc.h"
PgSubprocess *MySubprocess;
--
2.21.0
0004-Fix-allocation-of-MySubprocess.patch (text/x-patch, 1.0K) download
From d071fb374257dded5854c8cf905b9f9820ebcdbd Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 4 Mar 2020 01:58:39 +0000
Subject: [PATCH 04/11] Fix allocation of MySubprocess
---
src/backend/postmaster/subprocess.c | 7 ++-----
1 file changed, 2 insertions(+), 5 deletions(-)
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index e127236513..c4fe641456 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -27,7 +27,7 @@
PgSubprocess *MySubprocess;
-PgSubprocess process_types[] = {
+static PgSubprocess process_types[] = {
{
.progname = "boot",
.name = "checker",
@@ -201,9 +201,6 @@ PgSubprocess process_types[] = {
void
InitializeMySubprocess(SubprocessType type)
{
- /* Globally set subprocess information */
MySubprocessType = type;
- MySubprocess = (PgSubprocess *)malloc(sizeof(PgSubprocess));
-
- memcpy(MySubprocess, &process_types[type], sizeof(PgSubprocess));
+ MySubprocess = &process_types[type];
}
--
2.21.0
0002-Get-rid-of-pstrdup-for-fork-exec-args.patch (text/x-patch, 772B) download
From ba43e8a54c784676d55748656eb7b238a6d9bf47 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 4 Mar 2020 02:05:52 +0000
Subject: [PATCH 02/11] Get rid of pstrdup for fork/exec args
---
src/backend/postmaster/postmaster.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index dcfc6990c1..ffc3afa9d2 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -4434,7 +4434,7 @@ BackendRun(Port *port)
maxac * sizeof(char *));
ac = 0;
- av[ac++] = pstrdup("postgres");
+ av[ac++] = "postgres";
/*
* Pass any backend switches specified with -o on the postmaster's own
--
2.21.0
0001-Introduce-subprocess-infrastructure.patch (text/x-patch, 103.1K) download
From 6386af8b8b1b2fe820a8284717a534e1c06f22d1 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Fri, 27 Sep 2019 12:28:19 -0400
Subject: [PATCH 01/11] Introduce subprocess infrastructure
This is an initial attempt at coalescing all of the postmaster
subprocess startups into one central framework. The patchset introduces
a MySubprocess global, which contains an PgSubprocess entry from the
process_types array. Each entry has defined entrypoints, cleanup
functions, prep functions, and a few other control-oriented variables.
This is a rework of https://commitfest.postgresql.org/25/2259/, which
attempts to address the feedback from Andres Freund.
---
src/backend/bootstrap/bootstrap.c | 103 +--
src/backend/main/main.c | 1 +
src/backend/postmaster/Makefile | 1 +
src/backend/postmaster/autovacuum.c | 170 +---
src/backend/postmaster/bgworker.c | 157 +++-
src/backend/postmaster/bgwriter.c | 2 +-
src/backend/postmaster/checkpointer.c | 2 +-
src/backend/postmaster/pgarch.c | 83 +-
src/backend/postmaster/pgstat.c | 195 +----
src/backend/postmaster/postmaster.c | 890 +++++++-------------
src/backend/postmaster/startup.c | 8 +-
src/backend/postmaster/subprocess.c | 208 +++++
src/backend/postmaster/syslogger.c | 277 +++---
src/backend/postmaster/walwriter.c | 2 +-
src/backend/replication/walreceiver.c | 2 +-
src/backend/utils/adt/pgstatfuncs.c | 9 +-
src/backend/utils/init/globals.c | 3 +-
src/backend/utils/init/postinit.c | 2 +-
src/include/bootstrap/bootstrap.h | 4 +-
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 8 +-
src/include/postmaster/autovacuum.h | 13 +-
src/include/postmaster/bgworker_internals.h | 7 +-
src/include/postmaster/bgwriter.h | 4 +-
src/include/postmaster/fork_process.h | 6 +
src/include/postmaster/pgarch.h | 9 +-
src/include/postmaster/postmaster.h | 62 +-
src/include/postmaster/startup.h | 5 +-
src/include/postmaster/subprocess.h | 66 ++
src/include/postmaster/syslogger.h | 8 +-
src/include/postmaster/walwriter.h | 2 +-
src/include/replication/walreceiver.h | 2 +-
32 files changed, 1041 insertions(+), 1271 deletions(-)
create mode 100644 src/backend/postmaster/subprocess.c
create mode 100644 src/include/postmaster/subprocess.h
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index bfc629c753..76a84a15f5 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -35,6 +35,7 @@
#include "pgstat.h"
#include "postmaster/bgwriter.h"
#include "postmaster/startup.h"
+#include "postmaster/subprocess.h"
#include "postmaster/walwriter.h"
#include "replication/walreceiver.h"
#include "storage/bufmgr.h"
@@ -56,8 +57,6 @@ uint32 bootstrap_data_checksum_version = 0; /* No checksum */
#define ALLOC(t, c) \
((t *) MemoryContextAllocZero(TopMemoryContext, (unsigned)(c) * sizeof(t)))
-static void CheckerModeMain(void);
-static void BootstrapModeMain(void);
static void bootstrap_signals(void);
static void ShutdownAuxiliaryProcess(int code, Datum arg);
static Form_pg_attribute AllocateAttribute(void);
@@ -314,35 +313,20 @@ AuxiliaryProcessMain(int argc, char *argv[])
proc_exit(1);
}
+ /*
+ * At this point, we should know what kind of process we are. If the
+ * MySubprocess global has not been initialized, just cast and pass along the
+ * MyAuxProcType enum
+ */
+ if (MySubprocess == NULL)
+ InitializeMySubprocess((SubprocessType)MyAuxProcType);
+
/*
* Identify myself via ps
*/
if (IsUnderPostmaster)
{
- const char *statmsg;
-
- switch (MyAuxProcType)
- {
- case StartupProcess:
- statmsg = pgstat_get_backend_desc(B_STARTUP);
- break;
- case BgWriterProcess:
- statmsg = pgstat_get_backend_desc(B_BG_WRITER);
- break;
- case CheckpointerProcess:
- statmsg = pgstat_get_backend_desc(B_CHECKPOINTER);
- break;
- case WalWriterProcess:
- statmsg = pgstat_get_backend_desc(B_WAL_WRITER);
- break;
- case WalReceiverProcess:
- statmsg = pgstat_get_backend_desc(B_WAL_RECEIVER);
- break;
- default:
- statmsg = "??? process";
- break;
- }
- init_ps_display(statmsg, "", "", "");
+ init_ps_display(MySubprocess->desc, "", "", "");
}
/* Acquire configuration parameters, unless inherited from postmaster */
@@ -426,56 +410,10 @@ AuxiliaryProcessMain(int argc, char *argv[])
*/
SetProcessingMode(NormalProcessing);
- switch (MyAuxProcType)
- {
- case CheckerProcess:
- /* don't set signals, they're useless here */
- CheckerModeMain();
- proc_exit(1); /* should never return */
-
- case BootstrapProcess:
-
- /*
- * There was a brief instant during which mode was Normal; this is
- * okay. We need to be in bootstrap mode during BootStrapXLOG for
- * the sake of multixact initialization.
- */
- SetProcessingMode(BootstrapProcessing);
- bootstrap_signals();
- BootStrapXLOG();
- BootstrapModeMain();
- proc_exit(1); /* should never return */
-
- case StartupProcess:
- /* don't set signals, startup process has its own agenda */
- StartupProcessMain();
- proc_exit(1); /* should never return */
-
- case BgWriterProcess:
- /* don't set signals, bgwriter has its own agenda */
- BackgroundWriterMain();
- proc_exit(1); /* should never return */
-
- case CheckpointerProcess:
- /* don't set signals, checkpointer has its own agenda */
- CheckpointerMain();
- proc_exit(1); /* should never return */
-
- case WalWriterProcess:
- /* don't set signals, walwriter has its own agenda */
- InitXLOGAccess();
- WalWriterMain();
- proc_exit(1); /* should never return */
-
- case WalReceiverProcess:
- /* don't set signals, walreceiver has its own agenda */
- WalReceiverMain();
- proc_exit(1); /* should never return */
-
- default:
- elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
- proc_exit(1);
- }
+ /* Now jump into the subprocess main function and never look back! */
+ MySubprocess->entrypoint(argc, argv);
+
+ proc_exit(1); /* should never return */
}
/*
@@ -484,8 +422,8 @@ AuxiliaryProcessMain(int argc, char *argv[])
* settings). Since, in fact, that was already done by BaseInit(),
* we have nothing more to do here.
*/
-static void
-CheckerModeMain(void)
+void
+CheckerModeMain(int argc, char *argv[])
{
proc_exit(0);
}
@@ -497,11 +435,15 @@ CheckerModeMain(void)
* The bootstrap backend doesn't speak SQL, but instead expects
* commands in a special bootstrap language.
*/
-static void
-BootstrapModeMain(void)
+void
+BootstrapModeMain(int argc, char *argv[])
{
int i;
+ SetProcessingMode(BootstrapProcessing);
+ bootstrap_signals();
+ BootStrapXLOG();
+
Assert(!IsUnderPostmaster);
Assert(IsBootstrapProcessingMode());
@@ -516,7 +458,6 @@ BootstrapModeMain(void)
* Do backend-like initialization for bootstrap mode
*/
InitProcess();
-
InitPostgres(NULL, InvalidOid, NULL, InvalidOid, NULL, false);
/* Initialize stuff for bootstrap-file processing */
diff --git a/src/backend/main/main.c b/src/backend/main/main.c
index da3dae9e25..68b8b3a918 100644
--- a/src/backend/main/main.c
+++ b/src/backend/main/main.c
@@ -35,6 +35,7 @@
#include "common/username.h"
#include "port/atomics.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "storage/s_lock.h"
#include "storage/spin.h"
#include "tcop/tcopprot.h"
diff --git a/src/backend/postmaster/Makefile b/src/backend/postmaster/Makefile
index bfdf6a833d..52b4c2142c 100644
--- a/src/backend/postmaster/Makefile
+++ b/src/backend/postmaster/Makefile
@@ -23,6 +23,7 @@ OBJS = \
pgstat.o \
postmaster.o \
startup.o \
+ subprocess.o \
syslogger.o \
walwriter.o
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index e3a43d3296..78a4031b40 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -83,7 +83,6 @@
#include "nodes/makefuncs.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "storage/bufmgr.h"
@@ -95,6 +94,7 @@
#include "storage/procsignal.h"
#include "storage/sinvaladt.h"
#include "storage/smgr.h"
+#include "postmaster/subprocess.h"
#include "tcop/tcopprot.h"
#include "utils/fmgroids.h"
#include "utils/fmgrprotos.h"
@@ -135,8 +135,8 @@ int Log_autovacuum_min_duration = -1;
#define MAX_AUTOVAC_SLEEPTIME 300 /* seconds */
/* Flags to tell if we are in an autovacuum process */
-static bool am_autovacuum_launcher = false;
-static bool am_autovacuum_worker = false;
+bool am_autovacuum_launcher = false;
+bool am_autovacuum_worker = false;
/* Flags set by signal handlers */
static volatile sig_atomic_t got_SIGUSR2 = false;
@@ -302,13 +302,6 @@ static WorkerInfo MyWorkerInfo = NULL;
/* PID of launcher, valid only in worker while shutting down */
int AutovacuumLauncherPid = 0;
-#ifdef EXEC_BACKEND
-static pid_t avlauncher_forkexec(void);
-static pid_t avworker_forkexec(void);
-#endif
-NON_EXEC_STATIC void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-NON_EXEC_STATIC void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
-
static Oid do_start_worker(void);
static void HandleAutoVacLauncherInterrupts(void);
static void AutoVacLauncherShutdown(void) pg_attribute_noreturn();
@@ -346,95 +339,22 @@ static void autovac_report_workitem(AutoVacuumWorkItem *workitem,
static void avl_sigusr2_handler(SIGNAL_ARGS);
static void autovac_refresh_stats(void);
-
-
/********************************************************************
* AUTOVACUUM LAUNCHER CODE
********************************************************************/
-#ifdef EXEC_BACKEND
-/*
- * forkexec routine for the autovacuum launcher process.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avlauncher_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavlauncher";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
-/*
- * We need this set from the outside, before InitProcess is called
- */
-void
-AutovacuumLauncherIAm(void)
-{
- am_autovacuum_launcher = true;
-}
-#endif
-
-/*
- * Main entry point for autovacuum launcher process, to be called from the
- * postmaster.
- */
-int
-StartAutoVacLauncher(void)
-{
- pid_t AutoVacPID;
-
-#ifdef EXEC_BACKEND
- switch ((AutoVacPID = avlauncher_forkexec()))
-#else
- switch ((AutoVacPID = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum launcher process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacLauncherMain(0, NULL);
- break;
-#endif
- default:
- return (int) AutoVacPID;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
/*
* Main loop for the autovacuum launcher process.
*/
-NON_EXEC_STATIC void
-AutoVacLauncherMain(int argc, char *argv[])
+void
+AutoVacLauncherMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
am_autovacuum_launcher = true;
/* Identify myself via ps */
- init_ps_display(pgstat_get_backend_desc(B_AUTOVAC_LAUNCHER), "", "", "");
+ init_ps_display(MySubprocess->desc, "", "", "");
ereport(DEBUG1,
(errmsg("autovacuum launcher started")));
@@ -1423,83 +1343,11 @@ avl_sigusr2_handler(SIGNAL_ARGS)
* AUTOVACUUM WORKER CODE
********************************************************************/
-#ifdef EXEC_BACKEND
-/*
- * forkexec routines for the autovacuum worker.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avworker_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavworker";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
-/*
- * We need this set from the outside, before InitProcess is called
- */
-void
-AutovacuumWorkerIAm(void)
-{
- am_autovacuum_worker = true;
-}
-#endif
-
-/*
- * Main entry point for autovacuum worker process.
- *
- * This code is heavily based on pgarch.c, q.v.
- */
-int
-StartAutoVacWorker(void)
-{
- pid_t worker_pid;
-
-#ifdef EXEC_BACKEND
- switch ((worker_pid = avworker_forkexec()))
-#else
- switch ((worker_pid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum worker process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacWorkerMain(0, NULL);
- break;
-#endif
- default:
- return (int) worker_pid;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
/*
* AutoVacWorkerMain
*/
-NON_EXEC_STATIC void
-AutoVacWorkerMain(int argc, char *argv[])
+void
+AutoVacWorkerMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
Oid dbid;
@@ -1507,7 +1355,7 @@ AutoVacWorkerMain(int argc, char *argv[])
am_autovacuum_worker = true;
/* Identify myself via ps */
- init_ps_display(pgstat_get_backend_desc(B_AUTOVAC_WORKER), "", "", "");
+ init_ps_display(MySubprocess->desc, "", "", "");
SetProcessingMode(InitProcessing);
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index 75fc0d5d33..7993a0e360 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -33,8 +33,10 @@
#include "storage/shmem.h"
#include "tcop/tcopprot.h"
#include "utils/ascii.h"
+#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/timeout.h"
+#include "utils/timestamp.h"
/*
* The postmaster's list of registered background workers, in private memory.
@@ -133,7 +135,106 @@ static const struct
/* Private functions. */
static bgworker_main_type LookupBackgroundWorkerFunction(const char *libraryname, const char *funcname);
+static bool assign_backendlist_entry(RegisteredBgWorker *rw);
+bool BackgroundWorkerCleanup(int argc, char *argv[]);
+void BackgroundWorkerParentMain(int argc, char *argv[]);
+
+/*
+ * Allocate the Backend struct for a connected background worker, but don't
+ * add it to the list of backends just yet.
+ *
+ * On failure, return false without changing any worker state.
+ *
+ * Some info from the Backend is copied into the passed rw.
+ */
+static bool
+assign_backendlist_entry(RegisteredBgWorker *rw)
+{
+ Backend *bn;
+
+ /*
+ * Compute the cancel key that will be assigned to this session. We
+ * probably don't need cancel keys for background workers, but we'd better
+ * have something random in the field to prevent unfriendly people from
+ * sending cancels to them.
+ */
+ if (!RandomCancelKey(&MyCancelKey))
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random cancel key")));
+ return false;
+ }
+
+ bn = malloc(sizeof(Backend));
+ if (bn == NULL)
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+ return false;
+ }
+
+ bn->cancel_key = MyCancelKey;
+ bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
+ bn->bkend_type = BACKEND_TYPE_BGWORKER;
+ bn->dead_end = false;
+ bn->bgworker_notify = false;
+
+ rw->rw_backend = bn;
+ rw->rw_child_slot = bn->child_slot;
+
+ return true;
+}
+
+/*
+ * BgWorkerPrep
+ *
+ * Postmaster subroutine to prepare a bgworker subprocess.
+ *
+ * Returns 0 on success or -1 on failure.
+ *
+ * Note: if fail, we will be called again from the postmaster main loop.
+ */
+int
+BgWorkerPrep(int argc, char *argv[])
+{
+ RegisteredBgWorker *rw = CurrentBgWorker;
+#ifdef EXEC_BACKEND
+ char forkav[MAXPGPATH];
+#endif
+
+ Assert(CurrentBgWorker);
+ Assert(rw->rw_pid == 0);
+
+ /*
+ * Allocate and assign the Backend element. Note we must do this before
+ * forking, so that we can handle out of memory properly.
+ *
+ * Treat failure as though the worker had crashed. That way, the
+ * postmaster will wait a bit before attempting to start it again; if it
+ * tried again right away, most likely it'd find itself repeating the
+ * out-of-memory or fork failure condition.
+ */
+ if (!assign_backendlist_entry(rw))
+ {
+ rw->rw_crashed_at = GetCurrentTimestamp();
+ return -1;
+ }
+
+ ereport(DEBUG1,
+ (errmsg("starting background worker process \"%s\"",
+ rw->rw_worker.bgw_name)));
+
+#ifdef EXEC_BACKEND
+ /* Rewrite bgworker fork parameter with shmem_slot */
+ snprintf(forkav, MAXPGPATH, "%s=%d", argv[1], rw->rw_shmem_slot);
+ argv[1] = forkav;
+#endif
+
+ return 0;
+}
/*
* Calculate shared memory needed.
@@ -677,12 +778,28 @@ bgworker_sigusr1_handler(SIGNAL_ARGS)
* postmaster.
*/
void
-StartBackgroundWorker(void)
+BackgroundWorkerMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
- BackgroundWorker *worker = MyBgworkerEntry;
+ BackgroundWorker *worker;
bgworker_main_type entrypt;
+#ifndef EXEC_BACKEND
+ /*
+ * Before blowing away PostmasterContext, save this bgworker's
+ * data where it can find it.
+ */
+ MyBgworkerEntry = (BackgroundWorker *)
+ MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
+ memcpy(MyBgworkerEntry, &(CurrentBgWorker->rw_worker), sizeof(BackgroundWorker));
+
+ /* Now we can release postmaster's working memory context */
+ MemoryContextSwitchTo(TopMemoryContext);
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+#endif
+
+ worker = MyBgworkerEntry;
if (worker == NULL)
elog(FATAL, "unable to find bgworker entry");
@@ -816,6 +933,42 @@ StartBackgroundWorker(void)
proc_exit(0);
}
+bool
+BackgroundWorkerCleanup(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ RegisteredBgWorker *rw = CurrentBgWorker;
+
+ /* in postmaster, fork failed ... */
+ ereport(LOG,
+ (errmsg("could not fork worker process: %m")));
+
+ /* undo what assign_backendlist_entry did */
+ ReleasePostmasterChildSlot(rw->rw_child_slot);
+ rw->rw_child_slot = 0;
+ free(rw->rw_backend);
+ rw->rw_backend = NULL;
+ /* mark entry as crashed, so we'll try again later */
+ rw->rw_crashed_at = GetCurrentTimestamp();
+
+ /* Don't panic. */
+ return false;
+}
+
+void
+BackgroundWorkerParentMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ RegisteredBgWorker *rw = CurrentBgWorker;
+
+ rw->rw_pid = MyChildProcPid;
+ rw->rw_backend->pid = rw->rw_pid;
+ ReportBackgroundWorkerPID(rw);
+ /* add new worker to lists of backends */
+ dlist_push_head(&BackendList, &(rw->rw_backend->elem));
+#ifdef EXEC_BACKEND
+ ShmemBackendArrayAdd(rw->rw_backend);
+#endif
+}
+
/*
* Register a new static background worker.
*
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 069e27e427..b2170938f3 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -91,7 +91,7 @@ static XLogRecPtr last_snapshot_lsn = InvalidXLogRecPtr;
* basic execution environment, but not enabled signals yet.
*/
void
-BackgroundWriterMain(void)
+BackgroundWriterMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
MemoryContext bgwriter_context;
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index e354a78725..83f7be1b1e 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -180,7 +180,7 @@ static void ReqCheckpointHandler(SIGNAL_ARGS);
* basic execution environment, but not enabled signals yet.
*/
void
-CheckpointerMain(void)
+CheckpointerMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
MemoryContext checkpointer_context;
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 3ca30badb2..02c98d5d9f 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -91,11 +91,7 @@ static volatile sig_atomic_t ready_to_stop = false;
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgarch_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
+void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgarch_exit(SIGNAL_ARGS);
static void pgarch_waken(SIGNAL_ARGS);
static void pgarch_waken_stop(SIGNAL_ARGS);
@@ -112,7 +108,7 @@ static void pgarch_archiveDone(char *xlog);
*/
/*
- * pgarch_start
+ * PgArchiverPrep
*
* Called from postmaster at startup or after an existing archiver
* died. Attempt to fire up a fresh archiver process.
@@ -122,10 +118,9 @@ static void pgarch_archiveDone(char *xlog);
* Note: if fail, we will be called again from the postmaster main loop.
*/
int
-pgarch_start(void)
+PgArchiverPrep(int argc, char *argv[])
{
time_t curtime;
- pid_t pgArchPid;
/*
* Do nothing if no archiver needed
@@ -140,43 +135,10 @@ pgarch_start(void)
* the postmaster main loop, we will get another chance later.
*/
curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgarch_start_time) <
+ if ((unsigned int) (curtime - last_pgarch_start_time) >=
(unsigned int) PGARCH_RESTART_INTERVAL)
- return 0;
- last_pgarch_start_time = curtime;
-
-#ifdef EXEC_BACKEND
- switch ((pgArchPid = pgarch_forkexec()))
-#else
- switch ((pgArchPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork archiver: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgArchiverMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgArchPid;
- }
+ last_pgarch_start_time = curtime;
- /* shouldn't get here */
return 0;
}
@@ -186,42 +148,19 @@ pgarch_start(void)
*/
-#ifdef EXEC_BACKEND
-
-/*
- * pgarch_forkexec() -
- *
- * Format up the arglist for, then fork and exec, archive process
- */
-static pid_t
-pgarch_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
-
- av[ac++] = "--forkarch";
-
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
/*
* PgArchiverMain
*
* The argc/argv parameters are valid only in EXEC_BACKEND case. However,
* since we don't use 'em, it hardly matters...
*/
-NON_EXEC_STATIC void
-PgArchiverMain(int argc, char *argv[])
+void
+PgArchiverMain(pg_attribute_unused() int argc, pg_attribute_unused() char *arg[])
{
+ /* Drop our connection to postmaster's shared memory, as well */
+ dsm_detach_all();
+ PGSharedMemoryDetach();
+
/*
* Ignore all signals usually bound to some action in the postmaster,
* except for SIGHUP, SIGTERM, SIGUSR1, SIGUSR2, and SIGQUIT.
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 462b4d7e06..1df3d83f3f 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -61,6 +61,7 @@
#include "storage/pg_shmem.h"
#include "storage/procsignal.h"
#include "storage/sinvaladt.h"
+#include "postmaster/subprocess.h"
#include "utils/ascii.h"
#include "utils/guc.h"
#include "utils/memutils.h"
@@ -275,11 +276,7 @@ static instr_time total_func_time;
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgstat_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
+void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgstat_beshutdown_hook(int code, Datum arg);
static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
@@ -685,46 +682,18 @@ pgstat_reset_all(void)
pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
}
-#ifdef EXEC_BACKEND
/*
- * pgstat_forkexec() -
- *
- * Format up the arglist for, then fork and exec, statistics collector process
- */
-static pid_t
-pgstat_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkcol";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
-/*
- * pgstat_start() -
+ * PgstatCollectorPrep
*
* Called from postmaster at startup or after an existing collector
* died. Attempt to fire up a fresh statistics collector.
*
- * Returns PID of child process, or 0 if fail.
- *
- * Note: if fail, we will be called again from the postmaster main loop.
*/
int
-pgstat_start(void)
+PgstatCollectorPrep(int argc, char *argv[])
{
time_t curtime;
- pid_t pgStatPid;
/*
* Check that the socket is there, else pgstat_init failed and we can do
@@ -733,53 +702,18 @@ pgstat_start(void)
if (pgStatSock == PGINVALID_SOCKET)
return 0;
+ curtime = time(NULL);
+
/*
* Do nothing if too soon since last collector start. This is a safety
* valve to protect against continuous respawn attempts if the collector
* is dying immediately at launch. Note that since we will be re-called
* from the postmaster main loop, we will get another chance later.
*/
- curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgstat_start_time) <
+ if ((unsigned int) (curtime - last_pgstat_start_time) >=
(unsigned int) PGSTAT_RESTART_INTERVAL)
- return 0;
- last_pgstat_start_time = curtime;
-
- /*
- * Okay, fork off the collector.
- */
-#ifdef EXEC_BACKEND
- switch ((pgStatPid = pgstat_forkexec()))
-#else
- switch ((pgStatPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork statistics collector: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
+ last_pgstat_start_time = curtime;
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgstatCollectorMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgStatPid;
- }
-
- /* shouldn't get here */
return 0;
}
@@ -2893,61 +2827,6 @@ pgstat_bestart(void)
*/
lbeentry.st_procpid = MyProcPid;
- if (MyBackendId != InvalidBackendId)
- {
- if (IsAutoVacuumLauncherProcess())
- {
- /* Autovacuum Launcher */
- lbeentry.st_backendType = B_AUTOVAC_LAUNCHER;
- }
- else if (IsAutoVacuumWorkerProcess())
- {
- /* Autovacuum Worker */
- lbeentry.st_backendType = B_AUTOVAC_WORKER;
- }
- else if (am_walsender)
- {
- /* Wal sender */
- lbeentry.st_backendType = B_WAL_SENDER;
- }
- else if (IsBackgroundWorker)
- {
- /* bgworker */
- lbeentry.st_backendType = B_BG_WORKER;
- }
- else
- {
- /* client-backend */
- lbeentry.st_backendType = B_BACKEND;
- }
- }
- else
- {
- /* Must be an auxiliary process */
- Assert(MyAuxProcType != NotAnAuxProcess);
- switch (MyAuxProcType)
- {
- case StartupProcess:
- lbeentry.st_backendType = B_STARTUP;
- break;
- case BgWriterProcess:
- lbeentry.st_backendType = B_BG_WRITER;
- break;
- case CheckpointerProcess:
- lbeentry.st_backendType = B_CHECKPOINTER;
- break;
- case WalWriterProcess:
- lbeentry.st_backendType = B_WAL_WRITER;
- break;
- case WalReceiverProcess:
- lbeentry.st_backendType = B_WAL_RECEIVER;
- break;
- default:
- elog(FATAL, "unrecognized process type: %d",
- (int) MyAuxProcType);
- }
- }
-
lbeentry.st_proc_start_timestamp = MyStartTimestamp;
lbeentry.st_activity_start_timestamp = 0;
lbeentry.st_state_start_timestamp = 0;
@@ -2955,9 +2834,9 @@ pgstat_bestart(void)
lbeentry.st_databaseid = MyDatabaseId;
/* We have userid for client-backends, wal-sender and bgworker processes */
- if (lbeentry.st_backendType == B_BACKEND
- || lbeentry.st_backendType == B_WAL_SENDER
- || lbeentry.st_backendType == B_BG_WORKER)
+ if (MySubprocessType == ClientBackendType
+ || MySubprocessType == WalSenderType
+ || MySubprocessType == BgWorkerType)
lbeentry.st_userid = GetSessionUserId();
else
lbeentry.st_userid = InvalidOid;
@@ -4269,48 +4148,6 @@ pgstat_get_crashed_backend_activity(int pid, char *buffer, int buflen)
return NULL;
}
-const char *
-pgstat_get_backend_desc(BackendType backendType)
-{
- const char *backendDesc = "unknown process type";
-
- switch (backendType)
- {
- case B_AUTOVAC_LAUNCHER:
- backendDesc = "autovacuum launcher";
- break;
- case B_AUTOVAC_WORKER:
- backendDesc = "autovacuum worker";
- break;
- case B_BACKEND:
- backendDesc = "client backend";
- break;
- case B_BG_WORKER:
- backendDesc = "background worker";
- break;
- case B_BG_WRITER:
- backendDesc = "background writer";
- break;
- case B_CHECKPOINTER:
- backendDesc = "checkpointer";
- break;
- case B_STARTUP:
- backendDesc = "startup";
- break;
- case B_WAL_RECEIVER:
- backendDesc = "walreceiver";
- break;
- case B_WAL_SENDER:
- backendDesc = "walsender";
- break;
- case B_WAL_WRITER:
- backendDesc = "walwriter";
- break;
- }
-
- return backendDesc;
-}
-
/* ------------------------------------------------------------
* Local support functions follow
* ------------------------------------------------------------
@@ -4423,13 +4260,19 @@ pgstat_send_bgwriter(void)
* The argc/argv parameters are valid only in EXEC_BACKEND case.
* ----------
*/
-NON_EXEC_STATIC void
-PgstatCollectorMain(int argc, char *argv[])
+void
+PgstatCollectorMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
int len;
PgStat_Msg msg;
int wr;
+#ifndef EXEC_BACKEND
+ /* Drop our connection to postmaster's shared memory, as well */
+ dsm_detach_all();
+ PGSharedMemoryDetach();
+#endif
+
/*
* Ignore all signals usually bound to some action in the postmaster,
* except SIGHUP and SIGQUIT. Note we don't need a SIGUSR1 handler to
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 55187eb910..dcfc6990c1 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -114,6 +114,7 @@
#include "postmaster/fork_process.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "postmaster/syslogger.h"
#include "replication/logicallauncher.h"
#include "replication/walsender.h"
@@ -136,62 +137,19 @@
#include "storage/spin.h"
#endif
-
-/*
- * Possible types of a backend. Beyond being the possible bkend_type values in
- * struct bkend, these are OR-able request flag bits for SignalSomeChildren()
- * and CountChildren().
- */
-#define BACKEND_TYPE_NORMAL 0x0001 /* normal backend */
-#define BACKEND_TYPE_AUTOVAC 0x0002 /* autovacuum worker process */
-#define BACKEND_TYPE_WALSND 0x0004 /* walsender process */
-#define BACKEND_TYPE_BGWORKER 0x0008 /* bgworker process */
-#define BACKEND_TYPE_ALL 0x000F /* OR of all the above */
-
-#define BACKEND_TYPE_WORKER (BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER)
-
-/*
- * List of active backends (or child processes anyway; we don't actually
- * know whether a given child has become a backend or is still in the
- * authorization phase). This is used mainly to keep track of how many
- * children we have and send them appropriate signals when necessary.
- *
- * "Special" children such as the startup, bgwriter and autovacuum launcher
- * tasks are not in this list. Autovacuum worker and walsender are in it.
- * Also, "dead_end" children are in it: these are children launched just for
- * the purpose of sending a friendly rejection message to a would-be client.
- * We must track them because they are attached to shared memory, but we know
- * they will never become live backends. dead_end children are not assigned a
- * PMChildSlot.
- *
- * Background workers are in this list, too.
- */
-typedef struct bkend
-{
- pid_t pid; /* process id of backend */
- int32 cancel_key; /* cancel key for cancels for this backend */
- int child_slot; /* PMChildSlot for this backend, if any */
-
- /*
- * Flavor of backend or auxiliary process. Note that BACKEND_TYPE_WALSND
- * backends initially announce themselves as BACKEND_TYPE_NORMAL, so if
- * bkend_type is normal, you should check for a recent transition.
- */
- int bkend_type;
- bool dead_end; /* is it going to send an error and quit? */
- bool bgworker_notify; /* gets bgworker start/stop notifications */
- dlist_node elem; /* list link in BackendList */
-} Backend;
-
-static dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
+dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
#ifdef EXEC_BACKEND
static Backend *ShmemBackendArray;
#endif
BackgroundWorker *MyBgworkerEntry = NULL;
+RegisteredBgWorker *CurrentBgWorker = NULL;
+static Backend *MyBackend;
+static int child_errno;
-
+/* Struct containing postmaster subprocess control info */
+SubprocessType MySubprocessType;
/* The socket number we are listening for connections on */
int PostPortNumber;
@@ -403,25 +361,25 @@ static void BackendInitialize(Port *port);
static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
-static int BackendStartup(Port *port);
static int ProcessStartupPacket(Port *port, bool secure_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
static int initMasks(fd_set *rmask);
static void report_fork_failure_to_client(Port *port, int errnum);
static CAC_state canAcceptConnections(int backend_type);
-static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
+#ifdef EXEC_BACKEND
+static void shmemSetup(void);
+#endif
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
-static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
-static pid_t StartChildProcess(AuxProcType type);
+static pid_t StartSubprocess(SubprocessType type);
static void StartAutovacuumWorker(void);
static void MaybeStartWalReceiver(void);
static void InitPostmasterDeathWatchHandle(void);
@@ -455,8 +413,7 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(Port *port);
-static pid_t internal_forkexec(int argc, char *argv[], Port *port);
+static pid_t internal_forkexec(int argc, char *argv[]);
/* Type for a socket that can be inherited to a client process */
#ifdef WIN32
@@ -507,6 +464,9 @@ typedef struct
TimestampTz PgStartTime;
TimestampTz PgReloadTime;
pg_time_t first_syslogger_file_time;
+
+ SubprocessType MySubprocessType;
+
bool redirection_done;
bool IsBinaryUpgrade;
int max_safe_fds;
@@ -534,15 +494,10 @@ static bool save_backend_variables(BackendParameters *param, Port *port,
HANDLE childProcess, pid_t childPid);
#endif
-static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
-#define StartupDataBase() StartChildProcess(StartupProcess)
-#define StartBackgroundWriter() StartChildProcess(BgWriterProcess)
-#define StartCheckpointer() StartChildProcess(CheckpointerProcess)
-#define StartWalWriter() StartChildProcess(WalWriterProcess)
-#define StartWalReceiver() StartChildProcess(WalReceiverProcess)
+#define BGWORKER_LEN 15
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -560,6 +515,129 @@ int postmaster_alive_fds[2] = {-1, -1};
HANDLE PostmasterHandle;
#endif
+static Port *ConnProcPort = NULL;
+
+/*
+ * BackendMain
+ *
+ * Child code when forking a Backend.
+ */
+void BackendMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ /*
+ * Perform additional initialization and collect startup packet.
+ *
+ * We want to do this before InitProcess() for a couple of reasons: 1.
+ * so that we aren't eating up a PGPROC slot while waiting on the
+ * client. 2. so that if InitProcess() fails due to being out of
+ * PGPROC slots, we have already initialized libpq and are able to
+ * report the error to the client.
+ */
+ BackendInitialize(ConnProcPort);
+
+ /* And run the backend */
+ BackendRun(ConnProcPort); /* does not return */
+}
+
+/*
+ * BackendPostmasterMain
+ *
+ * Parent code when forking a Backend.
+ */
+void BackendParentMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ /* in parent, successful fork */
+ ereport(DEBUG2,
+ (errmsg_internal("forked new backend, pid=%d socket=%d",
+ (int) MyChildProcPid, (int) MyProcPort->sock)));
+
+ /*
+ * Everything's been successful, it's safe to add this backend to our list
+ * of backends.
+ */
+ MyBackend->pid = MyChildProcPid;
+ MyBackend->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
+ dlist_push_head(&BackendList, &MyBackend->elem);
+
+#ifdef EXEC_BACKEND
+ if (!MyBackend->dead_end)
+ ShmemBackendArrayAdd(MyBackend);
+#endif
+}
+
+/*
+ * BackendCleanup
+ *
+ * Backend cleanup in case a failure occurs forking a new Backend.
+ */
+bool
+BackendCleanup(int argc, char *argv[])
+{
+ if (!MyBackend->dead_end)
+ (void) ReleasePostmasterChildSlot(MyBackend->child_slot);
+ free(MyBackend);
+
+ report_fork_failure_to_client(MyProcPort, child_errno);
+
+ /* Don't panic */
+ return false;
+}
+
+/*
+ * BackendPrep
+ *
+ * Prepare a ForkProcType struct for starting a Backend.
+ * This does all prep related to av parameters and error messages.
+*/
+int
+BackendPrep(int argc, char *argv[])
+{
+ /*
+ * Create backend data structure. Better before the fork() so we can
+ * handle failure cleanly.
+ */
+ MyBackend = (Backend *) malloc(sizeof(Backend));
+ if (!MyBackend)
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+ }
+
+ /*
+ * Compute the cancel key that will be assigned to this backend. The
+ * backend will have its own copy in the forked-off process' value of
+ * MyCancelKey, so that it can transmit the key to the frontend.
+ */
+ if (!RandomCancelKey(&MyCancelKey))
+ {
+ free(MyBackend);
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random cancel key")));
+ }
+
+ MyBackend->cancel_key = MyCancelKey;
+
+ /* Pass down canAcceptConnections state */
+ ConnProcPort->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
+ MyBackend->dead_end = (ConnProcPort->canAcceptConnections != CAC_OK &&
+ ConnProcPort->canAcceptConnections != CAC_WAITBACKUP);
+
+ /*
+ * Unless it's a dead_end child, assign it a child slot number
+ */
+ if (!MyBackend->dead_end)
+ MyBackend->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
+ else
+ MyBackend->child_slot = 0;
+
+ /* Hasn't asked to be notified about any bgworkers yet */
+ MyBackend->bgworker_notify = false;
+
+ return 0;
+}
+
/*
* Postmaster main entry point
*/
@@ -1083,7 +1161,7 @@ PostmasterMain(int argc, char *argv[])
/*
* If enabled, start up syslogger collection subprocess
*/
- SysLoggerPID = SysLogger_Start();
+ SysLoggerPID = StartSubprocess(SysLoggerType);
/*
* Reset whereToSendOutput from DestDebug (its starting state) to
@@ -1389,7 +1467,7 @@ PostmasterMain(int argc, char *argv[])
/*
* We're ready to rock and roll...
*/
- StartupPID = StartupDataBase();
+ StartupPID = StartSubprocess(StartupType);
Assert(StartupPID != 0);
StartupStatus = STARTUP_RUNNING;
pmState = PM_STARTUP;
@@ -1719,19 +1797,17 @@ ServerLoop(void)
break;
if (FD_ISSET(ListenSocket[i], &rmask))
{
- Port *port;
-
- port = ConnCreate(ListenSocket[i]);
- if (port)
+ ConnProcPort = ConnCreate(ListenSocket[i]);
+ if (ConnProcPort)
{
- BackendStartup(port);
+ StartSubprocess(ClientBackendType);
/*
* We no longer need the open socket or port structure
* in this process
*/
- StreamClose(port->sock);
- ConnFree(port);
+ StreamClose(ConnProcPort->sock);
+ ConnFree(ConnProcPort);
}
}
}
@@ -1739,7 +1815,7 @@ ServerLoop(void)
/* If we have lost the log collector, try to start a new one */
if (SysLoggerPID == 0 && Logging_collector)
- SysLoggerPID = SysLogger_Start();
+ SysLoggerPID = StartSubprocess(SysLoggerType);
/*
* If no background writer process is running, and we are not in a
@@ -1750,9 +1826,9 @@ ServerLoop(void)
pmState == PM_HOT_STANDBY)
{
if (CheckpointerPID == 0)
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
if (BgWriterPID == 0)
- BgWriterPID = StartBackgroundWriter();
+ BgWriterPID = StartSubprocess(BgWriterType);
}
/*
@@ -1761,7 +1837,7 @@ ServerLoop(void)
* be writing any new WAL).
*/
if (WalWriterPID == 0 && pmState == PM_RUN)
- WalWriterPID = StartWalWriter();
+ WalWriterPID = StartSubprocess(WalWriterType);
/*
* If we have lost the autovacuum launcher, try to start a new one. We
@@ -1773,7 +1849,7 @@ ServerLoop(void)
(AutoVacuumingActive() || start_autovac_launcher) &&
pmState == PM_RUN)
{
- AutoVacPID = StartAutoVacLauncher();
+ AutoVacPID = StartSubprocess(AutoVacLauncherType);
if (AutoVacPID != 0)
start_autovac_launcher = false; /* signal processed */
}
@@ -1781,11 +1857,11 @@ ServerLoop(void)
/* If we have lost the stats collector, try to start a new one */
if (PgStatPID == 0 &&
(pmState == PM_RUN || pmState == PM_HOT_STANDBY))
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
/* If we need to signal the autovacuum launcher, do so now */
if (avlauncher_needs_signal)
@@ -2533,12 +2609,9 @@ ConnFree(Port *conn)
* This is called during child process startup to release file descriptors
* that are not needed by that child process. The postmaster still has
* them open, of course.
- *
- * Note: we pass am_syslogger as a boolean because we don't want to set
- * the global variable yet when this is called.
*/
void
-ClosePostmasterPorts(bool am_syslogger)
+ClosePostmasterPorts(void)
{
int i;
@@ -2575,7 +2648,8 @@ ClosePostmasterPorts(bool am_syslogger)
* If using syslogger, close the read side of the pipe. We don't bother
* tracking this in fd.c, either.
*/
- if (!am_syslogger)
+ if (MySubprocessType == SysLoggerType ||
+ MySubprocessType == ClientBackendType)
{
#ifndef WIN32
if (syslogPipe[0] >= 0)
@@ -3037,22 +3111,22 @@ reaper(SIGNAL_ARGS)
* if this fails, we'll just try again later.
*/
if (CheckpointerPID == 0)
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
if (BgWriterPID == 0)
- BgWriterPID = StartBackgroundWriter();
+ BgWriterPID = StartSubprocess(BgWriterType);
if (WalWriterPID == 0)
- WalWriterPID = StartWalWriter();
+ WalWriterPID = StartSubprocess(WalWriterType);
/*
* Likewise, start other special children as needed. In a restart
* situation, some of them may be alive already.
*/
if (!IsBinaryUpgrade && AutoVacuumingActive() && AutoVacPID == 0)
- AutoVacPID = StartAutoVacLauncher();
+ AutoVacPID = StartSubprocess(AutoVacLauncherType);
if (PgArchStartupAllowed() && PgArchPID == 0)
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
if (PgStatPID == 0)
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
/* workers may be scheduled to start now */
maybe_start_bgworkers();
@@ -3198,7 +3272,7 @@ reaper(SIGNAL_ARGS)
LogChildExit(LOG, _("archiver process"),
pid, exitstatus);
if (PgArchStartupAllowed())
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
continue;
}
@@ -3214,7 +3288,7 @@ reaper(SIGNAL_ARGS)
LogChildExit(LOG, _("statistics collector process"),
pid, exitstatus);
if (pmState == PM_RUN || pmState == PM_HOT_STANDBY)
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
continue;
}
@@ -3223,7 +3297,7 @@ reaper(SIGNAL_ARGS)
{
SysLoggerPID = 0;
/* for safety's sake, launch new logger *first* */
- SysLoggerPID = SysLogger_Start();
+ SysLoggerPID = StartSubprocess(SysLoggerType);
if (!EXIT_STATUS_0(exitstatus))
LogChildExit(LOG, _("system logger process"),
pid, exitstatus);
@@ -3855,7 +3929,7 @@ PostmasterStateMachine(void)
Assert(Shutdown > NoShutdown);
/* Start the checkpointer if not running */
if (CheckpointerPID == 0)
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
/* And tell it to shut down */
if (CheckpointerPID != 0)
{
@@ -3996,7 +4070,7 @@ PostmasterStateMachine(void)
reset_shared();
- StartupPID = StartupDataBase();
+ StartupPID = StartSubprocess(StartupType);
Assert(StartupPID != 0);
StartupStatus = STARTUP_RUNNING;
pmState = PM_STARTUP;
@@ -4118,122 +4192,6 @@ TerminateChildren(int signal)
signal_child(PgStatPID, signal);
}
-/*
- * BackendStartup -- start backend process
- *
- * returns: STATUS_ERROR if the fork failed, STATUS_OK otherwise.
- *
- * Note: if you change this code, also consider StartAutovacuumWorker.
- */
-static int
-BackendStartup(Port *port)
-{
- Backend *bn; /* for backend cleanup */
- pid_t pid;
-
- /*
- * Create backend data structure. Better before the fork() so we can
- * handle failure cleanly.
- */
- bn = (Backend *) malloc(sizeof(Backend));
- if (!bn)
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- return STATUS_ERROR;
- }
-
- /*
- * Compute the cancel key that will be assigned to this backend. The
- * backend will have its own copy in the forked-off process' value of
- * MyCancelKey, so that it can transmit the key to the frontend.
- */
- if (!RandomCancelKey(&MyCancelKey))
- {
- free(bn);
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("could not generate random cancel key")));
- return STATUS_ERROR;
- }
-
- bn->cancel_key = MyCancelKey;
-
- /* Pass down canAcceptConnections state */
- port->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
- bn->dead_end = (port->canAcceptConnections != CAC_OK &&
- port->canAcceptConnections != CAC_WAITBACKUP);
-
- /*
- * Unless it's a dead_end child, assign it a child slot number
- */
- if (!bn->dead_end)
- bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
- else
- bn->child_slot = 0;
-
- /* Hasn't asked to be notified about any bgworkers yet */
- bn->bgworker_notify = false;
-
-#ifdef EXEC_BACKEND
- pid = backend_forkexec(port);
-#else /* !EXEC_BACKEND */
- pid = fork_process();
- if (pid == 0) /* child */
- {
- free(bn);
-
- /* Detangle from postmaster */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Perform additional initialization and collect startup packet */
- BackendInitialize(port);
-
- /* And run the backend */
- BackendRun(port);
- }
-#endif /* EXEC_BACKEND */
-
- if (pid < 0)
- {
- /* in parent, fork failed */
- int save_errno = errno;
-
- if (!bn->dead_end)
- (void) ReleasePostmasterChildSlot(bn->child_slot);
- free(bn);
- errno = save_errno;
- ereport(LOG,
- (errmsg("could not fork new process for connection: %m")));
- report_fork_failure_to_client(port, save_errno);
- return STATUS_ERROR;
- }
-
- /* in parent, successful fork */
- ereport(DEBUG2,
- (errmsg_internal("forked new backend, pid=%d socket=%d",
- (int) pid, (int) port->sock)));
-
- /*
- * Everything's been successful, it's safe to add this backend to our list
- * of backends.
- */
- bn->pid = pid;
- bn->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
- dlist_push_head(&BackendList, &bn->elem);
-
-#ifdef EXEC_BACKEND
- if (!bn->dead_end)
- ShmemBackendArrayAdd(bn);
-#endif
-
- return STATUS_OK;
-}
-
/*
* Try to report backend fork() failure to client before we close the
* connection. Since we do not care to risk blocking the postmaster on
@@ -4433,7 +4391,7 @@ BackendInitialize(Port *port)
* init_ps_display() to avoid abusing the parameters like this.
*/
if (am_walsender)
- init_ps_display(pgstat_get_backend_desc(B_WAL_SENDER), port->user_name, remote_ps_data,
+ init_ps_display(MySubprocess->desc, port->user_name, remote_ps_data,
update_process_title ? "authentication" : "");
else
init_ps_display(port->user_name, port->database_name, remote_ps_data,
@@ -4476,7 +4434,7 @@ BackendRun(Port *port)
maxac * sizeof(char *));
ac = 0;
- av[ac++] = "postgres";
+ av[ac++] = pstrdup("postgres");
/*
* Pass any backend switches specified with -o on the postmaster's own
@@ -4527,36 +4485,11 @@ BackendRun(Port *port)
pid_t
postmaster_forkexec(int argc, char *argv[])
{
- Port port;
-
/* This entry point passes dummy values for the Port variables */
- memset(&port, 0, sizeof(port));
- return internal_forkexec(argc, argv, &port);
-}
+ if (!ConnProcPort)
+ ConnProcPort = palloc0(sizeof(*ConnProcPort));
-/*
- * backend_forkexec -- fork/exec off a backend process
- *
- * Some operating systems (WIN32) don't have fork() so we have to simulate
- * it by storing parameters that need to be passed to the child and
- * then create a new child process.
- *
- * returns the pid of the fork/exec'd process, or -1 on failure
- */
-static pid_t
-backend_forkexec(Port *port)
-{
- char *av[4];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkbackend";
- av[ac++] = NULL; /* filled in by internal_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return internal_forkexec(ac, av, port);
+ return internal_forkexec(argc, argv);
}
#ifndef WIN32
@@ -4568,7 +4501,7 @@ backend_forkexec(Port *port)
* - fork():s, and then exec():s the child process
*/
static pid_t
-internal_forkexec(int argc, char *argv[], Port *port)
+internal_forkexec(int argc, char *argv[])
{
static unsigned long tmpBackendFileNum = 0;
pid_t pid;
@@ -4576,7 +4509,7 @@ internal_forkexec(int argc, char *argv[], Port *port)
BackendParameters param;
FILE *fp;
- if (!save_backend_variables(¶m, port))
+ if (!save_backend_variables(¶m, ConnProcPort))
return -1; /* log made by save_backend_variables */
/* Calculate name for temp file */
@@ -4861,6 +4794,28 @@ retry:
}
#endif /* WIN32 */
+/*
+ * shmemSetup
+ *
+ * Helper function for a child to set up shmem before
+ * executing.
+ *
+ * aux_process - set to true if an auxiliary process.
+ */
+static void
+shmemSetup(void)
+{
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
+
+ if (MySubprocess->needs_aux_proc)
+ InitAuxiliaryProcess();
+ else
+ InitProcess();
+
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores();
+}
/*
* SubPostmasterMain -- Get the fork/exec'd process into a state equivalent
@@ -4877,6 +4832,7 @@ void
SubPostmasterMain(int argc, char *argv[])
{
Port port;
+ int shmem_slot;
/* In EXEC_BACKEND case we will not have inherited these settings */
IsPostmasterEnvironment = true;
@@ -4892,12 +4848,15 @@ SubPostmasterMain(int argc, char *argv[])
if (argc < 3)
elog(FATAL, "invalid subpostmaster invocation");
+ Assert(!ConnProcPort);
+
/* Read in the variables file */
memset(&port, 0, sizeof(Port));
read_backend_variables(argv[2], &port);
+ ConnProcPort = &port;
/* Close the postmaster's sockets (as soon as we know them) */
- ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
+ ClosePostmasterPorts();
/*
* Set reference point for stack-depth checking
@@ -4916,6 +4875,9 @@ SubPostmasterMain(int argc, char *argv[])
errmsg("out of memory")));
#endif
+ if (!MySubprocess)
+ InitializeMySubprocess(MySubprocessType);
+
/*
* If appropriate, physically re-attach to shared memory segment. We want
* to do this before going any further to ensure that we can attach at the
@@ -4932,20 +4894,16 @@ SubPostmasterMain(int argc, char *argv[])
* sometimes impossible to attach to shared memory at the desired address.
* Return the setting to its old value (usually '1' or '2') when finished.
*/
- if (strcmp(argv[1], "--forkbackend") == 0 ||
- strcmp(argv[1], "--forkavlauncher") == 0 ||
- strcmp(argv[1], "--forkavworker") == 0 ||
- strcmp(argv[1], "--forkboot") == 0 ||
- strncmp(argv[1], "--forkbgworker=", 15) == 0)
+ if (MySubprocess->needs_shmem)
PGSharedMemoryReAttach();
else
PGSharedMemoryNoReAttach();
/* autovacuum needs this set before calling InitProcess */
if (strcmp(argv[1], "--forkavlauncher") == 0)
- AutovacuumLauncherIAm();
+ am_autovacuum_launcher = true;
if (strcmp(argv[1], "--forkavworker") == 0)
- AutovacuumWorkerIAm();
+ am_autovacuum_worker = true;
/*
* Start our win32 signal implementation. This has to be done after we
@@ -4985,8 +4943,7 @@ SubPostmasterMain(int argc, char *argv[])
*/
process_shared_preload_libraries();
- /* Run backend or appropriate child */
- if (strcmp(argv[1], "--forkbackend") == 0)
+ if (MySubprocessType == ClientBackendType)
{
Assert(argc == 3); /* shouldn't be any more args */
@@ -5024,96 +4981,43 @@ SubPostmasterMain(int argc, char *argv[])
*/
BackendInitialize(&port);
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
+ shmemSetup();
/* And run the backend */
BackendRun(&port); /* does not return */
}
- if (strcmp(argv[1], "--forkboot") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitAuxiliaryProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
- }
- if (strcmp(argv[1], "--forkavlauncher") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
- }
- if (strcmp(argv[1], "--forkavworker") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
- }
- if (strncmp(argv[1], "--forkbgworker=", 15) == 0)
- {
- int shmem_slot;
- /* do this as early as possible; in particular, before InitProcess() */
+ /* do this as early as possible; in particular, before InitProcess() */
+ if (MySubprocessType == BgWorkerType)
IsBackgroundWorker = true;
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
+ shmemSetup();
- /* Fetch MyBgworkerEntry from shared memory */
- shmem_slot = atoi(argv[1] + 15);
- MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
-
- StartBackgroundWorker();
- }
- if (strcmp(argv[1], "--forkarch") == 0)
+ if (MySubprocess->needs_aux_proc)
{
- /* Do not want to attach to shared memory */
-
- PgArchiverMain(argc, argv); /* does not return */
+ AuxiliaryProcessMain(argc-2, argv+2);
}
- if (strcmp(argv[1], "--forkcol") == 0)
- {
- /* Do not want to attach to shared memory */
- PgstatCollectorMain(argc, argv); /* does not return */
- }
- if (strcmp(argv[1], "--forklog") == 0)
+ switch(MySubprocessType)
{
- /* Do not want to attach to shared memory */
-
- SysLoggerMain(argc, argv); /* does not return */
+ case BgWorkerType:
+ /* Fetch MyBgworkerEntry from shared memory */
+ shmem_slot = atoi(argv[1] + 15);
+ MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
+ /* fallthrough */
+ case PgArchiverType:
+ case PgstatCollectorType:
+ case SysLoggerType:
+ MySubprocess->entrypoint(argc, argv); /* no return */
+ break;
+ case AutoVacLauncherType:
+ case AutoVacWorkerType:
+ MySubprocess->entrypoint(argc-2, argv+2);
+ break;
+ default:
+ ereport(LOG,
+ (errmsg("could not start unknown process type (%d) under postmaster",
+ MySubprocessType)));
}
abort(); /* shouldn't get here */
@@ -5197,9 +5101,9 @@ sigusr1_handler(SIGNAL_ARGS)
* we'll just try again later.
*/
Assert(CheckpointerPID == 0);
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
Assert(BgWriterPID == 0);
- BgWriterPID = StartBackgroundWriter();
+ BgWriterPID = StartSubprocess(BgWriterType);
/*
* Start the archiver if we're responsible for (re-)archiving received
@@ -5207,7 +5111,7 @@ sigusr1_handler(SIGNAL_ARGS)
*/
Assert(PgArchPID == 0);
if (XLogArchivingAlways())
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
/*
* If we aren't planning to enter hot standby mode later, treat
@@ -5231,7 +5135,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Likewise, start other special children as needed.
*/
Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5378,7 +5282,7 @@ StartupPacketTimeoutHandler(void)
/*
* Generate a random cancel key.
*/
-static bool
+bool
RandomCancelKey(int32 *cancel_key)
{
return pg_strong_random(cancel_key, sizeof(int32));
@@ -5424,7 +5328,6 @@ CountChildren(int target)
return cnt;
}
-
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
@@ -5435,93 +5338,110 @@ CountChildren(int target)
* to start subprocess.
*/
static pid_t
-StartChildProcess(AuxProcType type)
+StartSubprocess(SubprocessType type)
{
pid_t pid;
- char *av[10];
- int ac = 0;
+ char *argv[10];
+ int argc = 0;
char typebuf[32];
+#ifdef EXEC_BACKEND
+ char forkname[32];
+#endif
- /*
- * Set up command-line arguments for subprocess
- */
- av[ac++] = "postgres";
+ /* This should be our first time in this function */
+ InitializeMySubprocess(type);
+
+ argv[argc++] = "postgres";
#ifdef EXEC_BACKEND
- av[ac++] = "--forkboot";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
+ snprintf(forkname, sizeof(forkname), "--%s", MySubprocess->progname);
+ argv[argc++] = psprintf("--%s", MySubprocess->progname);
+ argv[argc++] = NULL;
#endif
snprintf(typebuf, sizeof(typebuf), "-x%d", type);
- av[ac++] = typebuf;
+ argv[argc++] = typebuf;
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ argv[argc] = NULL;
+ Assert(argc < lengthof(argv));
+
+ /* Prep the subprocesses for a fork */
+ if (MySubprocess->init)
+ {
+ int rc = 0;
+ rc = MySubprocess->init(argc, argv);
+
+ if (rc != 0)
+ return 0;
+ }
#ifdef EXEC_BACKEND
- pid = postmaster_forkexec(ac, av);
-#else /* !EXEC_BACKEND */
+ pid = postmaster_forkexec(argc, argv);
+#else
pid = fork_process();
+#endif
+ /* some processes like backends and bgworkers need the pid */
+ MyChildProcPid = pid;
+
+#ifndef EXEC_BACKEND
if (pid == 0) /* child */
{
InitPostmasterChild();
+ ClosePostmasterPorts();
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
+ /*
+ * Release postmaster's working memory context unless we're still using
+ * it.
+ */
+ if (!MySubprocess->keep_postmaster_memcontext)
+ {
+ MemoryContextSwitchTo(TopMemoryContext);
+ MemoryContextDelete(PostmasterContext);
+
+ PostmasterContext = NULL;
+ }
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
+ /* Call the process's main subroutine */
+ if (MySubprocess->needs_aux_proc)
+ AuxiliaryProcessMain(argc, argv);
+ else
+ MySubprocess->entrypoint(argc, argv);
- AuxiliaryProcessMain(ac, av);
ExitPostmaster(0);
}
-#endif /* EXEC_BACKEND */
-
+#endif
if (pid < 0)
{
/* in parent, fork failed */
- int save_errno = errno;
+ child_errno = errno;
- errno = save_errno;
- switch (type)
- {
- case StartupProcess:
- ereport(LOG,
- (errmsg("could not fork startup process: %m")));
- break;
- case BgWriterProcess:
- ereport(LOG,
- (errmsg("could not fork background writer process: %m")));
- break;
- case CheckpointerProcess:
- ereport(LOG,
- (errmsg("could not fork checkpointer process: %m")));
- break;
- case WalWriterProcess:
- ereport(LOG,
- (errmsg("could not fork WAL writer process: %m")));
- break;
- case WalReceiverProcess:
- ereport(LOG,
- (errmsg("could not fork WAL receiver process: %m")));
- break;
- default:
- ereport(LOG,
- (errmsg("could not fork process: %m")));
- break;
- }
+ ereport(LOG,
+ (errmsg("could not fork %s: %m", MySubprocess->desc)));
/*
* fork failure is fatal during startup, but there's no need to choke
* immediately if starting other child types fails.
*/
- if (type == StartupProcess)
- ExitPostmaster(1);
+ if (MySubprocess->cleanup)
+ {
+ /*
+ * Panic if the cleanup function tells us to. Otherwise, just bail
+ * with -1.
+ */
+ if (MySubprocess->cleanup(argc, argv))
+ ExitPostmaster(1);
+ else
+ return -1;
+ }
+
return 0;
}
+ else
+ {
+ if (MySubprocess->parent_main)
+ MySubprocess->parent_main(argc, argv);
+ }
/*
* in parent, successful fork
@@ -5576,7 +5496,7 @@ StartAutovacuumWorker(void)
bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
bn->bgworker_notify = false;
- bn->pid = StartAutoVacWorker();
+ bn->pid = StartSubprocess(AutoVacWorkerType);
if (bn->pid > 0)
{
bn->bkend_type = BACKEND_TYPE_AUTOVAC;
@@ -5637,7 +5557,7 @@ MaybeStartWalReceiver(void)
pmState == PM_HOT_STANDBY || pmState == PM_WAIT_READONLY) &&
Shutdown == NoShutdown)
{
- WalReceiverPID = StartWalReceiver();
+ WalReceiverPID = StartSubprocess(WalReceiverType);
if (WalReceiverPID != 0)
WalReceiverRequested = false;
/* else leave the flag set, so we'll try again later */
@@ -5756,124 +5676,6 @@ BackgroundWorkerUnblockSignals(void)
PG_SETMASK(&UnBlockSig);
}
-#ifdef EXEC_BACKEND
-static pid_t
-bgworker_forkexec(int shmem_slot)
-{
- char *av[10];
- int ac = 0;
- char forkav[MAXPGPATH];
-
- snprintf(forkav, MAXPGPATH, "--forkbgworker=%d", shmem_slot);
-
- av[ac++] = "postgres";
- av[ac++] = forkav;
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif
-
-/*
- * Start a new bgworker.
- * Starting time conditions must have been checked already.
- *
- * Returns true on success, false on failure.
- * In either case, update the RegisteredBgWorker's state appropriately.
- *
- * This code is heavily based on autovacuum.c, q.v.
- */
-static bool
-do_start_bgworker(RegisteredBgWorker *rw)
-{
- pid_t worker_pid;
-
- Assert(rw->rw_pid == 0);
-
- /*
- * Allocate and assign the Backend element. Note we must do this before
- * forking, so that we can handle failures (out of memory or child-process
- * slots) cleanly.
- *
- * Treat failure as though the worker had crashed. That way, the
- * postmaster will wait a bit before attempting to start it again; if we
- * tried again right away, most likely we'd find ourselves hitting the
- * same resource-exhaustion condition.
- */
- if (!assign_backendlist_entry(rw))
- {
- rw->rw_crashed_at = GetCurrentTimestamp();
- return false;
- }
-
- ereport(DEBUG1,
- (errmsg("starting background worker process \"%s\"",
- rw->rw_worker.bgw_name)));
-
-#ifdef EXEC_BACKEND
- switch ((worker_pid = bgworker_forkexec(rw->rw_shmem_slot)))
-#else
- switch ((worker_pid = fork_process()))
-#endif
- {
- case -1:
- /* in postmaster, fork failed ... */
- ereport(LOG,
- (errmsg("could not fork worker process: %m")));
- /* undo what assign_backendlist_entry did */
- ReleasePostmasterChildSlot(rw->rw_child_slot);
- rw->rw_child_slot = 0;
- free(rw->rw_backend);
- rw->rw_backend = NULL;
- /* mark entry as crashed, so we'll try again later */
- rw->rw_crashed_at = GetCurrentTimestamp();
- break;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /*
- * Before blowing away PostmasterContext, save this bgworker's
- * data where it can find it.
- */
- MyBgworkerEntry = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(MyBgworkerEntry, &rw->rw_worker, sizeof(BackgroundWorker));
-
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
-
- StartBackgroundWorker();
-
- exit(1); /* should not get here */
- break;
-#endif
- default:
- /* in postmaster, fork successful ... */
- rw->rw_pid = worker_pid;
- rw->rw_backend->pid = rw->rw_pid;
- ReportBackgroundWorkerPID(rw);
- /* add new worker to lists of backends */
- dlist_push_head(&BackendList, &rw->rw_backend->elem);
-#ifdef EXEC_BACKEND
- ShmemBackendArrayAdd(rw->rw_backend);
-#endif
- return true;
- }
-
- return false;
-}
-
/*
* Does the current postmaster state require starting a worker with the
* specified start_time?
@@ -5914,67 +5716,6 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
return false;
}
-/*
- * Allocate the Backend struct for a connected background worker, but don't
- * add it to the list of backends just yet.
- *
- * On failure, return false without changing any worker state.
- *
- * Some info from the Backend is copied into the passed rw.
- */
-static bool
-assign_backendlist_entry(RegisteredBgWorker *rw)
-{
- Backend *bn;
-
- /*
- * Check that database state allows another connection. Currently the
- * only possible failure is CAC_TOOMANY, so we just log an error message
- * based on that rather than checking the error code precisely.
- */
- if (canAcceptConnections(BACKEND_TYPE_BGWORKER) != CAC_OK)
- {
- ereport(LOG,
- (errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("no slot available for new worker process")));
- return false;
- }
-
- /*
- * Compute the cancel key that will be assigned to this session. We
- * probably don't need cancel keys for background workers, but we'd better
- * have something random in the field to prevent unfriendly people from
- * sending cancels to them.
- */
- if (!RandomCancelKey(&MyCancelKey))
- {
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("could not generate random cancel key")));
- return false;
- }
-
- bn = malloc(sizeof(Backend));
- if (bn == NULL)
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- return false;
- }
-
- bn->cancel_key = MyCancelKey;
- bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
- bn->bkend_type = BACKEND_TYPE_BGWORKER;
- bn->dead_end = false;
- bn->bgworker_notify = false;
-
- rw->rw_backend = bn;
- rw->rw_child_slot = bn->child_slot;
-
- return true;
-}
-
/*
* If the time is right, start background worker(s).
*
@@ -6079,7 +5820,8 @@ maybe_start_bgworkers(void)
* crashed, but there's no need because the next run of this
* function will do that.
*/
- if (!do_start_bgworker(rw))
+ CurrentBgWorker = rw;
+ if (StartSubprocess(BgWorkerType) <= 0)
{
StartWorkerNeeded = true;
return;
@@ -6197,6 +5939,8 @@ save_backend_variables(BackendParameters *param, Port *port,
param->PgReloadTime = PgReloadTime;
param->first_syslogger_file_time = first_syslogger_file_time;
+ param->MySubprocessType = MySubprocessType;
+
param->redirection_done = redirection_done;
param->IsBinaryUpgrade = IsBinaryUpgrade;
param->max_safe_fds = max_safe_fds;
@@ -6432,6 +6176,8 @@ restore_backend_variables(BackendParameters *param, Port *port)
PgReloadTime = param->PgReloadTime;
first_syslogger_file_time = param->first_syslogger_file_time;
+ MySubprocessType = param->MySubprocessType;
+
redirection_done = param->redirection_done;
IsBinaryUpgrade = param->IsBinaryUpgrade;
max_safe_fds = param->max_safe_fds;
@@ -6486,7 +6232,7 @@ ShmemBackendArrayAllocation(void)
memset(ShmemBackendArray, 0, size);
}
-static void
+void
ShmemBackendArrayAdd(Backend *bn)
{
/* The array slot corresponding to my PMChildSlot should be free */
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index c2250d7d4e..8957b341b9 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -127,13 +127,19 @@ HandleStartupProcInterrupts(void)
ProcessProcSignalBarrier();
}
+bool
+StartupCleanup(int argc, char *argv[])
+{
+ /* Panic! */
+ return true;
+}
/* ----------------------------------
* Startup Process main entry point
* ----------------------------------
*/
void
-StartupProcessMain(void)
+StartupProcessMain(int argc, char *argv[])
{
/*
* Properly accept or ignore signals the postmaster might send us.
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
new file mode 100644
index 0000000000..1c91c5f0c3
--- /dev/null
+++ b/src/backend/postmaster/subprocess.c
@@ -0,0 +1,208 @@
+/*-------------------------------------------------------------------------
+ *
+ * subprocess.c
+ *
+ * Copyright (c) 2004-2020, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/postmaster/syslogger.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "postmaster/subprocess.h"
+#include "bootstrap/bootstrap.h"
+#include "postmaster/bgwriter.h"
+#include "postmaster/walwriter.h"
+#include "postmaster/syslogger.h"
+#include "postmaster/autovacuum.h"
+#include "postmaster/bgworker_internals.h"
+#include "postmaster/bgworker.h"
+#include "postmaster/postmaster.h"
+#include "postmaster/startup.h"
+#include "pgstat.h"
+#include "replication/walreceiver.h"
+
+PgSubprocess *MySubprocess;
+
+PgSubprocess process_types[] = {
+ {
+ .progname = "boot",
+ .name = "checker",
+ .desc = "checker process",
+ .needs_shmem = true,
+ .needs_aux_proc = true,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = CheckerModeMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "boot",
+ .name = "bootstrap",
+ .desc = "bootstrap process",
+ .needs_shmem = true,
+ .needs_aux_proc = true,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = BootstrapModeMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forkboot",
+ .name = "startup",
+ .desc = "startup process",
+ .needs_shmem = true,
+ .needs_aux_proc = true,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = StartupProcessMain,
+ .cleanup = StartupCleanup,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forkboot",
+ .name = "bgwriter",
+ .desc = "background writer process",
+ .needs_shmem = true,
+ .needs_aux_proc = true,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = BackgroundWriterMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forkboot",
+ .name = "checkpointer",
+ .desc = "checkpointer process",
+ .needs_shmem = true,
+ .needs_aux_proc = true,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = CheckpointerMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forkboot",
+ .name = "walwriter",
+ .desc = "WAL writer process",
+ .needs_shmem = true,
+ .needs_aux_proc = true,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = WalWriterMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forkboot",
+ .name = "walreceiver",
+ .desc = "WAL receiver process",
+ .needs_shmem = true,
+ .needs_aux_proc = true,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = WalReceiverMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forkavlauncher",
+ .name = "autovacuum launcher",
+ .desc = "autovacuum launcher process",
+ .needs_shmem = true,
+ .needs_aux_proc = false,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = AutoVacLauncherMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forkavworker",
+ .name = "autovacuum worker",
+ .desc = "autovacuum worker process",
+ .needs_shmem = true,
+ .needs_aux_proc = false,
+ .keep_postmaster_memcontext = false,
+ .init = NULL,
+ .entrypoint = AutoVacWorkerMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forkcol",
+ .name = "pgstat collector",
+ .desc = "statistics collector process",
+ .needs_shmem = true,
+ .needs_aux_proc = false,
+ .keep_postmaster_memcontext = false,
+ .init = PgstatCollectorPrep,
+ .entrypoint = PgstatCollectorMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forkarch",
+ .name = "archiver",
+ .desc = "archiver process",
+ .needs_shmem = false,
+ .needs_aux_proc = false,
+ .keep_postmaster_memcontext = false,
+ .init = PgArchiverPrep,
+ .entrypoint = PgArchiverMain,
+ .cleanup = NULL,
+ .parent_main = NULL
+ },
+ {
+ .progname = "forklog",
+ .name = "logger",
+ .desc = "system logger",
+ .needs_shmem = false,
+ .needs_aux_proc = false,
+ .keep_postmaster_memcontext = true,
+ .init = SysLoggerPrep,
+ .entrypoint = SysLoggerMain,
+ .cleanup = NULL,
+ .parent_main = SysLoggerParentMain
+ },
+ {
+ .progname = "forkbgworker",
+ .name = "bgworker",
+ .desc = "background worker",
+ .needs_shmem = true,
+ .needs_aux_proc = false,
+ .keep_postmaster_memcontext = true,
+ .init = BgWorkerPrep,
+ .entrypoint = BackgroundWorkerMain,
+ .cleanup = BackgroundWorkerCleanup,
+ .parent_main = BackgroundWorkerParentMain
+ },
+ {
+ .progname = "forkbackend",
+ .name = "backend",
+ .desc = "client backend",
+ .needs_shmem = true,
+ .needs_aux_proc = false,
+ .keep_postmaster_memcontext = true,
+ .init = BackendPrep,
+ .entrypoint = BackendMain,
+ .cleanup = BackendCleanup,
+ .parent_main = BackendParentMain
+ }
+};
+
+void
+InitializeMySubprocess(SubprocessType type)
+{
+ /* Globally set subprocess information */
+ MySubprocessType = type;
+ MySubprocess = (PgSubprocess *)malloc(sizeof(PgSubprocess));
+
+ memcpy(MySubprocess, &process_types[type], sizeof(PgSubprocess));
+}
diff --git a/src/backend/postmaster/syslogger.c b/src/backend/postmaster/syslogger.c
index cf7b535e4e..59497e829b 100644
--- a/src/backend/postmaster/syslogger.c
+++ b/src/backend/postmaster/syslogger.c
@@ -48,6 +48,7 @@
#include "storage/pg_shmem.h"
#include "tcop/tcopprot.h"
#include "utils/guc.h"
+#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/timestamp.h"
@@ -93,6 +94,7 @@ NON_EXEC_STATIC pg_time_t first_syslogger_file_time = 0;
static char *last_file_name = NULL;
static char *last_csv_file_name = NULL;
+
/*
* Buffers for saving partial messages from different backends.
*
@@ -131,13 +133,11 @@ static CRITICAL_SECTION sysloggerSection;
static volatile sig_atomic_t got_SIGHUP = false;
static volatile sig_atomic_t rotation_requested = false;
-
/* Local subroutines */
#ifdef EXEC_BACKEND
-static pid_t syslogger_forkexec(void);
static void syslogger_parseArgs(int argc, char *argv[]);
#endif
-NON_EXEC_STATIC void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
+
static void process_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static void flush_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static FILE *logfile_open(const char *filename, const char *mode,
@@ -158,7 +158,7 @@ static void update_metainfo_datafile(void);
* Main entry point for syslogger process
* argc/argv parameters are valid only in EXEC_BACKEND case.
*/
-NON_EXEC_STATIC void
+void
SysLoggerMain(int argc, char *argv[])
{
#ifndef WIN32
@@ -171,12 +171,16 @@ SysLoggerMain(int argc, char *argv[])
pg_time_t now;
WaitEventSet *wes;
- now = MyStartTime;
-
#ifdef EXEC_BACKEND
syslogger_parseArgs(argc, argv);
+#else
+ /* Drop our connection to postmaster's shared memory, as well */
+ dsm_detach_all();
+ PGSharedMemoryDetach();
#endif /* EXEC_BACKEND */
+ now = MyStartTime;
+
am_syslogger = true;
init_ps_display("logger", "", "", "");
@@ -540,16 +544,40 @@ SysLoggerMain(int argc, char *argv[])
}
/*
- * Postmaster subroutine to start a syslogger subprocess.
+ * Helper function for setting up a file number buffer
+ */
+static void
+write_filenobuf(char *filenobuf, FILE *file)
+{
+ /* static variables (those not passed by write_backend_variables) */
+#ifndef WIN32
+ if (file != NULL)
+ snprintf(filenobuf, sizeof(filenobuf), "%d", fileno(file));
+ else
+ strcpy(filenobuf, "-1");
+#else /* WIN32 */
+ if (file != NULL)
+ snprintf(filenobuf, sizeof(filenobuf), "%ld", (long) _get_osfhandle(_fileno(file)));
+ else
+ strcpy(filenobuf, "-1");
+#endif /* WIN32 */
+}
+
+/*
+ * SysLoggerPrep
+ *
+ * Postmaster subroutine to prepare a syslogger subprocess.
*/
int
-SysLogger_Start(void)
+SysLoggerPrep(int argc, char *argv[])
{
- pid_t sysloggerPid;
char *filename;
+ char filenobuf[32];
+ char csvfilenobuf[32];
+ /* Tell StartupSubprocess not to continue forking */
if (!Logging_collector)
- return 0;
+ return -1;
/*
* If first time through, create the pipe which will receive stderr
@@ -610,11 +638,9 @@ SysLogger_Start(void)
* a time-based rotation.
*/
first_syslogger_file_time = time(NULL);
-
filename = logfile_getname(first_syslogger_file_time, NULL);
-
syslogFile = logfile_open(filename, "a", false);
-
+ write_filenobuf(filenobuf, syslogFile);
pfree(filename);
/*
@@ -625,173 +651,97 @@ SysLogger_Start(void)
if (Log_destination & LOG_DESTINATION_CSVLOG)
{
filename = logfile_getname(first_syslogger_file_time, ".csv");
-
csvlogFile = logfile_open(filename, "a", false);
-
+ write_filenobuf(csvfilenobuf, csvlogFile);
pfree(filename);
}
-#ifdef EXEC_BACKEND
- switch ((sysloggerPid = syslogger_forkexec()))
-#else
- switch ((sysloggerPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork system logger: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(true);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- /* do the work */
- SysLoggerMain(0, NULL);
- break;
-#endif
-
- default:
- /* success, in postmaster */
-
- /* now we redirect stderr, if not done already */
- if (!redirection_done)
- {
-#ifdef WIN32
- int fd;
-#endif
-
- /*
- * Leave a breadcrumb trail when redirecting, in case the user
- * forgets that redirection is active and looks only at the
- * original stderr target file.
- */
- ereport(LOG,
- (errmsg("redirecting log output to logging collector process"),
- errhint("Future log output will appear in directory \"%s\".",
- Log_directory)));
-
-#ifndef WIN32
- fflush(stdout);
- if (dup2(syslogPipe[1], fileno(stdout)) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stdout: %m")));
- fflush(stderr);
- if (dup2(syslogPipe[1], fileno(stderr)) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- /* Now we are done with the write end of the pipe. */
- close(syslogPipe[1]);
- syslogPipe[1] = -1;
-#else
-
- /*
- * open the pipe in binary mode and make sure stderr is binary
- * after it's been dup'ed into, to avoid disturbing the pipe
- * chunking protocol.
- */
- fflush(stderr);
- fd = _open_osfhandle((intptr_t) syslogPipe[1],
- _O_APPEND | _O_BINARY);
- if (dup2(fd, _fileno(stderr)) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- close(fd);
- _setmode(_fileno(stderr), _O_BINARY);
-
- /*
- * Now we are done with the write end of the pipe.
- * CloseHandle() must not be called because the preceding
- * close() closes the underlying handle.
- */
- syslogPipe[1] = 0;
-#endif
- redirection_done = true;
- }
+ argv[argc++] = filenobuf;
+ argv[argc++] = csvfilenobuf;
- /* postmaster will never write the file(s); close 'em */
- fclose(syslogFile);
- syslogFile = NULL;
- if (csvlogFile != NULL)
- {
- fclose(csvlogFile);
- csvlogFile = NULL;
- }
- return (int) sysloggerPid;
- }
+ argv[argc] = NULL;
+ Assert(argc < lengthof(argv));
- /* we should never reach here */
return 0;
}
-
-#ifdef EXEC_BACKEND
-
/*
- * syslogger_forkexec() -
+ * SysLoggerParentMain
*
- * Format up the arglist for, then fork and exec, a syslogger process
+ * Fallthrough subroutine for parent running syslogger.
*/
-static pid_t
-syslogger_forkexec(void)
+void
+SysLoggerParentMain(int argc, char *argv[])
{
- char *av[10];
- int ac = 0;
- char filenobuf[32];
- char csvfilenobuf[32];
+ /* now we redirect stderr, if not done already */
+ if (!redirection_done)
+ {
+#ifdef WIN32
+ int fd;
+#endif
- av[ac++] = "postgres";
- av[ac++] = "--forklog";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
+ /*
+ * Leave a breadcrumb trail when redirecting, in case the user
+ * forgets that redirection is active and looks only at the
+ * original stderr target file.
+ */
+ ereport(LOG,
+ (errmsg("redirecting log output to logging collector process"),
+ errhint("Future log output will appear in directory \"%s\".",
+ Log_directory)));
- /* static variables (those not passed by write_backend_variables) */
#ifndef WIN32
- if (syslogFile != NULL)
- snprintf(filenobuf, sizeof(filenobuf), "%d",
- fileno(syslogFile));
- else
- strcpy(filenobuf, "-1");
-#else /* WIN32 */
- if (syslogFile != NULL)
- snprintf(filenobuf, sizeof(filenobuf), "%ld",
- (long) _get_osfhandle(_fileno(syslogFile)));
- else
- strcpy(filenobuf, "0");
-#endif /* WIN32 */
- av[ac++] = filenobuf;
+ fflush(stdout);
+ if (dup2(syslogPipe[1], fileno(stdout)) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stdout: %m")));
+ fflush(stderr);
+ if (dup2(syslogPipe[1], fileno(stderr)) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ /* Now we are done with the write end of the pipe. */
+ close(syslogPipe[1]);
+ syslogPipe[1] = -1;
+#else
-#ifndef WIN32
- if (csvlogFile != NULL)
- snprintf(csvfilenobuf, sizeof(csvfilenobuf), "%d",
- fileno(csvlogFile));
- else
- strcpy(csvfilenobuf, "-1");
-#else /* WIN32 */
- if (csvlogFile != NULL)
- snprintf(csvfilenobuf, sizeof(csvfilenobuf), "%ld",
- (long) _get_osfhandle(_fileno(csvlogFile)));
- else
- strcpy(csvfilenobuf, "0");
-#endif /* WIN32 */
- av[ac++] = csvfilenobuf;
+ /*
+ * open the pipe in binary mode and make sure stderr is binary
+ * after it's been dup'ed into, to avoid disturbing the pipe
+ * chunking protocol.
+ */
+ fflush(stderr);
+ fd = _open_osfhandle((intptr_t) syslogPipe[1],
+ _O_APPEND | _O_BINARY);
+ if (dup2(fd, _fileno(stderr)) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ close(fd);
+ _setmode(_fileno(stderr), _O_BINARY);
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ /*
+ * Now we are done with the write end of the pipe.
+ * CloseHandle() must not be called because the preceding
+ * close() closes the underlying handle.
+ */
+ syslogPipe[1] = 0;
+#endif
+ redirection_done = true;
+ }
- return postmaster_forkexec(ac, av);
+ /* postmaster will never write the file(s); close 'em */
+ fclose(syslogFile);
+ syslogFile = NULL;
+ if (csvlogFile != NULL)
+ {
+ fclose(csvlogFile);
+ csvlogFile = NULL;
+ }
}
+#ifdef EXEC_BACKEND
/*
* syslogger_parseArgs() -
*
@@ -800,7 +750,7 @@ syslogger_forkexec(void)
static void
syslogger_parseArgs(int argc, char *argv[])
{
- int fd;
+ int fd;
Assert(argc == 5);
argv += 3;
@@ -825,7 +775,7 @@ syslogger_parseArgs(int argc, char *argv[])
csvlogFile = fdopen(fd, "a");
setvbuf(csvlogFile, NULL, PG_IOLBF, 0);
}
-#else /* WIN32 */
+#else /* WIN32 */
fd = atoi(*argv++);
if (fd != 0)
{
@@ -846,10 +796,9 @@ syslogger_parseArgs(int argc, char *argv[])
setvbuf(csvlogFile, NULL, PG_IOLBF, 0);
}
}
-#endif /* WIN32 */
+#endif /* WIN32 */
}
-#endif /* EXEC_BACKEND */
-
+#endif
/* --------------------------------
* pipe protocol handling
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index 45a2757969..3a0cec3371 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -85,7 +85,7 @@ int WalWriterFlushAfter = 128;
* basic execution environment, but not enabled signals yet.
*/
void
-WalWriterMain(void)
+WalWriterMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
MemoryContext walwriter_context;
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index 2ab15c3cbb..725023f0f6 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -164,7 +164,7 @@ ProcessWalRcvInterrupts(void)
/* Main entry point for walreceiver process */
void
-WalReceiverMain(void)
+WalReceiverMain(int argc, char *argv[])
{
char conninfo[MAXCONNINFO];
char *tmp_conninfo;
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 54d2673254..b5692676d6 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -24,6 +24,7 @@
#include "pgstat.h"
#include "postmaster/bgworker_internals.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "storage/proc.h"
#include "storage/procarray.h"
#include "utils/acl.h"
@@ -693,7 +694,7 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
proc = BackendPidGetProc(beentry->st_procpid);
- if (proc == NULL && (beentry->st_backendType != B_BACKEND))
+ if (proc == NULL && (MySubprocessType != ClientBackendType))
{
/*
* For an auxiliary process, retrieve process info from
@@ -741,7 +742,7 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
* date.
*/
if (beentry->st_xact_start_timestamp != 0 &&
- beentry->st_backendType != B_WAL_SENDER)
+ MySubprocessType != WalSenderType)
values[8] = TimestampTzGetDatum(beentry->st_xact_start_timestamp);
else
nulls[8] = true;
@@ -829,7 +830,7 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
}
}
/* Add backend type */
- if (beentry->st_backendType == B_BG_WORKER)
+ if (MySubprocessType == BgWorkerType)
{
const char *bgw_type;
@@ -841,7 +842,7 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
}
else
values[17] =
- CStringGetTextDatum(pgstat_get_backend_desc(beentry->st_backendType));
+ CStringGetTextDatum(MySubprocess->desc);
/* SSL information */
if (beentry->st_ssl)
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index eb19644419..3e7be0bfcd 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -38,9 +38,10 @@ volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
int MyProcPid;
+int MyChildProcPid;
pg_time_t MyStartTime;
TimestampTz MyStartTimestamp;
-struct Port *MyProcPort;
+struct Port *MyProcPort = NULL;
int32 MyCancelKey;
int MyPMChildSlot;
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 8a47dcdcb1..91e43382c0 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -1112,7 +1112,7 @@ process_startup_options(Port *port, bool am_superuser)
av = (char **) palloc(maxac * sizeof(char *));
ac = 0;
- av[ac++] = "postgres";
+ av[ac++] = pstrdup("postgres");
pg_split_opts(av, &ac, port->cmdline_options);
diff --git a/src/include/bootstrap/bootstrap.h b/src/include/bootstrap/bootstrap.h
index b67fa9acc1..5d851e0081 100644
--- a/src/include/bootstrap/bootstrap.h
+++ b/src/include/bootstrap/bootstrap.h
@@ -15,7 +15,7 @@
#define BOOTSTRAP_H
#include "nodes/execnodes.h"
-
+#include "postmaster/fork_process.h"
/*
* MAXATTR is the maximum number of attributes in a relation supported
@@ -31,6 +31,8 @@ extern Relation boot_reldesc;
extern Form_pg_attribute attrtypes[MAXATTR];
extern int numattr;
+extern void CheckerModeMain(int argc, char *argv[]);
+extern void BootstrapModeMain(int argc, char *argv[]);
extern void AuxiliaryProcessMain(int argc, char *argv[]) pg_attribute_noreturn();
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index f985453ec3..823f5258d8 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -163,6 +163,7 @@ extern PGDLLIMPORT int max_worker_processes;
extern PGDLLIMPORT int max_parallel_workers;
extern PGDLLIMPORT int MyProcPid;
+extern PGDLLIMPORT int MyChildProcPid;
extern PGDLLIMPORT pg_time_t MyStartTime;
extern PGDLLIMPORT TimestampTz MyStartTimestamp;
extern PGDLLIMPORT struct Port *MyProcPort;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 7bc36c6583..811c34682b 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1049,9 +1049,6 @@ typedef struct PgBackendStatus
/* The entry is valid iff st_procpid > 0, unused if st_procpid == 0 */
int st_procpid;
- /* Type of backends */
- BackendType st_backendType;
-
/* Times when current backend, transaction, and activity started */
TimestampTz st_proc_start_timestamp;
TimestampTz st_xact_start_timestamp;
@@ -1238,13 +1235,12 @@ extern Size BackendStatusShmemSize(void);
extern void CreateSharedBackendStatus(void);
extern void pgstat_init(void);
-extern int pgstat_start(void);
extern void pgstat_reset_all(void);
extern void allow_immediate_pgstat_restart(void);
-#ifdef EXEC_BACKEND
+/* Stat collector subprocess startup functions */
+extern int PgstatCollectorPrep(int argc, char *argv[]);
extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
/* ----------
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index d40ed55531..dc02de745e 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -45,6 +45,10 @@ extern int AutovacuumLauncherPid;
extern int Log_autovacuum_min_duration;
+/* autovacuum identification */
+extern bool am_autovacuum_launcher;
+extern bool am_autovacuum_worker;
+
/* Status inquiry functions */
extern bool AutoVacuumingActive(void);
extern bool IsAutoVacuumLauncherProcess(void);
@@ -58,18 +62,15 @@ extern void autovac_init(void);
extern int StartAutoVacLauncher(void);
extern int StartAutoVacWorker(void);
+extern void AutoVacWorkerMain(int argc, char *argv[]);
+extern void AutoVacLauncherMain(int argc, char *argv[]);
+
/* called from postmaster when a worker could not be forked */
extern void AutoVacWorkerFailed(void);
/* autovacuum cost-delay balancer */
extern void AutoVacuumUpdateDelay(void);
-#ifdef EXEC_BACKEND
-extern void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern void AutovacuumWorkerIAm(void);
-extern void AutovacuumLauncherIAm(void);
-#endif
extern bool AutoVacuumRequestWork(AutoVacuumWorkItemType type,
Oid relationId, BlockNumber blkno);
diff --git a/src/include/postmaster/bgworker_internals.h b/src/include/postmaster/bgworker_internals.h
index f7e24664d5..366bbcf1e3 100644
--- a/src/include/postmaster/bgworker_internals.h
+++ b/src/include/postmaster/bgworker_internals.h
@@ -43,6 +43,7 @@ typedef struct RegisteredBgWorker
} RegisteredBgWorker;
extern slist_head BackgroundWorkerList;
+extern RegisteredBgWorker *CurrentBgWorker;
extern Size BackgroundWorkerShmemSize(void);
extern void BackgroundWorkerShmemInit(void);
@@ -54,7 +55,11 @@ extern void BackgroundWorkerStopNotifications(pid_t pid);
extern void ResetBackgroundWorkerCrashTimes(void);
/* Function to start a background worker, called from postmaster.c */
-extern void StartBackgroundWorker(void) pg_attribute_noreturn();
+extern int BgWorkerPrep(int argc, char *argv[]);
+extern void BackgroundWorkerMain(int argc, char *argv[]);
+extern void BackgroundWorkerParentMain(int argc, char *argv[]);
+extern bool BackgroundWorkerCleanup(int argc, char *argv[]);
+
#ifdef EXEC_BACKEND
extern BackgroundWorker *BackgroundWorkerEntry(int slotno);
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..6fb1ee1d6d 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -27,8 +27,8 @@ extern int CheckPointTimeout;
extern int CheckPointWarning;
extern double CheckPointCompletionTarget;
-extern void BackgroundWriterMain(void) pg_attribute_noreturn();
-extern void CheckpointerMain(void) pg_attribute_noreturn();
+extern void BackgroundWriterMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern void CheckpointerMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void RequestCheckpoint(int flags);
extern void CheckpointWriteDelay(int flags, double progress);
diff --git a/src/include/postmaster/fork_process.h b/src/include/postmaster/fork_process.h
index 8cb568fc63..a42f859a08 100644
--- a/src/include/postmaster/fork_process.h
+++ b/src/include/postmaster/fork_process.h
@@ -12,6 +12,12 @@
#ifndef FORK_PROCESS_H
#define FORK_PROCESS_H
+#include "postmaster.h"
+
extern pid_t fork_process(void);
+#ifdef EXEC_BACKEND
+extern pid_t postmaster_forkexec(int argc, char *argvp[]);
+#endif
+
#endif /* FORK_PROCESS_H */
diff --git a/src/include/postmaster/pgarch.h b/src/include/postmaster/pgarch.h
index b3200874ca..ba5b012839 100644
--- a/src/include/postmaster/pgarch.h
+++ b/src/include/postmaster/pgarch.h
@@ -26,14 +26,9 @@
#define MAX_XFN_CHARS 40
#define VALID_XFN_CHARS "0123456789ABCDEF.history.backup.partial"
-/* ----------
- * Functions called from postmaster
- * ----------
- */
-extern int pgarch_start(void);
-#ifdef EXEC_BACKEND
+/* Archiver subprocess startup functions */
+extern int PgArchiverPrep(int argc, char *argv[]);
extern void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
#endif /* _PGARCH_H */
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index babc87dfc9..d4c0f46220 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -13,6 +13,41 @@
#ifndef _POSTMASTER_H
#define _POSTMASTER_H
+#include "lib/ilist.h"
+
+/*
+ * List of active backends (or child processes anyway; we don't actually
+ * know whether a given child has become a backend or is still in the
+ * authorization phase). This is used mainly to keep track of how many
+ * children we have and send them appropriate signals when necessary.
+ *
+ * "Special" children such as the startup, bgwriter and autovacuum launcher
+ * tasks are not in this list. Autovacuum worker and walsender are in it.
+ * Also, "dead_end" children are in it: these are children launched just for
+ * the purpose of sending a friendly rejection message to a would-be client.
+ * We must track them because they are attached to shared memory, but we know
+ * they will never become live backends. dead_end children are not assigned a
+ * PMChildSlot.
+ *
+ * Background workers are in this list, too.
+ */
+typedef struct bkend
+{
+ pid_t pid; /* process id of backend */
+ int32 cancel_key; /* cancel key for cancels for this backend */
+ int child_slot; /* PMChildSlot for this backend, if any */
+
+ /*
+ * Flavor of backend or auxiliary process. Note that BACKEND_TYPE_WALSND
+ * backends initially announce themselves as BACKEND_TYPE_NORMAL, so if
+ * bkend_type is normal, you should check for a recent transition.
+ */
+ int bkend_type;
+ bool dead_end; /* is it going to send an error and quit? */
+ bool bgworker_notify; /* gets bgworker start/stop notifications */
+ dlist_node elem; /* list link in BackendList */
+} Backend;
+
/* GUC options */
extern bool EnableSSL;
extern int ReservedBackends;
@@ -47,21 +82,31 @@ extern int postmaster_alive_fds[2];
extern PGDLLIMPORT const char *progname;
extern void PostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern void ClosePostmasterPorts(bool am_syslogger);
+extern void ClosePostmasterPorts(void);
extern void InitProcessGlobals(void);
extern int MaxLivePostmasterChildren(void);
extern bool PostmasterMarkPIDForWorkerNotify(int);
+extern bool RandomCancelKey(int32 *cancel_key);
+
+extern int BackendPrep(int argc, char *argv[]);
+extern void BackendMain(int argc, char *argv[]);
+extern void BackendParentMain(int argc, char *argv[]);
+extern bool BackendCleanup(int argc, char *argv[]);
+extern void BackgroundWorkerMain(int argc, char *argv[]);
+
#ifdef EXEC_BACKEND
-extern pid_t postmaster_forkexec(int argc, char *argv[]);
extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
extern Size ShmemBackendArraySize(void);
extern void ShmemBackendArrayAllocation(void);
+extern void ShmemBackendArrayAdd(Backend *bn);
#endif
+extern PGDLLIMPORT dlist_head BackendList;
+
/*
* Note: MAX_BACKENDS is limited to 2^18-1 because that's the width reserved
* for buffer references in buf_internals.h. This limitation could be lifted
@@ -74,4 +119,17 @@ extern void ShmemBackendArrayAllocation(void);
*/
#define MAX_BACKENDS 0x3FFFF
+/*
+ * Possible types of a backend. Beyond being the possible bkend_type values in
+ * struct bkend, these are OR-able request flag bits for SignalSomeChildren()
+ * and CountChildren().
+ */
+#define BACKEND_TYPE_NORMAL 0x0001 /* normal backend */
+#define BACKEND_TYPE_AUTOVAC 0x0002 /* autovacuum worker process */
+#define BACKEND_TYPE_WALSND 0x0004 /* walsender process */
+#define BACKEND_TYPE_BGWORKER 0x0008 /* bgworker process */
+#define BACKEND_TYPE_ALL 0x000F /* OR of all the above */
+
+#define BACKEND_TYPE_WORKER (BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER)
+
#endif /* _POSTMASTER_H */
diff --git a/src/include/postmaster/startup.h b/src/include/postmaster/startup.h
index 9f59c1ffa3..14a646136f 100644
--- a/src/include/postmaster/startup.h
+++ b/src/include/postmaster/startup.h
@@ -13,10 +13,13 @@
#define _STARTUP_H
extern void HandleStartupProcInterrupts(void);
-extern void StartupProcessMain(void) pg_attribute_noreturn();
extern void PreRestoreCommand(void);
extern void PostRestoreCommand(void);
extern bool IsPromoteTriggered(void);
extern void ResetPromoteTriggered(void);
+/* Startup subprocess functions */
+extern void StartupProcessMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern bool StartupCleanup(int argc, char *argv[]);
+
#endif /* _STARTUP_H */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
new file mode 100644
index 0000000000..b65fc75475
--- /dev/null
+++ b/src/include/postmaster/subprocess.h
@@ -0,0 +1,66 @@
+/*-------------------------------------------------------------------------
+ *
+ * subprocess.h
+ *
+ * Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ *
+ * src/include/postmaster/subprocess.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef SUBPROCESS_H
+#define SUBPROCESS_H
+
+#include "postmaster.h"
+
+typedef enum
+{
+ NoProcessType = -1,
+ CheckerType = 0,
+ BootstrapType,
+ StartupType,
+ BgWriterType,
+ CheckpointerType,
+ WalWriterType,
+ WalReceiverType, /* end of Auxiliary Process Forks */
+ AutoVacLauncherType,
+ AutoVacWorkerType,
+ PgstatCollectorType,
+ PgArchiverType,
+ SysLoggerType,
+ BgWorkerType,
+ ClientBackendType,
+
+ WalSenderType, /* Dummy type for wal sender. No process struct needed as
+ * this process just takes over an existing bgworker */
+
+ NUMSUBPROCESSTYPES /* Must be last! */
+} SubprocessType;
+
+typedef int (*SubprocessInit) (int argc, char *argv[]);
+typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
+typedef bool (*SubprocessCleanup) (int argc, char *argv[]);
+typedef void (*SubprocessParent) (int argc, char *argv[]);
+
+/* Current subprocess initializer */
+extern void InitializeMySubprocess(SubprocessType type);
+
+typedef struct PgSubprocess
+{
+ const char *progname;
+ const char *name;
+ const char *desc;
+ bool needs_shmem;
+ bool needs_aux_proc;
+ bool needs_full_proc;
+ bool keep_postmaster_memcontext;
+ SubprocessInit init;
+ SubprocessEntryPoint entrypoint;
+ SubprocessCleanup cleanup;
+ SubprocessParent parent_main;
+} PgSubprocess;
+
+extern SubprocessType MySubprocessType;
+extern PGDLLIMPORT PgSubprocess *MySubprocess;
+
+#endif /* SUBPROCESS_H */
diff --git a/src/include/postmaster/syslogger.h b/src/include/postmaster/syslogger.h
index 9b7a386dbd..ededfd7c23 100644
--- a/src/include/postmaster/syslogger.h
+++ b/src/include/postmaster/syslogger.h
@@ -14,7 +14,6 @@
#include <limits.h> /* for PIPE_BUF */
-
/*
* Primitive protocol structure for writing to syslogger pipe(s). The idea
* here is to divide long messages into chunks that are not more than
@@ -83,9 +82,10 @@ extern int SysLogger_Start(void);
extern void write_syslogger_file(const char *buffer, int count, int dest);
-#ifdef EXEC_BACKEND
-extern void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+/* Subprocess startup functions */
+extern int SysLoggerPrep(int argc, char *argv[]);
+extern void SysLoggerParentMain(int argc, char *argv[]);
+extern void SysLoggerMain(int argc, char *argv[]);
extern bool CheckLogrotateSignal(void);
extern void RemoveLogrotateSignalFiles(void);
diff --git a/src/include/postmaster/walwriter.h b/src/include/postmaster/walwriter.h
index 011dc26a6e..596c437c54 100644
--- a/src/include/postmaster/walwriter.h
+++ b/src/include/postmaster/walwriter.h
@@ -16,6 +16,6 @@
extern int WalWriterDelay;
extern int WalWriterFlushAfter;
-extern void WalWriterMain(void) pg_attribute_noreturn();
+extern void WalWriterMain(int argc, char *argv[]) pg_attribute_noreturn();
#endif /* _WALWRITER_H */
diff --git a/src/include/replication/walreceiver.h b/src/include/replication/walreceiver.h
index e08afc6548..2025f19ab3 100644
--- a/src/include/replication/walreceiver.h
+++ b/src/include/replication/walreceiver.h
@@ -311,7 +311,7 @@ walrcv_clear_result(WalRcvExecResult *walres)
}
/* prototypes for functions in walreceiver.c */
-extern void WalReceiverMain(void) pg_attribute_noreturn();
+extern void WalReceiverMain(int argc, char *argv[]);
extern void ProcessWalRcvInterrupts(void);
/* prototypes for functions in walreceiverfuncs.c */
--
2.21.0
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-02 14:53 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-04 04:56 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
@ 2020-03-17 18:50 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 21:52 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
0 siblings, 1 reply; 38+ messages in thread
From: Mike Palmiotto @ 2020-03-17 18:50 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: Andres Freund <andres@anarazel.de>; Alvaro Herrera <alvherre@2ndquadrant.com>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Tue, Mar 3, 2020 at 11:56 PM Mike Palmiotto
<mike.palmiotto@crunchydata.com> wrote:
>
> On Mon, Mar 2, 2020 at 9:53 AM Mike Palmiotto
> <mike.palmiotto@crunchydata.com> wrote:
> >
> > On Sat, Feb 29, 2020 at 9:51 PM Mike Palmiotto
> > <mike.palmiotto@crunchydata.com> wrote:
> > >
> <snip>
> > Okay, here is an updated and rebased patch that passes all regression
> > tests with and without EXEC_BACKEND. This also treats more of the
> > issues raised by Andres.
> >
> > I still need to do the following:
> > - split giant patch out into separate functional commits
> > - add translated process descriptions
> > - fix some variable names here and there (notably the PgSubprocess
> > struct ".progname" member -- that's a misnomer.
> > - address some valgrind findings
> >
> > I should be able to split this out into smaller commits sometime today
> > and will continue iterating to scratch the other items off the list.
>
> I've addressed all of the points above (except splitting the base
> patch up) and rebased on master. I didn't squash before generating the
> patches so it may be easier to read the progress here, or annoying
> that there are so many attachments.
>
> This patchset hits all of Andres' points with the exception of the
> reshuffling of functions like AuxiliaryProcessMain, but it does get
> rid of a lot of direct argv string comparison. I'll reorganize a bit
> when I re-write the patchset and will likely move
> AuxiliaryProcessMain, along with the Backend subprocess functions but
> just wanted to get this up here in the meantime.
>
> There is quite a lot to be gained from this patchset in the form of
> re-organization and removal of redundant code-paths/globals. I see
> this as a good first step and am willing to continue to push it
> forward with any new suggestions.
The patchset is now split out. I've just noticed that Peter Eisentraut
included some changes for a generic MyBackendType, which I should have
been aware of. I was unable to rebase due to these changes, but can
fold these patches into that framework if others think it's
worthwhile.
The resulting diffstat is as follows:
% git diff 24d85952a57b16090ca8ad9cf800fbdd9ddd104f --shortstat
29 files changed, 1029 insertions(+), 1201 deletions(-)
--
Mike Palmiotto
https://crunchydata.com
Attachments: download all (12 files, .tar.gz)
0001-Add-subprocess-infrastructure.patch (text/x-patch, 14.6K) download
From b0ea4771d38a20d84a86c2069d82fdbab60b6ecc Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Mon, 9 Mar 2020 22:12:10 +0000
Subject: [PATCH 01/11] Add subprocess infrastructure
This commit introduces a new infrastructure for postmaster subprocesses
initialization, forking, exec'ing and main function control. The intent
is to have all subprocess control information live in the process_types
struct, defined in subprocess.h. As more processes are added to the
struct, the fields will be expanded to differentiate actions.
There are several advantages to such an infrastructure:
1) Centralized definition of subprocess information
2) Standardized function arguments/naming schemes
3) Code minimization and removal of duplicate code
---
src/backend/bootstrap/bootstrap.c | 100 +++++---------------------
src/backend/postmaster/Makefile | 1 +
src/backend/postmaster/bgwriter.c | 2 +-
src/backend/postmaster/checkpointer.c | 2 +-
src/backend/postmaster/startup.c | 3 +-
src/backend/postmaster/subprocess.c | 62 ++++++++++++++++
src/backend/postmaster/walwriter.c | 2 +-
src/backend/replication/walreceiver.c | 2 +-
src/include/bootstrap/bootstrap.h | 3 +-
src/include/postmaster/bgwriter.h | 4 +-
src/include/postmaster/startup.h | 5 +-
src/include/postmaster/subprocess.h | 44 ++++++++++++
src/include/postmaster/walwriter.h | 2 +-
src/include/replication/walreceiver.h | 2 +-
14 files changed, 141 insertions(+), 93 deletions(-)
create mode 100644 src/backend/postmaster/subprocess.c
create mode 100644 src/include/postmaster/subprocess.h
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 657b18ecc8..21a44d6d00 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -35,6 +35,7 @@
#include "pgstat.h"
#include "postmaster/bgwriter.h"
#include "postmaster/startup.h"
+#include "postmaster/subprocess.h"
#include "postmaster/walwriter.h"
#include "replication/walreceiver.h"
#include "storage/bufmgr.h"
@@ -56,8 +57,6 @@ uint32 bootstrap_data_checksum_version = 0; /* No checksum */
#define ALLOC(t, c) \
((t *) MemoryContextAllocZero(TopMemoryContext, (unsigned)(c) * sizeof(t)))
-static void CheckerModeMain(void);
-static void BootstrapModeMain(void);
static void bootstrap_signals(void);
static void ShutdownAuxiliaryProcess(int code, Datum arg);
static Form_pg_attribute AllocateAttribute(void);
@@ -314,35 +313,17 @@ AuxiliaryProcessMain(int argc, char *argv[])
proc_exit(1);
}
+ /*
+ * We've read the arguments and know what backend type we are.
+ */
+ InitMySubprocess((SubprocessType)MyAuxProcType);
+
/*
* Identify myself via ps
*/
if (IsUnderPostmaster)
{
- const char *statmsg;
-
- switch (MyAuxProcType)
- {
- case StartupProcess:
- statmsg = pgstat_get_backend_desc(B_STARTUP);
- break;
- case BgWriterProcess:
- statmsg = pgstat_get_backend_desc(B_BG_WRITER);
- break;
- case CheckpointerProcess:
- statmsg = pgstat_get_backend_desc(B_CHECKPOINTER);
- break;
- case WalWriterProcess:
- statmsg = pgstat_get_backend_desc(B_WAL_WRITER);
- break;
- case WalReceiverProcess:
- statmsg = pgstat_get_backend_desc(B_WAL_RECEIVER);
- break;
- default:
- statmsg = "??? process";
- break;
- }
- init_ps_display(statmsg, "", "", "");
+ init_ps_display(MySubprocess->desc, "", "", "");
}
/* Acquire configuration parameters, unless inherited from postmaster */
@@ -426,56 +407,10 @@ AuxiliaryProcessMain(int argc, char *argv[])
*/
SetProcessingMode(NormalProcessing);
- switch (MyAuxProcType)
- {
- case CheckerProcess:
- /* don't set signals, they're useless here */
- CheckerModeMain();
- proc_exit(1); /* should never return */
-
- case BootstrapProcess:
-
- /*
- * There was a brief instant during which mode was Normal; this is
- * okay. We need to be in bootstrap mode during BootStrapXLOG for
- * the sake of multixact initialization.
- */
- SetProcessingMode(BootstrapProcessing);
- bootstrap_signals();
- BootStrapXLOG();
- BootstrapModeMain();
- proc_exit(1); /* should never return */
-
- case StartupProcess:
- /* don't set signals, startup process has its own agenda */
- StartupProcessMain();
- proc_exit(1); /* should never return */
-
- case BgWriterProcess:
- /* don't set signals, bgwriter has its own agenda */
- BackgroundWriterMain();
- proc_exit(1); /* should never return */
-
- case CheckpointerProcess:
- /* don't set signals, checkpointer has its own agenda */
- CheckpointerMain();
- proc_exit(1); /* should never return */
-
- case WalWriterProcess:
- /* don't set signals, walwriter has its own agenda */
- InitXLOGAccess();
- WalWriterMain();
- proc_exit(1); /* should never return */
-
- case WalReceiverProcess:
- /* don't set signals, walreceiver has its own agenda */
- WalReceiverMain();
- proc_exit(1); /* should never return */
-
- default:
- elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
- proc_exit(1);
- }
+ /* Now jump into the subprocess main function and never look back! */
+ MySubprocess->entrypoint(argc, argv);
+
+ proc_exit(1); /* should never return */
}
/*
@@ -484,8 +419,8 @@ AuxiliaryProcessMain(int argc, char *argv[])
* settings). Since, in fact, that was already done by BaseInit(),
* we have nothing more to do here.
*/
-static void
-CheckerModeMain(void)
+void
+CheckerModeMain(int argc, char *argv[])
{
proc_exit(0);
}
@@ -497,11 +432,15 @@ CheckerModeMain(void)
* The bootstrap backend doesn't speak SQL, but instead expects
* commands in a special bootstrap language.
*/
-static void
-BootstrapModeMain(void)
+void
+BootstrapModeMain(int argc, char *argv[])
{
int i;
+ SetProcessingMode(BootstrapProcessing);
+ bootstrap_signals();
+ BootStrapXLOG();
+
Assert(!IsUnderPostmaster);
Assert(IsBootstrapProcessingMode());
@@ -516,7 +455,6 @@ BootstrapModeMain(void)
* Do backend-like initialization for bootstrap mode
*/
InitProcess();
-
InitPostgres(NULL, InvalidOid, NULL, InvalidOid, NULL, false);
/* Initialize stuff for bootstrap-file processing */
diff --git a/src/backend/postmaster/Makefile b/src/backend/postmaster/Makefile
index bfdf6a833d..52b4c2142c 100644
--- a/src/backend/postmaster/Makefile
+++ b/src/backend/postmaster/Makefile
@@ -23,6 +23,7 @@ OBJS = \
pgstat.o \
postmaster.o \
startup.o \
+ subprocess.o \
syslogger.o \
walwriter.o
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 069e27e427..b2170938f3 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -91,7 +91,7 @@ static XLogRecPtr last_snapshot_lsn = InvalidXLogRecPtr;
* basic execution environment, but not enabled signals yet.
*/
void
-BackgroundWriterMain(void)
+BackgroundWriterMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
MemoryContext bgwriter_context;
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index e354a78725..83f7be1b1e 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -180,7 +180,7 @@ static void ReqCheckpointHandler(SIGNAL_ARGS);
* basic execution environment, but not enabled signals yet.
*/
void
-CheckpointerMain(void)
+CheckpointerMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
MemoryContext checkpointer_context;
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index c2250d7d4e..e29649a6a1 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -127,13 +127,12 @@ HandleStartupProcInterrupts(void)
ProcessProcSignalBarrier();
}
-
/* ----------------------------------
* Startup Process main entry point
* ----------------------------------
*/
void
-StartupProcessMain(void)
+StartupProcessMain(int argc, char *argv[])
{
/*
* Properly accept or ignore signals the postmaster might send us.
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
new file mode 100644
index 0000000000..3e7a45bf10
--- /dev/null
+++ b/src/backend/postmaster/subprocess.c
@@ -0,0 +1,62 @@
+/*-------------------------------------------------------------------------
+ *
+ * subprocess.c
+ *
+ * Copyright (c) 2004-2020, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/postmaster/syslogger.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "bootstrap/bootstrap.h"
+#include "postmaster/bgwriter.h"
+#include "postmaster/postmaster.h"
+#include "postmaster/startup.h"
+#include "postmaster/subprocess.h"
+#include "postmaster/walwriter.h"
+#include "replication/walreceiver.h"
+
+SubprocessType MySubprocessType;
+PgSubprocess *MySubprocess;
+
+static PgSubprocess process_types[] = {
+ {
+ .desc = "checker",
+ .entrypoint = CheckerModeMain
+ },
+ {
+ .desc = "bootstrap",
+ .entrypoint = BootstrapModeMain
+ },
+ {
+ .desc = "startup",
+ .entrypoint = StartupProcessMain
+ },
+ {
+ .desc = "background writer",
+ .entrypoint = BackgroundWriterMain
+ },
+ {
+ .desc = "checkpointer",
+ .entrypoint = CheckpointerMain
+ },
+ {
+ .desc = "wal writer",
+ .entrypoint = WalWriterMain
+ },
+ {
+ .desc = "wal receiver",
+ .entrypoint = WalReceiverMain
+ }
+};
+
+void
+InitMySubprocess(SubprocessType type)
+{
+ MySubprocessType = type;
+ MySubprocess = &process_types[type];
+ MySubprocess->desc = gettext(MySubprocess->desc);
+}
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index 45a2757969..3a0cec3371 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -85,7 +85,7 @@ int WalWriterFlushAfter = 128;
* basic execution environment, but not enabled signals yet.
*/
void
-WalWriterMain(void)
+WalWriterMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
MemoryContext walwriter_context;
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index 2ab15c3cbb..725023f0f6 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -164,7 +164,7 @@ ProcessWalRcvInterrupts(void)
/* Main entry point for walreceiver process */
void
-WalReceiverMain(void)
+WalReceiverMain(int argc, char *argv[])
{
char conninfo[MAXCONNINFO];
char *tmp_conninfo;
diff --git a/src/include/bootstrap/bootstrap.h b/src/include/bootstrap/bootstrap.h
index b67fa9acc1..f85cfb5202 100644
--- a/src/include/bootstrap/bootstrap.h
+++ b/src/include/bootstrap/bootstrap.h
@@ -16,7 +16,6 @@
#include "nodes/execnodes.h"
-
/*
* MAXATTR is the maximum number of attributes in a relation supported
* at bootstrap time (i.e., the max possible in a system table).
@@ -31,6 +30,8 @@ extern Relation boot_reldesc;
extern Form_pg_attribute attrtypes[MAXATTR];
extern int numattr;
+extern void CheckerModeMain(int argc, char *argv[]);
+extern void BootstrapModeMain(int argc, char *argv[]);
extern void AuxiliaryProcessMain(int argc, char *argv[]) pg_attribute_noreturn();
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..6fb1ee1d6d 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -27,8 +27,8 @@ extern int CheckPointTimeout;
extern int CheckPointWarning;
extern double CheckPointCompletionTarget;
-extern void BackgroundWriterMain(void) pg_attribute_noreturn();
-extern void CheckpointerMain(void) pg_attribute_noreturn();
+extern void BackgroundWriterMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern void CheckpointerMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void RequestCheckpoint(int flags);
extern void CheckpointWriteDelay(int flags, double progress);
diff --git a/src/include/postmaster/startup.h b/src/include/postmaster/startup.h
index 9f59c1ffa3..6a8108dfe0 100644
--- a/src/include/postmaster/startup.h
+++ b/src/include/postmaster/startup.h
@@ -13,10 +13,13 @@
#define _STARTUP_H
extern void HandleStartupProcInterrupts(void);
-extern void StartupProcessMain(void) pg_attribute_noreturn();
extern void PreRestoreCommand(void);
extern void PostRestoreCommand(void);
extern bool IsPromoteTriggered(void);
extern void ResetPromoteTriggered(void);
+/* Startup subprocess functions */
+extern void StartupProcessMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern bool StartupCleanup(int child_errno);
+
#endif /* _STARTUP_H */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
new file mode 100644
index 0000000000..56e8edf2d8
--- /dev/null
+++ b/src/include/postmaster/subprocess.h
@@ -0,0 +1,44 @@
+/*-------------------------------------------------------------------------
+ *
+ * subprocess.h
+ *
+ * Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ *
+ * src/include/postmaster/subprocess.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef SUBPROCESS_H
+#define SUBPROCESS_H
+
+#include "postmaster.h"
+
+typedef enum
+{
+ NoProcessType = -1,
+ CheckerType = 0,
+ BootstrapType,
+ StartupType,
+ BgWriterType,
+ CheckpointerType,
+ WalWriterType,
+ WalReceiverType, /* end of Auxiliary Process Forks */
+
+ NUMSUBPROCESSTYPES /* Must be last! */
+} SubprocessType;
+
+typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
+
+/* Current subprocess initializer */
+extern void InitMySubprocess(SubprocessType type);
+
+typedef struct PgSubprocess
+{
+ const char *desc;
+ SubprocessEntryPoint entrypoint;
+} PgSubprocess;
+
+extern SubprocessType MySubprocessType;
+extern PGDLLIMPORT PgSubprocess *MySubprocess;
+
+#endif /* SUBPROCESS_H */
diff --git a/src/include/postmaster/walwriter.h b/src/include/postmaster/walwriter.h
index 011dc26a6e..596c437c54 100644
--- a/src/include/postmaster/walwriter.h
+++ b/src/include/postmaster/walwriter.h
@@ -16,6 +16,6 @@
extern int WalWriterDelay;
extern int WalWriterFlushAfter;
-extern void WalWriterMain(void) pg_attribute_noreturn();
+extern void WalWriterMain(int argc, char *argv[]) pg_attribute_noreturn();
#endif /* _WALWRITER_H */
diff --git a/src/include/replication/walreceiver.h b/src/include/replication/walreceiver.h
index e08afc6548..2025f19ab3 100644
--- a/src/include/replication/walreceiver.h
+++ b/src/include/replication/walreceiver.h
@@ -311,7 +311,7 @@ walrcv_clear_result(WalRcvExecResult *walres)
}
/* prototypes for functions in walreceiver.c */
-extern void WalReceiverMain(void) pg_attribute_noreturn();
+extern void WalReceiverMain(int argc, char *argv[]);
extern void ProcessWalRcvInterrupts(void);
/* prototypes for functions in walreceiverfuncs.c */
--
2.21.1
0002-Use-centralized-StartSubprocess-for-aux-procs.patch (text/x-patch, 14.9K) download
From b78170a58339a91f9482d11b0e3a9e67f26d99ef Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 10 Mar 2020 14:58:07 +0000
Subject: [PATCH 02/11] Use centralized StartSubprocess for aux procs
Modifies StartChildProcess and moves the auxiliary processes under its
control. This is the next step toward making use of the Subprocess
infrastructure.
This patch introduces the fork_failure function, which may be
implemented by each subprocess to control failure when attempting to
fork the subprocess. This is used by the startup subprocess to exit
postmaster on fork failure. Future subprocesses may use it to do cleanup
(and optionally exit postmaster) after a fork failure.
Additionally, the patch adds a new backend variable for establishing a
backend variable to be shared across fork/exec.
---
src/backend/postmaster/postmaster.c | 156 ++++++++++++++--------------
src/backend/postmaster/startup.c | 12 +++
src/backend/postmaster/subprocess.c | 35 +++++--
src/include/postmaster/startup.h | 2 +-
src/include/postmaster/subprocess.h | 4 +
5 files changed, 121 insertions(+), 88 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 55187eb910..a902140284 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -114,6 +114,7 @@
#include "postmaster/fork_process.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "postmaster/syslogger.h"
#include "replication/logicallauncher.h"
#include "replication/walsender.h"
@@ -191,7 +192,8 @@ static Backend *ShmemBackendArray;
BackgroundWorker *MyBgworkerEntry = NULL;
-
+/* Struct containing postmaster subprocess control info */
+SubprocessType MySubprocessType;
/* The socket number we are listening for connections on */
int PostPortNumber;
@@ -421,7 +423,7 @@ static int CountChildren(int target);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
-static pid_t StartChildProcess(AuxProcType type);
+static pid_t StartSubprocess(SubprocessType type);
static void StartAutovacuumWorker(void);
static void MaybeStartWalReceiver(void);
static void InitPostmasterDeathWatchHandle(void);
@@ -507,6 +509,9 @@ typedef struct
TimestampTz PgStartTime;
TimestampTz PgReloadTime;
pg_time_t first_syslogger_file_time;
+
+ SubprocessType MySubprocessType;
+
bool redirection_done;
bool IsBinaryUpgrade;
int max_safe_fds;
@@ -538,11 +543,6 @@ static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
-#define StartupDataBase() StartChildProcess(StartupProcess)
-#define StartBackgroundWriter() StartChildProcess(BgWriterProcess)
-#define StartCheckpointer() StartChildProcess(CheckpointerProcess)
-#define StartWalWriter() StartChildProcess(WalWriterProcess)
-#define StartWalReceiver() StartChildProcess(WalReceiverProcess)
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -1389,7 +1389,7 @@ PostmasterMain(int argc, char *argv[])
/*
* We're ready to rock and roll...
*/
- StartupPID = StartupDataBase();
+ StartupPID = StartSubprocess(StartupType);
Assert(StartupPID != 0);
StartupStatus = STARTUP_RUNNING;
pmState = PM_STARTUP;
@@ -1750,9 +1750,9 @@ ServerLoop(void)
pmState == PM_HOT_STANDBY)
{
if (CheckpointerPID == 0)
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
if (BgWriterPID == 0)
- BgWriterPID = StartBackgroundWriter();
+ BgWriterPID = StartSubprocess(BgWriterType);
}
/*
@@ -1761,7 +1761,7 @@ ServerLoop(void)
* be writing any new WAL).
*/
if (WalWriterPID == 0 && pmState == PM_RUN)
- WalWriterPID = StartWalWriter();
+ WalWriterPID = StartSubprocess(WalWriterType);
/*
* If we have lost the autovacuum launcher, try to start a new one. We
@@ -3037,11 +3037,11 @@ reaper(SIGNAL_ARGS)
* if this fails, we'll just try again later.
*/
if (CheckpointerPID == 0)
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
if (BgWriterPID == 0)
- BgWriterPID = StartBackgroundWriter();
+ BgWriterPID = StartSubprocess(BgWriterType);
if (WalWriterPID == 0)
- WalWriterPID = StartWalWriter();
+ WalWriterPID = StartSubprocess(WalWriterType);
/*
* Likewise, start other special children as needed. In a restart
@@ -3855,7 +3855,7 @@ PostmasterStateMachine(void)
Assert(Shutdown > NoShutdown);
/* Start the checkpointer if not running */
if (CheckpointerPID == 0)
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
/* And tell it to shut down */
if (CheckpointerPID != 0)
{
@@ -3996,7 +3996,7 @@ PostmasterStateMachine(void)
reset_shared();
- StartupPID = StartupDataBase();
+ StartupPID = StartSubprocess(StartupType);
Assert(StartupPID != 0);
StartupStatus = STARTUP_RUNNING;
pmState = PM_STARTUP;
@@ -4916,6 +4916,10 @@ SubPostmasterMain(int argc, char *argv[])
errmsg("out of memory")));
#endif
+ /* We should only be here once per fork */
+ Assert(!MySubprocess);
+ InitMySubprocess(MySubprocessType);
+
/*
* If appropriate, physically re-attach to shared memory segment. We want
* to do this before going any further to ensure that we can attach at the
@@ -5036,19 +5040,6 @@ SubPostmasterMain(int argc, char *argv[])
/* And run the backend */
BackendRun(&port); /* does not return */
}
- if (strcmp(argv[1], "--forkboot") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitAuxiliaryProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
- }
if (strcmp(argv[1], "--forkavlauncher") == 0)
{
/* Restore basic shared memory pointers */
@@ -5116,6 +5107,20 @@ SubPostmasterMain(int argc, char *argv[])
SysLoggerMain(argc, argv); /* does not return */
}
+ if (MySubprocess->needs_aux_proc)
+ {
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
+
+ /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ InitAuxiliaryProcess();
+
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores();
+
+ AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
+ }
+
abort(); /* shouldn't get here */
}
#endif /* EXEC_BACKEND */
@@ -5197,9 +5202,9 @@ sigusr1_handler(SIGNAL_ARGS)
* we'll just try again later.
*/
Assert(CheckpointerPID == 0);
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
Assert(BgWriterPID == 0);
- BgWriterPID = StartBackgroundWriter();
+ BgWriterPID = StartSubprocess(BgWriterType);
/*
* Start the archiver if we're responsible for (re-)archiving received
@@ -5424,42 +5429,49 @@ CountChildren(int target)
return cnt;
}
-
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
- * "type" determines what kind of child will be started. All child types
- * initially go to AuxiliaryProcessMain, which will handle common setup.
+ * "type" determines what kind of child will be started. This code relies
+ * heavily on MySubprocess, which is an entry in the process_types struct in
+ * subprocess.c. All auxiliary processes go through AuxiliaryProcessMain after
+ * forking. Everything else goes through the designated entrypoint.
*
- * Return value of StartChildProcess is subprocess' PID, or 0 if failed
- * to start subprocess.
+ * Return value of StartChildProcess is subprocess' PID on success. On failure,
+ * the subprocess cleanup function decides whether or not to panic.
*/
static pid_t
-StartChildProcess(AuxProcType type)
+StartSubprocess(SubprocessType type)
{
pid_t pid;
- char *av[10];
- int ac = 0;
+ char *argv[10];
+ int argc = 0;
char typebuf[32];
+#ifdef EXEC_BACKEND
+ char forkname[32];
+#endif
/*
- * Set up command-line arguments for subprocess
+ * Get new subprocess data every time we start a new subprocess.
*/
- av[ac++] = "postgres";
+ InitMySubprocess(type);
+
+ argv[argc++] = "postgres";
#ifdef EXEC_BACKEND
- av[ac++] = "--forkboot";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
+ snprintf(forkname, sizeof(forkname), "--fork%s", MySubprocess->name);
+ argv[argc++] = forkname;
+ argv[argc++] = NULL;
#endif
snprintf(typebuf, sizeof(typebuf), "-x%d", type);
- av[ac++] = typebuf;
+ argv[argc++] = typebuf;
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ argv[argc] = NULL;
+ Assert(argc < lengthof(argv));
#ifdef EXEC_BACKEND
- pid = postmaster_forkexec(ac, av);
+ pid = postmaster_forkexec(argc, argv);
#else /* !EXEC_BACKEND */
pid = fork_process();
@@ -5475,7 +5487,7 @@ StartChildProcess(AuxProcType type)
MemoryContextDelete(PostmasterContext);
PostmasterContext = NULL;
- AuxiliaryProcessMain(ac, av);
+ AuxiliaryProcessMain(argc, argv);
ExitPostmaster(0);
}
#endif /* EXEC_BACKEND */
@@ -5486,40 +5498,20 @@ StartChildProcess(AuxProcType type)
int save_errno = errno;
errno = save_errno;
- switch (type)
- {
- case StartupProcess:
- ereport(LOG,
- (errmsg("could not fork startup process: %m")));
- break;
- case BgWriterProcess:
- ereport(LOG,
- (errmsg("could not fork background writer process: %m")));
- break;
- case CheckpointerProcess:
- ereport(LOG,
- (errmsg("could not fork checkpointer process: %m")));
- break;
- case WalWriterProcess:
- ereport(LOG,
- (errmsg("could not fork WAL writer process: %m")));
- break;
- case WalReceiverProcess:
- ereport(LOG,
- (errmsg("could not fork WAL receiver process: %m")));
- break;
- default:
- ereport(LOG,
- (errmsg("could not fork process: %m")));
- break;
- }
+ ereport(LOG,
+ (errmsg("could not fork %s process: %m", MySubprocess->desc)));
/*
- * fork failure is fatal during startup, but there's no need to choke
- * immediately if starting other child types fails.
+ * Handle fork-failure per subprocess type. Each subprocess handles
+ * fork-failure by returning a boolean which tells postmaster whether or
+ * not to panic.
*/
- if (type == StartupProcess)
- ExitPostmaster(1);
+ if (MySubprocess->fork_failure)
+ {
+ if (MySubprocess->fork_failure(save_errno))
+ ExitPostmaster(1);
+ }
+
return 0;
}
@@ -5637,7 +5629,7 @@ MaybeStartWalReceiver(void)
pmState == PM_HOT_STANDBY || pmState == PM_WAIT_READONLY) &&
Shutdown == NoShutdown)
{
- WalReceiverPID = StartWalReceiver();
+ WalReceiverPID = StartSubprocess(WalReceiverType);
if (WalReceiverPID != 0)
WalReceiverRequested = false;
/* else leave the flag set, so we'll try again later */
@@ -6197,6 +6189,8 @@ save_backend_variables(BackendParameters *param, Port *port,
param->PgReloadTime = PgReloadTime;
param->first_syslogger_file_time = first_syslogger_file_time;
+ param->MySubprocessType = MySubprocessType;
+
param->redirection_done = redirection_done;
param->IsBinaryUpgrade = IsBinaryUpgrade;
param->max_safe_fds = max_safe_fds;
@@ -6432,6 +6426,8 @@ restore_backend_variables(BackendParameters *param, Port *port)
PgReloadTime = param->PgReloadTime;
first_syslogger_file_time = param->first_syslogger_file_time;
+ MySubprocessType = param->MySubprocessType;
+
redirection_done = param->redirection_done;
IsBinaryUpgrade = param->IsBinaryUpgrade;
max_safe_fds = param->max_safe_fds;
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index e29649a6a1..782cd608ff 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -175,6 +175,18 @@ StartupProcessMain(int argc, char *argv[])
proc_exit(0);
}
+/*
+ * StartupForkFailure
+ *
+ * Tell the postmaster to exit/panic on failure to fork.
+ */
+bool
+StartupForkFailure(int fork_errno)
+{
+ /* Panic! */
+ return true;
+}
+
void
PreRestoreCommand(void)
{
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 3e7a45bf10..7e96678dfb 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -24,32 +24,53 @@ PgSubprocess *MySubprocess;
static PgSubprocess process_types[] = {
{
+ .name = "boot",
.desc = "checker",
- .entrypoint = CheckerModeMain
+ .needs_aux_proc = true,
+ .entrypoint = CheckerModeMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "bootstrap",
- .entrypoint = BootstrapModeMain
+ .needs_aux_proc = true,
+ .entrypoint = BootstrapModeMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "startup",
- .entrypoint = StartupProcessMain
+ .needs_aux_proc = true,
+ .entrypoint = StartupProcessMain,
+ .fork_failure = StartupForkFailure
},
{
+ .name = "boot",
.desc = "background writer",
- .entrypoint = BackgroundWriterMain
+ .needs_aux_proc = true,
+ .entrypoint = BackgroundWriterMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "checkpointer",
- .entrypoint = CheckpointerMain
+ .needs_aux_proc = true,
+ .entrypoint = CheckpointerMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "wal writer",
- .entrypoint = WalWriterMain
+ .needs_aux_proc = true,
+ .entrypoint = WalWriterMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "wal receiver",
- .entrypoint = WalReceiverMain
+ .needs_aux_proc = true,
+ .entrypoint = WalReceiverMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/postmaster/startup.h b/src/include/postmaster/startup.h
index 6a8108dfe0..c182c76960 100644
--- a/src/include/postmaster/startup.h
+++ b/src/include/postmaster/startup.h
@@ -20,6 +20,6 @@ extern void ResetPromoteTriggered(void);
/* Startup subprocess functions */
extern void StartupProcessMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern bool StartupCleanup(int child_errno);
+extern bool StartupForkFailure(int fork_errno);
#endif /* _STARTUP_H */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 56e8edf2d8..33f6a7ab04 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -28,14 +28,18 @@ typedef enum
} SubprocessType;
typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
+typedef bool (*SubprocessForkFailure) (int fork_errno);
/* Current subprocess initializer */
extern void InitMySubprocess(SubprocessType type);
typedef struct PgSubprocess
{
+ const char *name;
const char *desc;
+ bool needs_aux_proc;
SubprocessEntryPoint entrypoint;
+ SubprocessForkFailure fork_failure;
} PgSubprocess;
extern SubprocessType MySubprocessType;
--
2.21.1
0003-Add-AutoVacLauncherType-to-subprocess-struct.patch (text/x-patch, 7.6K) download
From 055d2d7f006bee98fd1671f89ca5d426710c8c31 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 10 Mar 2020 22:29:44 +0000
Subject: [PATCH 03/11] Add AutoVacLauncherType to subprocess struct
---
src/backend/postmaster/autovacuum.c | 68 ++---------------------------
src/backend/postmaster/postmaster.c | 36 ++++++++-------
src/backend/postmaster/subprocess.c | 8 ++++
src/include/postmaster/autovacuum.h | 3 +-
src/include/postmaster/subprocess.h | 1 +
5 files changed, 35 insertions(+), 81 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index e3a43d3296..ea44f68cd6 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -95,6 +95,7 @@
#include "storage/procsignal.h"
#include "storage/sinvaladt.h"
#include "storage/smgr.h"
+#include "postmaster/subprocess.h"
#include "tcop/tcopprot.h"
#include "utils/fmgroids.h"
#include "utils/fmgrprotos.h"
@@ -303,11 +304,9 @@ static WorkerInfo MyWorkerInfo = NULL;
int AutovacuumLauncherPid = 0;
#ifdef EXEC_BACKEND
-static pid_t avlauncher_forkexec(void);
static pid_t avworker_forkexec(void);
#endif
NON_EXEC_STATIC void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-NON_EXEC_STATIC void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
static Oid do_start_worker(void);
static void HandleAutoVacLauncherInterrupts(void);
@@ -353,26 +352,6 @@ static void autovac_refresh_stats(void);
********************************************************************/
#ifdef EXEC_BACKEND
-/*
- * forkexec routine for the autovacuum launcher process.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avlauncher_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavlauncher";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
/*
* We need this set from the outside, before InitProcess is called
@@ -384,57 +363,18 @@ AutovacuumLauncherIAm(void)
}
#endif
-/*
- * Main entry point for autovacuum launcher process, to be called from the
- * postmaster.
- */
-int
-StartAutoVacLauncher(void)
-{
- pid_t AutoVacPID;
-
-#ifdef EXEC_BACKEND
- switch ((AutoVacPID = avlauncher_forkexec()))
-#else
- switch ((AutoVacPID = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum launcher process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacLauncherMain(0, NULL);
- break;
-#endif
- default:
- return (int) AutoVacPID;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
/*
* Main loop for the autovacuum launcher process.
*/
-NON_EXEC_STATIC void
-AutoVacLauncherMain(int argc, char *argv[])
+void
+AutoVacLauncherMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
am_autovacuum_launcher = true;
/* Identify myself via ps */
- init_ps_display(pgstat_get_backend_desc(B_AUTOVAC_LAUNCHER), "", "", "");
+ init_ps_display(MySubprocess->desc, "", "", "");
ereport(DEBUG1,
(errmsg("autovacuum launcher started")));
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index a902140284..b4352b1dcc 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1773,7 +1773,7 @@ ServerLoop(void)
(AutoVacuumingActive() || start_autovac_launcher) &&
pmState == PM_RUN)
{
- AutoVacPID = StartAutoVacLauncher();
+ AutoVacPID = StartSubprocess(AutoVacuumLauncherType);
if (AutoVacPID != 0)
start_autovac_launcher = false; /* signal processed */
}
@@ -3048,7 +3048,7 @@ reaper(SIGNAL_ARGS)
* situation, some of them may be alive already.
*/
if (!IsBinaryUpgrade && AutoVacuumingActive() && AutoVacPID == 0)
- AutoVacPID = StartAutoVacLauncher();
+ AutoVacPID = StartSubprocess(AutoVacuumLauncherType);
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = pgarch_start();
if (PgStatPID == 0)
@@ -5040,19 +5040,6 @@ SubPostmasterMain(int argc, char *argv[])
/* And run the backend */
BackendRun(&port); /* does not return */
}
- if (strcmp(argv[1], "--forkavlauncher") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
- }
if (strcmp(argv[1], "--forkavworker") == 0)
{
/* Restore basic shared memory pointers */
@@ -5120,6 +5107,19 @@ SubPostmasterMain(int argc, char *argv[])
AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
}
+ else
+ {
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
+
+ /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ InitProcess();
+
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores();
+
+ MySubprocess->entrypoint(argc - 2, argv + 2);
+ }
abort(); /* shouldn't get here */
}
@@ -5487,7 +5487,11 @@ StartSubprocess(SubprocessType type)
MemoryContextDelete(PostmasterContext);
PostmasterContext = NULL;
- AuxiliaryProcessMain(argc, argv);
+ if (MySubprocess->needs_aux_proc)
+ AuxiliaryProcessMain(argc, argv);
+ else
+ MySubprocess->entrypoint(argc, argv);
+
ExitPostmaster(0);
}
#endif /* EXEC_BACKEND */
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 7e96678dfb..9aa60cc93e 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -12,6 +12,7 @@
*/
#include "postgres.h"
#include "bootstrap/bootstrap.h"
+#include "postmaster/autovacuum.h"
#include "postmaster/bgwriter.h"
#include "postmaster/postmaster.h"
#include "postmaster/startup.h"
@@ -71,6 +72,13 @@ static PgSubprocess process_types[] = {
.needs_aux_proc = true,
.entrypoint = WalReceiverMain,
.fork_failure = NULL
+ },
+ {
+ .name = "avlauncher",
+ .desc = "autovacuum launcher",
+ .needs_aux_proc = false,
+ .entrypoint = AutoVacLauncherMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index d40ed55531..08050cd81c 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -58,6 +58,8 @@ extern void autovac_init(void);
extern int StartAutoVacLauncher(void);
extern int StartAutoVacWorker(void);
+extern void AutoVacLauncherMain(int argc, char *argv[]);
+
/* called from postmaster when a worker could not be forked */
extern void AutoVacWorkerFailed(void);
@@ -65,7 +67,6 @@ extern void AutoVacWorkerFailed(void);
extern void AutoVacuumUpdateDelay(void);
#ifdef EXEC_BACKEND
-extern void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void AutovacuumWorkerIAm(void);
extern void AutovacuumLauncherIAm(void);
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 33f6a7ab04..9e3c743333 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -23,6 +23,7 @@ typedef enum
CheckpointerType,
WalWriterType,
WalReceiverType, /* end of Auxiliary Process Forks */
+ AutoVacuumLauncherType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.21.1
0004-Add-AutoVacuumWorkerType-to-subprocess-struct.patch (text/x-patch, 6.5K) download
From 6a06171a967af74e0a59d47acd2824ff1fad6160 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 10 Mar 2020 23:33:39 +0000
Subject: [PATCH 04/11] Add AutoVacuumWorkerType to subprocess struct
---
src/backend/postmaster/autovacuum.c | 71 +----------------------------
src/backend/postmaster/postmaster.c | 17 +------
src/backend/postmaster/subprocess.c | 7 +++
src/include/postmaster/autovacuum.h | 4 +-
src/include/postmaster/subprocess.h | 1 +
5 files changed, 14 insertions(+), 86 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index ea44f68cd6..a0409e30cb 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -83,7 +83,6 @@
#include "nodes/makefuncs.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "storage/bufmgr.h"
@@ -303,11 +302,6 @@ static WorkerInfo MyWorkerInfo = NULL;
/* PID of launcher, valid only in worker while shutting down */
int AutovacuumLauncherPid = 0;
-#ifdef EXEC_BACKEND
-static pid_t avworker_forkexec(void);
-#endif
-NON_EXEC_STATIC void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-
static Oid do_start_worker(void);
static void HandleAutoVacLauncherInterrupts(void);
static void AutoVacLauncherShutdown(void) pg_attribute_noreturn();
@@ -1364,27 +1358,6 @@ avl_sigusr2_handler(SIGNAL_ARGS)
********************************************************************/
#ifdef EXEC_BACKEND
-/*
- * forkexec routines for the autovacuum worker.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avworker_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavworker";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
/*
* We need this set from the outside, before InitProcess is called
*/
@@ -1395,50 +1368,10 @@ AutovacuumWorkerIAm(void)
}
#endif
-/*
- * Main entry point for autovacuum worker process.
- *
- * This code is heavily based on pgarch.c, q.v.
- */
-int
-StartAutoVacWorker(void)
-{
- pid_t worker_pid;
-
-#ifdef EXEC_BACKEND
- switch ((worker_pid = avworker_forkexec()))
-#else
- switch ((worker_pid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum worker process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacWorkerMain(0, NULL);
- break;
-#endif
- default:
- return (int) worker_pid;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
/*
* AutoVacWorkerMain
*/
-NON_EXEC_STATIC void
+void
AutoVacWorkerMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
@@ -1447,7 +1380,7 @@ AutoVacWorkerMain(int argc, char *argv[])
am_autovacuum_worker = true;
/* Identify myself via ps */
- init_ps_display(pgstat_get_backend_desc(B_AUTOVAC_WORKER), "", "", "");
+ init_ps_display(MySubprocess->desc, "", "", "");
SetProcessingMode(InitProcessing);
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index b4352b1dcc..d6e4cde92d 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -5040,19 +5040,6 @@ SubPostmasterMain(int argc, char *argv[])
/* And run the backend */
BackendRun(&port); /* does not return */
}
- if (strcmp(argv[1], "--forkavworker") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
- }
if (strncmp(argv[1], "--forkbgworker=", 15) == 0)
{
int shmem_slot;
@@ -5572,7 +5559,7 @@ StartAutovacuumWorker(void)
bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
bn->bgworker_notify = false;
- bn->pid = StartAutoVacWorker();
+ bn->pid = StartSubprocess(AutoVacuumWorkerType);
if (bn->pid > 0)
{
bn->bkend_type = BACKEND_TYPE_AUTOVAC;
@@ -5586,7 +5573,7 @@ StartAutovacuumWorker(void)
/*
* fork failed, fall through to report -- actual error message was
- * logged by StartAutoVacWorker
+ * logged by StartSubprocess
*/
(void) ReleasePostmasterChildSlot(bn->child_slot);
free(bn);
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 9aa60cc93e..0fbd067310 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -79,6 +79,13 @@ static PgSubprocess process_types[] = {
.needs_aux_proc = false,
.entrypoint = AutoVacLauncherMain,
.fork_failure = NULL
+ },
+ {
+ .name = "avworker",
+ .desc = "autovacuum worker",
+ .needs_aux_proc = false,
+ .entrypoint = AutoVacWorkerMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 08050cd81c..51546fd4c9 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -58,7 +58,8 @@ extern void autovac_init(void);
extern int StartAutoVacLauncher(void);
extern int StartAutoVacWorker(void);
-extern void AutoVacLauncherMain(int argc, char *argv[]);
+extern void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
/* called from postmaster when a worker could not be forked */
extern void AutoVacWorkerFailed(void);
@@ -67,7 +68,6 @@ extern void AutoVacWorkerFailed(void);
extern void AutoVacuumUpdateDelay(void);
#ifdef EXEC_BACKEND
-extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void AutovacuumWorkerIAm(void);
extern void AutovacuumLauncherIAm(void);
#endif
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 9e3c743333..e11c24cc62 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -24,6 +24,7 @@ typedef enum
WalWriterType,
WalReceiverType, /* end of Auxiliary Process Forks */
AutoVacuumLauncherType,
+ AutoVacuumWorkerType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.21.1
0005-Add-PgstatCollectorType-to-subprocess-struct.patch (text/x-patch, 11.6K) download
From 92ae2435e449bb1b017d5b927f029870f6e1ef01 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 11 Mar 2020 00:24:42 +0000
Subject: [PATCH 05/11] Add PgstatCollectorType to subprocess struct
Introduce two new fields to handle subprocess control:
1) needs_shmem controls whether or not the subprocess needs to attach
to shmem. Up until now, all subprocesses needed to attach. This field
paves the way for easier handling of subprocesses which do not.
2) fork_prep allows StartSubprocess to do some preparation for
fork/exec. This is necessary to split out any functionality that
previously happened in Start functions prior to fork/exec.
---
src/backend/postmaster/pgstat.c | 86 +++--------------------------
src/backend/postmaster/postmaster.c | 39 +++++++------
src/backend/postmaster/subprocess.c | 28 ++++++++++
src/include/pgstat.h | 4 +-
src/include/postmaster/subprocess.h | 8 ++-
5 files changed, 65 insertions(+), 100 deletions(-)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 462b4d7e06..648b5eadb5 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -48,9 +48,9 @@
#include "pg_trace.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "replication/walsender.h"
#include "storage/backendid.h"
#include "storage/dsm.h"
@@ -275,11 +275,6 @@ static instr_time total_func_time;
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgstat_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgstat_beshutdown_hook(int code, Datum arg);
static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
@@ -685,46 +680,15 @@ pgstat_reset_all(void)
pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
}
-#ifdef EXEC_BACKEND
-
/*
- * pgstat_forkexec() -
+ * PgstatCollectorPrep
*
- * Format up the arglist for, then fork and exec, statistics collector process
- */
-static pid_t
-pgstat_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkcol";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
-/*
- * pgstat_start() -
- *
- * Called from postmaster at startup or after an existing collector
- * died. Attempt to fire up a fresh statistics collector.
- *
- * Returns PID of child process, or 0 if fail.
- *
- * Note: if fail, we will be called again from the postmaster main loop.
+ * Called from StartSubprocess to prepare a new collector fork
*/
int
-pgstat_start(void)
+PgstatCollectorPrep(int argc, char *argv[])
{
time_t curtime;
- pid_t pgStatPid;
/*
* Check that the socket is there, else pgstat_init failed and we can do
@@ -740,46 +704,10 @@ pgstat_start(void)
* from the postmaster main loop, we will get another chance later.
*/
curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgstat_start_time) <
+ if ((unsigned int) (curtime - last_pgstat_start_time) >=
(unsigned int) PGSTAT_RESTART_INTERVAL)
- return 0;
- last_pgstat_start_time = curtime;
+ last_pgstat_start_time = curtime;
- /*
- * Okay, fork off the collector.
- */
-#ifdef EXEC_BACKEND
- switch ((pgStatPid = pgstat_forkexec()))
-#else
- switch ((pgStatPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork statistics collector: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgstatCollectorMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgStatPid;
- }
-
- /* shouldn't get here */
return 0;
}
@@ -4423,7 +4351,7 @@ pgstat_send_bgwriter(void)
* The argc/argv parameters are valid only in EXEC_BACKEND case.
* ----------
*/
-NON_EXEC_STATIC void
+void
PgstatCollectorMain(int argc, char *argv[])
{
int len;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index d6e4cde92d..76e2185d87 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1781,7 +1781,7 @@ ServerLoop(void)
/* If we have lost the stats collector, try to start a new one */
if (PgStatPID == 0 &&
(pmState == PM_RUN || pmState == PM_HOT_STANDBY))
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
@@ -3052,7 +3052,7 @@ reaper(SIGNAL_ARGS)
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = pgarch_start();
if (PgStatPID == 0)
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
/* workers may be scheduled to start now */
maybe_start_bgworkers();
@@ -3214,7 +3214,7 @@ reaper(SIGNAL_ARGS)
LogChildExit(LOG, _("statistics collector process"),
pid, exitstatus);
if (pmState == PM_RUN || pmState == PM_HOT_STANDBY)
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
continue;
}
@@ -5068,12 +5068,6 @@ SubPostmasterMain(int argc, char *argv[])
PgArchiverMain(argc, argv); /* does not return */
}
- if (strcmp(argv[1], "--forkcol") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgstatCollectorMain(argc, argv); /* does not return */
- }
if (strcmp(argv[1], "--forklog") == 0)
{
/* Do not want to attach to shared memory */
@@ -5096,14 +5090,17 @@ SubPostmasterMain(int argc, char *argv[])
}
else
{
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
+ if (MySubprocess->needs_shmem)
+ {
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
+ /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ InitProcess();
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores();
+ }
MySubprocess->entrypoint(argc - 2, argv + 2);
}
@@ -5223,7 +5220,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Likewise, start other special children as needed.
*/
Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5474,6 +5471,16 @@ StartSubprocess(SubprocessType type)
MemoryContextDelete(PostmasterContext);
PostmasterContext = NULL;
+ /*
+ * Drop connection to postmaster's shared memory if the subprocess
+ * doesn't need it.
+ */
+ if (!MySubprocess->needs_shmem)
+ {
+ dsm_detach_all();
+ PGSharedMemoryDetach();
+ }
+
if (MySubprocess->needs_aux_proc)
AuxiliaryProcessMain(argc, argv);
else
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 0fbd067310..ee44ef0789 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -12,6 +12,7 @@
*/
#include "postgres.h"
#include "bootstrap/bootstrap.h"
+#include "pgstat.h"
#include "postmaster/autovacuum.h"
#include "postmaster/bgwriter.h"
#include "postmaster/postmaster.h"
@@ -28,6 +29,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "checker",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = CheckerModeMain,
.fork_failure = NULL
},
@@ -35,6 +38,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "bootstrap",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = BootstrapModeMain,
.fork_failure = NULL
},
@@ -42,6 +47,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "startup",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = StartupProcessMain,
.fork_failure = StartupForkFailure
},
@@ -49,6 +56,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "background writer",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = BackgroundWriterMain,
.fork_failure = NULL
},
@@ -56,6 +65,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "checkpointer",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = CheckpointerMain,
.fork_failure = NULL
},
@@ -63,6 +74,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "wal writer",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = WalWriterMain,
.fork_failure = NULL
},
@@ -70,6 +83,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "wal receiver",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = WalReceiverMain,
.fork_failure = NULL
},
@@ -77,6 +92,8 @@ static PgSubprocess process_types[] = {
.name = "avlauncher",
.desc = "autovacuum launcher",
.needs_aux_proc = false,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = AutoVacLauncherMain,
.fork_failure = NULL
},
@@ -84,8 +101,19 @@ static PgSubprocess process_types[] = {
.name = "avworker",
.desc = "autovacuum worker",
.needs_aux_proc = false,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = AutoVacWorkerMain,
.fork_failure = NULL
+ },
+ {
+ .name = "col",
+ .desc = "statistics collector",
+ .needs_aux_proc = false,
+ .needs_shmem = false,
+ .fork_prep = PgstatCollectorPrep,
+ .entrypoint = PgstatCollectorMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 7bc36c6583..c7ddfaecc2 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1242,10 +1242,8 @@ extern int pgstat_start(void);
extern void pgstat_reset_all(void);
extern void allow_immediate_pgstat_restart(void);
-#ifdef EXEC_BACKEND
+extern int PgstatCollectorPrep(int argc, char *argv[]);
extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
-
/* ----------
* Functions called from backends
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index e11c24cc62..a7ab036b7b 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -25,12 +25,14 @@ typedef enum
WalReceiverType, /* end of Auxiliary Process Forks */
AutoVacuumLauncherType,
AutoVacuumWorkerType,
+ PgstatCollectorType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
-typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
-typedef bool (*SubprocessForkFailure) (int fork_errno);
+typedef int (*SubprocessPrep) (int argc, char *argv[]);
+typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
+typedef bool (*SubprocessForkFailure) (int fork_errno);
/* Current subprocess initializer */
extern void InitMySubprocess(SubprocessType type);
@@ -40,6 +42,8 @@ typedef struct PgSubprocess
const char *name;
const char *desc;
bool needs_aux_proc;
+ bool needs_shmem;
+ SubprocessPrep fork_prep;
SubprocessEntryPoint entrypoint;
SubprocessForkFailure fork_failure;
} PgSubprocess;
--
2.21.1
0006-Add-PgArchiverType-to-subprocess-struct.patch (text/x-patch, 7.6K) download
From 7c469faed58b81438955feae640824dee61969e7 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 11 Mar 2020 01:07:05 +0000
Subject: [PATCH 06/11] Add PgArchiverType to subprocess struct
---
src/backend/postmaster/pgarch.c | 89 ++++-------------------------
src/backend/postmaster/postmaster.c | 14 ++---
src/backend/postmaster/subprocess.c | 10 ++++
src/include/postmaster/pgarch.h | 9 +--
src/include/postmaster/subprocess.h | 1 +
5 files changed, 27 insertions(+), 96 deletions(-)
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 3ca30badb2..02f5c53b1f 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -38,10 +38,10 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "storage/dsm.h"
#include "storage/fd.h"
#include "storage/ipc.h"
@@ -91,11 +91,6 @@ static volatile sig_atomic_t ready_to_stop = false;
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgarch_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgarch_exit(SIGNAL_ARGS);
static void pgarch_waken(SIGNAL_ARGS);
static void pgarch_waken_stop(SIGNAL_ARGS);
@@ -122,10 +117,9 @@ static void pgarch_archiveDone(char *xlog);
* Note: if fail, we will be called again from the postmaster main loop.
*/
int
-pgarch_start(void)
+PgArchiverPrep(int argc, char *argv[])
{
time_t curtime;
- pid_t pgArchPid;
/*
* Do nothing if no archiver needed
@@ -140,86 +134,20 @@ pgarch_start(void)
* the postmaster main loop, we will get another chance later.
*/
curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgarch_start_time) <
+ if ((unsigned int) (curtime - last_pgarch_start_time) >=
(unsigned int) PGARCH_RESTART_INTERVAL)
- return 0;
- last_pgarch_start_time = curtime;
-
-#ifdef EXEC_BACKEND
- switch ((pgArchPid = pgarch_forkexec()))
-#else
- switch ((pgArchPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork archiver: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
+ last_pgarch_start_time = curtime;
- PgArchiverMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgArchPid;
- }
-
- /* shouldn't get here */
return 0;
}
-/* ------------------------------------------------------------
- * Local functions called by archiver follow
- * ------------------------------------------------------------
- */
-
-
-#ifdef EXEC_BACKEND
-
-/*
- * pgarch_forkexec() -
- *
- * Format up the arglist for, then fork and exec, archive process
- */
-static pid_t
-pgarch_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
-
- av[ac++] = "--forkarch";
-
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
/*
* PgArchiverMain
*
* The argc/argv parameters are valid only in EXEC_BACKEND case. However,
* since we don't use 'em, it hardly matters...
*/
-NON_EXEC_STATIC void
+void
PgArchiverMain(int argc, char *argv[])
{
/*
@@ -241,13 +169,18 @@ PgArchiverMain(int argc, char *argv[])
/*
* Identify myself via ps
*/
- init_ps_display("archiver", "", "", "");
+ init_ps_display(MySubprocess->desc, "", "", "");
pgarch_MainLoop();
exit(0);
}
+/* ------------------------------------------------------------
+ * Local functions called by archiver follow
+ * ------------------------------------------------------------
+ */
+
/* SIGQUIT signal handler for archiver process */
static void
pgarch_exit(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 76e2185d87..83287231f9 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1785,7 +1785,7 @@ ServerLoop(void)
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
/* If we need to signal the autovacuum launcher, do so now */
if (avlauncher_needs_signal)
@@ -3050,7 +3050,7 @@ reaper(SIGNAL_ARGS)
if (!IsBinaryUpgrade && AutoVacuumingActive() && AutoVacPID == 0)
AutoVacPID = StartSubprocess(AutoVacuumLauncherType);
if (PgArchStartupAllowed() && PgArchPID == 0)
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
if (PgStatPID == 0)
PgStatPID = StartSubprocess(PgstatCollectorType);
@@ -3198,7 +3198,7 @@ reaper(SIGNAL_ARGS)
LogChildExit(LOG, _("archiver process"),
pid, exitstatus);
if (PgArchStartupAllowed())
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
continue;
}
@@ -5062,12 +5062,6 @@ SubPostmasterMain(int argc, char *argv[])
StartBackgroundWorker();
}
- if (strcmp(argv[1], "--forkarch") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgArchiverMain(argc, argv); /* does not return */
- }
if (strcmp(argv[1], "--forklog") == 0)
{
/* Do not want to attach to shared memory */
@@ -5196,7 +5190,7 @@ sigusr1_handler(SIGNAL_ARGS)
*/
Assert(PgArchPID == 0);
if (XLogArchivingAlways())
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
/*
* If we aren't planning to enter hot standby mode later, treat
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index ee44ef0789..f9d71a627c 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -15,6 +15,7 @@
#include "pgstat.h"
#include "postmaster/autovacuum.h"
#include "postmaster/bgwriter.h"
+#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
#include "postmaster/startup.h"
#include "postmaster/subprocess.h"
@@ -114,6 +115,15 @@ static PgSubprocess process_types[] = {
.fork_prep = PgstatCollectorPrep,
.entrypoint = PgstatCollectorMain,
.fork_failure = NULL
+ },
+ {
+ .name = "arch",
+ .desc = "archiver",
+ .needs_aux_proc = false,
+ .needs_shmem = false,
+ .fork_prep = PgArchiverPrep,
+ .entrypoint = PgArchiverMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/postmaster/pgarch.h b/src/include/postmaster/pgarch.h
index b3200874ca..ac527212b1 100644
--- a/src/include/postmaster/pgarch.h
+++ b/src/include/postmaster/pgarch.h
@@ -26,14 +26,7 @@
#define MAX_XFN_CHARS 40
#define VALID_XFN_CHARS "0123456789ABCDEF.history.backup.partial"
-/* ----------
- * Functions called from postmaster
- * ----------
- */
-extern int pgarch_start(void);
-
-#ifdef EXEC_BACKEND
+extern int PgArchiverPrep(int argc, char *argv[]);
extern void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
#endif /* _PGARCH_H */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index a7ab036b7b..3f00f904dc 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -26,6 +26,7 @@ typedef enum
AutoVacuumLauncherType,
AutoVacuumWorkerType,
PgstatCollectorType,
+ PgArchiverType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.21.1
0007-Add-SysLoggerType-to-subprocess-struct.patch (text/x-patch, 17.0K) download
From 1eaf90d10d7614ef6272c94a41167a1b101b887d Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 11 Mar 2020 02:58:08 +0000
Subject: [PATCH 07/11] Add SysLoggerType to subprocess struct
This commit introduces two new fields to the subprocess struct:
1) keep_postmaster_memcontext allows subprocesses to specify whether or
not the PostmasterMemoryContext should be kept around after a fork.
2) postmaster_main allows postmaster to gain control to do some
necessary processing after a successful fork.
---
src/backend/postmaster/postmaster.c | 39 +++--
src/backend/postmaster/subprocess.c | 56 +++++--
src/backend/postmaster/syslogger.c | 229 +++++++++++-----------------
src/include/postmaster/subprocess.h | 18 ++-
src/include/postmaster/syslogger.h | 5 +-
5 files changed, 178 insertions(+), 169 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 83287231f9..af4fed4733 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1083,7 +1083,7 @@ PostmasterMain(int argc, char *argv[])
/*
* If enabled, start up syslogger collection subprocess
*/
- SysLoggerPID = SysLogger_Start();
+ SysLoggerPID = StartSubprocess(SysLoggerType);
/*
* Reset whereToSendOutput from DestDebug (its starting state) to
@@ -1739,7 +1739,7 @@ ServerLoop(void)
/* If we have lost the log collector, try to start a new one */
if (SysLoggerPID == 0 && Logging_collector)
- SysLoggerPID = SysLogger_Start();
+ SysLoggerPID = StartSubprocess(SysLoggerType);
/*
* If no background writer process is running, and we are not in a
@@ -3223,7 +3223,7 @@ reaper(SIGNAL_ARGS)
{
SysLoggerPID = 0;
/* for safety's sake, launch new logger *first* */
- SysLoggerPID = SysLogger_Start();
+ SysLoggerPID = StartSubprocess(SysLoggerType);
if (!EXIT_STATUS_0(exitstatus))
LogChildExit(LOG, _("system logger process"),
pid, exitstatus);
@@ -5062,12 +5062,6 @@ SubPostmasterMain(int argc, char *argv[])
StartBackgroundWorker();
}
- if (strcmp(argv[1], "--forklog") == 0)
- {
- /* Do not want to attach to shared memory */
-
- SysLoggerMain(argc, argv); /* does not return */
- }
if (MySubprocess->needs_aux_proc)
{
@@ -5445,6 +5439,15 @@ StartSubprocess(SubprocessType type)
snprintf(typebuf, sizeof(typebuf), "-x%d", type);
argv[argc++] = typebuf;
+ /* Prep subprocess for forking */
+ if (MySubprocess->fork_prep)
+ {
+ if (MySubprocess->fork_prep(argc, argv))
+ {
+ return 0;
+ }
+ }
+
argv[argc] = NULL;
Assert(argc < lengthof(argv));
@@ -5460,10 +5463,16 @@ StartSubprocess(SubprocessType type)
/* Close the postmaster's sockets */
ClosePostmasterPorts(false);
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
+ /*
+ * Release postmaster's working memory context if the subprocess doesn't
+ * need it
+ */
+ if (!MySubprocess->keep_postmaster_memcontext)
+ {
+ MemoryContextSwitchTo(TopMemoryContext);
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+ }
/*
* Drop connection to postmaster's shared memory if the subprocess
@@ -5507,6 +5516,10 @@ StartSubprocess(SubprocessType type)
return 0;
}
+ /* Some subprocesses need to do more work in postmaster. */
+ if (MySubprocess->postmaster_main)
+ MySubprocess->postmaster_main(argc, argv);
+
/*
* in parent, successful fork
*/
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index f9d71a627c..f9d07de4ae 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -19,6 +19,7 @@
#include "postmaster/postmaster.h"
#include "postmaster/startup.h"
#include "postmaster/subprocess.h"
+#include "postmaster/syslogger.h"
#include "postmaster/walwriter.h"
#include "replication/walreceiver.h"
@@ -31,99 +32,132 @@ static PgSubprocess process_types[] = {
.desc = "checker",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = CheckerModeMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "boot",
.desc = "bootstrap",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = BootstrapModeMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "boot",
.desc = "startup",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = StartupProcessMain,
- .fork_failure = StartupForkFailure
+ .fork_failure = StartupForkFailure,
+ .postmaster_main = NULL
},
{
.name = "boot",
.desc = "background writer",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = BackgroundWriterMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "boot",
.desc = "checkpointer",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = CheckpointerMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "boot",
.desc = "wal writer",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = WalWriterMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "boot",
.desc = "wal receiver",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = WalReceiverMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "avlauncher",
.desc = "autovacuum launcher",
.needs_aux_proc = false,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = AutoVacLauncherMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "avworker",
.desc = "autovacuum worker",
.needs_aux_proc = false,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = AutoVacWorkerMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "col",
.desc = "statistics collector",
.needs_aux_proc = false,
.needs_shmem = false,
+ .keep_postmaster_memcontext = false,
.fork_prep = PgstatCollectorPrep,
.entrypoint = PgstatCollectorMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "arch",
.desc = "archiver",
.needs_aux_proc = false,
.needs_shmem = false,
+ .keep_postmaster_memcontext = false,
.fork_prep = PgArchiverPrep,
.entrypoint = PgArchiverMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
+ },
+ {
+ .name = "log",
+ .desc = "syslogger",
+ .needs_aux_proc = false,
+ .needs_shmem = false,
+ .keep_postmaster_memcontext = true,
+ .fork_prep = SysLoggerPrep,
+ .entrypoint = SysLoggerMain,
+ .fork_failure = NULL,
+ .postmaster_main = SysLoggerPostmasterMain
}
};
diff --git a/src/backend/postmaster/syslogger.c b/src/backend/postmaster/syslogger.c
index cf7b535e4e..19ec55dcba 100644
--- a/src/backend/postmaster/syslogger.c
+++ b/src/backend/postmaster/syslogger.c
@@ -38,8 +38,8 @@
#include "nodes/pg_list.h"
#include "pgstat.h"
#include "pgtime.h"
-#include "postmaster/fork_process.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "postmaster/syslogger.h"
#include "storage/dsm.h"
#include "storage/fd.h"
@@ -134,10 +134,8 @@ static volatile sig_atomic_t rotation_requested = false;
/* Local subroutines */
#ifdef EXEC_BACKEND
-static pid_t syslogger_forkexec(void);
static void syslogger_parseArgs(int argc, char *argv[]);
#endif
-NON_EXEC_STATIC void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
static void process_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static void flush_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static FILE *logfile_open(const char *filename, const char *mode,
@@ -158,7 +156,7 @@ static void update_metainfo_datafile(void);
* Main entry point for syslogger process
* argc/argv parameters are valid only in EXEC_BACKEND case.
*/
-NON_EXEC_STATIC void
+void
SysLoggerMain(int argc, char *argv[])
{
#ifndef WIN32
@@ -179,7 +177,7 @@ SysLoggerMain(int argc, char *argv[])
am_syslogger = true;
- init_ps_display("logger", "", "", "");
+ init_ps_display(MySubprocess->desc, "", "", "");
/*
* If we restarted, our stderr is already redirected into our own input
@@ -543,13 +541,17 @@ SysLoggerMain(int argc, char *argv[])
* Postmaster subroutine to start a syslogger subprocess.
*/
int
-SysLogger_Start(void)
+SysLoggerPrep(int argc, char *argv[])
{
- pid_t sysloggerPid;
char *filename;
+#ifdef EXEC_BACKEND
+ char filenobuf[32];
+ char csvfilenobuf[32];
+#endif
+ /* Tell postmaster to stop forking */
if (!Logging_collector)
- return 0;
+ return -1;
/*
* If first time through, create the pipe which will receive stderr
@@ -632,129 +634,6 @@ SysLogger_Start(void)
}
#ifdef EXEC_BACKEND
- switch ((sysloggerPid = syslogger_forkexec()))
-#else
- switch ((sysloggerPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork system logger: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(true);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- /* do the work */
- SysLoggerMain(0, NULL);
- break;
-#endif
-
- default:
- /* success, in postmaster */
-
- /* now we redirect stderr, if not done already */
- if (!redirection_done)
- {
-#ifdef WIN32
- int fd;
-#endif
-
- /*
- * Leave a breadcrumb trail when redirecting, in case the user
- * forgets that redirection is active and looks only at the
- * original stderr target file.
- */
- ereport(LOG,
- (errmsg("redirecting log output to logging collector process"),
- errhint("Future log output will appear in directory \"%s\".",
- Log_directory)));
-
-#ifndef WIN32
- fflush(stdout);
- if (dup2(syslogPipe[1], fileno(stdout)) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stdout: %m")));
- fflush(stderr);
- if (dup2(syslogPipe[1], fileno(stderr)) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- /* Now we are done with the write end of the pipe. */
- close(syslogPipe[1]);
- syslogPipe[1] = -1;
-#else
-
- /*
- * open the pipe in binary mode and make sure stderr is binary
- * after it's been dup'ed into, to avoid disturbing the pipe
- * chunking protocol.
- */
- fflush(stderr);
- fd = _open_osfhandle((intptr_t) syslogPipe[1],
- _O_APPEND | _O_BINARY);
- if (dup2(fd, _fileno(stderr)) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- close(fd);
- _setmode(_fileno(stderr), _O_BINARY);
-
- /*
- * Now we are done with the write end of the pipe.
- * CloseHandle() must not be called because the preceding
- * close() closes the underlying handle.
- */
- syslogPipe[1] = 0;
-#endif
- redirection_done = true;
- }
-
- /* postmaster will never write the file(s); close 'em */
- fclose(syslogFile);
- syslogFile = NULL;
- if (csvlogFile != NULL)
- {
- fclose(csvlogFile);
- csvlogFile = NULL;
- }
- return (int) sysloggerPid;
- }
-
- /* we should never reach here */
- return 0;
-}
-
-
-#ifdef EXEC_BACKEND
-
-/*
- * syslogger_forkexec() -
- *
- * Format up the arglist for, then fork and exec, a syslogger process
- */
-static pid_t
-syslogger_forkexec(void)
-{
- char *av[10];
- int ac = 0;
- char filenobuf[32];
- char csvfilenobuf[32];
-
- av[ac++] = "postgres";
- av[ac++] = "--forklog";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
/* static variables (those not passed by write_backend_variables) */
#ifndef WIN32
if (syslogFile != NULL)
@@ -769,7 +648,7 @@ syslogger_forkexec(void)
else
strcpy(filenobuf, "0");
#endif /* WIN32 */
- av[ac++] = filenobuf;
+ argv[argc++] = filenobuf;
#ifndef WIN32
if (csvlogFile != NULL)
@@ -784,14 +663,92 @@ syslogger_forkexec(void)
else
strcpy(csvfilenobuf, "0");
#endif /* WIN32 */
- av[ac++] = csvfilenobuf;
+ argv[argc++] = csvfilenobuf;
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+#endif /* EXEC_BACKEND */
- return postmaster_forkexec(ac, av);
+ return 0;
}
+/*
+ * SysLoggerPostmasterMain
+ *
+ * Control function for subprocess parent (postmaster) after a successful fork.
+ */
+void
+SysLoggerPostmasterMain(int argc, char *argv[])
+{
+ /* success, in postmaster */
+
+ /* now we redirect stderr, if not done already */
+ if (!redirection_done)
+ {
+#ifdef WIN32
+ int fd;
+#endif
+
+ /*
+ * Leave a breadcrumb trail when redirecting, in case the user
+ * forgets that redirection is active and looks only at the
+ * original stderr target file.
+ */
+ ereport(LOG,
+ (errmsg("redirecting log output to logging collector process"),
+ errhint("Future log output will appear in directory \"%s\".",
+ Log_directory)));
+
+#ifndef WIN32
+ fflush(stdout);
+ if (dup2(syslogPipe[1], fileno(stdout)) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stdout: %m")));
+ fflush(stderr);
+ if (dup2(syslogPipe[1], fileno(stderr)) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ /* Now we are done with the write end of the pipe. */
+ close(syslogPipe[1]);
+ syslogPipe[1] = -1;
+#else
+
+ /*
+ * open the pipe in binary mode and make sure stderr is binary
+ * after it's been dup'ed into, to avoid disturbing the pipe
+ * chunking protocol.
+ */
+ fflush(stderr);
+ fd = _open_osfhandle((intptr_t) syslogPipe[1],
+ _O_APPEND | _O_BINARY);
+ if (dup2(fd, _fileno(stderr)) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ close(fd);
+ _setmode(_fileno(stderr), _O_BINARY);
+
+ /*
+ * Now we are done with the write end of the pipe.
+ * CloseHandle() must not be called because the preceding
+ * close() closes the underlying handle.
+ */
+ syslogPipe[1] = 0;
+#endif
+ redirection_done = true;
+ }
+
+ /* postmaster will never write the file(s); close 'em */
+ fclose(syslogFile);
+ syslogFile = NULL;
+ if (csvlogFile != NULL)
+ {
+ fclose(csvlogFile);
+ csvlogFile = NULL;
+ }
+}
+
+#ifdef EXEC_BACKEND
/*
* syslogger_parseArgs() -
*
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 3f00f904dc..d0d98a5d69 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -27,6 +27,7 @@ typedef enum
AutoVacuumWorkerType,
PgstatCollectorType,
PgArchiverType,
+ SysLoggerType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
@@ -34,19 +35,22 @@ typedef enum
typedef int (*SubprocessPrep) (int argc, char *argv[]);
typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
typedef bool (*SubprocessForkFailure) (int fork_errno);
+typedef void (*SubprocessPostmasterMain) (int argc, char *argv[]);
/* Current subprocess initializer */
extern void InitMySubprocess(SubprocessType type);
typedef struct PgSubprocess
{
- const char *name;
- const char *desc;
- bool needs_aux_proc;
- bool needs_shmem;
- SubprocessPrep fork_prep;
- SubprocessEntryPoint entrypoint;
- SubprocessForkFailure fork_failure;
+ const char *name;
+ const char *desc;
+ bool needs_aux_proc;
+ bool needs_shmem;
+ bool keep_postmaster_memcontext;
+ SubprocessPrep fork_prep;
+ SubprocessEntryPoint entrypoint;
+ SubprocessForkFailure fork_failure;
+ SubprocessPostmasterMain postmaster_main;
} PgSubprocess;
extern SubprocessType MySubprocessType;
diff --git a/src/include/postmaster/syslogger.h b/src/include/postmaster/syslogger.h
index 9b7a386dbd..0c544bbe9a 100644
--- a/src/include/postmaster/syslogger.h
+++ b/src/include/postmaster/syslogger.h
@@ -83,9 +83,10 @@ extern int SysLogger_Start(void);
extern void write_syslogger_file(const char *buffer, int count, int dest);
-#ifdef EXEC_BACKEND
+/* Subprocess startup functions */
+extern int SysLoggerPrep(int argc, char *argv[]);
extern void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+extern void SysLoggerPostmasterMain(int argc, char *argv[]);
extern bool CheckLogrotateSignal(void);
extern void RemoveLogrotateSignalFiles(void);
--
2.21.1
0008-Add-BgWorkerType-to-subprocess-struct.patch (text/x-patch, 25.6K) download
From d884cff99cacd7cd8cdab9f0aab8abbfc125a9e1 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 11 Mar 2020 03:32:41 +0000
Subject: [PATCH 08/11] Add BgWorkerType to subprocess struct
Export some of the backend-specific code and move the some bgworker
startup code out to src/backend/postmaster/bgworker.c.
---
src/backend/postmaster/bgworker.c | 165 ++++++++++-
src/backend/postmaster/postmaster.c | 294 +++-----------------
src/backend/postmaster/subprocess.c | 12 +
src/backend/utils/init/globals.c | 1 +
src/include/miscadmin.h | 1 +
src/include/postmaster/bgworker_internals.h | 8 +-
src/include/postmaster/postmaster.h | 53 ++++
src/include/postmaster/subprocess.h | 1 +
8 files changed, 268 insertions(+), 267 deletions(-)
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index 75fc0d5d33..c031181bfe 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -33,8 +33,10 @@
#include "storage/shmem.h"
#include "tcop/tcopprot.h"
#include "utils/ascii.h"
+#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/timeout.h"
+#include "utils/timestamp.h"
/*
* The postmaster's list of registered background workers, in private memory.
@@ -133,7 +135,55 @@ static const struct
/* Private functions. */
static bgworker_main_type LookupBackgroundWorkerFunction(const char *libraryname, const char *funcname);
+static bool assign_backendlist_entry(RegisteredBgWorker *rw);
+/*
+ * Allocate the Backend struct for a connected background worker, but don't
+ * add it to the list of backends just yet.
+ *
+ * On failure, return false without changing any worker state.
+ *
+ * Some info from the Backend is copied into the passed rw.
+ */
+static bool
+assign_backendlist_entry(RegisteredBgWorker *rw)
+{
+ Backend *bn;
+
+ /*
+ * Compute the cancel key that will be assigned to this session. We
+ * probably don't need cancel keys for background workers, but we'd better
+ * have something random in the field to prevent unfriendly people from
+ * sending cancels to them.
+ */
+ if (!RandomCancelKey(&MyCancelKey))
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random cancel key")));
+ return false;
+ }
+
+ bn = malloc(sizeof(Backend));
+ if (bn == NULL)
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+ return false;
+ }
+
+ bn->cancel_key = MyCancelKey;
+ bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
+ bn->bkend_type = BACKEND_TYPE_BGWORKER;
+ bn->dead_end = false;
+ bn->bgworker_notify = false;
+
+ rw->rw_backend = bn;
+ rw->rw_child_slot = bn->child_slot;
+
+ return true;
+}
/*
* Calculate shared memory needed.
@@ -670,6 +720,55 @@ bgworker_sigusr1_handler(SIGNAL_ARGS)
errno = save_errno;
}
+/*
+ * BackgroundWorkerPrep
+ *
+ * Postmaster subroutine to prepare a bgworker subprocess.
+ *
+ * Returns 0 on success or -1 on failure.
+ *
+ * Note: if fail, we will be called again from the postmaster main loop.
+ */
+int
+BackgroundWorkerPrep(int argc, char *argv[])
+{
+ RegisteredBgWorker *rw = MyRegisteredBgWorker;
+#ifdef EXEC_BACKEND
+ char forkav[MAXPGPATH];
+#endif
+
+ Assert(MyRegisteredBgWorker);
+ Assert(rw->rw_pid == 0);
+
+ /*
+ * Allocate and assign the Backend element. Note we must do this before
+ * forking, so that we can handle out of memory properly.
+ *
+ * Treat failure as though the worker had crashed. That way, the
+ * postmaster will wait a bit before attempting to start it again; if it
+ * tried again right away, most likely it'd find itself repeating the
+ * out-of-memory or fork failure condition.
+ */
+ if (!assign_backendlist_entry(rw))
+ {
+ rw->rw_crashed_at = GetCurrentTimestamp();
+ return -1;
+ }
+
+ ereport(DEBUG1,
+ (errmsg("starting background worker process \"%s\"",
+ rw->rw_worker.bgw_name)));
+
+#ifdef EXEC_BACKEND
+ /* Rewrite bgworker fork parameter with shmem_slot */
+ snprintf(forkav, MAXPGPATH, "%s=%d", argv[1], rw->rw_shmem_slot);
+ argv[1] = forkav;
+#endif
+
+ return 0;
+}
+
+
/*
* Start a new background worker
*
@@ -677,12 +776,28 @@ bgworker_sigusr1_handler(SIGNAL_ARGS)
* postmaster.
*/
void
-StartBackgroundWorker(void)
+BackgroundWorkerMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
- BackgroundWorker *worker = MyBgworkerEntry;
+ BackgroundWorker *worker;
bgworker_main_type entrypt;
+#ifndef EXEC_BACKEND
+ /*
+ * Before blowing away PostmasterContext, save this bgworker's
+ * data where it can find it.
+ */
+ MyBgworkerEntry = (BackgroundWorker *)
+ MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
+ memcpy(MyBgworkerEntry, &(MyRegisteredBgWorker->rw_worker), sizeof(BackgroundWorker));
+
+ /* Now we can release postmaster's working memory context */
+ MemoryContextSwitchTo(TopMemoryContext);
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+#endif
+
+ worker = MyBgworkerEntry;
if (worker == NULL)
elog(FATAL, "unable to find bgworker entry");
@@ -816,6 +931,52 @@ StartBackgroundWorker(void)
proc_exit(0);
}
+/*
+ * BackgroundWorkerForkFailure
+ *
+ * Handle bgworker fork failure gracefully.
+ */
+bool
+BackgroundWorkerForkFailure(int fork_errno)
+{
+ RegisteredBgWorker *rw = MyRegisteredBgWorker;
+
+ errno = fork_errno;
+
+ /* in postmaster, fork failed ... */
+ ereport(LOG,
+ (errmsg("could not fork worker process: %m")));
+
+ /* undo what assign_backendlist_entry did */
+ ReleasePostmasterChildSlot(rw->rw_child_slot);
+ rw->rw_child_slot = 0;
+ free(rw->rw_backend);
+ rw->rw_backend = NULL;
+ /* mark entry as crashed, so we'll try again later */
+ rw->rw_crashed_at = GetCurrentTimestamp();
+
+ /* Don't panic. */
+ return false;
+}
+
+/*
+ * BackgroundWorkerPostmasterMain
+ */
+void
+BackgroundWorkerPostmasterMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ RegisteredBgWorker *rw = MyRegisteredBgWorker;
+
+ rw->rw_pid = MyChildProcPid;
+ rw->rw_backend->pid = rw->rw_pid;
+ ReportBackgroundWorkerPID(rw);
+ /* add new worker to lists of backends */
+ dlist_push_head(&BackendList, &(rw->rw_backend->elem));
+#ifdef EXEC_BACKEND
+ ShmemBackendArrayAdd(rw->rw_backend);
+#endif
+}
+
/*
* Register a new static background worker.
*
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index af4fed4733..94b6737b7e 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -137,60 +137,14 @@
#include "storage/spin.h"
#endif
-
-/*
- * Possible types of a backend. Beyond being the possible bkend_type values in
- * struct bkend, these are OR-able request flag bits for SignalSomeChildren()
- * and CountChildren().
- */
-#define BACKEND_TYPE_NORMAL 0x0001 /* normal backend */
-#define BACKEND_TYPE_AUTOVAC 0x0002 /* autovacuum worker process */
-#define BACKEND_TYPE_WALSND 0x0004 /* walsender process */
-#define BACKEND_TYPE_BGWORKER 0x0008 /* bgworker process */
-#define BACKEND_TYPE_ALL 0x000F /* OR of all the above */
-
-#define BACKEND_TYPE_WORKER (BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER)
-
-/*
- * List of active backends (or child processes anyway; we don't actually
- * know whether a given child has become a backend or is still in the
- * authorization phase). This is used mainly to keep track of how many
- * children we have and send them appropriate signals when necessary.
- *
- * "Special" children such as the startup, bgwriter and autovacuum launcher
- * tasks are not in this list. Autovacuum worker and walsender are in it.
- * Also, "dead_end" children are in it: these are children launched just for
- * the purpose of sending a friendly rejection message to a would-be client.
- * We must track them because they are attached to shared memory, but we know
- * they will never become live backends. dead_end children are not assigned a
- * PMChildSlot.
- *
- * Background workers are in this list, too.
- */
-typedef struct bkend
-{
- pid_t pid; /* process id of backend */
- int32 cancel_key; /* cancel key for cancels for this backend */
- int child_slot; /* PMChildSlot for this backend, if any */
-
- /*
- * Flavor of backend or auxiliary process. Note that BACKEND_TYPE_WALSND
- * backends initially announce themselves as BACKEND_TYPE_NORMAL, so if
- * bkend_type is normal, you should check for a recent transition.
- */
- int bkend_type;
- bool dead_end; /* is it going to send an error and quit? */
- bool bgworker_notify; /* gets bgworker start/stop notifications */
- dlist_node elem; /* list link in BackendList */
-} Backend;
-
-static dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
+dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
#ifdef EXEC_BACKEND
static Backend *ShmemBackendArray;
#endif
BackgroundWorker *MyBgworkerEntry = NULL;
+RegisteredBgWorker *MyRegisteredBgWorker = NULL;
/* Struct containing postmaster subprocess control info */
SubprocessType MySubprocessType;
@@ -412,7 +366,6 @@ static void processCancelRequest(Port *port, void *pkt);
static int initMasks(fd_set *rmask);
static void report_fork_failure_to_client(Port *port, int errnum);
static CAC_state canAcceptConnections(int backend_type);
-static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
@@ -420,7 +373,6 @@ static void TerminateChildren(int signal);
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
-static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
static pid_t StartSubprocess(SubprocessType type);
@@ -539,10 +491,10 @@ static bool save_backend_variables(BackendParameters *param, Port *port,
HANDLE childProcess, pid_t childPid);
#endif
-static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
+#define BGWORKER_LEN 15
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -4877,6 +4829,7 @@ void
SubPostmasterMain(int argc, char *argv[])
{
Port port;
+ int shmem_slot;
/* In EXEC_BACKEND case we will not have inherited these settings */
IsPostmasterEnvironment = true;
@@ -5040,59 +4993,48 @@ SubPostmasterMain(int argc, char *argv[])
/* And run the backend */
BackendRun(&port); /* does not return */
}
- if (strncmp(argv[1], "--forkbgworker=", 15) == 0)
- {
- int shmem_slot;
-
- /* do this as early as possible; in particular, before InitProcess() */
- IsBackgroundWorker = true;
+ if (MySubprocess->needs_aux_proc)
+ {
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
/* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
+ InitAuxiliaryProcess();
/* Attach process to shared data structures */
CreateSharedMemoryAndSemaphores();
- /* Fetch MyBgworkerEntry from shared memory */
- shmem_slot = atoi(argv[1] + 15);
- MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
+ AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
+ }
- StartBackgroundWorker();
+ if (MySubprocessType == BgWorkerType)
+ {
+ /* do this as early as possible; in particular, before InitProcess() */
+ IsBackgroundWorker = true;
}
- if (MySubprocess->needs_aux_proc)
+ if (MySubprocess->needs_shmem)
{
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
/* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitAuxiliaryProcess();
+ InitProcess();
/* Attach process to shared data structures */
CreateSharedMemoryAndSemaphores();
-
- AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
}
- else
- {
- if (MySubprocess->needs_shmem)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
- }
-
- MySubprocess->entrypoint(argc - 2, argv + 2);
+ if (MySubprocessType == BgWorkerType)
+ {
+ /* Fetch MyBgworkerEntry from shared memory */
+ shmem_slot = atoi(argv[1] + 15);
+ MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
}
+ MySubprocess->entrypoint(argc - 2, argv + 2);
+
abort(); /* shouldn't get here */
}
#endif /* EXEC_BACKEND */
@@ -5355,7 +5297,7 @@ StartupPacketTimeoutHandler(void)
/*
* Generate a random cancel key.
*/
-static bool
+bool
RandomCancelKey(int32 *cancel_key)
{
return pg_strong_random(cancel_key, sizeof(int32));
@@ -5453,15 +5395,17 @@ StartSubprocess(SubprocessType type)
#ifdef EXEC_BACKEND
pid = postmaster_forkexec(argc, argv);
+ MyChildProcPid = pid;
#else /* !EXEC_BACKEND */
pid = fork_process();
+ MyChildProcPid = pid;
if (pid == 0) /* child */
{
InitPostmasterChild();
/* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
+ ClosePostmasterPorts(MySubprocessType == SysLoggerType);
/*
* Release postmaster's working memory context if the subprocess doesn't
@@ -5511,6 +5455,8 @@ StartSubprocess(SubprocessType type)
{
if (MySubprocess->fork_failure(save_errno))
ExitPostmaster(1);
+ else
+ return -1;
}
return 0;
@@ -5753,124 +5699,6 @@ BackgroundWorkerUnblockSignals(void)
PG_SETMASK(&UnBlockSig);
}
-#ifdef EXEC_BACKEND
-static pid_t
-bgworker_forkexec(int shmem_slot)
-{
- char *av[10];
- int ac = 0;
- char forkav[MAXPGPATH];
-
- snprintf(forkav, MAXPGPATH, "--forkbgworker=%d", shmem_slot);
-
- av[ac++] = "postgres";
- av[ac++] = forkav;
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif
-
-/*
- * Start a new bgworker.
- * Starting time conditions must have been checked already.
- *
- * Returns true on success, false on failure.
- * In either case, update the RegisteredBgWorker's state appropriately.
- *
- * This code is heavily based on autovacuum.c, q.v.
- */
-static bool
-do_start_bgworker(RegisteredBgWorker *rw)
-{
- pid_t worker_pid;
-
- Assert(rw->rw_pid == 0);
-
- /*
- * Allocate and assign the Backend element. Note we must do this before
- * forking, so that we can handle failures (out of memory or child-process
- * slots) cleanly.
- *
- * Treat failure as though the worker had crashed. That way, the
- * postmaster will wait a bit before attempting to start it again; if we
- * tried again right away, most likely we'd find ourselves hitting the
- * same resource-exhaustion condition.
- */
- if (!assign_backendlist_entry(rw))
- {
- rw->rw_crashed_at = GetCurrentTimestamp();
- return false;
- }
-
- ereport(DEBUG1,
- (errmsg("starting background worker process \"%s\"",
- rw->rw_worker.bgw_name)));
-
-#ifdef EXEC_BACKEND
- switch ((worker_pid = bgworker_forkexec(rw->rw_shmem_slot)))
-#else
- switch ((worker_pid = fork_process()))
-#endif
- {
- case -1:
- /* in postmaster, fork failed ... */
- ereport(LOG,
- (errmsg("could not fork worker process: %m")));
- /* undo what assign_backendlist_entry did */
- ReleasePostmasterChildSlot(rw->rw_child_slot);
- rw->rw_child_slot = 0;
- free(rw->rw_backend);
- rw->rw_backend = NULL;
- /* mark entry as crashed, so we'll try again later */
- rw->rw_crashed_at = GetCurrentTimestamp();
- break;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /*
- * Before blowing away PostmasterContext, save this bgworker's
- * data where it can find it.
- */
- MyBgworkerEntry = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(MyBgworkerEntry, &rw->rw_worker, sizeof(BackgroundWorker));
-
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
-
- StartBackgroundWorker();
-
- exit(1); /* should not get here */
- break;
-#endif
- default:
- /* in postmaster, fork successful ... */
- rw->rw_pid = worker_pid;
- rw->rw_backend->pid = rw->rw_pid;
- ReportBackgroundWorkerPID(rw);
- /* add new worker to lists of backends */
- dlist_push_head(&BackendList, &rw->rw_backend->elem);
-#ifdef EXEC_BACKEND
- ShmemBackendArrayAdd(rw->rw_backend);
-#endif
- return true;
- }
-
- return false;
-}
-
/*
* Does the current postmaster state require starting a worker with the
* specified start_time?
@@ -5911,67 +5739,6 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
return false;
}
-/*
- * Allocate the Backend struct for a connected background worker, but don't
- * add it to the list of backends just yet.
- *
- * On failure, return false without changing any worker state.
- *
- * Some info from the Backend is copied into the passed rw.
- */
-static bool
-assign_backendlist_entry(RegisteredBgWorker *rw)
-{
- Backend *bn;
-
- /*
- * Check that database state allows another connection. Currently the
- * only possible failure is CAC_TOOMANY, so we just log an error message
- * based on that rather than checking the error code precisely.
- */
- if (canAcceptConnections(BACKEND_TYPE_BGWORKER) != CAC_OK)
- {
- ereport(LOG,
- (errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("no slot available for new worker process")));
- return false;
- }
-
- /*
- * Compute the cancel key that will be assigned to this session. We
- * probably don't need cancel keys for background workers, but we'd better
- * have something random in the field to prevent unfriendly people from
- * sending cancels to them.
- */
- if (!RandomCancelKey(&MyCancelKey))
- {
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("could not generate random cancel key")));
- return false;
- }
-
- bn = malloc(sizeof(Backend));
- if (bn == NULL)
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- return false;
- }
-
- bn->cancel_key = MyCancelKey;
- bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
- bn->bkend_type = BACKEND_TYPE_BGWORKER;
- bn->dead_end = false;
- bn->bgworker_notify = false;
-
- rw->rw_backend = bn;
- rw->rw_child_slot = bn->child_slot;
-
- return true;
-}
-
/*
* If the time is right, start background worker(s).
*
@@ -6076,7 +5843,8 @@ maybe_start_bgworkers(void)
* crashed, but there's no need because the next run of this
* function will do that.
*/
- if (!do_start_bgworker(rw))
+ MyRegisteredBgWorker = rw;
+ if (StartSubprocess(BgWorkerType) <= 0)
{
StartWorkerNeeded = true;
return;
@@ -6487,7 +6255,7 @@ ShmemBackendArrayAllocation(void)
memset(ShmemBackendArray, 0, size);
}
-static void
+void
ShmemBackendArrayAdd(Backend *bn)
{
/* The array slot corresponding to my PMChildSlot should be free */
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index f9d07de4ae..d4df791aeb 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -14,6 +14,7 @@
#include "bootstrap/bootstrap.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
+#include "postmaster/bgworker_internals.h"
#include "postmaster/bgwriter.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
@@ -158,6 +159,17 @@ static PgSubprocess process_types[] = {
.entrypoint = SysLoggerMain,
.fork_failure = NULL,
.postmaster_main = SysLoggerPostmasterMain
+ },
+ {
+ .name = "bgworker",
+ .desc = "background worker",
+ .needs_aux_proc = false,
+ .needs_shmem = true,
+ .keep_postmaster_memcontext = true,
+ .fork_prep = BackgroundWorkerPrep,
+ .entrypoint = BackgroundWorkerMain,
+ .fork_failure = BackgroundWorkerForkFailure,
+ .postmaster_main = BackgroundWorkerPostmasterMain
}
};
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index eb19644419..b31ce82f75 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -38,6 +38,7 @@ volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
int MyProcPid;
+int MyChildProcPid;
pg_time_t MyStartTime;
TimestampTz MyStartTimestamp;
struct Port *MyProcPort;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index f985453ec3..823f5258d8 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -163,6 +163,7 @@ extern PGDLLIMPORT int max_worker_processes;
extern PGDLLIMPORT int max_parallel_workers;
extern PGDLLIMPORT int MyProcPid;
+extern PGDLLIMPORT int MyChildProcPid;
extern PGDLLIMPORT pg_time_t MyStartTime;
extern PGDLLIMPORT TimestampTz MyStartTimestamp;
extern PGDLLIMPORT struct Port *MyProcPort;
diff --git a/src/include/postmaster/bgworker_internals.h b/src/include/postmaster/bgworker_internals.h
index f7e24664d5..874d5a203f 100644
--- a/src/include/postmaster/bgworker_internals.h
+++ b/src/include/postmaster/bgworker_internals.h
@@ -43,6 +43,7 @@ typedef struct RegisteredBgWorker
} RegisteredBgWorker;
extern slist_head BackgroundWorkerList;
+extern RegisteredBgWorker *MyRegisteredBgWorker;
extern Size BackgroundWorkerShmemSize(void);
extern void BackgroundWorkerShmemInit(void);
@@ -53,8 +54,11 @@ extern void ReportBackgroundWorkerExit(slist_mutable_iter *cur);
extern void BackgroundWorkerStopNotifications(pid_t pid);
extern void ResetBackgroundWorkerCrashTimes(void);
-/* Function to start a background worker, called from postmaster.c */
-extern void StartBackgroundWorker(void) pg_attribute_noreturn();
+/* BgWorkerType subprocess */
+extern int BackgroundWorkerPrep(int argc, char *argv[]);
+extern void BackgroundWorkerMain(int argc, char *argv[]);
+extern bool BackgroundWorkerForkFailure(int fork_errno);
+extern void BackgroundWorkerPostmasterMain(int argc, char *argv[]);
#ifdef EXEC_BACKEND
extern BackgroundWorker *BackgroundWorkerEntry(int slotno);
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index babc87dfc9..b1411c1ed2 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -13,6 +13,41 @@
#ifndef _POSTMASTER_H
#define _POSTMASTER_H
+#include "lib/ilist.h"
+
+/*
+ * List of active backends (or child processes anyway; we don't actually
+ * know whether a given child has become a backend or is still in the
+ * authorization phase). This is used mainly to keep track of how many
+ * children we have and send them appropriate signals when necessary.
+ *
+ * "Special" children such as the startup, bgwriter and autovacuum launcher
+ * tasks are not in this list. Autovacuum worker and walsender are in it.
+ * Also, "dead_end" children are in it: these are children launched just for
+ * the purpose of sending a friendly rejection message to a would-be client.
+ * We must track them because they are attached to shared memory, but we know
+ * they will never become live backends. dead_end children are not assigned a
+ * PMChildSlot.
+ *
+ * Background workers are in this list, too.
+ */
+typedef struct bkend
+{
+ pid_t pid; /* process id of backend */
+ int32 cancel_key; /* cancel key for cancels for this backend */
+ int child_slot; /* PMChildSlot for this backend, if any */
+
+ /*
+ * Flavor of backend or auxiliary process. Note that BACKEND_TYPE_WALSND
+ * backends initially announce themselves as BACKEND_TYPE_NORMAL, so if
+ * bkend_type is normal, you should check for a recent transition.
+ */
+ int bkend_type;
+ bool dead_end; /* is it going to send an error and quit? */
+ bool bgworker_notify; /* gets bgworker start/stop notifications */
+ dlist_node elem; /* list link in BackendList */
+} Backend;
+
/* GUC options */
extern bool EnableSSL;
extern int ReservedBackends;
@@ -54,14 +89,19 @@ extern int MaxLivePostmasterChildren(void);
extern bool PostmasterMarkPIDForWorkerNotify(int);
+extern bool RandomCancelKey(int32 *cancel_key);
+
#ifdef EXEC_BACKEND
extern pid_t postmaster_forkexec(int argc, char *argv[]);
extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
extern Size ShmemBackendArraySize(void);
extern void ShmemBackendArrayAllocation(void);
+extern void ShmemBackendArrayAdd(Backend *bn);
#endif
+extern PGDLLIMPORT dlist_head BackendList;
+
/*
* Note: MAX_BACKENDS is limited to 2^18-1 because that's the width reserved
* for buffer references in buf_internals.h. This limitation could be lifted
@@ -74,4 +114,17 @@ extern void ShmemBackendArrayAllocation(void);
*/
#define MAX_BACKENDS 0x3FFFF
+/*
+ * Possible types of a backend. Beyond being the possible bkend_type values in
+ * struct bkend, these are OR-able request flag bits for SignalSomeChildren()
+ * and CountChildren().
+ */
+#define BACKEND_TYPE_NORMAL 0x0001 /* normal backend */
+#define BACKEND_TYPE_AUTOVAC 0x0002 /* autovacuum worker process */
+#define BACKEND_TYPE_WALSND 0x0004 /* walsender process */
+#define BACKEND_TYPE_BGWORKER 0x0008 /* bgworker process */
+#define BACKEND_TYPE_ALL 0x000F /* OR of all the above */
+
+#define BACKEND_TYPE_WORKER (BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER)
+
#endif /* _POSTMASTER_H */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index d0d98a5d69..706d60e4d6 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -28,6 +28,7 @@ typedef enum
PgstatCollectorType,
PgArchiverType,
SysLoggerType,
+ BgWorkerType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.21.1
0009-Add-Backends-to-subprocess-struct.patch (text/x-patch, 16.0K) download
From 728acdc853cc0ae3c2e5894ed0555bd14d0566f6 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Mon, 16 Mar 2020 23:05:33 +0000
Subject: [PATCH 09/11] Add Backends to subprocess struct
---
src/backend/postmaster/postmaster.c | 407 +++++++++++++++-------------
src/backend/postmaster/subprocess.c | 11 +
src/include/postmaster/postmaster.h | 6 +
src/include/postmaster/subprocess.h | 1 +
4 files changed, 235 insertions(+), 190 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 94b6737b7e..09f8bbb298 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -359,7 +359,6 @@ static void BackendInitialize(Port *port);
static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
-static int BackendStartup(Port *port);
static int ProcessStartupPacket(Port *port, bool secure_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
@@ -409,7 +408,6 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(Port *port);
static pid_t internal_forkexec(int argc, char *argv[], Port *port);
/* Type for a socket that can be inherited to a client process */
@@ -512,6 +510,8 @@ int postmaster_alive_fds[2] = {-1, -1};
HANDLE PostmasterHandle;
#endif
+static Port *ConnProcPort = NULL;
+
/*
* Postmaster main entry point
*/
@@ -1671,19 +1671,18 @@ ServerLoop(void)
break;
if (FD_ISSET(ListenSocket[i], &rmask))
{
- Port *port;
+ ConnProcPort = ConnCreate(ListenSocket[i]);
- port = ConnCreate(ListenSocket[i]);
- if (port)
+ if (ConnProcPort)
{
- BackendStartup(port);
+ StartSubprocess(ClientBackendType);
/*
* We no longer need the open socket or port structure
* in this process
*/
- StreamClose(port->sock);
- ConnFree(port);
+ StreamClose(ConnProcPort->sock);
+ ConnFree(ConnProcPort);
}
}
}
@@ -3210,6 +3209,174 @@ reaper(SIGNAL_ARGS)
errno = save_errno;
}
+/*
+ * BackendPrep
+ *
+ * Do all of the pre-fork setup for a new Backend.
+*/
+int
+BackendPrep(int argc, char *argv[])
+{
+ /*
+ * Create backend data structure. Better before the fork() so we can
+ * handle failure cleanly.
+ */
+ Backend *bn;
+
+ bn = (Backend *) malloc(sizeof(Backend));
+ if (!bn)
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+ }
+
+ /*
+ * Compute the cancel key that will be assigned to this backend. The
+ * backend will have its own copy in the forked-off process' value of
+ * MyCancelKey, so that it can transmit the key to the frontend.
+ */
+ if (!RandomCancelKey(&MyCancelKey))
+ {
+ free(bn);
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random cancel key")));
+ }
+
+ bn->cancel_key = MyCancelKey;
+
+ /* Pass down canAcceptConnections state */
+ ConnProcPort->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
+ bn->dead_end = (ConnProcPort->canAcceptConnections != CAC_OK &&
+ ConnProcPort->canAcceptConnections != CAC_WAITBACKUP);
+
+ /*
+ * Unless it's a dead_end child, assign it a child slot number
+ */
+ if (!bn->dead_end)
+ bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
+ else
+ bn->child_slot = 0;
+
+ /* Hasn't asked to be notified about any bgworkers yet */
+ bn->bgworker_notify = false;
+
+ /*
+ * Push the Backend to the stack as-is so it can be retreived by the parent
+ * on the other side of the fork/exec. They will modify this entry in place.
+ */
+ dlist_push_head(&BackendList, &bn->elem);
+
+ return 0;
+}
+
+/*
+ * BackendMain
+ *
+ * Child code when forking a Backend.
+ */
+void BackendMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ /*
+ * Perform additional initialization and collect startup packet.
+ *
+ * We want to do this before InitProcess() for a couple of reasons: 1.
+ * so that we aren't eating up a PGPROC slot while waiting on the
+ * client. 2. so that if InitProcess() fails due to being out of
+ * PGPROC slots, we have already initialized libpq and are able to
+ * report the error to the client.
+ */
+#ifndef EXEC_BACKEND
+ BackendInitialize(ConnProcPort);
+#endif
+
+ /* And run the backend */
+ BackendRun(ConnProcPort); /* does not return */
+}
+
+/*
+ * BackendPostmasterMain
+ *
+ * Parent code when forking a Backend.
+ */
+void BackendPostmasterMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ dlist_mutable_iter iter;
+
+ /*
+ * Find our backend node by popping the element with a matching child slot
+ * off the list. This was pushed to the stack (incomplete) by BackendPrep.
+ * We need to fix it up and push back.
+ */
+ dlist_foreach_modify(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ /* MyPMChildSlot is guaranteed to match the child MyPMChildSlot */
+ if (bp->child_slot == MyPMChildSlot)
+ {
+ /*
+ * Everything's been successful, it's safe to add this backend to our list
+ * of backends.
+ */
+ bp->pid = MyChildProcPid;
+ bp->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
+
+#ifdef EXEC_BACKEND
+ if (!bp->dead_end)
+ ShmemBackendArrayAdd(bp);
+#endif
+ break; /* There is only one entry and we've found it */
+ }
+ }
+
+ /* in parent, successful fork */
+ ereport(DEBUG2,
+ (errmsg_internal("forked new backend, pid=%d socket=%d",
+ (int) MyChildProcPid, (int) MyProcPort->sock)));
+}
+
+/*
+ * BackendForkFailure
+ *
+ * Backend cleanup in case a failure occurs forking a new Backend.
+ */
+bool
+BackendForkFailure(int child_errno)
+{
+ dlist_mutable_iter iter;
+
+ dlist_foreach_modify(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->child_slot == MyPMChildSlot)
+ {
+ if (!bp->dead_end)
+ {
+ if (!ReleasePostmasterChildSlot(bp->child_slot))
+ {
+ /* Could not release child slot! Panic! */
+ return true;
+ }
+#ifdef EXEC_BACKEND
+ ShmemBackendArrayRemove(bp);
+#endif
+ }
+ dlist_delete(iter.cur);
+ free(bp);
+ break;
+ }
+ }
+
+ report_fork_failure_to_client(MyProcPort, child_errno);
+
+ /* Don't panic */
+ return false;
+}
+
+
/*
* Scan the bgworkers list and see if the given PID (which has just stopped
* or crashed) is in it. Handle its shutdown if so, and return true. If not a
@@ -4070,122 +4237,6 @@ TerminateChildren(int signal)
signal_child(PgStatPID, signal);
}
-/*
- * BackendStartup -- start backend process
- *
- * returns: STATUS_ERROR if the fork failed, STATUS_OK otherwise.
- *
- * Note: if you change this code, also consider StartAutovacuumWorker.
- */
-static int
-BackendStartup(Port *port)
-{
- Backend *bn; /* for backend cleanup */
- pid_t pid;
-
- /*
- * Create backend data structure. Better before the fork() so we can
- * handle failure cleanly.
- */
- bn = (Backend *) malloc(sizeof(Backend));
- if (!bn)
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- return STATUS_ERROR;
- }
-
- /*
- * Compute the cancel key that will be assigned to this backend. The
- * backend will have its own copy in the forked-off process' value of
- * MyCancelKey, so that it can transmit the key to the frontend.
- */
- if (!RandomCancelKey(&MyCancelKey))
- {
- free(bn);
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("could not generate random cancel key")));
- return STATUS_ERROR;
- }
-
- bn->cancel_key = MyCancelKey;
-
- /* Pass down canAcceptConnections state */
- port->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
- bn->dead_end = (port->canAcceptConnections != CAC_OK &&
- port->canAcceptConnections != CAC_WAITBACKUP);
-
- /*
- * Unless it's a dead_end child, assign it a child slot number
- */
- if (!bn->dead_end)
- bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
- else
- bn->child_slot = 0;
-
- /* Hasn't asked to be notified about any bgworkers yet */
- bn->bgworker_notify = false;
-
-#ifdef EXEC_BACKEND
- pid = backend_forkexec(port);
-#else /* !EXEC_BACKEND */
- pid = fork_process();
- if (pid == 0) /* child */
- {
- free(bn);
-
- /* Detangle from postmaster */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Perform additional initialization and collect startup packet */
- BackendInitialize(port);
-
- /* And run the backend */
- BackendRun(port);
- }
-#endif /* EXEC_BACKEND */
-
- if (pid < 0)
- {
- /* in parent, fork failed */
- int save_errno = errno;
-
- if (!bn->dead_end)
- (void) ReleasePostmasterChildSlot(bn->child_slot);
- free(bn);
- errno = save_errno;
- ereport(LOG,
- (errmsg("could not fork new process for connection: %m")));
- report_fork_failure_to_client(port, save_errno);
- return STATUS_ERROR;
- }
-
- /* in parent, successful fork */
- ereport(DEBUG2,
- (errmsg_internal("forked new backend, pid=%d socket=%d",
- (int) pid, (int) port->sock)));
-
- /*
- * Everything's been successful, it's safe to add this backend to our list
- * of backends.
- */
- bn->pid = pid;
- bn->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
- dlist_push_head(&BackendList, &bn->elem);
-
-#ifdef EXEC_BACKEND
- if (!bn->dead_end)
- ShmemBackendArrayAdd(bn);
-#endif
-
- return STATUS_OK;
-}
-
/*
* Try to report backend fork() failure to client before we close the
* connection. Since we do not care to risk blocking the postmaster on
@@ -4482,33 +4533,12 @@ postmaster_forkexec(int argc, char *argv[])
Port port;
/* This entry point passes dummy values for the Port variables */
- memset(&port, 0, sizeof(port));
- return internal_forkexec(argc, argv, &port);
-}
-
-/*
- * backend_forkexec -- fork/exec off a backend process
- *
- * Some operating systems (WIN32) don't have fork() so we have to simulate
- * it by storing parameters that need to be passed to the child and
- * then create a new child process.
- *
- * returns the pid of the fork/exec'd process, or -1 on failure
- */
-static pid_t
-backend_forkexec(Port *port)
-{
- char *av[4];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkbackend";
- av[ac++] = NULL; /* filled in by internal_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ if (!ConnProcPort)
+ memset(&port, 0, sizeof(port));
+ else
+ port = *ConnProcPort;
- return internal_forkexec(ac, av, port);
+ return internal_forkexec(argc, argv, &port);
}
#ifndef WIN32
@@ -4848,6 +4878,7 @@ SubPostmasterMain(int argc, char *argv[])
/* Read in the variables file */
memset(&port, 0, sizeof(Port));
read_backend_variables(argv[2], &port);
+ ConnProcPort = &port;
/* Close the postmaster's sockets (as soon as we know them) */
ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
@@ -4889,11 +4920,7 @@ SubPostmasterMain(int argc, char *argv[])
* sometimes impossible to attach to shared memory at the desired address.
* Return the setting to its old value (usually '1' or '2') when finished.
*/
- if (strcmp(argv[1], "--forkbackend") == 0 ||
- strcmp(argv[1], "--forkavlauncher") == 0 ||
- strcmp(argv[1], "--forkavworker") == 0 ||
- strcmp(argv[1], "--forkboot") == 0 ||
- strncmp(argv[1], "--forkbgworker=", 15) == 0)
+ if (MySubprocess->needs_shmem)
PGSharedMemoryReAttach();
else
PGSharedMemoryNoReAttach();
@@ -4942,11 +4969,23 @@ SubPostmasterMain(int argc, char *argv[])
*/
process_shared_preload_libraries();
- /* Run backend or appropriate child */
- if (strcmp(argv[1], "--forkbackend") == 0)
+ if (MySubprocess->needs_aux_proc)
{
- Assert(argc == 3); /* shouldn't be any more args */
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
+ /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ InitAuxiliaryProcess();
+
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores();
+
+ AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
+ }
+
+ /* Run backend or appropriate child */
+ if (MySubprocessType == ClientBackendType)
+ {
/*
* Need to reinitialize the SSL library in the backend, since the
* context structures contain function pointers and cannot be passed
@@ -4980,32 +5019,6 @@ SubPostmasterMain(int argc, char *argv[])
* report the error to the client.
*/
BackendInitialize(&port);
-
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- /* And run the backend */
- BackendRun(&port); /* does not return */
- }
-
- if (MySubprocess->needs_aux_proc)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitAuxiliaryProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
}
if (MySubprocessType == BgWorkerType)
@@ -5026,15 +5039,29 @@ SubPostmasterMain(int argc, char *argv[])
CreateSharedMemoryAndSemaphores();
}
- if (MySubprocessType == BgWorkerType)
+ switch (MySubprocessType)
{
- /* Fetch MyBgworkerEntry from shared memory */
- shmem_slot = atoi(argv[1] + 15);
- MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
+ case AutoVacuumLauncherType:
+ case AutoVacuumWorkerType:
+ MySubprocess->entrypoint(argc - 2, argv + 2);
+ break;
+ case BgWorkerType:
+ /* Fetch MyBgworkerEntry from shared memory */
+ shmem_slot = atoi(argv[1] + 15);
+ MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
+ /* fallthrough */
+ case PgArchiverType:
+ case PgstatCollectorType:
+ case SysLoggerType:
+ case ClientBackendType:
+ MySubprocess->entrypoint(argc, argv);
+ break;
+ default:
+ ereport(LOG,
+ (errmsg("could not start unknown process type (%d) under postmaster",
+ MySubprocessType)));
}
- MySubprocess->entrypoint(argc - 2, argv + 2);
-
abort(); /* shouldn't get here */
}
#endif /* EXEC_BACKEND */
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index d4df791aeb..9d7ce86918 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -170,6 +170,17 @@ static PgSubprocess process_types[] = {
.entrypoint = BackgroundWorkerMain,
.fork_failure = BackgroundWorkerForkFailure,
.postmaster_main = BackgroundWorkerPostmasterMain
+ },
+ {
+ .name = "backend",
+ .desc = "backend",
+ .needs_aux_proc = false,
+ .needs_shmem = true,
+ .keep_postmaster_memcontext = true,
+ .fork_prep = BackendPrep,
+ .entrypoint = BackendMain,
+ .fork_failure = BackendForkFailure,
+ .postmaster_main = BackendPostmasterMain
}
};
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index b1411c1ed2..6bc0af7c2b 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -91,6 +91,12 @@ extern bool PostmasterMarkPIDForWorkerNotify(int);
extern bool RandomCancelKey(int32 *cancel_key);
+extern int BackendPrep(int argc, char *argv[]);
+extern void BackendMain(int argc, char *argv[]);
+extern void BackendPostmasterMain(int argc, char *argv[]);
+extern bool BackendForkFailure(int child_errno);
+extern void BackgroundWorkerMain(int argc, char *argv[]);
+
#ifdef EXEC_BACKEND
extern pid_t postmaster_forkexec(int argc, char *argv[]);
extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 706d60e4d6..06e67efecc 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -29,6 +29,7 @@ typedef enum
PgArchiverType,
SysLoggerType,
BgWorkerType,
+ ClientBackendType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.21.1
0010-Add-WalSenderType-to-subprocess-struct.patch (text/x-patch, 1.4K) download
From 230eaf3002c1ab1b74aed209804cff761dcceac4 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 17 Mar 2020 18:39:12 +0000
Subject: [PATCH 10/11] Add WalSenderType to subprocess struct
---
src/backend/postmaster/subprocess.c | 11 +++++++++++
src/include/postmaster/subprocess.h | 1 +
2 files changed, 12 insertions(+)
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 9d7ce86918..8cc88792ea 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -181,6 +181,17 @@ static PgSubprocess process_types[] = {
.entrypoint = BackendMain,
.fork_failure = BackendForkFailure,
.postmaster_main = BackendPostmasterMain
+ },
+ { /* Wal Sender placeholder */
+ .name = "walsender",
+ .desc = "wal sender",
+ .needs_aux_proc = false,
+ .needs_shmem = false,
+ .keep_postmaster_memcontext = false,
+ .fork_prep = NULL,
+ .entrypoint = NULL,
+ .fork_failure = NULL,
+ .postmaster_main = NULL
}
};
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 06e67efecc..867e1cb499 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -31,6 +31,7 @@ typedef enum
BgWorkerType,
ClientBackendType,
+ WalSenderType, /* placeholder for wal sender so it can be identified in pgstat */
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.21.1
0011-Get-rid-of-st_beckendType-and-use-MySubprocess.patch (text/x-patch, 6.6K) download
From 25200a43631b664859ed256dea20b6db2ef97d66 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 17 Mar 2020 18:39:30 +0000
Subject: [PATCH 11/11] Get rid of st_beckendType and use MySubprocess
---
src/backend/postmaster/pgstat.c | 103 +---------------------------
src/backend/postmaster/postmaster.c | 3 +-
src/backend/utils/adt/pgstatfuncs.c | 9 +--
src/include/pgstat.h | 3 -
4 files changed, 10 insertions(+), 108 deletions(-)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 648b5eadb5..b1356a28df 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -2821,61 +2821,6 @@ pgstat_bestart(void)
*/
lbeentry.st_procpid = MyProcPid;
- if (MyBackendId != InvalidBackendId)
- {
- if (IsAutoVacuumLauncherProcess())
- {
- /* Autovacuum Launcher */
- lbeentry.st_backendType = B_AUTOVAC_LAUNCHER;
- }
- else if (IsAutoVacuumWorkerProcess())
- {
- /* Autovacuum Worker */
- lbeentry.st_backendType = B_AUTOVAC_WORKER;
- }
- else if (am_walsender)
- {
- /* Wal sender */
- lbeentry.st_backendType = B_WAL_SENDER;
- }
- else if (IsBackgroundWorker)
- {
- /* bgworker */
- lbeentry.st_backendType = B_BG_WORKER;
- }
- else
- {
- /* client-backend */
- lbeentry.st_backendType = B_BACKEND;
- }
- }
- else
- {
- /* Must be an auxiliary process */
- Assert(MyAuxProcType != NotAnAuxProcess);
- switch (MyAuxProcType)
- {
- case StartupProcess:
- lbeentry.st_backendType = B_STARTUP;
- break;
- case BgWriterProcess:
- lbeentry.st_backendType = B_BG_WRITER;
- break;
- case CheckpointerProcess:
- lbeentry.st_backendType = B_CHECKPOINTER;
- break;
- case WalWriterProcess:
- lbeentry.st_backendType = B_WAL_WRITER;
- break;
- case WalReceiverProcess:
- lbeentry.st_backendType = B_WAL_RECEIVER;
- break;
- default:
- elog(FATAL, "unrecognized process type: %d",
- (int) MyAuxProcType);
- }
- }
-
lbeentry.st_proc_start_timestamp = MyStartTimestamp;
lbeentry.st_activity_start_timestamp = 0;
lbeentry.st_state_start_timestamp = 0;
@@ -2883,9 +2828,9 @@ pgstat_bestart(void)
lbeentry.st_databaseid = MyDatabaseId;
/* We have userid for client-backends, wal-sender and bgworker processes */
- if (lbeentry.st_backendType == B_BACKEND
- || lbeentry.st_backendType == B_WAL_SENDER
- || lbeentry.st_backendType == B_BG_WORKER)
+ if (MySubprocessType == ClientBackendType
+ || MySubprocessType == WalSenderType
+ || MySubprocessType == BgWorkerType)
lbeentry.st_userid = GetSessionUserId();
else
lbeentry.st_userid = InvalidOid;
@@ -4197,48 +4142,6 @@ pgstat_get_crashed_backend_activity(int pid, char *buffer, int buflen)
return NULL;
}
-const char *
-pgstat_get_backend_desc(BackendType backendType)
-{
- const char *backendDesc = "unknown process type";
-
- switch (backendType)
- {
- case B_AUTOVAC_LAUNCHER:
- backendDesc = "autovacuum launcher";
- break;
- case B_AUTOVAC_WORKER:
- backendDesc = "autovacuum worker";
- break;
- case B_BACKEND:
- backendDesc = "client backend";
- break;
- case B_BG_WORKER:
- backendDesc = "background worker";
- break;
- case B_BG_WRITER:
- backendDesc = "background writer";
- break;
- case B_CHECKPOINTER:
- backendDesc = "checkpointer";
- break;
- case B_STARTUP:
- backendDesc = "startup";
- break;
- case B_WAL_RECEIVER:
- backendDesc = "walreceiver";
- break;
- case B_WAL_SENDER:
- backendDesc = "walsender";
- break;
- case B_WAL_WRITER:
- backendDesc = "walwriter";
- break;
- }
-
- return backendDesc;
-}
-
/* ------------------------------------------------------------
* Local support functions follow
* ------------------------------------------------------------
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 09f8bbb298..65b8c120b5 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -2085,6 +2085,7 @@ retry1:
*/
if (strcmp(valptr, "database") == 0)
{
+ InitMySubprocess(WalSenderType);
am_walsender = true;
am_db_walsender = true;
}
@@ -4436,7 +4437,7 @@ BackendInitialize(Port *port)
* init_ps_display() to avoid abusing the parameters like this.
*/
if (am_walsender)
- init_ps_display(pgstat_get_backend_desc(B_WAL_SENDER), port->user_name, remote_ps_data,
+ init_ps_display(MySubprocess->desc, port->user_name, remote_ps_data,
update_process_title ? "authentication" : "");
else
init_ps_display(port->user_name, port->database_name, remote_ps_data,
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 54d2673254..b5692676d6 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -24,6 +24,7 @@
#include "pgstat.h"
#include "postmaster/bgworker_internals.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "storage/proc.h"
#include "storage/procarray.h"
#include "utils/acl.h"
@@ -693,7 +694,7 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
proc = BackendPidGetProc(beentry->st_procpid);
- if (proc == NULL && (beentry->st_backendType != B_BACKEND))
+ if (proc == NULL && (MySubprocessType != ClientBackendType))
{
/*
* For an auxiliary process, retrieve process info from
@@ -741,7 +742,7 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
* date.
*/
if (beentry->st_xact_start_timestamp != 0 &&
- beentry->st_backendType != B_WAL_SENDER)
+ MySubprocessType != WalSenderType)
values[8] = TimestampTzGetDatum(beentry->st_xact_start_timestamp);
else
nulls[8] = true;
@@ -829,7 +830,7 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
}
}
/* Add backend type */
- if (beentry->st_backendType == B_BG_WORKER)
+ if (MySubprocessType == BgWorkerType)
{
const char *bgw_type;
@@ -841,7 +842,7 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
}
else
values[17] =
- CStringGetTextDatum(pgstat_get_backend_desc(beentry->st_backendType));
+ CStringGetTextDatum(MySubprocess->desc);
/* SSL information */
if (beentry->st_ssl)
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index c7ddfaecc2..97c1683e47 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1049,9 +1049,6 @@ typedef struct PgBackendStatus
/* The entry is valid iff st_procpid > 0, unused if st_procpid == 0 */
int st_procpid;
- /* Type of backends */
- BackendType st_backendType;
-
/* Times when current backend, transaction, and activity started */
TimestampTz st_proc_start_timestamp;
TimestampTz st_xact_start_timestamp;
--
2.21.1
0012-Move-away-from-argv-string-comparison.patch (text/x-patch, 1.3K) download
From f0b8d67fc61ee7e43960c084bce0adc1f88c8c46 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 17 Mar 2020 14:43:52 -0400
Subject: [PATCH 12/12] Move away from argv string comparison
---
src/backend/postmaster/postmaster.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 65b8c120b5..67a85c61d0 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -4882,7 +4882,7 @@ SubPostmasterMain(int argc, char *argv[])
ConnProcPort = &port;
/* Close the postmaster's sockets (as soon as we know them) */
- ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
+ ClosePostmasterPorts(MySubprocessType == SysLoggerType);
/*
* Set reference point for stack-depth checking
@@ -4927,9 +4927,9 @@ SubPostmasterMain(int argc, char *argv[])
PGSharedMemoryNoReAttach();
/* autovacuum needs this set before calling InitProcess */
- if (strcmp(argv[1], "--forkavlauncher") == 0)
+ if (MySubprocessType == AutoVacuumLauncherType)
AutovacuumLauncherIAm();
- if (strcmp(argv[1], "--forkavworker") == 0)
+ if (MySubprocessType == AutoVacuumWorkerType)
AutovacuumWorkerIAm();
/*
--
2.21.1
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-02 14:53 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-04 04:56 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 18:50 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
@ 2020-03-17 21:52 ` Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 01:04 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
0 siblings, 1 reply; 38+ messages in thread
From: Justin Pryzby @ 2020-03-17 21:52 UTC (permalink / raw)
To: Mike Palmiotto <mike.palmiotto@crunchydata.com>; +Cc: Michael Paquier <michael@paquier.xyz>; Andres Freund <andres@anarazel.de>; Alvaro Herrera <alvherre@2ndquadrant.com>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Tue, Mar 17, 2020 at 02:50:19PM -0400, Mike Palmiotto wrote:
> The patchset is now split out. I've just noticed that Peter Eisentraut
> included some changes for a generic MyBackendType, which I should have
> been aware of. I was unable to rebase due to these changes, but can
> fold these patches into that framework if others think it's
> worthwhile.
I don't have many comments on the patch, but it's easy enough to rebase.
I think maybe you'll want to do something more with this new function:
GetBackendTypeDesc()
+ /* Don't panic. */
+1
--
Justin
Attachments: download all (12 files, .tar.gz)
v2-0001-Add-subprocess-infrastructure.patch (text/x-diff, 13.9K) download
From 4234a083913d93b27cd6763f41c092adc8d4cf6c Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Mon, 9 Mar 2020 22:12:10 +0000
Subject: [PATCH v2 01/12] Add subprocess infrastructure
This commit introduces a new infrastructure for postmaster subprocesses
initialization, forking, exec'ing and main function control. The intent
is to have all subprocess control information live in the process_types
struct, defined in subprocess.h. As more processes are added to the
struct, the fields will be expanded to differentiate actions.
There are several advantages to such an infrastructure:
1) Centralized definition of subprocess information
2) Standardized function arguments/naming schemes
3) Code minimization and removal of duplicate code
---
src/backend/bootstrap/bootstrap.c | 74 +++++++--------------------
src/backend/postmaster/Makefile | 1 +
src/backend/postmaster/bgwriter.c | 2 +-
src/backend/postmaster/checkpointer.c | 2 +-
src/backend/postmaster/startup.c | 3 +-
src/backend/postmaster/subprocess.c | 62 ++++++++++++++++++++++
src/backend/postmaster/walwriter.c | 2 +-
src/backend/replication/walreceiver.c | 2 +-
src/include/bootstrap/bootstrap.h | 3 +-
src/include/postmaster/bgwriter.h | 4 +-
src/include/postmaster/startup.h | 5 +-
src/include/postmaster/subprocess.h | 44 ++++++++++++++++
src/include/postmaster/walwriter.h | 2 +-
src/include/replication/walreceiver.h | 2 +-
14 files changed, 140 insertions(+), 68 deletions(-)
create mode 100644 src/backend/postmaster/subprocess.c
create mode 100644 src/include/postmaster/subprocess.h
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 5480a024e0..9d41567467 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -35,6 +35,7 @@
#include "pgstat.h"
#include "postmaster/bgwriter.h"
#include "postmaster/startup.h"
+#include "postmaster/subprocess.h"
#include "postmaster/walwriter.h"
#include "replication/walreceiver.h"
#include "storage/bufmgr.h"
@@ -56,8 +57,6 @@ uint32 bootstrap_data_checksum_version = 0; /* No checksum */
#define ALLOC(t, c) \
((t *) MemoryContextAllocZero(TopMemoryContext, (unsigned)(c) * sizeof(t)))
-static void CheckerModeMain(void);
-static void BootstrapModeMain(void);
static void bootstrap_signals(void);
static void ShutdownAuxiliaryProcess(int code, Datum arg);
static Form_pg_attribute AllocateAttribute(void);
@@ -334,6 +333,12 @@ AuxiliaryProcessMain(int argc, char *argv[])
default:
MyBackendType = B_INVALID;
}
+
+ /*
+ * We've read the arguments and know what backend type we are.
+ */
+ InitMySubprocess((SubprocessType)MyAuxProcType);
+
if (IsUnderPostmaster)
init_ps_display(NULL);
@@ -418,56 +423,10 @@ AuxiliaryProcessMain(int argc, char *argv[])
*/
SetProcessingMode(NormalProcessing);
- switch (MyAuxProcType)
- {
- case CheckerProcess:
- /* don't set signals, they're useless here */
- CheckerModeMain();
- proc_exit(1); /* should never return */
-
- case BootstrapProcess:
-
- /*
- * There was a brief instant during which mode was Normal; this is
- * okay. We need to be in bootstrap mode during BootStrapXLOG for
- * the sake of multixact initialization.
- */
- SetProcessingMode(BootstrapProcessing);
- bootstrap_signals();
- BootStrapXLOG();
- BootstrapModeMain();
- proc_exit(1); /* should never return */
-
- case StartupProcess:
- /* don't set signals, startup process has its own agenda */
- StartupProcessMain();
- proc_exit(1); /* should never return */
-
- case BgWriterProcess:
- /* don't set signals, bgwriter has its own agenda */
- BackgroundWriterMain();
- proc_exit(1); /* should never return */
-
- case CheckpointerProcess:
- /* don't set signals, checkpointer has its own agenda */
- CheckpointerMain();
- proc_exit(1); /* should never return */
-
- case WalWriterProcess:
- /* don't set signals, walwriter has its own agenda */
- InitXLOGAccess();
- WalWriterMain();
- proc_exit(1); /* should never return */
-
- case WalReceiverProcess:
- /* don't set signals, walreceiver has its own agenda */
- WalReceiverMain();
- proc_exit(1); /* should never return */
+ /* Now jump into the subprocess main function and never look back! */
+ MySubprocess->entrypoint(argc, argv);
- default:
- elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
- proc_exit(1);
- }
+ proc_exit(1); /* should never return */
}
/*
@@ -476,8 +435,8 @@ AuxiliaryProcessMain(int argc, char *argv[])
* settings). Since, in fact, that was already done by BaseInit(),
* we have nothing more to do here.
*/
-static void
-CheckerModeMain(void)
+void
+CheckerModeMain(int argc, char *argv[])
{
proc_exit(0);
}
@@ -489,11 +448,15 @@ CheckerModeMain(void)
* The bootstrap backend doesn't speak SQL, but instead expects
* commands in a special bootstrap language.
*/
-static void
-BootstrapModeMain(void)
+void
+BootstrapModeMain(int argc, char *argv[])
{
int i;
+ SetProcessingMode(BootstrapProcessing);
+ bootstrap_signals();
+ BootStrapXLOG();
+
Assert(!IsUnderPostmaster);
Assert(IsBootstrapProcessingMode());
@@ -508,7 +471,6 @@ BootstrapModeMain(void)
* Do backend-like initialization for bootstrap mode
*/
InitProcess();
-
InitPostgres(NULL, InvalidOid, NULL, InvalidOid, NULL, false);
/* Initialize stuff for bootstrap-file processing */
diff --git a/src/backend/postmaster/Makefile b/src/backend/postmaster/Makefile
index bfdf6a833d..52b4c2142c 100644
--- a/src/backend/postmaster/Makefile
+++ b/src/backend/postmaster/Makefile
@@ -23,6 +23,7 @@ OBJS = \
pgstat.o \
postmaster.o \
startup.o \
+ subprocess.o \
syslogger.o \
walwriter.o
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 069e27e427..b2170938f3 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -91,7 +91,7 @@ static XLogRecPtr last_snapshot_lsn = InvalidXLogRecPtr;
* basic execution environment, but not enabled signals yet.
*/
void
-BackgroundWriterMain(void)
+BackgroundWriterMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
MemoryContext bgwriter_context;
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index e354a78725..83f7be1b1e 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -180,7 +180,7 @@ static void ReqCheckpointHandler(SIGNAL_ARGS);
* basic execution environment, but not enabled signals yet.
*/
void
-CheckpointerMain(void)
+CheckpointerMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
MemoryContext checkpointer_context;
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index c2250d7d4e..e29649a6a1 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -127,13 +127,12 @@ HandleStartupProcInterrupts(void)
ProcessProcSignalBarrier();
}
-
/* ----------------------------------
* Startup Process main entry point
* ----------------------------------
*/
void
-StartupProcessMain(void)
+StartupProcessMain(int argc, char *argv[])
{
/*
* Properly accept or ignore signals the postmaster might send us.
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
new file mode 100644
index 0000000000..3e7a45bf10
--- /dev/null
+++ b/src/backend/postmaster/subprocess.c
@@ -0,0 +1,62 @@
+/*-------------------------------------------------------------------------
+ *
+ * subprocess.c
+ *
+ * Copyright (c) 2004-2020, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/postmaster/syslogger.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "bootstrap/bootstrap.h"
+#include "postmaster/bgwriter.h"
+#include "postmaster/postmaster.h"
+#include "postmaster/startup.h"
+#include "postmaster/subprocess.h"
+#include "postmaster/walwriter.h"
+#include "replication/walreceiver.h"
+
+SubprocessType MySubprocessType;
+PgSubprocess *MySubprocess;
+
+static PgSubprocess process_types[] = {
+ {
+ .desc = "checker",
+ .entrypoint = CheckerModeMain
+ },
+ {
+ .desc = "bootstrap",
+ .entrypoint = BootstrapModeMain
+ },
+ {
+ .desc = "startup",
+ .entrypoint = StartupProcessMain
+ },
+ {
+ .desc = "background writer",
+ .entrypoint = BackgroundWriterMain
+ },
+ {
+ .desc = "checkpointer",
+ .entrypoint = CheckpointerMain
+ },
+ {
+ .desc = "wal writer",
+ .entrypoint = WalWriterMain
+ },
+ {
+ .desc = "wal receiver",
+ .entrypoint = WalReceiverMain
+ }
+};
+
+void
+InitMySubprocess(SubprocessType type)
+{
+ MySubprocessType = type;
+ MySubprocess = &process_types[type];
+ MySubprocess->desc = gettext(MySubprocess->desc);
+}
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index 45a2757969..3a0cec3371 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -85,7 +85,7 @@ int WalWriterFlushAfter = 128;
* basic execution environment, but not enabled signals yet.
*/
void
-WalWriterMain(void)
+WalWriterMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
MemoryContext walwriter_context;
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index 25e0333c9e..cd39a8c982 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -165,7 +165,7 @@ ProcessWalRcvInterrupts(void)
/* Main entry point for walreceiver process */
void
-WalReceiverMain(void)
+WalReceiverMain(int argc, char *argv[])
{
char conninfo[MAXCONNINFO];
char *tmp_conninfo;
diff --git a/src/include/bootstrap/bootstrap.h b/src/include/bootstrap/bootstrap.h
index b67fa9acc1..f85cfb5202 100644
--- a/src/include/bootstrap/bootstrap.h
+++ b/src/include/bootstrap/bootstrap.h
@@ -16,7 +16,6 @@
#include "nodes/execnodes.h"
-
/*
* MAXATTR is the maximum number of attributes in a relation supported
* at bootstrap time (i.e., the max possible in a system table).
@@ -31,6 +30,8 @@ extern Relation boot_reldesc;
extern Form_pg_attribute attrtypes[MAXATTR];
extern int numattr;
+extern void CheckerModeMain(int argc, char *argv[]);
+extern void BootstrapModeMain(int argc, char *argv[]);
extern void AuxiliaryProcessMain(int argc, char *argv[]) pg_attribute_noreturn();
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..6fb1ee1d6d 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -27,8 +27,8 @@ extern int CheckPointTimeout;
extern int CheckPointWarning;
extern double CheckPointCompletionTarget;
-extern void BackgroundWriterMain(void) pg_attribute_noreturn();
-extern void CheckpointerMain(void) pg_attribute_noreturn();
+extern void BackgroundWriterMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern void CheckpointerMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void RequestCheckpoint(int flags);
extern void CheckpointWriteDelay(int flags, double progress);
diff --git a/src/include/postmaster/startup.h b/src/include/postmaster/startup.h
index 9f59c1ffa3..6a8108dfe0 100644
--- a/src/include/postmaster/startup.h
+++ b/src/include/postmaster/startup.h
@@ -13,10 +13,13 @@
#define _STARTUP_H
extern void HandleStartupProcInterrupts(void);
-extern void StartupProcessMain(void) pg_attribute_noreturn();
extern void PreRestoreCommand(void);
extern void PostRestoreCommand(void);
extern bool IsPromoteTriggered(void);
extern void ResetPromoteTriggered(void);
+/* Startup subprocess functions */
+extern void StartupProcessMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern bool StartupCleanup(int child_errno);
+
#endif /* _STARTUP_H */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
new file mode 100644
index 0000000000..56e8edf2d8
--- /dev/null
+++ b/src/include/postmaster/subprocess.h
@@ -0,0 +1,44 @@
+/*-------------------------------------------------------------------------
+ *
+ * subprocess.h
+ *
+ * Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ *
+ * src/include/postmaster/subprocess.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef SUBPROCESS_H
+#define SUBPROCESS_H
+
+#include "postmaster.h"
+
+typedef enum
+{
+ NoProcessType = -1,
+ CheckerType = 0,
+ BootstrapType,
+ StartupType,
+ BgWriterType,
+ CheckpointerType,
+ WalWriterType,
+ WalReceiverType, /* end of Auxiliary Process Forks */
+
+ NUMSUBPROCESSTYPES /* Must be last! */
+} SubprocessType;
+
+typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
+
+/* Current subprocess initializer */
+extern void InitMySubprocess(SubprocessType type);
+
+typedef struct PgSubprocess
+{
+ const char *desc;
+ SubprocessEntryPoint entrypoint;
+} PgSubprocess;
+
+extern SubprocessType MySubprocessType;
+extern PGDLLIMPORT PgSubprocess *MySubprocess;
+
+#endif /* SUBPROCESS_H */
diff --git a/src/include/postmaster/walwriter.h b/src/include/postmaster/walwriter.h
index 011dc26a6e..596c437c54 100644
--- a/src/include/postmaster/walwriter.h
+++ b/src/include/postmaster/walwriter.h
@@ -16,6 +16,6 @@
extern int WalWriterDelay;
extern int WalWriterFlushAfter;
-extern void WalWriterMain(void) pg_attribute_noreturn();
+extern void WalWriterMain(int argc, char *argv[]) pg_attribute_noreturn();
#endif /* _WALWRITER_H */
diff --git a/src/include/replication/walreceiver.h b/src/include/replication/walreceiver.h
index e08afc6548..2025f19ab3 100644
--- a/src/include/replication/walreceiver.h
+++ b/src/include/replication/walreceiver.h
@@ -311,7 +311,7 @@ walrcv_clear_result(WalRcvExecResult *walres)
}
/* prototypes for functions in walreceiver.c */
-extern void WalReceiverMain(void) pg_attribute_noreturn();
+extern void WalReceiverMain(int argc, char *argv[]);
extern void ProcessWalRcvInterrupts(void);
/* prototypes for functions in walreceiverfuncs.c */
--
2.17.0
v2-0002-Use-centralized-StartSubprocess-for-aux-procs.patch (text/x-diff, 14.9K) download
From e468f43243a3a82ea3a5b782a0bda3ed42c1286a Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 10 Mar 2020 14:58:07 +0000
Subject: [PATCH v2 02/12] Use centralized StartSubprocess for aux procs
Modifies StartChildProcess and moves the auxiliary processes under its
control. This is the next step toward making use of the Subprocess
infrastructure.
This patch introduces the fork_failure function, which may be
implemented by each subprocess to control failure when attempting to
fork the subprocess. This is used by the startup subprocess to exit
postmaster on fork failure. Future subprocesses may use it to do cleanup
(and optionally exit postmaster) after a fork failure.
Additionally, the patch adds a new backend variable for establishing a
backend variable to be shared across fork/exec.
---
src/backend/postmaster/postmaster.c | 156 ++++++++++++++--------------
src/backend/postmaster/startup.c | 12 +++
src/backend/postmaster/subprocess.c | 35 +++++--
src/include/postmaster/startup.h | 2 +-
src/include/postmaster/subprocess.h | 4 +
5 files changed, 121 insertions(+), 88 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 2b9ab32293..4b86aeec2d 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -114,6 +114,7 @@
#include "postmaster/fork_process.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "postmaster/syslogger.h"
#include "replication/logicallauncher.h"
#include "replication/walsender.h"
@@ -191,7 +192,8 @@ static Backend *ShmemBackendArray;
BackgroundWorker *MyBgworkerEntry = NULL;
-
+/* Struct containing postmaster subprocess control info */
+SubprocessType MySubprocessType;
/* The socket number we are listening for connections on */
int PostPortNumber;
@@ -421,7 +423,7 @@ static int CountChildren(int target);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
-static pid_t StartChildProcess(AuxProcType type);
+static pid_t StartSubprocess(SubprocessType type);
static void StartAutovacuumWorker(void);
static void MaybeStartWalReceiver(void);
static void InitPostmasterDeathWatchHandle(void);
@@ -507,6 +509,9 @@ typedef struct
TimestampTz PgStartTime;
TimestampTz PgReloadTime;
pg_time_t first_syslogger_file_time;
+
+ SubprocessType MySubprocessType;
+
bool redirection_done;
bool IsBinaryUpgrade;
int max_safe_fds;
@@ -538,11 +543,6 @@ static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
-#define StartupDataBase() StartChildProcess(StartupProcess)
-#define StartBackgroundWriter() StartChildProcess(BgWriterProcess)
-#define StartCheckpointer() StartChildProcess(CheckpointerProcess)
-#define StartWalWriter() StartChildProcess(WalWriterProcess)
-#define StartWalReceiver() StartChildProcess(WalReceiverProcess)
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -1389,7 +1389,7 @@ PostmasterMain(int argc, char *argv[])
/*
* We're ready to rock and roll...
*/
- StartupPID = StartupDataBase();
+ StartupPID = StartSubprocess(StartupType);
Assert(StartupPID != 0);
StartupStatus = STARTUP_RUNNING;
pmState = PM_STARTUP;
@@ -1750,9 +1750,9 @@ ServerLoop(void)
pmState == PM_HOT_STANDBY)
{
if (CheckpointerPID == 0)
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
if (BgWriterPID == 0)
- BgWriterPID = StartBackgroundWriter();
+ BgWriterPID = StartSubprocess(BgWriterType);
}
/*
@@ -1761,7 +1761,7 @@ ServerLoop(void)
* be writing any new WAL).
*/
if (WalWriterPID == 0 && pmState == PM_RUN)
- WalWriterPID = StartWalWriter();
+ WalWriterPID = StartSubprocess(WalWriterType);
/*
* If we have lost the autovacuum launcher, try to start a new one. We
@@ -3042,11 +3042,11 @@ reaper(SIGNAL_ARGS)
* if this fails, we'll just try again later.
*/
if (CheckpointerPID == 0)
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
if (BgWriterPID == 0)
- BgWriterPID = StartBackgroundWriter();
+ BgWriterPID = StartSubprocess(BgWriterType);
if (WalWriterPID == 0)
- WalWriterPID = StartWalWriter();
+ WalWriterPID = StartSubprocess(WalWriterType);
/*
* Likewise, start other special children as needed. In a restart
@@ -3860,7 +3860,7 @@ PostmasterStateMachine(void)
Assert(Shutdown > NoShutdown);
/* Start the checkpointer if not running */
if (CheckpointerPID == 0)
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
/* And tell it to shut down */
if (CheckpointerPID != 0)
{
@@ -4001,7 +4001,7 @@ PostmasterStateMachine(void)
reset_shared();
- StartupPID = StartupDataBase();
+ StartupPID = StartSubprocess(StartupType);
Assert(StartupPID != 0);
StartupStatus = STARTUP_RUNNING;
pmState = PM_STARTUP;
@@ -4917,6 +4917,10 @@ SubPostmasterMain(int argc, char *argv[])
errmsg("out of memory")));
#endif
+ /* We should only be here once per fork */
+ Assert(!MySubprocess);
+ InitMySubprocess(MySubprocessType);
+
/*
* If appropriate, physically re-attach to shared memory segment. We want
* to do this before going any further to ensure that we can attach at the
@@ -5037,19 +5041,6 @@ SubPostmasterMain(int argc, char *argv[])
/* And run the backend */
BackendRun(&port); /* does not return */
}
- if (strcmp(argv[1], "--forkboot") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitAuxiliaryProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
- }
if (strcmp(argv[1], "--forkavlauncher") == 0)
{
/* Restore basic shared memory pointers */
@@ -5117,6 +5108,20 @@ SubPostmasterMain(int argc, char *argv[])
SysLoggerMain(argc, argv); /* does not return */
}
+ if (MySubprocess->needs_aux_proc)
+ {
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
+
+ /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ InitAuxiliaryProcess();
+
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores();
+
+ AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
+ }
+
abort(); /* shouldn't get here */
}
#endif /* EXEC_BACKEND */
@@ -5198,9 +5203,9 @@ sigusr1_handler(SIGNAL_ARGS)
* we'll just try again later.
*/
Assert(CheckpointerPID == 0);
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
Assert(BgWriterPID == 0);
- BgWriterPID = StartBackgroundWriter();
+ BgWriterPID = StartSubprocess(BgWriterType);
/*
* Start the archiver if we're responsible for (re-)archiving received
@@ -5425,42 +5430,49 @@ CountChildren(int target)
return cnt;
}
-
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
- * "type" determines what kind of child will be started. All child types
- * initially go to AuxiliaryProcessMain, which will handle common setup.
+ * "type" determines what kind of child will be started. This code relies
+ * heavily on MySubprocess, which is an entry in the process_types struct in
+ * subprocess.c. All auxiliary processes go through AuxiliaryProcessMain after
+ * forking. Everything else goes through the designated entrypoint.
*
- * Return value of StartChildProcess is subprocess' PID, or 0 if failed
- * to start subprocess.
+ * Return value of StartChildProcess is subprocess' PID on success. On failure,
+ * the subprocess cleanup function decides whether or not to panic.
*/
static pid_t
-StartChildProcess(AuxProcType type)
+StartSubprocess(SubprocessType type)
{
pid_t pid;
- char *av[10];
- int ac = 0;
+ char *argv[10];
+ int argc = 0;
char typebuf[32];
+#ifdef EXEC_BACKEND
+ char forkname[32];
+#endif
/*
- * Set up command-line arguments for subprocess
+ * Get new subprocess data every time we start a new subprocess.
*/
- av[ac++] = "postgres";
+ InitMySubprocess(type);
+
+ argv[argc++] = "postgres";
#ifdef EXEC_BACKEND
- av[ac++] = "--forkboot";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
+ snprintf(forkname, sizeof(forkname), "--fork%s", MySubprocess->name);
+ argv[argc++] = forkname;
+ argv[argc++] = NULL;
#endif
snprintf(typebuf, sizeof(typebuf), "-x%d", type);
- av[ac++] = typebuf;
+ argv[argc++] = typebuf;
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ argv[argc] = NULL;
+ Assert(argc < lengthof(argv));
#ifdef EXEC_BACKEND
- pid = postmaster_forkexec(ac, av);
+ pid = postmaster_forkexec(argc, argv);
#else /* !EXEC_BACKEND */
pid = fork_process();
@@ -5476,7 +5488,7 @@ StartChildProcess(AuxProcType type)
MemoryContextDelete(PostmasterContext);
PostmasterContext = NULL;
- AuxiliaryProcessMain(ac, av);
+ AuxiliaryProcessMain(argc, argv);
ExitPostmaster(0);
}
#endif /* EXEC_BACKEND */
@@ -5487,40 +5499,20 @@ StartChildProcess(AuxProcType type)
int save_errno = errno;
errno = save_errno;
- switch (type)
- {
- case StartupProcess:
- ereport(LOG,
- (errmsg("could not fork startup process: %m")));
- break;
- case BgWriterProcess:
- ereport(LOG,
- (errmsg("could not fork background writer process: %m")));
- break;
- case CheckpointerProcess:
- ereport(LOG,
- (errmsg("could not fork checkpointer process: %m")));
- break;
- case WalWriterProcess:
- ereport(LOG,
- (errmsg("could not fork WAL writer process: %m")));
- break;
- case WalReceiverProcess:
- ereport(LOG,
- (errmsg("could not fork WAL receiver process: %m")));
- break;
- default:
- ereport(LOG,
- (errmsg("could not fork process: %m")));
- break;
- }
+ ereport(LOG,
+ (errmsg("could not fork %s process: %m", MySubprocess->desc)));
/*
- * fork failure is fatal during startup, but there's no need to choke
- * immediately if starting other child types fails.
+ * Handle fork-failure per subprocess type. Each subprocess handles
+ * fork-failure by returning a boolean which tells postmaster whether or
+ * not to panic.
*/
- if (type == StartupProcess)
- ExitPostmaster(1);
+ if (MySubprocess->fork_failure)
+ {
+ if (MySubprocess->fork_failure(save_errno))
+ ExitPostmaster(1);
+ }
+
return 0;
}
@@ -5638,7 +5630,7 @@ MaybeStartWalReceiver(void)
pmState == PM_HOT_STANDBY || pmState == PM_WAIT_READONLY) &&
Shutdown == NoShutdown)
{
- WalReceiverPID = StartWalReceiver();
+ WalReceiverPID = StartSubprocess(WalReceiverType);
if (WalReceiverPID != 0)
WalReceiverRequested = false;
/* else leave the flag set, so we'll try again later */
@@ -6198,6 +6190,8 @@ save_backend_variables(BackendParameters *param, Port *port,
param->PgReloadTime = PgReloadTime;
param->first_syslogger_file_time = first_syslogger_file_time;
+ param->MySubprocessType = MySubprocessType;
+
param->redirection_done = redirection_done;
param->IsBinaryUpgrade = IsBinaryUpgrade;
param->max_safe_fds = max_safe_fds;
@@ -6433,6 +6427,8 @@ restore_backend_variables(BackendParameters *param, Port *port)
PgReloadTime = param->PgReloadTime;
first_syslogger_file_time = param->first_syslogger_file_time;
+ MySubprocessType = param->MySubprocessType;
+
redirection_done = param->redirection_done;
IsBinaryUpgrade = param->IsBinaryUpgrade;
max_safe_fds = param->max_safe_fds;
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index e29649a6a1..782cd608ff 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -175,6 +175,18 @@ StartupProcessMain(int argc, char *argv[])
proc_exit(0);
}
+/*
+ * StartupForkFailure
+ *
+ * Tell the postmaster to exit/panic on failure to fork.
+ */
+bool
+StartupForkFailure(int fork_errno)
+{
+ /* Panic! */
+ return true;
+}
+
void
PreRestoreCommand(void)
{
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 3e7a45bf10..7e96678dfb 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -24,32 +24,53 @@ PgSubprocess *MySubprocess;
static PgSubprocess process_types[] = {
{
+ .name = "boot",
.desc = "checker",
- .entrypoint = CheckerModeMain
+ .needs_aux_proc = true,
+ .entrypoint = CheckerModeMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "bootstrap",
- .entrypoint = BootstrapModeMain
+ .needs_aux_proc = true,
+ .entrypoint = BootstrapModeMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "startup",
- .entrypoint = StartupProcessMain
+ .needs_aux_proc = true,
+ .entrypoint = StartupProcessMain,
+ .fork_failure = StartupForkFailure
},
{
+ .name = "boot",
.desc = "background writer",
- .entrypoint = BackgroundWriterMain
+ .needs_aux_proc = true,
+ .entrypoint = BackgroundWriterMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "checkpointer",
- .entrypoint = CheckpointerMain
+ .needs_aux_proc = true,
+ .entrypoint = CheckpointerMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "wal writer",
- .entrypoint = WalWriterMain
+ .needs_aux_proc = true,
+ .entrypoint = WalWriterMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "wal receiver",
- .entrypoint = WalReceiverMain
+ .needs_aux_proc = true,
+ .entrypoint = WalReceiverMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/postmaster/startup.h b/src/include/postmaster/startup.h
index 6a8108dfe0..c182c76960 100644
--- a/src/include/postmaster/startup.h
+++ b/src/include/postmaster/startup.h
@@ -20,6 +20,6 @@ extern void ResetPromoteTriggered(void);
/* Startup subprocess functions */
extern void StartupProcessMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern bool StartupCleanup(int child_errno);
+extern bool StartupForkFailure(int fork_errno);
#endif /* _STARTUP_H */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 56e8edf2d8..33f6a7ab04 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -28,14 +28,18 @@ typedef enum
} SubprocessType;
typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
+typedef bool (*SubprocessForkFailure) (int fork_errno);
/* Current subprocess initializer */
extern void InitMySubprocess(SubprocessType type);
typedef struct PgSubprocess
{
+ const char *name;
const char *desc;
+ bool needs_aux_proc;
SubprocessEntryPoint entrypoint;
+ SubprocessForkFailure fork_failure;
} PgSubprocess;
extern SubprocessType MySubprocessType;
--
2.17.0
v2-0003-Add-AutoVacLauncherType-to-subprocess-struct.patch (text/x-diff, 7.4K) download
From 9e649e32ee68d9a7145a35d02c59bdc03442d165 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 10 Mar 2020 22:29:44 +0000
Subject: [PATCH v2 03/12] Add AutoVacLauncherType to subprocess struct
---
src/backend/postmaster/autovacuum.c | 66 ++---------------------------
src/backend/postmaster/postmaster.c | 36 +++++++++-------
src/backend/postmaster/subprocess.c | 8 ++++
src/include/postmaster/autovacuum.h | 3 +-
src/include/postmaster/subprocess.h | 1 +
5 files changed, 34 insertions(+), 80 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index da75e755f0..d0fdb7e5ee 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -95,6 +95,7 @@
#include "storage/procsignal.h"
#include "storage/sinvaladt.h"
#include "storage/smgr.h"
+#include "postmaster/subprocess.h"
#include "tcop/tcopprot.h"
#include "utils/fmgroids.h"
#include "utils/fmgrprotos.h"
@@ -303,11 +304,9 @@ static WorkerInfo MyWorkerInfo = NULL;
int AutovacuumLauncherPid = 0;
#ifdef EXEC_BACKEND
-static pid_t avlauncher_forkexec(void);
static pid_t avworker_forkexec(void);
#endif
NON_EXEC_STATIC void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-NON_EXEC_STATIC void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
static Oid do_start_worker(void);
static void HandleAutoVacLauncherInterrupts(void);
@@ -353,26 +352,6 @@ static void autovac_refresh_stats(void);
********************************************************************/
#ifdef EXEC_BACKEND
-/*
- * forkexec routine for the autovacuum launcher process.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avlauncher_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavlauncher";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
/*
* We need this set from the outside, before InitProcess is called
@@ -384,50 +363,11 @@ AutovacuumLauncherIAm(void)
}
#endif
-/*
- * Main entry point for autovacuum launcher process, to be called from the
- * postmaster.
- */
-int
-StartAutoVacLauncher(void)
-{
- pid_t AutoVacPID;
-
-#ifdef EXEC_BACKEND
- switch ((AutoVacPID = avlauncher_forkexec()))
-#else
- switch ((AutoVacPID = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum launcher process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacLauncherMain(0, NULL);
- break;
-#endif
- default:
- return (int) AutoVacPID;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
/*
* Main loop for the autovacuum launcher process.
*/
-NON_EXEC_STATIC void
-AutoVacLauncherMain(int argc, char *argv[])
+void
+AutoVacLauncherMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 4b86aeec2d..2251303589 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1773,7 +1773,7 @@ ServerLoop(void)
(AutoVacuumingActive() || start_autovac_launcher) &&
pmState == PM_RUN)
{
- AutoVacPID = StartAutoVacLauncher();
+ AutoVacPID = StartSubprocess(AutoVacuumLauncherType);
if (AutoVacPID != 0)
start_autovac_launcher = false; /* signal processed */
}
@@ -3053,7 +3053,7 @@ reaper(SIGNAL_ARGS)
* situation, some of them may be alive already.
*/
if (!IsBinaryUpgrade && AutoVacuumingActive() && AutoVacPID == 0)
- AutoVacPID = StartAutoVacLauncher();
+ AutoVacPID = StartSubprocess(AutoVacuumLauncherType);
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = pgarch_start();
if (PgStatPID == 0)
@@ -5041,19 +5041,6 @@ SubPostmasterMain(int argc, char *argv[])
/* And run the backend */
BackendRun(&port); /* does not return */
}
- if (strcmp(argv[1], "--forkavlauncher") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
- }
if (strcmp(argv[1], "--forkavworker") == 0)
{
/* Restore basic shared memory pointers */
@@ -5121,6 +5108,19 @@ SubPostmasterMain(int argc, char *argv[])
AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
}
+ else
+ {
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
+
+ /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ InitProcess();
+
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores();
+
+ MySubprocess->entrypoint(argc - 2, argv + 2);
+ }
abort(); /* shouldn't get here */
}
@@ -5488,7 +5488,11 @@ StartSubprocess(SubprocessType type)
MemoryContextDelete(PostmasterContext);
PostmasterContext = NULL;
- AuxiliaryProcessMain(argc, argv);
+ if (MySubprocess->needs_aux_proc)
+ AuxiliaryProcessMain(argc, argv);
+ else
+ MySubprocess->entrypoint(argc, argv);
+
ExitPostmaster(0);
}
#endif /* EXEC_BACKEND */
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 7e96678dfb..9aa60cc93e 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -12,6 +12,7 @@
*/
#include "postgres.h"
#include "bootstrap/bootstrap.h"
+#include "postmaster/autovacuum.h"
#include "postmaster/bgwriter.h"
#include "postmaster/postmaster.h"
#include "postmaster/startup.h"
@@ -71,6 +72,13 @@ static PgSubprocess process_types[] = {
.needs_aux_proc = true,
.entrypoint = WalReceiverMain,
.fork_failure = NULL
+ },
+ {
+ .name = "avlauncher",
+ .desc = "autovacuum launcher",
+ .needs_aux_proc = false,
+ .entrypoint = AutoVacLauncherMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index d40ed55531..08050cd81c 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -58,6 +58,8 @@ extern void autovac_init(void);
extern int StartAutoVacLauncher(void);
extern int StartAutoVacWorker(void);
+extern void AutoVacLauncherMain(int argc, char *argv[]);
+
/* called from postmaster when a worker could not be forked */
extern void AutoVacWorkerFailed(void);
@@ -65,7 +67,6 @@ extern void AutoVacWorkerFailed(void);
extern void AutoVacuumUpdateDelay(void);
#ifdef EXEC_BACKEND
-extern void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void AutovacuumWorkerIAm(void);
extern void AutovacuumLauncherIAm(void);
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 33f6a7ab04..9e3c743333 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -23,6 +23,7 @@ typedef enum
CheckpointerType,
WalWriterType,
WalReceiverType, /* end of Auxiliary Process Forks */
+ AutoVacuumLauncherType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.17.0
v2-0004-Add-AutoVacuumWorkerType-to-subprocess-struct.patch (text/x-diff, 6.2K) download
From f804f37d6e0cba40fdad14f91e6ca08d626c0ffe Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 10 Mar 2020 23:33:39 +0000
Subject: [PATCH v2 04/12] Add AutoVacuumWorkerType to subprocess struct
---
src/backend/postmaster/autovacuum.c | 69 +----------------------------
src/backend/postmaster/postmaster.c | 17 +------
src/backend/postmaster/subprocess.c | 7 +++
src/include/postmaster/autovacuum.h | 4 +-
src/include/postmaster/subprocess.h | 1 +
5 files changed, 13 insertions(+), 85 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index d0fdb7e5ee..ba19a9c5d2 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -83,7 +83,6 @@
#include "nodes/makefuncs.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "storage/bufmgr.h"
@@ -303,11 +302,6 @@ static WorkerInfo MyWorkerInfo = NULL;
/* PID of launcher, valid only in worker while shutting down */
int AutovacuumLauncherPid = 0;
-#ifdef EXEC_BACKEND
-static pid_t avworker_forkexec(void);
-#endif
-NON_EXEC_STATIC void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-
static Oid do_start_worker(void);
static void HandleAutoVacLauncherInterrupts(void);
static void AutoVacLauncherShutdown(void) pg_attribute_noreturn();
@@ -1364,27 +1358,6 @@ avl_sigusr2_handler(SIGNAL_ARGS)
********************************************************************/
#ifdef EXEC_BACKEND
-/*
- * forkexec routines for the autovacuum worker.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avworker_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavworker";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
/*
* We need this set from the outside, before InitProcess is called
*/
@@ -1395,50 +1368,10 @@ AutovacuumWorkerIAm(void)
}
#endif
-/*
- * Main entry point for autovacuum worker process.
- *
- * This code is heavily based on pgarch.c, q.v.
- */
-int
-StartAutoVacWorker(void)
-{
- pid_t worker_pid;
-
-#ifdef EXEC_BACKEND
- switch ((worker_pid = avworker_forkexec()))
-#else
- switch ((worker_pid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum worker process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacWorkerMain(0, NULL);
- break;
-#endif
- default:
- return (int) worker_pid;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
/*
* AutoVacWorkerMain
*/
-NON_EXEC_STATIC void
+void
AutoVacWorkerMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 2251303589..21bbbf8194 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -5041,19 +5041,6 @@ SubPostmasterMain(int argc, char *argv[])
/* And run the backend */
BackendRun(&port); /* does not return */
}
- if (strcmp(argv[1], "--forkavworker") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
- }
if (strncmp(argv[1], "--forkbgworker=", 15) == 0)
{
int shmem_slot;
@@ -5573,7 +5560,7 @@ StartAutovacuumWorker(void)
bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
bn->bgworker_notify = false;
- bn->pid = StartAutoVacWorker();
+ bn->pid = StartSubprocess(AutoVacuumWorkerType);
if (bn->pid > 0)
{
bn->bkend_type = BACKEND_TYPE_AUTOVAC;
@@ -5587,7 +5574,7 @@ StartAutovacuumWorker(void)
/*
* fork failed, fall through to report -- actual error message was
- * logged by StartAutoVacWorker
+ * logged by StartSubprocess
*/
(void) ReleasePostmasterChildSlot(bn->child_slot);
free(bn);
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 9aa60cc93e..0fbd067310 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -79,6 +79,13 @@ static PgSubprocess process_types[] = {
.needs_aux_proc = false,
.entrypoint = AutoVacLauncherMain,
.fork_failure = NULL
+ },
+ {
+ .name = "avworker",
+ .desc = "autovacuum worker",
+ .needs_aux_proc = false,
+ .entrypoint = AutoVacWorkerMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 08050cd81c..51546fd4c9 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -58,7 +58,8 @@ extern void autovac_init(void);
extern int StartAutoVacLauncher(void);
extern int StartAutoVacWorker(void);
-extern void AutoVacLauncherMain(int argc, char *argv[]);
+extern void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
/* called from postmaster when a worker could not be forked */
extern void AutoVacWorkerFailed(void);
@@ -67,7 +68,6 @@ extern void AutoVacWorkerFailed(void);
extern void AutoVacuumUpdateDelay(void);
#ifdef EXEC_BACKEND
-extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void AutovacuumWorkerIAm(void);
extern void AutovacuumLauncherIAm(void);
#endif
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 9e3c743333..e11c24cc62 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -24,6 +24,7 @@ typedef enum
WalWriterType,
WalReceiverType, /* end of Auxiliary Process Forks */
AutoVacuumLauncherType,
+ AutoVacuumWorkerType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.17.0
v2-0005-Add-PgstatCollectorType-to-subprocess-struct.patch (text/x-diff, 11.6K) download
From dfb5814036358afa44670f377ef9768f782753a1 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 11 Mar 2020 00:24:42 +0000
Subject: [PATCH v2 05/12] Add PgstatCollectorType to subprocess struct
Introduce two new fields to handle subprocess control:
1) needs_shmem controls whether or not the subprocess needs to attach
to shmem. Up until now, all subprocesses needed to attach. This field
paves the way for easier handling of subprocesses which do not.
2) fork_prep allows StartSubprocess to do some preparation for
fork/exec. This is necessary to split out any functionality that
previously happened in Start functions prior to fork/exec.
---
src/backend/postmaster/pgstat.c | 86 +++--------------------------
src/backend/postmaster/postmaster.c | 39 +++++++------
src/backend/postmaster/subprocess.c | 28 ++++++++++
src/include/pgstat.h | 4 +-
src/include/postmaster/subprocess.h | 8 ++-
5 files changed, 65 insertions(+), 100 deletions(-)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index f9287b7942..6d4f2ace56 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -48,9 +48,9 @@
#include "pg_trace.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "replication/walsender.h"
#include "storage/backendid.h"
#include "storage/dsm.h"
@@ -275,11 +275,6 @@ static instr_time total_func_time;
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgstat_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgstat_beshutdown_hook(int code, Datum arg);
static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
@@ -685,46 +680,15 @@ pgstat_reset_all(void)
pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
}
-#ifdef EXEC_BACKEND
-
/*
- * pgstat_forkexec() -
+ * PgstatCollectorPrep
*
- * Format up the arglist for, then fork and exec, statistics collector process
- */
-static pid_t
-pgstat_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkcol";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
-/*
- * pgstat_start() -
- *
- * Called from postmaster at startup or after an existing collector
- * died. Attempt to fire up a fresh statistics collector.
- *
- * Returns PID of child process, or 0 if fail.
- *
- * Note: if fail, we will be called again from the postmaster main loop.
+ * Called from StartSubprocess to prepare a new collector fork
*/
int
-pgstat_start(void)
+PgstatCollectorPrep(int argc, char *argv[])
{
time_t curtime;
- pid_t pgStatPid;
/*
* Check that the socket is there, else pgstat_init failed and we can do
@@ -740,46 +704,10 @@ pgstat_start(void)
* from the postmaster main loop, we will get another chance later.
*/
curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgstat_start_time) <
+ if ((unsigned int) (curtime - last_pgstat_start_time) >=
(unsigned int) PGSTAT_RESTART_INTERVAL)
- return 0;
- last_pgstat_start_time = curtime;
+ last_pgstat_start_time = curtime;
- /*
- * Okay, fork off the collector.
- */
-#ifdef EXEC_BACKEND
- switch ((pgStatPid = pgstat_forkexec()))
-#else
- switch ((pgStatPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork statistics collector: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgstatCollectorMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgStatPid;
- }
-
- /* shouldn't get here */
return 0;
}
@@ -4326,7 +4254,7 @@ pgstat_send_bgwriter(void)
* The argc/argv parameters are valid only in EXEC_BACKEND case.
* ----------
*/
-NON_EXEC_STATIC void
+void
PgstatCollectorMain(int argc, char *argv[])
{
int len;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 21bbbf8194..e2ad879452 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1781,7 +1781,7 @@ ServerLoop(void)
/* If we have lost the stats collector, try to start a new one */
if (PgStatPID == 0 &&
(pmState == PM_RUN || pmState == PM_HOT_STANDBY))
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
@@ -3057,7 +3057,7 @@ reaper(SIGNAL_ARGS)
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = pgarch_start();
if (PgStatPID == 0)
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
/* workers may be scheduled to start now */
maybe_start_bgworkers();
@@ -3219,7 +3219,7 @@ reaper(SIGNAL_ARGS)
LogChildExit(LOG, _("statistics collector process"),
pid, exitstatus);
if (pmState == PM_RUN || pmState == PM_HOT_STANDBY)
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
continue;
}
@@ -5069,12 +5069,6 @@ SubPostmasterMain(int argc, char *argv[])
PgArchiverMain(argc, argv); /* does not return */
}
- if (strcmp(argv[1], "--forkcol") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgstatCollectorMain(argc, argv); /* does not return */
- }
if (strcmp(argv[1], "--forklog") == 0)
{
/* Do not want to attach to shared memory */
@@ -5097,14 +5091,17 @@ SubPostmasterMain(int argc, char *argv[])
}
else
{
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
+ if (MySubprocess->needs_shmem)
+ {
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
+ /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ InitProcess();
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores();
+ }
MySubprocess->entrypoint(argc - 2, argv + 2);
}
@@ -5224,7 +5221,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Likewise, start other special children as needed.
*/
Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5475,6 +5472,16 @@ StartSubprocess(SubprocessType type)
MemoryContextDelete(PostmasterContext);
PostmasterContext = NULL;
+ /*
+ * Drop connection to postmaster's shared memory if the subprocess
+ * doesn't need it.
+ */
+ if (!MySubprocess->needs_shmem)
+ {
+ dsm_detach_all();
+ PGSharedMemoryDetach();
+ }
+
if (MySubprocess->needs_aux_proc)
AuxiliaryProcessMain(argc, argv);
else
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 0fbd067310..ee44ef0789 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -12,6 +12,7 @@
*/
#include "postgres.h"
#include "bootstrap/bootstrap.h"
+#include "pgstat.h"
#include "postmaster/autovacuum.h"
#include "postmaster/bgwriter.h"
#include "postmaster/postmaster.h"
@@ -28,6 +29,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "checker",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = CheckerModeMain,
.fork_failure = NULL
},
@@ -35,6 +38,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "bootstrap",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = BootstrapModeMain,
.fork_failure = NULL
},
@@ -42,6 +47,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "startup",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = StartupProcessMain,
.fork_failure = StartupForkFailure
},
@@ -49,6 +56,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "background writer",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = BackgroundWriterMain,
.fork_failure = NULL
},
@@ -56,6 +65,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "checkpointer",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = CheckpointerMain,
.fork_failure = NULL
},
@@ -63,6 +74,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "wal writer",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = WalWriterMain,
.fork_failure = NULL
},
@@ -70,6 +83,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "wal receiver",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = WalReceiverMain,
.fork_failure = NULL
},
@@ -77,6 +92,8 @@ static PgSubprocess process_types[] = {
.name = "avlauncher",
.desc = "autovacuum launcher",
.needs_aux_proc = false,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = AutoVacLauncherMain,
.fork_failure = NULL
},
@@ -84,8 +101,19 @@ static PgSubprocess process_types[] = {
.name = "avworker",
.desc = "autovacuum worker",
.needs_aux_proc = false,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = AutoVacWorkerMain,
.fork_failure = NULL
+ },
+ {
+ .name = "col",
+ .desc = "statistics collector",
+ .needs_aux_proc = false,
+ .needs_shmem = false,
+ .fork_prep = PgstatCollectorPrep,
+ .entrypoint = PgstatCollectorMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1a19921f80..92943f5cff 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1224,10 +1224,8 @@ extern int pgstat_start(void);
extern void pgstat_reset_all(void);
extern void allow_immediate_pgstat_restart(void);
-#ifdef EXEC_BACKEND
+extern int PgstatCollectorPrep(int argc, char *argv[]);
extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
-
/* ----------
* Functions called from backends
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index e11c24cc62..a7ab036b7b 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -25,12 +25,14 @@ typedef enum
WalReceiverType, /* end of Auxiliary Process Forks */
AutoVacuumLauncherType,
AutoVacuumWorkerType,
+ PgstatCollectorType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
-typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
-typedef bool (*SubprocessForkFailure) (int fork_errno);
+typedef int (*SubprocessPrep) (int argc, char *argv[]);
+typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
+typedef bool (*SubprocessForkFailure) (int fork_errno);
/* Current subprocess initializer */
extern void InitMySubprocess(SubprocessType type);
@@ -40,6 +42,8 @@ typedef struct PgSubprocess
const char *name;
const char *desc;
bool needs_aux_proc;
+ bool needs_shmem;
+ SubprocessPrep fork_prep;
SubprocessEntryPoint entrypoint;
SubprocessForkFailure fork_failure;
} PgSubprocess;
--
2.17.0
v2-0006-Add-PgArchiverType-to-subprocess-struct.patch (text/x-diff, 7.5K) download
From 7db474baf6b386261688760c8f32f9bf1c4a549b Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 11 Mar 2020 01:07:05 +0000
Subject: [PATCH v2 06/12] Add PgArchiverType to subprocess struct
---
src/backend/postmaster/pgarch.c | 87 ++++-------------------------
src/backend/postmaster/postmaster.c | 14 ++---
src/backend/postmaster/subprocess.c | 10 ++++
src/include/postmaster/pgarch.h | 9 +--
src/include/postmaster/subprocess.h | 1 +
5 files changed, 26 insertions(+), 95 deletions(-)
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 01ffd6513c..9c1761ad94 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -38,10 +38,10 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "storage/dsm.h"
#include "storage/fd.h"
#include "storage/ipc.h"
@@ -91,11 +91,6 @@ static volatile sig_atomic_t ready_to_stop = false;
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgarch_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgarch_exit(SIGNAL_ARGS);
static void pgarch_waken(SIGNAL_ARGS);
static void pgarch_waken_stop(SIGNAL_ARGS);
@@ -122,10 +117,9 @@ static void pgarch_archiveDone(char *xlog);
* Note: if fail, we will be called again from the postmaster main loop.
*/
int
-pgarch_start(void)
+PgArchiverPrep(int argc, char *argv[])
{
time_t curtime;
- pid_t pgArchPid;
/*
* Do nothing if no archiver needed
@@ -140,86 +134,20 @@ pgarch_start(void)
* the postmaster main loop, we will get another chance later.
*/
curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgarch_start_time) <
+ if ((unsigned int) (curtime - last_pgarch_start_time) >=
(unsigned int) PGARCH_RESTART_INTERVAL)
- return 0;
- last_pgarch_start_time = curtime;
-
-#ifdef EXEC_BACKEND
- switch ((pgArchPid = pgarch_forkexec()))
-#else
- switch ((pgArchPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork archiver: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
+ last_pgarch_start_time = curtime;
- PgArchiverMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgArchPid;
- }
-
- /* shouldn't get here */
return 0;
}
-/* ------------------------------------------------------------
- * Local functions called by archiver follow
- * ------------------------------------------------------------
- */
-
-
-#ifdef EXEC_BACKEND
-
-/*
- * pgarch_forkexec() -
- *
- * Format up the arglist for, then fork and exec, archive process
- */
-static pid_t
-pgarch_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
-
- av[ac++] = "--forkarch";
-
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
/*
* PgArchiverMain
*
* The argc/argv parameters are valid only in EXEC_BACKEND case. However,
* since we don't use 'em, it hardly matters...
*/
-NON_EXEC_STATIC void
+void
PgArchiverMain(int argc, char *argv[])
{
/*
@@ -246,6 +174,11 @@ PgArchiverMain(int argc, char *argv[])
exit(0);
}
+/* ------------------------------------------------------------
+ * Local functions called by archiver follow
+ * ------------------------------------------------------------
+ */
+
/* SIGQUIT signal handler for archiver process */
static void
pgarch_exit(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index e2ad879452..88d010ceb1 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1785,7 +1785,7 @@ ServerLoop(void)
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
/* If we need to signal the autovacuum launcher, do so now */
if (avlauncher_needs_signal)
@@ -3055,7 +3055,7 @@ reaper(SIGNAL_ARGS)
if (!IsBinaryUpgrade && AutoVacuumingActive() && AutoVacPID == 0)
AutoVacPID = StartSubprocess(AutoVacuumLauncherType);
if (PgArchStartupAllowed() && PgArchPID == 0)
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
if (PgStatPID == 0)
PgStatPID = StartSubprocess(PgstatCollectorType);
@@ -3203,7 +3203,7 @@ reaper(SIGNAL_ARGS)
LogChildExit(LOG, _("archiver process"),
pid, exitstatus);
if (PgArchStartupAllowed())
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
continue;
}
@@ -5063,12 +5063,6 @@ SubPostmasterMain(int argc, char *argv[])
StartBackgroundWorker();
}
- if (strcmp(argv[1], "--forkarch") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgArchiverMain(argc, argv); /* does not return */
- }
if (strcmp(argv[1], "--forklog") == 0)
{
/* Do not want to attach to shared memory */
@@ -5197,7 +5191,7 @@ sigusr1_handler(SIGNAL_ARGS)
*/
Assert(PgArchPID == 0);
if (XLogArchivingAlways())
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
/*
* If we aren't planning to enter hot standby mode later, treat
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index ee44ef0789..f9d71a627c 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -15,6 +15,7 @@
#include "pgstat.h"
#include "postmaster/autovacuum.h"
#include "postmaster/bgwriter.h"
+#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
#include "postmaster/startup.h"
#include "postmaster/subprocess.h"
@@ -114,6 +115,15 @@ static PgSubprocess process_types[] = {
.fork_prep = PgstatCollectorPrep,
.entrypoint = PgstatCollectorMain,
.fork_failure = NULL
+ },
+ {
+ .name = "arch",
+ .desc = "archiver",
+ .needs_aux_proc = false,
+ .needs_shmem = false,
+ .fork_prep = PgArchiverPrep,
+ .entrypoint = PgArchiverMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/postmaster/pgarch.h b/src/include/postmaster/pgarch.h
index b3200874ca..ac527212b1 100644
--- a/src/include/postmaster/pgarch.h
+++ b/src/include/postmaster/pgarch.h
@@ -26,14 +26,7 @@
#define MAX_XFN_CHARS 40
#define VALID_XFN_CHARS "0123456789ABCDEF.history.backup.partial"
-/* ----------
- * Functions called from postmaster
- * ----------
- */
-extern int pgarch_start(void);
-
-#ifdef EXEC_BACKEND
+extern int PgArchiverPrep(int argc, char *argv[]);
extern void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
#endif /* _PGARCH_H */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index a7ab036b7b..3f00f904dc 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -26,6 +26,7 @@ typedef enum
AutoVacuumLauncherType,
AutoVacuumWorkerType,
PgstatCollectorType,
+ PgArchiverType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.17.0
v2-0007-Add-SysLoggerType-to-subprocess-struct.patch (text/x-diff, 16.7K) download
From 0da9a9de5ab07e90e59ce496a39bf1e48e354170 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 11 Mar 2020 02:58:08 +0000
Subject: [PATCH v2 07/12] Add SysLoggerType to subprocess struct
This commit introduces two new fields to the subprocess struct:
1) keep_postmaster_memcontext allows subprocesses to specify whether or
not the PostmasterMemoryContext should be kept around after a fork.
2) postmaster_main allows postmaster to gain control to do some
necessary processing after a successful fork.
---
src/backend/postmaster/postmaster.c | 39 +++--
src/backend/postmaster/subprocess.c | 56 +++++--
src/backend/postmaster/syslogger.c | 227 +++++++++++-----------------
src/include/postmaster/subprocess.h | 18 ++-
src/include/postmaster/syslogger.h | 5 +-
5 files changed, 177 insertions(+), 168 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 88d010ceb1..f724727b1e 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1083,7 +1083,7 @@ PostmasterMain(int argc, char *argv[])
/*
* If enabled, start up syslogger collection subprocess
*/
- SysLoggerPID = SysLogger_Start();
+ SysLoggerPID = StartSubprocess(SysLoggerType);
/*
* Reset whereToSendOutput from DestDebug (its starting state) to
@@ -1739,7 +1739,7 @@ ServerLoop(void)
/* If we have lost the log collector, try to start a new one */
if (SysLoggerPID == 0 && Logging_collector)
- SysLoggerPID = SysLogger_Start();
+ SysLoggerPID = StartSubprocess(SysLoggerType);
/*
* If no background writer process is running, and we are not in a
@@ -3228,7 +3228,7 @@ reaper(SIGNAL_ARGS)
{
SysLoggerPID = 0;
/* for safety's sake, launch new logger *first* */
- SysLoggerPID = SysLogger_Start();
+ SysLoggerPID = StartSubprocess(SysLoggerType);
if (!EXIT_STATUS_0(exitstatus))
LogChildExit(LOG, _("system logger process"),
pid, exitstatus);
@@ -5063,12 +5063,6 @@ SubPostmasterMain(int argc, char *argv[])
StartBackgroundWorker();
}
- if (strcmp(argv[1], "--forklog") == 0)
- {
- /* Do not want to attach to shared memory */
-
- SysLoggerMain(argc, argv); /* does not return */
- }
if (MySubprocess->needs_aux_proc)
{
@@ -5446,6 +5440,15 @@ StartSubprocess(SubprocessType type)
snprintf(typebuf, sizeof(typebuf), "-x%d", type);
argv[argc++] = typebuf;
+ /* Prep subprocess for forking */
+ if (MySubprocess->fork_prep)
+ {
+ if (MySubprocess->fork_prep(argc, argv))
+ {
+ return 0;
+ }
+ }
+
argv[argc] = NULL;
Assert(argc < lengthof(argv));
@@ -5461,10 +5464,16 @@ StartSubprocess(SubprocessType type)
/* Close the postmaster's sockets */
ClosePostmasterPorts(false);
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
+ /*
+ * Release postmaster's working memory context if the subprocess doesn't
+ * need it
+ */
+ if (!MySubprocess->keep_postmaster_memcontext)
+ {
+ MemoryContextSwitchTo(TopMemoryContext);
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+ }
/*
* Drop connection to postmaster's shared memory if the subprocess
@@ -5508,6 +5517,10 @@ StartSubprocess(SubprocessType type)
return 0;
}
+ /* Some subprocesses need to do more work in postmaster. */
+ if (MySubprocess->postmaster_main)
+ MySubprocess->postmaster_main(argc, argv);
+
/*
* in parent, successful fork
*/
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index f9d71a627c..f9d07de4ae 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -19,6 +19,7 @@
#include "postmaster/postmaster.h"
#include "postmaster/startup.h"
#include "postmaster/subprocess.h"
+#include "postmaster/syslogger.h"
#include "postmaster/walwriter.h"
#include "replication/walreceiver.h"
@@ -31,99 +32,132 @@ static PgSubprocess process_types[] = {
.desc = "checker",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = CheckerModeMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "boot",
.desc = "bootstrap",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = BootstrapModeMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "boot",
.desc = "startup",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = StartupProcessMain,
- .fork_failure = StartupForkFailure
+ .fork_failure = StartupForkFailure,
+ .postmaster_main = NULL
},
{
.name = "boot",
.desc = "background writer",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = BackgroundWriterMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "boot",
.desc = "checkpointer",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = CheckpointerMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "boot",
.desc = "wal writer",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = WalWriterMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "boot",
.desc = "wal receiver",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = WalReceiverMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "avlauncher",
.desc = "autovacuum launcher",
.needs_aux_proc = false,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = AutoVacLauncherMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "avworker",
.desc = "autovacuum worker",
.needs_aux_proc = false,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = AutoVacWorkerMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "col",
.desc = "statistics collector",
.needs_aux_proc = false,
.needs_shmem = false,
+ .keep_postmaster_memcontext = false,
.fork_prep = PgstatCollectorPrep,
.entrypoint = PgstatCollectorMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "arch",
.desc = "archiver",
.needs_aux_proc = false,
.needs_shmem = false,
+ .keep_postmaster_memcontext = false,
.fork_prep = PgArchiverPrep,
.entrypoint = PgArchiverMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
+ },
+ {
+ .name = "log",
+ .desc = "syslogger",
+ .needs_aux_proc = false,
+ .needs_shmem = false,
+ .keep_postmaster_memcontext = true,
+ .fork_prep = SysLoggerPrep,
+ .entrypoint = SysLoggerMain,
+ .fork_failure = NULL,
+ .postmaster_main = SysLoggerPostmasterMain
}
};
diff --git a/src/backend/postmaster/syslogger.c b/src/backend/postmaster/syslogger.c
index ffcb54968f..e56b0a77dd 100644
--- a/src/backend/postmaster/syslogger.c
+++ b/src/backend/postmaster/syslogger.c
@@ -38,8 +38,8 @@
#include "nodes/pg_list.h"
#include "pgstat.h"
#include "pgtime.h"
-#include "postmaster/fork_process.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "postmaster/syslogger.h"
#include "storage/dsm.h"
#include "storage/fd.h"
@@ -129,10 +129,8 @@ static volatile sig_atomic_t rotation_requested = false;
/* Local subroutines */
#ifdef EXEC_BACKEND
-static pid_t syslogger_forkexec(void);
static void syslogger_parseArgs(int argc, char *argv[]);
#endif
-NON_EXEC_STATIC void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
static void process_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static void flush_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static FILE *logfile_open(const char *filename, const char *mode,
@@ -153,7 +151,7 @@ static void update_metainfo_datafile(void);
* Main entry point for syslogger process
* argc/argv parameters are valid only in EXEC_BACKEND case.
*/
-NON_EXEC_STATIC void
+void
SysLoggerMain(int argc, char *argv[])
{
#ifndef WIN32
@@ -537,13 +535,17 @@ SysLoggerMain(int argc, char *argv[])
* Postmaster subroutine to start a syslogger subprocess.
*/
int
-SysLogger_Start(void)
+SysLoggerPrep(int argc, char *argv[])
{
- pid_t sysloggerPid;
char *filename;
+#ifdef EXEC_BACKEND
+ char filenobuf[32];
+ char csvfilenobuf[32];
+#endif
+ /* Tell postmaster to stop forking */
if (!Logging_collector)
- return 0;
+ return -1;
/*
* If first time through, create the pipe which will receive stderr
@@ -626,129 +628,6 @@ SysLogger_Start(void)
}
#ifdef EXEC_BACKEND
- switch ((sysloggerPid = syslogger_forkexec()))
-#else
- switch ((sysloggerPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork system logger: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(true);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- /* do the work */
- SysLoggerMain(0, NULL);
- break;
-#endif
-
- default:
- /* success, in postmaster */
-
- /* now we redirect stderr, if not done already */
- if (!redirection_done)
- {
-#ifdef WIN32
- int fd;
-#endif
-
- /*
- * Leave a breadcrumb trail when redirecting, in case the user
- * forgets that redirection is active and looks only at the
- * original stderr target file.
- */
- ereport(LOG,
- (errmsg("redirecting log output to logging collector process"),
- errhint("Future log output will appear in directory \"%s\".",
- Log_directory)));
-
-#ifndef WIN32
- fflush(stdout);
- if (dup2(syslogPipe[1], fileno(stdout)) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stdout: %m")));
- fflush(stderr);
- if (dup2(syslogPipe[1], fileno(stderr)) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- /* Now we are done with the write end of the pipe. */
- close(syslogPipe[1]);
- syslogPipe[1] = -1;
-#else
-
- /*
- * open the pipe in binary mode and make sure stderr is binary
- * after it's been dup'ed into, to avoid disturbing the pipe
- * chunking protocol.
- */
- fflush(stderr);
- fd = _open_osfhandle((intptr_t) syslogPipe[1],
- _O_APPEND | _O_BINARY);
- if (dup2(fd, _fileno(stderr)) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- close(fd);
- _setmode(_fileno(stderr), _O_BINARY);
-
- /*
- * Now we are done with the write end of the pipe.
- * CloseHandle() must not be called because the preceding
- * close() closes the underlying handle.
- */
- syslogPipe[1] = 0;
-#endif
- redirection_done = true;
- }
-
- /* postmaster will never write the file(s); close 'em */
- fclose(syslogFile);
- syslogFile = NULL;
- if (csvlogFile != NULL)
- {
- fclose(csvlogFile);
- csvlogFile = NULL;
- }
- return (int) sysloggerPid;
- }
-
- /* we should never reach here */
- return 0;
-}
-
-
-#ifdef EXEC_BACKEND
-
-/*
- * syslogger_forkexec() -
- *
- * Format up the arglist for, then fork and exec, a syslogger process
- */
-static pid_t
-syslogger_forkexec(void)
-{
- char *av[10];
- int ac = 0;
- char filenobuf[32];
- char csvfilenobuf[32];
-
- av[ac++] = "postgres";
- av[ac++] = "--forklog";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
/* static variables (those not passed by write_backend_variables) */
#ifndef WIN32
if (syslogFile != NULL)
@@ -763,7 +642,7 @@ syslogger_forkexec(void)
else
strcpy(filenobuf, "0");
#endif /* WIN32 */
- av[ac++] = filenobuf;
+ argv[argc++] = filenobuf;
#ifndef WIN32
if (csvlogFile != NULL)
@@ -778,14 +657,92 @@ syslogger_forkexec(void)
else
strcpy(csvfilenobuf, "0");
#endif /* WIN32 */
- av[ac++] = csvfilenobuf;
+ argv[argc++] = csvfilenobuf;
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+#endif /* EXEC_BACKEND */
- return postmaster_forkexec(ac, av);
+ return 0;
}
+/*
+ * SysLoggerPostmasterMain
+ *
+ * Control function for subprocess parent (postmaster) after a successful fork.
+ */
+void
+SysLoggerPostmasterMain(int argc, char *argv[])
+{
+ /* success, in postmaster */
+
+ /* now we redirect stderr, if not done already */
+ if (!redirection_done)
+ {
+#ifdef WIN32
+ int fd;
+#endif
+
+ /*
+ * Leave a breadcrumb trail when redirecting, in case the user
+ * forgets that redirection is active and looks only at the
+ * original stderr target file.
+ */
+ ereport(LOG,
+ (errmsg("redirecting log output to logging collector process"),
+ errhint("Future log output will appear in directory \"%s\".",
+ Log_directory)));
+
+#ifndef WIN32
+ fflush(stdout);
+ if (dup2(syslogPipe[1], fileno(stdout)) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stdout: %m")));
+ fflush(stderr);
+ if (dup2(syslogPipe[1], fileno(stderr)) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ /* Now we are done with the write end of the pipe. */
+ close(syslogPipe[1]);
+ syslogPipe[1] = -1;
+#else
+
+ /*
+ * open the pipe in binary mode and make sure stderr is binary
+ * after it's been dup'ed into, to avoid disturbing the pipe
+ * chunking protocol.
+ */
+ fflush(stderr);
+ fd = _open_osfhandle((intptr_t) syslogPipe[1],
+ _O_APPEND | _O_BINARY);
+ if (dup2(fd, _fileno(stderr)) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ close(fd);
+ _setmode(_fileno(stderr), _O_BINARY);
+
+ /*
+ * Now we are done with the write end of the pipe.
+ * CloseHandle() must not be called because the preceding
+ * close() closes the underlying handle.
+ */
+ syslogPipe[1] = 0;
+#endif
+ redirection_done = true;
+ }
+
+ /* postmaster will never write the file(s); close 'em */
+ fclose(syslogFile);
+ syslogFile = NULL;
+ if (csvlogFile != NULL)
+ {
+ fclose(csvlogFile);
+ csvlogFile = NULL;
+ }
+}
+
+#ifdef EXEC_BACKEND
/*
* syslogger_parseArgs() -
*
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 3f00f904dc..d0d98a5d69 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -27,6 +27,7 @@ typedef enum
AutoVacuumWorkerType,
PgstatCollectorType,
PgArchiverType,
+ SysLoggerType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
@@ -34,19 +35,22 @@ typedef enum
typedef int (*SubprocessPrep) (int argc, char *argv[]);
typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
typedef bool (*SubprocessForkFailure) (int fork_errno);
+typedef void (*SubprocessPostmasterMain) (int argc, char *argv[]);
/* Current subprocess initializer */
extern void InitMySubprocess(SubprocessType type);
typedef struct PgSubprocess
{
- const char *name;
- const char *desc;
- bool needs_aux_proc;
- bool needs_shmem;
- SubprocessPrep fork_prep;
- SubprocessEntryPoint entrypoint;
- SubprocessForkFailure fork_failure;
+ const char *name;
+ const char *desc;
+ bool needs_aux_proc;
+ bool needs_shmem;
+ bool keep_postmaster_memcontext;
+ SubprocessPrep fork_prep;
+ SubprocessEntryPoint entrypoint;
+ SubprocessForkFailure fork_failure;
+ SubprocessPostmasterMain postmaster_main;
} PgSubprocess;
extern SubprocessType MySubprocessType;
diff --git a/src/include/postmaster/syslogger.h b/src/include/postmaster/syslogger.h
index f611bd1411..a498c73121 100644
--- a/src/include/postmaster/syslogger.h
+++ b/src/include/postmaster/syslogger.h
@@ -81,9 +81,10 @@ extern int SysLogger_Start(void);
extern void write_syslogger_file(const char *buffer, int count, int dest);
-#ifdef EXEC_BACKEND
+/* Subprocess startup functions */
+extern int SysLoggerPrep(int argc, char *argv[]);
extern void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+extern void SysLoggerPostmasterMain(int argc, char *argv[]);
extern bool CheckLogrotateSignal(void);
extern void RemoveLogrotateSignalFiles(void);
--
2.17.0
v2-0008-Add-BgWorkerType-to-subprocess-struct.patch (text/x-diff, 25.6K) download
From e4e1f4360d44508e9d562cb32543cce0bb424a95 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 11 Mar 2020 03:32:41 +0000
Subject: [PATCH v2 08/12] Add BgWorkerType to subprocess struct
Export some of the backend-specific code and move the some bgworker
startup code out to src/backend/postmaster/bgworker.c.
---
src/backend/postmaster/bgworker.c | 165 ++++++++++-
src/backend/postmaster/postmaster.c | 294 +++-----------------
src/backend/postmaster/subprocess.c | 12 +
src/backend/utils/init/globals.c | 1 +
src/include/miscadmin.h | 1 +
src/include/postmaster/bgworker_internals.h | 8 +-
src/include/postmaster/postmaster.h | 53 ++++
src/include/postmaster/subprocess.h | 1 +
8 files changed, 268 insertions(+), 267 deletions(-)
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index 6c684b5e12..6f62cd4889 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -33,8 +33,10 @@
#include "storage/shmem.h"
#include "tcop/tcopprot.h"
#include "utils/ascii.h"
+#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/timeout.h"
+#include "utils/timestamp.h"
/*
* The postmaster's list of registered background workers, in private memory.
@@ -133,7 +135,55 @@ static const struct
/* Private functions. */
static bgworker_main_type LookupBackgroundWorkerFunction(const char *libraryname, const char *funcname);
+static bool assign_backendlist_entry(RegisteredBgWorker *rw);
+/*
+ * Allocate the Backend struct for a connected background worker, but don't
+ * add it to the list of backends just yet.
+ *
+ * On failure, return false without changing any worker state.
+ *
+ * Some info from the Backend is copied into the passed rw.
+ */
+static bool
+assign_backendlist_entry(RegisteredBgWorker *rw)
+{
+ Backend *bn;
+
+ /*
+ * Compute the cancel key that will be assigned to this session. We
+ * probably don't need cancel keys for background workers, but we'd better
+ * have something random in the field to prevent unfriendly people from
+ * sending cancels to them.
+ */
+ if (!RandomCancelKey(&MyCancelKey))
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random cancel key")));
+ return false;
+ }
+
+ bn = malloc(sizeof(Backend));
+ if (bn == NULL)
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+ return false;
+ }
+
+ bn->cancel_key = MyCancelKey;
+ bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
+ bn->bkend_type = BACKEND_TYPE_BGWORKER;
+ bn->dead_end = false;
+ bn->bgworker_notify = false;
+
+ rw->rw_backend = bn;
+ rw->rw_child_slot = bn->child_slot;
+
+ return true;
+}
/*
* Calculate shared memory needed.
@@ -670,6 +720,55 @@ bgworker_sigusr1_handler(SIGNAL_ARGS)
errno = save_errno;
}
+/*
+ * BackgroundWorkerPrep
+ *
+ * Postmaster subroutine to prepare a bgworker subprocess.
+ *
+ * Returns 0 on success or -1 on failure.
+ *
+ * Note: if fail, we will be called again from the postmaster main loop.
+ */
+int
+BackgroundWorkerPrep(int argc, char *argv[])
+{
+ RegisteredBgWorker *rw = MyRegisteredBgWorker;
+#ifdef EXEC_BACKEND
+ char forkav[MAXPGPATH];
+#endif
+
+ Assert(MyRegisteredBgWorker);
+ Assert(rw->rw_pid == 0);
+
+ /*
+ * Allocate and assign the Backend element. Note we must do this before
+ * forking, so that we can handle out of memory properly.
+ *
+ * Treat failure as though the worker had crashed. That way, the
+ * postmaster will wait a bit before attempting to start it again; if it
+ * tried again right away, most likely it'd find itself repeating the
+ * out-of-memory or fork failure condition.
+ */
+ if (!assign_backendlist_entry(rw))
+ {
+ rw->rw_crashed_at = GetCurrentTimestamp();
+ return -1;
+ }
+
+ ereport(DEBUG1,
+ (errmsg("starting background worker process \"%s\"",
+ rw->rw_worker.bgw_name)));
+
+#ifdef EXEC_BACKEND
+ /* Rewrite bgworker fork parameter with shmem_slot */
+ snprintf(forkav, MAXPGPATH, "%s=%d", argv[1], rw->rw_shmem_slot);
+ argv[1] = forkav;
+#endif
+
+ return 0;
+}
+
+
/*
* Start a new background worker
*
@@ -677,12 +776,28 @@ bgworker_sigusr1_handler(SIGNAL_ARGS)
* postmaster.
*/
void
-StartBackgroundWorker(void)
+BackgroundWorkerMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
- BackgroundWorker *worker = MyBgworkerEntry;
+ BackgroundWorker *worker;
bgworker_main_type entrypt;
+#ifndef EXEC_BACKEND
+ /*
+ * Before blowing away PostmasterContext, save this bgworker's
+ * data where it can find it.
+ */
+ MyBgworkerEntry = (BackgroundWorker *)
+ MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
+ memcpy(MyBgworkerEntry, &(MyRegisteredBgWorker->rw_worker), sizeof(BackgroundWorker));
+
+ /* Now we can release postmaster's working memory context */
+ MemoryContextSwitchTo(TopMemoryContext);
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+#endif
+
+ worker = MyBgworkerEntry;
if (worker == NULL)
elog(FATAL, "unable to find bgworker entry");
@@ -816,6 +931,52 @@ StartBackgroundWorker(void)
proc_exit(0);
}
+/*
+ * BackgroundWorkerForkFailure
+ *
+ * Handle bgworker fork failure gracefully.
+ */
+bool
+BackgroundWorkerForkFailure(int fork_errno)
+{
+ RegisteredBgWorker *rw = MyRegisteredBgWorker;
+
+ errno = fork_errno;
+
+ /* in postmaster, fork failed ... */
+ ereport(LOG,
+ (errmsg("could not fork worker process: %m")));
+
+ /* undo what assign_backendlist_entry did */
+ ReleasePostmasterChildSlot(rw->rw_child_slot);
+ rw->rw_child_slot = 0;
+ free(rw->rw_backend);
+ rw->rw_backend = NULL;
+ /* mark entry as crashed, so we'll try again later */
+ rw->rw_crashed_at = GetCurrentTimestamp();
+
+ /* Don't panic. */
+ return false;
+}
+
+/*
+ * BackgroundWorkerPostmasterMain
+ */
+void
+BackgroundWorkerPostmasterMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ RegisteredBgWorker *rw = MyRegisteredBgWorker;
+
+ rw->rw_pid = MyChildProcPid;
+ rw->rw_backend->pid = rw->rw_pid;
+ ReportBackgroundWorkerPID(rw);
+ /* add new worker to lists of backends */
+ dlist_push_head(&BackendList, &(rw->rw_backend->elem));
+#ifdef EXEC_BACKEND
+ ShmemBackendArrayAdd(rw->rw_backend);
+#endif
+}
+
/*
* Register a new static background worker.
*
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index f724727b1e..8343a81923 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -137,60 +137,14 @@
#include "storage/spin.h"
#endif
-
-/*
- * Possible types of a backend. Beyond being the possible bkend_type values in
- * struct bkend, these are OR-able request flag bits for SignalSomeChildren()
- * and CountChildren().
- */
-#define BACKEND_TYPE_NORMAL 0x0001 /* normal backend */
-#define BACKEND_TYPE_AUTOVAC 0x0002 /* autovacuum worker process */
-#define BACKEND_TYPE_WALSND 0x0004 /* walsender process */
-#define BACKEND_TYPE_BGWORKER 0x0008 /* bgworker process */
-#define BACKEND_TYPE_ALL 0x000F /* OR of all the above */
-
-#define BACKEND_TYPE_WORKER (BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER)
-
-/*
- * List of active backends (or child processes anyway; we don't actually
- * know whether a given child has become a backend or is still in the
- * authorization phase). This is used mainly to keep track of how many
- * children we have and send them appropriate signals when necessary.
- *
- * "Special" children such as the startup, bgwriter and autovacuum launcher
- * tasks are not in this list. Autovacuum worker and walsender are in it.
- * Also, "dead_end" children are in it: these are children launched just for
- * the purpose of sending a friendly rejection message to a would-be client.
- * We must track them because they are attached to shared memory, but we know
- * they will never become live backends. dead_end children are not assigned a
- * PMChildSlot.
- *
- * Background workers are in this list, too.
- */
-typedef struct bkend
-{
- pid_t pid; /* process id of backend */
- int32 cancel_key; /* cancel key for cancels for this backend */
- int child_slot; /* PMChildSlot for this backend, if any */
-
- /*
- * Flavor of backend or auxiliary process. Note that BACKEND_TYPE_WALSND
- * backends initially announce themselves as BACKEND_TYPE_NORMAL, so if
- * bkend_type is normal, you should check for a recent transition.
- */
- int bkend_type;
- bool dead_end; /* is it going to send an error and quit? */
- bool bgworker_notify; /* gets bgworker start/stop notifications */
- dlist_node elem; /* list link in BackendList */
-} Backend;
-
-static dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
+dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
#ifdef EXEC_BACKEND
static Backend *ShmemBackendArray;
#endif
BackgroundWorker *MyBgworkerEntry = NULL;
+RegisteredBgWorker *MyRegisteredBgWorker = NULL;
/* Struct containing postmaster subprocess control info */
SubprocessType MySubprocessType;
@@ -412,7 +366,6 @@ static void processCancelRequest(Port *port, void *pkt);
static int initMasks(fd_set *rmask);
static void report_fork_failure_to_client(Port *port, int errnum);
static CAC_state canAcceptConnections(int backend_type);
-static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
@@ -420,7 +373,6 @@ static void TerminateChildren(int signal);
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
-static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
static pid_t StartSubprocess(SubprocessType type);
@@ -539,10 +491,10 @@ static bool save_backend_variables(BackendParameters *param, Port *port,
HANDLE childProcess, pid_t childPid);
#endif
-static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
+#define BGWORKER_LEN 15
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -4878,6 +4830,7 @@ void
SubPostmasterMain(int argc, char *argv[])
{
Port port;
+ int shmem_slot;
/* In EXEC_BACKEND case we will not have inherited these settings */
IsPostmasterEnvironment = true;
@@ -5041,59 +4994,48 @@ SubPostmasterMain(int argc, char *argv[])
/* And run the backend */
BackendRun(&port); /* does not return */
}
- if (strncmp(argv[1], "--forkbgworker=", 15) == 0)
- {
- int shmem_slot;
-
- /* do this as early as possible; in particular, before InitProcess() */
- IsBackgroundWorker = true;
+ if (MySubprocess->needs_aux_proc)
+ {
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
/* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
+ InitAuxiliaryProcess();
/* Attach process to shared data structures */
CreateSharedMemoryAndSemaphores();
- /* Fetch MyBgworkerEntry from shared memory */
- shmem_slot = atoi(argv[1] + 15);
- MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
+ AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
+ }
- StartBackgroundWorker();
+ if (MySubprocessType == BgWorkerType)
+ {
+ /* do this as early as possible; in particular, before InitProcess() */
+ IsBackgroundWorker = true;
}
- if (MySubprocess->needs_aux_proc)
+ if (MySubprocess->needs_shmem)
{
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
/* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitAuxiliaryProcess();
+ InitProcess();
/* Attach process to shared data structures */
CreateSharedMemoryAndSemaphores();
-
- AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
}
- else
- {
- if (MySubprocess->needs_shmem)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
- }
-
- MySubprocess->entrypoint(argc - 2, argv + 2);
+ if (MySubprocessType == BgWorkerType)
+ {
+ /* Fetch MyBgworkerEntry from shared memory */
+ shmem_slot = atoi(argv[1] + 15);
+ MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
}
+ MySubprocess->entrypoint(argc - 2, argv + 2);
+
abort(); /* shouldn't get here */
}
#endif /* EXEC_BACKEND */
@@ -5356,7 +5298,7 @@ StartupPacketTimeoutHandler(void)
/*
* Generate a random cancel key.
*/
-static bool
+bool
RandomCancelKey(int32 *cancel_key)
{
return pg_strong_random(cancel_key, sizeof(int32));
@@ -5454,15 +5396,17 @@ StartSubprocess(SubprocessType type)
#ifdef EXEC_BACKEND
pid = postmaster_forkexec(argc, argv);
+ MyChildProcPid = pid;
#else /* !EXEC_BACKEND */
pid = fork_process();
+ MyChildProcPid = pid;
if (pid == 0) /* child */
{
InitPostmasterChild();
/* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
+ ClosePostmasterPorts(MySubprocessType == SysLoggerType);
/*
* Release postmaster's working memory context if the subprocess doesn't
@@ -5512,6 +5456,8 @@ StartSubprocess(SubprocessType type)
{
if (MySubprocess->fork_failure(save_errno))
ExitPostmaster(1);
+ else
+ return -1;
}
return 0;
@@ -5754,124 +5700,6 @@ BackgroundWorkerUnblockSignals(void)
PG_SETMASK(&UnBlockSig);
}
-#ifdef EXEC_BACKEND
-static pid_t
-bgworker_forkexec(int shmem_slot)
-{
- char *av[10];
- int ac = 0;
- char forkav[MAXPGPATH];
-
- snprintf(forkav, MAXPGPATH, "--forkbgworker=%d", shmem_slot);
-
- av[ac++] = "postgres";
- av[ac++] = forkav;
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif
-
-/*
- * Start a new bgworker.
- * Starting time conditions must have been checked already.
- *
- * Returns true on success, false on failure.
- * In either case, update the RegisteredBgWorker's state appropriately.
- *
- * This code is heavily based on autovacuum.c, q.v.
- */
-static bool
-do_start_bgworker(RegisteredBgWorker *rw)
-{
- pid_t worker_pid;
-
- Assert(rw->rw_pid == 0);
-
- /*
- * Allocate and assign the Backend element. Note we must do this before
- * forking, so that we can handle failures (out of memory or child-process
- * slots) cleanly.
- *
- * Treat failure as though the worker had crashed. That way, the
- * postmaster will wait a bit before attempting to start it again; if we
- * tried again right away, most likely we'd find ourselves hitting the
- * same resource-exhaustion condition.
- */
- if (!assign_backendlist_entry(rw))
- {
- rw->rw_crashed_at = GetCurrentTimestamp();
- return false;
- }
-
- ereport(DEBUG1,
- (errmsg("starting background worker process \"%s\"",
- rw->rw_worker.bgw_name)));
-
-#ifdef EXEC_BACKEND
- switch ((worker_pid = bgworker_forkexec(rw->rw_shmem_slot)))
-#else
- switch ((worker_pid = fork_process()))
-#endif
- {
- case -1:
- /* in postmaster, fork failed ... */
- ereport(LOG,
- (errmsg("could not fork worker process: %m")));
- /* undo what assign_backendlist_entry did */
- ReleasePostmasterChildSlot(rw->rw_child_slot);
- rw->rw_child_slot = 0;
- free(rw->rw_backend);
- rw->rw_backend = NULL;
- /* mark entry as crashed, so we'll try again later */
- rw->rw_crashed_at = GetCurrentTimestamp();
- break;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /*
- * Before blowing away PostmasterContext, save this bgworker's
- * data where it can find it.
- */
- MyBgworkerEntry = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(MyBgworkerEntry, &rw->rw_worker, sizeof(BackgroundWorker));
-
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
-
- StartBackgroundWorker();
-
- exit(1); /* should not get here */
- break;
-#endif
- default:
- /* in postmaster, fork successful ... */
- rw->rw_pid = worker_pid;
- rw->rw_backend->pid = rw->rw_pid;
- ReportBackgroundWorkerPID(rw);
- /* add new worker to lists of backends */
- dlist_push_head(&BackendList, &rw->rw_backend->elem);
-#ifdef EXEC_BACKEND
- ShmemBackendArrayAdd(rw->rw_backend);
-#endif
- return true;
- }
-
- return false;
-}
-
/*
* Does the current postmaster state require starting a worker with the
* specified start_time?
@@ -5912,67 +5740,6 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
return false;
}
-/*
- * Allocate the Backend struct for a connected background worker, but don't
- * add it to the list of backends just yet.
- *
- * On failure, return false without changing any worker state.
- *
- * Some info from the Backend is copied into the passed rw.
- */
-static bool
-assign_backendlist_entry(RegisteredBgWorker *rw)
-{
- Backend *bn;
-
- /*
- * Check that database state allows another connection. Currently the
- * only possible failure is CAC_TOOMANY, so we just log an error message
- * based on that rather than checking the error code precisely.
- */
- if (canAcceptConnections(BACKEND_TYPE_BGWORKER) != CAC_OK)
- {
- ereport(LOG,
- (errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("no slot available for new worker process")));
- return false;
- }
-
- /*
- * Compute the cancel key that will be assigned to this session. We
- * probably don't need cancel keys for background workers, but we'd better
- * have something random in the field to prevent unfriendly people from
- * sending cancels to them.
- */
- if (!RandomCancelKey(&MyCancelKey))
- {
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("could not generate random cancel key")));
- return false;
- }
-
- bn = malloc(sizeof(Backend));
- if (bn == NULL)
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- return false;
- }
-
- bn->cancel_key = MyCancelKey;
- bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
- bn->bkend_type = BACKEND_TYPE_BGWORKER;
- bn->dead_end = false;
- bn->bgworker_notify = false;
-
- rw->rw_backend = bn;
- rw->rw_child_slot = bn->child_slot;
-
- return true;
-}
-
/*
* If the time is right, start background worker(s).
*
@@ -6077,7 +5844,8 @@ maybe_start_bgworkers(void)
* crashed, but there's no need because the next run of this
* function will do that.
*/
- if (!do_start_bgworker(rw))
+ MyRegisteredBgWorker = rw;
+ if (StartSubprocess(BgWorkerType) <= 0)
{
StartWorkerNeeded = true;
return;
@@ -6488,7 +6256,7 @@ ShmemBackendArrayAllocation(void)
memset(ShmemBackendArray, 0, size);
}
-static void
+void
ShmemBackendArrayAdd(Backend *bn)
{
/* The array slot corresponding to my PMChildSlot should be free */
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index f9d07de4ae..d4df791aeb 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -14,6 +14,7 @@
#include "bootstrap/bootstrap.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
+#include "postmaster/bgworker_internals.h"
#include "postmaster/bgwriter.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
@@ -158,6 +159,17 @@ static PgSubprocess process_types[] = {
.entrypoint = SysLoggerMain,
.fork_failure = NULL,
.postmaster_main = SysLoggerPostmasterMain
+ },
+ {
+ .name = "bgworker",
+ .desc = "background worker",
+ .needs_aux_proc = false,
+ .needs_shmem = true,
+ .keep_postmaster_memcontext = true,
+ .fork_prep = BackgroundWorkerPrep,
+ .entrypoint = BackgroundWorkerMain,
+ .fork_failure = BackgroundWorkerForkFailure,
+ .postmaster_main = BackgroundWorkerPostmasterMain
}
};
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index eb19644419..b31ce82f75 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -38,6 +38,7 @@ volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
int MyProcPid;
+int MyChildProcPid;
pg_time_t MyStartTime;
TimestampTz MyStartTimestamp;
struct Port *MyProcPort;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 14fa127ab1..541ca5145d 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -163,6 +163,7 @@ extern PGDLLIMPORT int max_worker_processes;
extern PGDLLIMPORT int max_parallel_workers;
extern PGDLLIMPORT int MyProcPid;
+extern PGDLLIMPORT int MyChildProcPid;
extern PGDLLIMPORT pg_time_t MyStartTime;
extern PGDLLIMPORT TimestampTz MyStartTimestamp;
extern PGDLLIMPORT struct Port *MyProcPort;
diff --git a/src/include/postmaster/bgworker_internals.h b/src/include/postmaster/bgworker_internals.h
index f7e24664d5..874d5a203f 100644
--- a/src/include/postmaster/bgworker_internals.h
+++ b/src/include/postmaster/bgworker_internals.h
@@ -43,6 +43,7 @@ typedef struct RegisteredBgWorker
} RegisteredBgWorker;
extern slist_head BackgroundWorkerList;
+extern RegisteredBgWorker *MyRegisteredBgWorker;
extern Size BackgroundWorkerShmemSize(void);
extern void BackgroundWorkerShmemInit(void);
@@ -53,8 +54,11 @@ extern void ReportBackgroundWorkerExit(slist_mutable_iter *cur);
extern void BackgroundWorkerStopNotifications(pid_t pid);
extern void ResetBackgroundWorkerCrashTimes(void);
-/* Function to start a background worker, called from postmaster.c */
-extern void StartBackgroundWorker(void) pg_attribute_noreturn();
+/* BgWorkerType subprocess */
+extern int BackgroundWorkerPrep(int argc, char *argv[]);
+extern void BackgroundWorkerMain(int argc, char *argv[]);
+extern bool BackgroundWorkerForkFailure(int fork_errno);
+extern void BackgroundWorkerPostmasterMain(int argc, char *argv[]);
#ifdef EXEC_BACKEND
extern BackgroundWorker *BackgroundWorkerEntry(int slotno);
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index babc87dfc9..b1411c1ed2 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -13,6 +13,41 @@
#ifndef _POSTMASTER_H
#define _POSTMASTER_H
+#include "lib/ilist.h"
+
+/*
+ * List of active backends (or child processes anyway; we don't actually
+ * know whether a given child has become a backend or is still in the
+ * authorization phase). This is used mainly to keep track of how many
+ * children we have and send them appropriate signals when necessary.
+ *
+ * "Special" children such as the startup, bgwriter and autovacuum launcher
+ * tasks are not in this list. Autovacuum worker and walsender are in it.
+ * Also, "dead_end" children are in it: these are children launched just for
+ * the purpose of sending a friendly rejection message to a would-be client.
+ * We must track them because they are attached to shared memory, but we know
+ * they will never become live backends. dead_end children are not assigned a
+ * PMChildSlot.
+ *
+ * Background workers are in this list, too.
+ */
+typedef struct bkend
+{
+ pid_t pid; /* process id of backend */
+ int32 cancel_key; /* cancel key for cancels for this backend */
+ int child_slot; /* PMChildSlot for this backend, if any */
+
+ /*
+ * Flavor of backend or auxiliary process. Note that BACKEND_TYPE_WALSND
+ * backends initially announce themselves as BACKEND_TYPE_NORMAL, so if
+ * bkend_type is normal, you should check for a recent transition.
+ */
+ int bkend_type;
+ bool dead_end; /* is it going to send an error and quit? */
+ bool bgworker_notify; /* gets bgworker start/stop notifications */
+ dlist_node elem; /* list link in BackendList */
+} Backend;
+
/* GUC options */
extern bool EnableSSL;
extern int ReservedBackends;
@@ -54,14 +89,19 @@ extern int MaxLivePostmasterChildren(void);
extern bool PostmasterMarkPIDForWorkerNotify(int);
+extern bool RandomCancelKey(int32 *cancel_key);
+
#ifdef EXEC_BACKEND
extern pid_t postmaster_forkexec(int argc, char *argv[]);
extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
extern Size ShmemBackendArraySize(void);
extern void ShmemBackendArrayAllocation(void);
+extern void ShmemBackendArrayAdd(Backend *bn);
#endif
+extern PGDLLIMPORT dlist_head BackendList;
+
/*
* Note: MAX_BACKENDS is limited to 2^18-1 because that's the width reserved
* for buffer references in buf_internals.h. This limitation could be lifted
@@ -74,4 +114,17 @@ extern void ShmemBackendArrayAllocation(void);
*/
#define MAX_BACKENDS 0x3FFFF
+/*
+ * Possible types of a backend. Beyond being the possible bkend_type values in
+ * struct bkend, these are OR-able request flag bits for SignalSomeChildren()
+ * and CountChildren().
+ */
+#define BACKEND_TYPE_NORMAL 0x0001 /* normal backend */
+#define BACKEND_TYPE_AUTOVAC 0x0002 /* autovacuum worker process */
+#define BACKEND_TYPE_WALSND 0x0004 /* walsender process */
+#define BACKEND_TYPE_BGWORKER 0x0008 /* bgworker process */
+#define BACKEND_TYPE_ALL 0x000F /* OR of all the above */
+
+#define BACKEND_TYPE_WORKER (BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER)
+
#endif /* _POSTMASTER_H */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index d0d98a5d69..706d60e4d6 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -28,6 +28,7 @@ typedef enum
PgstatCollectorType,
PgArchiverType,
SysLoggerType,
+ BgWorkerType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.17.0
v2-0009-Add-Backends-to-subprocess-struct.patch (text/x-diff, 16.0K) download
From aa2c3209969fe4b6d193b37d92959dfd3809c804 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Mon, 16 Mar 2020 23:05:33 +0000
Subject: [PATCH v2 09/12] Add Backends to subprocess struct
---
src/backend/postmaster/postmaster.c | 407 +++++++++++++++-------------
src/backend/postmaster/subprocess.c | 11 +
src/include/postmaster/postmaster.h | 6 +
src/include/postmaster/subprocess.h | 1 +
4 files changed, 235 insertions(+), 190 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 8343a81923..f979d505df 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -359,7 +359,6 @@ static void BackendInitialize(Port *port);
static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
-static int BackendStartup(Port *port);
static int ProcessStartupPacket(Port *port, bool secure_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
@@ -409,7 +408,6 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(Port *port);
static pid_t internal_forkexec(int argc, char *argv[], Port *port);
/* Type for a socket that can be inherited to a client process */
@@ -512,6 +510,8 @@ int postmaster_alive_fds[2] = {-1, -1};
HANDLE PostmasterHandle;
#endif
+static Port *ConnProcPort = NULL;
+
/*
* Postmaster main entry point
*/
@@ -1671,19 +1671,18 @@ ServerLoop(void)
break;
if (FD_ISSET(ListenSocket[i], &rmask))
{
- Port *port;
+ ConnProcPort = ConnCreate(ListenSocket[i]);
- port = ConnCreate(ListenSocket[i]);
- if (port)
+ if (ConnProcPort)
{
- BackendStartup(port);
+ StartSubprocess(ClientBackendType);
/*
* We no longer need the open socket or port structure
* in this process
*/
- StreamClose(port->sock);
- ConnFree(port);
+ StreamClose(ConnProcPort->sock);
+ ConnFree(ConnProcPort);
}
}
}
@@ -3215,6 +3214,174 @@ reaper(SIGNAL_ARGS)
errno = save_errno;
}
+/*
+ * BackendPrep
+ *
+ * Do all of the pre-fork setup for a new Backend.
+*/
+int
+BackendPrep(int argc, char *argv[])
+{
+ /*
+ * Create backend data structure. Better before the fork() so we can
+ * handle failure cleanly.
+ */
+ Backend *bn;
+
+ bn = (Backend *) malloc(sizeof(Backend));
+ if (!bn)
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+ }
+
+ /*
+ * Compute the cancel key that will be assigned to this backend. The
+ * backend will have its own copy in the forked-off process' value of
+ * MyCancelKey, so that it can transmit the key to the frontend.
+ */
+ if (!RandomCancelKey(&MyCancelKey))
+ {
+ free(bn);
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random cancel key")));
+ }
+
+ bn->cancel_key = MyCancelKey;
+
+ /* Pass down canAcceptConnections state */
+ ConnProcPort->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
+ bn->dead_end = (ConnProcPort->canAcceptConnections != CAC_OK &&
+ ConnProcPort->canAcceptConnections != CAC_WAITBACKUP);
+
+ /*
+ * Unless it's a dead_end child, assign it a child slot number
+ */
+ if (!bn->dead_end)
+ bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
+ else
+ bn->child_slot = 0;
+
+ /* Hasn't asked to be notified about any bgworkers yet */
+ bn->bgworker_notify = false;
+
+ /*
+ * Push the Backend to the stack as-is so it can be retreived by the parent
+ * on the other side of the fork/exec. They will modify this entry in place.
+ */
+ dlist_push_head(&BackendList, &bn->elem);
+
+ return 0;
+}
+
+/*
+ * BackendMain
+ *
+ * Child code when forking a Backend.
+ */
+void BackendMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ /*
+ * Perform additional initialization and collect startup packet.
+ *
+ * We want to do this before InitProcess() for a couple of reasons: 1.
+ * so that we aren't eating up a PGPROC slot while waiting on the
+ * client. 2. so that if InitProcess() fails due to being out of
+ * PGPROC slots, we have already initialized libpq and are able to
+ * report the error to the client.
+ */
+#ifndef EXEC_BACKEND
+ BackendInitialize(ConnProcPort);
+#endif
+
+ /* And run the backend */
+ BackendRun(ConnProcPort); /* does not return */
+}
+
+/*
+ * BackendPostmasterMain
+ *
+ * Parent code when forking a Backend.
+ */
+void BackendPostmasterMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ dlist_mutable_iter iter;
+
+ /*
+ * Find our backend node by popping the element with a matching child slot
+ * off the list. This was pushed to the stack (incomplete) by BackendPrep.
+ * We need to fix it up and push back.
+ */
+ dlist_foreach_modify(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ /* MyPMChildSlot is guaranteed to match the child MyPMChildSlot */
+ if (bp->child_slot == MyPMChildSlot)
+ {
+ /*
+ * Everything's been successful, it's safe to add this backend to our list
+ * of backends.
+ */
+ bp->pid = MyChildProcPid;
+ bp->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
+
+#ifdef EXEC_BACKEND
+ if (!bp->dead_end)
+ ShmemBackendArrayAdd(bp);
+#endif
+ break; /* There is only one entry and we've found it */
+ }
+ }
+
+ /* in parent, successful fork */
+ ereport(DEBUG2,
+ (errmsg_internal("forked new backend, pid=%d socket=%d",
+ (int) MyChildProcPid, (int) MyProcPort->sock)));
+}
+
+/*
+ * BackendForkFailure
+ *
+ * Backend cleanup in case a failure occurs forking a new Backend.
+ */
+bool
+BackendForkFailure(int child_errno)
+{
+ dlist_mutable_iter iter;
+
+ dlist_foreach_modify(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->child_slot == MyPMChildSlot)
+ {
+ if (!bp->dead_end)
+ {
+ if (!ReleasePostmasterChildSlot(bp->child_slot))
+ {
+ /* Could not release child slot! Panic! */
+ return true;
+ }
+#ifdef EXEC_BACKEND
+ ShmemBackendArrayRemove(bp);
+#endif
+ }
+ dlist_delete(iter.cur);
+ free(bp);
+ break;
+ }
+ }
+
+ report_fork_failure_to_client(MyProcPort, child_errno);
+
+ /* Don't panic */
+ return false;
+}
+
+
/*
* Scan the bgworkers list and see if the given PID (which has just stopped
* or crashed) is in it. Handle its shutdown if so, and return true. If not a
@@ -4075,122 +4242,6 @@ TerminateChildren(int signal)
signal_child(PgStatPID, signal);
}
-/*
- * BackendStartup -- start backend process
- *
- * returns: STATUS_ERROR if the fork failed, STATUS_OK otherwise.
- *
- * Note: if you change this code, also consider StartAutovacuumWorker.
- */
-static int
-BackendStartup(Port *port)
-{
- Backend *bn; /* for backend cleanup */
- pid_t pid;
-
- /*
- * Create backend data structure. Better before the fork() so we can
- * handle failure cleanly.
- */
- bn = (Backend *) malloc(sizeof(Backend));
- if (!bn)
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- return STATUS_ERROR;
- }
-
- /*
- * Compute the cancel key that will be assigned to this backend. The
- * backend will have its own copy in the forked-off process' value of
- * MyCancelKey, so that it can transmit the key to the frontend.
- */
- if (!RandomCancelKey(&MyCancelKey))
- {
- free(bn);
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("could not generate random cancel key")));
- return STATUS_ERROR;
- }
-
- bn->cancel_key = MyCancelKey;
-
- /* Pass down canAcceptConnections state */
- port->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
- bn->dead_end = (port->canAcceptConnections != CAC_OK &&
- port->canAcceptConnections != CAC_WAITBACKUP);
-
- /*
- * Unless it's a dead_end child, assign it a child slot number
- */
- if (!bn->dead_end)
- bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
- else
- bn->child_slot = 0;
-
- /* Hasn't asked to be notified about any bgworkers yet */
- bn->bgworker_notify = false;
-
-#ifdef EXEC_BACKEND
- pid = backend_forkexec(port);
-#else /* !EXEC_BACKEND */
- pid = fork_process();
- if (pid == 0) /* child */
- {
- free(bn);
-
- /* Detangle from postmaster */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Perform additional initialization and collect startup packet */
- BackendInitialize(port);
-
- /* And run the backend */
- BackendRun(port);
- }
-#endif /* EXEC_BACKEND */
-
- if (pid < 0)
- {
- /* in parent, fork failed */
- int save_errno = errno;
-
- if (!bn->dead_end)
- (void) ReleasePostmasterChildSlot(bn->child_slot);
- free(bn);
- errno = save_errno;
- ereport(LOG,
- (errmsg("could not fork new process for connection: %m")));
- report_fork_failure_to_client(port, save_errno);
- return STATUS_ERROR;
- }
-
- /* in parent, successful fork */
- ereport(DEBUG2,
- (errmsg_internal("forked new backend, pid=%d socket=%d",
- (int) pid, (int) port->sock)));
-
- /*
- * Everything's been successful, it's safe to add this backend to our list
- * of backends.
- */
- bn->pid = pid;
- bn->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
- dlist_push_head(&BackendList, &bn->elem);
-
-#ifdef EXEC_BACKEND
- if (!bn->dead_end)
- ShmemBackendArrayAdd(bn);
-#endif
-
- return STATUS_OK;
-}
-
/*
* Try to report backend fork() failure to client before we close the
* connection. Since we do not care to risk blocking the postmaster on
@@ -4483,33 +4534,12 @@ postmaster_forkexec(int argc, char *argv[])
Port port;
/* This entry point passes dummy values for the Port variables */
- memset(&port, 0, sizeof(port));
- return internal_forkexec(argc, argv, &port);
-}
-
-/*
- * backend_forkexec -- fork/exec off a backend process
- *
- * Some operating systems (WIN32) don't have fork() so we have to simulate
- * it by storing parameters that need to be passed to the child and
- * then create a new child process.
- *
- * returns the pid of the fork/exec'd process, or -1 on failure
- */
-static pid_t
-backend_forkexec(Port *port)
-{
- char *av[4];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkbackend";
- av[ac++] = NULL; /* filled in by internal_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ if (!ConnProcPort)
+ memset(&port, 0, sizeof(port));
+ else
+ port = *ConnProcPort;
- return internal_forkexec(ac, av, port);
+ return internal_forkexec(argc, argv, &port);
}
#ifndef WIN32
@@ -4849,6 +4879,7 @@ SubPostmasterMain(int argc, char *argv[])
/* Read in the variables file */
memset(&port, 0, sizeof(Port));
read_backend_variables(argv[2], &port);
+ ConnProcPort = &port;
/* Close the postmaster's sockets (as soon as we know them) */
ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
@@ -4890,11 +4921,7 @@ SubPostmasterMain(int argc, char *argv[])
* sometimes impossible to attach to shared memory at the desired address.
* Return the setting to its old value (usually '1' or '2') when finished.
*/
- if (strcmp(argv[1], "--forkbackend") == 0 ||
- strcmp(argv[1], "--forkavlauncher") == 0 ||
- strcmp(argv[1], "--forkavworker") == 0 ||
- strcmp(argv[1], "--forkboot") == 0 ||
- strncmp(argv[1], "--forkbgworker=", 15) == 0)
+ if (MySubprocess->needs_shmem)
PGSharedMemoryReAttach();
else
PGSharedMemoryNoReAttach();
@@ -4943,11 +4970,23 @@ SubPostmasterMain(int argc, char *argv[])
*/
process_shared_preload_libraries();
- /* Run backend or appropriate child */
- if (strcmp(argv[1], "--forkbackend") == 0)
+ if (MySubprocess->needs_aux_proc)
{
- Assert(argc == 3); /* shouldn't be any more args */
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
+ /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ InitAuxiliaryProcess();
+
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores();
+
+ AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
+ }
+
+ /* Run backend or appropriate child */
+ if (MySubprocessType == ClientBackendType)
+ {
/*
* Need to reinitialize the SSL library in the backend, since the
* context structures contain function pointers and cannot be passed
@@ -4981,32 +5020,6 @@ SubPostmasterMain(int argc, char *argv[])
* report the error to the client.
*/
BackendInitialize(&port);
-
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- /* And run the backend */
- BackendRun(&port); /* does not return */
- }
-
- if (MySubprocess->needs_aux_proc)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitAuxiliaryProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
}
if (MySubprocessType == BgWorkerType)
@@ -5027,15 +5040,29 @@ SubPostmasterMain(int argc, char *argv[])
CreateSharedMemoryAndSemaphores();
}
- if (MySubprocessType == BgWorkerType)
+ switch (MySubprocessType)
{
- /* Fetch MyBgworkerEntry from shared memory */
- shmem_slot = atoi(argv[1] + 15);
- MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
+ case AutoVacuumLauncherType:
+ case AutoVacuumWorkerType:
+ MySubprocess->entrypoint(argc - 2, argv + 2);
+ break;
+ case BgWorkerType:
+ /* Fetch MyBgworkerEntry from shared memory */
+ shmem_slot = atoi(argv[1] + 15);
+ MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
+ /* fallthrough */
+ case PgArchiverType:
+ case PgstatCollectorType:
+ case SysLoggerType:
+ case ClientBackendType:
+ MySubprocess->entrypoint(argc, argv);
+ break;
+ default:
+ ereport(LOG,
+ (errmsg("could not start unknown process type (%d) under postmaster",
+ MySubprocessType)));
}
- MySubprocess->entrypoint(argc - 2, argv + 2);
-
abort(); /* shouldn't get here */
}
#endif /* EXEC_BACKEND */
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index d4df791aeb..9d7ce86918 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -170,6 +170,17 @@ static PgSubprocess process_types[] = {
.entrypoint = BackgroundWorkerMain,
.fork_failure = BackgroundWorkerForkFailure,
.postmaster_main = BackgroundWorkerPostmasterMain
+ },
+ {
+ .name = "backend",
+ .desc = "backend",
+ .needs_aux_proc = false,
+ .needs_shmem = true,
+ .keep_postmaster_memcontext = true,
+ .fork_prep = BackendPrep,
+ .entrypoint = BackendMain,
+ .fork_failure = BackendForkFailure,
+ .postmaster_main = BackendPostmasterMain
}
};
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index b1411c1ed2..6bc0af7c2b 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -91,6 +91,12 @@ extern bool PostmasterMarkPIDForWorkerNotify(int);
extern bool RandomCancelKey(int32 *cancel_key);
+extern int BackendPrep(int argc, char *argv[]);
+extern void BackendMain(int argc, char *argv[]);
+extern void BackendPostmasterMain(int argc, char *argv[]);
+extern bool BackendForkFailure(int child_errno);
+extern void BackgroundWorkerMain(int argc, char *argv[]);
+
#ifdef EXEC_BACKEND
extern pid_t postmaster_forkexec(int argc, char *argv[]);
extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 706d60e4d6..06e67efecc 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -29,6 +29,7 @@ typedef enum
PgArchiverType,
SysLoggerType,
BgWorkerType,
+ ClientBackendType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.17.0
v2-0010-Add-WalSenderType-to-subprocess-struct.patch (text/x-diff, 1.4K) download
From f0f10fce0d28847d33a1145ab93b478fac2308fa Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 17 Mar 2020 18:39:12 +0000
Subject: [PATCH v2 10/12] Add WalSenderType to subprocess struct
---
src/backend/postmaster/subprocess.c | 11 +++++++++++
src/include/postmaster/subprocess.h | 1 +
2 files changed, 12 insertions(+)
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 9d7ce86918..8cc88792ea 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -181,6 +181,17 @@ static PgSubprocess process_types[] = {
.entrypoint = BackendMain,
.fork_failure = BackendForkFailure,
.postmaster_main = BackendPostmasterMain
+ },
+ { /* Wal Sender placeholder */
+ .name = "walsender",
+ .desc = "wal sender",
+ .needs_aux_proc = false,
+ .needs_shmem = false,
+ .keep_postmaster_memcontext = false,
+ .fork_prep = NULL,
+ .entrypoint = NULL,
+ .fork_failure = NULL,
+ .postmaster_main = NULL
}
};
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 06e67efecc..867e1cb499 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -31,6 +31,7 @@ typedef enum
BgWorkerType,
ClientBackendType,
+ WalSenderType, /* placeholder for wal sender so it can be identified in pgstat */
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.17.0
v2-0011-Get-rid-of-st_beckendType-and-use-MySubprocess.patch (text/x-diff, 4.0K) download
From b6108d834dec272440f4bc69ad8a5f170e9054b1 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 17 Mar 2020 18:39:30 +0000
Subject: [PATCH v2 11/12] Get rid of st_beckendType and use MySubprocess
---
src/backend/postmaster/pgstat.c | 7 +++----
src/backend/postmaster/postmaster.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 10 ++++++----
src/include/pgstat.h | 3 ---
4 files changed, 10 insertions(+), 11 deletions(-)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 6d4f2ace56..6bb063420f 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -2820,7 +2820,6 @@ pgstat_bestart(void)
* out-of-line data. Those have to be handled separately, below.
*/
lbeentry.st_procpid = MyProcPid;
- lbeentry.st_backendType = MyBackendType;
lbeentry.st_proc_start_timestamp = MyStartTimestamp;
lbeentry.st_activity_start_timestamp = 0;
lbeentry.st_state_start_timestamp = 0;
@@ -2828,9 +2827,9 @@ pgstat_bestart(void)
lbeentry.st_databaseid = MyDatabaseId;
/* We have userid for client-backends, wal-sender and bgworker processes */
- if (lbeentry.st_backendType == B_BACKEND
- || lbeentry.st_backendType == B_WAL_SENDER
- || lbeentry.st_backendType == B_BG_WORKER)
+ if (MySubprocessType == ClientBackendType
+ || MySubprocessType == WalSenderType
+ || MySubprocessType == BgWorkerType)
lbeentry.st_userid = GetSessionUserId();
else
lbeentry.st_userid = InvalidOid;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index f979d505df..65305a18ee 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -2085,6 +2085,7 @@ retry1:
*/
if (strcmp(valptr, "database") == 0)
{
+ InitMySubprocess(WalSenderType);
am_walsender = true;
am_db_walsender = true;
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index cea01534a5..599e9219ec 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -24,6 +24,7 @@
#include "pgstat.h"
#include "postmaster/bgworker_internals.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "storage/proc.h"
#include "storage/procarray.h"
#include "utils/acl.h"
@@ -693,7 +694,7 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
proc = BackendPidGetProc(beentry->st_procpid);
- if (proc == NULL && (beentry->st_backendType != B_BACKEND))
+ if (proc == NULL && (MySubprocessType != ClientBackendType))
{
/*
* For an auxiliary process, retrieve process info from
@@ -741,7 +742,7 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
* date.
*/
if (beentry->st_xact_start_timestamp != 0 &&
- beentry->st_backendType != B_WAL_SENDER)
+ MySubprocessType != WalSenderType)
values[8] = TimestampTzGetDatum(beentry->st_xact_start_timestamp);
else
nulls[8] = true;
@@ -829,7 +830,7 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
}
}
/* Add backend type */
- if (beentry->st_backendType == B_BG_WORKER)
+ if (MySubprocessType == BgWorkerType)
{
const char *bgw_type;
@@ -841,7 +842,8 @@ pg_stat_get_activity(PG_FUNCTION_ARGS)
}
else
values[17] =
- CStringGetTextDatum(GetBackendTypeDesc(beentry->st_backendType));
+ // CStringGetTextDatum(GetBackendTypeDesc(beentry->st_backendType));
+ CStringGetTextDatum(MySubprocess->desc);
/* SSL information */
if (beentry->st_ssl)
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 92943f5cff..405a3224cd 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1031,9 +1031,6 @@ typedef struct PgBackendStatus
/* The entry is valid iff st_procpid > 0, unused if st_procpid == 0 */
int st_procpid;
- /* Type of backends */
- BackendType st_backendType;
-
/* Times when current backend, transaction, and activity started */
TimestampTz st_proc_start_timestamp;
TimestampTz st_xact_start_timestamp;
--
2.17.0
v2-0012-Move-away-from-argv-string-comparison.patch (text/x-diff, 1.3K) download
From e34e34399e65ab40660f927eedc2cdbae0b4d76e Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 17 Mar 2020 14:43:52 -0400
Subject: [PATCH v2 12/12] Move away from argv string comparison
---
src/backend/postmaster/postmaster.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 65305a18ee..b31fcc7d12 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -4883,7 +4883,7 @@ SubPostmasterMain(int argc, char *argv[])
ConnProcPort = &port;
/* Close the postmaster's sockets (as soon as we know them) */
- ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
+ ClosePostmasterPorts(MySubprocessType == SysLoggerType);
/*
* Set reference point for stack-depth checking
@@ -4928,9 +4928,9 @@ SubPostmasterMain(int argc, char *argv[])
PGSharedMemoryNoReAttach();
/* autovacuum needs this set before calling InitProcess */
- if (strcmp(argv[1], "--forkavlauncher") == 0)
+ if (MySubprocessType == AutoVacuumLauncherType)
AutovacuumLauncherIAm();
- if (strcmp(argv[1], "--forkavworker") == 0)
+ if (MySubprocessType == AutoVacuumWorkerType)
AutovacuumWorkerIAm();
/*
--
2.17.0
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-02 14:53 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-04 04:56 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 18:50 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 21:52 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
@ 2020-03-18 01:04 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2020-03-18 13:22 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
0 siblings, 1 reply; 38+ messages in thread
From: Alvaro Herrera @ 2020-03-18 01:04 UTC (permalink / raw)
To: Justin Pryzby <pryzby@telsasoft.com>; +Cc: Mike Palmiotto <mike.palmiotto@crunchydata.com>; Michael Paquier <michael@paquier.xyz>; Andres Freund <andres@anarazel.de>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On 2020-Mar-17, Justin Pryzby wrote:
> +static PgSubprocess process_types[] = {
> + {
> + .desc = "checker",
> + .entrypoint = CheckerModeMain
> + },
> + {
> + .desc = "bootstrap",
> + .entrypoint = BootstrapModeMain
> + },
Maybe this stuff can be resolved using a technique like rmgrlist.h or
cmdtaglist.h. That way it's not in two separate files.
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-02 14:53 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-04 04:56 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 18:50 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 21:52 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 01:04 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
@ 2020-03-18 13:22 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 14:17 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
0 siblings, 1 reply; 38+ messages in thread
From: Mike Palmiotto @ 2020-03-18 13:22 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@2ndquadrant.com>; +Cc: Justin Pryzby <pryzby@telsasoft.com>; Michael Paquier <michael@paquier.xyz>; Andres Freund <andres@anarazel.de>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Tue, Mar 17, 2020 at 9:04 PM Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
>
> On 2020-Mar-17, Justin Pryzby wrote:
>
> > +static PgSubprocess process_types[] = {
> > + {
> > + .desc = "checker",
> > + .entrypoint = CheckerModeMain
> > + },
> > + {
> > + .desc = "bootstrap",
> > + .entrypoint = BootstrapModeMain
> > + },
>
> Maybe this stuff can be resolved using a technique like rmgrlist.h or
> cmdtaglist.h. That way it's not in two separate files.
Great suggestion, thanks! I'll try this out in the next version.
--
Mike Palmiotto
https://crunchydata.com
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-02 14:53 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-04 04:56 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 18:50 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 21:52 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 01:04 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2020-03-18 13:22 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
@ 2020-03-18 14:17 ` Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 15:26 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
0 siblings, 1 reply; 38+ messages in thread
From: Justin Pryzby @ 2020-03-18 14:17 UTC (permalink / raw)
To: Mike Palmiotto <mike.palmiotto@crunchydata.com>; +Cc: Alvaro Herrera <alvherre@2ndquadrant.com>; Michael Paquier <michael@paquier.xyz>; Andres Freund <andres@anarazel.de>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Wed, Mar 18, 2020 at 09:22:58AM -0400, Mike Palmiotto wrote:
> On Tue, Mar 17, 2020 at 9:04 PM Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
> >
> > On 2020-Mar-17, Justin Pryzby wrote:
> >
> > > +static PgSubprocess process_types[] = {
> > > + {
> > > + .desc = "checker",
> > > + .entrypoint = CheckerModeMain
> > > + },
> > > + {
> > > + .desc = "bootstrap",
> > > + .entrypoint = BootstrapModeMain
> > > + },
> >
> > Maybe this stuff can be resolved using a technique like rmgrlist.h or
> > cmdtaglist.h. That way it's not in two separate files.
>
> Great suggestion, thanks! I'll try this out in the next version.
Also, I saw this was failing tests both before and after my rebase.
http://cfbot.cputube.org/
https://ci.appveyor.com/project/postgresql-cfbot/postgresql/builds/31535161
https://ci.appveyor.com/project/postgresql-cfbot/postgresql/builds/31386446
--
Justin
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-02 14:53 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-04 04:56 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 18:50 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 21:52 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 01:04 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2020-03-18 13:22 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 14:17 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
@ 2020-03-18 15:26 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 16:07 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
0 siblings, 1 reply; 38+ messages in thread
From: Mike Palmiotto @ 2020-03-18 15:26 UTC (permalink / raw)
To: Justin Pryzby <pryzby@telsasoft.com>; +Cc: Alvaro Herrera <alvherre@2ndquadrant.com>; Michael Paquier <michael@paquier.xyz>; Andres Freund <andres@anarazel.de>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Wed, Mar 18, 2020 at 10:17 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
>
> On Wed, Mar 18, 2020 at 09:22:58AM -0400, Mike Palmiotto wrote:
> > On Tue, Mar 17, 2020 at 9:04 PM Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
> > >
> > > On 2020-Mar-17, Justin Pryzby wrote:
> > >
> > > > +static PgSubprocess process_types[] = {
> > > > + {
> > > > + .desc = "checker",
> > > > + .entrypoint = CheckerModeMain
> > > > + },
> > > > + {
> > > > + .desc = "bootstrap",
> > > > + .entrypoint = BootstrapModeMain
> > > > + },
> > >
> > > Maybe this stuff can be resolved using a technique like rmgrlist.h or
> > > cmdtaglist.h. That way it's not in two separate files.
> >
> > Great suggestion, thanks! I'll try this out in the next version.
>
> Also, I saw this was failing tests both before and after my rebase.
>
> http://cfbot.cputube.org/
> https://ci.appveyor.com/project/postgresql-cfbot/postgresql/builds/31535161
> https://ci.appveyor.com/project/postgresql-cfbot/postgresql/builds/31386446
Good catch, thanks. Will address this as well in the next round. Just
need to set up a Windows dev environment to see if I can
reproduce/fix.
--
Mike Palmiotto
https://crunchydata.com
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-02 14:53 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-04 04:56 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 18:50 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 21:52 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 01:04 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2020-03-18 13:22 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 14:17 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 15:26 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
@ 2020-03-18 16:07 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 16:54 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2020-03-19 10:35 ` Re: Auxiliary Processes and MyAuxProc Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
0 siblings, 2 replies; 38+ messages in thread
From: Mike Palmiotto @ 2020-03-18 16:07 UTC (permalink / raw)
To: Justin Pryzby <pryzby@telsasoft.com>; +Cc: Alvaro Herrera <alvherre@2ndquadrant.com>; Michael Paquier <michael@paquier.xyz>; Andres Freund <andres@anarazel.de>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Wed, Mar 18, 2020 at 11:26 AM Mike Palmiotto
<mike.palmiotto@crunchydata.com> wrote:
>
> On Wed, Mar 18, 2020 at 10:17 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
> > Also, I saw this was failing tests both before and after my rebase.
> >
> > http://cfbot.cputube.org/
> > https://ci.appveyor.com/project/postgresql-cfbot/postgresql/builds/31535161
> > https://ci.appveyor.com/project/postgresql-cfbot/postgresql/builds/31386446
>
> Good catch, thanks. Will address this as well in the next round. Just
> need to set up a Windows dev environment to see if I can
> reproduce/fix.
While I track this down, here is a rebased patchset, which drops
MySubprocessType and makes use of the MyBackendType.
Once I get the fix for Windows builds, I'll see about making use of
the rmgrlist/cmdtaglist-style technique that Alvaro suggested.
Thanks,
--
Mike Palmiotto
https://crunchydata.com
Attachments: download all (11 files, .tar.gz)
0001-Add-subprocess-infrastructure.patch (text/x-patch, 14.3K) download
From 156b90798007b805d1cff26742146929aa99ef7a Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Mon, 9 Mar 2020 22:12:10 +0000
Subject: [PATCH 01/11] Add subprocess infrastructure
This commit introduces a new infrastructure for postmaster subprocesses
initialization, forking, exec'ing and main function control. The intent
is to have all subprocess control information live in the process_types
struct, defined in subprocess.h. As more processes are added to the
struct, the fields will be expanded to differentiate actions.
There are several advantages to such an infrastructure:
1) Centralized definition of subprocess information
2) Standardized function arguments/naming schemes
3) Code minimization and removal of duplicate code
---
src/backend/bootstrap/bootstrap.c | 98 ++++++---------------------
src/backend/postmaster/Makefile | 1 +
src/backend/postmaster/bgwriter.c | 2 +-
src/backend/postmaster/checkpointer.c | 2 +-
src/backend/postmaster/startup.c | 3 +-
src/backend/postmaster/subprocess.c | 62 +++++++++++++++++
src/backend/postmaster/walwriter.c | 2 +-
src/backend/replication/walreceiver.c | 2 +-
src/include/bootstrap/bootstrap.h | 3 +-
src/include/postmaster/bgwriter.h | 4 +-
src/include/postmaster/startup.h | 5 +-
src/include/postmaster/subprocess.h | 44 ++++++++++++
src/include/postmaster/walwriter.h | 2 +-
src/include/replication/walreceiver.h | 2 +-
14 files changed, 143 insertions(+), 89 deletions(-)
create mode 100644 src/backend/postmaster/subprocess.c
create mode 100644 src/include/postmaster/subprocess.h
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 5480a024e0..38d45d9163 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -35,6 +35,7 @@
#include "pgstat.h"
#include "postmaster/bgwriter.h"
#include "postmaster/startup.h"
+#include "postmaster/subprocess.h"
#include "postmaster/walwriter.h"
#include "replication/walreceiver.h"
#include "storage/bufmgr.h"
@@ -56,8 +57,6 @@ uint32 bootstrap_data_checksum_version = 0; /* No checksum */
#define ALLOC(t, c) \
((t *) MemoryContextAllocZero(TopMemoryContext, (unsigned)(c) * sizeof(t)))
-static void CheckerModeMain(void);
-static void BootstrapModeMain(void);
static void bootstrap_signals(void);
static void ShutdownAuxiliaryProcess(int code, Datum arg);
static Form_pg_attribute AllocateAttribute(void);
@@ -314,26 +313,14 @@ AuxiliaryProcessMain(int argc, char *argv[])
proc_exit(1);
}
- switch (MyAuxProcType)
- {
- case StartupProcess:
- MyBackendType = B_STARTUP;
- break;
- case BgWriterProcess:
- MyBackendType = B_BG_WRITER;
- break;
- case CheckpointerProcess:
- MyBackendType = B_CHECKPOINTER;
- break;
- case WalWriterProcess:
- MyBackendType = B_WAL_WRITER;
- break;
- case WalReceiverProcess:
- MyBackendType = B_WAL_RECEIVER;
- break;
- default:
- MyBackendType = B_INVALID;
- }
+ /*
+ * We've read the arguments and know what backend type we are.
+ */
+ InitMySubprocess((SubprocessType)MyAuxProcType);
+
+ /*
+ * Identify myself via ps
+ */
if (IsUnderPostmaster)
init_ps_display(NULL);
@@ -418,56 +405,10 @@ AuxiliaryProcessMain(int argc, char *argv[])
*/
SetProcessingMode(NormalProcessing);
- switch (MyAuxProcType)
- {
- case CheckerProcess:
- /* don't set signals, they're useless here */
- CheckerModeMain();
- proc_exit(1); /* should never return */
-
- case BootstrapProcess:
-
- /*
- * There was a brief instant during which mode was Normal; this is
- * okay. We need to be in bootstrap mode during BootStrapXLOG for
- * the sake of multixact initialization.
- */
- SetProcessingMode(BootstrapProcessing);
- bootstrap_signals();
- BootStrapXLOG();
- BootstrapModeMain();
- proc_exit(1); /* should never return */
-
- case StartupProcess:
- /* don't set signals, startup process has its own agenda */
- StartupProcessMain();
- proc_exit(1); /* should never return */
-
- case BgWriterProcess:
- /* don't set signals, bgwriter has its own agenda */
- BackgroundWriterMain();
- proc_exit(1); /* should never return */
-
- case CheckpointerProcess:
- /* don't set signals, checkpointer has its own agenda */
- CheckpointerMain();
- proc_exit(1); /* should never return */
-
- case WalWriterProcess:
- /* don't set signals, walwriter has its own agenda */
- InitXLOGAccess();
- WalWriterMain();
- proc_exit(1); /* should never return */
-
- case WalReceiverProcess:
- /* don't set signals, walreceiver has its own agenda */
- WalReceiverMain();
- proc_exit(1); /* should never return */
-
- default:
- elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
- proc_exit(1);
- }
+ /* Now jump into the subprocess main function and never look back! */
+ MySubprocess->entrypoint(argc, argv);
+
+ proc_exit(1); /* should never return */
}
/*
@@ -476,8 +417,8 @@ AuxiliaryProcessMain(int argc, char *argv[])
* settings). Since, in fact, that was already done by BaseInit(),
* we have nothing more to do here.
*/
-static void
-CheckerModeMain(void)
+void
+CheckerModeMain(int argc, char *argv[])
{
proc_exit(0);
}
@@ -489,11 +430,15 @@ CheckerModeMain(void)
* The bootstrap backend doesn't speak SQL, but instead expects
* commands in a special bootstrap language.
*/
-static void
-BootstrapModeMain(void)
+void
+BootstrapModeMain(int argc, char *argv[])
{
int i;
+ SetProcessingMode(BootstrapProcessing);
+ bootstrap_signals();
+ BootStrapXLOG();
+
Assert(!IsUnderPostmaster);
Assert(IsBootstrapProcessingMode());
@@ -508,7 +453,6 @@ BootstrapModeMain(void)
* Do backend-like initialization for bootstrap mode
*/
InitProcess();
-
InitPostgres(NULL, InvalidOid, NULL, InvalidOid, NULL, false);
/* Initialize stuff for bootstrap-file processing */
diff --git a/src/backend/postmaster/Makefile b/src/backend/postmaster/Makefile
index bfdf6a833d..52b4c2142c 100644
--- a/src/backend/postmaster/Makefile
+++ b/src/backend/postmaster/Makefile
@@ -23,6 +23,7 @@ OBJS = \
pgstat.o \
postmaster.o \
startup.o \
+ subprocess.o \
syslogger.o \
walwriter.o
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 069e27e427..b2170938f3 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -91,7 +91,7 @@ static XLogRecPtr last_snapshot_lsn = InvalidXLogRecPtr;
* basic execution environment, but not enabled signals yet.
*/
void
-BackgroundWriterMain(void)
+BackgroundWriterMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
MemoryContext bgwriter_context;
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index e354a78725..83f7be1b1e 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -180,7 +180,7 @@ static void ReqCheckpointHandler(SIGNAL_ARGS);
* basic execution environment, but not enabled signals yet.
*/
void
-CheckpointerMain(void)
+CheckpointerMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
MemoryContext checkpointer_context;
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index c2250d7d4e..e29649a6a1 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -127,13 +127,12 @@ HandleStartupProcInterrupts(void)
ProcessProcSignalBarrier();
}
-
/* ----------------------------------
* Startup Process main entry point
* ----------------------------------
*/
void
-StartupProcessMain(void)
+StartupProcessMain(int argc, char *argv[])
{
/*
* Properly accept or ignore signals the postmaster might send us.
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
new file mode 100644
index 0000000000..3e7a45bf10
--- /dev/null
+++ b/src/backend/postmaster/subprocess.c
@@ -0,0 +1,62 @@
+/*-------------------------------------------------------------------------
+ *
+ * subprocess.c
+ *
+ * Copyright (c) 2004-2020, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/postmaster/syslogger.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "bootstrap/bootstrap.h"
+#include "postmaster/bgwriter.h"
+#include "postmaster/postmaster.h"
+#include "postmaster/startup.h"
+#include "postmaster/subprocess.h"
+#include "postmaster/walwriter.h"
+#include "replication/walreceiver.h"
+
+SubprocessType MySubprocessType;
+PgSubprocess *MySubprocess;
+
+static PgSubprocess process_types[] = {
+ {
+ .desc = "checker",
+ .entrypoint = CheckerModeMain
+ },
+ {
+ .desc = "bootstrap",
+ .entrypoint = BootstrapModeMain
+ },
+ {
+ .desc = "startup",
+ .entrypoint = StartupProcessMain
+ },
+ {
+ .desc = "background writer",
+ .entrypoint = BackgroundWriterMain
+ },
+ {
+ .desc = "checkpointer",
+ .entrypoint = CheckpointerMain
+ },
+ {
+ .desc = "wal writer",
+ .entrypoint = WalWriterMain
+ },
+ {
+ .desc = "wal receiver",
+ .entrypoint = WalReceiverMain
+ }
+};
+
+void
+InitMySubprocess(SubprocessType type)
+{
+ MySubprocessType = type;
+ MySubprocess = &process_types[type];
+ MySubprocess->desc = gettext(MySubprocess->desc);
+}
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index 45a2757969..3a0cec3371 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -85,7 +85,7 @@ int WalWriterFlushAfter = 128;
* basic execution environment, but not enabled signals yet.
*/
void
-WalWriterMain(void)
+WalWriterMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
MemoryContext walwriter_context;
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index 25e0333c9e..cd39a8c982 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -165,7 +165,7 @@ ProcessWalRcvInterrupts(void)
/* Main entry point for walreceiver process */
void
-WalReceiverMain(void)
+WalReceiverMain(int argc, char *argv[])
{
char conninfo[MAXCONNINFO];
char *tmp_conninfo;
diff --git a/src/include/bootstrap/bootstrap.h b/src/include/bootstrap/bootstrap.h
index b67fa9acc1..f85cfb5202 100644
--- a/src/include/bootstrap/bootstrap.h
+++ b/src/include/bootstrap/bootstrap.h
@@ -16,7 +16,6 @@
#include "nodes/execnodes.h"
-
/*
* MAXATTR is the maximum number of attributes in a relation supported
* at bootstrap time (i.e., the max possible in a system table).
@@ -31,6 +30,8 @@ extern Relation boot_reldesc;
extern Form_pg_attribute attrtypes[MAXATTR];
extern int numattr;
+extern void CheckerModeMain(int argc, char *argv[]);
+extern void BootstrapModeMain(int argc, char *argv[]);
extern void AuxiliaryProcessMain(int argc, char *argv[]) pg_attribute_noreturn();
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..6fb1ee1d6d 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -27,8 +27,8 @@ extern int CheckPointTimeout;
extern int CheckPointWarning;
extern double CheckPointCompletionTarget;
-extern void BackgroundWriterMain(void) pg_attribute_noreturn();
-extern void CheckpointerMain(void) pg_attribute_noreturn();
+extern void BackgroundWriterMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern void CheckpointerMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void RequestCheckpoint(int flags);
extern void CheckpointWriteDelay(int flags, double progress);
diff --git a/src/include/postmaster/startup.h b/src/include/postmaster/startup.h
index 9f59c1ffa3..6a8108dfe0 100644
--- a/src/include/postmaster/startup.h
+++ b/src/include/postmaster/startup.h
@@ -13,10 +13,13 @@
#define _STARTUP_H
extern void HandleStartupProcInterrupts(void);
-extern void StartupProcessMain(void) pg_attribute_noreturn();
extern void PreRestoreCommand(void);
extern void PostRestoreCommand(void);
extern bool IsPromoteTriggered(void);
extern void ResetPromoteTriggered(void);
+/* Startup subprocess functions */
+extern void StartupProcessMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern bool StartupCleanup(int child_errno);
+
#endif /* _STARTUP_H */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
new file mode 100644
index 0000000000..56e8edf2d8
--- /dev/null
+++ b/src/include/postmaster/subprocess.h
@@ -0,0 +1,44 @@
+/*-------------------------------------------------------------------------
+ *
+ * subprocess.h
+ *
+ * Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ *
+ * src/include/postmaster/subprocess.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef SUBPROCESS_H
+#define SUBPROCESS_H
+
+#include "postmaster.h"
+
+typedef enum
+{
+ NoProcessType = -1,
+ CheckerType = 0,
+ BootstrapType,
+ StartupType,
+ BgWriterType,
+ CheckpointerType,
+ WalWriterType,
+ WalReceiverType, /* end of Auxiliary Process Forks */
+
+ NUMSUBPROCESSTYPES /* Must be last! */
+} SubprocessType;
+
+typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
+
+/* Current subprocess initializer */
+extern void InitMySubprocess(SubprocessType type);
+
+typedef struct PgSubprocess
+{
+ const char *desc;
+ SubprocessEntryPoint entrypoint;
+} PgSubprocess;
+
+extern SubprocessType MySubprocessType;
+extern PGDLLIMPORT PgSubprocess *MySubprocess;
+
+#endif /* SUBPROCESS_H */
diff --git a/src/include/postmaster/walwriter.h b/src/include/postmaster/walwriter.h
index 011dc26a6e..596c437c54 100644
--- a/src/include/postmaster/walwriter.h
+++ b/src/include/postmaster/walwriter.h
@@ -16,6 +16,6 @@
extern int WalWriterDelay;
extern int WalWriterFlushAfter;
-extern void WalWriterMain(void) pg_attribute_noreturn();
+extern void WalWriterMain(int argc, char *argv[]) pg_attribute_noreturn();
#endif /* _WALWRITER_H */
diff --git a/src/include/replication/walreceiver.h b/src/include/replication/walreceiver.h
index e08afc6548..2025f19ab3 100644
--- a/src/include/replication/walreceiver.h
+++ b/src/include/replication/walreceiver.h
@@ -311,7 +311,7 @@ walrcv_clear_result(WalRcvExecResult *walres)
}
/* prototypes for functions in walreceiver.c */
-extern void WalReceiverMain(void) pg_attribute_noreturn();
+extern void WalReceiverMain(int argc, char *argv[]);
extern void ProcessWalRcvInterrupts(void);
/* prototypes for functions in walreceiverfuncs.c */
--
2.21.1
0002-Use-centralized-StartSubprocess-for-aux-procs.patch (text/x-patch, 14.9K) download
From 16c89689c39f88f474bb24f3dbfabbd171bef049 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 10 Mar 2020 14:58:07 +0000
Subject: [PATCH 02/11] Use centralized StartSubprocess for aux procs
Modifies StartChildProcess and moves the auxiliary processes under its
control. This is the next step toward making use of the Subprocess
infrastructure.
This patch introduces the fork_failure function, which may be
implemented by each subprocess to control failure when attempting to
fork the subprocess. This is used by the startup subprocess to exit
postmaster on fork failure. Future subprocesses may use it to do cleanup
(and optionally exit postmaster) after a fork failure.
Additionally, the patch adds a new backend variable for establishing a
backend variable to be shared across fork/exec.
---
src/backend/postmaster/postmaster.c | 156 ++++++++++++++--------------
src/backend/postmaster/startup.c | 12 +++
src/backend/postmaster/subprocess.c | 35 +++++--
src/include/postmaster/startup.h | 2 +-
src/include/postmaster/subprocess.h | 4 +
5 files changed, 121 insertions(+), 88 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 2b9ab32293..4b86aeec2d 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -114,6 +114,7 @@
#include "postmaster/fork_process.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "postmaster/syslogger.h"
#include "replication/logicallauncher.h"
#include "replication/walsender.h"
@@ -191,7 +192,8 @@ static Backend *ShmemBackendArray;
BackgroundWorker *MyBgworkerEntry = NULL;
-
+/* Struct containing postmaster subprocess control info */
+SubprocessType MySubprocessType;
/* The socket number we are listening for connections on */
int PostPortNumber;
@@ -421,7 +423,7 @@ static int CountChildren(int target);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
-static pid_t StartChildProcess(AuxProcType type);
+static pid_t StartSubprocess(SubprocessType type);
static void StartAutovacuumWorker(void);
static void MaybeStartWalReceiver(void);
static void InitPostmasterDeathWatchHandle(void);
@@ -507,6 +509,9 @@ typedef struct
TimestampTz PgStartTime;
TimestampTz PgReloadTime;
pg_time_t first_syslogger_file_time;
+
+ SubprocessType MySubprocessType;
+
bool redirection_done;
bool IsBinaryUpgrade;
int max_safe_fds;
@@ -538,11 +543,6 @@ static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
-#define StartupDataBase() StartChildProcess(StartupProcess)
-#define StartBackgroundWriter() StartChildProcess(BgWriterProcess)
-#define StartCheckpointer() StartChildProcess(CheckpointerProcess)
-#define StartWalWriter() StartChildProcess(WalWriterProcess)
-#define StartWalReceiver() StartChildProcess(WalReceiverProcess)
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -1389,7 +1389,7 @@ PostmasterMain(int argc, char *argv[])
/*
* We're ready to rock and roll...
*/
- StartupPID = StartupDataBase();
+ StartupPID = StartSubprocess(StartupType);
Assert(StartupPID != 0);
StartupStatus = STARTUP_RUNNING;
pmState = PM_STARTUP;
@@ -1750,9 +1750,9 @@ ServerLoop(void)
pmState == PM_HOT_STANDBY)
{
if (CheckpointerPID == 0)
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
if (BgWriterPID == 0)
- BgWriterPID = StartBackgroundWriter();
+ BgWriterPID = StartSubprocess(BgWriterType);
}
/*
@@ -1761,7 +1761,7 @@ ServerLoop(void)
* be writing any new WAL).
*/
if (WalWriterPID == 0 && pmState == PM_RUN)
- WalWriterPID = StartWalWriter();
+ WalWriterPID = StartSubprocess(WalWriterType);
/*
* If we have lost the autovacuum launcher, try to start a new one. We
@@ -3042,11 +3042,11 @@ reaper(SIGNAL_ARGS)
* if this fails, we'll just try again later.
*/
if (CheckpointerPID == 0)
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
if (BgWriterPID == 0)
- BgWriterPID = StartBackgroundWriter();
+ BgWriterPID = StartSubprocess(BgWriterType);
if (WalWriterPID == 0)
- WalWriterPID = StartWalWriter();
+ WalWriterPID = StartSubprocess(WalWriterType);
/*
* Likewise, start other special children as needed. In a restart
@@ -3860,7 +3860,7 @@ PostmasterStateMachine(void)
Assert(Shutdown > NoShutdown);
/* Start the checkpointer if not running */
if (CheckpointerPID == 0)
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
/* And tell it to shut down */
if (CheckpointerPID != 0)
{
@@ -4001,7 +4001,7 @@ PostmasterStateMachine(void)
reset_shared();
- StartupPID = StartupDataBase();
+ StartupPID = StartSubprocess(StartupType);
Assert(StartupPID != 0);
StartupStatus = STARTUP_RUNNING;
pmState = PM_STARTUP;
@@ -4917,6 +4917,10 @@ SubPostmasterMain(int argc, char *argv[])
errmsg("out of memory")));
#endif
+ /* We should only be here once per fork */
+ Assert(!MySubprocess);
+ InitMySubprocess(MySubprocessType);
+
/*
* If appropriate, physically re-attach to shared memory segment. We want
* to do this before going any further to ensure that we can attach at the
@@ -5037,19 +5041,6 @@ SubPostmasterMain(int argc, char *argv[])
/* And run the backend */
BackendRun(&port); /* does not return */
}
- if (strcmp(argv[1], "--forkboot") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitAuxiliaryProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
- }
if (strcmp(argv[1], "--forkavlauncher") == 0)
{
/* Restore basic shared memory pointers */
@@ -5117,6 +5108,20 @@ SubPostmasterMain(int argc, char *argv[])
SysLoggerMain(argc, argv); /* does not return */
}
+ if (MySubprocess->needs_aux_proc)
+ {
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
+
+ /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ InitAuxiliaryProcess();
+
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores();
+
+ AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
+ }
+
abort(); /* shouldn't get here */
}
#endif /* EXEC_BACKEND */
@@ -5198,9 +5203,9 @@ sigusr1_handler(SIGNAL_ARGS)
* we'll just try again later.
*/
Assert(CheckpointerPID == 0);
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
Assert(BgWriterPID == 0);
- BgWriterPID = StartBackgroundWriter();
+ BgWriterPID = StartSubprocess(BgWriterType);
/*
* Start the archiver if we're responsible for (re-)archiving received
@@ -5425,42 +5430,49 @@ CountChildren(int target)
return cnt;
}
-
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
- * "type" determines what kind of child will be started. All child types
- * initially go to AuxiliaryProcessMain, which will handle common setup.
+ * "type" determines what kind of child will be started. This code relies
+ * heavily on MySubprocess, which is an entry in the process_types struct in
+ * subprocess.c. All auxiliary processes go through AuxiliaryProcessMain after
+ * forking. Everything else goes through the designated entrypoint.
*
- * Return value of StartChildProcess is subprocess' PID, or 0 if failed
- * to start subprocess.
+ * Return value of StartChildProcess is subprocess' PID on success. On failure,
+ * the subprocess cleanup function decides whether or not to panic.
*/
static pid_t
-StartChildProcess(AuxProcType type)
+StartSubprocess(SubprocessType type)
{
pid_t pid;
- char *av[10];
- int ac = 0;
+ char *argv[10];
+ int argc = 0;
char typebuf[32];
+#ifdef EXEC_BACKEND
+ char forkname[32];
+#endif
/*
- * Set up command-line arguments for subprocess
+ * Get new subprocess data every time we start a new subprocess.
*/
- av[ac++] = "postgres";
+ InitMySubprocess(type);
+
+ argv[argc++] = "postgres";
#ifdef EXEC_BACKEND
- av[ac++] = "--forkboot";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
+ snprintf(forkname, sizeof(forkname), "--fork%s", MySubprocess->name);
+ argv[argc++] = forkname;
+ argv[argc++] = NULL;
#endif
snprintf(typebuf, sizeof(typebuf), "-x%d", type);
- av[ac++] = typebuf;
+ argv[argc++] = typebuf;
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ argv[argc] = NULL;
+ Assert(argc < lengthof(argv));
#ifdef EXEC_BACKEND
- pid = postmaster_forkexec(ac, av);
+ pid = postmaster_forkexec(argc, argv);
#else /* !EXEC_BACKEND */
pid = fork_process();
@@ -5476,7 +5488,7 @@ StartChildProcess(AuxProcType type)
MemoryContextDelete(PostmasterContext);
PostmasterContext = NULL;
- AuxiliaryProcessMain(ac, av);
+ AuxiliaryProcessMain(argc, argv);
ExitPostmaster(0);
}
#endif /* EXEC_BACKEND */
@@ -5487,40 +5499,20 @@ StartChildProcess(AuxProcType type)
int save_errno = errno;
errno = save_errno;
- switch (type)
- {
- case StartupProcess:
- ereport(LOG,
- (errmsg("could not fork startup process: %m")));
- break;
- case BgWriterProcess:
- ereport(LOG,
- (errmsg("could not fork background writer process: %m")));
- break;
- case CheckpointerProcess:
- ereport(LOG,
- (errmsg("could not fork checkpointer process: %m")));
- break;
- case WalWriterProcess:
- ereport(LOG,
- (errmsg("could not fork WAL writer process: %m")));
- break;
- case WalReceiverProcess:
- ereport(LOG,
- (errmsg("could not fork WAL receiver process: %m")));
- break;
- default:
- ereport(LOG,
- (errmsg("could not fork process: %m")));
- break;
- }
+ ereport(LOG,
+ (errmsg("could not fork %s process: %m", MySubprocess->desc)));
/*
- * fork failure is fatal during startup, but there's no need to choke
- * immediately if starting other child types fails.
+ * Handle fork-failure per subprocess type. Each subprocess handles
+ * fork-failure by returning a boolean which tells postmaster whether or
+ * not to panic.
*/
- if (type == StartupProcess)
- ExitPostmaster(1);
+ if (MySubprocess->fork_failure)
+ {
+ if (MySubprocess->fork_failure(save_errno))
+ ExitPostmaster(1);
+ }
+
return 0;
}
@@ -5638,7 +5630,7 @@ MaybeStartWalReceiver(void)
pmState == PM_HOT_STANDBY || pmState == PM_WAIT_READONLY) &&
Shutdown == NoShutdown)
{
- WalReceiverPID = StartWalReceiver();
+ WalReceiverPID = StartSubprocess(WalReceiverType);
if (WalReceiverPID != 0)
WalReceiverRequested = false;
/* else leave the flag set, so we'll try again later */
@@ -6198,6 +6190,8 @@ save_backend_variables(BackendParameters *param, Port *port,
param->PgReloadTime = PgReloadTime;
param->first_syslogger_file_time = first_syslogger_file_time;
+ param->MySubprocessType = MySubprocessType;
+
param->redirection_done = redirection_done;
param->IsBinaryUpgrade = IsBinaryUpgrade;
param->max_safe_fds = max_safe_fds;
@@ -6433,6 +6427,8 @@ restore_backend_variables(BackendParameters *param, Port *port)
PgReloadTime = param->PgReloadTime;
first_syslogger_file_time = param->first_syslogger_file_time;
+ MySubprocessType = param->MySubprocessType;
+
redirection_done = param->redirection_done;
IsBinaryUpgrade = param->IsBinaryUpgrade;
max_safe_fds = param->max_safe_fds;
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index e29649a6a1..782cd608ff 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -175,6 +175,18 @@ StartupProcessMain(int argc, char *argv[])
proc_exit(0);
}
+/*
+ * StartupForkFailure
+ *
+ * Tell the postmaster to exit/panic on failure to fork.
+ */
+bool
+StartupForkFailure(int fork_errno)
+{
+ /* Panic! */
+ return true;
+}
+
void
PreRestoreCommand(void)
{
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 3e7a45bf10..7e96678dfb 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -24,32 +24,53 @@ PgSubprocess *MySubprocess;
static PgSubprocess process_types[] = {
{
+ .name = "boot",
.desc = "checker",
- .entrypoint = CheckerModeMain
+ .needs_aux_proc = true,
+ .entrypoint = CheckerModeMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "bootstrap",
- .entrypoint = BootstrapModeMain
+ .needs_aux_proc = true,
+ .entrypoint = BootstrapModeMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "startup",
- .entrypoint = StartupProcessMain
+ .needs_aux_proc = true,
+ .entrypoint = StartupProcessMain,
+ .fork_failure = StartupForkFailure
},
{
+ .name = "boot",
.desc = "background writer",
- .entrypoint = BackgroundWriterMain
+ .needs_aux_proc = true,
+ .entrypoint = BackgroundWriterMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "checkpointer",
- .entrypoint = CheckpointerMain
+ .needs_aux_proc = true,
+ .entrypoint = CheckpointerMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "wal writer",
- .entrypoint = WalWriterMain
+ .needs_aux_proc = true,
+ .entrypoint = WalWriterMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "wal receiver",
- .entrypoint = WalReceiverMain
+ .needs_aux_proc = true,
+ .entrypoint = WalReceiverMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/postmaster/startup.h b/src/include/postmaster/startup.h
index 6a8108dfe0..c182c76960 100644
--- a/src/include/postmaster/startup.h
+++ b/src/include/postmaster/startup.h
@@ -20,6 +20,6 @@ extern void ResetPromoteTriggered(void);
/* Startup subprocess functions */
extern void StartupProcessMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern bool StartupCleanup(int child_errno);
+extern bool StartupForkFailure(int fork_errno);
#endif /* _STARTUP_H */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 56e8edf2d8..33f6a7ab04 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -28,14 +28,18 @@ typedef enum
} SubprocessType;
typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
+typedef bool (*SubprocessForkFailure) (int fork_errno);
/* Current subprocess initializer */
extern void InitMySubprocess(SubprocessType type);
typedef struct PgSubprocess
{
+ const char *name;
const char *desc;
+ bool needs_aux_proc;
SubprocessEntryPoint entrypoint;
+ SubprocessForkFailure fork_failure;
} PgSubprocess;
extern SubprocessType MySubprocessType;
--
2.21.1
0003-Add-AutoVacLauncherType-to-subprocess-struct.patch (text/x-patch, 7.5K) download
From f24ad5db32f63805ddd7f19082036c808846ffab Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 10 Mar 2020 22:29:44 +0000
Subject: [PATCH 03/11] Add AutoVacLauncherType to subprocess struct
---
src/backend/postmaster/autovacuum.c | 68 ++---------------------------
src/backend/postmaster/postmaster.c | 36 ++++++++-------
src/backend/postmaster/subprocess.c | 8 ++++
src/include/postmaster/autovacuum.h | 3 +-
src/include/postmaster/subprocess.h | 1 +
5 files changed, 35 insertions(+), 81 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index da75e755f0..7397b944d3 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -95,6 +95,7 @@
#include "storage/procsignal.h"
#include "storage/sinvaladt.h"
#include "storage/smgr.h"
+#include "postmaster/subprocess.h"
#include "tcop/tcopprot.h"
#include "utils/fmgroids.h"
#include "utils/fmgrprotos.h"
@@ -303,11 +304,9 @@ static WorkerInfo MyWorkerInfo = NULL;
int AutovacuumLauncherPid = 0;
#ifdef EXEC_BACKEND
-static pid_t avlauncher_forkexec(void);
static pid_t avworker_forkexec(void);
#endif
NON_EXEC_STATIC void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-NON_EXEC_STATIC void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
static Oid do_start_worker(void);
static void HandleAutoVacLauncherInterrupts(void);
@@ -353,26 +352,6 @@ static void autovac_refresh_stats(void);
********************************************************************/
#ifdef EXEC_BACKEND
-/*
- * forkexec routine for the autovacuum launcher process.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avlauncher_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavlauncher";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
/*
* We need this set from the outside, before InitProcess is called
@@ -384,56 +363,17 @@ AutovacuumLauncherIAm(void)
}
#endif
-/*
- * Main entry point for autovacuum launcher process, to be called from the
- * postmaster.
- */
-int
-StartAutoVacLauncher(void)
-{
- pid_t AutoVacPID;
-
-#ifdef EXEC_BACKEND
- switch ((AutoVacPID = avlauncher_forkexec()))
-#else
- switch ((AutoVacPID = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum launcher process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacLauncherMain(0, NULL);
- break;
-#endif
- default:
- return (int) AutoVacPID;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
/*
* Main loop for the autovacuum launcher process.
*/
-NON_EXEC_STATIC void
-AutoVacLauncherMain(int argc, char *argv[])
+void
+AutoVacLauncherMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
am_autovacuum_launcher = true;
- MyBackendType = B_AUTOVAC_LAUNCHER;
+ /* Identify myself via ps */
init_ps_display(NULL);
ereport(DEBUG1,
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 4b86aeec2d..2251303589 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1773,7 +1773,7 @@ ServerLoop(void)
(AutoVacuumingActive() || start_autovac_launcher) &&
pmState == PM_RUN)
{
- AutoVacPID = StartAutoVacLauncher();
+ AutoVacPID = StartSubprocess(AutoVacuumLauncherType);
if (AutoVacPID != 0)
start_autovac_launcher = false; /* signal processed */
}
@@ -3053,7 +3053,7 @@ reaper(SIGNAL_ARGS)
* situation, some of them may be alive already.
*/
if (!IsBinaryUpgrade && AutoVacuumingActive() && AutoVacPID == 0)
- AutoVacPID = StartAutoVacLauncher();
+ AutoVacPID = StartSubprocess(AutoVacuumLauncherType);
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = pgarch_start();
if (PgStatPID == 0)
@@ -5041,19 +5041,6 @@ SubPostmasterMain(int argc, char *argv[])
/* And run the backend */
BackendRun(&port); /* does not return */
}
- if (strcmp(argv[1], "--forkavlauncher") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
- }
if (strcmp(argv[1], "--forkavworker") == 0)
{
/* Restore basic shared memory pointers */
@@ -5121,6 +5108,19 @@ SubPostmasterMain(int argc, char *argv[])
AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
}
+ else
+ {
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
+
+ /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ InitProcess();
+
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores();
+
+ MySubprocess->entrypoint(argc - 2, argv + 2);
+ }
abort(); /* shouldn't get here */
}
@@ -5488,7 +5488,11 @@ StartSubprocess(SubprocessType type)
MemoryContextDelete(PostmasterContext);
PostmasterContext = NULL;
- AuxiliaryProcessMain(argc, argv);
+ if (MySubprocess->needs_aux_proc)
+ AuxiliaryProcessMain(argc, argv);
+ else
+ MySubprocess->entrypoint(argc, argv);
+
ExitPostmaster(0);
}
#endif /* EXEC_BACKEND */
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 7e96678dfb..9aa60cc93e 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -12,6 +12,7 @@
*/
#include "postgres.h"
#include "bootstrap/bootstrap.h"
+#include "postmaster/autovacuum.h"
#include "postmaster/bgwriter.h"
#include "postmaster/postmaster.h"
#include "postmaster/startup.h"
@@ -71,6 +72,13 @@ static PgSubprocess process_types[] = {
.needs_aux_proc = true,
.entrypoint = WalReceiverMain,
.fork_failure = NULL
+ },
+ {
+ .name = "avlauncher",
+ .desc = "autovacuum launcher",
+ .needs_aux_proc = false,
+ .entrypoint = AutoVacLauncherMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index d40ed55531..08050cd81c 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -58,6 +58,8 @@ extern void autovac_init(void);
extern int StartAutoVacLauncher(void);
extern int StartAutoVacWorker(void);
+extern void AutoVacLauncherMain(int argc, char *argv[]);
+
/* called from postmaster when a worker could not be forked */
extern void AutoVacWorkerFailed(void);
@@ -65,7 +67,6 @@ extern void AutoVacWorkerFailed(void);
extern void AutoVacuumUpdateDelay(void);
#ifdef EXEC_BACKEND
-extern void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void AutovacuumWorkerIAm(void);
extern void AutovacuumLauncherIAm(void);
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 33f6a7ab04..9e3c743333 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -23,6 +23,7 @@ typedef enum
CheckpointerType,
WalWriterType,
WalReceiverType, /* end of Auxiliary Process Forks */
+ AutoVacuumLauncherType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.21.1
0004-Add-AutoVacuumWorkerType-to-subprocess-struct.patch (text/x-patch, 6.4K) download
From 086e15389e523be661dcb29961f072b622680b6c Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 10 Mar 2020 23:33:39 +0000
Subject: [PATCH 04/11] Add AutoVacuumWorkerType to subprocess struct
---
src/backend/postmaster/autovacuum.c | 71 +----------------------------
src/backend/postmaster/postmaster.c | 17 +------
src/backend/postmaster/subprocess.c | 7 +++
src/include/postmaster/autovacuum.h | 4 +-
src/include/postmaster/subprocess.h | 1 +
5 files changed, 14 insertions(+), 86 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 7397b944d3..cf0a74a4bb 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -83,7 +83,6 @@
#include "nodes/makefuncs.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "storage/bufmgr.h"
@@ -303,11 +302,6 @@ static WorkerInfo MyWorkerInfo = NULL;
/* PID of launcher, valid only in worker while shutting down */
int AutovacuumLauncherPid = 0;
-#ifdef EXEC_BACKEND
-static pid_t avworker_forkexec(void);
-#endif
-NON_EXEC_STATIC void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-
static Oid do_start_worker(void);
static void HandleAutoVacLauncherInterrupts(void);
static void AutoVacLauncherShutdown(void) pg_attribute_noreturn();
@@ -1364,27 +1358,6 @@ avl_sigusr2_handler(SIGNAL_ARGS)
********************************************************************/
#ifdef EXEC_BACKEND
-/*
- * forkexec routines for the autovacuum worker.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avworker_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavworker";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
/*
* We need this set from the outside, before InitProcess is called
*/
@@ -1395,50 +1368,10 @@ AutovacuumWorkerIAm(void)
}
#endif
-/*
- * Main entry point for autovacuum worker process.
- *
- * This code is heavily based on pgarch.c, q.v.
- */
-int
-StartAutoVacWorker(void)
-{
- pid_t worker_pid;
-
-#ifdef EXEC_BACKEND
- switch ((worker_pid = avworker_forkexec()))
-#else
- switch ((worker_pid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum worker process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacWorkerMain(0, NULL);
- break;
-#endif
- default:
- return (int) worker_pid;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
/*
* AutoVacWorkerMain
*/
-NON_EXEC_STATIC void
+void
AutoVacWorkerMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
@@ -1446,7 +1379,7 @@ AutoVacWorkerMain(int argc, char *argv[])
am_autovacuum_worker = true;
- MyBackendType = B_AUTOVAC_WORKER;
+ /* Identify myself via ps */
init_ps_display(NULL);
SetProcessingMode(InitProcessing);
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 2251303589..21bbbf8194 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -5041,19 +5041,6 @@ SubPostmasterMain(int argc, char *argv[])
/* And run the backend */
BackendRun(&port); /* does not return */
}
- if (strcmp(argv[1], "--forkavworker") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
- }
if (strncmp(argv[1], "--forkbgworker=", 15) == 0)
{
int shmem_slot;
@@ -5573,7 +5560,7 @@ StartAutovacuumWorker(void)
bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
bn->bgworker_notify = false;
- bn->pid = StartAutoVacWorker();
+ bn->pid = StartSubprocess(AutoVacuumWorkerType);
if (bn->pid > 0)
{
bn->bkend_type = BACKEND_TYPE_AUTOVAC;
@@ -5587,7 +5574,7 @@ StartAutovacuumWorker(void)
/*
* fork failed, fall through to report -- actual error message was
- * logged by StartAutoVacWorker
+ * logged by StartSubprocess
*/
(void) ReleasePostmasterChildSlot(bn->child_slot);
free(bn);
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 9aa60cc93e..0fbd067310 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -79,6 +79,13 @@ static PgSubprocess process_types[] = {
.needs_aux_proc = false,
.entrypoint = AutoVacLauncherMain,
.fork_failure = NULL
+ },
+ {
+ .name = "avworker",
+ .desc = "autovacuum worker",
+ .needs_aux_proc = false,
+ .entrypoint = AutoVacWorkerMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 08050cd81c..51546fd4c9 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -58,7 +58,8 @@ extern void autovac_init(void);
extern int StartAutoVacLauncher(void);
extern int StartAutoVacWorker(void);
-extern void AutoVacLauncherMain(int argc, char *argv[]);
+extern void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
/* called from postmaster when a worker could not be forked */
extern void AutoVacWorkerFailed(void);
@@ -67,7 +68,6 @@ extern void AutoVacWorkerFailed(void);
extern void AutoVacuumUpdateDelay(void);
#ifdef EXEC_BACKEND
-extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void AutovacuumWorkerIAm(void);
extern void AutovacuumLauncherIAm(void);
#endif
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 9e3c743333..e11c24cc62 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -24,6 +24,7 @@ typedef enum
WalWriterType,
WalReceiverType, /* end of Auxiliary Process Forks */
AutoVacuumLauncherType,
+ AutoVacuumWorkerType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.21.1
0005-Add-PgstatCollectorType-to-subprocess-struct.patch (text/x-patch, 11.6K) download
From 82820045efe748b3901685f1ad48b33c7b82b205 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 11 Mar 2020 00:24:42 +0000
Subject: [PATCH 05/11] Add PgstatCollectorType to subprocess struct
Introduce two new fields to handle subprocess control:
1) needs_shmem controls whether or not the subprocess needs to attach
to shmem. Up until now, all subprocesses needed to attach. This field
paves the way for easier handling of subprocesses which do not.
2) fork_prep allows StartSubprocess to do some preparation for
fork/exec. This is necessary to split out any functionality that
previously happened in Start functions prior to fork/exec.
---
src/backend/postmaster/pgstat.c | 86 +++--------------------------
src/backend/postmaster/postmaster.c | 39 +++++++------
src/backend/postmaster/subprocess.c | 28 ++++++++++
src/include/pgstat.h | 4 +-
src/include/postmaster/subprocess.h | 8 ++-
5 files changed, 65 insertions(+), 100 deletions(-)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index f9287b7942..6d4f2ace56 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -48,9 +48,9 @@
#include "pg_trace.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "replication/walsender.h"
#include "storage/backendid.h"
#include "storage/dsm.h"
@@ -275,11 +275,6 @@ static instr_time total_func_time;
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgstat_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgstat_beshutdown_hook(int code, Datum arg);
static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
@@ -685,46 +680,15 @@ pgstat_reset_all(void)
pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
}
-#ifdef EXEC_BACKEND
-
/*
- * pgstat_forkexec() -
+ * PgstatCollectorPrep
*
- * Format up the arglist for, then fork and exec, statistics collector process
- */
-static pid_t
-pgstat_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkcol";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
-/*
- * pgstat_start() -
- *
- * Called from postmaster at startup or after an existing collector
- * died. Attempt to fire up a fresh statistics collector.
- *
- * Returns PID of child process, or 0 if fail.
- *
- * Note: if fail, we will be called again from the postmaster main loop.
+ * Called from StartSubprocess to prepare a new collector fork
*/
int
-pgstat_start(void)
+PgstatCollectorPrep(int argc, char *argv[])
{
time_t curtime;
- pid_t pgStatPid;
/*
* Check that the socket is there, else pgstat_init failed and we can do
@@ -740,46 +704,10 @@ pgstat_start(void)
* from the postmaster main loop, we will get another chance later.
*/
curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgstat_start_time) <
+ if ((unsigned int) (curtime - last_pgstat_start_time) >=
(unsigned int) PGSTAT_RESTART_INTERVAL)
- return 0;
- last_pgstat_start_time = curtime;
+ last_pgstat_start_time = curtime;
- /*
- * Okay, fork off the collector.
- */
-#ifdef EXEC_BACKEND
- switch ((pgStatPid = pgstat_forkexec()))
-#else
- switch ((pgStatPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork statistics collector: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgstatCollectorMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgStatPid;
- }
-
- /* shouldn't get here */
return 0;
}
@@ -4326,7 +4254,7 @@ pgstat_send_bgwriter(void)
* The argc/argv parameters are valid only in EXEC_BACKEND case.
* ----------
*/
-NON_EXEC_STATIC void
+void
PgstatCollectorMain(int argc, char *argv[])
{
int len;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 21bbbf8194..e2ad879452 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1781,7 +1781,7 @@ ServerLoop(void)
/* If we have lost the stats collector, try to start a new one */
if (PgStatPID == 0 &&
(pmState == PM_RUN || pmState == PM_HOT_STANDBY))
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
@@ -3057,7 +3057,7 @@ reaper(SIGNAL_ARGS)
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = pgarch_start();
if (PgStatPID == 0)
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
/* workers may be scheduled to start now */
maybe_start_bgworkers();
@@ -3219,7 +3219,7 @@ reaper(SIGNAL_ARGS)
LogChildExit(LOG, _("statistics collector process"),
pid, exitstatus);
if (pmState == PM_RUN || pmState == PM_HOT_STANDBY)
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
continue;
}
@@ -5069,12 +5069,6 @@ SubPostmasterMain(int argc, char *argv[])
PgArchiverMain(argc, argv); /* does not return */
}
- if (strcmp(argv[1], "--forkcol") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgstatCollectorMain(argc, argv); /* does not return */
- }
if (strcmp(argv[1], "--forklog") == 0)
{
/* Do not want to attach to shared memory */
@@ -5097,14 +5091,17 @@ SubPostmasterMain(int argc, char *argv[])
}
else
{
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
+ if (MySubprocess->needs_shmem)
+ {
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
+ /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ InitProcess();
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores();
+ }
MySubprocess->entrypoint(argc - 2, argv + 2);
}
@@ -5224,7 +5221,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Likewise, start other special children as needed.
*/
Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5475,6 +5472,16 @@ StartSubprocess(SubprocessType type)
MemoryContextDelete(PostmasterContext);
PostmasterContext = NULL;
+ /*
+ * Drop connection to postmaster's shared memory if the subprocess
+ * doesn't need it.
+ */
+ if (!MySubprocess->needs_shmem)
+ {
+ dsm_detach_all();
+ PGSharedMemoryDetach();
+ }
+
if (MySubprocess->needs_aux_proc)
AuxiliaryProcessMain(argc, argv);
else
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 0fbd067310..ee44ef0789 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -12,6 +12,7 @@
*/
#include "postgres.h"
#include "bootstrap/bootstrap.h"
+#include "pgstat.h"
#include "postmaster/autovacuum.h"
#include "postmaster/bgwriter.h"
#include "postmaster/postmaster.h"
@@ -28,6 +29,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "checker",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = CheckerModeMain,
.fork_failure = NULL
},
@@ -35,6 +38,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "bootstrap",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = BootstrapModeMain,
.fork_failure = NULL
},
@@ -42,6 +47,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "startup",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = StartupProcessMain,
.fork_failure = StartupForkFailure
},
@@ -49,6 +56,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "background writer",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = BackgroundWriterMain,
.fork_failure = NULL
},
@@ -56,6 +65,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "checkpointer",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = CheckpointerMain,
.fork_failure = NULL
},
@@ -63,6 +74,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "wal writer",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = WalWriterMain,
.fork_failure = NULL
},
@@ -70,6 +83,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "wal receiver",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = WalReceiverMain,
.fork_failure = NULL
},
@@ -77,6 +92,8 @@ static PgSubprocess process_types[] = {
.name = "avlauncher",
.desc = "autovacuum launcher",
.needs_aux_proc = false,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = AutoVacLauncherMain,
.fork_failure = NULL
},
@@ -84,8 +101,19 @@ static PgSubprocess process_types[] = {
.name = "avworker",
.desc = "autovacuum worker",
.needs_aux_proc = false,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = AutoVacWorkerMain,
.fork_failure = NULL
+ },
+ {
+ .name = "col",
+ .desc = "statistics collector",
+ .needs_aux_proc = false,
+ .needs_shmem = false,
+ .fork_prep = PgstatCollectorPrep,
+ .entrypoint = PgstatCollectorMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1a19921f80..92943f5cff 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1224,10 +1224,8 @@ extern int pgstat_start(void);
extern void pgstat_reset_all(void);
extern void allow_immediate_pgstat_restart(void);
-#ifdef EXEC_BACKEND
+extern int PgstatCollectorPrep(int argc, char *argv[]);
extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
-
/* ----------
* Functions called from backends
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index e11c24cc62..a7ab036b7b 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -25,12 +25,14 @@ typedef enum
WalReceiverType, /* end of Auxiliary Process Forks */
AutoVacuumLauncherType,
AutoVacuumWorkerType,
+ PgstatCollectorType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
-typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
-typedef bool (*SubprocessForkFailure) (int fork_errno);
+typedef int (*SubprocessPrep) (int argc, char *argv[]);
+typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
+typedef bool (*SubprocessForkFailure) (int fork_errno);
/* Current subprocess initializer */
extern void InitMySubprocess(SubprocessType type);
@@ -40,6 +42,8 @@ typedef struct PgSubprocess
const char *name;
const char *desc;
bool needs_aux_proc;
+ bool needs_shmem;
+ SubprocessPrep fork_prep;
SubprocessEntryPoint entrypoint;
SubprocessForkFailure fork_failure;
} PgSubprocess;
--
2.21.1
0006-Add-PgArchiverType-to-subprocess-struct.patch (text/x-patch, 7.7K) download
From 451847b00c68815261d24aa689627d877ba0059b Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 11 Mar 2020 01:07:05 +0000
Subject: [PATCH 06/11] Add PgArchiverType to subprocess struct
---
src/backend/postmaster/pgarch.c | 89 ++++-------------------------
src/backend/postmaster/postmaster.c | 14 ++---
src/backend/postmaster/subprocess.c | 10 ++++
src/include/postmaster/pgarch.h | 9 +--
src/include/postmaster/subprocess.h | 1 +
5 files changed, 27 insertions(+), 96 deletions(-)
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 01ffd6513c..b2cbe80361 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -38,10 +38,10 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "storage/dsm.h"
#include "storage/fd.h"
#include "storage/ipc.h"
@@ -91,11 +91,6 @@ static volatile sig_atomic_t ready_to_stop = false;
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgarch_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgarch_exit(SIGNAL_ARGS);
static void pgarch_waken(SIGNAL_ARGS);
static void pgarch_waken_stop(SIGNAL_ARGS);
@@ -122,10 +117,9 @@ static void pgarch_archiveDone(char *xlog);
* Note: if fail, we will be called again from the postmaster main loop.
*/
int
-pgarch_start(void)
+PgArchiverPrep(int argc, char *argv[])
{
time_t curtime;
- pid_t pgArchPid;
/*
* Do nothing if no archiver needed
@@ -140,86 +134,20 @@ pgarch_start(void)
* the postmaster main loop, we will get another chance later.
*/
curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgarch_start_time) <
+ if ((unsigned int) (curtime - last_pgarch_start_time) >=
(unsigned int) PGARCH_RESTART_INTERVAL)
- return 0;
- last_pgarch_start_time = curtime;
-
-#ifdef EXEC_BACKEND
- switch ((pgArchPid = pgarch_forkexec()))
-#else
- switch ((pgArchPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork archiver: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
+ last_pgarch_start_time = curtime;
- PgArchiverMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgArchPid;
- }
-
- /* shouldn't get here */
return 0;
}
-/* ------------------------------------------------------------
- * Local functions called by archiver follow
- * ------------------------------------------------------------
- */
-
-
-#ifdef EXEC_BACKEND
-
-/*
- * pgarch_forkexec() -
- *
- * Format up the arglist for, then fork and exec, archive process
- */
-static pid_t
-pgarch_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
-
- av[ac++] = "--forkarch";
-
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
/*
* PgArchiverMain
*
* The argc/argv parameters are valid only in EXEC_BACKEND case. However,
* since we don't use 'em, it hardly matters...
*/
-NON_EXEC_STATIC void
+void
PgArchiverMain(int argc, char *argv[])
{
/*
@@ -238,7 +166,7 @@ PgArchiverMain(int argc, char *argv[])
pqsignal(SIGCHLD, SIG_DFL);
PG_SETMASK(&UnBlockSig);
- MyBackendType = B_ARCHIVER;
+ /* Identify myself via ps */
init_ps_display(NULL);
pgarch_MainLoop();
@@ -246,6 +174,11 @@ PgArchiverMain(int argc, char *argv[])
exit(0);
}
+/* ------------------------------------------------------------
+ * Local functions called by archiver follow
+ * ------------------------------------------------------------
+ */
+
/* SIGQUIT signal handler for archiver process */
static void
pgarch_exit(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index e2ad879452..88d010ceb1 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1785,7 +1785,7 @@ ServerLoop(void)
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
/* If we need to signal the autovacuum launcher, do so now */
if (avlauncher_needs_signal)
@@ -3055,7 +3055,7 @@ reaper(SIGNAL_ARGS)
if (!IsBinaryUpgrade && AutoVacuumingActive() && AutoVacPID == 0)
AutoVacPID = StartSubprocess(AutoVacuumLauncherType);
if (PgArchStartupAllowed() && PgArchPID == 0)
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
if (PgStatPID == 0)
PgStatPID = StartSubprocess(PgstatCollectorType);
@@ -3203,7 +3203,7 @@ reaper(SIGNAL_ARGS)
LogChildExit(LOG, _("archiver process"),
pid, exitstatus);
if (PgArchStartupAllowed())
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
continue;
}
@@ -5063,12 +5063,6 @@ SubPostmasterMain(int argc, char *argv[])
StartBackgroundWorker();
}
- if (strcmp(argv[1], "--forkarch") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgArchiverMain(argc, argv); /* does not return */
- }
if (strcmp(argv[1], "--forklog") == 0)
{
/* Do not want to attach to shared memory */
@@ -5197,7 +5191,7 @@ sigusr1_handler(SIGNAL_ARGS)
*/
Assert(PgArchPID == 0);
if (XLogArchivingAlways())
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
/*
* If we aren't planning to enter hot standby mode later, treat
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index ee44ef0789..f9d71a627c 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -15,6 +15,7 @@
#include "pgstat.h"
#include "postmaster/autovacuum.h"
#include "postmaster/bgwriter.h"
+#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
#include "postmaster/startup.h"
#include "postmaster/subprocess.h"
@@ -114,6 +115,15 @@ static PgSubprocess process_types[] = {
.fork_prep = PgstatCollectorPrep,
.entrypoint = PgstatCollectorMain,
.fork_failure = NULL
+ },
+ {
+ .name = "arch",
+ .desc = "archiver",
+ .needs_aux_proc = false,
+ .needs_shmem = false,
+ .fork_prep = PgArchiverPrep,
+ .entrypoint = PgArchiverMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/postmaster/pgarch.h b/src/include/postmaster/pgarch.h
index b3200874ca..ac527212b1 100644
--- a/src/include/postmaster/pgarch.h
+++ b/src/include/postmaster/pgarch.h
@@ -26,14 +26,7 @@
#define MAX_XFN_CHARS 40
#define VALID_XFN_CHARS "0123456789ABCDEF.history.backup.partial"
-/* ----------
- * Functions called from postmaster
- * ----------
- */
-extern int pgarch_start(void);
-
-#ifdef EXEC_BACKEND
+extern int PgArchiverPrep(int argc, char *argv[]);
extern void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
#endif /* _PGARCH_H */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index a7ab036b7b..3f00f904dc 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -26,6 +26,7 @@ typedef enum
AutoVacuumLauncherType,
AutoVacuumWorkerType,
PgstatCollectorType,
+ PgArchiverType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.21.1
0007-Add-SysLoggerType-to-subprocess-struct.patch (text/x-patch, 16.9K) download
From 9e7065d0f37ef6042e5604162e74aeed2df34d42 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 11 Mar 2020 02:58:08 +0000
Subject: [PATCH 07/11] Add SysLoggerType to subprocess struct
This commit introduces two new fields to the subprocess struct:
1) keep_postmaster_memcontext allows subprocesses to specify whether or
not the PostmasterMemoryContext should be kept around after a fork.
2) postmaster_main allows postmaster to gain control to do some
necessary processing after a successful fork.
---
src/backend/postmaster/postmaster.c | 39 +++--
src/backend/postmaster/subprocess.c | 56 +++++--
src/backend/postmaster/syslogger.c | 228 +++++++++++-----------------
src/include/postmaster/subprocess.h | 18 ++-
src/include/postmaster/syslogger.h | 5 +-
5 files changed, 177 insertions(+), 169 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 88d010ceb1..f724727b1e 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1083,7 +1083,7 @@ PostmasterMain(int argc, char *argv[])
/*
* If enabled, start up syslogger collection subprocess
*/
- SysLoggerPID = SysLogger_Start();
+ SysLoggerPID = StartSubprocess(SysLoggerType);
/*
* Reset whereToSendOutput from DestDebug (its starting state) to
@@ -1739,7 +1739,7 @@ ServerLoop(void)
/* If we have lost the log collector, try to start a new one */
if (SysLoggerPID == 0 && Logging_collector)
- SysLoggerPID = SysLogger_Start();
+ SysLoggerPID = StartSubprocess(SysLoggerType);
/*
* If no background writer process is running, and we are not in a
@@ -3228,7 +3228,7 @@ reaper(SIGNAL_ARGS)
{
SysLoggerPID = 0;
/* for safety's sake, launch new logger *first* */
- SysLoggerPID = SysLogger_Start();
+ SysLoggerPID = StartSubprocess(SysLoggerType);
if (!EXIT_STATUS_0(exitstatus))
LogChildExit(LOG, _("system logger process"),
pid, exitstatus);
@@ -5063,12 +5063,6 @@ SubPostmasterMain(int argc, char *argv[])
StartBackgroundWorker();
}
- if (strcmp(argv[1], "--forklog") == 0)
- {
- /* Do not want to attach to shared memory */
-
- SysLoggerMain(argc, argv); /* does not return */
- }
if (MySubprocess->needs_aux_proc)
{
@@ -5446,6 +5440,15 @@ StartSubprocess(SubprocessType type)
snprintf(typebuf, sizeof(typebuf), "-x%d", type);
argv[argc++] = typebuf;
+ /* Prep subprocess for forking */
+ if (MySubprocess->fork_prep)
+ {
+ if (MySubprocess->fork_prep(argc, argv))
+ {
+ return 0;
+ }
+ }
+
argv[argc] = NULL;
Assert(argc < lengthof(argv));
@@ -5461,10 +5464,16 @@ StartSubprocess(SubprocessType type)
/* Close the postmaster's sockets */
ClosePostmasterPorts(false);
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
+ /*
+ * Release postmaster's working memory context if the subprocess doesn't
+ * need it
+ */
+ if (!MySubprocess->keep_postmaster_memcontext)
+ {
+ MemoryContextSwitchTo(TopMemoryContext);
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+ }
/*
* Drop connection to postmaster's shared memory if the subprocess
@@ -5508,6 +5517,10 @@ StartSubprocess(SubprocessType type)
return 0;
}
+ /* Some subprocesses need to do more work in postmaster. */
+ if (MySubprocess->postmaster_main)
+ MySubprocess->postmaster_main(argc, argv);
+
/*
* in parent, successful fork
*/
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index f9d71a627c..f9d07de4ae 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -19,6 +19,7 @@
#include "postmaster/postmaster.h"
#include "postmaster/startup.h"
#include "postmaster/subprocess.h"
+#include "postmaster/syslogger.h"
#include "postmaster/walwriter.h"
#include "replication/walreceiver.h"
@@ -31,99 +32,132 @@ static PgSubprocess process_types[] = {
.desc = "checker",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = CheckerModeMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "boot",
.desc = "bootstrap",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = BootstrapModeMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "boot",
.desc = "startup",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = StartupProcessMain,
- .fork_failure = StartupForkFailure
+ .fork_failure = StartupForkFailure,
+ .postmaster_main = NULL
},
{
.name = "boot",
.desc = "background writer",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = BackgroundWriterMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "boot",
.desc = "checkpointer",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = CheckpointerMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "boot",
.desc = "wal writer",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = WalWriterMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "boot",
.desc = "wal receiver",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = WalReceiverMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "avlauncher",
.desc = "autovacuum launcher",
.needs_aux_proc = false,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = AutoVacLauncherMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "avworker",
.desc = "autovacuum worker",
.needs_aux_proc = false,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = AutoVacWorkerMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "col",
.desc = "statistics collector",
.needs_aux_proc = false,
.needs_shmem = false,
+ .keep_postmaster_memcontext = false,
.fork_prep = PgstatCollectorPrep,
.entrypoint = PgstatCollectorMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "arch",
.desc = "archiver",
.needs_aux_proc = false,
.needs_shmem = false,
+ .keep_postmaster_memcontext = false,
.fork_prep = PgArchiverPrep,
.entrypoint = PgArchiverMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
+ },
+ {
+ .name = "log",
+ .desc = "syslogger",
+ .needs_aux_proc = false,
+ .needs_shmem = false,
+ .keep_postmaster_memcontext = true,
+ .fork_prep = SysLoggerPrep,
+ .entrypoint = SysLoggerMain,
+ .fork_failure = NULL,
+ .postmaster_main = SysLoggerPostmasterMain
}
};
diff --git a/src/backend/postmaster/syslogger.c b/src/backend/postmaster/syslogger.c
index ffcb54968f..0fa0fadcbf 100644
--- a/src/backend/postmaster/syslogger.c
+++ b/src/backend/postmaster/syslogger.c
@@ -38,8 +38,8 @@
#include "nodes/pg_list.h"
#include "pgstat.h"
#include "pgtime.h"
-#include "postmaster/fork_process.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "postmaster/syslogger.h"
#include "storage/dsm.h"
#include "storage/fd.h"
@@ -129,10 +129,8 @@ static volatile sig_atomic_t rotation_requested = false;
/* Local subroutines */
#ifdef EXEC_BACKEND
-static pid_t syslogger_forkexec(void);
static void syslogger_parseArgs(int argc, char *argv[]);
#endif
-NON_EXEC_STATIC void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
static void process_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static void flush_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static FILE *logfile_open(const char *filename, const char *mode,
@@ -153,7 +151,7 @@ static void update_metainfo_datafile(void);
* Main entry point for syslogger process
* argc/argv parameters are valid only in EXEC_BACKEND case.
*/
-NON_EXEC_STATIC void
+void
SysLoggerMain(int argc, char *argv[])
{
#ifndef WIN32
@@ -172,7 +170,6 @@ SysLoggerMain(int argc, char *argv[])
syslogger_parseArgs(argc, argv);
#endif /* EXEC_BACKEND */
- MyBackendType = B_LOGGER;
init_ps_display(NULL);
/*
@@ -537,13 +534,17 @@ SysLoggerMain(int argc, char *argv[])
* Postmaster subroutine to start a syslogger subprocess.
*/
int
-SysLogger_Start(void)
+SysLoggerPrep(int argc, char *argv[])
{
- pid_t sysloggerPid;
char *filename;
+#ifdef EXEC_BACKEND
+ char filenobuf[32];
+ char csvfilenobuf[32];
+#endif
+ /* Tell postmaster to stop forking */
if (!Logging_collector)
- return 0;
+ return -1;
/*
* If first time through, create the pipe which will receive stderr
@@ -626,129 +627,6 @@ SysLogger_Start(void)
}
#ifdef EXEC_BACKEND
- switch ((sysloggerPid = syslogger_forkexec()))
-#else
- switch ((sysloggerPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork system logger: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(true);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- /* do the work */
- SysLoggerMain(0, NULL);
- break;
-#endif
-
- default:
- /* success, in postmaster */
-
- /* now we redirect stderr, if not done already */
- if (!redirection_done)
- {
-#ifdef WIN32
- int fd;
-#endif
-
- /*
- * Leave a breadcrumb trail when redirecting, in case the user
- * forgets that redirection is active and looks only at the
- * original stderr target file.
- */
- ereport(LOG,
- (errmsg("redirecting log output to logging collector process"),
- errhint("Future log output will appear in directory \"%s\".",
- Log_directory)));
-
-#ifndef WIN32
- fflush(stdout);
- if (dup2(syslogPipe[1], fileno(stdout)) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stdout: %m")));
- fflush(stderr);
- if (dup2(syslogPipe[1], fileno(stderr)) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- /* Now we are done with the write end of the pipe. */
- close(syslogPipe[1]);
- syslogPipe[1] = -1;
-#else
-
- /*
- * open the pipe in binary mode and make sure stderr is binary
- * after it's been dup'ed into, to avoid disturbing the pipe
- * chunking protocol.
- */
- fflush(stderr);
- fd = _open_osfhandle((intptr_t) syslogPipe[1],
- _O_APPEND | _O_BINARY);
- if (dup2(fd, _fileno(stderr)) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- close(fd);
- _setmode(_fileno(stderr), _O_BINARY);
-
- /*
- * Now we are done with the write end of the pipe.
- * CloseHandle() must not be called because the preceding
- * close() closes the underlying handle.
- */
- syslogPipe[1] = 0;
-#endif
- redirection_done = true;
- }
-
- /* postmaster will never write the file(s); close 'em */
- fclose(syslogFile);
- syslogFile = NULL;
- if (csvlogFile != NULL)
- {
- fclose(csvlogFile);
- csvlogFile = NULL;
- }
- return (int) sysloggerPid;
- }
-
- /* we should never reach here */
- return 0;
-}
-
-
-#ifdef EXEC_BACKEND
-
-/*
- * syslogger_forkexec() -
- *
- * Format up the arglist for, then fork and exec, a syslogger process
- */
-static pid_t
-syslogger_forkexec(void)
-{
- char *av[10];
- int ac = 0;
- char filenobuf[32];
- char csvfilenobuf[32];
-
- av[ac++] = "postgres";
- av[ac++] = "--forklog";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
/* static variables (those not passed by write_backend_variables) */
#ifndef WIN32
if (syslogFile != NULL)
@@ -763,7 +641,7 @@ syslogger_forkexec(void)
else
strcpy(filenobuf, "0");
#endif /* WIN32 */
- av[ac++] = filenobuf;
+ argv[argc++] = filenobuf;
#ifndef WIN32
if (csvlogFile != NULL)
@@ -778,14 +656,92 @@ syslogger_forkexec(void)
else
strcpy(csvfilenobuf, "0");
#endif /* WIN32 */
- av[ac++] = csvfilenobuf;
+ argv[argc++] = csvfilenobuf;
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+#endif /* EXEC_BACKEND */
- return postmaster_forkexec(ac, av);
+ return 0;
}
+/*
+ * SysLoggerPostmasterMain
+ *
+ * Control function for subprocess parent (postmaster) after a successful fork.
+ */
+void
+SysLoggerPostmasterMain(int argc, char *argv[])
+{
+ /* success, in postmaster */
+
+ /* now we redirect stderr, if not done already */
+ if (!redirection_done)
+ {
+#ifdef WIN32
+ int fd;
+#endif
+
+ /*
+ * Leave a breadcrumb trail when redirecting, in case the user
+ * forgets that redirection is active and looks only at the
+ * original stderr target file.
+ */
+ ereport(LOG,
+ (errmsg("redirecting log output to logging collector process"),
+ errhint("Future log output will appear in directory \"%s\".",
+ Log_directory)));
+
+#ifndef WIN32
+ fflush(stdout);
+ if (dup2(syslogPipe[1], fileno(stdout)) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stdout: %m")));
+ fflush(stderr);
+ if (dup2(syslogPipe[1], fileno(stderr)) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ /* Now we are done with the write end of the pipe. */
+ close(syslogPipe[1]);
+ syslogPipe[1] = -1;
+#else
+
+ /*
+ * open the pipe in binary mode and make sure stderr is binary
+ * after it's been dup'ed into, to avoid disturbing the pipe
+ * chunking protocol.
+ */
+ fflush(stderr);
+ fd = _open_osfhandle((intptr_t) syslogPipe[1],
+ _O_APPEND | _O_BINARY);
+ if (dup2(fd, _fileno(stderr)) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ close(fd);
+ _setmode(_fileno(stderr), _O_BINARY);
+
+ /*
+ * Now we are done with the write end of the pipe.
+ * CloseHandle() must not be called because the preceding
+ * close() closes the underlying handle.
+ */
+ syslogPipe[1] = 0;
+#endif
+ redirection_done = true;
+ }
+
+ /* postmaster will never write the file(s); close 'em */
+ fclose(syslogFile);
+ syslogFile = NULL;
+ if (csvlogFile != NULL)
+ {
+ fclose(csvlogFile);
+ csvlogFile = NULL;
+ }
+}
+
+#ifdef EXEC_BACKEND
/*
* syslogger_parseArgs() -
*
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 3f00f904dc..d0d98a5d69 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -27,6 +27,7 @@ typedef enum
AutoVacuumWorkerType,
PgstatCollectorType,
PgArchiverType,
+ SysLoggerType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
@@ -34,19 +35,22 @@ typedef enum
typedef int (*SubprocessPrep) (int argc, char *argv[]);
typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
typedef bool (*SubprocessForkFailure) (int fork_errno);
+typedef void (*SubprocessPostmasterMain) (int argc, char *argv[]);
/* Current subprocess initializer */
extern void InitMySubprocess(SubprocessType type);
typedef struct PgSubprocess
{
- const char *name;
- const char *desc;
- bool needs_aux_proc;
- bool needs_shmem;
- SubprocessPrep fork_prep;
- SubprocessEntryPoint entrypoint;
- SubprocessForkFailure fork_failure;
+ const char *name;
+ const char *desc;
+ bool needs_aux_proc;
+ bool needs_shmem;
+ bool keep_postmaster_memcontext;
+ SubprocessPrep fork_prep;
+ SubprocessEntryPoint entrypoint;
+ SubprocessForkFailure fork_failure;
+ SubprocessPostmasterMain postmaster_main;
} PgSubprocess;
extern SubprocessType MySubprocessType;
diff --git a/src/include/postmaster/syslogger.h b/src/include/postmaster/syslogger.h
index f611bd1411..a498c73121 100644
--- a/src/include/postmaster/syslogger.h
+++ b/src/include/postmaster/syslogger.h
@@ -81,9 +81,10 @@ extern int SysLogger_Start(void);
extern void write_syslogger_file(const char *buffer, int count, int dest);
-#ifdef EXEC_BACKEND
+/* Subprocess startup functions */
+extern int SysLoggerPrep(int argc, char *argv[]);
extern void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+extern void SysLoggerPostmasterMain(int argc, char *argv[]);
extern bool CheckLogrotateSignal(void);
extern void RemoveLogrotateSignalFiles(void);
--
2.21.1
0008-Add-BgWorkerType-to-subprocess-struct.patch (text/x-patch, 25.6K) download
From 6b58297612ea18f1943feee3cda0ef7d038471e6 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 11 Mar 2020 03:32:41 +0000
Subject: [PATCH 08/11] Add BgWorkerType to subprocess struct
Export some of the backend-specific code and move the some bgworker
startup code out to src/backend/postmaster/bgworker.c.
---
src/backend/postmaster/bgworker.c | 165 ++++++++++-
src/backend/postmaster/postmaster.c | 294 +++-----------------
src/backend/postmaster/subprocess.c | 12 +
src/backend/utils/init/globals.c | 1 +
src/include/miscadmin.h | 1 +
src/include/postmaster/bgworker_internals.h | 8 +-
src/include/postmaster/postmaster.h | 53 ++++
src/include/postmaster/subprocess.h | 1 +
8 files changed, 268 insertions(+), 267 deletions(-)
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index 6c684b5e12..6f62cd4889 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -33,8 +33,10 @@
#include "storage/shmem.h"
#include "tcop/tcopprot.h"
#include "utils/ascii.h"
+#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/timeout.h"
+#include "utils/timestamp.h"
/*
* The postmaster's list of registered background workers, in private memory.
@@ -133,7 +135,55 @@ static const struct
/* Private functions. */
static bgworker_main_type LookupBackgroundWorkerFunction(const char *libraryname, const char *funcname);
+static bool assign_backendlist_entry(RegisteredBgWorker *rw);
+/*
+ * Allocate the Backend struct for a connected background worker, but don't
+ * add it to the list of backends just yet.
+ *
+ * On failure, return false without changing any worker state.
+ *
+ * Some info from the Backend is copied into the passed rw.
+ */
+static bool
+assign_backendlist_entry(RegisteredBgWorker *rw)
+{
+ Backend *bn;
+
+ /*
+ * Compute the cancel key that will be assigned to this session. We
+ * probably don't need cancel keys for background workers, but we'd better
+ * have something random in the field to prevent unfriendly people from
+ * sending cancels to them.
+ */
+ if (!RandomCancelKey(&MyCancelKey))
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random cancel key")));
+ return false;
+ }
+
+ bn = malloc(sizeof(Backend));
+ if (bn == NULL)
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+ return false;
+ }
+
+ bn->cancel_key = MyCancelKey;
+ bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
+ bn->bkend_type = BACKEND_TYPE_BGWORKER;
+ bn->dead_end = false;
+ bn->bgworker_notify = false;
+
+ rw->rw_backend = bn;
+ rw->rw_child_slot = bn->child_slot;
+
+ return true;
+}
/*
* Calculate shared memory needed.
@@ -670,6 +720,55 @@ bgworker_sigusr1_handler(SIGNAL_ARGS)
errno = save_errno;
}
+/*
+ * BackgroundWorkerPrep
+ *
+ * Postmaster subroutine to prepare a bgworker subprocess.
+ *
+ * Returns 0 on success or -1 on failure.
+ *
+ * Note: if fail, we will be called again from the postmaster main loop.
+ */
+int
+BackgroundWorkerPrep(int argc, char *argv[])
+{
+ RegisteredBgWorker *rw = MyRegisteredBgWorker;
+#ifdef EXEC_BACKEND
+ char forkav[MAXPGPATH];
+#endif
+
+ Assert(MyRegisteredBgWorker);
+ Assert(rw->rw_pid == 0);
+
+ /*
+ * Allocate and assign the Backend element. Note we must do this before
+ * forking, so that we can handle out of memory properly.
+ *
+ * Treat failure as though the worker had crashed. That way, the
+ * postmaster will wait a bit before attempting to start it again; if it
+ * tried again right away, most likely it'd find itself repeating the
+ * out-of-memory or fork failure condition.
+ */
+ if (!assign_backendlist_entry(rw))
+ {
+ rw->rw_crashed_at = GetCurrentTimestamp();
+ return -1;
+ }
+
+ ereport(DEBUG1,
+ (errmsg("starting background worker process \"%s\"",
+ rw->rw_worker.bgw_name)));
+
+#ifdef EXEC_BACKEND
+ /* Rewrite bgworker fork parameter with shmem_slot */
+ snprintf(forkav, MAXPGPATH, "%s=%d", argv[1], rw->rw_shmem_slot);
+ argv[1] = forkav;
+#endif
+
+ return 0;
+}
+
+
/*
* Start a new background worker
*
@@ -677,12 +776,28 @@ bgworker_sigusr1_handler(SIGNAL_ARGS)
* postmaster.
*/
void
-StartBackgroundWorker(void)
+BackgroundWorkerMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
- BackgroundWorker *worker = MyBgworkerEntry;
+ BackgroundWorker *worker;
bgworker_main_type entrypt;
+#ifndef EXEC_BACKEND
+ /*
+ * Before blowing away PostmasterContext, save this bgworker's
+ * data where it can find it.
+ */
+ MyBgworkerEntry = (BackgroundWorker *)
+ MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
+ memcpy(MyBgworkerEntry, &(MyRegisteredBgWorker->rw_worker), sizeof(BackgroundWorker));
+
+ /* Now we can release postmaster's working memory context */
+ MemoryContextSwitchTo(TopMemoryContext);
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+#endif
+
+ worker = MyBgworkerEntry;
if (worker == NULL)
elog(FATAL, "unable to find bgworker entry");
@@ -816,6 +931,52 @@ StartBackgroundWorker(void)
proc_exit(0);
}
+/*
+ * BackgroundWorkerForkFailure
+ *
+ * Handle bgworker fork failure gracefully.
+ */
+bool
+BackgroundWorkerForkFailure(int fork_errno)
+{
+ RegisteredBgWorker *rw = MyRegisteredBgWorker;
+
+ errno = fork_errno;
+
+ /* in postmaster, fork failed ... */
+ ereport(LOG,
+ (errmsg("could not fork worker process: %m")));
+
+ /* undo what assign_backendlist_entry did */
+ ReleasePostmasterChildSlot(rw->rw_child_slot);
+ rw->rw_child_slot = 0;
+ free(rw->rw_backend);
+ rw->rw_backend = NULL;
+ /* mark entry as crashed, so we'll try again later */
+ rw->rw_crashed_at = GetCurrentTimestamp();
+
+ /* Don't panic. */
+ return false;
+}
+
+/*
+ * BackgroundWorkerPostmasterMain
+ */
+void
+BackgroundWorkerPostmasterMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ RegisteredBgWorker *rw = MyRegisteredBgWorker;
+
+ rw->rw_pid = MyChildProcPid;
+ rw->rw_backend->pid = rw->rw_pid;
+ ReportBackgroundWorkerPID(rw);
+ /* add new worker to lists of backends */
+ dlist_push_head(&BackendList, &(rw->rw_backend->elem));
+#ifdef EXEC_BACKEND
+ ShmemBackendArrayAdd(rw->rw_backend);
+#endif
+}
+
/*
* Register a new static background worker.
*
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index f724727b1e..8343a81923 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -137,60 +137,14 @@
#include "storage/spin.h"
#endif
-
-/*
- * Possible types of a backend. Beyond being the possible bkend_type values in
- * struct bkend, these are OR-able request flag bits for SignalSomeChildren()
- * and CountChildren().
- */
-#define BACKEND_TYPE_NORMAL 0x0001 /* normal backend */
-#define BACKEND_TYPE_AUTOVAC 0x0002 /* autovacuum worker process */
-#define BACKEND_TYPE_WALSND 0x0004 /* walsender process */
-#define BACKEND_TYPE_BGWORKER 0x0008 /* bgworker process */
-#define BACKEND_TYPE_ALL 0x000F /* OR of all the above */
-
-#define BACKEND_TYPE_WORKER (BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER)
-
-/*
- * List of active backends (or child processes anyway; we don't actually
- * know whether a given child has become a backend or is still in the
- * authorization phase). This is used mainly to keep track of how many
- * children we have and send them appropriate signals when necessary.
- *
- * "Special" children such as the startup, bgwriter and autovacuum launcher
- * tasks are not in this list. Autovacuum worker and walsender are in it.
- * Also, "dead_end" children are in it: these are children launched just for
- * the purpose of sending a friendly rejection message to a would-be client.
- * We must track them because they are attached to shared memory, but we know
- * they will never become live backends. dead_end children are not assigned a
- * PMChildSlot.
- *
- * Background workers are in this list, too.
- */
-typedef struct bkend
-{
- pid_t pid; /* process id of backend */
- int32 cancel_key; /* cancel key for cancels for this backend */
- int child_slot; /* PMChildSlot for this backend, if any */
-
- /*
- * Flavor of backend or auxiliary process. Note that BACKEND_TYPE_WALSND
- * backends initially announce themselves as BACKEND_TYPE_NORMAL, so if
- * bkend_type is normal, you should check for a recent transition.
- */
- int bkend_type;
- bool dead_end; /* is it going to send an error and quit? */
- bool bgworker_notify; /* gets bgworker start/stop notifications */
- dlist_node elem; /* list link in BackendList */
-} Backend;
-
-static dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
+dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
#ifdef EXEC_BACKEND
static Backend *ShmemBackendArray;
#endif
BackgroundWorker *MyBgworkerEntry = NULL;
+RegisteredBgWorker *MyRegisteredBgWorker = NULL;
/* Struct containing postmaster subprocess control info */
SubprocessType MySubprocessType;
@@ -412,7 +366,6 @@ static void processCancelRequest(Port *port, void *pkt);
static int initMasks(fd_set *rmask);
static void report_fork_failure_to_client(Port *port, int errnum);
static CAC_state canAcceptConnections(int backend_type);
-static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
@@ -420,7 +373,6 @@ static void TerminateChildren(int signal);
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
-static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
static pid_t StartSubprocess(SubprocessType type);
@@ -539,10 +491,10 @@ static bool save_backend_variables(BackendParameters *param, Port *port,
HANDLE childProcess, pid_t childPid);
#endif
-static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
+#define BGWORKER_LEN 15
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -4878,6 +4830,7 @@ void
SubPostmasterMain(int argc, char *argv[])
{
Port port;
+ int shmem_slot;
/* In EXEC_BACKEND case we will not have inherited these settings */
IsPostmasterEnvironment = true;
@@ -5041,59 +4994,48 @@ SubPostmasterMain(int argc, char *argv[])
/* And run the backend */
BackendRun(&port); /* does not return */
}
- if (strncmp(argv[1], "--forkbgworker=", 15) == 0)
- {
- int shmem_slot;
-
- /* do this as early as possible; in particular, before InitProcess() */
- IsBackgroundWorker = true;
+ if (MySubprocess->needs_aux_proc)
+ {
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
/* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
+ InitAuxiliaryProcess();
/* Attach process to shared data structures */
CreateSharedMemoryAndSemaphores();
- /* Fetch MyBgworkerEntry from shared memory */
- shmem_slot = atoi(argv[1] + 15);
- MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
+ AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
+ }
- StartBackgroundWorker();
+ if (MySubprocessType == BgWorkerType)
+ {
+ /* do this as early as possible; in particular, before InitProcess() */
+ IsBackgroundWorker = true;
}
- if (MySubprocess->needs_aux_proc)
+ if (MySubprocess->needs_shmem)
{
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
/* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitAuxiliaryProcess();
+ InitProcess();
/* Attach process to shared data structures */
CreateSharedMemoryAndSemaphores();
-
- AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
}
- else
- {
- if (MySubprocess->needs_shmem)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
- }
-
- MySubprocess->entrypoint(argc - 2, argv + 2);
+ if (MySubprocessType == BgWorkerType)
+ {
+ /* Fetch MyBgworkerEntry from shared memory */
+ shmem_slot = atoi(argv[1] + 15);
+ MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
}
+ MySubprocess->entrypoint(argc - 2, argv + 2);
+
abort(); /* shouldn't get here */
}
#endif /* EXEC_BACKEND */
@@ -5356,7 +5298,7 @@ StartupPacketTimeoutHandler(void)
/*
* Generate a random cancel key.
*/
-static bool
+bool
RandomCancelKey(int32 *cancel_key)
{
return pg_strong_random(cancel_key, sizeof(int32));
@@ -5454,15 +5396,17 @@ StartSubprocess(SubprocessType type)
#ifdef EXEC_BACKEND
pid = postmaster_forkexec(argc, argv);
+ MyChildProcPid = pid;
#else /* !EXEC_BACKEND */
pid = fork_process();
+ MyChildProcPid = pid;
if (pid == 0) /* child */
{
InitPostmasterChild();
/* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
+ ClosePostmasterPorts(MySubprocessType == SysLoggerType);
/*
* Release postmaster's working memory context if the subprocess doesn't
@@ -5512,6 +5456,8 @@ StartSubprocess(SubprocessType type)
{
if (MySubprocess->fork_failure(save_errno))
ExitPostmaster(1);
+ else
+ return -1;
}
return 0;
@@ -5754,124 +5700,6 @@ BackgroundWorkerUnblockSignals(void)
PG_SETMASK(&UnBlockSig);
}
-#ifdef EXEC_BACKEND
-static pid_t
-bgworker_forkexec(int shmem_slot)
-{
- char *av[10];
- int ac = 0;
- char forkav[MAXPGPATH];
-
- snprintf(forkav, MAXPGPATH, "--forkbgworker=%d", shmem_slot);
-
- av[ac++] = "postgres";
- av[ac++] = forkav;
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif
-
-/*
- * Start a new bgworker.
- * Starting time conditions must have been checked already.
- *
- * Returns true on success, false on failure.
- * In either case, update the RegisteredBgWorker's state appropriately.
- *
- * This code is heavily based on autovacuum.c, q.v.
- */
-static bool
-do_start_bgworker(RegisteredBgWorker *rw)
-{
- pid_t worker_pid;
-
- Assert(rw->rw_pid == 0);
-
- /*
- * Allocate and assign the Backend element. Note we must do this before
- * forking, so that we can handle failures (out of memory or child-process
- * slots) cleanly.
- *
- * Treat failure as though the worker had crashed. That way, the
- * postmaster will wait a bit before attempting to start it again; if we
- * tried again right away, most likely we'd find ourselves hitting the
- * same resource-exhaustion condition.
- */
- if (!assign_backendlist_entry(rw))
- {
- rw->rw_crashed_at = GetCurrentTimestamp();
- return false;
- }
-
- ereport(DEBUG1,
- (errmsg("starting background worker process \"%s\"",
- rw->rw_worker.bgw_name)));
-
-#ifdef EXEC_BACKEND
- switch ((worker_pid = bgworker_forkexec(rw->rw_shmem_slot)))
-#else
- switch ((worker_pid = fork_process()))
-#endif
- {
- case -1:
- /* in postmaster, fork failed ... */
- ereport(LOG,
- (errmsg("could not fork worker process: %m")));
- /* undo what assign_backendlist_entry did */
- ReleasePostmasterChildSlot(rw->rw_child_slot);
- rw->rw_child_slot = 0;
- free(rw->rw_backend);
- rw->rw_backend = NULL;
- /* mark entry as crashed, so we'll try again later */
- rw->rw_crashed_at = GetCurrentTimestamp();
- break;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /*
- * Before blowing away PostmasterContext, save this bgworker's
- * data where it can find it.
- */
- MyBgworkerEntry = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(MyBgworkerEntry, &rw->rw_worker, sizeof(BackgroundWorker));
-
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
-
- StartBackgroundWorker();
-
- exit(1); /* should not get here */
- break;
-#endif
- default:
- /* in postmaster, fork successful ... */
- rw->rw_pid = worker_pid;
- rw->rw_backend->pid = rw->rw_pid;
- ReportBackgroundWorkerPID(rw);
- /* add new worker to lists of backends */
- dlist_push_head(&BackendList, &rw->rw_backend->elem);
-#ifdef EXEC_BACKEND
- ShmemBackendArrayAdd(rw->rw_backend);
-#endif
- return true;
- }
-
- return false;
-}
-
/*
* Does the current postmaster state require starting a worker with the
* specified start_time?
@@ -5912,67 +5740,6 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
return false;
}
-/*
- * Allocate the Backend struct for a connected background worker, but don't
- * add it to the list of backends just yet.
- *
- * On failure, return false without changing any worker state.
- *
- * Some info from the Backend is copied into the passed rw.
- */
-static bool
-assign_backendlist_entry(RegisteredBgWorker *rw)
-{
- Backend *bn;
-
- /*
- * Check that database state allows another connection. Currently the
- * only possible failure is CAC_TOOMANY, so we just log an error message
- * based on that rather than checking the error code precisely.
- */
- if (canAcceptConnections(BACKEND_TYPE_BGWORKER) != CAC_OK)
- {
- ereport(LOG,
- (errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("no slot available for new worker process")));
- return false;
- }
-
- /*
- * Compute the cancel key that will be assigned to this session. We
- * probably don't need cancel keys for background workers, but we'd better
- * have something random in the field to prevent unfriendly people from
- * sending cancels to them.
- */
- if (!RandomCancelKey(&MyCancelKey))
- {
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("could not generate random cancel key")));
- return false;
- }
-
- bn = malloc(sizeof(Backend));
- if (bn == NULL)
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- return false;
- }
-
- bn->cancel_key = MyCancelKey;
- bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
- bn->bkend_type = BACKEND_TYPE_BGWORKER;
- bn->dead_end = false;
- bn->bgworker_notify = false;
-
- rw->rw_backend = bn;
- rw->rw_child_slot = bn->child_slot;
-
- return true;
-}
-
/*
* If the time is right, start background worker(s).
*
@@ -6077,7 +5844,8 @@ maybe_start_bgworkers(void)
* crashed, but there's no need because the next run of this
* function will do that.
*/
- if (!do_start_bgworker(rw))
+ MyRegisteredBgWorker = rw;
+ if (StartSubprocess(BgWorkerType) <= 0)
{
StartWorkerNeeded = true;
return;
@@ -6488,7 +6256,7 @@ ShmemBackendArrayAllocation(void)
memset(ShmemBackendArray, 0, size);
}
-static void
+void
ShmemBackendArrayAdd(Backend *bn)
{
/* The array slot corresponding to my PMChildSlot should be free */
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index f9d07de4ae..d4df791aeb 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -14,6 +14,7 @@
#include "bootstrap/bootstrap.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
+#include "postmaster/bgworker_internals.h"
#include "postmaster/bgwriter.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
@@ -158,6 +159,17 @@ static PgSubprocess process_types[] = {
.entrypoint = SysLoggerMain,
.fork_failure = NULL,
.postmaster_main = SysLoggerPostmasterMain
+ },
+ {
+ .name = "bgworker",
+ .desc = "background worker",
+ .needs_aux_proc = false,
+ .needs_shmem = true,
+ .keep_postmaster_memcontext = true,
+ .fork_prep = BackgroundWorkerPrep,
+ .entrypoint = BackgroundWorkerMain,
+ .fork_failure = BackgroundWorkerForkFailure,
+ .postmaster_main = BackgroundWorkerPostmasterMain
}
};
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index eb19644419..b31ce82f75 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -38,6 +38,7 @@ volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
int MyProcPid;
+int MyChildProcPid;
pg_time_t MyStartTime;
TimestampTz MyStartTimestamp;
struct Port *MyProcPort;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 14fa127ab1..541ca5145d 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -163,6 +163,7 @@ extern PGDLLIMPORT int max_worker_processes;
extern PGDLLIMPORT int max_parallel_workers;
extern PGDLLIMPORT int MyProcPid;
+extern PGDLLIMPORT int MyChildProcPid;
extern PGDLLIMPORT pg_time_t MyStartTime;
extern PGDLLIMPORT TimestampTz MyStartTimestamp;
extern PGDLLIMPORT struct Port *MyProcPort;
diff --git a/src/include/postmaster/bgworker_internals.h b/src/include/postmaster/bgworker_internals.h
index f7e24664d5..874d5a203f 100644
--- a/src/include/postmaster/bgworker_internals.h
+++ b/src/include/postmaster/bgworker_internals.h
@@ -43,6 +43,7 @@ typedef struct RegisteredBgWorker
} RegisteredBgWorker;
extern slist_head BackgroundWorkerList;
+extern RegisteredBgWorker *MyRegisteredBgWorker;
extern Size BackgroundWorkerShmemSize(void);
extern void BackgroundWorkerShmemInit(void);
@@ -53,8 +54,11 @@ extern void ReportBackgroundWorkerExit(slist_mutable_iter *cur);
extern void BackgroundWorkerStopNotifications(pid_t pid);
extern void ResetBackgroundWorkerCrashTimes(void);
-/* Function to start a background worker, called from postmaster.c */
-extern void StartBackgroundWorker(void) pg_attribute_noreturn();
+/* BgWorkerType subprocess */
+extern int BackgroundWorkerPrep(int argc, char *argv[]);
+extern void BackgroundWorkerMain(int argc, char *argv[]);
+extern bool BackgroundWorkerForkFailure(int fork_errno);
+extern void BackgroundWorkerPostmasterMain(int argc, char *argv[]);
#ifdef EXEC_BACKEND
extern BackgroundWorker *BackgroundWorkerEntry(int slotno);
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index babc87dfc9..b1411c1ed2 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -13,6 +13,41 @@
#ifndef _POSTMASTER_H
#define _POSTMASTER_H
+#include "lib/ilist.h"
+
+/*
+ * List of active backends (or child processes anyway; we don't actually
+ * know whether a given child has become a backend or is still in the
+ * authorization phase). This is used mainly to keep track of how many
+ * children we have and send them appropriate signals when necessary.
+ *
+ * "Special" children such as the startup, bgwriter and autovacuum launcher
+ * tasks are not in this list. Autovacuum worker and walsender are in it.
+ * Also, "dead_end" children are in it: these are children launched just for
+ * the purpose of sending a friendly rejection message to a would-be client.
+ * We must track them because they are attached to shared memory, but we know
+ * they will never become live backends. dead_end children are not assigned a
+ * PMChildSlot.
+ *
+ * Background workers are in this list, too.
+ */
+typedef struct bkend
+{
+ pid_t pid; /* process id of backend */
+ int32 cancel_key; /* cancel key for cancels for this backend */
+ int child_slot; /* PMChildSlot for this backend, if any */
+
+ /*
+ * Flavor of backend or auxiliary process. Note that BACKEND_TYPE_WALSND
+ * backends initially announce themselves as BACKEND_TYPE_NORMAL, so if
+ * bkend_type is normal, you should check for a recent transition.
+ */
+ int bkend_type;
+ bool dead_end; /* is it going to send an error and quit? */
+ bool bgworker_notify; /* gets bgworker start/stop notifications */
+ dlist_node elem; /* list link in BackendList */
+} Backend;
+
/* GUC options */
extern bool EnableSSL;
extern int ReservedBackends;
@@ -54,14 +89,19 @@ extern int MaxLivePostmasterChildren(void);
extern bool PostmasterMarkPIDForWorkerNotify(int);
+extern bool RandomCancelKey(int32 *cancel_key);
+
#ifdef EXEC_BACKEND
extern pid_t postmaster_forkexec(int argc, char *argv[]);
extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
extern Size ShmemBackendArraySize(void);
extern void ShmemBackendArrayAllocation(void);
+extern void ShmemBackendArrayAdd(Backend *bn);
#endif
+extern PGDLLIMPORT dlist_head BackendList;
+
/*
* Note: MAX_BACKENDS is limited to 2^18-1 because that's the width reserved
* for buffer references in buf_internals.h. This limitation could be lifted
@@ -74,4 +114,17 @@ extern void ShmemBackendArrayAllocation(void);
*/
#define MAX_BACKENDS 0x3FFFF
+/*
+ * Possible types of a backend. Beyond being the possible bkend_type values in
+ * struct bkend, these are OR-able request flag bits for SignalSomeChildren()
+ * and CountChildren().
+ */
+#define BACKEND_TYPE_NORMAL 0x0001 /* normal backend */
+#define BACKEND_TYPE_AUTOVAC 0x0002 /* autovacuum worker process */
+#define BACKEND_TYPE_WALSND 0x0004 /* walsender process */
+#define BACKEND_TYPE_BGWORKER 0x0008 /* bgworker process */
+#define BACKEND_TYPE_ALL 0x000F /* OR of all the above */
+
+#define BACKEND_TYPE_WORKER (BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER)
+
#endif /* _POSTMASTER_H */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index d0d98a5d69..706d60e4d6 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -28,6 +28,7 @@ typedef enum
PgstatCollectorType,
PgArchiverType,
SysLoggerType,
+ BgWorkerType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.21.1
0009-Add-Backends-to-subprocess-struct.patch (text/x-patch, 16.0K) download
From 06ad41d6c563c0258696eb965706b9f1d24436e2 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Mon, 16 Mar 2020 23:05:33 +0000
Subject: [PATCH 09/11] Add Backends to subprocess struct
---
src/backend/postmaster/postmaster.c | 407 +++++++++++++++-------------
src/backend/postmaster/subprocess.c | 11 +
src/include/postmaster/postmaster.h | 6 +
src/include/postmaster/subprocess.h | 1 +
4 files changed, 235 insertions(+), 190 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 8343a81923..f979d505df 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -359,7 +359,6 @@ static void BackendInitialize(Port *port);
static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
-static int BackendStartup(Port *port);
static int ProcessStartupPacket(Port *port, bool secure_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
@@ -409,7 +408,6 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(Port *port);
static pid_t internal_forkexec(int argc, char *argv[], Port *port);
/* Type for a socket that can be inherited to a client process */
@@ -512,6 +510,8 @@ int postmaster_alive_fds[2] = {-1, -1};
HANDLE PostmasterHandle;
#endif
+static Port *ConnProcPort = NULL;
+
/*
* Postmaster main entry point
*/
@@ -1671,19 +1671,18 @@ ServerLoop(void)
break;
if (FD_ISSET(ListenSocket[i], &rmask))
{
- Port *port;
+ ConnProcPort = ConnCreate(ListenSocket[i]);
- port = ConnCreate(ListenSocket[i]);
- if (port)
+ if (ConnProcPort)
{
- BackendStartup(port);
+ StartSubprocess(ClientBackendType);
/*
* We no longer need the open socket or port structure
* in this process
*/
- StreamClose(port->sock);
- ConnFree(port);
+ StreamClose(ConnProcPort->sock);
+ ConnFree(ConnProcPort);
}
}
}
@@ -3215,6 +3214,174 @@ reaper(SIGNAL_ARGS)
errno = save_errno;
}
+/*
+ * BackendPrep
+ *
+ * Do all of the pre-fork setup for a new Backend.
+*/
+int
+BackendPrep(int argc, char *argv[])
+{
+ /*
+ * Create backend data structure. Better before the fork() so we can
+ * handle failure cleanly.
+ */
+ Backend *bn;
+
+ bn = (Backend *) malloc(sizeof(Backend));
+ if (!bn)
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+ }
+
+ /*
+ * Compute the cancel key that will be assigned to this backend. The
+ * backend will have its own copy in the forked-off process' value of
+ * MyCancelKey, so that it can transmit the key to the frontend.
+ */
+ if (!RandomCancelKey(&MyCancelKey))
+ {
+ free(bn);
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random cancel key")));
+ }
+
+ bn->cancel_key = MyCancelKey;
+
+ /* Pass down canAcceptConnections state */
+ ConnProcPort->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
+ bn->dead_end = (ConnProcPort->canAcceptConnections != CAC_OK &&
+ ConnProcPort->canAcceptConnections != CAC_WAITBACKUP);
+
+ /*
+ * Unless it's a dead_end child, assign it a child slot number
+ */
+ if (!bn->dead_end)
+ bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
+ else
+ bn->child_slot = 0;
+
+ /* Hasn't asked to be notified about any bgworkers yet */
+ bn->bgworker_notify = false;
+
+ /*
+ * Push the Backend to the stack as-is so it can be retreived by the parent
+ * on the other side of the fork/exec. They will modify this entry in place.
+ */
+ dlist_push_head(&BackendList, &bn->elem);
+
+ return 0;
+}
+
+/*
+ * BackendMain
+ *
+ * Child code when forking a Backend.
+ */
+void BackendMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ /*
+ * Perform additional initialization and collect startup packet.
+ *
+ * We want to do this before InitProcess() for a couple of reasons: 1.
+ * so that we aren't eating up a PGPROC slot while waiting on the
+ * client. 2. so that if InitProcess() fails due to being out of
+ * PGPROC slots, we have already initialized libpq and are able to
+ * report the error to the client.
+ */
+#ifndef EXEC_BACKEND
+ BackendInitialize(ConnProcPort);
+#endif
+
+ /* And run the backend */
+ BackendRun(ConnProcPort); /* does not return */
+}
+
+/*
+ * BackendPostmasterMain
+ *
+ * Parent code when forking a Backend.
+ */
+void BackendPostmasterMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ dlist_mutable_iter iter;
+
+ /*
+ * Find our backend node by popping the element with a matching child slot
+ * off the list. This was pushed to the stack (incomplete) by BackendPrep.
+ * We need to fix it up and push back.
+ */
+ dlist_foreach_modify(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ /* MyPMChildSlot is guaranteed to match the child MyPMChildSlot */
+ if (bp->child_slot == MyPMChildSlot)
+ {
+ /*
+ * Everything's been successful, it's safe to add this backend to our list
+ * of backends.
+ */
+ bp->pid = MyChildProcPid;
+ bp->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
+
+#ifdef EXEC_BACKEND
+ if (!bp->dead_end)
+ ShmemBackendArrayAdd(bp);
+#endif
+ break; /* There is only one entry and we've found it */
+ }
+ }
+
+ /* in parent, successful fork */
+ ereport(DEBUG2,
+ (errmsg_internal("forked new backend, pid=%d socket=%d",
+ (int) MyChildProcPid, (int) MyProcPort->sock)));
+}
+
+/*
+ * BackendForkFailure
+ *
+ * Backend cleanup in case a failure occurs forking a new Backend.
+ */
+bool
+BackendForkFailure(int child_errno)
+{
+ dlist_mutable_iter iter;
+
+ dlist_foreach_modify(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->child_slot == MyPMChildSlot)
+ {
+ if (!bp->dead_end)
+ {
+ if (!ReleasePostmasterChildSlot(bp->child_slot))
+ {
+ /* Could not release child slot! Panic! */
+ return true;
+ }
+#ifdef EXEC_BACKEND
+ ShmemBackendArrayRemove(bp);
+#endif
+ }
+ dlist_delete(iter.cur);
+ free(bp);
+ break;
+ }
+ }
+
+ report_fork_failure_to_client(MyProcPort, child_errno);
+
+ /* Don't panic */
+ return false;
+}
+
+
/*
* Scan the bgworkers list and see if the given PID (which has just stopped
* or crashed) is in it. Handle its shutdown if so, and return true. If not a
@@ -4075,122 +4242,6 @@ TerminateChildren(int signal)
signal_child(PgStatPID, signal);
}
-/*
- * BackendStartup -- start backend process
- *
- * returns: STATUS_ERROR if the fork failed, STATUS_OK otherwise.
- *
- * Note: if you change this code, also consider StartAutovacuumWorker.
- */
-static int
-BackendStartup(Port *port)
-{
- Backend *bn; /* for backend cleanup */
- pid_t pid;
-
- /*
- * Create backend data structure. Better before the fork() so we can
- * handle failure cleanly.
- */
- bn = (Backend *) malloc(sizeof(Backend));
- if (!bn)
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- return STATUS_ERROR;
- }
-
- /*
- * Compute the cancel key that will be assigned to this backend. The
- * backend will have its own copy in the forked-off process' value of
- * MyCancelKey, so that it can transmit the key to the frontend.
- */
- if (!RandomCancelKey(&MyCancelKey))
- {
- free(bn);
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("could not generate random cancel key")));
- return STATUS_ERROR;
- }
-
- bn->cancel_key = MyCancelKey;
-
- /* Pass down canAcceptConnections state */
- port->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
- bn->dead_end = (port->canAcceptConnections != CAC_OK &&
- port->canAcceptConnections != CAC_WAITBACKUP);
-
- /*
- * Unless it's a dead_end child, assign it a child slot number
- */
- if (!bn->dead_end)
- bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
- else
- bn->child_slot = 0;
-
- /* Hasn't asked to be notified about any bgworkers yet */
- bn->bgworker_notify = false;
-
-#ifdef EXEC_BACKEND
- pid = backend_forkexec(port);
-#else /* !EXEC_BACKEND */
- pid = fork_process();
- if (pid == 0) /* child */
- {
- free(bn);
-
- /* Detangle from postmaster */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Perform additional initialization and collect startup packet */
- BackendInitialize(port);
-
- /* And run the backend */
- BackendRun(port);
- }
-#endif /* EXEC_BACKEND */
-
- if (pid < 0)
- {
- /* in parent, fork failed */
- int save_errno = errno;
-
- if (!bn->dead_end)
- (void) ReleasePostmasterChildSlot(bn->child_slot);
- free(bn);
- errno = save_errno;
- ereport(LOG,
- (errmsg("could not fork new process for connection: %m")));
- report_fork_failure_to_client(port, save_errno);
- return STATUS_ERROR;
- }
-
- /* in parent, successful fork */
- ereport(DEBUG2,
- (errmsg_internal("forked new backend, pid=%d socket=%d",
- (int) pid, (int) port->sock)));
-
- /*
- * Everything's been successful, it's safe to add this backend to our list
- * of backends.
- */
- bn->pid = pid;
- bn->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
- dlist_push_head(&BackendList, &bn->elem);
-
-#ifdef EXEC_BACKEND
- if (!bn->dead_end)
- ShmemBackendArrayAdd(bn);
-#endif
-
- return STATUS_OK;
-}
-
/*
* Try to report backend fork() failure to client before we close the
* connection. Since we do not care to risk blocking the postmaster on
@@ -4483,33 +4534,12 @@ postmaster_forkexec(int argc, char *argv[])
Port port;
/* This entry point passes dummy values for the Port variables */
- memset(&port, 0, sizeof(port));
- return internal_forkexec(argc, argv, &port);
-}
-
-/*
- * backend_forkexec -- fork/exec off a backend process
- *
- * Some operating systems (WIN32) don't have fork() so we have to simulate
- * it by storing parameters that need to be passed to the child and
- * then create a new child process.
- *
- * returns the pid of the fork/exec'd process, or -1 on failure
- */
-static pid_t
-backend_forkexec(Port *port)
-{
- char *av[4];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkbackend";
- av[ac++] = NULL; /* filled in by internal_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ if (!ConnProcPort)
+ memset(&port, 0, sizeof(port));
+ else
+ port = *ConnProcPort;
- return internal_forkexec(ac, av, port);
+ return internal_forkexec(argc, argv, &port);
}
#ifndef WIN32
@@ -4849,6 +4879,7 @@ SubPostmasterMain(int argc, char *argv[])
/* Read in the variables file */
memset(&port, 0, sizeof(Port));
read_backend_variables(argv[2], &port);
+ ConnProcPort = &port;
/* Close the postmaster's sockets (as soon as we know them) */
ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
@@ -4890,11 +4921,7 @@ SubPostmasterMain(int argc, char *argv[])
* sometimes impossible to attach to shared memory at the desired address.
* Return the setting to its old value (usually '1' or '2') when finished.
*/
- if (strcmp(argv[1], "--forkbackend") == 0 ||
- strcmp(argv[1], "--forkavlauncher") == 0 ||
- strcmp(argv[1], "--forkavworker") == 0 ||
- strcmp(argv[1], "--forkboot") == 0 ||
- strncmp(argv[1], "--forkbgworker=", 15) == 0)
+ if (MySubprocess->needs_shmem)
PGSharedMemoryReAttach();
else
PGSharedMemoryNoReAttach();
@@ -4943,11 +4970,23 @@ SubPostmasterMain(int argc, char *argv[])
*/
process_shared_preload_libraries();
- /* Run backend or appropriate child */
- if (strcmp(argv[1], "--forkbackend") == 0)
+ if (MySubprocess->needs_aux_proc)
{
- Assert(argc == 3); /* shouldn't be any more args */
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
+ /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ InitAuxiliaryProcess();
+
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores();
+
+ AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
+ }
+
+ /* Run backend or appropriate child */
+ if (MySubprocessType == ClientBackendType)
+ {
/*
* Need to reinitialize the SSL library in the backend, since the
* context structures contain function pointers and cannot be passed
@@ -4981,32 +5020,6 @@ SubPostmasterMain(int argc, char *argv[])
* report the error to the client.
*/
BackendInitialize(&port);
-
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- /* And run the backend */
- BackendRun(&port); /* does not return */
- }
-
- if (MySubprocess->needs_aux_proc)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitAuxiliaryProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
}
if (MySubprocessType == BgWorkerType)
@@ -5027,15 +5040,29 @@ SubPostmasterMain(int argc, char *argv[])
CreateSharedMemoryAndSemaphores();
}
- if (MySubprocessType == BgWorkerType)
+ switch (MySubprocessType)
{
- /* Fetch MyBgworkerEntry from shared memory */
- shmem_slot = atoi(argv[1] + 15);
- MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
+ case AutoVacuumLauncherType:
+ case AutoVacuumWorkerType:
+ MySubprocess->entrypoint(argc - 2, argv + 2);
+ break;
+ case BgWorkerType:
+ /* Fetch MyBgworkerEntry from shared memory */
+ shmem_slot = atoi(argv[1] + 15);
+ MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
+ /* fallthrough */
+ case PgArchiverType:
+ case PgstatCollectorType:
+ case SysLoggerType:
+ case ClientBackendType:
+ MySubprocess->entrypoint(argc, argv);
+ break;
+ default:
+ ereport(LOG,
+ (errmsg("could not start unknown process type (%d) under postmaster",
+ MySubprocessType)));
}
- MySubprocess->entrypoint(argc - 2, argv + 2);
-
abort(); /* shouldn't get here */
}
#endif /* EXEC_BACKEND */
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index d4df791aeb..9d7ce86918 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -170,6 +170,17 @@ static PgSubprocess process_types[] = {
.entrypoint = BackgroundWorkerMain,
.fork_failure = BackgroundWorkerForkFailure,
.postmaster_main = BackgroundWorkerPostmasterMain
+ },
+ {
+ .name = "backend",
+ .desc = "backend",
+ .needs_aux_proc = false,
+ .needs_shmem = true,
+ .keep_postmaster_memcontext = true,
+ .fork_prep = BackendPrep,
+ .entrypoint = BackendMain,
+ .fork_failure = BackendForkFailure,
+ .postmaster_main = BackendPostmasterMain
}
};
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index b1411c1ed2..6bc0af7c2b 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -91,6 +91,12 @@ extern bool PostmasterMarkPIDForWorkerNotify(int);
extern bool RandomCancelKey(int32 *cancel_key);
+extern int BackendPrep(int argc, char *argv[]);
+extern void BackendMain(int argc, char *argv[]);
+extern void BackendPostmasterMain(int argc, char *argv[]);
+extern bool BackendForkFailure(int child_errno);
+extern void BackgroundWorkerMain(int argc, char *argv[]);
+
#ifdef EXEC_BACKEND
extern pid_t postmaster_forkexec(int argc, char *argv[]);
extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 706d60e4d6..06e67efecc 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -29,6 +29,7 @@ typedef enum
PgArchiverType,
SysLoggerType,
BgWorkerType,
+ ClientBackendType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.21.1
0010-Add-WalSenderType-to-subprocess-struct.patch (text/x-patch, 1.4K) download
From 052bbf6f80d75651da06e72f591c7cdeaf1c677f Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 17 Mar 2020 18:39:12 +0000
Subject: [PATCH 10/11] Add WalSenderType to subprocess struct
---
src/backend/postmaster/subprocess.c | 11 +++++++++++
src/include/postmaster/subprocess.h | 1 +
2 files changed, 12 insertions(+)
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 9d7ce86918..8cc88792ea 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -181,6 +181,17 @@ static PgSubprocess process_types[] = {
.entrypoint = BackendMain,
.fork_failure = BackendForkFailure,
.postmaster_main = BackendPostmasterMain
+ },
+ { /* Wal Sender placeholder */
+ .name = "walsender",
+ .desc = "wal sender",
+ .needs_aux_proc = false,
+ .needs_shmem = false,
+ .keep_postmaster_memcontext = false,
+ .fork_prep = NULL,
+ .entrypoint = NULL,
+ .fork_failure = NULL,
+ .postmaster_main = NULL
}
};
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 06e67efecc..867e1cb499 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -31,6 +31,7 @@ typedef enum
BgWorkerType,
ClientBackendType,
+ WalSenderType, /* placeholder for wal sender so it can be identified in pgstat */
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.21.1
0011-Move-to-new-MyBackendType.patch (text/x-patch, 19.7K) download
From f5219e4a0c845d86c8f48d975701ecd368cd0903 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 17 Mar 2020 23:37:05 +0000
Subject: [PATCH 11/11] Move to new MyBackendType
- Replace MySubprocessType with MyBackendType
- Reshuffle the order of BackendType enums and move to subprocess.h
- Move GetBackendTypeDesc to subprocess.c and use process_types struct
- Change InitMySubprocess to more appropriate SetMySubprocess
- Handle resetting of MyBackendType on StartSubprocess failure
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/postmaster/postmaster.c | 102 ++++++++++++++--------------
src/backend/postmaster/subprocess.c | 12 +++-
src/backend/utils/error/elog.c | 1 +
src/backend/utils/init/miscinit.c | 54 ---------------
src/include/miscadmin.h | 22 ------
src/include/pgstat.h | 1 +
src/include/postmaster/subprocess.h | 45 ++++++------
8 files changed, 87 insertions(+), 152 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 38d45d9163..7c6673c0fc 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -316,7 +316,7 @@ AuxiliaryProcessMain(int argc, char *argv[])
/*
* We've read the arguments and know what backend type we are.
*/
- InitMySubprocess((SubprocessType)MyAuxProcType);
+ SetMySubprocess((BackendType)MyAuxProcType);
/*
* Identify myself via ps
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index f979d505df..143b13cca6 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -147,7 +147,7 @@ BackgroundWorker *MyBgworkerEntry = NULL;
RegisteredBgWorker *MyRegisteredBgWorker = NULL;
/* Struct containing postmaster subprocess control info */
-SubprocessType MySubprocessType;
+BackendType MyBackendType;
/* The socket number we are listening for connections on */
int PostPortNumber;
@@ -374,7 +374,7 @@ static void TerminateChildren(int signal);
static int CountChildren(int target);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
-static pid_t StartSubprocess(SubprocessType type);
+static pid_t StartSubprocess(BackendType type);
static void StartAutovacuumWorker(void);
static void MaybeStartWalReceiver(void);
static void InitPostmasterDeathWatchHandle(void);
@@ -460,7 +460,7 @@ typedef struct
TimestampTz PgReloadTime;
pg_time_t first_syslogger_file_time;
- SubprocessType MySubprocessType;
+ BackendType MyBackendType;
bool redirection_done;
bool IsBinaryUpgrade;
@@ -1035,7 +1035,7 @@ PostmasterMain(int argc, char *argv[])
/*
* If enabled, start up syslogger collection subprocess
*/
- SysLoggerPID = StartSubprocess(SysLoggerType);
+ SysLoggerPID = StartSubprocess(B_LOGGER);
/*
* Reset whereToSendOutput from DestDebug (its starting state) to
@@ -1341,7 +1341,7 @@ PostmasterMain(int argc, char *argv[])
/*
* We're ready to rock and roll...
*/
- StartupPID = StartSubprocess(StartupType);
+ StartupPID = StartSubprocess(B_STARTUP);
Assert(StartupPID != 0);
StartupStatus = STARTUP_RUNNING;
pmState = PM_STARTUP;
@@ -1675,7 +1675,7 @@ ServerLoop(void)
if (ConnProcPort)
{
- StartSubprocess(ClientBackendType);
+ StartSubprocess(B_BACKEND);
/*
* We no longer need the open socket or port structure
@@ -1690,7 +1690,7 @@ ServerLoop(void)
/* If we have lost the log collector, try to start a new one */
if (SysLoggerPID == 0 && Logging_collector)
- SysLoggerPID = StartSubprocess(SysLoggerType);
+ SysLoggerPID = StartSubprocess(B_LOGGER);
/*
* If no background writer process is running, and we are not in a
@@ -1701,9 +1701,9 @@ ServerLoop(void)
pmState == PM_HOT_STANDBY)
{
if (CheckpointerPID == 0)
- CheckpointerPID = StartSubprocess(CheckpointerType);
+ CheckpointerPID = StartSubprocess(B_CHECKPOINTER);
if (BgWriterPID == 0)
- BgWriterPID = StartSubprocess(BgWriterType);
+ BgWriterPID = StartSubprocess(B_BG_WRITER);
}
/*
@@ -1712,7 +1712,7 @@ ServerLoop(void)
* be writing any new WAL).
*/
if (WalWriterPID == 0 && pmState == PM_RUN)
- WalWriterPID = StartSubprocess(WalWriterType);
+ WalWriterPID = StartSubprocess(B_WAL_WRITER);
/*
* If we have lost the autovacuum launcher, try to start a new one. We
@@ -1724,7 +1724,7 @@ ServerLoop(void)
(AutoVacuumingActive() || start_autovac_launcher) &&
pmState == PM_RUN)
{
- AutoVacPID = StartSubprocess(AutoVacuumLauncherType);
+ AutoVacPID = StartSubprocess(B_AUTOVAC_LAUNCHER);
if (AutoVacPID != 0)
start_autovac_launcher = false; /* signal processed */
}
@@ -1732,11 +1732,11 @@ ServerLoop(void)
/* If we have lost the stats collector, try to start a new one */
if (PgStatPID == 0 &&
(pmState == PM_RUN || pmState == PM_HOT_STANDBY))
- PgStatPID = StartSubprocess(PgstatCollectorType);
+ PgStatPID = StartSubprocess(B_STATS_COLLECTOR);
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
- PgArchPID = StartSubprocess(PgArchiverType);
+ PgArchPID = StartSubprocess(B_ARCHIVER);
/* If we need to signal the autovacuum launcher, do so now */
if (avlauncher_needs_signal)
@@ -2993,22 +2993,22 @@ reaper(SIGNAL_ARGS)
* if this fails, we'll just try again later.
*/
if (CheckpointerPID == 0)
- CheckpointerPID = StartSubprocess(CheckpointerType);
+ CheckpointerPID = StartSubprocess(B_CHECKPOINTER);
if (BgWriterPID == 0)
- BgWriterPID = StartSubprocess(BgWriterType);
+ BgWriterPID = StartSubprocess(B_BG_WRITER);
if (WalWriterPID == 0)
- WalWriterPID = StartSubprocess(WalWriterType);
+ WalWriterPID = StartSubprocess(B_WAL_WRITER);
/*
* Likewise, start other special children as needed. In a restart
* situation, some of them may be alive already.
*/
if (!IsBinaryUpgrade && AutoVacuumingActive() && AutoVacPID == 0)
- AutoVacPID = StartSubprocess(AutoVacuumLauncherType);
+ AutoVacPID = StartSubprocess(B_AUTOVAC_LAUNCHER);
if (PgArchStartupAllowed() && PgArchPID == 0)
- PgArchPID = StartSubprocess(PgArchiverType);
+ PgArchPID = StartSubprocess(B_ARCHIVER);
if (PgStatPID == 0)
- PgStatPID = StartSubprocess(PgstatCollectorType);
+ PgStatPID = StartSubprocess(B_STATS_COLLECTOR);
/* workers may be scheduled to start now */
maybe_start_bgworkers();
@@ -3154,7 +3154,7 @@ reaper(SIGNAL_ARGS)
LogChildExit(LOG, _("archiver process"),
pid, exitstatus);
if (PgArchStartupAllowed())
- PgArchPID = StartSubprocess(PgArchiverType);
+ PgArchPID = StartSubprocess(B_ARCHIVER);
continue;
}
@@ -3170,7 +3170,7 @@ reaper(SIGNAL_ARGS)
LogChildExit(LOG, _("statistics collector process"),
pid, exitstatus);
if (pmState == PM_RUN || pmState == PM_HOT_STANDBY)
- PgStatPID = StartSubprocess(PgstatCollectorType);
+ PgStatPID = StartSubprocess(B_STATS_COLLECTOR);
continue;
}
@@ -3179,7 +3179,7 @@ reaper(SIGNAL_ARGS)
{
SysLoggerPID = 0;
/* for safety's sake, launch new logger *first* */
- SysLoggerPID = StartSubprocess(SysLoggerType);
+ SysLoggerPID = StartSubprocess(B_LOGGER);
if (!EXIT_STATUS_0(exitstatus))
LogChildExit(LOG, _("system logger process"),
pid, exitstatus);
@@ -3979,7 +3979,7 @@ PostmasterStateMachine(void)
Assert(Shutdown > NoShutdown);
/* Start the checkpointer if not running */
if (CheckpointerPID == 0)
- CheckpointerPID = StartSubprocess(CheckpointerType);
+ CheckpointerPID = StartSubprocess(B_CHECKPOINTER);
/* And tell it to shut down */
if (CheckpointerPID != 0)
{
@@ -4120,7 +4120,7 @@ PostmasterStateMachine(void)
reset_shared();
- StartupPID = StartSubprocess(StartupType);
+ StartupPID = StartSubprocess(B_STARTUP);
Assert(StartupPID != 0);
StartupStatus = STARTUP_RUNNING;
pmState = PM_STARTUP;
@@ -4903,7 +4903,7 @@ SubPostmasterMain(int argc, char *argv[])
/* We should only be here once per fork */
Assert(!MySubprocess);
- InitMySubprocess(MySubprocessType);
+ SetMySubprocess(MyBackendType);
/*
* If appropriate, physically re-attach to shared memory segment. We want
@@ -4927,9 +4927,9 @@ SubPostmasterMain(int argc, char *argv[])
PGSharedMemoryNoReAttach();
/* autovacuum needs this set before calling InitProcess */
- if (strcmp(argv[1], "--forkavlauncher") == 0)
+ if (MyBackendType == B_AUTOVAC_LAUNCHER)
AutovacuumLauncherIAm();
- if (strcmp(argv[1], "--forkavworker") == 0)
+ if (MyBackendType == B_AUTOVAC_WORKER)
AutovacuumWorkerIAm();
/*
@@ -4985,7 +4985,7 @@ SubPostmasterMain(int argc, char *argv[])
}
/* Run backend or appropriate child */
- if (MySubprocessType == ClientBackendType)
+ if (MyBackendType == B_BACKEND)
{
/*
* Need to reinitialize the SSL library in the backend, since the
@@ -5022,7 +5022,7 @@ SubPostmasterMain(int argc, char *argv[])
BackendInitialize(&port);
}
- if (MySubprocessType == BgWorkerType)
+ if (MyBackendType == B_BG_WORKER)
{
/* do this as early as possible; in particular, before InitProcess() */
IsBackgroundWorker = true;
@@ -5040,27 +5040,27 @@ SubPostmasterMain(int argc, char *argv[])
CreateSharedMemoryAndSemaphores();
}
- switch (MySubprocessType)
+ switch (MyBackendType)
{
- case AutoVacuumLauncherType:
- case AutoVacuumWorkerType:
+ case B_AUTOVAC_LAUNCHER:
+ case B_AUTOVAC_WORKER:
MySubprocess->entrypoint(argc - 2, argv + 2);
break;
- case BgWorkerType:
+ case B_BG_WORKER:
/* Fetch MyBgworkerEntry from shared memory */
shmem_slot = atoi(argv[1] + 15);
MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
/* fallthrough */
- case PgArchiverType:
- case PgstatCollectorType:
- case SysLoggerType:
- case ClientBackendType:
+ case B_ARCHIVER:
+ case B_STATS_COLLECTOR:
+ case B_LOGGER:
+ case B_BACKEND:
MySubprocess->entrypoint(argc, argv);
break;
default:
ereport(LOG,
(errmsg("could not start unknown process type (%d) under postmaster",
- MySubprocessType)));
+ MyBackendType)));
}
abort(); /* shouldn't get here */
@@ -5144,9 +5144,9 @@ sigusr1_handler(SIGNAL_ARGS)
* we'll just try again later.
*/
Assert(CheckpointerPID == 0);
- CheckpointerPID = StartSubprocess(CheckpointerType);
+ CheckpointerPID = StartSubprocess(B_CHECKPOINTER);
Assert(BgWriterPID == 0);
- BgWriterPID = StartSubprocess(BgWriterType);
+ BgWriterPID = StartSubprocess(B_BG_WRITER);
/*
* Start the archiver if we're responsible for (re-)archiving received
@@ -5154,7 +5154,7 @@ sigusr1_handler(SIGNAL_ARGS)
*/
Assert(PgArchPID == 0);
if (XLogArchivingAlways())
- PgArchPID = StartSubprocess(PgArchiverType);
+ PgArchPID = StartSubprocess(B_ARCHIVER);
/*
* If we aren't planning to enter hot standby mode later, treat
@@ -5178,7 +5178,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Likewise, start other special children as needed.
*/
Assert(PgStatPID == 0);
- PgStatPID = StartSubprocess(PgstatCollectorType);
+ PgStatPID = StartSubprocess(B_STATS_COLLECTOR);
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5383,8 +5383,9 @@ CountChildren(int target)
* the subprocess cleanup function decides whether or not to panic.
*/
static pid_t
-StartSubprocess(SubprocessType type)
+StartSubprocess(BackendType type)
{
+ BackendType prev_backend_type = MyBackendType;
pid_t pid;
char *argv[10];
int argc = 0;
@@ -5396,7 +5397,7 @@ StartSubprocess(SubprocessType type)
/*
* Get new subprocess data every time we start a new subprocess.
*/
- InitMySubprocess(type);
+ SetMySubprocess(type);
argv[argc++] = "postgres";
@@ -5414,6 +5415,7 @@ StartSubprocess(SubprocessType type)
{
if (MySubprocess->fork_prep(argc, argv))
{
+ SetMySubprocess(prev_backend_type);
return 0;
}
}
@@ -5433,7 +5435,7 @@ StartSubprocess(SubprocessType type)
InitPostmasterChild();
/* Close the postmaster's sockets */
- ClosePostmasterPorts(MySubprocessType == SysLoggerType);
+ ClosePostmasterPorts(MyBackendType == B_LOGGER)
/*
* Release postmaster's working memory context if the subprocess doesn't
@@ -5547,7 +5549,7 @@ StartAutovacuumWorker(void)
bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
bn->bgworker_notify = false;
- bn->pid = StartSubprocess(AutoVacuumWorkerType);
+ bn->pid = StartSubprocess(B_AUTOVAC_WORKER);
if (bn->pid > 0)
{
bn->bkend_type = BACKEND_TYPE_AUTOVAC;
@@ -5608,7 +5610,7 @@ MaybeStartWalReceiver(void)
pmState == PM_HOT_STANDBY || pmState == PM_WAIT_READONLY) &&
Shutdown == NoShutdown)
{
- WalReceiverPID = StartSubprocess(WalReceiverType);
+ WalReceiverPID = StartSubprocess(B_WAL_RECEIVER);
if (WalReceiverPID != 0)
WalReceiverRequested = false;
/* else leave the flag set, so we'll try again later */
@@ -5872,7 +5874,7 @@ maybe_start_bgworkers(void)
* function will do that.
*/
MyRegisteredBgWorker = rw;
- if (StartSubprocess(BgWorkerType) <= 0)
+ if (StartSubprocess(B_BG_WORKER) <= 0)
{
StartWorkerNeeded = true;
return;
@@ -5990,7 +5992,7 @@ save_backend_variables(BackendParameters *param, Port *port,
param->PgReloadTime = PgReloadTime;
param->first_syslogger_file_time = first_syslogger_file_time;
- param->MySubprocessType = MySubprocessType;
+ param->MyBackendType = MyBackendType;
param->redirection_done = redirection_done;
param->IsBinaryUpgrade = IsBinaryUpgrade;
@@ -6227,7 +6229,7 @@ restore_backend_variables(BackendParameters *param, Port *port)
PgReloadTime = param->PgReloadTime;
first_syslogger_file_time = param->first_syslogger_file_time;
- MySubprocessType = param->MySubprocessType;
+ MyBackendType = param->MyBackendType;
redirection_done = param->redirection_done;
IsBinaryUpgrade = param->IsBinaryUpgrade;
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 8cc88792ea..3bd9136f83 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -24,7 +24,7 @@
#include "postmaster/walwriter.h"
#include "replication/walreceiver.h"
-SubprocessType MySubprocessType;
+BackendType MyBackendType;
PgSubprocess *MySubprocess;
static PgSubprocess process_types[] = {
@@ -196,9 +196,15 @@ static PgSubprocess process_types[] = {
};
void
-InitMySubprocess(SubprocessType type)
+SetMySubprocess(BackendType type)
{
- MySubprocessType = type;
+ MyBackendType = type;
MySubprocess = &process_types[type];
MySubprocess->desc = gettext(MySubprocess->desc);
}
+
+const char *
+GetBackendTypeDesc(BackendType type)
+{
+ return gettext((&process_types[type])->desc);
+}
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 62eef7b71f..214fcb0522 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -74,6 +74,7 @@
#include "miscadmin.h"
#include "postmaster/bgworker.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "postmaster/syslogger.h"
#include "storage/ipc.h"
#include "storage/proc.h"
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index a7b7b12249..3d36836413 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -189,60 +189,6 @@ SwitchBackToLocalLatch(void)
SetLatch(MyLatch);
}
-const char *
-GetBackendTypeDesc(BackendType backendType)
-{
- const char *backendDesc = "unknown process type";
-
- switch (backendType)
- {
- case B_INVALID:
- backendDesc = "not initialized";
- break;
- case B_AUTOVAC_LAUNCHER:
- backendDesc = "autovacuum launcher";
- break;
- case B_AUTOVAC_WORKER:
- backendDesc = "autovacuum worker";
- break;
- case B_BACKEND:
- backendDesc = "client backend";
- break;
- case B_BG_WORKER:
- backendDesc = "background worker";
- break;
- case B_BG_WRITER:
- backendDesc = "background writer";
- break;
- case B_CHECKPOINTER:
- backendDesc = "checkpointer";
- break;
- case B_STARTUP:
- backendDesc = "startup";
- break;
- case B_WAL_RECEIVER:
- backendDesc = "walreceiver";
- break;
- case B_WAL_SENDER:
- backendDesc = "walsender";
- break;
- case B_WAL_WRITER:
- backendDesc = "walwriter";
- break;
- case B_ARCHIVER:
- backendDesc = "archiver";
- break;
- case B_STATS_COLLECTOR:
- backendDesc = "stats collector";
- break;
- case B_LOGGER:
- backendDesc = "logger";
- break;
- }
-
- return backendDesc;
-}
-
/* ----------------------------------------------------------------
* database path / name support stuff
* ----------------------------------------------------------------
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 541ca5145d..36a65865ab 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -307,28 +307,6 @@ extern void InitStandaloneProcess(const char *argv0);
extern void SwitchToSharedLatch(void);
extern void SwitchBackToLocalLatch(void);
-typedef enum BackendType
-{
- B_INVALID = 0,
- B_AUTOVAC_LAUNCHER,
- B_AUTOVAC_WORKER,
- B_BACKEND,
- B_BG_WORKER,
- B_BG_WRITER,
- B_CHECKPOINTER,
- B_STARTUP,
- B_WAL_RECEIVER,
- B_WAL_SENDER,
- B_WAL_WRITER,
- B_ARCHIVER,
- B_STATS_COLLECTOR,
- B_LOGGER,
-} BackendType;
-
-extern BackendType MyBackendType;
-
-extern const char *GetBackendTypeDesc(BackendType backendType);
-
extern void SetDatabasePath(const char *path);
extern void checkDataDir(void);
extern void SetDataDir(const char *dir);
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 92943f5cff..48f9e313a6 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -17,6 +17,7 @@
#include "port/atomics.h"
#include "portability/instr_time.h"
#include "postmaster/pgarch.h"
+#include "postmaster/subprocess.h"
#include "storage/proc.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 867e1cb499..7c86690daf 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -13,27 +13,27 @@
#include "postmaster.h"
-typedef enum
+typedef enum BackendType
{
- NoProcessType = -1,
- CheckerType = 0,
- BootstrapType,
- StartupType,
- BgWriterType,
- CheckpointerType,
- WalWriterType,
- WalReceiverType, /* end of Auxiliary Process Forks */
- AutoVacuumLauncherType,
- AutoVacuumWorkerType,
- PgstatCollectorType,
- PgArchiverType,
- SysLoggerType,
- BgWorkerType,
- ClientBackendType,
-
- WalSenderType, /* placeholder for wal sender so it can be identified in pgstat */
- NUMSUBPROCESSTYPES /* Must be last! */
-} SubprocessType;
+ B_INVALID = -1,
+ B_CHECKER = 0,
+ B_BOOTSTRAP,
+ B_STARTUP,
+ B_BG_WRITER,
+ B_CHECKPOINTER,
+ B_WAL_WRITER,
+ B_WAL_RECEIVER, /* end of Auxiliary Process Forks */
+ B_AUTOVAC_LAUNCHER,
+ B_AUTOVAC_WORKER,
+ B_STATS_COLLECTOR,
+ B_ARCHIVER,
+ B_LOGGER,
+ B_BG_WORKER,
+ B_BACKEND,
+
+ B_WAL_SENDER, /* placeholder for wal sender so it can be identified in pgstat */
+ NUMBACKENDTYPES /* Must be last! */
+} BackendType;
typedef int (*SubprocessPrep) (int argc, char *argv[]);
typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
@@ -41,7 +41,8 @@ typedef bool (*SubprocessForkFailure) (int fork_errno);
typedef void (*SubprocessPostmasterMain) (int argc, char *argv[]);
/* Current subprocess initializer */
-extern void InitMySubprocess(SubprocessType type);
+extern void SetMySubprocess(BackendType type);
+extern const char *GetBackendTypeDesc(BackendType type);
typedef struct PgSubprocess
{
@@ -56,7 +57,7 @@ typedef struct PgSubprocess
SubprocessPostmasterMain postmaster_main;
} PgSubprocess;
-extern SubprocessType MySubprocessType;
+extern BackendType MyBackendType;
extern PGDLLIMPORT PgSubprocess *MySubprocess;
#endif /* SUBPROCESS_H */
--
2.21.1
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-02 14:53 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-04 04:56 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 18:50 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 21:52 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 01:04 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2020-03-18 13:22 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 14:17 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 15:26 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 16:07 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
@ 2020-03-18 16:54 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2020-03-18 16:59 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
1 sibling, 1 reply; 38+ messages in thread
From: Alvaro Herrera @ 2020-03-18 16:54 UTC (permalink / raw)
To: Mike Palmiotto <mike.palmiotto@crunchydata.com>; +Cc: Justin Pryzby <pryzby@telsasoft.com>; Michael Paquier <michael@paquier.xyz>; Andres Freund <andres@anarazel.de>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On 2020-Mar-18, Mike Palmiotto wrote:
> On Wed, Mar 18, 2020 at 11:26 AM Mike Palmiotto
> <mike.palmiotto@crunchydata.com> wrote:
> >
> > On Wed, Mar 18, 2020 at 10:17 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
> > > Also, I saw this was failing tests both before and after my rebase.
> > >
> > > http://cfbot.cputube.org/
> > > https://ci.appveyor.com/project/postgresql-cfbot/postgresql/builds/31535161
> > > https://ci.appveyor.com/project/postgresql-cfbot/postgresql/builds/31386446
> >
> > Good catch, thanks. Will address this as well in the next round. Just
> > need to set up a Windows dev environment to see if I can
> > reproduce/fix.
>
> While I track this down, here is a rebased patchset, which drops
> MySubprocessType and makes use of the MyBackendType.
Note that you can compile with -DEXEC_BACKEND to use in a *nix build the
same technique used to spawn processes in Windows, which might be an
easier way to discover some problems without a proper Windows build.
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-02 14:53 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-04 04:56 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 18:50 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 21:52 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 01:04 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2020-03-18 13:22 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 14:17 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 15:26 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 16:07 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 16:54 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
@ 2020-03-18 16:59 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
0 siblings, 0 replies; 38+ messages in thread
From: Mike Palmiotto @ 2020-03-18 16:59 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@2ndquadrant.com>; +Cc: Justin Pryzby <pryzby@telsasoft.com>; Michael Paquier <michael@paquier.xyz>; Andres Freund <andres@anarazel.de>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Wed, Mar 18, 2020 at 12:54 PM Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:
>
> On 2020-Mar-18, Mike Palmiotto wrote:
>
> > On Wed, Mar 18, 2020 at 11:26 AM Mike Palmiotto
> > <mike.palmiotto@crunchydata.com> wrote:
> > >
> > > On Wed, Mar 18, 2020 at 10:17 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
> > > > Also, I saw this was failing tests both before and after my rebase.
> > > >
> > > > http://cfbot.cputube.org/
> > > > https://ci.appveyor.com/project/postgresql-cfbot/postgresql/builds/31535161
> > > > https://ci.appveyor.com/project/postgresql-cfbot/postgresql/builds/31386446
> > >
> > > Good catch, thanks. Will address this as well in the next round. Just
> > > need to set up a Windows dev environment to see if I can
> > > reproduce/fix.
> >
> > While I track this down, here is a rebased patchset, which drops
> > MySubprocessType and makes use of the MyBackendType.
>
> Note that you can compile with -DEXEC_BACKEND to use in a *nix build the
> same technique used to spawn processes in Windows, which might be an
> easier way to discover some problems without a proper Windows build.
Good suggestion. Unfortunately,that's how I've been testing all along.
I thought that would be sufficient, but seems like this might be more
specific to the Windows #ifdef's.
I have another version on my devbox which fixes the (embarrassing)
Travis failure for non-EXEC_BACKEND due to a dropped semicolon during
rebase. I'm setting up my own appveyor instance with Tomas's config
and will see if I can get to the bottom of this.
--
Mike Palmiotto
https://crunchydata.com
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-02 14:53 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-04 04:56 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 18:50 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 21:52 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 01:04 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2020-03-18 13:22 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 14:17 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 15:26 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 16:07 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
@ 2020-03-19 10:35 ` Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2020-03-19 13:29 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-19 20:57 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
1 sibling, 2 replies; 38+ messages in thread
From: Peter Eisentraut @ 2020-03-19 10:35 UTC (permalink / raw)
To: Mike Palmiotto <mike.palmiotto@crunchydata.com>; Justin Pryzby <pryzby@telsasoft.com>; +Cc: Alvaro Herrera <alvherre@2ndquadrant.com>; Michael Paquier <michael@paquier.xyz>; Andres Freund <andres@anarazel.de>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On 2020-03-18 17:07, Mike Palmiotto wrote:
> On Wed, Mar 18, 2020 at 11:26 AM Mike Palmiotto
> <mike.palmiotto@crunchydata.com> wrote:
>>
>> On Wed, Mar 18, 2020 at 10:17 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
>>> Also, I saw this was failing tests both before and after my rebase.
>>>
>>> http://cfbot.cputube.org/
>>> https://ci.appveyor.com/project/postgresql-cfbot/postgresql/builds/31535161
>>> https://ci.appveyor.com/project/postgresql-cfbot/postgresql/builds/31386446
>>
>> Good catch, thanks. Will address this as well in the next round. Just
>> need to set up a Windows dev environment to see if I can
>> reproduce/fix.
>
> While I track this down, here is a rebased patchset, which drops
> MySubprocessType and makes use of the MyBackendType.
While working on (My)BackendType, I was attempting to get rid of
(My)AuxProcType altogether. This would mostly work except that it's
sort of wired into the pgstats subsystem (see NumBackendStatSlots).
This can probably be reorganized, but I didn't pursue it further.
Now, I'm a sucker for refactoring, but I feel this proposal is going
into a direction I don't understand. I'd prefer that we focus around
building out background workers as the preferred subprocess mechanism.
Building out a second generic mechanism, again, I don't understand the
direction. Are we hoping to add more of these processes? Make it
extensible? The net lines added/removed by this patch series seems
pretty neutral. What are we getting at the end of this?
More specifically, I don't agree with the wholesale renaming of
auxiliary process to subprocess. Besides the massive code churn, the
old naming seemed pretty good to distinguish them from background
workers, the new naming is more ambiguous.
Btw., if I had a lot of time I would attempt to rewrite walreceiver and
perhaps the autovacuum system as background workers and thus further
reduce the footprint of the aux process system.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-02 14:53 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-04 04:56 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 18:50 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 21:52 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 01:04 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2020-03-18 13:22 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 14:17 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 15:26 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 16:07 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-19 10:35 ` Re: Auxiliary Processes and MyAuxProc Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
@ 2020-03-19 13:29 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-21 17:40 ` Re: Auxiliary Processes and MyAuxProc Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
1 sibling, 1 reply; 38+ messages in thread
From: Mike Palmiotto @ 2020-03-19 13:29 UTC (permalink / raw)
To: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>; +Cc: Justin Pryzby <pryzby@telsasoft.com>; Alvaro Herrera <alvherre@2ndquadrant.com>; Michael Paquier <michael@paquier.xyz>; Andres Freund <andres@anarazel.de>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Thu, Mar 19, 2020 at 6:35 AM Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
>
> While working on (My)BackendType, I was attempting to get rid of
> (My)AuxProcType altogether. This would mostly work except that it's
> sort of wired into the pgstats subsystem (see NumBackendStatSlots).
> This can probably be reorganized, but I didn't pursue it further.
This patchset has a different goal: to remove redundant startup code
and interspersed variants of fork/forkexec code so that we can
centralize the postmaster child startup.
The goal of centralizing postmaster startup stems from the desire to
be able to control the process security attributes immediately
before/after fork/exec. This is simply not possible with the existing
infrastructure, since processes are identified in Main functions,
which is too late (and again too scattered) to be able to do anything
useful.
By providing a mechanism to set child process metadata prior to
spawning the subprocess, we gain the ability to identify the process
type and thus set security attributes on that process.
In an earlier spin of the patchset, I included a fork_process hook,
which would be where an extension could set security attributes on a
process. I have since dropped the fork (as advised), but now realize
that it actually demonstrates the main motivation of the patchset.
Perhaps I should add that back in for the next version.
>
> Now, I'm a sucker for refactoring, but I feel this proposal is going
> into a direction I don't understand. I'd prefer that we focus around
> building out background workers as the preferred subprocess mechanism.
> Building out a second generic mechanism, again, I don't understand the
> direction. Are we hoping to add more of these processes? Make it
> extensible? The net lines added/removed by this patch series seems
> pretty neutral. What are we getting at the end of this?
As stated above, the primary goal is to centralize the startup code.
One nice side-effect is the introduction of a mechanism that is now
both extensible and provides the ability to remove a lot of redundant
code. I see no reason to have 5 different variants of process forkexec
functions for the sole purpose of building up argv. This patchset
intends to get rid of such an architecture.
Note that this is not intended to be the complete product here -- it
is just a first step at swapping in and making use of a new
infrastructure. There will be follow-up work required to really get
the most out of this infrastructure. For instance, we could drop a
large portion of the SubPostmasterMain switch logic. There are a
number of other areas throughout the codebase (including the example
provided in the last commit, which changes the way we retrieve process
descriptions), that can utilize this new infrastructure to get rid of
code.
>
> More specifically, I don't agree with the wholesale renaming of
> auxiliary process to subprocess. Besides the massive code churn, the
I'm not sure I understand the argument here. Where do you see
wholesale renaming of AuxProc to Subprocess? Subprocess is the name
for postmaster subprocesses, whereas Auxiliary Processes are a subset
of those processes.
> old naming seemed pretty good to distinguish them from background
> workers, the new naming is more ambiguous.
I do not see any conflation between Background Workers and Auxiliary
Processes in this patchset. Since Auxiliary Processes are included in
the full set of subprocesses and are delineated with a boolean:
needs_aux_proc, it seems fairly straightforward to me which
subprocesses are in fact Auxiliary Processes.
Thanks,
--
Mike Palmiotto
https://crunchydata.com
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-02 14:53 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-04 04:56 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 18:50 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 21:52 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 01:04 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2020-03-18 13:22 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 14:17 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 15:26 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 16:07 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-19 10:35 ` Re: Auxiliary Processes and MyAuxProc Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2020-03-19 13:29 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
@ 2020-03-21 17:40 ` Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
0 siblings, 0 replies; 38+ messages in thread
From: Peter Eisentraut @ 2020-03-21 17:40 UTC (permalink / raw)
To: Mike Palmiotto <mike.palmiotto@crunchydata.com>; +Cc: Justin Pryzby <pryzby@telsasoft.com>; Alvaro Herrera <alvherre@2ndquadrant.com>; Michael Paquier <michael@paquier.xyz>; Andres Freund <andres@anarazel.de>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On 2020-03-19 14:29, Mike Palmiotto wrote:
>> More specifically, I don't agree with the wholesale renaming of
>> auxiliary process to subprocess. Besides the massive code churn, the
>
> I'm not sure I understand the argument here. Where do you see
> wholesale renaming of AuxProc to Subprocess? Subprocess is the name
> for postmaster subprocesses, whereas Auxiliary Processes are a subset
> of those processes.
Specifically renaming AuxProcType to SubprocessType and everything that
follows from that, including the name of new files.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-02 14:53 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-04 04:56 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 18:50 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 21:52 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 01:04 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2020-03-18 13:22 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 14:17 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 15:26 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 16:07 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-19 10:35 ` Re: Auxiliary Processes and MyAuxProc Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
@ 2020-03-19 20:57 ` Andres Freund <andres@anarazel.de>
2020-03-20 22:17 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
1 sibling, 1 reply; 38+ messages in thread
From: Andres Freund @ 2020-03-19 20:57 UTC (permalink / raw)
To: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>; +Cc: Mike Palmiotto <mike.palmiotto@crunchydata.com>; Justin Pryzby <pryzby@telsasoft.com>; Alvaro Herrera <alvherre@2ndquadrant.com>; Michael Paquier <michael@paquier.xyz>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
Hi,
On 2020-03-19 11:35:41 +0100, Peter Eisentraut wrote:
> On 2020-03-18 17:07, Mike Palmiotto wrote:
> > On Wed, Mar 18, 2020 at 11:26 AM Mike Palmiotto
> > <mike.palmiotto@crunchydata.com> wrote:
> > >
> > > On Wed, Mar 18, 2020 at 10:17 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
> > > > Also, I saw this was failing tests both before and after my rebase.
> > > >
> > > > http://cfbot.cputube.org/
> > > > https://ci.appveyor.com/project/postgresql-cfbot/postgresql/builds/31535161
> > > > https://ci.appveyor.com/project/postgresql-cfbot/postgresql/builds/31386446
> > >
> > > Good catch, thanks. Will address this as well in the next round. Just
> > > need to set up a Windows dev environment to see if I can
> > > reproduce/fix.
> >
> > While I track this down, here is a rebased patchset, which drops
> > MySubprocessType and makes use of the MyBackendType.
>
> While working on (My)BackendType, I was attempting to get rid of
> (My)AuxProcType altogether. This would mostly work except that it's sort of
> wired into the pgstats subsystem (see NumBackendStatSlots). This can
> probably be reorganized, but I didn't pursue it further.
Hm. Why does the number of stat slots prevent dropping AuxProcType?
> Now, I'm a sucker for refactoring, but I feel this proposal is going into a
> direction I don't understand. I'd prefer that we focus around building out
> background workers as the preferred subprocess mechanism. Building out a
> second generic mechanism, again, I don't understand the direction. Are we
> hoping to add more of these processes? Make it extensible? The net lines
> added/removed by this patch series seems pretty neutral. What are we
> getting at the end of this?
I think background workers for internal processes are the *wrong*
direction to go. They were used as a shortcut for parallelism, and then
that was extended for logical replication. In my opinion that was done,
to a significant degree, because the aux proc stuff is/was too painful
to deal with, but that's something that should be fixed, instead of
building more and more parallel infrastructure.
Bgworkers are imo not actually a very good fit for internal
processes. We have to be able to guarantee that there's a free "slot" to
start internal processes, we should be able to efficiently reference
their pids (especially from postmaster, but also other processes), we
want to precisely know which shared PGPROC is being used, etc.
We now have somewhat different systems for at least: non-shmem
postmaster children, aux processes, autovacuum workers, internal
bgworkers, extension bgworkers. That's just insane.
We should merge those as much as possible. There's obviously going to be
some differences, but it needs to be less than now. I think we're
mostly on the same page on that, I just don't see bgworkers getting us
there.
The worst part about the current situation imo is that there's way too
many places that one needs to modify / check to create / understand a
process. Moving towards having a single c file w/ associated header that
defines 95% of that seems like a good direction. I've not looked at
recent versions of the patch, but there was some movement towards that
in earlier versions.
On a green field, I would say that there should be one or two C
arrays-of-structs defining subprocesses. And most behaviour should be
channeled through that.
struct PgProcessType
{
const char* name;
PgProcessBeforeShmem before_shmem;
PgProcessEntry entrypoint;
uint8:1 only_one_exists;
uint8:1 uses_shared_memory;
uint8:1 client_connected;
uint8:1 database_connected;
...
};
PgProcessType ProcessTypes[] = {
[StartupType] = {
.name = "startup",
.entrypoint = StartupProcessMain,
.only_one_exists = true,
.uses_shared_memory = true,
.client_connected = false,
.database_connected = false,
...
},
...
[UserBackendType] = {
.name = "backend",
.before_shmem = BackendInitialize,
.entrypoint = BackendRun, // fixme
.only_one_exists = false,
.uses_shared_memory = true,
.client_connected = true,
.database_connected = true,
...
}
...
Then there should be a single startup routine for all postmaster
children. Since most of the startup is actually shared between all the
different types of processes, we can declutter it a lot.
When starting a process postmaster would just specify the process type,
and if relevant, an argument (struct Port for backends, whatever
relevant for bgworkers etc) . Generic code should handle all the work
until the process type entry point - and likely we should move more work
from the individual process types into generic code.
If a process is 'only_one_exists' (to be renamed), the generic code
would also (in postmaster) register the pid as
subprocess_pids[type] = pid;
which would make it easy to only have per-type code in the few locations
that need to be aware, instead of many locations in
postmaster.c. Perhaps also some shared memory location.
Coming back to earth, from green field imagination land: I think the
patchset does go in that direction (to be fair, I think I outlined
something like the above elsewhere in this thread in a discussion with
Mike). And that's good.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-02 14:53 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-04 04:56 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 18:50 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 21:52 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 01:04 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2020-03-18 13:22 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 14:17 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 15:26 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 16:07 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-19 10:35 ` Re: Auxiliary Processes and MyAuxProc Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2020-03-19 20:57 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
@ 2020-03-20 22:17 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-26 23:30 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
0 siblings, 1 reply; 38+ messages in thread
From: Mike Palmiotto @ 2020-03-20 22:17 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>; Justin Pryzby <pryzby@telsasoft.com>; Alvaro Herrera <alvherre@2ndquadrant.com>; Michael Paquier <michael@paquier.xyz>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Thu, Mar 19, 2020 at 4:57 PM Andres Freund <andres@anarazel.de> wrote:
> On 2020-03-19 11:35:41 +0100, Peter Eisentraut wrote:
> > On 2020-03-18 17:07, Mike Palmiotto wrote:
> > > On Wed, Mar 18, 2020 at 11:26 AM Mike Palmiotto
> > > <mike.palmiotto@crunchydata.com> wrote:
> > > >
> > > > On Wed, Mar 18, 2020 at 10:17 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
> > > > > Also, I saw this was failing tests both before and after my rebase.
> > > > >
> > > > > http://cfbot.cputube.org/
> > > > > https://ci.appveyor.com/project/postgresql-cfbot/postgresql/builds/31535161
> > > > > https://ci.appveyor.com/project/postgresql-cfbot/postgresql/builds/31386446
> > > >
> > > > Good catch, thanks. Will address this as well in the next round. Just
> > > > need to set up a Windows dev environment to see if I can
> > > > reproduce/fix.
I'm still working on wiring up an AppVeyor instance, as seemingly
builds don't work on any of the default Azure/Visual Studio images. In
the meantime, I've fixed some spurious whitespace changes and the
compile error for non-EXEC_BACKEND. I'm posting a new version to keep
Travis happy at least while I keep working on that. Sorry for the
delay.
> On a green field, I would say that there should be one or two C
> arrays-of-structs defining subprocesses. And most behaviour should be
> channeled through that.
>
> struct PgProcessType
> {
> const char* name;
> PgProcessBeforeShmem before_shmem;
> PgProcessEntry entrypoint;
> uint8:1 only_one_exists;
> uint8:1 uses_shared_memory;
> uint8:1 client_connected;
> uint8:1 database_connected;
> ...
> };
Only some of these are currently included in the process_types struct,
but this demonstrates the extensibility of the architecture and room
for future centralization. Do you see any items in this set that
aren't currently included but are must-haves for this round?
> Then there should be a single startup routine for all postmaster
> children. Since most of the startup is actually shared between all the
> different types of processes, we can declutter it a lot.
Agreed.
> When starting a process postmaster would just specify the process type,
> and if relevant, an argument (struct Port for backends, whatever
> relevant for bgworkers etc) . Generic code should handle all the work
> until the process type entry point - and likely we should move more work
> from the individual process types into generic code.
>
> If a process is 'only_one_exists' (to be renamed), the generic code
> would also (in postmaster) register the pid as
> subprocess_pids[type] = pid;
> which would make it easy to only have per-type code in the few locations
> that need to be aware, instead of many locations in
> postmaster.c. Perhaps also some shared memory location.
All of this sounds good.
> Coming back to earth, from green field imagination land: I think the
> patchset does go in that direction (to be fair, I think I outlined
> something like the above elsewhere in this thread in a discussion with
> Mike). And that's good.
Thanks for chiming in. Is there anything else you think needs to go in
this version to push things along -- besides fixing Windows builds, of
course?
Regards,
--
Mike Palmiotto
https://crunchydata.com
Attachments: download all (11 files, .tar.gz)
0001-Add-subprocess-infrastructure.patch (text/x-patch, 14.0K) download
From 2c91e804533193a2dbc0841f815033a74bf0e980 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Mon, 9 Mar 2020 22:12:10 +0000
Subject: [PATCH 01/11] Add subprocess infrastructure
This commit introduces a new infrastructure for postmaster subprocesses
initialization, forking, exec'ing and main function control. The intent
is to have all subprocess control information live in the process_types
struct, defined in subprocess.h. As more processes are added to the
struct, the fields will be expanded to differentiate actions.
There are several advantages to such an infrastructure:
1) Centralized definition of subprocess information
2) Standardized function arguments/naming schemes
3) Code minimization and removal of duplicate code
---
src/backend/bootstrap/bootstrap.c | 98 ++++++---------------------
src/backend/postmaster/Makefile | 1 +
src/backend/postmaster/bgwriter.c | 2 +-
src/backend/postmaster/checkpointer.c | 2 +-
src/backend/postmaster/startup.c | 2 +-
src/backend/postmaster/subprocess.c | 62 +++++++++++++++++
src/backend/postmaster/walwriter.c | 2 +-
src/backend/replication/walreceiver.c | 2 +-
src/include/bootstrap/bootstrap.h | 2 +
src/include/postmaster/bgwriter.h | 4 +-
src/include/postmaster/startup.h | 5 +-
src/include/postmaster/subprocess.h | 44 ++++++++++++
src/include/postmaster/walwriter.h | 2 +-
src/include/replication/walreceiver.h | 2 +-
14 files changed, 143 insertions(+), 87 deletions(-)
create mode 100644 src/backend/postmaster/subprocess.c
create mode 100644 src/include/postmaster/subprocess.h
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 5480a024e0..38d45d9163 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -35,6 +35,7 @@
#include "pgstat.h"
#include "postmaster/bgwriter.h"
#include "postmaster/startup.h"
+#include "postmaster/subprocess.h"
#include "postmaster/walwriter.h"
#include "replication/walreceiver.h"
#include "storage/bufmgr.h"
@@ -56,8 +57,6 @@ uint32 bootstrap_data_checksum_version = 0; /* No checksum */
#define ALLOC(t, c) \
((t *) MemoryContextAllocZero(TopMemoryContext, (unsigned)(c) * sizeof(t)))
-static void CheckerModeMain(void);
-static void BootstrapModeMain(void);
static void bootstrap_signals(void);
static void ShutdownAuxiliaryProcess(int code, Datum arg);
static Form_pg_attribute AllocateAttribute(void);
@@ -314,26 +313,14 @@ AuxiliaryProcessMain(int argc, char *argv[])
proc_exit(1);
}
- switch (MyAuxProcType)
- {
- case StartupProcess:
- MyBackendType = B_STARTUP;
- break;
- case BgWriterProcess:
- MyBackendType = B_BG_WRITER;
- break;
- case CheckpointerProcess:
- MyBackendType = B_CHECKPOINTER;
- break;
- case WalWriterProcess:
- MyBackendType = B_WAL_WRITER;
- break;
- case WalReceiverProcess:
- MyBackendType = B_WAL_RECEIVER;
- break;
- default:
- MyBackendType = B_INVALID;
- }
+ /*
+ * We've read the arguments and know what backend type we are.
+ */
+ InitMySubprocess((SubprocessType)MyAuxProcType);
+
+ /*
+ * Identify myself via ps
+ */
if (IsUnderPostmaster)
init_ps_display(NULL);
@@ -418,56 +405,10 @@ AuxiliaryProcessMain(int argc, char *argv[])
*/
SetProcessingMode(NormalProcessing);
- switch (MyAuxProcType)
- {
- case CheckerProcess:
- /* don't set signals, they're useless here */
- CheckerModeMain();
- proc_exit(1); /* should never return */
-
- case BootstrapProcess:
-
- /*
- * There was a brief instant during which mode was Normal; this is
- * okay. We need to be in bootstrap mode during BootStrapXLOG for
- * the sake of multixact initialization.
- */
- SetProcessingMode(BootstrapProcessing);
- bootstrap_signals();
- BootStrapXLOG();
- BootstrapModeMain();
- proc_exit(1); /* should never return */
-
- case StartupProcess:
- /* don't set signals, startup process has its own agenda */
- StartupProcessMain();
- proc_exit(1); /* should never return */
-
- case BgWriterProcess:
- /* don't set signals, bgwriter has its own agenda */
- BackgroundWriterMain();
- proc_exit(1); /* should never return */
-
- case CheckpointerProcess:
- /* don't set signals, checkpointer has its own agenda */
- CheckpointerMain();
- proc_exit(1); /* should never return */
-
- case WalWriterProcess:
- /* don't set signals, walwriter has its own agenda */
- InitXLOGAccess();
- WalWriterMain();
- proc_exit(1); /* should never return */
-
- case WalReceiverProcess:
- /* don't set signals, walreceiver has its own agenda */
- WalReceiverMain();
- proc_exit(1); /* should never return */
-
- default:
- elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
- proc_exit(1);
- }
+ /* Now jump into the subprocess main function and never look back! */
+ MySubprocess->entrypoint(argc, argv);
+
+ proc_exit(1); /* should never return */
}
/*
@@ -476,8 +417,8 @@ AuxiliaryProcessMain(int argc, char *argv[])
* settings). Since, in fact, that was already done by BaseInit(),
* we have nothing more to do here.
*/
-static void
-CheckerModeMain(void)
+void
+CheckerModeMain(int argc, char *argv[])
{
proc_exit(0);
}
@@ -489,11 +430,15 @@ CheckerModeMain(void)
* The bootstrap backend doesn't speak SQL, but instead expects
* commands in a special bootstrap language.
*/
-static void
-BootstrapModeMain(void)
+void
+BootstrapModeMain(int argc, char *argv[])
{
int i;
+ SetProcessingMode(BootstrapProcessing);
+ bootstrap_signals();
+ BootStrapXLOG();
+
Assert(!IsUnderPostmaster);
Assert(IsBootstrapProcessingMode());
@@ -508,7 +453,6 @@ BootstrapModeMain(void)
* Do backend-like initialization for bootstrap mode
*/
InitProcess();
-
InitPostgres(NULL, InvalidOid, NULL, InvalidOid, NULL, false);
/* Initialize stuff for bootstrap-file processing */
diff --git a/src/backend/postmaster/Makefile b/src/backend/postmaster/Makefile
index bfdf6a833d..52b4c2142c 100644
--- a/src/backend/postmaster/Makefile
+++ b/src/backend/postmaster/Makefile
@@ -23,6 +23,7 @@ OBJS = \
pgstat.o \
postmaster.o \
startup.o \
+ subprocess.o \
syslogger.o \
walwriter.o
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 069e27e427..b2170938f3 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -91,7 +91,7 @@ static XLogRecPtr last_snapshot_lsn = InvalidXLogRecPtr;
* basic execution environment, but not enabled signals yet.
*/
void
-BackgroundWriterMain(void)
+BackgroundWriterMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
MemoryContext bgwriter_context;
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index e354a78725..83f7be1b1e 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -180,7 +180,7 @@ static void ReqCheckpointHandler(SIGNAL_ARGS);
* basic execution environment, but not enabled signals yet.
*/
void
-CheckpointerMain(void)
+CheckpointerMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
MemoryContext checkpointer_context;
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index c2250d7d4e..9351820d69 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -133,7 +133,7 @@ HandleStartupProcInterrupts(void)
* ----------------------------------
*/
void
-StartupProcessMain(void)
+StartupProcessMain(int argc, char *argv[])
{
/*
* Properly accept or ignore signals the postmaster might send us.
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
new file mode 100644
index 0000000000..3e7a45bf10
--- /dev/null
+++ b/src/backend/postmaster/subprocess.c
@@ -0,0 +1,62 @@
+/*-------------------------------------------------------------------------
+ *
+ * subprocess.c
+ *
+ * Copyright (c) 2004-2020, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/postmaster/syslogger.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "bootstrap/bootstrap.h"
+#include "postmaster/bgwriter.h"
+#include "postmaster/postmaster.h"
+#include "postmaster/startup.h"
+#include "postmaster/subprocess.h"
+#include "postmaster/walwriter.h"
+#include "replication/walreceiver.h"
+
+SubprocessType MySubprocessType;
+PgSubprocess *MySubprocess;
+
+static PgSubprocess process_types[] = {
+ {
+ .desc = "checker",
+ .entrypoint = CheckerModeMain
+ },
+ {
+ .desc = "bootstrap",
+ .entrypoint = BootstrapModeMain
+ },
+ {
+ .desc = "startup",
+ .entrypoint = StartupProcessMain
+ },
+ {
+ .desc = "background writer",
+ .entrypoint = BackgroundWriterMain
+ },
+ {
+ .desc = "checkpointer",
+ .entrypoint = CheckpointerMain
+ },
+ {
+ .desc = "wal writer",
+ .entrypoint = WalWriterMain
+ },
+ {
+ .desc = "wal receiver",
+ .entrypoint = WalReceiverMain
+ }
+};
+
+void
+InitMySubprocess(SubprocessType type)
+{
+ MySubprocessType = type;
+ MySubprocess = &process_types[type];
+ MySubprocess->desc = gettext(MySubprocess->desc);
+}
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index 45a2757969..3a0cec3371 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -85,7 +85,7 @@ int WalWriterFlushAfter = 128;
* basic execution environment, but not enabled signals yet.
*/
void
-WalWriterMain(void)
+WalWriterMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
MemoryContext walwriter_context;
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index 25e0333c9e..cd39a8c982 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -165,7 +165,7 @@ ProcessWalRcvInterrupts(void)
/* Main entry point for walreceiver process */
void
-WalReceiverMain(void)
+WalReceiverMain(int argc, char *argv[])
{
char conninfo[MAXCONNINFO];
char *tmp_conninfo;
diff --git a/src/include/bootstrap/bootstrap.h b/src/include/bootstrap/bootstrap.h
index b67fa9acc1..0247ca7f23 100644
--- a/src/include/bootstrap/bootstrap.h
+++ b/src/include/bootstrap/bootstrap.h
@@ -31,6 +31,8 @@ extern Relation boot_reldesc;
extern Form_pg_attribute attrtypes[MAXATTR];
extern int numattr;
+extern void CheckerModeMain(int argc, char *argv[]);
+extern void BootstrapModeMain(int argc, char *argv[]);
extern void AuxiliaryProcessMain(int argc, char *argv[]) pg_attribute_noreturn();
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..6fb1ee1d6d 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -27,8 +27,8 @@ extern int CheckPointTimeout;
extern int CheckPointWarning;
extern double CheckPointCompletionTarget;
-extern void BackgroundWriterMain(void) pg_attribute_noreturn();
-extern void CheckpointerMain(void) pg_attribute_noreturn();
+extern void BackgroundWriterMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern void CheckpointerMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void RequestCheckpoint(int flags);
extern void CheckpointWriteDelay(int flags, double progress);
diff --git a/src/include/postmaster/startup.h b/src/include/postmaster/startup.h
index 9f59c1ffa3..6a8108dfe0 100644
--- a/src/include/postmaster/startup.h
+++ b/src/include/postmaster/startup.h
@@ -13,10 +13,13 @@
#define _STARTUP_H
extern void HandleStartupProcInterrupts(void);
-extern void StartupProcessMain(void) pg_attribute_noreturn();
extern void PreRestoreCommand(void);
extern void PostRestoreCommand(void);
extern bool IsPromoteTriggered(void);
extern void ResetPromoteTriggered(void);
+/* Startup subprocess functions */
+extern void StartupProcessMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern bool StartupCleanup(int child_errno);
+
#endif /* _STARTUP_H */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
new file mode 100644
index 0000000000..56e8edf2d8
--- /dev/null
+++ b/src/include/postmaster/subprocess.h
@@ -0,0 +1,44 @@
+/*-------------------------------------------------------------------------
+ *
+ * subprocess.h
+ *
+ * Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ *
+ * src/include/postmaster/subprocess.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef SUBPROCESS_H
+#define SUBPROCESS_H
+
+#include "postmaster.h"
+
+typedef enum
+{
+ NoProcessType = -1,
+ CheckerType = 0,
+ BootstrapType,
+ StartupType,
+ BgWriterType,
+ CheckpointerType,
+ WalWriterType,
+ WalReceiverType, /* end of Auxiliary Process Forks */
+
+ NUMSUBPROCESSTYPES /* Must be last! */
+} SubprocessType;
+
+typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
+
+/* Current subprocess initializer */
+extern void InitMySubprocess(SubprocessType type);
+
+typedef struct PgSubprocess
+{
+ const char *desc;
+ SubprocessEntryPoint entrypoint;
+} PgSubprocess;
+
+extern SubprocessType MySubprocessType;
+extern PGDLLIMPORT PgSubprocess *MySubprocess;
+
+#endif /* SUBPROCESS_H */
diff --git a/src/include/postmaster/walwriter.h b/src/include/postmaster/walwriter.h
index 011dc26a6e..596c437c54 100644
--- a/src/include/postmaster/walwriter.h
+++ b/src/include/postmaster/walwriter.h
@@ -16,6 +16,6 @@
extern int WalWriterDelay;
extern int WalWriterFlushAfter;
-extern void WalWriterMain(void) pg_attribute_noreturn();
+extern void WalWriterMain(int argc, char *argv[]) pg_attribute_noreturn();
#endif /* _WALWRITER_H */
diff --git a/src/include/replication/walreceiver.h b/src/include/replication/walreceiver.h
index e08afc6548..2025f19ab3 100644
--- a/src/include/replication/walreceiver.h
+++ b/src/include/replication/walreceiver.h
@@ -311,7 +311,7 @@ walrcv_clear_result(WalRcvExecResult *walres)
}
/* prototypes for functions in walreceiver.c */
-extern void WalReceiverMain(void) pg_attribute_noreturn();
+extern void WalReceiverMain(int argc, char *argv[]);
extern void ProcessWalRcvInterrupts(void);
/* prototypes for functions in walreceiverfuncs.c */
--
2.21.1
0002-Use-centralized-StartSubprocess-for-aux-procs.patch (text/x-patch, 14.9K) download
From 6a43c11e09c88f4b208bc5a9e38b9f8290375aa2 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 10 Mar 2020 14:58:07 +0000
Subject: [PATCH 02/11] Use centralized StartSubprocess for aux procs
Modifies StartChildProcess and moves the auxiliary processes under its
control. This is the next step toward making use of the Subprocess
infrastructure.
This patch introduces the fork_failure function, which may be
implemented by each subprocess to control failure when attempting to
fork the subprocess. This is used by the startup subprocess to exit
postmaster on fork failure. Future subprocesses may use it to do cleanup
(and optionally exit postmaster) after a fork failure.
Additionally, the patch adds a new backend variable for establishing a
backend variable to be shared across fork/exec.
---
src/backend/postmaster/postmaster.c | 156 ++++++++++++++--------------
src/backend/postmaster/startup.c | 12 +++
src/backend/postmaster/subprocess.c | 35 +++++--
src/include/postmaster/startup.h | 2 +-
src/include/postmaster/subprocess.h | 4 +
5 files changed, 121 insertions(+), 88 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 2b9ab32293..4b86aeec2d 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -114,6 +114,7 @@
#include "postmaster/fork_process.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "postmaster/syslogger.h"
#include "replication/logicallauncher.h"
#include "replication/walsender.h"
@@ -191,7 +192,8 @@ static Backend *ShmemBackendArray;
BackgroundWorker *MyBgworkerEntry = NULL;
-
+/* Struct containing postmaster subprocess control info */
+SubprocessType MySubprocessType;
/* The socket number we are listening for connections on */
int PostPortNumber;
@@ -421,7 +423,7 @@ static int CountChildren(int target);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
-static pid_t StartChildProcess(AuxProcType type);
+static pid_t StartSubprocess(SubprocessType type);
static void StartAutovacuumWorker(void);
static void MaybeStartWalReceiver(void);
static void InitPostmasterDeathWatchHandle(void);
@@ -507,6 +509,9 @@ typedef struct
TimestampTz PgStartTime;
TimestampTz PgReloadTime;
pg_time_t first_syslogger_file_time;
+
+ SubprocessType MySubprocessType;
+
bool redirection_done;
bool IsBinaryUpgrade;
int max_safe_fds;
@@ -538,11 +543,6 @@ static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
-#define StartupDataBase() StartChildProcess(StartupProcess)
-#define StartBackgroundWriter() StartChildProcess(BgWriterProcess)
-#define StartCheckpointer() StartChildProcess(CheckpointerProcess)
-#define StartWalWriter() StartChildProcess(WalWriterProcess)
-#define StartWalReceiver() StartChildProcess(WalReceiverProcess)
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -1389,7 +1389,7 @@ PostmasterMain(int argc, char *argv[])
/*
* We're ready to rock and roll...
*/
- StartupPID = StartupDataBase();
+ StartupPID = StartSubprocess(StartupType);
Assert(StartupPID != 0);
StartupStatus = STARTUP_RUNNING;
pmState = PM_STARTUP;
@@ -1750,9 +1750,9 @@ ServerLoop(void)
pmState == PM_HOT_STANDBY)
{
if (CheckpointerPID == 0)
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
if (BgWriterPID == 0)
- BgWriterPID = StartBackgroundWriter();
+ BgWriterPID = StartSubprocess(BgWriterType);
}
/*
@@ -1761,7 +1761,7 @@ ServerLoop(void)
* be writing any new WAL).
*/
if (WalWriterPID == 0 && pmState == PM_RUN)
- WalWriterPID = StartWalWriter();
+ WalWriterPID = StartSubprocess(WalWriterType);
/*
* If we have lost the autovacuum launcher, try to start a new one. We
@@ -3042,11 +3042,11 @@ reaper(SIGNAL_ARGS)
* if this fails, we'll just try again later.
*/
if (CheckpointerPID == 0)
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
if (BgWriterPID == 0)
- BgWriterPID = StartBackgroundWriter();
+ BgWriterPID = StartSubprocess(BgWriterType);
if (WalWriterPID == 0)
- WalWriterPID = StartWalWriter();
+ WalWriterPID = StartSubprocess(WalWriterType);
/*
* Likewise, start other special children as needed. In a restart
@@ -3860,7 +3860,7 @@ PostmasterStateMachine(void)
Assert(Shutdown > NoShutdown);
/* Start the checkpointer if not running */
if (CheckpointerPID == 0)
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
/* And tell it to shut down */
if (CheckpointerPID != 0)
{
@@ -4001,7 +4001,7 @@ PostmasterStateMachine(void)
reset_shared();
- StartupPID = StartupDataBase();
+ StartupPID = StartSubprocess(StartupType);
Assert(StartupPID != 0);
StartupStatus = STARTUP_RUNNING;
pmState = PM_STARTUP;
@@ -4917,6 +4917,10 @@ SubPostmasterMain(int argc, char *argv[])
errmsg("out of memory")));
#endif
+ /* We should only be here once per fork */
+ Assert(!MySubprocess);
+ InitMySubprocess(MySubprocessType);
+
/*
* If appropriate, physically re-attach to shared memory segment. We want
* to do this before going any further to ensure that we can attach at the
@@ -5037,19 +5041,6 @@ SubPostmasterMain(int argc, char *argv[])
/* And run the backend */
BackendRun(&port); /* does not return */
}
- if (strcmp(argv[1], "--forkboot") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitAuxiliaryProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
- }
if (strcmp(argv[1], "--forkavlauncher") == 0)
{
/* Restore basic shared memory pointers */
@@ -5117,6 +5108,20 @@ SubPostmasterMain(int argc, char *argv[])
SysLoggerMain(argc, argv); /* does not return */
}
+ if (MySubprocess->needs_aux_proc)
+ {
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
+
+ /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ InitAuxiliaryProcess();
+
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores();
+
+ AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
+ }
+
abort(); /* shouldn't get here */
}
#endif /* EXEC_BACKEND */
@@ -5198,9 +5203,9 @@ sigusr1_handler(SIGNAL_ARGS)
* we'll just try again later.
*/
Assert(CheckpointerPID == 0);
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
Assert(BgWriterPID == 0);
- BgWriterPID = StartBackgroundWriter();
+ BgWriterPID = StartSubprocess(BgWriterType);
/*
* Start the archiver if we're responsible for (re-)archiving received
@@ -5425,42 +5430,49 @@ CountChildren(int target)
return cnt;
}
-
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
- * "type" determines what kind of child will be started. All child types
- * initially go to AuxiliaryProcessMain, which will handle common setup.
+ * "type" determines what kind of child will be started. This code relies
+ * heavily on MySubprocess, which is an entry in the process_types struct in
+ * subprocess.c. All auxiliary processes go through AuxiliaryProcessMain after
+ * forking. Everything else goes through the designated entrypoint.
*
- * Return value of StartChildProcess is subprocess' PID, or 0 if failed
- * to start subprocess.
+ * Return value of StartChildProcess is subprocess' PID on success. On failure,
+ * the subprocess cleanup function decides whether or not to panic.
*/
static pid_t
-StartChildProcess(AuxProcType type)
+StartSubprocess(SubprocessType type)
{
pid_t pid;
- char *av[10];
- int ac = 0;
+ char *argv[10];
+ int argc = 0;
char typebuf[32];
+#ifdef EXEC_BACKEND
+ char forkname[32];
+#endif
/*
- * Set up command-line arguments for subprocess
+ * Get new subprocess data every time we start a new subprocess.
*/
- av[ac++] = "postgres";
+ InitMySubprocess(type);
+
+ argv[argc++] = "postgres";
#ifdef EXEC_BACKEND
- av[ac++] = "--forkboot";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
+ snprintf(forkname, sizeof(forkname), "--fork%s", MySubprocess->name);
+ argv[argc++] = forkname;
+ argv[argc++] = NULL;
#endif
snprintf(typebuf, sizeof(typebuf), "-x%d", type);
- av[ac++] = typebuf;
+ argv[argc++] = typebuf;
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ argv[argc] = NULL;
+ Assert(argc < lengthof(argv));
#ifdef EXEC_BACKEND
- pid = postmaster_forkexec(ac, av);
+ pid = postmaster_forkexec(argc, argv);
#else /* !EXEC_BACKEND */
pid = fork_process();
@@ -5476,7 +5488,7 @@ StartChildProcess(AuxProcType type)
MemoryContextDelete(PostmasterContext);
PostmasterContext = NULL;
- AuxiliaryProcessMain(ac, av);
+ AuxiliaryProcessMain(argc, argv);
ExitPostmaster(0);
}
#endif /* EXEC_BACKEND */
@@ -5487,40 +5499,20 @@ StartChildProcess(AuxProcType type)
int save_errno = errno;
errno = save_errno;
- switch (type)
- {
- case StartupProcess:
- ereport(LOG,
- (errmsg("could not fork startup process: %m")));
- break;
- case BgWriterProcess:
- ereport(LOG,
- (errmsg("could not fork background writer process: %m")));
- break;
- case CheckpointerProcess:
- ereport(LOG,
- (errmsg("could not fork checkpointer process: %m")));
- break;
- case WalWriterProcess:
- ereport(LOG,
- (errmsg("could not fork WAL writer process: %m")));
- break;
- case WalReceiverProcess:
- ereport(LOG,
- (errmsg("could not fork WAL receiver process: %m")));
- break;
- default:
- ereport(LOG,
- (errmsg("could not fork process: %m")));
- break;
- }
+ ereport(LOG,
+ (errmsg("could not fork %s process: %m", MySubprocess->desc)));
/*
- * fork failure is fatal during startup, but there's no need to choke
- * immediately if starting other child types fails.
+ * Handle fork-failure per subprocess type. Each subprocess handles
+ * fork-failure by returning a boolean which tells postmaster whether or
+ * not to panic.
*/
- if (type == StartupProcess)
- ExitPostmaster(1);
+ if (MySubprocess->fork_failure)
+ {
+ if (MySubprocess->fork_failure(save_errno))
+ ExitPostmaster(1);
+ }
+
return 0;
}
@@ -5638,7 +5630,7 @@ MaybeStartWalReceiver(void)
pmState == PM_HOT_STANDBY || pmState == PM_WAIT_READONLY) &&
Shutdown == NoShutdown)
{
- WalReceiverPID = StartWalReceiver();
+ WalReceiverPID = StartSubprocess(WalReceiverType);
if (WalReceiverPID != 0)
WalReceiverRequested = false;
/* else leave the flag set, so we'll try again later */
@@ -6198,6 +6190,8 @@ save_backend_variables(BackendParameters *param, Port *port,
param->PgReloadTime = PgReloadTime;
param->first_syslogger_file_time = first_syslogger_file_time;
+ param->MySubprocessType = MySubprocessType;
+
param->redirection_done = redirection_done;
param->IsBinaryUpgrade = IsBinaryUpgrade;
param->max_safe_fds = max_safe_fds;
@@ -6433,6 +6427,8 @@ restore_backend_variables(BackendParameters *param, Port *port)
PgReloadTime = param->PgReloadTime;
first_syslogger_file_time = param->first_syslogger_file_time;
+ MySubprocessType = param->MySubprocessType;
+
redirection_done = param->redirection_done;
IsBinaryUpgrade = param->IsBinaryUpgrade;
max_safe_fds = param->max_safe_fds;
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index 9351820d69..1893af2e7b 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -176,6 +176,18 @@ StartupProcessMain(int argc, char *argv[])
proc_exit(0);
}
+/*
+ * StartupForkFailure
+ *
+ * Tell the postmaster to exit/panic on failure to fork.
+ */
+bool
+StartupForkFailure(int fork_errno)
+{
+ /* Panic! */
+ return true;
+}
+
void
PreRestoreCommand(void)
{
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 3e7a45bf10..7e96678dfb 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -24,32 +24,53 @@ PgSubprocess *MySubprocess;
static PgSubprocess process_types[] = {
{
+ .name = "boot",
.desc = "checker",
- .entrypoint = CheckerModeMain
+ .needs_aux_proc = true,
+ .entrypoint = CheckerModeMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "bootstrap",
- .entrypoint = BootstrapModeMain
+ .needs_aux_proc = true,
+ .entrypoint = BootstrapModeMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "startup",
- .entrypoint = StartupProcessMain
+ .needs_aux_proc = true,
+ .entrypoint = StartupProcessMain,
+ .fork_failure = StartupForkFailure
},
{
+ .name = "boot",
.desc = "background writer",
- .entrypoint = BackgroundWriterMain
+ .needs_aux_proc = true,
+ .entrypoint = BackgroundWriterMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "checkpointer",
- .entrypoint = CheckpointerMain
+ .needs_aux_proc = true,
+ .entrypoint = CheckpointerMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "wal writer",
- .entrypoint = WalWriterMain
+ .needs_aux_proc = true,
+ .entrypoint = WalWriterMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "wal receiver",
- .entrypoint = WalReceiverMain
+ .needs_aux_proc = true,
+ .entrypoint = WalReceiverMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/postmaster/startup.h b/src/include/postmaster/startup.h
index 6a8108dfe0..c182c76960 100644
--- a/src/include/postmaster/startup.h
+++ b/src/include/postmaster/startup.h
@@ -20,6 +20,6 @@ extern void ResetPromoteTriggered(void);
/* Startup subprocess functions */
extern void StartupProcessMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern bool StartupCleanup(int child_errno);
+extern bool StartupForkFailure(int fork_errno);
#endif /* _STARTUP_H */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 56e8edf2d8..33f6a7ab04 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -28,14 +28,18 @@ typedef enum
} SubprocessType;
typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
+typedef bool (*SubprocessForkFailure) (int fork_errno);
/* Current subprocess initializer */
extern void InitMySubprocess(SubprocessType type);
typedef struct PgSubprocess
{
+ const char *name;
const char *desc;
+ bool needs_aux_proc;
SubprocessEntryPoint entrypoint;
+ SubprocessForkFailure fork_failure;
} PgSubprocess;
extern SubprocessType MySubprocessType;
--
2.21.1
0003-Add-AutoVacLauncherType-to-subprocess-struct.patch (text/x-patch, 7.5K) download
From 0e41e5211cdb3c8268f742ef00655fa93c41fe3c Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 10 Mar 2020 22:29:44 +0000
Subject: [PATCH 03/11] Add AutoVacLauncherType to subprocess struct
---
src/backend/postmaster/autovacuum.c | 68 ++---------------------------
src/backend/postmaster/postmaster.c | 36 ++++++++-------
src/backend/postmaster/subprocess.c | 8 ++++
src/include/postmaster/autovacuum.h | 3 +-
src/include/postmaster/subprocess.h | 1 +
5 files changed, 35 insertions(+), 81 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index da75e755f0..7397b944d3 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -95,6 +95,7 @@
#include "storage/procsignal.h"
#include "storage/sinvaladt.h"
#include "storage/smgr.h"
+#include "postmaster/subprocess.h"
#include "tcop/tcopprot.h"
#include "utils/fmgroids.h"
#include "utils/fmgrprotos.h"
@@ -303,11 +304,9 @@ static WorkerInfo MyWorkerInfo = NULL;
int AutovacuumLauncherPid = 0;
#ifdef EXEC_BACKEND
-static pid_t avlauncher_forkexec(void);
static pid_t avworker_forkexec(void);
#endif
NON_EXEC_STATIC void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-NON_EXEC_STATIC void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
static Oid do_start_worker(void);
static void HandleAutoVacLauncherInterrupts(void);
@@ -353,26 +352,6 @@ static void autovac_refresh_stats(void);
********************************************************************/
#ifdef EXEC_BACKEND
-/*
- * forkexec routine for the autovacuum launcher process.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avlauncher_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavlauncher";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
/*
* We need this set from the outside, before InitProcess is called
@@ -384,56 +363,17 @@ AutovacuumLauncherIAm(void)
}
#endif
-/*
- * Main entry point for autovacuum launcher process, to be called from the
- * postmaster.
- */
-int
-StartAutoVacLauncher(void)
-{
- pid_t AutoVacPID;
-
-#ifdef EXEC_BACKEND
- switch ((AutoVacPID = avlauncher_forkexec()))
-#else
- switch ((AutoVacPID = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum launcher process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacLauncherMain(0, NULL);
- break;
-#endif
- default:
- return (int) AutoVacPID;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
/*
* Main loop for the autovacuum launcher process.
*/
-NON_EXEC_STATIC void
-AutoVacLauncherMain(int argc, char *argv[])
+void
+AutoVacLauncherMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
am_autovacuum_launcher = true;
- MyBackendType = B_AUTOVAC_LAUNCHER;
+ /* Identify myself via ps */
init_ps_display(NULL);
ereport(DEBUG1,
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 4b86aeec2d..2251303589 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1773,7 +1773,7 @@ ServerLoop(void)
(AutoVacuumingActive() || start_autovac_launcher) &&
pmState == PM_RUN)
{
- AutoVacPID = StartAutoVacLauncher();
+ AutoVacPID = StartSubprocess(AutoVacuumLauncherType);
if (AutoVacPID != 0)
start_autovac_launcher = false; /* signal processed */
}
@@ -3053,7 +3053,7 @@ reaper(SIGNAL_ARGS)
* situation, some of them may be alive already.
*/
if (!IsBinaryUpgrade && AutoVacuumingActive() && AutoVacPID == 0)
- AutoVacPID = StartAutoVacLauncher();
+ AutoVacPID = StartSubprocess(AutoVacuumLauncherType);
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = pgarch_start();
if (PgStatPID == 0)
@@ -5041,19 +5041,6 @@ SubPostmasterMain(int argc, char *argv[])
/* And run the backend */
BackendRun(&port); /* does not return */
}
- if (strcmp(argv[1], "--forkavlauncher") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
- }
if (strcmp(argv[1], "--forkavworker") == 0)
{
/* Restore basic shared memory pointers */
@@ -5121,6 +5108,19 @@ SubPostmasterMain(int argc, char *argv[])
AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
}
+ else
+ {
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
+
+ /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ InitProcess();
+
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores();
+
+ MySubprocess->entrypoint(argc - 2, argv + 2);
+ }
abort(); /* shouldn't get here */
}
@@ -5488,7 +5488,11 @@ StartSubprocess(SubprocessType type)
MemoryContextDelete(PostmasterContext);
PostmasterContext = NULL;
- AuxiliaryProcessMain(argc, argv);
+ if (MySubprocess->needs_aux_proc)
+ AuxiliaryProcessMain(argc, argv);
+ else
+ MySubprocess->entrypoint(argc, argv);
+
ExitPostmaster(0);
}
#endif /* EXEC_BACKEND */
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 7e96678dfb..9aa60cc93e 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -12,6 +12,7 @@
*/
#include "postgres.h"
#include "bootstrap/bootstrap.h"
+#include "postmaster/autovacuum.h"
#include "postmaster/bgwriter.h"
#include "postmaster/postmaster.h"
#include "postmaster/startup.h"
@@ -71,6 +72,13 @@ static PgSubprocess process_types[] = {
.needs_aux_proc = true,
.entrypoint = WalReceiverMain,
.fork_failure = NULL
+ },
+ {
+ .name = "avlauncher",
+ .desc = "autovacuum launcher",
+ .needs_aux_proc = false,
+ .entrypoint = AutoVacLauncherMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index d40ed55531..08050cd81c 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -58,6 +58,8 @@ extern void autovac_init(void);
extern int StartAutoVacLauncher(void);
extern int StartAutoVacWorker(void);
+extern void AutoVacLauncherMain(int argc, char *argv[]);
+
/* called from postmaster when a worker could not be forked */
extern void AutoVacWorkerFailed(void);
@@ -65,7 +67,6 @@ extern void AutoVacWorkerFailed(void);
extern void AutoVacuumUpdateDelay(void);
#ifdef EXEC_BACKEND
-extern void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void AutovacuumWorkerIAm(void);
extern void AutovacuumLauncherIAm(void);
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 33f6a7ab04..9e3c743333 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -23,6 +23,7 @@ typedef enum
CheckpointerType,
WalWriterType,
WalReceiverType, /* end of Auxiliary Process Forks */
+ AutoVacuumLauncherType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.21.1
0004-Add-AutoVacuumWorkerType-to-subprocess-struct.patch (text/x-patch, 6.4K) download
From 35aa6817b8e64f37e51a1d20be70fc50b4ea0156 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 10 Mar 2020 23:33:39 +0000
Subject: [PATCH 04/11] Add AutoVacuumWorkerType to subprocess struct
---
src/backend/postmaster/autovacuum.c | 71 +----------------------------
src/backend/postmaster/postmaster.c | 17 +------
src/backend/postmaster/subprocess.c | 7 +++
src/include/postmaster/autovacuum.h | 4 +-
src/include/postmaster/subprocess.h | 1 +
5 files changed, 14 insertions(+), 86 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 7397b944d3..cf0a74a4bb 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -83,7 +83,6 @@
#include "nodes/makefuncs.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "storage/bufmgr.h"
@@ -303,11 +302,6 @@ static WorkerInfo MyWorkerInfo = NULL;
/* PID of launcher, valid only in worker while shutting down */
int AutovacuumLauncherPid = 0;
-#ifdef EXEC_BACKEND
-static pid_t avworker_forkexec(void);
-#endif
-NON_EXEC_STATIC void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-
static Oid do_start_worker(void);
static void HandleAutoVacLauncherInterrupts(void);
static void AutoVacLauncherShutdown(void) pg_attribute_noreturn();
@@ -1364,27 +1358,6 @@ avl_sigusr2_handler(SIGNAL_ARGS)
********************************************************************/
#ifdef EXEC_BACKEND
-/*
- * forkexec routines for the autovacuum worker.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avworker_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavworker";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
/*
* We need this set from the outside, before InitProcess is called
*/
@@ -1395,50 +1368,10 @@ AutovacuumWorkerIAm(void)
}
#endif
-/*
- * Main entry point for autovacuum worker process.
- *
- * This code is heavily based on pgarch.c, q.v.
- */
-int
-StartAutoVacWorker(void)
-{
- pid_t worker_pid;
-
-#ifdef EXEC_BACKEND
- switch ((worker_pid = avworker_forkexec()))
-#else
- switch ((worker_pid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum worker process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacWorkerMain(0, NULL);
- break;
-#endif
- default:
- return (int) worker_pid;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
/*
* AutoVacWorkerMain
*/
-NON_EXEC_STATIC void
+void
AutoVacWorkerMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
@@ -1446,7 +1379,7 @@ AutoVacWorkerMain(int argc, char *argv[])
am_autovacuum_worker = true;
- MyBackendType = B_AUTOVAC_WORKER;
+ /* Identify myself via ps */
init_ps_display(NULL);
SetProcessingMode(InitProcessing);
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 2251303589..21bbbf8194 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -5041,19 +5041,6 @@ SubPostmasterMain(int argc, char *argv[])
/* And run the backend */
BackendRun(&port); /* does not return */
}
- if (strcmp(argv[1], "--forkavworker") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
- }
if (strncmp(argv[1], "--forkbgworker=", 15) == 0)
{
int shmem_slot;
@@ -5573,7 +5560,7 @@ StartAutovacuumWorker(void)
bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
bn->bgworker_notify = false;
- bn->pid = StartAutoVacWorker();
+ bn->pid = StartSubprocess(AutoVacuumWorkerType);
if (bn->pid > 0)
{
bn->bkend_type = BACKEND_TYPE_AUTOVAC;
@@ -5587,7 +5574,7 @@ StartAutovacuumWorker(void)
/*
* fork failed, fall through to report -- actual error message was
- * logged by StartAutoVacWorker
+ * logged by StartSubprocess
*/
(void) ReleasePostmasterChildSlot(bn->child_slot);
free(bn);
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 9aa60cc93e..0fbd067310 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -79,6 +79,13 @@ static PgSubprocess process_types[] = {
.needs_aux_proc = false,
.entrypoint = AutoVacLauncherMain,
.fork_failure = NULL
+ },
+ {
+ .name = "avworker",
+ .desc = "autovacuum worker",
+ .needs_aux_proc = false,
+ .entrypoint = AutoVacWorkerMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 08050cd81c..51546fd4c9 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -58,7 +58,8 @@ extern void autovac_init(void);
extern int StartAutoVacLauncher(void);
extern int StartAutoVacWorker(void);
-extern void AutoVacLauncherMain(int argc, char *argv[]);
+extern void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
/* called from postmaster when a worker could not be forked */
extern void AutoVacWorkerFailed(void);
@@ -67,7 +68,6 @@ extern void AutoVacWorkerFailed(void);
extern void AutoVacuumUpdateDelay(void);
#ifdef EXEC_BACKEND
-extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void AutovacuumWorkerIAm(void);
extern void AutovacuumLauncherIAm(void);
#endif
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 9e3c743333..e11c24cc62 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -24,6 +24,7 @@ typedef enum
WalWriterType,
WalReceiverType, /* end of Auxiliary Process Forks */
AutoVacuumLauncherType,
+ AutoVacuumWorkerType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.21.1
0005-Add-PgstatCollectorType-to-subprocess-struct.patch (text/x-patch, 11.6K) download
From 8b43297a8e497ef6d14aef3c3cc8d7bd81483180 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 11 Mar 2020 00:24:42 +0000
Subject: [PATCH 05/11] Add PgstatCollectorType to subprocess struct
Introduce two new fields to handle subprocess control:
1) needs_shmem controls whether or not the subprocess needs to attach
to shmem. Up until now, all subprocesses needed to attach. This field
paves the way for easier handling of subprocesses which do not.
2) fork_prep allows StartSubprocess to do some preparation for
fork/exec. This is necessary to split out any functionality that
previously happened in Start functions prior to fork/exec.
---
src/backend/postmaster/pgstat.c | 86 +++--------------------------
src/backend/postmaster/postmaster.c | 39 +++++++------
src/backend/postmaster/subprocess.c | 28 ++++++++++
src/include/pgstat.h | 4 +-
src/include/postmaster/subprocess.h | 8 ++-
5 files changed, 65 insertions(+), 100 deletions(-)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index f9287b7942..6d4f2ace56 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -48,9 +48,9 @@
#include "pg_trace.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "replication/walsender.h"
#include "storage/backendid.h"
#include "storage/dsm.h"
@@ -275,11 +275,6 @@ static instr_time total_func_time;
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgstat_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgstat_beshutdown_hook(int code, Datum arg);
static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
@@ -685,46 +680,15 @@ pgstat_reset_all(void)
pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
}
-#ifdef EXEC_BACKEND
-
/*
- * pgstat_forkexec() -
+ * PgstatCollectorPrep
*
- * Format up the arglist for, then fork and exec, statistics collector process
- */
-static pid_t
-pgstat_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkcol";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
-/*
- * pgstat_start() -
- *
- * Called from postmaster at startup or after an existing collector
- * died. Attempt to fire up a fresh statistics collector.
- *
- * Returns PID of child process, or 0 if fail.
- *
- * Note: if fail, we will be called again from the postmaster main loop.
+ * Called from StartSubprocess to prepare a new collector fork
*/
int
-pgstat_start(void)
+PgstatCollectorPrep(int argc, char *argv[])
{
time_t curtime;
- pid_t pgStatPid;
/*
* Check that the socket is there, else pgstat_init failed and we can do
@@ -740,46 +704,10 @@ pgstat_start(void)
* from the postmaster main loop, we will get another chance later.
*/
curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgstat_start_time) <
+ if ((unsigned int) (curtime - last_pgstat_start_time) >=
(unsigned int) PGSTAT_RESTART_INTERVAL)
- return 0;
- last_pgstat_start_time = curtime;
+ last_pgstat_start_time = curtime;
- /*
- * Okay, fork off the collector.
- */
-#ifdef EXEC_BACKEND
- switch ((pgStatPid = pgstat_forkexec()))
-#else
- switch ((pgStatPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork statistics collector: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgstatCollectorMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgStatPid;
- }
-
- /* shouldn't get here */
return 0;
}
@@ -4326,7 +4254,7 @@ pgstat_send_bgwriter(void)
* The argc/argv parameters are valid only in EXEC_BACKEND case.
* ----------
*/
-NON_EXEC_STATIC void
+void
PgstatCollectorMain(int argc, char *argv[])
{
int len;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 21bbbf8194..e2ad879452 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1781,7 +1781,7 @@ ServerLoop(void)
/* If we have lost the stats collector, try to start a new one */
if (PgStatPID == 0 &&
(pmState == PM_RUN || pmState == PM_HOT_STANDBY))
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
@@ -3057,7 +3057,7 @@ reaper(SIGNAL_ARGS)
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = pgarch_start();
if (PgStatPID == 0)
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
/* workers may be scheduled to start now */
maybe_start_bgworkers();
@@ -3219,7 +3219,7 @@ reaper(SIGNAL_ARGS)
LogChildExit(LOG, _("statistics collector process"),
pid, exitstatus);
if (pmState == PM_RUN || pmState == PM_HOT_STANDBY)
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
continue;
}
@@ -5069,12 +5069,6 @@ SubPostmasterMain(int argc, char *argv[])
PgArchiverMain(argc, argv); /* does not return */
}
- if (strcmp(argv[1], "--forkcol") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgstatCollectorMain(argc, argv); /* does not return */
- }
if (strcmp(argv[1], "--forklog") == 0)
{
/* Do not want to attach to shared memory */
@@ -5097,14 +5091,17 @@ SubPostmasterMain(int argc, char *argv[])
}
else
{
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
+ if (MySubprocess->needs_shmem)
+ {
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
+ /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ InitProcess();
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores();
+ }
MySubprocess->entrypoint(argc - 2, argv + 2);
}
@@ -5224,7 +5221,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Likewise, start other special children as needed.
*/
Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5475,6 +5472,16 @@ StartSubprocess(SubprocessType type)
MemoryContextDelete(PostmasterContext);
PostmasterContext = NULL;
+ /*
+ * Drop connection to postmaster's shared memory if the subprocess
+ * doesn't need it.
+ */
+ if (!MySubprocess->needs_shmem)
+ {
+ dsm_detach_all();
+ PGSharedMemoryDetach();
+ }
+
if (MySubprocess->needs_aux_proc)
AuxiliaryProcessMain(argc, argv);
else
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 0fbd067310..ee44ef0789 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -12,6 +12,7 @@
*/
#include "postgres.h"
#include "bootstrap/bootstrap.h"
+#include "pgstat.h"
#include "postmaster/autovacuum.h"
#include "postmaster/bgwriter.h"
#include "postmaster/postmaster.h"
@@ -28,6 +29,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "checker",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = CheckerModeMain,
.fork_failure = NULL
},
@@ -35,6 +38,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "bootstrap",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = BootstrapModeMain,
.fork_failure = NULL
},
@@ -42,6 +47,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "startup",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = StartupProcessMain,
.fork_failure = StartupForkFailure
},
@@ -49,6 +56,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "background writer",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = BackgroundWriterMain,
.fork_failure = NULL
},
@@ -56,6 +65,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "checkpointer",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = CheckpointerMain,
.fork_failure = NULL
},
@@ -63,6 +74,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "wal writer",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = WalWriterMain,
.fork_failure = NULL
},
@@ -70,6 +83,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "wal receiver",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = WalReceiverMain,
.fork_failure = NULL
},
@@ -77,6 +92,8 @@ static PgSubprocess process_types[] = {
.name = "avlauncher",
.desc = "autovacuum launcher",
.needs_aux_proc = false,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = AutoVacLauncherMain,
.fork_failure = NULL
},
@@ -84,8 +101,19 @@ static PgSubprocess process_types[] = {
.name = "avworker",
.desc = "autovacuum worker",
.needs_aux_proc = false,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = AutoVacWorkerMain,
.fork_failure = NULL
+ },
+ {
+ .name = "col",
+ .desc = "statistics collector",
+ .needs_aux_proc = false,
+ .needs_shmem = false,
+ .fork_prep = PgstatCollectorPrep,
+ .entrypoint = PgstatCollectorMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 1a19921f80..92943f5cff 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1224,10 +1224,8 @@ extern int pgstat_start(void);
extern void pgstat_reset_all(void);
extern void allow_immediate_pgstat_restart(void);
-#ifdef EXEC_BACKEND
+extern int PgstatCollectorPrep(int argc, char *argv[]);
extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
-
/* ----------
* Functions called from backends
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index e11c24cc62..a7ab036b7b 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -25,12 +25,14 @@ typedef enum
WalReceiverType, /* end of Auxiliary Process Forks */
AutoVacuumLauncherType,
AutoVacuumWorkerType,
+ PgstatCollectorType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
-typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
-typedef bool (*SubprocessForkFailure) (int fork_errno);
+typedef int (*SubprocessPrep) (int argc, char *argv[]);
+typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
+typedef bool (*SubprocessForkFailure) (int fork_errno);
/* Current subprocess initializer */
extern void InitMySubprocess(SubprocessType type);
@@ -40,6 +42,8 @@ typedef struct PgSubprocess
const char *name;
const char *desc;
bool needs_aux_proc;
+ bool needs_shmem;
+ SubprocessPrep fork_prep;
SubprocessEntryPoint entrypoint;
SubprocessForkFailure fork_failure;
} PgSubprocess;
--
2.21.1
0006-Add-PgArchiverType-to-subprocess-struct.patch (text/x-patch, 7.7K) download
From 9d39838534349a970a19764f081d4ae4ae0afc13 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 11 Mar 2020 01:07:05 +0000
Subject: [PATCH 06/11] Add PgArchiverType to subprocess struct
---
src/backend/postmaster/pgarch.c | 89 ++++-------------------------
src/backend/postmaster/postmaster.c | 14 ++---
src/backend/postmaster/subprocess.c | 10 ++++
src/include/postmaster/pgarch.h | 9 +--
src/include/postmaster/subprocess.h | 1 +
5 files changed, 27 insertions(+), 96 deletions(-)
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 01ffd6513c..b2cbe80361 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -38,10 +38,10 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "storage/dsm.h"
#include "storage/fd.h"
#include "storage/ipc.h"
@@ -91,11 +91,6 @@ static volatile sig_atomic_t ready_to_stop = false;
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgarch_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgarch_exit(SIGNAL_ARGS);
static void pgarch_waken(SIGNAL_ARGS);
static void pgarch_waken_stop(SIGNAL_ARGS);
@@ -122,10 +117,9 @@ static void pgarch_archiveDone(char *xlog);
* Note: if fail, we will be called again from the postmaster main loop.
*/
int
-pgarch_start(void)
+PgArchiverPrep(int argc, char *argv[])
{
time_t curtime;
- pid_t pgArchPid;
/*
* Do nothing if no archiver needed
@@ -140,86 +134,20 @@ pgarch_start(void)
* the postmaster main loop, we will get another chance later.
*/
curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgarch_start_time) <
+ if ((unsigned int) (curtime - last_pgarch_start_time) >=
(unsigned int) PGARCH_RESTART_INTERVAL)
- return 0;
- last_pgarch_start_time = curtime;
-
-#ifdef EXEC_BACKEND
- switch ((pgArchPid = pgarch_forkexec()))
-#else
- switch ((pgArchPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork archiver: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
+ last_pgarch_start_time = curtime;
- PgArchiverMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgArchPid;
- }
-
- /* shouldn't get here */
return 0;
}
-/* ------------------------------------------------------------
- * Local functions called by archiver follow
- * ------------------------------------------------------------
- */
-
-
-#ifdef EXEC_BACKEND
-
-/*
- * pgarch_forkexec() -
- *
- * Format up the arglist for, then fork and exec, archive process
- */
-static pid_t
-pgarch_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
-
- av[ac++] = "--forkarch";
-
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
/*
* PgArchiverMain
*
* The argc/argv parameters are valid only in EXEC_BACKEND case. However,
* since we don't use 'em, it hardly matters...
*/
-NON_EXEC_STATIC void
+void
PgArchiverMain(int argc, char *argv[])
{
/*
@@ -238,7 +166,7 @@ PgArchiverMain(int argc, char *argv[])
pqsignal(SIGCHLD, SIG_DFL);
PG_SETMASK(&UnBlockSig);
- MyBackendType = B_ARCHIVER;
+ /* Identify myself via ps */
init_ps_display(NULL);
pgarch_MainLoop();
@@ -246,6 +174,11 @@ PgArchiverMain(int argc, char *argv[])
exit(0);
}
+/* ------------------------------------------------------------
+ * Local functions called by archiver follow
+ * ------------------------------------------------------------
+ */
+
/* SIGQUIT signal handler for archiver process */
static void
pgarch_exit(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index e2ad879452..88d010ceb1 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1785,7 +1785,7 @@ ServerLoop(void)
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
/* If we need to signal the autovacuum launcher, do so now */
if (avlauncher_needs_signal)
@@ -3055,7 +3055,7 @@ reaper(SIGNAL_ARGS)
if (!IsBinaryUpgrade && AutoVacuumingActive() && AutoVacPID == 0)
AutoVacPID = StartSubprocess(AutoVacuumLauncherType);
if (PgArchStartupAllowed() && PgArchPID == 0)
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
if (PgStatPID == 0)
PgStatPID = StartSubprocess(PgstatCollectorType);
@@ -3203,7 +3203,7 @@ reaper(SIGNAL_ARGS)
LogChildExit(LOG, _("archiver process"),
pid, exitstatus);
if (PgArchStartupAllowed())
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
continue;
}
@@ -5063,12 +5063,6 @@ SubPostmasterMain(int argc, char *argv[])
StartBackgroundWorker();
}
- if (strcmp(argv[1], "--forkarch") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgArchiverMain(argc, argv); /* does not return */
- }
if (strcmp(argv[1], "--forklog") == 0)
{
/* Do not want to attach to shared memory */
@@ -5197,7 +5191,7 @@ sigusr1_handler(SIGNAL_ARGS)
*/
Assert(PgArchPID == 0);
if (XLogArchivingAlways())
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
/*
* If we aren't planning to enter hot standby mode later, treat
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index ee44ef0789..f9d71a627c 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -15,6 +15,7 @@
#include "pgstat.h"
#include "postmaster/autovacuum.h"
#include "postmaster/bgwriter.h"
+#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
#include "postmaster/startup.h"
#include "postmaster/subprocess.h"
@@ -114,6 +115,15 @@ static PgSubprocess process_types[] = {
.fork_prep = PgstatCollectorPrep,
.entrypoint = PgstatCollectorMain,
.fork_failure = NULL
+ },
+ {
+ .name = "arch",
+ .desc = "archiver",
+ .needs_aux_proc = false,
+ .needs_shmem = false,
+ .fork_prep = PgArchiverPrep,
+ .entrypoint = PgArchiverMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/postmaster/pgarch.h b/src/include/postmaster/pgarch.h
index b3200874ca..ac527212b1 100644
--- a/src/include/postmaster/pgarch.h
+++ b/src/include/postmaster/pgarch.h
@@ -26,14 +26,7 @@
#define MAX_XFN_CHARS 40
#define VALID_XFN_CHARS "0123456789ABCDEF.history.backup.partial"
-/* ----------
- * Functions called from postmaster
- * ----------
- */
-extern int pgarch_start(void);
-
-#ifdef EXEC_BACKEND
+extern int PgArchiverPrep(int argc, char *argv[]);
extern void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
#endif /* _PGARCH_H */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index a7ab036b7b..3f00f904dc 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -26,6 +26,7 @@ typedef enum
AutoVacuumLauncherType,
AutoVacuumWorkerType,
PgstatCollectorType,
+ PgArchiverType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.21.1
0007-Add-SysLoggerType-to-subprocess-struct.patch (text/x-patch, 16.9K) download
From e2200cbc6ca9a0e6318301c8c6fa0fa21c3ffae3 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 11 Mar 2020 02:58:08 +0000
Subject: [PATCH 07/11] Add SysLoggerType to subprocess struct
This commit introduces two new fields to the subprocess struct:
1) keep_postmaster_memcontext allows subprocesses to specify whether or
not the PostmasterMemoryContext should be kept around after a fork.
2) postmaster_main allows postmaster to gain control to do some
necessary processing after a successful fork.
---
src/backend/postmaster/postmaster.c | 39 +++--
src/backend/postmaster/subprocess.c | 56 +++++--
src/backend/postmaster/syslogger.c | 228 +++++++++++-----------------
src/include/postmaster/subprocess.h | 18 ++-
src/include/postmaster/syslogger.h | 5 +-
5 files changed, 177 insertions(+), 169 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 88d010ceb1..f724727b1e 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1083,7 +1083,7 @@ PostmasterMain(int argc, char *argv[])
/*
* If enabled, start up syslogger collection subprocess
*/
- SysLoggerPID = SysLogger_Start();
+ SysLoggerPID = StartSubprocess(SysLoggerType);
/*
* Reset whereToSendOutput from DestDebug (its starting state) to
@@ -1739,7 +1739,7 @@ ServerLoop(void)
/* If we have lost the log collector, try to start a new one */
if (SysLoggerPID == 0 && Logging_collector)
- SysLoggerPID = SysLogger_Start();
+ SysLoggerPID = StartSubprocess(SysLoggerType);
/*
* If no background writer process is running, and we are not in a
@@ -3228,7 +3228,7 @@ reaper(SIGNAL_ARGS)
{
SysLoggerPID = 0;
/* for safety's sake, launch new logger *first* */
- SysLoggerPID = SysLogger_Start();
+ SysLoggerPID = StartSubprocess(SysLoggerType);
if (!EXIT_STATUS_0(exitstatus))
LogChildExit(LOG, _("system logger process"),
pid, exitstatus);
@@ -5063,12 +5063,6 @@ SubPostmasterMain(int argc, char *argv[])
StartBackgroundWorker();
}
- if (strcmp(argv[1], "--forklog") == 0)
- {
- /* Do not want to attach to shared memory */
-
- SysLoggerMain(argc, argv); /* does not return */
- }
if (MySubprocess->needs_aux_proc)
{
@@ -5446,6 +5440,15 @@ StartSubprocess(SubprocessType type)
snprintf(typebuf, sizeof(typebuf), "-x%d", type);
argv[argc++] = typebuf;
+ /* Prep subprocess for forking */
+ if (MySubprocess->fork_prep)
+ {
+ if (MySubprocess->fork_prep(argc, argv))
+ {
+ return 0;
+ }
+ }
+
argv[argc] = NULL;
Assert(argc < lengthof(argv));
@@ -5461,10 +5464,16 @@ StartSubprocess(SubprocessType type)
/* Close the postmaster's sockets */
ClosePostmasterPorts(false);
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
+ /*
+ * Release postmaster's working memory context if the subprocess doesn't
+ * need it
+ */
+ if (!MySubprocess->keep_postmaster_memcontext)
+ {
+ MemoryContextSwitchTo(TopMemoryContext);
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+ }
/*
* Drop connection to postmaster's shared memory if the subprocess
@@ -5508,6 +5517,10 @@ StartSubprocess(SubprocessType type)
return 0;
}
+ /* Some subprocesses need to do more work in postmaster. */
+ if (MySubprocess->postmaster_main)
+ MySubprocess->postmaster_main(argc, argv);
+
/*
* in parent, successful fork
*/
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index f9d71a627c..f9d07de4ae 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -19,6 +19,7 @@
#include "postmaster/postmaster.h"
#include "postmaster/startup.h"
#include "postmaster/subprocess.h"
+#include "postmaster/syslogger.h"
#include "postmaster/walwriter.h"
#include "replication/walreceiver.h"
@@ -31,99 +32,132 @@ static PgSubprocess process_types[] = {
.desc = "checker",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = CheckerModeMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "boot",
.desc = "bootstrap",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = BootstrapModeMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "boot",
.desc = "startup",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = StartupProcessMain,
- .fork_failure = StartupForkFailure
+ .fork_failure = StartupForkFailure,
+ .postmaster_main = NULL
},
{
.name = "boot",
.desc = "background writer",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = BackgroundWriterMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "boot",
.desc = "checkpointer",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = CheckpointerMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "boot",
.desc = "wal writer",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = WalWriterMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "boot",
.desc = "wal receiver",
.needs_aux_proc = true,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = WalReceiverMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "avlauncher",
.desc = "autovacuum launcher",
.needs_aux_proc = false,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = AutoVacLauncherMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "avworker",
.desc = "autovacuum worker",
.needs_aux_proc = false,
.needs_shmem = true,
+ .keep_postmaster_memcontext = false,
.fork_prep = NULL,
.entrypoint = AutoVacWorkerMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "col",
.desc = "statistics collector",
.needs_aux_proc = false,
.needs_shmem = false,
+ .keep_postmaster_memcontext = false,
.fork_prep = PgstatCollectorPrep,
.entrypoint = PgstatCollectorMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
},
{
.name = "arch",
.desc = "archiver",
.needs_aux_proc = false,
.needs_shmem = false,
+ .keep_postmaster_memcontext = false,
.fork_prep = PgArchiverPrep,
.entrypoint = PgArchiverMain,
- .fork_failure = NULL
+ .fork_failure = NULL,
+ .postmaster_main = NULL
+ },
+ {
+ .name = "log",
+ .desc = "syslogger",
+ .needs_aux_proc = false,
+ .needs_shmem = false,
+ .keep_postmaster_memcontext = true,
+ .fork_prep = SysLoggerPrep,
+ .entrypoint = SysLoggerMain,
+ .fork_failure = NULL,
+ .postmaster_main = SysLoggerPostmasterMain
}
};
diff --git a/src/backend/postmaster/syslogger.c b/src/backend/postmaster/syslogger.c
index ffcb54968f..0fa0fadcbf 100644
--- a/src/backend/postmaster/syslogger.c
+++ b/src/backend/postmaster/syslogger.c
@@ -38,8 +38,8 @@
#include "nodes/pg_list.h"
#include "pgstat.h"
#include "pgtime.h"
-#include "postmaster/fork_process.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "postmaster/syslogger.h"
#include "storage/dsm.h"
#include "storage/fd.h"
@@ -129,10 +129,8 @@ static volatile sig_atomic_t rotation_requested = false;
/* Local subroutines */
#ifdef EXEC_BACKEND
-static pid_t syslogger_forkexec(void);
static void syslogger_parseArgs(int argc, char *argv[]);
#endif
-NON_EXEC_STATIC void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
static void process_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static void flush_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static FILE *logfile_open(const char *filename, const char *mode,
@@ -153,7 +151,7 @@ static void update_metainfo_datafile(void);
* Main entry point for syslogger process
* argc/argv parameters are valid only in EXEC_BACKEND case.
*/
-NON_EXEC_STATIC void
+void
SysLoggerMain(int argc, char *argv[])
{
#ifndef WIN32
@@ -172,7 +170,6 @@ SysLoggerMain(int argc, char *argv[])
syslogger_parseArgs(argc, argv);
#endif /* EXEC_BACKEND */
- MyBackendType = B_LOGGER;
init_ps_display(NULL);
/*
@@ -537,13 +534,17 @@ SysLoggerMain(int argc, char *argv[])
* Postmaster subroutine to start a syslogger subprocess.
*/
int
-SysLogger_Start(void)
+SysLoggerPrep(int argc, char *argv[])
{
- pid_t sysloggerPid;
char *filename;
+#ifdef EXEC_BACKEND
+ char filenobuf[32];
+ char csvfilenobuf[32];
+#endif
+ /* Tell postmaster to stop forking */
if (!Logging_collector)
- return 0;
+ return -1;
/*
* If first time through, create the pipe which will receive stderr
@@ -626,129 +627,6 @@ SysLogger_Start(void)
}
#ifdef EXEC_BACKEND
- switch ((sysloggerPid = syslogger_forkexec()))
-#else
- switch ((sysloggerPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork system logger: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(true);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- /* do the work */
- SysLoggerMain(0, NULL);
- break;
-#endif
-
- default:
- /* success, in postmaster */
-
- /* now we redirect stderr, if not done already */
- if (!redirection_done)
- {
-#ifdef WIN32
- int fd;
-#endif
-
- /*
- * Leave a breadcrumb trail when redirecting, in case the user
- * forgets that redirection is active and looks only at the
- * original stderr target file.
- */
- ereport(LOG,
- (errmsg("redirecting log output to logging collector process"),
- errhint("Future log output will appear in directory \"%s\".",
- Log_directory)));
-
-#ifndef WIN32
- fflush(stdout);
- if (dup2(syslogPipe[1], fileno(stdout)) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stdout: %m")));
- fflush(stderr);
- if (dup2(syslogPipe[1], fileno(stderr)) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- /* Now we are done with the write end of the pipe. */
- close(syslogPipe[1]);
- syslogPipe[1] = -1;
-#else
-
- /*
- * open the pipe in binary mode and make sure stderr is binary
- * after it's been dup'ed into, to avoid disturbing the pipe
- * chunking protocol.
- */
- fflush(stderr);
- fd = _open_osfhandle((intptr_t) syslogPipe[1],
- _O_APPEND | _O_BINARY);
- if (dup2(fd, _fileno(stderr)) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- close(fd);
- _setmode(_fileno(stderr), _O_BINARY);
-
- /*
- * Now we are done with the write end of the pipe.
- * CloseHandle() must not be called because the preceding
- * close() closes the underlying handle.
- */
- syslogPipe[1] = 0;
-#endif
- redirection_done = true;
- }
-
- /* postmaster will never write the file(s); close 'em */
- fclose(syslogFile);
- syslogFile = NULL;
- if (csvlogFile != NULL)
- {
- fclose(csvlogFile);
- csvlogFile = NULL;
- }
- return (int) sysloggerPid;
- }
-
- /* we should never reach here */
- return 0;
-}
-
-
-#ifdef EXEC_BACKEND
-
-/*
- * syslogger_forkexec() -
- *
- * Format up the arglist for, then fork and exec, a syslogger process
- */
-static pid_t
-syslogger_forkexec(void)
-{
- char *av[10];
- int ac = 0;
- char filenobuf[32];
- char csvfilenobuf[32];
-
- av[ac++] = "postgres";
- av[ac++] = "--forklog";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
/* static variables (those not passed by write_backend_variables) */
#ifndef WIN32
if (syslogFile != NULL)
@@ -763,7 +641,7 @@ syslogger_forkexec(void)
else
strcpy(filenobuf, "0");
#endif /* WIN32 */
- av[ac++] = filenobuf;
+ argv[argc++] = filenobuf;
#ifndef WIN32
if (csvlogFile != NULL)
@@ -778,14 +656,92 @@ syslogger_forkexec(void)
else
strcpy(csvfilenobuf, "0");
#endif /* WIN32 */
- av[ac++] = csvfilenobuf;
+ argv[argc++] = csvfilenobuf;
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+#endif /* EXEC_BACKEND */
- return postmaster_forkexec(ac, av);
+ return 0;
}
+/*
+ * SysLoggerPostmasterMain
+ *
+ * Control function for subprocess parent (postmaster) after a successful fork.
+ */
+void
+SysLoggerPostmasterMain(int argc, char *argv[])
+{
+ /* success, in postmaster */
+
+ /* now we redirect stderr, if not done already */
+ if (!redirection_done)
+ {
+#ifdef WIN32
+ int fd;
+#endif
+
+ /*
+ * Leave a breadcrumb trail when redirecting, in case the user
+ * forgets that redirection is active and looks only at the
+ * original stderr target file.
+ */
+ ereport(LOG,
+ (errmsg("redirecting log output to logging collector process"),
+ errhint("Future log output will appear in directory \"%s\".",
+ Log_directory)));
+
+#ifndef WIN32
+ fflush(stdout);
+ if (dup2(syslogPipe[1], fileno(stdout)) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stdout: %m")));
+ fflush(stderr);
+ if (dup2(syslogPipe[1], fileno(stderr)) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ /* Now we are done with the write end of the pipe. */
+ close(syslogPipe[1]);
+ syslogPipe[1] = -1;
+#else
+
+ /*
+ * open the pipe in binary mode and make sure stderr is binary
+ * after it's been dup'ed into, to avoid disturbing the pipe
+ * chunking protocol.
+ */
+ fflush(stderr);
+ fd = _open_osfhandle((intptr_t) syslogPipe[1],
+ _O_APPEND | _O_BINARY);
+ if (dup2(fd, _fileno(stderr)) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ close(fd);
+ _setmode(_fileno(stderr), _O_BINARY);
+
+ /*
+ * Now we are done with the write end of the pipe.
+ * CloseHandle() must not be called because the preceding
+ * close() closes the underlying handle.
+ */
+ syslogPipe[1] = 0;
+#endif
+ redirection_done = true;
+ }
+
+ /* postmaster will never write the file(s); close 'em */
+ fclose(syslogFile);
+ syslogFile = NULL;
+ if (csvlogFile != NULL)
+ {
+ fclose(csvlogFile);
+ csvlogFile = NULL;
+ }
+}
+
+#ifdef EXEC_BACKEND
/*
* syslogger_parseArgs() -
*
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 3f00f904dc..d0d98a5d69 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -27,6 +27,7 @@ typedef enum
AutoVacuumWorkerType,
PgstatCollectorType,
PgArchiverType,
+ SysLoggerType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
@@ -34,19 +35,22 @@ typedef enum
typedef int (*SubprocessPrep) (int argc, char *argv[]);
typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
typedef bool (*SubprocessForkFailure) (int fork_errno);
+typedef void (*SubprocessPostmasterMain) (int argc, char *argv[]);
/* Current subprocess initializer */
extern void InitMySubprocess(SubprocessType type);
typedef struct PgSubprocess
{
- const char *name;
- const char *desc;
- bool needs_aux_proc;
- bool needs_shmem;
- SubprocessPrep fork_prep;
- SubprocessEntryPoint entrypoint;
- SubprocessForkFailure fork_failure;
+ const char *name;
+ const char *desc;
+ bool needs_aux_proc;
+ bool needs_shmem;
+ bool keep_postmaster_memcontext;
+ SubprocessPrep fork_prep;
+ SubprocessEntryPoint entrypoint;
+ SubprocessForkFailure fork_failure;
+ SubprocessPostmasterMain postmaster_main;
} PgSubprocess;
extern SubprocessType MySubprocessType;
diff --git a/src/include/postmaster/syslogger.h b/src/include/postmaster/syslogger.h
index f611bd1411..a498c73121 100644
--- a/src/include/postmaster/syslogger.h
+++ b/src/include/postmaster/syslogger.h
@@ -81,9 +81,10 @@ extern int SysLogger_Start(void);
extern void write_syslogger_file(const char *buffer, int count, int dest);
-#ifdef EXEC_BACKEND
+/* Subprocess startup functions */
+extern int SysLoggerPrep(int argc, char *argv[]);
extern void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+extern void SysLoggerPostmasterMain(int argc, char *argv[]);
extern bool CheckLogrotateSignal(void);
extern void RemoveLogrotateSignalFiles(void);
--
2.21.1
0008-Add-BgWorkerType-to-subprocess-struct.patch (text/x-patch, 25.6K) download
From 747e2640455ba2088637888944f43546ab052a01 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 11 Mar 2020 03:32:41 +0000
Subject: [PATCH 08/11] Add BgWorkerType to subprocess struct
Export some of the backend-specific code and move the some bgworker
startup code out to src/backend/postmaster/bgworker.c.
---
src/backend/postmaster/bgworker.c | 165 ++++++++++-
src/backend/postmaster/postmaster.c | 294 +++-----------------
src/backend/postmaster/subprocess.c | 12 +
src/backend/utils/init/globals.c | 1 +
src/include/miscadmin.h | 1 +
src/include/postmaster/bgworker_internals.h | 8 +-
src/include/postmaster/postmaster.h | 53 ++++
src/include/postmaster/subprocess.h | 1 +
8 files changed, 268 insertions(+), 267 deletions(-)
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index 6c684b5e12..6f62cd4889 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -33,8 +33,10 @@
#include "storage/shmem.h"
#include "tcop/tcopprot.h"
#include "utils/ascii.h"
+#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/timeout.h"
+#include "utils/timestamp.h"
/*
* The postmaster's list of registered background workers, in private memory.
@@ -133,7 +135,55 @@ static const struct
/* Private functions. */
static bgworker_main_type LookupBackgroundWorkerFunction(const char *libraryname, const char *funcname);
+static bool assign_backendlist_entry(RegisteredBgWorker *rw);
+/*
+ * Allocate the Backend struct for a connected background worker, but don't
+ * add it to the list of backends just yet.
+ *
+ * On failure, return false without changing any worker state.
+ *
+ * Some info from the Backend is copied into the passed rw.
+ */
+static bool
+assign_backendlist_entry(RegisteredBgWorker *rw)
+{
+ Backend *bn;
+
+ /*
+ * Compute the cancel key that will be assigned to this session. We
+ * probably don't need cancel keys for background workers, but we'd better
+ * have something random in the field to prevent unfriendly people from
+ * sending cancels to them.
+ */
+ if (!RandomCancelKey(&MyCancelKey))
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random cancel key")));
+ return false;
+ }
+
+ bn = malloc(sizeof(Backend));
+ if (bn == NULL)
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+ return false;
+ }
+
+ bn->cancel_key = MyCancelKey;
+ bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
+ bn->bkend_type = BACKEND_TYPE_BGWORKER;
+ bn->dead_end = false;
+ bn->bgworker_notify = false;
+
+ rw->rw_backend = bn;
+ rw->rw_child_slot = bn->child_slot;
+
+ return true;
+}
/*
* Calculate shared memory needed.
@@ -670,6 +720,55 @@ bgworker_sigusr1_handler(SIGNAL_ARGS)
errno = save_errno;
}
+/*
+ * BackgroundWorkerPrep
+ *
+ * Postmaster subroutine to prepare a bgworker subprocess.
+ *
+ * Returns 0 on success or -1 on failure.
+ *
+ * Note: if fail, we will be called again from the postmaster main loop.
+ */
+int
+BackgroundWorkerPrep(int argc, char *argv[])
+{
+ RegisteredBgWorker *rw = MyRegisteredBgWorker;
+#ifdef EXEC_BACKEND
+ char forkav[MAXPGPATH];
+#endif
+
+ Assert(MyRegisteredBgWorker);
+ Assert(rw->rw_pid == 0);
+
+ /*
+ * Allocate and assign the Backend element. Note we must do this before
+ * forking, so that we can handle out of memory properly.
+ *
+ * Treat failure as though the worker had crashed. That way, the
+ * postmaster will wait a bit before attempting to start it again; if it
+ * tried again right away, most likely it'd find itself repeating the
+ * out-of-memory or fork failure condition.
+ */
+ if (!assign_backendlist_entry(rw))
+ {
+ rw->rw_crashed_at = GetCurrentTimestamp();
+ return -1;
+ }
+
+ ereport(DEBUG1,
+ (errmsg("starting background worker process \"%s\"",
+ rw->rw_worker.bgw_name)));
+
+#ifdef EXEC_BACKEND
+ /* Rewrite bgworker fork parameter with shmem_slot */
+ snprintf(forkav, MAXPGPATH, "%s=%d", argv[1], rw->rw_shmem_slot);
+ argv[1] = forkav;
+#endif
+
+ return 0;
+}
+
+
/*
* Start a new background worker
*
@@ -677,12 +776,28 @@ bgworker_sigusr1_handler(SIGNAL_ARGS)
* postmaster.
*/
void
-StartBackgroundWorker(void)
+BackgroundWorkerMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
- BackgroundWorker *worker = MyBgworkerEntry;
+ BackgroundWorker *worker;
bgworker_main_type entrypt;
+#ifndef EXEC_BACKEND
+ /*
+ * Before blowing away PostmasterContext, save this bgworker's
+ * data where it can find it.
+ */
+ MyBgworkerEntry = (BackgroundWorker *)
+ MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
+ memcpy(MyBgworkerEntry, &(MyRegisteredBgWorker->rw_worker), sizeof(BackgroundWorker));
+
+ /* Now we can release postmaster's working memory context */
+ MemoryContextSwitchTo(TopMemoryContext);
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+#endif
+
+ worker = MyBgworkerEntry;
if (worker == NULL)
elog(FATAL, "unable to find bgworker entry");
@@ -816,6 +931,52 @@ StartBackgroundWorker(void)
proc_exit(0);
}
+/*
+ * BackgroundWorkerForkFailure
+ *
+ * Handle bgworker fork failure gracefully.
+ */
+bool
+BackgroundWorkerForkFailure(int fork_errno)
+{
+ RegisteredBgWorker *rw = MyRegisteredBgWorker;
+
+ errno = fork_errno;
+
+ /* in postmaster, fork failed ... */
+ ereport(LOG,
+ (errmsg("could not fork worker process: %m")));
+
+ /* undo what assign_backendlist_entry did */
+ ReleasePostmasterChildSlot(rw->rw_child_slot);
+ rw->rw_child_slot = 0;
+ free(rw->rw_backend);
+ rw->rw_backend = NULL;
+ /* mark entry as crashed, so we'll try again later */
+ rw->rw_crashed_at = GetCurrentTimestamp();
+
+ /* Don't panic. */
+ return false;
+}
+
+/*
+ * BackgroundWorkerPostmasterMain
+ */
+void
+BackgroundWorkerPostmasterMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ RegisteredBgWorker *rw = MyRegisteredBgWorker;
+
+ rw->rw_pid = MyChildProcPid;
+ rw->rw_backend->pid = rw->rw_pid;
+ ReportBackgroundWorkerPID(rw);
+ /* add new worker to lists of backends */
+ dlist_push_head(&BackendList, &(rw->rw_backend->elem));
+#ifdef EXEC_BACKEND
+ ShmemBackendArrayAdd(rw->rw_backend);
+#endif
+}
+
/*
* Register a new static background worker.
*
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index f724727b1e..8343a81923 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -137,60 +137,14 @@
#include "storage/spin.h"
#endif
-
-/*
- * Possible types of a backend. Beyond being the possible bkend_type values in
- * struct bkend, these are OR-able request flag bits for SignalSomeChildren()
- * and CountChildren().
- */
-#define BACKEND_TYPE_NORMAL 0x0001 /* normal backend */
-#define BACKEND_TYPE_AUTOVAC 0x0002 /* autovacuum worker process */
-#define BACKEND_TYPE_WALSND 0x0004 /* walsender process */
-#define BACKEND_TYPE_BGWORKER 0x0008 /* bgworker process */
-#define BACKEND_TYPE_ALL 0x000F /* OR of all the above */
-
-#define BACKEND_TYPE_WORKER (BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER)
-
-/*
- * List of active backends (or child processes anyway; we don't actually
- * know whether a given child has become a backend or is still in the
- * authorization phase). This is used mainly to keep track of how many
- * children we have and send them appropriate signals when necessary.
- *
- * "Special" children such as the startup, bgwriter and autovacuum launcher
- * tasks are not in this list. Autovacuum worker and walsender are in it.
- * Also, "dead_end" children are in it: these are children launched just for
- * the purpose of sending a friendly rejection message to a would-be client.
- * We must track them because they are attached to shared memory, but we know
- * they will never become live backends. dead_end children are not assigned a
- * PMChildSlot.
- *
- * Background workers are in this list, too.
- */
-typedef struct bkend
-{
- pid_t pid; /* process id of backend */
- int32 cancel_key; /* cancel key for cancels for this backend */
- int child_slot; /* PMChildSlot for this backend, if any */
-
- /*
- * Flavor of backend or auxiliary process. Note that BACKEND_TYPE_WALSND
- * backends initially announce themselves as BACKEND_TYPE_NORMAL, so if
- * bkend_type is normal, you should check for a recent transition.
- */
- int bkend_type;
- bool dead_end; /* is it going to send an error and quit? */
- bool bgworker_notify; /* gets bgworker start/stop notifications */
- dlist_node elem; /* list link in BackendList */
-} Backend;
-
-static dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
+dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
#ifdef EXEC_BACKEND
static Backend *ShmemBackendArray;
#endif
BackgroundWorker *MyBgworkerEntry = NULL;
+RegisteredBgWorker *MyRegisteredBgWorker = NULL;
/* Struct containing postmaster subprocess control info */
SubprocessType MySubprocessType;
@@ -412,7 +366,6 @@ static void processCancelRequest(Port *port, void *pkt);
static int initMasks(fd_set *rmask);
static void report_fork_failure_to_client(Port *port, int errnum);
static CAC_state canAcceptConnections(int backend_type);
-static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
@@ -420,7 +373,6 @@ static void TerminateChildren(int signal);
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
-static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
static pid_t StartSubprocess(SubprocessType type);
@@ -539,10 +491,10 @@ static bool save_backend_variables(BackendParameters *param, Port *port,
HANDLE childProcess, pid_t childPid);
#endif
-static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
+#define BGWORKER_LEN 15
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -4878,6 +4830,7 @@ void
SubPostmasterMain(int argc, char *argv[])
{
Port port;
+ int shmem_slot;
/* In EXEC_BACKEND case we will not have inherited these settings */
IsPostmasterEnvironment = true;
@@ -5041,59 +4994,48 @@ SubPostmasterMain(int argc, char *argv[])
/* And run the backend */
BackendRun(&port); /* does not return */
}
- if (strncmp(argv[1], "--forkbgworker=", 15) == 0)
- {
- int shmem_slot;
-
- /* do this as early as possible; in particular, before InitProcess() */
- IsBackgroundWorker = true;
+ if (MySubprocess->needs_aux_proc)
+ {
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
/* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
+ InitAuxiliaryProcess();
/* Attach process to shared data structures */
CreateSharedMemoryAndSemaphores();
- /* Fetch MyBgworkerEntry from shared memory */
- shmem_slot = atoi(argv[1] + 15);
- MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
+ AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
+ }
- StartBackgroundWorker();
+ if (MySubprocessType == BgWorkerType)
+ {
+ /* do this as early as possible; in particular, before InitProcess() */
+ IsBackgroundWorker = true;
}
- if (MySubprocess->needs_aux_proc)
+ if (MySubprocess->needs_shmem)
{
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
/* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitAuxiliaryProcess();
+ InitProcess();
/* Attach process to shared data structures */
CreateSharedMemoryAndSemaphores();
-
- AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
}
- else
- {
- if (MySubprocess->needs_shmem)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
- }
-
- MySubprocess->entrypoint(argc - 2, argv + 2);
+ if (MySubprocessType == BgWorkerType)
+ {
+ /* Fetch MyBgworkerEntry from shared memory */
+ shmem_slot = atoi(argv[1] + 15);
+ MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
}
+ MySubprocess->entrypoint(argc - 2, argv + 2);
+
abort(); /* shouldn't get here */
}
#endif /* EXEC_BACKEND */
@@ -5356,7 +5298,7 @@ StartupPacketTimeoutHandler(void)
/*
* Generate a random cancel key.
*/
-static bool
+bool
RandomCancelKey(int32 *cancel_key)
{
return pg_strong_random(cancel_key, sizeof(int32));
@@ -5454,15 +5396,17 @@ StartSubprocess(SubprocessType type)
#ifdef EXEC_BACKEND
pid = postmaster_forkexec(argc, argv);
+ MyChildProcPid = pid;
#else /* !EXEC_BACKEND */
pid = fork_process();
+ MyChildProcPid = pid;
if (pid == 0) /* child */
{
InitPostmasterChild();
/* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
+ ClosePostmasterPorts(MySubprocessType == SysLoggerType);
/*
* Release postmaster's working memory context if the subprocess doesn't
@@ -5512,6 +5456,8 @@ StartSubprocess(SubprocessType type)
{
if (MySubprocess->fork_failure(save_errno))
ExitPostmaster(1);
+ else
+ return -1;
}
return 0;
@@ -5754,124 +5700,6 @@ BackgroundWorkerUnblockSignals(void)
PG_SETMASK(&UnBlockSig);
}
-#ifdef EXEC_BACKEND
-static pid_t
-bgworker_forkexec(int shmem_slot)
-{
- char *av[10];
- int ac = 0;
- char forkav[MAXPGPATH];
-
- snprintf(forkav, MAXPGPATH, "--forkbgworker=%d", shmem_slot);
-
- av[ac++] = "postgres";
- av[ac++] = forkav;
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif
-
-/*
- * Start a new bgworker.
- * Starting time conditions must have been checked already.
- *
- * Returns true on success, false on failure.
- * In either case, update the RegisteredBgWorker's state appropriately.
- *
- * This code is heavily based on autovacuum.c, q.v.
- */
-static bool
-do_start_bgworker(RegisteredBgWorker *rw)
-{
- pid_t worker_pid;
-
- Assert(rw->rw_pid == 0);
-
- /*
- * Allocate and assign the Backend element. Note we must do this before
- * forking, so that we can handle failures (out of memory or child-process
- * slots) cleanly.
- *
- * Treat failure as though the worker had crashed. That way, the
- * postmaster will wait a bit before attempting to start it again; if we
- * tried again right away, most likely we'd find ourselves hitting the
- * same resource-exhaustion condition.
- */
- if (!assign_backendlist_entry(rw))
- {
- rw->rw_crashed_at = GetCurrentTimestamp();
- return false;
- }
-
- ereport(DEBUG1,
- (errmsg("starting background worker process \"%s\"",
- rw->rw_worker.bgw_name)));
-
-#ifdef EXEC_BACKEND
- switch ((worker_pid = bgworker_forkexec(rw->rw_shmem_slot)))
-#else
- switch ((worker_pid = fork_process()))
-#endif
- {
- case -1:
- /* in postmaster, fork failed ... */
- ereport(LOG,
- (errmsg("could not fork worker process: %m")));
- /* undo what assign_backendlist_entry did */
- ReleasePostmasterChildSlot(rw->rw_child_slot);
- rw->rw_child_slot = 0;
- free(rw->rw_backend);
- rw->rw_backend = NULL;
- /* mark entry as crashed, so we'll try again later */
- rw->rw_crashed_at = GetCurrentTimestamp();
- break;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /*
- * Before blowing away PostmasterContext, save this bgworker's
- * data where it can find it.
- */
- MyBgworkerEntry = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(MyBgworkerEntry, &rw->rw_worker, sizeof(BackgroundWorker));
-
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
-
- StartBackgroundWorker();
-
- exit(1); /* should not get here */
- break;
-#endif
- default:
- /* in postmaster, fork successful ... */
- rw->rw_pid = worker_pid;
- rw->rw_backend->pid = rw->rw_pid;
- ReportBackgroundWorkerPID(rw);
- /* add new worker to lists of backends */
- dlist_push_head(&BackendList, &rw->rw_backend->elem);
-#ifdef EXEC_BACKEND
- ShmemBackendArrayAdd(rw->rw_backend);
-#endif
- return true;
- }
-
- return false;
-}
-
/*
* Does the current postmaster state require starting a worker with the
* specified start_time?
@@ -5912,67 +5740,6 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
return false;
}
-/*
- * Allocate the Backend struct for a connected background worker, but don't
- * add it to the list of backends just yet.
- *
- * On failure, return false without changing any worker state.
- *
- * Some info from the Backend is copied into the passed rw.
- */
-static bool
-assign_backendlist_entry(RegisteredBgWorker *rw)
-{
- Backend *bn;
-
- /*
- * Check that database state allows another connection. Currently the
- * only possible failure is CAC_TOOMANY, so we just log an error message
- * based on that rather than checking the error code precisely.
- */
- if (canAcceptConnections(BACKEND_TYPE_BGWORKER) != CAC_OK)
- {
- ereport(LOG,
- (errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("no slot available for new worker process")));
- return false;
- }
-
- /*
- * Compute the cancel key that will be assigned to this session. We
- * probably don't need cancel keys for background workers, but we'd better
- * have something random in the field to prevent unfriendly people from
- * sending cancels to them.
- */
- if (!RandomCancelKey(&MyCancelKey))
- {
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("could not generate random cancel key")));
- return false;
- }
-
- bn = malloc(sizeof(Backend));
- if (bn == NULL)
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- return false;
- }
-
- bn->cancel_key = MyCancelKey;
- bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
- bn->bkend_type = BACKEND_TYPE_BGWORKER;
- bn->dead_end = false;
- bn->bgworker_notify = false;
-
- rw->rw_backend = bn;
- rw->rw_child_slot = bn->child_slot;
-
- return true;
-}
-
/*
* If the time is right, start background worker(s).
*
@@ -6077,7 +5844,8 @@ maybe_start_bgworkers(void)
* crashed, but there's no need because the next run of this
* function will do that.
*/
- if (!do_start_bgworker(rw))
+ MyRegisteredBgWorker = rw;
+ if (StartSubprocess(BgWorkerType) <= 0)
{
StartWorkerNeeded = true;
return;
@@ -6488,7 +6256,7 @@ ShmemBackendArrayAllocation(void)
memset(ShmemBackendArray, 0, size);
}
-static void
+void
ShmemBackendArrayAdd(Backend *bn)
{
/* The array slot corresponding to my PMChildSlot should be free */
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index f9d07de4ae..d4df791aeb 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -14,6 +14,7 @@
#include "bootstrap/bootstrap.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
+#include "postmaster/bgworker_internals.h"
#include "postmaster/bgwriter.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
@@ -158,6 +159,17 @@ static PgSubprocess process_types[] = {
.entrypoint = SysLoggerMain,
.fork_failure = NULL,
.postmaster_main = SysLoggerPostmasterMain
+ },
+ {
+ .name = "bgworker",
+ .desc = "background worker",
+ .needs_aux_proc = false,
+ .needs_shmem = true,
+ .keep_postmaster_memcontext = true,
+ .fork_prep = BackgroundWorkerPrep,
+ .entrypoint = BackgroundWorkerMain,
+ .fork_failure = BackgroundWorkerForkFailure,
+ .postmaster_main = BackgroundWorkerPostmasterMain
}
};
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index eb19644419..b31ce82f75 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -38,6 +38,7 @@ volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
int MyProcPid;
+int MyChildProcPid;
pg_time_t MyStartTime;
TimestampTz MyStartTimestamp;
struct Port *MyProcPort;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 14fa127ab1..541ca5145d 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -163,6 +163,7 @@ extern PGDLLIMPORT int max_worker_processes;
extern PGDLLIMPORT int max_parallel_workers;
extern PGDLLIMPORT int MyProcPid;
+extern PGDLLIMPORT int MyChildProcPid;
extern PGDLLIMPORT pg_time_t MyStartTime;
extern PGDLLIMPORT TimestampTz MyStartTimestamp;
extern PGDLLIMPORT struct Port *MyProcPort;
diff --git a/src/include/postmaster/bgworker_internals.h b/src/include/postmaster/bgworker_internals.h
index f7e24664d5..874d5a203f 100644
--- a/src/include/postmaster/bgworker_internals.h
+++ b/src/include/postmaster/bgworker_internals.h
@@ -43,6 +43,7 @@ typedef struct RegisteredBgWorker
} RegisteredBgWorker;
extern slist_head BackgroundWorkerList;
+extern RegisteredBgWorker *MyRegisteredBgWorker;
extern Size BackgroundWorkerShmemSize(void);
extern void BackgroundWorkerShmemInit(void);
@@ -53,8 +54,11 @@ extern void ReportBackgroundWorkerExit(slist_mutable_iter *cur);
extern void BackgroundWorkerStopNotifications(pid_t pid);
extern void ResetBackgroundWorkerCrashTimes(void);
-/* Function to start a background worker, called from postmaster.c */
-extern void StartBackgroundWorker(void) pg_attribute_noreturn();
+/* BgWorkerType subprocess */
+extern int BackgroundWorkerPrep(int argc, char *argv[]);
+extern void BackgroundWorkerMain(int argc, char *argv[]);
+extern bool BackgroundWorkerForkFailure(int fork_errno);
+extern void BackgroundWorkerPostmasterMain(int argc, char *argv[]);
#ifdef EXEC_BACKEND
extern BackgroundWorker *BackgroundWorkerEntry(int slotno);
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index babc87dfc9..b1411c1ed2 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -13,6 +13,41 @@
#ifndef _POSTMASTER_H
#define _POSTMASTER_H
+#include "lib/ilist.h"
+
+/*
+ * List of active backends (or child processes anyway; we don't actually
+ * know whether a given child has become a backend or is still in the
+ * authorization phase). This is used mainly to keep track of how many
+ * children we have and send them appropriate signals when necessary.
+ *
+ * "Special" children such as the startup, bgwriter and autovacuum launcher
+ * tasks are not in this list. Autovacuum worker and walsender are in it.
+ * Also, "dead_end" children are in it: these are children launched just for
+ * the purpose of sending a friendly rejection message to a would-be client.
+ * We must track them because they are attached to shared memory, but we know
+ * they will never become live backends. dead_end children are not assigned a
+ * PMChildSlot.
+ *
+ * Background workers are in this list, too.
+ */
+typedef struct bkend
+{
+ pid_t pid; /* process id of backend */
+ int32 cancel_key; /* cancel key for cancels for this backend */
+ int child_slot; /* PMChildSlot for this backend, if any */
+
+ /*
+ * Flavor of backend or auxiliary process. Note that BACKEND_TYPE_WALSND
+ * backends initially announce themselves as BACKEND_TYPE_NORMAL, so if
+ * bkend_type is normal, you should check for a recent transition.
+ */
+ int bkend_type;
+ bool dead_end; /* is it going to send an error and quit? */
+ bool bgworker_notify; /* gets bgworker start/stop notifications */
+ dlist_node elem; /* list link in BackendList */
+} Backend;
+
/* GUC options */
extern bool EnableSSL;
extern int ReservedBackends;
@@ -54,14 +89,19 @@ extern int MaxLivePostmasterChildren(void);
extern bool PostmasterMarkPIDForWorkerNotify(int);
+extern bool RandomCancelKey(int32 *cancel_key);
+
#ifdef EXEC_BACKEND
extern pid_t postmaster_forkexec(int argc, char *argv[]);
extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
extern Size ShmemBackendArraySize(void);
extern void ShmemBackendArrayAllocation(void);
+extern void ShmemBackendArrayAdd(Backend *bn);
#endif
+extern PGDLLIMPORT dlist_head BackendList;
+
/*
* Note: MAX_BACKENDS is limited to 2^18-1 because that's the width reserved
* for buffer references in buf_internals.h. This limitation could be lifted
@@ -74,4 +114,17 @@ extern void ShmemBackendArrayAllocation(void);
*/
#define MAX_BACKENDS 0x3FFFF
+/*
+ * Possible types of a backend. Beyond being the possible bkend_type values in
+ * struct bkend, these are OR-able request flag bits for SignalSomeChildren()
+ * and CountChildren().
+ */
+#define BACKEND_TYPE_NORMAL 0x0001 /* normal backend */
+#define BACKEND_TYPE_AUTOVAC 0x0002 /* autovacuum worker process */
+#define BACKEND_TYPE_WALSND 0x0004 /* walsender process */
+#define BACKEND_TYPE_BGWORKER 0x0008 /* bgworker process */
+#define BACKEND_TYPE_ALL 0x000F /* OR of all the above */
+
+#define BACKEND_TYPE_WORKER (BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER)
+
#endif /* _POSTMASTER_H */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index d0d98a5d69..706d60e4d6 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -28,6 +28,7 @@ typedef enum
PgstatCollectorType,
PgArchiverType,
SysLoggerType,
+ BgWorkerType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.21.1
0009-Add-Backends-to-subprocess-struct.patch (text/x-patch, 16.0K) download
From d22ca88048beacea829ce3e6afabb1558fc80e52 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Mon, 16 Mar 2020 23:05:33 +0000
Subject: [PATCH 09/11] Add Backends to subprocess struct
---
src/backend/postmaster/postmaster.c | 406 +++++++++++++++-------------
src/backend/postmaster/subprocess.c | 11 +
src/include/postmaster/postmaster.h | 6 +
src/include/postmaster/subprocess.h | 1 +
4 files changed, 234 insertions(+), 190 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 8343a81923..490f7c8213 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -359,7 +359,6 @@ static void BackendInitialize(Port *port);
static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
-static int BackendStartup(Port *port);
static int ProcessStartupPacket(Port *port, bool secure_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
@@ -409,7 +408,6 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(Port *port);
static pid_t internal_forkexec(int argc, char *argv[], Port *port);
/* Type for a socket that can be inherited to a client process */
@@ -512,6 +510,8 @@ int postmaster_alive_fds[2] = {-1, -1};
HANDLE PostmasterHandle;
#endif
+static Port *ConnProcPort = NULL;
+
/*
* Postmaster main entry point
*/
@@ -1671,19 +1671,18 @@ ServerLoop(void)
break;
if (FD_ISSET(ListenSocket[i], &rmask))
{
- Port *port;
+ ConnProcPort = ConnCreate(ListenSocket[i]);
- port = ConnCreate(ListenSocket[i]);
- if (port)
+ if (ConnProcPort)
{
- BackendStartup(port);
+ StartSubprocess(ClientBackendType);
/*
* We no longer need the open socket or port structure
* in this process
*/
- StreamClose(port->sock);
- ConnFree(port);
+ StreamClose(ConnProcPort->sock);
+ ConnFree(ConnProcPort);
}
}
}
@@ -3215,6 +3214,173 @@ reaper(SIGNAL_ARGS)
errno = save_errno;
}
+/*
+ * BackendPrep
+ *
+ * Do all of the pre-fork setup for a new Backend.
+*/
+int
+BackendPrep(int argc, char *argv[])
+{
+ /*
+ * Create backend data structure. Better before the fork() so we can
+ * handle failure cleanly.
+ */
+ Backend *bn;
+
+ bn = (Backend *) malloc(sizeof(Backend));
+ if (!bn)
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+ }
+
+ /*
+ * Compute the cancel key that will be assigned to this backend. The
+ * backend will have its own copy in the forked-off process' value of
+ * MyCancelKey, so that it can transmit the key to the frontend.
+ */
+ if (!RandomCancelKey(&MyCancelKey))
+ {
+ free(bn);
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random cancel key")));
+ }
+
+ bn->cancel_key = MyCancelKey;
+
+ /* Pass down canAcceptConnections state */
+ ConnProcPort->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
+ bn->dead_end = (ConnProcPort->canAcceptConnections != CAC_OK &&
+ ConnProcPort->canAcceptConnections != CAC_WAITBACKUP);
+
+ /*
+ * Unless it's a dead_end child, assign it a child slot number
+ */
+ if (!bn->dead_end)
+ bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
+ else
+ bn->child_slot = 0;
+
+ /* Hasn't asked to be notified about any bgworkers yet */
+ bn->bgworker_notify = false;
+
+ /*
+ * Push the Backend to the stack as-is so it can be retreived by the parent
+ * on the other side of the fork/exec. They will modify this entry in place.
+ */
+ dlist_push_head(&BackendList, &bn->elem);
+
+ return 0;
+}
+
+/*
+ * BackendMain
+ *
+ * Child code when forking a Backend.
+ */
+void BackendMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ /*
+ * Perform additional initialization and collect startup packet.
+ *
+ * We want to do this before InitProcess() for a couple of reasons: 1.
+ * so that we aren't eating up a PGPROC slot while waiting on the
+ * client. 2. so that if InitProcess() fails due to being out of
+ * PGPROC slots, we have already initialized libpq and are able to
+ * report the error to the client.
+ */
+#ifndef EXEC_BACKEND
+ BackendInitialize(ConnProcPort);
+#endif
+
+ /* And run the backend */
+ BackendRun(ConnProcPort); /* does not return */
+}
+
+/*
+ * BackendPostmasterMain
+ *
+ * Parent code when forking a Backend.
+ */
+void BackendPostmasterMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ dlist_mutable_iter iter;
+
+ /*
+ * Find our backend node by popping the element with a matching child slot
+ * off the list. This was pushed to the stack (incomplete) by BackendPrep.
+ * We need to fix it up and push back.
+ */
+ dlist_foreach_modify(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ /* MyPMChildSlot is guaranteed to match the child MyPMChildSlot */
+ if (bp->child_slot == MyPMChildSlot)
+ {
+ /*
+ * Everything's been successful, it's safe to add this backend to our list
+ * of backends.
+ */
+ bp->pid = MyChildProcPid;
+ bp->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
+
+#ifdef EXEC_BACKEND
+ if (!bp->dead_end)
+ ShmemBackendArrayAdd(bp);
+#endif
+ break; /* There is only one entry and we've found it */
+ }
+ }
+
+ /* in parent, successful fork */
+ ereport(DEBUG2,
+ (errmsg_internal("forked new backend, pid=%d socket=%d",
+ (int) MyChildProcPid, (int) MyProcPort->sock)));
+}
+
+/*
+ * BackendForkFailure
+ *
+ * Backend cleanup in case a failure occurs forking a new Backend.
+ */
+bool
+BackendForkFailure(int child_errno)
+{
+ dlist_mutable_iter iter;
+
+ dlist_foreach_modify(iter, &BackendList)
+ {
+ Backend *bp = dlist_container(Backend, elem, iter.cur);
+
+ if (bp->child_slot == MyPMChildSlot)
+ {
+ if (!bp->dead_end)
+ {
+ if (!ReleasePostmasterChildSlot(bp->child_slot))
+ {
+ /* Could not release child slot! Panic! */
+ return true;
+ }
+#ifdef EXEC_BACKEND
+ ShmemBackendArrayRemove(bp);
+#endif
+ }
+ dlist_delete(iter.cur);
+ free(bp);
+ break;
+ }
+ }
+
+ report_fork_failure_to_client(MyProcPort, child_errno);
+
+ /* Don't panic */
+ return false;
+}
+
/*
* Scan the bgworkers list and see if the given PID (which has just stopped
* or crashed) is in it. Handle its shutdown if so, and return true. If not a
@@ -4075,122 +4241,6 @@ TerminateChildren(int signal)
signal_child(PgStatPID, signal);
}
-/*
- * BackendStartup -- start backend process
- *
- * returns: STATUS_ERROR if the fork failed, STATUS_OK otherwise.
- *
- * Note: if you change this code, also consider StartAutovacuumWorker.
- */
-static int
-BackendStartup(Port *port)
-{
- Backend *bn; /* for backend cleanup */
- pid_t pid;
-
- /*
- * Create backend data structure. Better before the fork() so we can
- * handle failure cleanly.
- */
- bn = (Backend *) malloc(sizeof(Backend));
- if (!bn)
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- return STATUS_ERROR;
- }
-
- /*
- * Compute the cancel key that will be assigned to this backend. The
- * backend will have its own copy in the forked-off process' value of
- * MyCancelKey, so that it can transmit the key to the frontend.
- */
- if (!RandomCancelKey(&MyCancelKey))
- {
- free(bn);
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("could not generate random cancel key")));
- return STATUS_ERROR;
- }
-
- bn->cancel_key = MyCancelKey;
-
- /* Pass down canAcceptConnections state */
- port->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
- bn->dead_end = (port->canAcceptConnections != CAC_OK &&
- port->canAcceptConnections != CAC_WAITBACKUP);
-
- /*
- * Unless it's a dead_end child, assign it a child slot number
- */
- if (!bn->dead_end)
- bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
- else
- bn->child_slot = 0;
-
- /* Hasn't asked to be notified about any bgworkers yet */
- bn->bgworker_notify = false;
-
-#ifdef EXEC_BACKEND
- pid = backend_forkexec(port);
-#else /* !EXEC_BACKEND */
- pid = fork_process();
- if (pid == 0) /* child */
- {
- free(bn);
-
- /* Detangle from postmaster */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Perform additional initialization and collect startup packet */
- BackendInitialize(port);
-
- /* And run the backend */
- BackendRun(port);
- }
-#endif /* EXEC_BACKEND */
-
- if (pid < 0)
- {
- /* in parent, fork failed */
- int save_errno = errno;
-
- if (!bn->dead_end)
- (void) ReleasePostmasterChildSlot(bn->child_slot);
- free(bn);
- errno = save_errno;
- ereport(LOG,
- (errmsg("could not fork new process for connection: %m")));
- report_fork_failure_to_client(port, save_errno);
- return STATUS_ERROR;
- }
-
- /* in parent, successful fork */
- ereport(DEBUG2,
- (errmsg_internal("forked new backend, pid=%d socket=%d",
- (int) pid, (int) port->sock)));
-
- /*
- * Everything's been successful, it's safe to add this backend to our list
- * of backends.
- */
- bn->pid = pid;
- bn->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
- dlist_push_head(&BackendList, &bn->elem);
-
-#ifdef EXEC_BACKEND
- if (!bn->dead_end)
- ShmemBackendArrayAdd(bn);
-#endif
-
- return STATUS_OK;
-}
-
/*
* Try to report backend fork() failure to client before we close the
* connection. Since we do not care to risk blocking the postmaster on
@@ -4483,33 +4533,12 @@ postmaster_forkexec(int argc, char *argv[])
Port port;
/* This entry point passes dummy values for the Port variables */
- memset(&port, 0, sizeof(port));
- return internal_forkexec(argc, argv, &port);
-}
-
-/*
- * backend_forkexec -- fork/exec off a backend process
- *
- * Some operating systems (WIN32) don't have fork() so we have to simulate
- * it by storing parameters that need to be passed to the child and
- * then create a new child process.
- *
- * returns the pid of the fork/exec'd process, or -1 on failure
- */
-static pid_t
-backend_forkexec(Port *port)
-{
- char *av[4];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkbackend";
- av[ac++] = NULL; /* filled in by internal_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ if (!ConnProcPort)
+ memset(&port, 0, sizeof(port));
+ else
+ port = *ConnProcPort;
- return internal_forkexec(ac, av, port);
+ return internal_forkexec(argc, argv, &port);
}
#ifndef WIN32
@@ -4849,6 +4878,7 @@ SubPostmasterMain(int argc, char *argv[])
/* Read in the variables file */
memset(&port, 0, sizeof(Port));
read_backend_variables(argv[2], &port);
+ ConnProcPort = &port;
/* Close the postmaster's sockets (as soon as we know them) */
ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
@@ -4890,11 +4920,7 @@ SubPostmasterMain(int argc, char *argv[])
* sometimes impossible to attach to shared memory at the desired address.
* Return the setting to its old value (usually '1' or '2') when finished.
*/
- if (strcmp(argv[1], "--forkbackend") == 0 ||
- strcmp(argv[1], "--forkavlauncher") == 0 ||
- strcmp(argv[1], "--forkavworker") == 0 ||
- strcmp(argv[1], "--forkboot") == 0 ||
- strncmp(argv[1], "--forkbgworker=", 15) == 0)
+ if (MySubprocess->needs_shmem)
PGSharedMemoryReAttach();
else
PGSharedMemoryNoReAttach();
@@ -4943,11 +4969,23 @@ SubPostmasterMain(int argc, char *argv[])
*/
process_shared_preload_libraries();
- /* Run backend or appropriate child */
- if (strcmp(argv[1], "--forkbackend") == 0)
+ if (MySubprocess->needs_aux_proc)
{
- Assert(argc == 3); /* shouldn't be any more args */
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
+
+ /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ InitAuxiliaryProcess();
+
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores();
+
+ AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
+ }
+ /* Run backend or appropriate child */
+ if (MySubprocessType == ClientBackendType)
+ {
/*
* Need to reinitialize the SSL library in the backend, since the
* context structures contain function pointers and cannot be passed
@@ -4981,32 +5019,6 @@ SubPostmasterMain(int argc, char *argv[])
* report the error to the client.
*/
BackendInitialize(&port);
-
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- /* And run the backend */
- BackendRun(&port); /* does not return */
- }
-
- if (MySubprocess->needs_aux_proc)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitAuxiliaryProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
}
if (MySubprocessType == BgWorkerType)
@@ -5027,15 +5039,29 @@ SubPostmasterMain(int argc, char *argv[])
CreateSharedMemoryAndSemaphores();
}
- if (MySubprocessType == BgWorkerType)
+ switch (MySubprocessType)
{
- /* Fetch MyBgworkerEntry from shared memory */
- shmem_slot = atoi(argv[1] + 15);
- MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
+ case AutoVacuumLauncherType:
+ case AutoVacuumWorkerType:
+ MySubprocess->entrypoint(argc - 2, argv + 2);
+ break;
+ case BgWorkerType:
+ /* Fetch MyBgworkerEntry from shared memory */
+ shmem_slot = atoi(argv[1] + 15);
+ MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
+ /* fallthrough */
+ case PgArchiverType:
+ case PgstatCollectorType:
+ case SysLoggerType:
+ case ClientBackendType:
+ MySubprocess->entrypoint(argc, argv);
+ break;
+ default:
+ ereport(LOG,
+ (errmsg("could not start unknown process type (%d) under postmaster",
+ MySubprocessType)));
}
- MySubprocess->entrypoint(argc - 2, argv + 2);
-
abort(); /* shouldn't get here */
}
#endif /* EXEC_BACKEND */
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index d4df791aeb..9d7ce86918 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -170,6 +170,17 @@ static PgSubprocess process_types[] = {
.entrypoint = BackgroundWorkerMain,
.fork_failure = BackgroundWorkerForkFailure,
.postmaster_main = BackgroundWorkerPostmasterMain
+ },
+ {
+ .name = "backend",
+ .desc = "backend",
+ .needs_aux_proc = false,
+ .needs_shmem = true,
+ .keep_postmaster_memcontext = true,
+ .fork_prep = BackendPrep,
+ .entrypoint = BackendMain,
+ .fork_failure = BackendForkFailure,
+ .postmaster_main = BackendPostmasterMain
}
};
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index b1411c1ed2..6bc0af7c2b 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -91,6 +91,12 @@ extern bool PostmasterMarkPIDForWorkerNotify(int);
extern bool RandomCancelKey(int32 *cancel_key);
+extern int BackendPrep(int argc, char *argv[]);
+extern void BackendMain(int argc, char *argv[]);
+extern void BackendPostmasterMain(int argc, char *argv[]);
+extern bool BackendForkFailure(int child_errno);
+extern void BackgroundWorkerMain(int argc, char *argv[]);
+
#ifdef EXEC_BACKEND
extern pid_t postmaster_forkexec(int argc, char *argv[]);
extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 706d60e4d6..06e67efecc 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -29,6 +29,7 @@ typedef enum
PgArchiverType,
SysLoggerType,
BgWorkerType,
+ ClientBackendType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.21.1
0010-Add-WalSenderType-to-subprocess-struct.patch (text/x-patch, 1.4K) download
From e14ee3e60267aa2caaa1cc828779d364d3bd821c Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 17 Mar 2020 18:39:12 +0000
Subject: [PATCH 10/11] Add WalSenderType to subprocess struct
---
src/backend/postmaster/subprocess.c | 11 +++++++++++
src/include/postmaster/subprocess.h | 1 +
2 files changed, 12 insertions(+)
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 9d7ce86918..8cc88792ea 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -181,6 +181,17 @@ static PgSubprocess process_types[] = {
.entrypoint = BackendMain,
.fork_failure = BackendForkFailure,
.postmaster_main = BackendPostmasterMain
+ },
+ { /* Wal Sender placeholder */
+ .name = "walsender",
+ .desc = "wal sender",
+ .needs_aux_proc = false,
+ .needs_shmem = false,
+ .keep_postmaster_memcontext = false,
+ .fork_prep = NULL,
+ .entrypoint = NULL,
+ .fork_failure = NULL,
+ .postmaster_main = NULL
}
};
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 06e67efecc..867e1cb499 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -31,6 +31,7 @@ typedef enum
BgWorkerType,
ClientBackendType,
+ WalSenderType, /* placeholder for wal sender so it can be identified in pgstat */
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.21.1
0011-Move-to-new-MyBackendType.patch (text/x-patch, 19.7K) download
From f9e419379157f11fda0d77abc6e3ef288f9d7962 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 17 Mar 2020 23:37:05 +0000
Subject: [PATCH 11/11] Move to new MyBackendType
- Replace MySubprocessType with MyBackendType
- Reshuffle the order of BackendType enums and move to subprocess.h
- Move GetBackendTypeDesc to subprocess.c and use process_types struct
- Change InitMySubprocess to more appropriate SetMySubprocess
- Handle resetting of MyBackendType on StartSubprocess failure
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/postmaster/postmaster.c | 102 ++++++++++++++--------------
src/backend/postmaster/subprocess.c | 12 +++-
src/backend/utils/error/elog.c | 1 +
src/backend/utils/init/miscinit.c | 54 ---------------
src/include/miscadmin.h | 22 ------
src/include/pgstat.h | 1 +
src/include/postmaster/subprocess.h | 45 ++++++------
8 files changed, 87 insertions(+), 152 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 38d45d9163..7c6673c0fc 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -316,7 +316,7 @@ AuxiliaryProcessMain(int argc, char *argv[])
/*
* We've read the arguments and know what backend type we are.
*/
- InitMySubprocess((SubprocessType)MyAuxProcType);
+ SetMySubprocess((BackendType)MyAuxProcType);
/*
* Identify myself via ps
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 490f7c8213..12e6e97bbf 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -147,7 +147,7 @@ BackgroundWorker *MyBgworkerEntry = NULL;
RegisteredBgWorker *MyRegisteredBgWorker = NULL;
/* Struct containing postmaster subprocess control info */
-SubprocessType MySubprocessType;
+BackendType MyBackendType;
/* The socket number we are listening for connections on */
int PostPortNumber;
@@ -374,7 +374,7 @@ static void TerminateChildren(int signal);
static int CountChildren(int target);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
-static pid_t StartSubprocess(SubprocessType type);
+static pid_t StartSubprocess(BackendType type);
static void StartAutovacuumWorker(void);
static void MaybeStartWalReceiver(void);
static void InitPostmasterDeathWatchHandle(void);
@@ -460,7 +460,7 @@ typedef struct
TimestampTz PgReloadTime;
pg_time_t first_syslogger_file_time;
- SubprocessType MySubprocessType;
+ BackendType MyBackendType;
bool redirection_done;
bool IsBinaryUpgrade;
@@ -1035,7 +1035,7 @@ PostmasterMain(int argc, char *argv[])
/*
* If enabled, start up syslogger collection subprocess
*/
- SysLoggerPID = StartSubprocess(SysLoggerType);
+ SysLoggerPID = StartSubprocess(B_LOGGER);
/*
* Reset whereToSendOutput from DestDebug (its starting state) to
@@ -1341,7 +1341,7 @@ PostmasterMain(int argc, char *argv[])
/*
* We're ready to rock and roll...
*/
- StartupPID = StartSubprocess(StartupType);
+ StartupPID = StartSubprocess(B_STARTUP);
Assert(StartupPID != 0);
StartupStatus = STARTUP_RUNNING;
pmState = PM_STARTUP;
@@ -1675,7 +1675,7 @@ ServerLoop(void)
if (ConnProcPort)
{
- StartSubprocess(ClientBackendType);
+ StartSubprocess(B_BACKEND);
/*
* We no longer need the open socket or port structure
@@ -1690,7 +1690,7 @@ ServerLoop(void)
/* If we have lost the log collector, try to start a new one */
if (SysLoggerPID == 0 && Logging_collector)
- SysLoggerPID = StartSubprocess(SysLoggerType);
+ SysLoggerPID = StartSubprocess(B_LOGGER);
/*
* If no background writer process is running, and we are not in a
@@ -1701,9 +1701,9 @@ ServerLoop(void)
pmState == PM_HOT_STANDBY)
{
if (CheckpointerPID == 0)
- CheckpointerPID = StartSubprocess(CheckpointerType);
+ CheckpointerPID = StartSubprocess(B_CHECKPOINTER);
if (BgWriterPID == 0)
- BgWriterPID = StartSubprocess(BgWriterType);
+ BgWriterPID = StartSubprocess(B_BG_WRITER);
}
/*
@@ -1712,7 +1712,7 @@ ServerLoop(void)
* be writing any new WAL).
*/
if (WalWriterPID == 0 && pmState == PM_RUN)
- WalWriterPID = StartSubprocess(WalWriterType);
+ WalWriterPID = StartSubprocess(B_WAL_WRITER);
/*
* If we have lost the autovacuum launcher, try to start a new one. We
@@ -1724,7 +1724,7 @@ ServerLoop(void)
(AutoVacuumingActive() || start_autovac_launcher) &&
pmState == PM_RUN)
{
- AutoVacPID = StartSubprocess(AutoVacuumLauncherType);
+ AutoVacPID = StartSubprocess(B_AUTOVAC_LAUNCHER);
if (AutoVacPID != 0)
start_autovac_launcher = false; /* signal processed */
}
@@ -1732,11 +1732,11 @@ ServerLoop(void)
/* If we have lost the stats collector, try to start a new one */
if (PgStatPID == 0 &&
(pmState == PM_RUN || pmState == PM_HOT_STANDBY))
- PgStatPID = StartSubprocess(PgstatCollectorType);
+ PgStatPID = StartSubprocess(B_STATS_COLLECTOR);
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
- PgArchPID = StartSubprocess(PgArchiverType);
+ PgArchPID = StartSubprocess(B_ARCHIVER);
/* If we need to signal the autovacuum launcher, do so now */
if (avlauncher_needs_signal)
@@ -2993,22 +2993,22 @@ reaper(SIGNAL_ARGS)
* if this fails, we'll just try again later.
*/
if (CheckpointerPID == 0)
- CheckpointerPID = StartSubprocess(CheckpointerType);
+ CheckpointerPID = StartSubprocess(B_CHECKPOINTER);
if (BgWriterPID == 0)
- BgWriterPID = StartSubprocess(BgWriterType);
+ BgWriterPID = StartSubprocess(B_BG_WRITER);
if (WalWriterPID == 0)
- WalWriterPID = StartSubprocess(WalWriterType);
+ WalWriterPID = StartSubprocess(B_WAL_WRITER);
/*
* Likewise, start other special children as needed. In a restart
* situation, some of them may be alive already.
*/
if (!IsBinaryUpgrade && AutoVacuumingActive() && AutoVacPID == 0)
- AutoVacPID = StartSubprocess(AutoVacuumLauncherType);
+ AutoVacPID = StartSubprocess(B_AUTOVAC_LAUNCHER);
if (PgArchStartupAllowed() && PgArchPID == 0)
- PgArchPID = StartSubprocess(PgArchiverType);
+ PgArchPID = StartSubprocess(B_ARCHIVER);
if (PgStatPID == 0)
- PgStatPID = StartSubprocess(PgstatCollectorType);
+ PgStatPID = StartSubprocess(B_STATS_COLLECTOR);
/* workers may be scheduled to start now */
maybe_start_bgworkers();
@@ -3154,7 +3154,7 @@ reaper(SIGNAL_ARGS)
LogChildExit(LOG, _("archiver process"),
pid, exitstatus);
if (PgArchStartupAllowed())
- PgArchPID = StartSubprocess(PgArchiverType);
+ PgArchPID = StartSubprocess(B_ARCHIVER);
continue;
}
@@ -3170,7 +3170,7 @@ reaper(SIGNAL_ARGS)
LogChildExit(LOG, _("statistics collector process"),
pid, exitstatus);
if (pmState == PM_RUN || pmState == PM_HOT_STANDBY)
- PgStatPID = StartSubprocess(PgstatCollectorType);
+ PgStatPID = StartSubprocess(B_STATS_COLLECTOR);
continue;
}
@@ -3179,7 +3179,7 @@ reaper(SIGNAL_ARGS)
{
SysLoggerPID = 0;
/* for safety's sake, launch new logger *first* */
- SysLoggerPID = StartSubprocess(SysLoggerType);
+ SysLoggerPID = StartSubprocess(B_LOGGER);
if (!EXIT_STATUS_0(exitstatus))
LogChildExit(LOG, _("system logger process"),
pid, exitstatus);
@@ -3978,7 +3978,7 @@ PostmasterStateMachine(void)
Assert(Shutdown > NoShutdown);
/* Start the checkpointer if not running */
if (CheckpointerPID == 0)
- CheckpointerPID = StartSubprocess(CheckpointerType);
+ CheckpointerPID = StartSubprocess(B_CHECKPOINTER);
/* And tell it to shut down */
if (CheckpointerPID != 0)
{
@@ -4119,7 +4119,7 @@ PostmasterStateMachine(void)
reset_shared();
- StartupPID = StartSubprocess(StartupType);
+ StartupPID = StartSubprocess(B_STARTUP);
Assert(StartupPID != 0);
StartupStatus = STARTUP_RUNNING;
pmState = PM_STARTUP;
@@ -4902,7 +4902,7 @@ SubPostmasterMain(int argc, char *argv[])
/* We should only be here once per fork */
Assert(!MySubprocess);
- InitMySubprocess(MySubprocessType);
+ SetMySubprocess(MyBackendType);
/*
* If appropriate, physically re-attach to shared memory segment. We want
@@ -4926,9 +4926,9 @@ SubPostmasterMain(int argc, char *argv[])
PGSharedMemoryNoReAttach();
/* autovacuum needs this set before calling InitProcess */
- if (strcmp(argv[1], "--forkavlauncher") == 0)
+ if (MyBackendType == B_AUTOVAC_LAUNCHER)
AutovacuumLauncherIAm();
- if (strcmp(argv[1], "--forkavworker") == 0)
+ if (MyBackendType == B_AUTOVAC_WORKER)
AutovacuumWorkerIAm();
/*
@@ -4984,7 +4984,7 @@ SubPostmasterMain(int argc, char *argv[])
}
/* Run backend or appropriate child */
- if (MySubprocessType == ClientBackendType)
+ if (MyBackendType == B_BACKEND)
{
/*
* Need to reinitialize the SSL library in the backend, since the
@@ -5021,7 +5021,7 @@ SubPostmasterMain(int argc, char *argv[])
BackendInitialize(&port);
}
- if (MySubprocessType == BgWorkerType)
+ if (MyBackendType == B_BG_WORKER)
{
/* do this as early as possible; in particular, before InitProcess() */
IsBackgroundWorker = true;
@@ -5039,27 +5039,27 @@ SubPostmasterMain(int argc, char *argv[])
CreateSharedMemoryAndSemaphores();
}
- switch (MySubprocessType)
+ switch (MyBackendType)
{
- case AutoVacuumLauncherType:
- case AutoVacuumWorkerType:
+ case B_AUTOVAC_LAUNCHER:
+ case B_AUTOVAC_WORKER:
MySubprocess->entrypoint(argc - 2, argv + 2);
break;
- case BgWorkerType:
+ case B_BG_WORKER:
/* Fetch MyBgworkerEntry from shared memory */
shmem_slot = atoi(argv[1] + 15);
MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
/* fallthrough */
- case PgArchiverType:
- case PgstatCollectorType:
- case SysLoggerType:
- case ClientBackendType:
+ case B_ARCHIVER:
+ case B_STATS_COLLECTOR:
+ case B_LOGGER:
+ case B_BACKEND:
MySubprocess->entrypoint(argc, argv);
break;
default:
ereport(LOG,
(errmsg("could not start unknown process type (%d) under postmaster",
- MySubprocessType)));
+ MyBackendType)));
}
abort(); /* shouldn't get here */
@@ -5143,9 +5143,9 @@ sigusr1_handler(SIGNAL_ARGS)
* we'll just try again later.
*/
Assert(CheckpointerPID == 0);
- CheckpointerPID = StartSubprocess(CheckpointerType);
+ CheckpointerPID = StartSubprocess(B_CHECKPOINTER);
Assert(BgWriterPID == 0);
- BgWriterPID = StartSubprocess(BgWriterType);
+ BgWriterPID = StartSubprocess(B_BG_WRITER);
/*
* Start the archiver if we're responsible for (re-)archiving received
@@ -5153,7 +5153,7 @@ sigusr1_handler(SIGNAL_ARGS)
*/
Assert(PgArchPID == 0);
if (XLogArchivingAlways())
- PgArchPID = StartSubprocess(PgArchiverType);
+ PgArchPID = StartSubprocess(B_ARCHIVER);
/*
* If we aren't planning to enter hot standby mode later, treat
@@ -5177,7 +5177,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Likewise, start other special children as needed.
*/
Assert(PgStatPID == 0);
- PgStatPID = StartSubprocess(PgstatCollectorType);
+ PgStatPID = StartSubprocess(B_STATS_COLLECTOR);
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5382,8 +5382,9 @@ CountChildren(int target)
* the subprocess cleanup function decides whether or not to panic.
*/
static pid_t
-StartSubprocess(SubprocessType type)
+StartSubprocess(BackendType type)
{
+ BackendType prev_backend_type = MyBackendType;
pid_t pid;
char *argv[10];
int argc = 0;
@@ -5395,7 +5396,7 @@ StartSubprocess(SubprocessType type)
/*
* Get new subprocess data every time we start a new subprocess.
*/
- InitMySubprocess(type);
+ SetMySubprocess(type);
argv[argc++] = "postgres";
@@ -5413,6 +5414,7 @@ StartSubprocess(SubprocessType type)
{
if (MySubprocess->fork_prep(argc, argv))
{
+ SetMySubprocess(prev_backend_type);
return 0;
}
}
@@ -5432,7 +5434,7 @@ StartSubprocess(SubprocessType type)
InitPostmasterChild();
/* Close the postmaster's sockets */
- ClosePostmasterPorts(MySubprocessType == SysLoggerType);
+ ClosePostmasterPorts(MyBackendType == B_LOGGER);
/*
* Release postmaster's working memory context if the subprocess doesn't
@@ -5546,7 +5548,7 @@ StartAutovacuumWorker(void)
bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
bn->bgworker_notify = false;
- bn->pid = StartSubprocess(AutoVacuumWorkerType);
+ bn->pid = StartSubprocess(B_AUTOVAC_WORKER);
if (bn->pid > 0)
{
bn->bkend_type = BACKEND_TYPE_AUTOVAC;
@@ -5607,7 +5609,7 @@ MaybeStartWalReceiver(void)
pmState == PM_HOT_STANDBY || pmState == PM_WAIT_READONLY) &&
Shutdown == NoShutdown)
{
- WalReceiverPID = StartSubprocess(WalReceiverType);
+ WalReceiverPID = StartSubprocess(B_WAL_RECEIVER);
if (WalReceiverPID != 0)
WalReceiverRequested = false;
/* else leave the flag set, so we'll try again later */
@@ -5871,7 +5873,7 @@ maybe_start_bgworkers(void)
* function will do that.
*/
MyRegisteredBgWorker = rw;
- if (StartSubprocess(BgWorkerType) <= 0)
+ if (StartSubprocess(B_BG_WORKER) <= 0)
{
StartWorkerNeeded = true;
return;
@@ -5989,7 +5991,7 @@ save_backend_variables(BackendParameters *param, Port *port,
param->PgReloadTime = PgReloadTime;
param->first_syslogger_file_time = first_syslogger_file_time;
- param->MySubprocessType = MySubprocessType;
+ param->MyBackendType = MyBackendType;
param->redirection_done = redirection_done;
param->IsBinaryUpgrade = IsBinaryUpgrade;
@@ -6226,7 +6228,7 @@ restore_backend_variables(BackendParameters *param, Port *port)
PgReloadTime = param->PgReloadTime;
first_syslogger_file_time = param->first_syslogger_file_time;
- MySubprocessType = param->MySubprocessType;
+ MyBackendType = param->MyBackendType;
redirection_done = param->redirection_done;
IsBinaryUpgrade = param->IsBinaryUpgrade;
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 8cc88792ea..3bd9136f83 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -24,7 +24,7 @@
#include "postmaster/walwriter.h"
#include "replication/walreceiver.h"
-SubprocessType MySubprocessType;
+BackendType MyBackendType;
PgSubprocess *MySubprocess;
static PgSubprocess process_types[] = {
@@ -196,9 +196,15 @@ static PgSubprocess process_types[] = {
};
void
-InitMySubprocess(SubprocessType type)
+SetMySubprocess(BackendType type)
{
- MySubprocessType = type;
+ MyBackendType = type;
MySubprocess = &process_types[type];
MySubprocess->desc = gettext(MySubprocess->desc);
}
+
+const char *
+GetBackendTypeDesc(BackendType type)
+{
+ return gettext((&process_types[type])->desc);
+}
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 62eef7b71f..214fcb0522 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -74,6 +74,7 @@
#include "miscadmin.h"
#include "postmaster/bgworker.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "postmaster/syslogger.h"
#include "storage/ipc.h"
#include "storage/proc.h"
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index a7b7b12249..3d36836413 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -189,60 +189,6 @@ SwitchBackToLocalLatch(void)
SetLatch(MyLatch);
}
-const char *
-GetBackendTypeDesc(BackendType backendType)
-{
- const char *backendDesc = "unknown process type";
-
- switch (backendType)
- {
- case B_INVALID:
- backendDesc = "not initialized";
- break;
- case B_AUTOVAC_LAUNCHER:
- backendDesc = "autovacuum launcher";
- break;
- case B_AUTOVAC_WORKER:
- backendDesc = "autovacuum worker";
- break;
- case B_BACKEND:
- backendDesc = "client backend";
- break;
- case B_BG_WORKER:
- backendDesc = "background worker";
- break;
- case B_BG_WRITER:
- backendDesc = "background writer";
- break;
- case B_CHECKPOINTER:
- backendDesc = "checkpointer";
- break;
- case B_STARTUP:
- backendDesc = "startup";
- break;
- case B_WAL_RECEIVER:
- backendDesc = "walreceiver";
- break;
- case B_WAL_SENDER:
- backendDesc = "walsender";
- break;
- case B_WAL_WRITER:
- backendDesc = "walwriter";
- break;
- case B_ARCHIVER:
- backendDesc = "archiver";
- break;
- case B_STATS_COLLECTOR:
- backendDesc = "stats collector";
- break;
- case B_LOGGER:
- backendDesc = "logger";
- break;
- }
-
- return backendDesc;
-}
-
/* ----------------------------------------------------------------
* database path / name support stuff
* ----------------------------------------------------------------
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 541ca5145d..36a65865ab 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -307,28 +307,6 @@ extern void InitStandaloneProcess(const char *argv0);
extern void SwitchToSharedLatch(void);
extern void SwitchBackToLocalLatch(void);
-typedef enum BackendType
-{
- B_INVALID = 0,
- B_AUTOVAC_LAUNCHER,
- B_AUTOVAC_WORKER,
- B_BACKEND,
- B_BG_WORKER,
- B_BG_WRITER,
- B_CHECKPOINTER,
- B_STARTUP,
- B_WAL_RECEIVER,
- B_WAL_SENDER,
- B_WAL_WRITER,
- B_ARCHIVER,
- B_STATS_COLLECTOR,
- B_LOGGER,
-} BackendType;
-
-extern BackendType MyBackendType;
-
-extern const char *GetBackendTypeDesc(BackendType backendType);
-
extern void SetDatabasePath(const char *path);
extern void checkDataDir(void);
extern void SetDataDir(const char *dir);
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 92943f5cff..48f9e313a6 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -17,6 +17,7 @@
#include "port/atomics.h"
#include "portability/instr_time.h"
#include "postmaster/pgarch.h"
+#include "postmaster/subprocess.h"
#include "storage/proc.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 867e1cb499..7affd48e3a 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -13,27 +13,27 @@
#include "postmaster.h"
-typedef enum
+typedef enum BackendType
{
- NoProcessType = -1,
- CheckerType = 0,
- BootstrapType,
- StartupType,
- BgWriterType,
- CheckpointerType,
- WalWriterType,
- WalReceiverType, /* end of Auxiliary Process Forks */
- AutoVacuumLauncherType,
- AutoVacuumWorkerType,
- PgstatCollectorType,
- PgArchiverType,
- SysLoggerType,
- BgWorkerType,
- ClientBackendType,
-
- WalSenderType, /* placeholder for wal sender so it can be identified in pgstat */
- NUMSUBPROCESSTYPES /* Must be last! */
-} SubprocessType;
+ B_INVALID = -1,
+ B_CHECKER = 0,
+ B_BOOTSTRAP,
+ B_STARTUP,
+ B_BG_WRITER,
+ B_CHECKPOINTER,
+ B_WAL_WRITER,
+ B_WAL_RECEIVER, /* end of Auxiliary Processes */
+ B_AUTOVAC_LAUNCHER,
+ B_AUTOVAC_WORKER,
+ B_STATS_COLLECTOR,
+ B_ARCHIVER,
+ B_LOGGER,
+ B_BG_WORKER,
+ B_BACKEND,
+
+ B_WAL_SENDER, /* placeholder for wal sender so it can be identified in pgstat */
+ NUMBACKENDTYPES /* Must be last! */
+} BackendType;
typedef int (*SubprocessPrep) (int argc, char *argv[]);
typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
@@ -41,7 +41,8 @@ typedef bool (*SubprocessForkFailure) (int fork_errno);
typedef void (*SubprocessPostmasterMain) (int argc, char *argv[]);
/* Current subprocess initializer */
-extern void InitMySubprocess(SubprocessType type);
+extern void SetMySubprocess(BackendType type);
+extern const char *GetBackendTypeDesc(BackendType type);
typedef struct PgSubprocess
{
@@ -56,7 +57,7 @@ typedef struct PgSubprocess
SubprocessPostmasterMain postmaster_main;
} PgSubprocess;
-extern SubprocessType MySubprocessType;
+extern BackendType MyBackendType;
extern PGDLLIMPORT PgSubprocess *MySubprocess;
#endif /* SUBPROCESS_H */
--
2.21.1
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-02 14:53 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-04 04:56 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 18:50 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 21:52 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 01:04 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2020-03-18 13:22 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 14:17 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 15:26 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 16:07 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-19 10:35 ` Re: Auxiliary Processes and MyAuxProc Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2020-03-19 20:57 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2020-03-20 22:17 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
@ 2020-03-26 23:30 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-07-02 14:26 ` Re: Auxiliary Processes and MyAuxProc Daniel Gustafsson <daniel@yesql.se>
2020-07-02 15:11 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
0 siblings, 2 replies; 38+ messages in thread
From: Mike Palmiotto @ 2020-03-26 23:30 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>; Justin Pryzby <pryzby@telsasoft.com>; Alvaro Herrera <alvherre@2ndquadrant.com>; Michael Paquier <michael@paquier.xyz>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Fri, Mar 20, 2020 at 6:17 PM Mike Palmiotto
<mike.palmiotto@crunchydata.com> wrote:
>
> On Thu, Mar 19, 2020 at 4:57 PM Andres Freund <andres@anarazel.de> wrote:
>
> I'm still working on wiring up an AppVeyor instance, as seemingly
> builds don't work on any of the default Azure/Visual Studio images. In
> the meantime, I've fixed some spurious whitespace changes and the
> compile error for non-EXEC_BACKEND. I'm posting a new version to keep
> Travis happy at least while I keep working on that. Sorry for the
> delay.
The attached patchset should be fixed. I rebased on master and tested
with TAP tests enabled (fork/EXEC_BACKEND/and Windows).
The AppVeyor configs that Peter posted in a separate thread were
extremely helpful. Thanks, Peter!
> > When starting a process postmaster would just specify the process type,
> > and if relevant, an argument (struct Port for backends, whatever
> > relevant for bgworkers etc) . Generic code should handle all the work
> > until the process type entry point - and likely we should move more work
> > from the individual process types into generic code.
> >
> > If a process is 'only_one_exists' (to be renamed), the generic code
> > would also (in postmaster) register the pid as
> > subprocess_pids[type] = pid;
> > which would make it easy to only have per-type code in the few locations
> > that need to be aware, instead of many locations in
> > postmaster.c. Perhaps also some shared memory location.
I played around with this a bit last weekend and have a local/untested
patch to move all subprocess_pids into the array. I think
'is_singleton_process' would be a decent name for 'only_one_exists'.
We can also probably add a field to the subprocess array to tell which
signals each gets from postmaster.
Are these pieces required to make this patchset committable? Is there
anything else needed at this point to make it committable?
Thanks again for everyone's feedback and comments.
Regards,
--
Mike Palmiotto
https://crunchydata.com
Attachments: download all (11 files, .tar.gz)
0001-Add-subprocess-infrastructure.patch (text/x-patch, 14.0K) download
From 4ffe8563f95ebaf5e436ee308b1e707c7134850b Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Mon, 9 Mar 2020 22:12:10 +0000
Subject: [PATCH 01/11] Add subprocess infrastructure
This commit introduces a new infrastructure for postmaster subprocesses
initialization, forking, exec'ing and main function control. The intent
is to have all subprocess control information live in the process_types
struct, defined in subprocess.h. As more processes are added to the
struct, the fields will be expanded to differentiate actions.
There are several advantages to such an infrastructure:
1) Centralized definition of subprocess information
2) Standardized function arguments/naming schemes
3) Code minimization and removal of duplicate code
---
src/backend/bootstrap/bootstrap.c | 98 ++++++---------------------
src/backend/postmaster/Makefile | 1 +
src/backend/postmaster/bgwriter.c | 2 +-
src/backend/postmaster/checkpointer.c | 2 +-
src/backend/postmaster/startup.c | 2 +-
src/backend/postmaster/subprocess.c | 62 +++++++++++++++++
src/backend/postmaster/walwriter.c | 2 +-
src/backend/replication/walreceiver.c | 2 +-
src/include/bootstrap/bootstrap.h | 2 +
src/include/postmaster/bgwriter.h | 4 +-
src/include/postmaster/startup.h | 5 +-
src/include/postmaster/subprocess.h | 44 ++++++++++++
src/include/postmaster/walwriter.h | 2 +-
src/include/replication/walreceiver.h | 2 +-
14 files changed, 143 insertions(+), 87 deletions(-)
create mode 100644 src/backend/postmaster/subprocess.c
create mode 100644 src/include/postmaster/subprocess.h
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 5480a024e0..38d45d9163 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -35,6 +35,7 @@
#include "pgstat.h"
#include "postmaster/bgwriter.h"
#include "postmaster/startup.h"
+#include "postmaster/subprocess.h"
#include "postmaster/walwriter.h"
#include "replication/walreceiver.h"
#include "storage/bufmgr.h"
@@ -56,8 +57,6 @@ uint32 bootstrap_data_checksum_version = 0; /* No checksum */
#define ALLOC(t, c) \
((t *) MemoryContextAllocZero(TopMemoryContext, (unsigned)(c) * sizeof(t)))
-static void CheckerModeMain(void);
-static void BootstrapModeMain(void);
static void bootstrap_signals(void);
static void ShutdownAuxiliaryProcess(int code, Datum arg);
static Form_pg_attribute AllocateAttribute(void);
@@ -314,26 +313,14 @@ AuxiliaryProcessMain(int argc, char *argv[])
proc_exit(1);
}
- switch (MyAuxProcType)
- {
- case StartupProcess:
- MyBackendType = B_STARTUP;
- break;
- case BgWriterProcess:
- MyBackendType = B_BG_WRITER;
- break;
- case CheckpointerProcess:
- MyBackendType = B_CHECKPOINTER;
- break;
- case WalWriterProcess:
- MyBackendType = B_WAL_WRITER;
- break;
- case WalReceiverProcess:
- MyBackendType = B_WAL_RECEIVER;
- break;
- default:
- MyBackendType = B_INVALID;
- }
+ /*
+ * We've read the arguments and know what backend type we are.
+ */
+ InitMySubprocess((SubprocessType)MyAuxProcType);
+
+ /*
+ * Identify myself via ps
+ */
if (IsUnderPostmaster)
init_ps_display(NULL);
@@ -418,56 +405,10 @@ AuxiliaryProcessMain(int argc, char *argv[])
*/
SetProcessingMode(NormalProcessing);
- switch (MyAuxProcType)
- {
- case CheckerProcess:
- /* don't set signals, they're useless here */
- CheckerModeMain();
- proc_exit(1); /* should never return */
-
- case BootstrapProcess:
-
- /*
- * There was a brief instant during which mode was Normal; this is
- * okay. We need to be in bootstrap mode during BootStrapXLOG for
- * the sake of multixact initialization.
- */
- SetProcessingMode(BootstrapProcessing);
- bootstrap_signals();
- BootStrapXLOG();
- BootstrapModeMain();
- proc_exit(1); /* should never return */
-
- case StartupProcess:
- /* don't set signals, startup process has its own agenda */
- StartupProcessMain();
- proc_exit(1); /* should never return */
-
- case BgWriterProcess:
- /* don't set signals, bgwriter has its own agenda */
- BackgroundWriterMain();
- proc_exit(1); /* should never return */
-
- case CheckpointerProcess:
- /* don't set signals, checkpointer has its own agenda */
- CheckpointerMain();
- proc_exit(1); /* should never return */
-
- case WalWriterProcess:
- /* don't set signals, walwriter has its own agenda */
- InitXLOGAccess();
- WalWriterMain();
- proc_exit(1); /* should never return */
-
- case WalReceiverProcess:
- /* don't set signals, walreceiver has its own agenda */
- WalReceiverMain();
- proc_exit(1); /* should never return */
-
- default:
- elog(PANIC, "unrecognized process type: %d", (int) MyAuxProcType);
- proc_exit(1);
- }
+ /* Now jump into the subprocess main function and never look back! */
+ MySubprocess->entrypoint(argc, argv);
+
+ proc_exit(1); /* should never return */
}
/*
@@ -476,8 +417,8 @@ AuxiliaryProcessMain(int argc, char *argv[])
* settings). Since, in fact, that was already done by BaseInit(),
* we have nothing more to do here.
*/
-static void
-CheckerModeMain(void)
+void
+CheckerModeMain(int argc, char *argv[])
{
proc_exit(0);
}
@@ -489,11 +430,15 @@ CheckerModeMain(void)
* The bootstrap backend doesn't speak SQL, but instead expects
* commands in a special bootstrap language.
*/
-static void
-BootstrapModeMain(void)
+void
+BootstrapModeMain(int argc, char *argv[])
{
int i;
+ SetProcessingMode(BootstrapProcessing);
+ bootstrap_signals();
+ BootStrapXLOG();
+
Assert(!IsUnderPostmaster);
Assert(IsBootstrapProcessingMode());
@@ -508,7 +453,6 @@ BootstrapModeMain(void)
* Do backend-like initialization for bootstrap mode
*/
InitProcess();
-
InitPostgres(NULL, InvalidOid, NULL, InvalidOid, NULL, false);
/* Initialize stuff for bootstrap-file processing */
diff --git a/src/backend/postmaster/Makefile b/src/backend/postmaster/Makefile
index bfdf6a833d..52b4c2142c 100644
--- a/src/backend/postmaster/Makefile
+++ b/src/backend/postmaster/Makefile
@@ -23,6 +23,7 @@ OBJS = \
pgstat.o \
postmaster.o \
startup.o \
+ subprocess.o \
syslogger.o \
walwriter.o
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 069e27e427..b2170938f3 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -91,7 +91,7 @@ static XLogRecPtr last_snapshot_lsn = InvalidXLogRecPtr;
* basic execution environment, but not enabled signals yet.
*/
void
-BackgroundWriterMain(void)
+BackgroundWriterMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
MemoryContext bgwriter_context;
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index e354a78725..83f7be1b1e 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -180,7 +180,7 @@ static void ReqCheckpointHandler(SIGNAL_ARGS);
* basic execution environment, but not enabled signals yet.
*/
void
-CheckpointerMain(void)
+CheckpointerMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
MemoryContext checkpointer_context;
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index 8952676765..3b6ef2b65c 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -133,7 +133,7 @@ HandleStartupProcInterrupts(void)
* ----------------------------------
*/
void
-StartupProcessMain(void)
+StartupProcessMain(int argc, char *argv[])
{
/*
* Properly accept or ignore signals the postmaster might send us.
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
new file mode 100644
index 0000000000..3e7a45bf10
--- /dev/null
+++ b/src/backend/postmaster/subprocess.c
@@ -0,0 +1,62 @@
+/*-------------------------------------------------------------------------
+ *
+ * subprocess.c
+ *
+ * Copyright (c) 2004-2020, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/postmaster/syslogger.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "bootstrap/bootstrap.h"
+#include "postmaster/bgwriter.h"
+#include "postmaster/postmaster.h"
+#include "postmaster/startup.h"
+#include "postmaster/subprocess.h"
+#include "postmaster/walwriter.h"
+#include "replication/walreceiver.h"
+
+SubprocessType MySubprocessType;
+PgSubprocess *MySubprocess;
+
+static PgSubprocess process_types[] = {
+ {
+ .desc = "checker",
+ .entrypoint = CheckerModeMain
+ },
+ {
+ .desc = "bootstrap",
+ .entrypoint = BootstrapModeMain
+ },
+ {
+ .desc = "startup",
+ .entrypoint = StartupProcessMain
+ },
+ {
+ .desc = "background writer",
+ .entrypoint = BackgroundWriterMain
+ },
+ {
+ .desc = "checkpointer",
+ .entrypoint = CheckpointerMain
+ },
+ {
+ .desc = "wal writer",
+ .entrypoint = WalWriterMain
+ },
+ {
+ .desc = "wal receiver",
+ .entrypoint = WalReceiverMain
+ }
+};
+
+void
+InitMySubprocess(SubprocessType type)
+{
+ MySubprocessType = type;
+ MySubprocess = &process_types[type];
+ MySubprocess->desc = gettext(MySubprocess->desc);
+}
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index 45a2757969..3a0cec3371 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -85,7 +85,7 @@ int WalWriterFlushAfter = 128;
* basic execution environment, but not enabled signals yet.
*/
void
-WalWriterMain(void)
+WalWriterMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
MemoryContext walwriter_context;
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index 25e0333c9e..cd39a8c982 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -165,7 +165,7 @@ ProcessWalRcvInterrupts(void)
/* Main entry point for walreceiver process */
void
-WalReceiverMain(void)
+WalReceiverMain(int argc, char *argv[])
{
char conninfo[MAXCONNINFO];
char *tmp_conninfo;
diff --git a/src/include/bootstrap/bootstrap.h b/src/include/bootstrap/bootstrap.h
index b67fa9acc1..0247ca7f23 100644
--- a/src/include/bootstrap/bootstrap.h
+++ b/src/include/bootstrap/bootstrap.h
@@ -31,6 +31,8 @@ extern Relation boot_reldesc;
extern Form_pg_attribute attrtypes[MAXATTR];
extern int numattr;
+extern void CheckerModeMain(int argc, char *argv[]);
+extern void BootstrapModeMain(int argc, char *argv[]);
extern void AuxiliaryProcessMain(int argc, char *argv[]) pg_attribute_noreturn();
diff --git a/src/include/postmaster/bgwriter.h b/src/include/postmaster/bgwriter.h
index 0a5708b32e..6fb1ee1d6d 100644
--- a/src/include/postmaster/bgwriter.h
+++ b/src/include/postmaster/bgwriter.h
@@ -27,8 +27,8 @@ extern int CheckPointTimeout;
extern int CheckPointWarning;
extern double CheckPointCompletionTarget;
-extern void BackgroundWriterMain(void) pg_attribute_noreturn();
-extern void CheckpointerMain(void) pg_attribute_noreturn();
+extern void BackgroundWriterMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern void CheckpointerMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void RequestCheckpoint(int flags);
extern void CheckpointWriteDelay(int flags, double progress);
diff --git a/src/include/postmaster/startup.h b/src/include/postmaster/startup.h
index bec313764a..2e9d18863a 100644
--- a/src/include/postmaster/startup.h
+++ b/src/include/postmaster/startup.h
@@ -13,10 +13,13 @@
#define _STARTUP_H
extern void HandleStartupProcInterrupts(void);
-extern void StartupProcessMain(void) pg_attribute_noreturn();
extern void PreRestoreCommand(void);
extern void PostRestoreCommand(void);
extern bool IsPromoteSignaled(void);
extern void ResetPromoteSignaled(void);
+/* Startup subprocess functions */
+extern void StartupProcessMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern bool StartupCleanup(int child_errno);
+
#endif /* _STARTUP_H */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
new file mode 100644
index 0000000000..56e8edf2d8
--- /dev/null
+++ b/src/include/postmaster/subprocess.h
@@ -0,0 +1,44 @@
+/*-------------------------------------------------------------------------
+ *
+ * subprocess.h
+ *
+ * Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ *
+ * src/include/postmaster/subprocess.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef SUBPROCESS_H
+#define SUBPROCESS_H
+
+#include "postmaster.h"
+
+typedef enum
+{
+ NoProcessType = -1,
+ CheckerType = 0,
+ BootstrapType,
+ StartupType,
+ BgWriterType,
+ CheckpointerType,
+ WalWriterType,
+ WalReceiverType, /* end of Auxiliary Process Forks */
+
+ NUMSUBPROCESSTYPES /* Must be last! */
+} SubprocessType;
+
+typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
+
+/* Current subprocess initializer */
+extern void InitMySubprocess(SubprocessType type);
+
+typedef struct PgSubprocess
+{
+ const char *desc;
+ SubprocessEntryPoint entrypoint;
+} PgSubprocess;
+
+extern SubprocessType MySubprocessType;
+extern PGDLLIMPORT PgSubprocess *MySubprocess;
+
+#endif /* SUBPROCESS_H */
diff --git a/src/include/postmaster/walwriter.h b/src/include/postmaster/walwriter.h
index 011dc26a6e..596c437c54 100644
--- a/src/include/postmaster/walwriter.h
+++ b/src/include/postmaster/walwriter.h
@@ -16,6 +16,6 @@
extern int WalWriterDelay;
extern int WalWriterFlushAfter;
-extern void WalWriterMain(void) pg_attribute_noreturn();
+extern void WalWriterMain(int argc, char *argv[]) pg_attribute_noreturn();
#endif /* _WALWRITER_H */
diff --git a/src/include/replication/walreceiver.h b/src/include/replication/walreceiver.h
index e08afc6548..2025f19ab3 100644
--- a/src/include/replication/walreceiver.h
+++ b/src/include/replication/walreceiver.h
@@ -311,7 +311,7 @@ walrcv_clear_result(WalRcvExecResult *walres)
}
/* prototypes for functions in walreceiver.c */
-extern void WalReceiverMain(void) pg_attribute_noreturn();
+extern void WalReceiverMain(int argc, char *argv[]);
extern void ProcessWalRcvInterrupts(void);
/* prototypes for functions in walreceiverfuncs.c */
--
2.21.1
0002-Use-centralized-StartSubprocess-for-aux-procs.patch (text/x-patch, 14.9K) download
From e6268eba78dd2390e05ad6f8c129f49084fab81f Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 10 Mar 2020 14:58:07 +0000
Subject: [PATCH 02/11] Use centralized StartSubprocess for aux procs
Modifies StartChildProcess and moves the auxiliary processes under its
control. This is the next step toward making use of the Subprocess
infrastructure.
This patch introduces the fork_failure function, which may be
implemented by each subprocess to control failure when attempting to
fork the subprocess. This is used by the startup subprocess to exit
postmaster on fork failure. Future subprocesses may use it to do cleanup
(and optionally exit postmaster) after a fork failure.
Additionally, the patch adds a new backend variable for establishing a
backend variable to be shared across fork/exec.
---
src/backend/postmaster/postmaster.c | 156 ++++++++++++++--------------
src/backend/postmaster/startup.c | 12 +++
src/backend/postmaster/subprocess.c | 35 +++++--
src/include/postmaster/startup.h | 2 +-
src/include/postmaster/subprocess.h | 4 +
5 files changed, 121 insertions(+), 88 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 73d278f3b2..cda8d69349 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -114,6 +114,7 @@
#include "postmaster/fork_process.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "postmaster/syslogger.h"
#include "replication/logicallauncher.h"
#include "replication/walsender.h"
@@ -191,7 +192,8 @@ static Backend *ShmemBackendArray;
BackgroundWorker *MyBgworkerEntry = NULL;
-
+/* Struct containing postmaster subprocess control info */
+SubprocessType MySubprocessType;
/* The socket number we are listening for connections on */
int PostPortNumber;
@@ -421,7 +423,7 @@ static int CountChildren(int target);
static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
-static pid_t StartChildProcess(AuxProcType type);
+static pid_t StartSubprocess(SubprocessType type);
static void StartAutovacuumWorker(void);
static void MaybeStartWalReceiver(void);
static void InitPostmasterDeathWatchHandle(void);
@@ -507,6 +509,9 @@ typedef struct
TimestampTz PgStartTime;
TimestampTz PgReloadTime;
pg_time_t first_syslogger_file_time;
+
+ SubprocessType MySubprocessType;
+
bool redirection_done;
bool IsBinaryUpgrade;
int max_safe_fds;
@@ -538,11 +543,6 @@ static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
-#define StartupDataBase() StartChildProcess(StartupProcess)
-#define StartBackgroundWriter() StartChildProcess(BgWriterProcess)
-#define StartCheckpointer() StartChildProcess(CheckpointerProcess)
-#define StartWalWriter() StartChildProcess(WalWriterProcess)
-#define StartWalReceiver() StartChildProcess(WalReceiverProcess)
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -1389,7 +1389,7 @@ PostmasterMain(int argc, char *argv[])
/*
* We're ready to rock and roll...
*/
- StartupPID = StartupDataBase();
+ StartupPID = StartSubprocess(StartupType);
Assert(StartupPID != 0);
StartupStatus = STARTUP_RUNNING;
pmState = PM_STARTUP;
@@ -1750,9 +1750,9 @@ ServerLoop(void)
pmState == PM_HOT_STANDBY)
{
if (CheckpointerPID == 0)
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
if (BgWriterPID == 0)
- BgWriterPID = StartBackgroundWriter();
+ BgWriterPID = StartSubprocess(BgWriterType);
}
/*
@@ -1761,7 +1761,7 @@ ServerLoop(void)
* be writing any new WAL).
*/
if (WalWriterPID == 0 && pmState == PM_RUN)
- WalWriterPID = StartWalWriter();
+ WalWriterPID = StartSubprocess(WalWriterType);
/*
* If we have lost the autovacuum launcher, try to start a new one. We
@@ -3042,11 +3042,11 @@ reaper(SIGNAL_ARGS)
* if this fails, we'll just try again later.
*/
if (CheckpointerPID == 0)
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
if (BgWriterPID == 0)
- BgWriterPID = StartBackgroundWriter();
+ BgWriterPID = StartSubprocess(BgWriterType);
if (WalWriterPID == 0)
- WalWriterPID = StartWalWriter();
+ WalWriterPID = StartSubprocess(WalWriterType);
/*
* Likewise, start other special children as needed. In a restart
@@ -3860,7 +3860,7 @@ PostmasterStateMachine(void)
Assert(Shutdown > NoShutdown);
/* Start the checkpointer if not running */
if (CheckpointerPID == 0)
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
/* And tell it to shut down */
if (CheckpointerPID != 0)
{
@@ -4001,7 +4001,7 @@ PostmasterStateMachine(void)
reset_shared();
- StartupPID = StartupDataBase();
+ StartupPID = StartSubprocess(StartupType);
Assert(StartupPID != 0);
StartupStatus = STARTUP_RUNNING;
pmState = PM_STARTUP;
@@ -4917,6 +4917,10 @@ SubPostmasterMain(int argc, char *argv[])
errmsg("out of memory")));
#endif
+ /* We should only be here once per fork */
+ Assert(!MySubprocess);
+ InitMySubprocess(MySubprocessType);
+
/*
* If appropriate, physically re-attach to shared memory segment. We want
* to do this before going any further to ensure that we can attach at the
@@ -5037,19 +5041,6 @@ SubPostmasterMain(int argc, char *argv[])
/* And run the backend */
BackendRun(&port); /* does not return */
}
- if (strcmp(argv[1], "--forkboot") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitAuxiliaryProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
- }
if (strcmp(argv[1], "--forkavlauncher") == 0)
{
/* Restore basic shared memory pointers */
@@ -5117,6 +5108,20 @@ SubPostmasterMain(int argc, char *argv[])
SysLoggerMain(argc, argv); /* does not return */
}
+ if (MySubprocess->needs_aux_proc)
+ {
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
+
+ /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ InitAuxiliaryProcess();
+
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores();
+
+ AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
+ }
+
abort(); /* shouldn't get here */
}
#endif /* EXEC_BACKEND */
@@ -5198,9 +5203,9 @@ sigusr1_handler(SIGNAL_ARGS)
* we'll just try again later.
*/
Assert(CheckpointerPID == 0);
- CheckpointerPID = StartCheckpointer();
+ CheckpointerPID = StartSubprocess(CheckpointerType);
Assert(BgWriterPID == 0);
- BgWriterPID = StartBackgroundWriter();
+ BgWriterPID = StartSubprocess(BgWriterType);
/*
* Start the archiver if we're responsible for (re-)archiving received
@@ -5425,42 +5430,49 @@ CountChildren(int target)
return cnt;
}
-
/*
* StartChildProcess -- start an auxiliary process for the postmaster
*
- * "type" determines what kind of child will be started. All child types
- * initially go to AuxiliaryProcessMain, which will handle common setup.
+ * "type" determines what kind of child will be started. This code relies
+ * heavily on MySubprocess, which is an entry in the process_types struct in
+ * subprocess.c. All auxiliary processes go through AuxiliaryProcessMain after
+ * forking. Everything else goes through the designated entrypoint.
*
- * Return value of StartChildProcess is subprocess' PID, or 0 if failed
- * to start subprocess.
+ * Return value of StartChildProcess is subprocess' PID on success. On failure,
+ * the subprocess cleanup function decides whether or not to panic.
*/
static pid_t
-StartChildProcess(AuxProcType type)
+StartSubprocess(SubprocessType type)
{
pid_t pid;
- char *av[10];
- int ac = 0;
+ char *argv[10];
+ int argc = 0;
char typebuf[32];
+#ifdef EXEC_BACKEND
+ char forkname[32];
+#endif
/*
- * Set up command-line arguments for subprocess
+ * Get new subprocess data every time we start a new subprocess.
*/
- av[ac++] = "postgres";
+ InitMySubprocess(type);
+
+ argv[argc++] = "postgres";
#ifdef EXEC_BACKEND
- av[ac++] = "--forkboot";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
+ snprintf(forkname, sizeof(forkname), "--fork%s", MySubprocess->name);
+ argv[argc++] = forkname;
+ argv[argc++] = NULL;
#endif
snprintf(typebuf, sizeof(typebuf), "-x%d", type);
- av[ac++] = typebuf;
+ argv[argc++] = typebuf;
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ argv[argc] = NULL;
+ Assert(argc < lengthof(argv));
#ifdef EXEC_BACKEND
- pid = postmaster_forkexec(ac, av);
+ pid = postmaster_forkexec(argc, argv);
#else /* !EXEC_BACKEND */
pid = fork_process();
@@ -5476,7 +5488,7 @@ StartChildProcess(AuxProcType type)
MemoryContextDelete(PostmasterContext);
PostmasterContext = NULL;
- AuxiliaryProcessMain(ac, av);
+ AuxiliaryProcessMain(argc, argv);
ExitPostmaster(0);
}
#endif /* EXEC_BACKEND */
@@ -5487,40 +5499,20 @@ StartChildProcess(AuxProcType type)
int save_errno = errno;
errno = save_errno;
- switch (type)
- {
- case StartupProcess:
- ereport(LOG,
- (errmsg("could not fork startup process: %m")));
- break;
- case BgWriterProcess:
- ereport(LOG,
- (errmsg("could not fork background writer process: %m")));
- break;
- case CheckpointerProcess:
- ereport(LOG,
- (errmsg("could not fork checkpointer process: %m")));
- break;
- case WalWriterProcess:
- ereport(LOG,
- (errmsg("could not fork WAL writer process: %m")));
- break;
- case WalReceiverProcess:
- ereport(LOG,
- (errmsg("could not fork WAL receiver process: %m")));
- break;
- default:
- ereport(LOG,
- (errmsg("could not fork process: %m")));
- break;
- }
+ ereport(LOG,
+ (errmsg("could not fork %s process: %m", MySubprocess->desc)));
/*
- * fork failure is fatal during startup, but there's no need to choke
- * immediately if starting other child types fails.
+ * Handle fork-failure per subprocess type. Each subprocess handles
+ * fork-failure by returning a boolean which tells postmaster whether or
+ * not to panic.
*/
- if (type == StartupProcess)
- ExitPostmaster(1);
+ if (MySubprocess->fork_failure)
+ {
+ if (MySubprocess->fork_failure(save_errno))
+ ExitPostmaster(1);
+ }
+
return 0;
}
@@ -5638,7 +5630,7 @@ MaybeStartWalReceiver(void)
pmState == PM_HOT_STANDBY || pmState == PM_WAIT_READONLY) &&
Shutdown == NoShutdown)
{
- WalReceiverPID = StartWalReceiver();
+ WalReceiverPID = StartSubprocess(WalReceiverType);
if (WalReceiverPID != 0)
WalReceiverRequested = false;
/* else leave the flag set, so we'll try again later */
@@ -6198,6 +6190,8 @@ save_backend_variables(BackendParameters *param, Port *port,
param->PgReloadTime = PgReloadTime;
param->first_syslogger_file_time = first_syslogger_file_time;
+ param->MySubprocessType = MySubprocessType;
+
param->redirection_done = redirection_done;
param->IsBinaryUpgrade = IsBinaryUpgrade;
param->max_safe_fds = max_safe_fds;
@@ -6433,6 +6427,8 @@ restore_backend_variables(BackendParameters *param, Port *port)
PgReloadTime = param->PgReloadTime;
first_syslogger_file_time = param->first_syslogger_file_time;
+ MySubprocessType = param->MySubprocessType;
+
redirection_done = param->redirection_done;
IsBinaryUpgrade = param->IsBinaryUpgrade;
max_safe_fds = param->max_safe_fds;
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index 3b6ef2b65c..b382573ed7 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -176,6 +176,18 @@ StartupProcessMain(int argc, char *argv[])
proc_exit(0);
}
+/*
+ * StartupForkFailure
+ *
+ * Tell the postmaster to exit/panic on failure to fork.
+ */
+bool
+StartupForkFailure(int fork_errno)
+{
+ /* Panic! */
+ return true;
+}
+
void
PreRestoreCommand(void)
{
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 3e7a45bf10..7e96678dfb 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -24,32 +24,53 @@ PgSubprocess *MySubprocess;
static PgSubprocess process_types[] = {
{
+ .name = "boot",
.desc = "checker",
- .entrypoint = CheckerModeMain
+ .needs_aux_proc = true,
+ .entrypoint = CheckerModeMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "bootstrap",
- .entrypoint = BootstrapModeMain
+ .needs_aux_proc = true,
+ .entrypoint = BootstrapModeMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "startup",
- .entrypoint = StartupProcessMain
+ .needs_aux_proc = true,
+ .entrypoint = StartupProcessMain,
+ .fork_failure = StartupForkFailure
},
{
+ .name = "boot",
.desc = "background writer",
- .entrypoint = BackgroundWriterMain
+ .needs_aux_proc = true,
+ .entrypoint = BackgroundWriterMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "checkpointer",
- .entrypoint = CheckpointerMain
+ .needs_aux_proc = true,
+ .entrypoint = CheckpointerMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "wal writer",
- .entrypoint = WalWriterMain
+ .needs_aux_proc = true,
+ .entrypoint = WalWriterMain,
+ .fork_failure = NULL
},
{
+ .name = "boot",
.desc = "wal receiver",
- .entrypoint = WalReceiverMain
+ .needs_aux_proc = true,
+ .entrypoint = WalReceiverMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/postmaster/startup.h b/src/include/postmaster/startup.h
index 2e9d18863a..7f064743a5 100644
--- a/src/include/postmaster/startup.h
+++ b/src/include/postmaster/startup.h
@@ -20,6 +20,6 @@ extern void ResetPromoteSignaled(void);
/* Startup subprocess functions */
extern void StartupProcessMain(int argc, char *argv[]) pg_attribute_noreturn();
-extern bool StartupCleanup(int child_errno);
+extern bool StartupForkFailure(int fork_errno);
#endif /* _STARTUP_H */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 56e8edf2d8..33f6a7ab04 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -28,14 +28,18 @@ typedef enum
} SubprocessType;
typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
+typedef bool (*SubprocessForkFailure) (int fork_errno);
/* Current subprocess initializer */
extern void InitMySubprocess(SubprocessType type);
typedef struct PgSubprocess
{
+ const char *name;
const char *desc;
+ bool needs_aux_proc;
SubprocessEntryPoint entrypoint;
+ SubprocessForkFailure fork_failure;
} PgSubprocess;
extern SubprocessType MySubprocessType;
--
2.21.1
0003-Add-AutoVacLauncherType-to-subprocess-struct.patch (text/x-patch, 7.5K) download
From f55f2010fae0a2aae91520299c0776eede56d958 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 10 Mar 2020 22:29:44 +0000
Subject: [PATCH 03/11] Add AutoVacLauncherType to subprocess struct
---
src/backend/postmaster/autovacuum.c | 68 ++---------------------------
src/backend/postmaster/postmaster.c | 36 ++++++++-------
src/backend/postmaster/subprocess.c | 8 ++++
src/include/postmaster/autovacuum.h | 3 +-
src/include/postmaster/subprocess.h | 1 +
5 files changed, 35 insertions(+), 81 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index da75e755f0..7397b944d3 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -95,6 +95,7 @@
#include "storage/procsignal.h"
#include "storage/sinvaladt.h"
#include "storage/smgr.h"
+#include "postmaster/subprocess.h"
#include "tcop/tcopprot.h"
#include "utils/fmgroids.h"
#include "utils/fmgrprotos.h"
@@ -303,11 +304,9 @@ static WorkerInfo MyWorkerInfo = NULL;
int AutovacuumLauncherPid = 0;
#ifdef EXEC_BACKEND
-static pid_t avlauncher_forkexec(void);
static pid_t avworker_forkexec(void);
#endif
NON_EXEC_STATIC void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-NON_EXEC_STATIC void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
static Oid do_start_worker(void);
static void HandleAutoVacLauncherInterrupts(void);
@@ -353,26 +352,6 @@ static void autovac_refresh_stats(void);
********************************************************************/
#ifdef EXEC_BACKEND
-/*
- * forkexec routine for the autovacuum launcher process.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avlauncher_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavlauncher";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
/*
* We need this set from the outside, before InitProcess is called
@@ -384,56 +363,17 @@ AutovacuumLauncherIAm(void)
}
#endif
-/*
- * Main entry point for autovacuum launcher process, to be called from the
- * postmaster.
- */
-int
-StartAutoVacLauncher(void)
-{
- pid_t AutoVacPID;
-
-#ifdef EXEC_BACKEND
- switch ((AutoVacPID = avlauncher_forkexec()))
-#else
- switch ((AutoVacPID = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum launcher process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacLauncherMain(0, NULL);
- break;
-#endif
- default:
- return (int) AutoVacPID;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
/*
* Main loop for the autovacuum launcher process.
*/
-NON_EXEC_STATIC void
-AutoVacLauncherMain(int argc, char *argv[])
+void
+AutoVacLauncherMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
am_autovacuum_launcher = true;
- MyBackendType = B_AUTOVAC_LAUNCHER;
+ /* Identify myself via ps */
init_ps_display(NULL);
ereport(DEBUG1,
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index cda8d69349..5c4b92d3bf 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1773,7 +1773,7 @@ ServerLoop(void)
(AutoVacuumingActive() || start_autovac_launcher) &&
pmState == PM_RUN)
{
- AutoVacPID = StartAutoVacLauncher();
+ AutoVacPID = StartSubprocess(AutoVacuumLauncherType);
if (AutoVacPID != 0)
start_autovac_launcher = false; /* signal processed */
}
@@ -3053,7 +3053,7 @@ reaper(SIGNAL_ARGS)
* situation, some of them may be alive already.
*/
if (!IsBinaryUpgrade && AutoVacuumingActive() && AutoVacPID == 0)
- AutoVacPID = StartAutoVacLauncher();
+ AutoVacPID = StartSubprocess(AutoVacuumLauncherType);
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = pgarch_start();
if (PgStatPID == 0)
@@ -5041,19 +5041,6 @@ SubPostmasterMain(int argc, char *argv[])
/* And run the backend */
BackendRun(&port); /* does not return */
}
- if (strcmp(argv[1], "--forkavlauncher") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AutoVacLauncherMain(argc - 2, argv + 2); /* does not return */
- }
if (strcmp(argv[1], "--forkavworker") == 0)
{
/* Restore basic shared memory pointers */
@@ -5121,6 +5108,19 @@ SubPostmasterMain(int argc, char *argv[])
AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
}
+ else
+ {
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
+
+ /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ InitProcess();
+
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores();
+
+ MySubprocess->entrypoint(argc - 2, argv + 2);
+ }
abort(); /* shouldn't get here */
}
@@ -5488,7 +5488,11 @@ StartSubprocess(SubprocessType type)
MemoryContextDelete(PostmasterContext);
PostmasterContext = NULL;
- AuxiliaryProcessMain(argc, argv);
+ if (MySubprocess->needs_aux_proc)
+ AuxiliaryProcessMain(argc, argv);
+ else
+ MySubprocess->entrypoint(argc, argv);
+
ExitPostmaster(0);
}
#endif /* EXEC_BACKEND */
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 7e96678dfb..9aa60cc93e 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -12,6 +12,7 @@
*/
#include "postgres.h"
#include "bootstrap/bootstrap.h"
+#include "postmaster/autovacuum.h"
#include "postmaster/bgwriter.h"
#include "postmaster/postmaster.h"
#include "postmaster/startup.h"
@@ -71,6 +72,13 @@ static PgSubprocess process_types[] = {
.needs_aux_proc = true,
.entrypoint = WalReceiverMain,
.fork_failure = NULL
+ },
+ {
+ .name = "avlauncher",
+ .desc = "autovacuum launcher",
+ .needs_aux_proc = false,
+ .entrypoint = AutoVacLauncherMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index d40ed55531..08050cd81c 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -58,6 +58,8 @@ extern void autovac_init(void);
extern int StartAutoVacLauncher(void);
extern int StartAutoVacWorker(void);
+extern void AutoVacLauncherMain(int argc, char *argv[]);
+
/* called from postmaster when a worker could not be forked */
extern void AutoVacWorkerFailed(void);
@@ -65,7 +67,6 @@ extern void AutoVacWorkerFailed(void);
extern void AutoVacuumUpdateDelay(void);
#ifdef EXEC_BACKEND
-extern void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void AutovacuumWorkerIAm(void);
extern void AutovacuumLauncherIAm(void);
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 33f6a7ab04..9e3c743333 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -23,6 +23,7 @@ typedef enum
CheckpointerType,
WalWriterType,
WalReceiverType, /* end of Auxiliary Process Forks */
+ AutoVacuumLauncherType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.21.1
0004-Add-AutoVacuumWorkerType-to-subprocess-struct.patch (text/x-patch, 6.4K) download
From 095efd4d4619b4633e821e048f448297437ee8e2 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 10 Mar 2020 23:33:39 +0000
Subject: [PATCH 04/11] Add AutoVacuumWorkerType to subprocess struct
---
src/backend/postmaster/autovacuum.c | 71 +----------------------------
src/backend/postmaster/postmaster.c | 17 +------
src/backend/postmaster/subprocess.c | 7 +++
src/include/postmaster/autovacuum.h | 4 +-
src/include/postmaster/subprocess.h | 1 +
5 files changed, 14 insertions(+), 86 deletions(-)
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 7397b944d3..cf0a74a4bb 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -83,7 +83,6 @@
#include "nodes/makefuncs.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
#include "storage/bufmgr.h"
@@ -303,11 +302,6 @@ static WorkerInfo MyWorkerInfo = NULL;
/* PID of launcher, valid only in worker while shutting down */
int AutovacuumLauncherPid = 0;
-#ifdef EXEC_BACKEND
-static pid_t avworker_forkexec(void);
-#endif
-NON_EXEC_STATIC void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
-
static Oid do_start_worker(void);
static void HandleAutoVacLauncherInterrupts(void);
static void AutoVacLauncherShutdown(void) pg_attribute_noreturn();
@@ -1364,27 +1358,6 @@ avl_sigusr2_handler(SIGNAL_ARGS)
********************************************************************/
#ifdef EXEC_BACKEND
-/*
- * forkexec routines for the autovacuum worker.
- *
- * Format up the arglist, then fork and exec.
- */
-static pid_t
-avworker_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkavworker";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-
/*
* We need this set from the outside, before InitProcess is called
*/
@@ -1395,50 +1368,10 @@ AutovacuumWorkerIAm(void)
}
#endif
-/*
- * Main entry point for autovacuum worker process.
- *
- * This code is heavily based on pgarch.c, q.v.
- */
-int
-StartAutoVacWorker(void)
-{
- pid_t worker_pid;
-
-#ifdef EXEC_BACKEND
- switch ((worker_pid = avworker_forkexec()))
-#else
- switch ((worker_pid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork autovacuum worker process: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- AutoVacWorkerMain(0, NULL);
- break;
-#endif
- default:
- return (int) worker_pid;
- }
-
- /* shouldn't get here */
- return 0;
-}
-
/*
* AutoVacWorkerMain
*/
-NON_EXEC_STATIC void
+void
AutoVacWorkerMain(int argc, char *argv[])
{
sigjmp_buf local_sigjmp_buf;
@@ -1446,7 +1379,7 @@ AutoVacWorkerMain(int argc, char *argv[])
am_autovacuum_worker = true;
- MyBackendType = B_AUTOVAC_WORKER;
+ /* Identify myself via ps */
init_ps_display(NULL);
SetProcessingMode(InitProcessing);
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 5c4b92d3bf..d93b62d285 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -5041,19 +5041,6 @@ SubPostmasterMain(int argc, char *argv[])
/* And run the backend */
BackendRun(&port); /* does not return */
}
- if (strcmp(argv[1], "--forkavworker") == 0)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AutoVacWorkerMain(argc - 2, argv + 2); /* does not return */
- }
if (strncmp(argv[1], "--forkbgworker=", 15) == 0)
{
int shmem_slot;
@@ -5573,7 +5560,7 @@ StartAutovacuumWorker(void)
bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
bn->bgworker_notify = false;
- bn->pid = StartAutoVacWorker();
+ bn->pid = StartSubprocess(AutoVacuumWorkerType);
if (bn->pid > 0)
{
bn->bkend_type = BACKEND_TYPE_AUTOVAC;
@@ -5587,7 +5574,7 @@ StartAutovacuumWorker(void)
/*
* fork failed, fall through to report -- actual error message was
- * logged by StartAutoVacWorker
+ * logged by StartSubprocess
*/
(void) ReleasePostmasterChildSlot(bn->child_slot);
free(bn);
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 9aa60cc93e..0fbd067310 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -79,6 +79,13 @@ static PgSubprocess process_types[] = {
.needs_aux_proc = false,
.entrypoint = AutoVacLauncherMain,
.fork_failure = NULL
+ },
+ {
+ .name = "avworker",
+ .desc = "autovacuum worker",
+ .needs_aux_proc = false,
+ .entrypoint = AutoVacWorkerMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/postmaster/autovacuum.h b/src/include/postmaster/autovacuum.h
index 08050cd81c..51546fd4c9 100644
--- a/src/include/postmaster/autovacuum.h
+++ b/src/include/postmaster/autovacuum.h
@@ -58,7 +58,8 @@ extern void autovac_init(void);
extern int StartAutoVacLauncher(void);
extern int StartAutoVacWorker(void);
-extern void AutoVacLauncherMain(int argc, char *argv[]);
+extern void AutoVacLauncherMain(int argc, char *argv[]) pg_attribute_noreturn();
+extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
/* called from postmaster when a worker could not be forked */
extern void AutoVacWorkerFailed(void);
@@ -67,7 +68,6 @@ extern void AutoVacWorkerFailed(void);
extern void AutoVacuumUpdateDelay(void);
#ifdef EXEC_BACKEND
-extern void AutoVacWorkerMain(int argc, char *argv[]) pg_attribute_noreturn();
extern void AutovacuumWorkerIAm(void);
extern void AutovacuumLauncherIAm(void);
#endif
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 9e3c743333..e11c24cc62 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -24,6 +24,7 @@ typedef enum
WalWriterType,
WalReceiverType, /* end of Auxiliary Process Forks */
AutoVacuumLauncherType,
+ AutoVacuumWorkerType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.21.1
0005-Add-PgstatCollectorType-to-subprocess-struct.patch (text/x-patch, 11.6K) download
From 2f143173a139b6b2cd6edf5aabeb624b5550316f Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 11 Mar 2020 00:24:42 +0000
Subject: [PATCH 05/11] Add PgstatCollectorType to subprocess struct
Introduce two new fields to handle subprocess control:
1) needs_shmem controls whether or not the subprocess needs to attach
to shmem. Up until now, all subprocesses needed to attach. This field
paves the way for easier handling of subprocesses which do not.
2) fork_prep allows StartSubprocess to do some preparation for
fork/exec. This is necessary to split out any functionality that
previously happened in Start functions prior to fork/exec.
---
src/backend/postmaster/pgstat.c | 86 +++--------------------------
src/backend/postmaster/postmaster.c | 39 +++++++------
src/backend/postmaster/subprocess.c | 28 ++++++++++
src/include/pgstat.h | 4 +-
src/include/postmaster/subprocess.h | 8 ++-
5 files changed, 65 insertions(+), 100 deletions(-)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 4763c24be9..8a4a8747f2 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -48,9 +48,9 @@
#include "pg_trace.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "replication/walsender.h"
#include "storage/backendid.h"
#include "storage/dsm.h"
@@ -275,11 +275,6 @@ static instr_time total_func_time;
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgstat_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgstat_beshutdown_hook(int code, Datum arg);
static PgStat_StatDBEntry *pgstat_get_db_entry(Oid databaseid, bool create);
@@ -685,46 +680,15 @@ pgstat_reset_all(void)
pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
}
-#ifdef EXEC_BACKEND
-
/*
- * pgstat_forkexec() -
+ * PgstatCollectorPrep
*
- * Format up the arglist for, then fork and exec, statistics collector process
- */
-static pid_t
-pgstat_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkcol";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
-/*
- * pgstat_start() -
- *
- * Called from postmaster at startup or after an existing collector
- * died. Attempt to fire up a fresh statistics collector.
- *
- * Returns PID of child process, or 0 if fail.
- *
- * Note: if fail, we will be called again from the postmaster main loop.
+ * Called from StartSubprocess to prepare a new collector fork
*/
int
-pgstat_start(void)
+PgstatCollectorPrep(int argc, char *argv[])
{
time_t curtime;
- pid_t pgStatPid;
/*
* Check that the socket is there, else pgstat_init failed and we can do
@@ -740,46 +704,10 @@ pgstat_start(void)
* from the postmaster main loop, we will get another chance later.
*/
curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgstat_start_time) <
+ if ((unsigned int) (curtime - last_pgstat_start_time) >=
(unsigned int) PGSTAT_RESTART_INTERVAL)
- return 0;
- last_pgstat_start_time = curtime;
+ last_pgstat_start_time = curtime;
- /*
- * Okay, fork off the collector.
- */
-#ifdef EXEC_BACKEND
- switch ((pgStatPid = pgstat_forkexec()))
-#else
- switch ((pgStatPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork statistics collector: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- PgstatCollectorMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgStatPid;
- }
-
- /* shouldn't get here */
return 0;
}
@@ -4335,7 +4263,7 @@ pgstat_send_bgwriter(void)
* The argc/argv parameters are valid only in EXEC_BACKEND case.
* ----------
*/
-NON_EXEC_STATIC void
+void
PgstatCollectorMain(int argc, char *argv[])
{
int len;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index d93b62d285..9dea8195b8 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1781,7 +1781,7 @@ ServerLoop(void)
/* If we have lost the stats collector, try to start a new one */
if (PgStatPID == 0 &&
(pmState == PM_RUN || pmState == PM_HOT_STANDBY))
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
@@ -3057,7 +3057,7 @@ reaper(SIGNAL_ARGS)
if (PgArchStartupAllowed() && PgArchPID == 0)
PgArchPID = pgarch_start();
if (PgStatPID == 0)
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
/* workers may be scheduled to start now */
maybe_start_bgworkers();
@@ -3219,7 +3219,7 @@ reaper(SIGNAL_ARGS)
LogChildExit(LOG, _("statistics collector process"),
pid, exitstatus);
if (pmState == PM_RUN || pmState == PM_HOT_STANDBY)
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
continue;
}
@@ -5069,12 +5069,6 @@ SubPostmasterMain(int argc, char *argv[])
PgArchiverMain(argc, argv); /* does not return */
}
- if (strcmp(argv[1], "--forkcol") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgstatCollectorMain(argc, argv); /* does not return */
- }
if (strcmp(argv[1], "--forklog") == 0)
{
/* Do not want to attach to shared memory */
@@ -5097,14 +5091,17 @@ SubPostmasterMain(int argc, char *argv[])
}
else
{
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
+ if (MySubprocess->needs_shmem)
+ {
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
+ /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ InitProcess();
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores();
+ }
MySubprocess->entrypoint(argc - 2, argv + 2);
}
@@ -5224,7 +5221,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Likewise, start other special children as needed.
*/
Assert(PgStatPID == 0);
- PgStatPID = pgstat_start();
+ PgStatPID = StartSubprocess(PgstatCollectorType);
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5475,6 +5472,16 @@ StartSubprocess(SubprocessType type)
MemoryContextDelete(PostmasterContext);
PostmasterContext = NULL;
+ /*
+ * Drop connection to postmaster's shared memory if the subprocess
+ * doesn't need it.
+ */
+ if (!MySubprocess->needs_shmem)
+ {
+ dsm_detach_all();
+ PGSharedMemoryDetach();
+ }
+
if (MySubprocess->needs_aux_proc)
AuxiliaryProcessMain(argc, argv);
else
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 0fbd067310..ee44ef0789 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -12,6 +12,7 @@
*/
#include "postgres.h"
#include "bootstrap/bootstrap.h"
+#include "pgstat.h"
#include "postmaster/autovacuum.h"
#include "postmaster/bgwriter.h"
#include "postmaster/postmaster.h"
@@ -28,6 +29,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "checker",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = CheckerModeMain,
.fork_failure = NULL
},
@@ -35,6 +38,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "bootstrap",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = BootstrapModeMain,
.fork_failure = NULL
},
@@ -42,6 +47,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "startup",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = StartupProcessMain,
.fork_failure = StartupForkFailure
},
@@ -49,6 +56,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "background writer",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = BackgroundWriterMain,
.fork_failure = NULL
},
@@ -56,6 +65,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "checkpointer",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = CheckpointerMain,
.fork_failure = NULL
},
@@ -63,6 +74,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "wal writer",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = WalWriterMain,
.fork_failure = NULL
},
@@ -70,6 +83,8 @@ static PgSubprocess process_types[] = {
.name = "boot",
.desc = "wal receiver",
.needs_aux_proc = true,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = WalReceiverMain,
.fork_failure = NULL
},
@@ -77,6 +92,8 @@ static PgSubprocess process_types[] = {
.name = "avlauncher",
.desc = "autovacuum launcher",
.needs_aux_proc = false,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = AutoVacLauncherMain,
.fork_failure = NULL
},
@@ -84,8 +101,19 @@ static PgSubprocess process_types[] = {
.name = "avworker",
.desc = "autovacuum worker",
.needs_aux_proc = false,
+ .needs_shmem = true,
+ .fork_prep = NULL,
.entrypoint = AutoVacWorkerMain,
.fork_failure = NULL
+ },
+ {
+ .name = "col",
+ .desc = "statistics collector",
+ .needs_aux_proc = false,
+ .needs_shmem = false,
+ .fork_prep = PgstatCollectorPrep,
+ .entrypoint = PgstatCollectorMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index a07012bf4b..92f4e7824c 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1227,10 +1227,8 @@ extern int pgstat_start(void);
extern void pgstat_reset_all(void);
extern void allow_immediate_pgstat_restart(void);
-#ifdef EXEC_BACKEND
+extern int PgstatCollectorPrep(int argc, char *argv[]);
extern void PgstatCollectorMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
-
/* ----------
* Functions called from backends
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index e11c24cc62..a7ab036b7b 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -25,12 +25,14 @@ typedef enum
WalReceiverType, /* end of Auxiliary Process Forks */
AutoVacuumLauncherType,
AutoVacuumWorkerType,
+ PgstatCollectorType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
-typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
-typedef bool (*SubprocessForkFailure) (int fork_errno);
+typedef int (*SubprocessPrep) (int argc, char *argv[]);
+typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
+typedef bool (*SubprocessForkFailure) (int fork_errno);
/* Current subprocess initializer */
extern void InitMySubprocess(SubprocessType type);
@@ -40,6 +42,8 @@ typedef struct PgSubprocess
const char *name;
const char *desc;
bool needs_aux_proc;
+ bool needs_shmem;
+ SubprocessPrep fork_prep;
SubprocessEntryPoint entrypoint;
SubprocessForkFailure fork_failure;
} PgSubprocess;
--
2.21.1
0006-Add-PgArchiverType-to-subprocess-struct.patch (text/x-patch, 7.7K) download
From d2f8dcfb056a24f9d608105d7292705d9201820b Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 11 Mar 2020 01:07:05 +0000
Subject: [PATCH 06/11] Add PgArchiverType to subprocess struct
---
src/backend/postmaster/pgarch.c | 89 ++++-------------------------
src/backend/postmaster/postmaster.c | 14 ++---
src/backend/postmaster/subprocess.c | 10 ++++
src/include/postmaster/pgarch.h | 9 +--
src/include/postmaster/subprocess.h | 1 +
5 files changed, 27 insertions(+), 96 deletions(-)
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 01ffd6513c..b2cbe80361 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -38,10 +38,10 @@
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "pgstat.h"
-#include "postmaster/fork_process.h"
#include "postmaster/interrupt.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "storage/dsm.h"
#include "storage/fd.h"
#include "storage/ipc.h"
@@ -91,11 +91,6 @@ static volatile sig_atomic_t ready_to_stop = false;
* Local function forward declarations
* ----------
*/
-#ifdef EXEC_BACKEND
-static pid_t pgarch_forkexec(void);
-#endif
-
-NON_EXEC_STATIC void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
static void pgarch_exit(SIGNAL_ARGS);
static void pgarch_waken(SIGNAL_ARGS);
static void pgarch_waken_stop(SIGNAL_ARGS);
@@ -122,10 +117,9 @@ static void pgarch_archiveDone(char *xlog);
* Note: if fail, we will be called again from the postmaster main loop.
*/
int
-pgarch_start(void)
+PgArchiverPrep(int argc, char *argv[])
{
time_t curtime;
- pid_t pgArchPid;
/*
* Do nothing if no archiver needed
@@ -140,86 +134,20 @@ pgarch_start(void)
* the postmaster main loop, we will get another chance later.
*/
curtime = time(NULL);
- if ((unsigned int) (curtime - last_pgarch_start_time) <
+ if ((unsigned int) (curtime - last_pgarch_start_time) >=
(unsigned int) PGARCH_RESTART_INTERVAL)
- return 0;
- last_pgarch_start_time = curtime;
-
-#ifdef EXEC_BACKEND
- switch ((pgArchPid = pgarch_forkexec()))
-#else
- switch ((pgArchPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork archiver: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
+ last_pgarch_start_time = curtime;
- PgArchiverMain(0, NULL);
- break;
-#endif
-
- default:
- return (int) pgArchPid;
- }
-
- /* shouldn't get here */
return 0;
}
-/* ------------------------------------------------------------
- * Local functions called by archiver follow
- * ------------------------------------------------------------
- */
-
-
-#ifdef EXEC_BACKEND
-
-/*
- * pgarch_forkexec() -
- *
- * Format up the arglist for, then fork and exec, archive process
- */
-static pid_t
-pgarch_forkexec(void)
-{
- char *av[10];
- int ac = 0;
-
- av[ac++] = "postgres";
-
- av[ac++] = "--forkarch";
-
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif /* EXEC_BACKEND */
-
-
/*
* PgArchiverMain
*
* The argc/argv parameters are valid only in EXEC_BACKEND case. However,
* since we don't use 'em, it hardly matters...
*/
-NON_EXEC_STATIC void
+void
PgArchiverMain(int argc, char *argv[])
{
/*
@@ -238,7 +166,7 @@ PgArchiverMain(int argc, char *argv[])
pqsignal(SIGCHLD, SIG_DFL);
PG_SETMASK(&UnBlockSig);
- MyBackendType = B_ARCHIVER;
+ /* Identify myself via ps */
init_ps_display(NULL);
pgarch_MainLoop();
@@ -246,6 +174,11 @@ PgArchiverMain(int argc, char *argv[])
exit(0);
}
+/* ------------------------------------------------------------
+ * Local functions called by archiver follow
+ * ------------------------------------------------------------
+ */
+
/* SIGQUIT signal handler for archiver process */
static void
pgarch_exit(SIGNAL_ARGS)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 9dea8195b8..d2e9301a1e 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -1785,7 +1785,7 @@ ServerLoop(void)
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
/* If we need to signal the autovacuum launcher, do so now */
if (avlauncher_needs_signal)
@@ -3055,7 +3055,7 @@ reaper(SIGNAL_ARGS)
if (!IsBinaryUpgrade && AutoVacuumingActive() && AutoVacPID == 0)
AutoVacPID = StartSubprocess(AutoVacuumLauncherType);
if (PgArchStartupAllowed() && PgArchPID == 0)
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
if (PgStatPID == 0)
PgStatPID = StartSubprocess(PgstatCollectorType);
@@ -3203,7 +3203,7 @@ reaper(SIGNAL_ARGS)
LogChildExit(LOG, _("archiver process"),
pid, exitstatus);
if (PgArchStartupAllowed())
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
continue;
}
@@ -5063,12 +5063,6 @@ SubPostmasterMain(int argc, char *argv[])
StartBackgroundWorker();
}
- if (strcmp(argv[1], "--forkarch") == 0)
- {
- /* Do not want to attach to shared memory */
-
- PgArchiverMain(argc, argv); /* does not return */
- }
if (strcmp(argv[1], "--forklog") == 0)
{
/* Do not want to attach to shared memory */
@@ -5197,7 +5191,7 @@ sigusr1_handler(SIGNAL_ARGS)
*/
Assert(PgArchPID == 0);
if (XLogArchivingAlways())
- PgArchPID = pgarch_start();
+ PgArchPID = StartSubprocess(PgArchiverType);
/*
* If we aren't planning to enter hot standby mode later, treat
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index ee44ef0789..f9d71a627c 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -15,6 +15,7 @@
#include "pgstat.h"
#include "postmaster/autovacuum.h"
#include "postmaster/bgwriter.h"
+#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
#include "postmaster/startup.h"
#include "postmaster/subprocess.h"
@@ -114,6 +115,15 @@ static PgSubprocess process_types[] = {
.fork_prep = PgstatCollectorPrep,
.entrypoint = PgstatCollectorMain,
.fork_failure = NULL
+ },
+ {
+ .name = "arch",
+ .desc = "archiver",
+ .needs_aux_proc = false,
+ .needs_shmem = false,
+ .fork_prep = PgArchiverPrep,
+ .entrypoint = PgArchiverMain,
+ .fork_failure = NULL
}
};
diff --git a/src/include/postmaster/pgarch.h b/src/include/postmaster/pgarch.h
index b3200874ca..ac527212b1 100644
--- a/src/include/postmaster/pgarch.h
+++ b/src/include/postmaster/pgarch.h
@@ -26,14 +26,7 @@
#define MAX_XFN_CHARS 40
#define VALID_XFN_CHARS "0123456789ABCDEF.history.backup.partial"
-/* ----------
- * Functions called from postmaster
- * ----------
- */
-extern int pgarch_start(void);
-
-#ifdef EXEC_BACKEND
+extern int PgArchiverPrep(int argc, char *argv[]);
extern void PgArchiverMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
#endif /* _PGARCH_H */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index a7ab036b7b..3f00f904dc 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -26,6 +26,7 @@ typedef enum
AutoVacuumLauncherType,
AutoVacuumWorkerType,
PgstatCollectorType,
+ PgArchiverType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.21.1
0007-Add-SysLoggerType-to-subprocess-struct.patch (text/x-patch, 21.2K) download
From 0f0e695510aeaee7ca24cb7ecaffa74f39bf9e67 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 11 Mar 2020 02:58:08 +0000
Subject: [PATCH 07/11] Add SysLoggerType to subprocess struct
This commit introduces two new fields to the subprocess struct:
1) keep_postmaster_memcontext allows subprocesses to specify whether or
not the PostmasterMemoryContext should be kept around after a fork.
2) postmaster_main allows postmaster to gain control to do some
necessary processing after a successful fork.
---
src/backend/postmaster/postmaster.c | 49 +++--
src/backend/postmaster/subprocess.c | 61 +++---
src/backend/postmaster/syslogger.c | 278 +++++++++++-----------------
src/backend/utils/error/elog.c | 4 +-
src/include/postmaster/subprocess.h | 18 +-
src/include/postmaster/syslogger.h | 10 +-
6 files changed, 199 insertions(+), 221 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index d2e9301a1e..94acaaf375 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -355,6 +355,8 @@ bool ClientAuthInProgress = false; /* T during new-client
* authentication */
bool redirection_done = false; /* stderr redirected for syslogger? */
+LogFileNo syslogFileNo = -1;
+LogFileNo csvlogFileNo = -1;
/* received START_AUTOVAC_LAUNCHER signal */
static volatile sig_atomic_t start_autovac_launcher = false;
@@ -513,6 +515,10 @@ typedef struct
SubprocessType MySubprocessType;
bool redirection_done;
+#ifdef EXEC_BACKEND
+ LogFileNo syslogFileNo;
+ LogFileNo csvlogFileNo;
+#endif
bool IsBinaryUpgrade;
int max_safe_fds;
int MaxBackends;
@@ -1083,7 +1089,7 @@ PostmasterMain(int argc, char *argv[])
/*
* If enabled, start up syslogger collection subprocess
*/
- SysLoggerPID = SysLogger_Start();
+ SysLoggerPID = StartSubprocess(SysLoggerType);
/*
* Reset whereToSendOutput from DestDebug (its starting state) to
@@ -1739,7 +1745,7 @@ ServerLoop(void)
/* If we have lost the log collector, try to start a new one */
if (SysLoggerPID == 0 && Logging_collector)
- SysLoggerPID = SysLogger_Start();
+ SysLoggerPID = StartSubprocess(SysLoggerType);
/*
* If no background writer process is running, and we are not in a
@@ -3228,7 +3234,7 @@ reaper(SIGNAL_ARGS)
{
SysLoggerPID = 0;
/* for safety's sake, launch new logger *first* */
- SysLoggerPID = SysLogger_Start();
+ SysLoggerPID = StartSubprocess(SysLoggerType);
if (!EXIT_STATUS_0(exitstatus))
LogChildExit(LOG, _("system logger process"),
pid, exitstatus);
@@ -5063,12 +5069,6 @@ SubPostmasterMain(int argc, char *argv[])
StartBackgroundWorker();
}
- if (strcmp(argv[1], "--forklog") == 0)
- {
- /* Do not want to attach to shared memory */
-
- SysLoggerMain(argc, argv); /* does not return */
- }
if (MySubprocess->needs_aux_proc)
{
@@ -5446,6 +5446,15 @@ StartSubprocess(SubprocessType type)
snprintf(typebuf, sizeof(typebuf), "-x%d", type);
argv[argc++] = typebuf;
+ /* Prep subprocess for forking */
+ if (MySubprocess->fork_prep)
+ {
+ if (MySubprocess->fork_prep(argc, argv))
+ {
+ return 0;
+ }
+ }
+
argv[argc] = NULL;
Assert(argc < lengthof(argv));
@@ -5461,10 +5470,16 @@ StartSubprocess(SubprocessType type)
/* Close the postmaster's sockets */
ClosePostmasterPorts(false);
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
+ /*
+ * Release postmaster's working memory context if the subprocess doesn't
+ * need it
+ */
+ if (!MySubprocess->keep_postmaster_memcontext)
+ {
+ MemoryContextSwitchTo(TopMemoryContext);
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+ }
/*
* Drop connection to postmaster's shared memory if the subprocess
@@ -5508,6 +5523,10 @@ StartSubprocess(SubprocessType type)
return 0;
}
+ /* Some subprocesses need to do more work in postmaster. */
+ if (MySubprocess->postmaster_main)
+ MySubprocess->postmaster_main(argc, argv);
+
/*
* in parent, successful fork
*/
@@ -6185,6 +6204,8 @@ save_backend_variables(BackendParameters *param, Port *port,
param->MySubprocessType = MySubprocessType;
param->redirection_done = redirection_done;
+ param->syslogFileNo = syslogFileNo;
+ param->csvlogFileNo = csvlogFileNo;
param->IsBinaryUpgrade = IsBinaryUpgrade;
param->max_safe_fds = max_safe_fds;
@@ -6422,6 +6443,8 @@ restore_backend_variables(BackendParameters *param, Port *port)
MySubprocessType = param->MySubprocessType;
redirection_done = param->redirection_done;
+ syslogFileNo = param->syslogFileNo;
+ csvlogFileNo = param->csvlogFileNo;
IsBinaryUpgrade = param->IsBinaryUpgrade;
max_safe_fds = param->max_safe_fds;
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index f9d71a627c..122eb31b41 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -19,6 +19,7 @@
#include "postmaster/postmaster.h"
#include "postmaster/startup.h"
#include "postmaster/subprocess.h"
+#include "postmaster/syslogger.h"
#include "postmaster/walwriter.h"
#include "replication/walreceiver.h"
@@ -31,25 +32,23 @@ static PgSubprocess process_types[] = {
.desc = "checker",
.needs_aux_proc = true,
.needs_shmem = true,
- .fork_prep = NULL,
- .entrypoint = CheckerModeMain,
- .fork_failure = NULL
+ .keep_postmaster_memcontext = false,
+ .entrypoint = CheckerModeMain
},
{
.name = "boot",
.desc = "bootstrap",
.needs_aux_proc = true,
.needs_shmem = true,
- .fork_prep = NULL,
- .entrypoint = BootstrapModeMain,
- .fork_failure = NULL
+ .keep_postmaster_memcontext = false,
+ .entrypoint = BootstrapModeMain
},
{
.name = "boot",
.desc = "startup",
.needs_aux_proc = true,
.needs_shmem = true,
- .fork_prep = NULL,
+ .keep_postmaster_memcontext = false,
.entrypoint = StartupProcessMain,
.fork_failure = StartupForkFailure
},
@@ -58,72 +57,76 @@ static PgSubprocess process_types[] = {
.desc = "background writer",
.needs_aux_proc = true,
.needs_shmem = true,
- .fork_prep = NULL,
- .entrypoint = BackgroundWriterMain,
- .fork_failure = NULL
+ .keep_postmaster_memcontext = false,
+ .entrypoint = BackgroundWriterMain
},
{
.name = "boot",
.desc = "checkpointer",
.needs_aux_proc = true,
.needs_shmem = true,
- .fork_prep = NULL,
- .entrypoint = CheckpointerMain,
- .fork_failure = NULL
+ .keep_postmaster_memcontext = false,
+ .entrypoint = CheckpointerMain
},
{
.name = "boot",
.desc = "wal writer",
.needs_aux_proc = true,
.needs_shmem = true,
- .fork_prep = NULL,
- .entrypoint = WalWriterMain,
- .fork_failure = NULL
+ .keep_postmaster_memcontext = false,
+ .entrypoint = WalWriterMain
},
{
.name = "boot",
.desc = "wal receiver",
.needs_aux_proc = true,
.needs_shmem = true,
- .fork_prep = NULL,
- .entrypoint = WalReceiverMain,
- .fork_failure = NULL
+ .keep_postmaster_memcontext = false,
+ .entrypoint = WalReceiverMain
},
{
.name = "avlauncher",
.desc = "autovacuum launcher",
.needs_aux_proc = false,
.needs_shmem = true,
- .fork_prep = NULL,
- .entrypoint = AutoVacLauncherMain,
- .fork_failure = NULL
+ .keep_postmaster_memcontext = false,
+ .entrypoint = AutoVacLauncherMain
},
{
.name = "avworker",
.desc = "autovacuum worker",
.needs_aux_proc = false,
.needs_shmem = true,
- .fork_prep = NULL,
- .entrypoint = AutoVacWorkerMain,
- .fork_failure = NULL
+ .keep_postmaster_memcontext = false,
+ .entrypoint = AutoVacWorkerMain
},
{
.name = "col",
.desc = "statistics collector",
.needs_aux_proc = false,
.needs_shmem = false,
+ .keep_postmaster_memcontext = false,
.fork_prep = PgstatCollectorPrep,
- .entrypoint = PgstatCollectorMain,
- .fork_failure = NULL
+ .entrypoint = PgstatCollectorMain
},
{
.name = "arch",
.desc = "archiver",
.needs_aux_proc = false,
.needs_shmem = false,
+ .keep_postmaster_memcontext = false,
.fork_prep = PgArchiverPrep,
- .entrypoint = PgArchiverMain,
- .fork_failure = NULL
+ .entrypoint = PgArchiverMain
+ },
+ {
+ .name = "log",
+ .desc = "syslogger",
+ .needs_aux_proc = false,
+ .needs_shmem = false,
+ .keep_postmaster_memcontext = true,
+ .fork_prep = SysLoggerPrep,
+ .entrypoint = SysLoggerMain,
+ .postmaster_main = SysLoggerPostmasterMain
}
};
diff --git a/src/backend/postmaster/syslogger.c b/src/backend/postmaster/syslogger.c
index ffcb54968f..8824196c43 100644
--- a/src/backend/postmaster/syslogger.c
+++ b/src/backend/postmaster/syslogger.c
@@ -38,8 +38,8 @@
#include "nodes/pg_list.h"
#include "pgstat.h"
#include "pgtime.h"
-#include "postmaster/fork_process.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "postmaster/syslogger.h"
#include "storage/dsm.h"
#include "storage/fd.h"
@@ -75,6 +75,10 @@ bool Log_truncate_on_rotation = false;
int Log_file_mode = S_IRUSR | S_IWUSR;
extern bool redirection_done;
+#ifdef EXEC_BACKEND
+extern LogFileNo syslogFileNo;
+extern LogFileNo csvlogFileNo;
+#endif
/*
* Private state
@@ -129,10 +133,8 @@ static volatile sig_atomic_t rotation_requested = false;
/* Local subroutines */
#ifdef EXEC_BACKEND
-static pid_t syslogger_forkexec(void);
-static void syslogger_parseArgs(int argc, char *argv[]);
+static void syslogger_openfds(void);
#endif
-NON_EXEC_STATIC void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
static void process_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static void flush_pipe_input(char *logbuffer, int *bytes_in_logbuffer);
static FILE *logfile_open(const char *filename, const char *mode,
@@ -153,7 +155,7 @@ static void update_metainfo_datafile(void);
* Main entry point for syslogger process
* argc/argv parameters are valid only in EXEC_BACKEND case.
*/
-NON_EXEC_STATIC void
+void
SysLoggerMain(int argc, char *argv[])
{
#ifndef WIN32
@@ -169,10 +171,9 @@ SysLoggerMain(int argc, char *argv[])
now = MyStartTime;
#ifdef EXEC_BACKEND
- syslogger_parseArgs(argc, argv);
+ syslogger_openfds();
#endif /* EXEC_BACKEND */
- MyBackendType = B_LOGGER;
init_ps_display(NULL);
/*
@@ -537,13 +538,13 @@ SysLoggerMain(int argc, char *argv[])
* Postmaster subroutine to start a syslogger subprocess.
*/
int
-SysLogger_Start(void)
+SysLoggerPrep(int argc, char *argv[])
{
- pid_t sysloggerPid;
char *filename;
+ /* Tell postmaster to stop forking */
if (!Logging_collector)
- return 0;
+ return -1;
/*
* If first time through, create the pipe which will receive stderr
@@ -626,179 +627,124 @@ SysLogger_Start(void)
}
#ifdef EXEC_BACKEND
- switch ((sysloggerPid = syslogger_forkexec()))
-#else
- switch ((sysloggerPid = fork_process()))
-#endif
- {
- case -1:
- ereport(LOG,
- (errmsg("could not fork system logger: %m")));
- return 0;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(true);
-
- /* Drop our connection to postmaster's shared memory, as well */
- dsm_detach_all();
- PGSharedMemoryDetach();
-
- /* do the work */
- SysLoggerMain(0, NULL);
- break;
-#endif
-
- default:
- /* success, in postmaster */
-
- /* now we redirect stderr, if not done already */
- if (!redirection_done)
- {
-#ifdef WIN32
- int fd;
-#endif
-
- /*
- * Leave a breadcrumb trail when redirecting, in case the user
- * forgets that redirection is active and looks only at the
- * original stderr target file.
- */
- ereport(LOG,
- (errmsg("redirecting log output to logging collector process"),
- errhint("Future log output will appear in directory \"%s\".",
- Log_directory)));
-
-#ifndef WIN32
- fflush(stdout);
- if (dup2(syslogPipe[1], fileno(stdout)) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stdout: %m")));
- fflush(stderr);
- if (dup2(syslogPipe[1], fileno(stderr)) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- /* Now we are done with the write end of the pipe. */
- close(syslogPipe[1]);
- syslogPipe[1] = -1;
-#else
-
- /*
- * open the pipe in binary mode and make sure stderr is binary
- * after it's been dup'ed into, to avoid disturbing the pipe
- * chunking protocol.
- */
- fflush(stderr);
- fd = _open_osfhandle((intptr_t) syslogPipe[1],
- _O_APPEND | _O_BINARY);
- if (dup2(fd, _fileno(stderr)) < 0)
- ereport(FATAL,
- (errcode_for_file_access(),
- errmsg("could not redirect stderr: %m")));
- close(fd);
- _setmode(_fileno(stderr), _O_BINARY);
-
- /*
- * Now we are done with the write end of the pipe.
- * CloseHandle() must not be called because the preceding
- * close() closes the underlying handle.
- */
- syslogPipe[1] = 0;
-#endif
- redirection_done = true;
- }
-
- /* postmaster will never write the file(s); close 'em */
- fclose(syslogFile);
- syslogFile = NULL;
- if (csvlogFile != NULL)
- {
- fclose(csvlogFile);
- csvlogFile = NULL;
- }
- return (int) sysloggerPid;
- }
-
- /* we should never reach here */
- return 0;
-}
-
-
-#ifdef EXEC_BACKEND
-
-/*
- * syslogger_forkexec() -
- *
- * Format up the arglist for, then fork and exec, a syslogger process
- */
-static pid_t
-syslogger_forkexec(void)
-{
- char *av[10];
- int ac = 0;
- char filenobuf[32];
- char csvfilenobuf[32];
-
- av[ac++] = "postgres";
- av[ac++] = "--forklog";
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
-
- /* static variables (those not passed by write_backend_variables) */
#ifndef WIN32
if (syslogFile != NULL)
- snprintf(filenobuf, sizeof(filenobuf), "%d",
- fileno(syslogFile));
+ syslogFileNo = fileno(syslogFile);
else
- strcpy(filenobuf, "-1");
+ syslogFileNo = -1;
#else /* WIN32 */
if (syslogFile != NULL)
- snprintf(filenobuf, sizeof(filenobuf), "%ld",
- (long) _get_osfhandle(_fileno(syslogFile)));
+ syslogFileNo = (long) _get_osfhandle(_fileno(syslogFile));
else
- strcpy(filenobuf, "0");
+ syslogFileNo = 0;
#endif /* WIN32 */
- av[ac++] = filenobuf;
#ifndef WIN32
if (csvlogFile != NULL)
- snprintf(csvfilenobuf, sizeof(csvfilenobuf), "%d",
- fileno(csvlogFile));
+ csvlogFileNo = fileno(csvlogFile);
else
- strcpy(csvfilenobuf, "-1");
+ csvlogFileNo = -1;
#else /* WIN32 */
if (csvlogFile != NULL)
- snprintf(csvfilenobuf, sizeof(csvfilenobuf), "%ld",
- (long) _get_osfhandle(_fileno(csvlogFile)));
+ csvlogFileNo = (long) _get_osfhandle(_fileno(csvlogFile));
else
- strcpy(csvfilenobuf, "0");
+ csvlogFileNo = 0;
#endif /* WIN32 */
- av[ac++] = csvfilenobuf;
+#endif /* EXEC_BACKEND */
+
+ return 0;
+}
+
+/*
+ * SysLoggerPostmasterMain
+ *
+ * Control function for subprocess parent (postmaster) after a successful fork.
+ */
+void
+SysLoggerPostmasterMain(int argc, char *argv[])
+{
+ /* success, in postmaster */
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ /* now we redirect stderr, if not done already */
+ if (!redirection_done)
+ {
+#ifdef WIN32
+ int fd;
+#endif
+
+ /*
+ * Leave a breadcrumb trail when redirecting, in case the user
+ * forgets that redirection is active and looks only at the
+ * original stderr target file.
+ */
+ ereport(LOG,
+ (errmsg("redirecting log output to logging collector process"),
+ errhint("Future log output will appear in directory \"%s\".",
+ Log_directory)));
- return postmaster_forkexec(ac, av);
+#ifndef WIN32
+ fflush(stdout);
+ if (dup2(syslogPipe[1], fileno(stdout)) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stdout: %m")));
+ fflush(stderr);
+ if (dup2(syslogPipe[1], fileno(stderr)) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ /* Now we are done with the write end of the pipe. */
+ close(syslogPipe[1]);
+ syslogPipe[1] = -1;
+#else
+
+ /*
+ * open the pipe in binary mode and make sure stderr is binary
+ * after it's been dup'ed into, to avoid disturbing the pipe
+ * chunking protocol.
+ */
+ fflush(stderr);
+ fd = _open_osfhandle((intptr_t) syslogPipe[1],
+ _O_APPEND | _O_BINARY);
+ if (dup2(fd, _fileno(stderr)) < 0)
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not redirect stderr: %m")));
+ close(fd);
+ _setmode(_fileno(stderr), _O_BINARY);
+
+ /*
+ * Now we are done with the write end of the pipe.
+ * CloseHandle() must not be called because the preceding
+ * close() closes the underlying handle.
+ */
+ syslogPipe[1] = 0;
+#endif
+ redirection_done = true;
+ }
+
+ /* postmaster will never write the file(s); close 'em */
+ fclose(syslogFile);
+ syslogFile = NULL;
+ if (csvlogFile != NULL)
+ {
+ fclose(csvlogFile);
+ csvlogFile = NULL;
+ }
}
+#ifdef EXEC_BACKEND
/*
- * syslogger_parseArgs() -
+ * syslogger_openfds() -
*
* Extract data from the arglist for exec'ed syslogger process
*/
static void
-syslogger_parseArgs(int argc, char *argv[])
+syslogger_openfds(void)
{
- int fd;
-
- Assert(argc == 5);
- argv += 3;
-
+#ifdef WIN32
+ int fd = 0;
+#endif
/*
* Re-open the error output files that were opened by SysLogger_Start().
*
@@ -807,21 +753,18 @@ syslogger_parseArgs(int argc, char *argv[])
* coded, we'll just crash on a null pointer dereference after failure...
*/
#ifndef WIN32
- fd = atoi(*argv++);
- if (fd != -1)
+ if (syslogFileNo != -1)
{
- syslogFile = fdopen(fd, "a");
+ syslogFile = fdopen(syslogFileNo, "a");
setvbuf(syslogFile, NULL, PG_IOLBF, 0);
}
- fd = atoi(*argv++);
- if (fd != -1)
+ if (csvlogFileNo != -1)
{
- csvlogFile = fdopen(fd, "a");
+ csvlogFile = fdopen(csvlogFileNo, "a");
setvbuf(csvlogFile, NULL, PG_IOLBF, 0);
}
#else /* WIN32 */
- fd = atoi(*argv++);
- if (fd != 0)
+ if (syslogFileNo != 0)
{
fd = _open_osfhandle(fd, _O_APPEND | _O_TEXT);
if (fd > 0)
@@ -830,8 +773,7 @@ syslogger_parseArgs(int argc, char *argv[])
setvbuf(syslogFile, NULL, PG_IOLBF, 0);
}
}
- fd = atoi(*argv++);
- if (fd != 0)
+ if (syslogFileNo != 0)
{
fd = _open_osfhandle(fd, _O_APPEND | _O_TEXT);
if (fd > 0)
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index b8858b132b..4ec598a28d 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -2839,7 +2839,7 @@ write_csvlog(ErrorData *edata)
appendStringInfoChar(&buf, '\n');
/* If in the syslogger process, try to write messages direct to file */
- if (MyBackendType == B_LOGGER)
+ if (MyBackendType == B_LOGGER && IsUnderPostmaster)
write_syslogger_file(buf.data, buf.len, LOG_DESTINATION_CSVLOG);
else
write_pipe_chunks(buf.data, buf.len, LOG_DESTINATION_CSVLOG);
@@ -3052,7 +3052,7 @@ send_message_to_server_log(ErrorData *edata)
}
/* If in the syslogger process, try to write messages direct to file */
- if (MyBackendType == B_LOGGER)
+ if (MyBackendType == B_LOGGER && IsUnderPostmaster)
write_syslogger_file(buf.data, buf.len, LOG_DESTINATION_STDERR);
/* Write to CSV log if enabled */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 3f00f904dc..d0d98a5d69 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -27,6 +27,7 @@ typedef enum
AutoVacuumWorkerType,
PgstatCollectorType,
PgArchiverType,
+ SysLoggerType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
@@ -34,19 +35,22 @@ typedef enum
typedef int (*SubprocessPrep) (int argc, char *argv[]);
typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
typedef bool (*SubprocessForkFailure) (int fork_errno);
+typedef void (*SubprocessPostmasterMain) (int argc, char *argv[]);
/* Current subprocess initializer */
extern void InitMySubprocess(SubprocessType type);
typedef struct PgSubprocess
{
- const char *name;
- const char *desc;
- bool needs_aux_proc;
- bool needs_shmem;
- SubprocessPrep fork_prep;
- SubprocessEntryPoint entrypoint;
- SubprocessForkFailure fork_failure;
+ const char *name;
+ const char *desc;
+ bool needs_aux_proc;
+ bool needs_shmem;
+ bool keep_postmaster_memcontext;
+ SubprocessPrep fork_prep;
+ SubprocessEntryPoint entrypoint;
+ SubprocessForkFailure fork_failure;
+ SubprocessPostmasterMain postmaster_main;
} PgSubprocess;
extern SubprocessType MySubprocessType;
diff --git a/src/include/postmaster/syslogger.h b/src/include/postmaster/syslogger.h
index f611bd1411..6fbcb0c509 100644
--- a/src/include/postmaster/syslogger.h
+++ b/src/include/postmaster/syslogger.h
@@ -76,14 +76,20 @@ extern int syslogPipe[2];
extern HANDLE syslogPipe[2];
#endif
+#ifndef WIN32
+typedef int LogFileNo;
+#else
+typedef long LogFileNo;
+#endif
extern int SysLogger_Start(void);
extern void write_syslogger_file(const char *buffer, int count, int dest);
-#ifdef EXEC_BACKEND
+/* Subprocess startup functions */
+extern int SysLoggerPrep(int argc, char *argv[]);
extern void SysLoggerMain(int argc, char *argv[]) pg_attribute_noreturn();
-#endif
+extern void SysLoggerPostmasterMain(int argc, char *argv[]);
extern bool CheckLogrotateSignal(void);
extern void RemoveLogrotateSignalFiles(void);
--
2.21.1
0008-Add-BgWorkerType-to-subprocess-struct.patch (text/x-patch, 26.4K) download
From 255fb32a532fd0acb399c5f4f53557e999140571 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Wed, 11 Mar 2020 03:32:41 +0000
Subject: [PATCH 08/11] Add BgWorkerType to subprocess struct
Export some of the backend-specific code and move the some bgworker
startup code out to src/backend/postmaster/bgworker.c.
---
src/backend/postmaster/bgworker.c | 157 ++++++++++-
src/backend/postmaster/postmaster.c | 297 +++-----------------
src/backend/postmaster/subprocess.c | 12 +
src/backend/utils/init/globals.c | 1 +
src/include/miscadmin.h | 1 +
src/include/postmaster/bgworker_internals.h | 11 +-
src/include/postmaster/postmaster.h | 53 ++++
src/include/postmaster/subprocess.h | 1 +
8 files changed, 266 insertions(+), 267 deletions(-)
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index 6c684b5e12..5a11adac0c 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -33,8 +33,10 @@
#include "storage/shmem.h"
#include "tcop/tcopprot.h"
#include "utils/ascii.h"
+#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/timeout.h"
+#include "utils/timestamp.h"
/*
* The postmaster's list of registered background workers, in private memory.
@@ -133,7 +135,55 @@ static const struct
/* Private functions. */
static bgworker_main_type LookupBackgroundWorkerFunction(const char *libraryname, const char *funcname);
+static bool assign_backendlist_entry(RegisteredBgWorker *rw);
+/*
+ * Allocate the Backend struct for a connected background worker, but don't
+ * add it to the list of backends just yet.
+ *
+ * On failure, return false without changing any worker state.
+ *
+ * Some info from the Backend is copied into the passed rw.
+ */
+static bool
+assign_backendlist_entry(RegisteredBgWorker *rw)
+{
+ Backend *bn;
+
+ /*
+ * Compute the cancel key that will be assigned to this session. We
+ * probably don't need cancel keys for background workers, but we'd better
+ * have something random in the field to prevent unfriendly people from
+ * sending cancels to them.
+ */
+ if (!RandomCancelKey(&MyCancelKey))
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random cancel key")));
+ return false;
+ }
+
+ bn = malloc(sizeof(Backend));
+ if (bn == NULL)
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+ return false;
+ }
+
+ bn->cancel_key = MyCancelKey;
+ bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
+ bn->bkend_type = BACKEND_TYPE_BGWORKER;
+ bn->dead_end = false;
+ bn->bgworker_notify = false;
+
+ rw->rw_backend = bn;
+ rw->rw_child_slot = bn->child_slot;
+
+ return true;
+}
/*
* Calculate shared memory needed.
@@ -670,6 +720,47 @@ bgworker_sigusr1_handler(SIGNAL_ARGS)
errno = save_errno;
}
+/*
+ * BackgroundWorkerPrep
+ *
+ * Postmaster subroutine to prepare a bgworker subprocess.
+ *
+ * Returns 0 on success or -1 on failure.
+ *
+ * Note: if fail, we will be called again from the postmaster main loop.
+ */
+int
+BackgroundWorkerPrep(int argc, char *argv[])
+{
+ RegisteredBgWorker *rw = MyRegisteredBgWorker;
+
+ Assert(MyRegisteredBgWorker);
+ Assert(rw->rw_pid == 0);
+
+ /*
+ * Allocate and assign the Backend element. Note we must do this before
+ * forking, so that we can handle out of memory properly.
+ *
+ * Treat failure as though the worker had crashed. That way, the
+ * postmaster will wait a bit before attempting to start it again; if it
+ * tried again right away, most likely it'd find itself repeating the
+ * out-of-memory or fork failure condition.
+ */
+ if (!assign_backendlist_entry(rw))
+ {
+ rw->rw_crashed_at = GetCurrentTimestamp();
+ return -1;
+ }
+
+ ereport(DEBUG1,
+ (errmsg("starting background worker process \"%s\"",
+ rw->rw_worker.bgw_name)));
+
+ bgworker_shmem_slot = rw->rw_shmem_slot;
+
+ return 0;
+}
+
/*
* Start a new background worker
*
@@ -677,12 +768,28 @@ bgworker_sigusr1_handler(SIGNAL_ARGS)
* postmaster.
*/
void
-StartBackgroundWorker(void)
+BackgroundWorkerMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
{
sigjmp_buf local_sigjmp_buf;
- BackgroundWorker *worker = MyBgworkerEntry;
+ BackgroundWorker *worker;
bgworker_main_type entrypt;
+#ifndef EXEC_BACKEND
+ /*
+ * Before blowing away PostmasterContext, save this bgworker's
+ * data where it can find it.
+ */
+ MyBgworkerEntry = (BackgroundWorker *)
+ MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
+ memcpy(MyBgworkerEntry, &(MyRegisteredBgWorker->rw_worker), sizeof(BackgroundWorker));
+
+ /* Now we can release postmaster's working memory context */
+ MemoryContextSwitchTo(TopMemoryContext);
+ MemoryContextDelete(PostmasterContext);
+ PostmasterContext = NULL;
+#endif
+
+ worker = MyBgworkerEntry;
if (worker == NULL)
elog(FATAL, "unable to find bgworker entry");
@@ -816,6 +923,52 @@ StartBackgroundWorker(void)
proc_exit(0);
}
+/*
+ * BackgroundWorkerForkFailure
+ *
+ * Handle bgworker fork failure gracefully.
+ */
+bool
+BackgroundWorkerForkFailure(int fork_errno)
+{
+ RegisteredBgWorker *rw = MyRegisteredBgWorker;
+
+ errno = fork_errno;
+
+ /* in postmaster, fork failed ... */
+ ereport(LOG,
+ (errmsg("could not fork worker process: %m")));
+
+ /* undo what assign_backendlist_entry did */
+ ReleasePostmasterChildSlot(rw->rw_child_slot);
+ rw->rw_child_slot = 0;
+ free(rw->rw_backend);
+ rw->rw_backend = NULL;
+ /* mark entry as crashed, so we'll try again later */
+ rw->rw_crashed_at = GetCurrentTimestamp();
+
+ /* Don't panic. */
+ return false;
+}
+
+/*
+ * BackgroundWorkerPostmasterMain
+ */
+void
+BackgroundWorkerPostmasterMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ RegisteredBgWorker *rw = MyRegisteredBgWorker;
+
+ rw->rw_pid = MyChildProcPid;
+ rw->rw_backend->pid = rw->rw_pid;
+ ReportBackgroundWorkerPID(rw);
+ /* add new worker to lists of backends */
+ dlist_push_head(&BackendList, &(rw->rw_backend->elem));
+#ifdef EXEC_BACKEND
+ ShmemBackendArrayAdd(rw->rw_backend);
+#endif
+}
+
/*
* Register a new static background worker.
*
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 94acaaf375..2e15b69055 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -137,60 +137,14 @@
#include "storage/spin.h"
#endif
-
-/*
- * Possible types of a backend. Beyond being the possible bkend_type values in
- * struct bkend, these are OR-able request flag bits for SignalSomeChildren()
- * and CountChildren().
- */
-#define BACKEND_TYPE_NORMAL 0x0001 /* normal backend */
-#define BACKEND_TYPE_AUTOVAC 0x0002 /* autovacuum worker process */
-#define BACKEND_TYPE_WALSND 0x0004 /* walsender process */
-#define BACKEND_TYPE_BGWORKER 0x0008 /* bgworker process */
-#define BACKEND_TYPE_ALL 0x000F /* OR of all the above */
-
-#define BACKEND_TYPE_WORKER (BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER)
-
-/*
- * List of active backends (or child processes anyway; we don't actually
- * know whether a given child has become a backend or is still in the
- * authorization phase). This is used mainly to keep track of how many
- * children we have and send them appropriate signals when necessary.
- *
- * "Special" children such as the startup, bgwriter and autovacuum launcher
- * tasks are not in this list. Autovacuum worker and walsender are in it.
- * Also, "dead_end" children are in it: these are children launched just for
- * the purpose of sending a friendly rejection message to a would-be client.
- * We must track them because they are attached to shared memory, but we know
- * they will never become live backends. dead_end children are not assigned a
- * PMChildSlot.
- *
- * Background workers are in this list, too.
- */
-typedef struct bkend
-{
- pid_t pid; /* process id of backend */
- int32 cancel_key; /* cancel key for cancels for this backend */
- int child_slot; /* PMChildSlot for this backend, if any */
-
- /*
- * Flavor of backend or auxiliary process. Note that BACKEND_TYPE_WALSND
- * backends initially announce themselves as BACKEND_TYPE_NORMAL, so if
- * bkend_type is normal, you should check for a recent transition.
- */
- int bkend_type;
- bool dead_end; /* is it going to send an error and quit? */
- bool bgworker_notify; /* gets bgworker start/stop notifications */
- dlist_node elem; /* list link in BackendList */
-} Backend;
-
-static dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
+dlist_head BackendList = DLIST_STATIC_INIT(BackendList);
#ifdef EXEC_BACKEND
static Backend *ShmemBackendArray;
#endif
BackgroundWorker *MyBgworkerEntry = NULL;
+RegisteredBgWorker *MyRegisteredBgWorker = NULL;
/* Struct containing postmaster subprocess control info */
SubprocessType MySubprocessType;
@@ -358,6 +312,8 @@ bool redirection_done = false; /* stderr redirected for syslogger? */
LogFileNo syslogFileNo = -1;
LogFileNo csvlogFileNo = -1;
+int bgworker_shmem_slot = -1;
+
/* received START_AUTOVAC_LAUNCHER signal */
static volatile sig_atomic_t start_autovac_launcher = false;
@@ -414,7 +370,6 @@ static void processCancelRequest(Port *port, void *pkt);
static int initMasks(fd_set *rmask);
static void report_fork_failure_to_client(Port *port, int errnum);
static CAC_state canAcceptConnections(int backend_type);
-static bool RandomCancelKey(int32 *cancel_key);
static void signal_child(pid_t pid, int signal);
static bool SignalSomeChildren(int signal, int targets);
static void TerminateChildren(int signal);
@@ -422,7 +377,6 @@ static void TerminateChildren(int signal);
#define SignalChildren(sig) SignalSomeChildren(sig, BACKEND_TYPE_ALL)
static int CountChildren(int target);
-static bool assign_backendlist_entry(RegisteredBgWorker *rw);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
static pid_t StartSubprocess(SubprocessType type);
@@ -513,6 +467,7 @@ typedef struct
pg_time_t first_syslogger_file_time;
SubprocessType MySubprocessType;
+ int bgworker_shmem_slot;
bool redirection_done;
#ifdef EXEC_BACKEND
@@ -545,10 +500,10 @@ static bool save_backend_variables(BackendParameters *param, Port *port,
HANDLE childProcess, pid_t childPid);
#endif
-static void ShmemBackendArrayAdd(Backend *bn);
static void ShmemBackendArrayRemove(Backend *bn);
#endif /* EXEC_BACKEND */
+#define BGWORKER_LEN 15
/* Macros to check exit status of a child process */
#define EXIT_STATUS_0(st) ((st) == 0)
@@ -5047,59 +5002,47 @@ SubPostmasterMain(int argc, char *argv[])
/* And run the backend */
BackendRun(&port); /* does not return */
}
- if (strncmp(argv[1], "--forkbgworker=", 15) == 0)
- {
- int shmem_slot;
-
- /* do this as early as possible; in particular, before InitProcess() */
- IsBackgroundWorker = true;
+ if (MySubprocess->needs_aux_proc)
+ {
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
/* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
+ InitAuxiliaryProcess();
/* Attach process to shared data structures */
CreateSharedMemoryAndSemaphores();
- /* Fetch MyBgworkerEntry from shared memory */
- shmem_slot = atoi(argv[1] + 15);
- MyBgworkerEntry = BackgroundWorkerEntry(shmem_slot);
+ AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
+ }
- StartBackgroundWorker();
+ if (MySubprocessType == BgWorkerType)
+ {
+ /* do this as early as possible; in particular, before InitProcess() */
+ IsBackgroundWorker = true;
}
- if (MySubprocess->needs_aux_proc)
+ if (MySubprocess->needs_shmem)
{
/* Restore basic shared memory pointers */
InitShmemAccess(UsedShmemSegAddr);
/* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitAuxiliaryProcess();
+ InitProcess();
/* Attach process to shared data structures */
CreateSharedMemoryAndSemaphores();
-
- AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
}
- else
- {
- if (MySubprocess->needs_shmem)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
- }
-
- MySubprocess->entrypoint(argc - 2, argv + 2);
+ if (MySubprocessType == BgWorkerType)
+ {
+ /* Fetch MyBgworkerEntry from shared memory */
+ MyBgworkerEntry = BackgroundWorkerEntry(bgworker_shmem_slot);
}
+ MySubprocess->entrypoint(argc - 2, argv + 2);
+
abort(); /* shouldn't get here */
}
#endif /* EXEC_BACKEND */
@@ -5362,7 +5305,7 @@ StartupPacketTimeoutHandler(void)
/*
* Generate a random cancel key.
*/
-static bool
+bool
RandomCancelKey(int32 *cancel_key)
{
return pg_strong_random(cancel_key, sizeof(int32));
@@ -5460,15 +5403,17 @@ StartSubprocess(SubprocessType type)
#ifdef EXEC_BACKEND
pid = postmaster_forkexec(argc, argv);
+ MyChildProcPid = pid;
#else /* !EXEC_BACKEND */
pid = fork_process();
+ MyChildProcPid = pid;
if (pid == 0) /* child */
{
InitPostmasterChild();
/* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
+ ClosePostmasterPorts(MySubprocessType == SysLoggerType);
/*
* Release postmaster's working memory context if the subprocess doesn't
@@ -5518,6 +5463,8 @@ StartSubprocess(SubprocessType type)
{
if (MySubprocess->fork_failure(save_errno))
ExitPostmaster(1);
+ else
+ return -1;
}
return 0;
@@ -5760,124 +5707,6 @@ BackgroundWorkerUnblockSignals(void)
PG_SETMASK(&UnBlockSig);
}
-#ifdef EXEC_BACKEND
-static pid_t
-bgworker_forkexec(int shmem_slot)
-{
- char *av[10];
- int ac = 0;
- char forkav[MAXPGPATH];
-
- snprintf(forkav, MAXPGPATH, "--forkbgworker=%d", shmem_slot);
-
- av[ac++] = "postgres";
- av[ac++] = forkav;
- av[ac++] = NULL; /* filled in by postmaster_forkexec */
- av[ac] = NULL;
-
- Assert(ac < lengthof(av));
-
- return postmaster_forkexec(ac, av);
-}
-#endif
-
-/*
- * Start a new bgworker.
- * Starting time conditions must have been checked already.
- *
- * Returns true on success, false on failure.
- * In either case, update the RegisteredBgWorker's state appropriately.
- *
- * This code is heavily based on autovacuum.c, q.v.
- */
-static bool
-do_start_bgworker(RegisteredBgWorker *rw)
-{
- pid_t worker_pid;
-
- Assert(rw->rw_pid == 0);
-
- /*
- * Allocate and assign the Backend element. Note we must do this before
- * forking, so that we can handle failures (out of memory or child-process
- * slots) cleanly.
- *
- * Treat failure as though the worker had crashed. That way, the
- * postmaster will wait a bit before attempting to start it again; if we
- * tried again right away, most likely we'd find ourselves hitting the
- * same resource-exhaustion condition.
- */
- if (!assign_backendlist_entry(rw))
- {
- rw->rw_crashed_at = GetCurrentTimestamp();
- return false;
- }
-
- ereport(DEBUG1,
- (errmsg("starting background worker process \"%s\"",
- rw->rw_worker.bgw_name)));
-
-#ifdef EXEC_BACKEND
- switch ((worker_pid = bgworker_forkexec(rw->rw_shmem_slot)))
-#else
- switch ((worker_pid = fork_process()))
-#endif
- {
- case -1:
- /* in postmaster, fork failed ... */
- ereport(LOG,
- (errmsg("could not fork worker process: %m")));
- /* undo what assign_backendlist_entry did */
- ReleasePostmasterChildSlot(rw->rw_child_slot);
- rw->rw_child_slot = 0;
- free(rw->rw_backend);
- rw->rw_backend = NULL;
- /* mark entry as crashed, so we'll try again later */
- rw->rw_crashed_at = GetCurrentTimestamp();
- break;
-
-#ifndef EXEC_BACKEND
- case 0:
- /* in postmaster child ... */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /*
- * Before blowing away PostmasterContext, save this bgworker's
- * data where it can find it.
- */
- MyBgworkerEntry = (BackgroundWorker *)
- MemoryContextAlloc(TopMemoryContext, sizeof(BackgroundWorker));
- memcpy(MyBgworkerEntry, &rw->rw_worker, sizeof(BackgroundWorker));
-
- /* Release postmaster's working memory context */
- MemoryContextSwitchTo(TopMemoryContext);
- MemoryContextDelete(PostmasterContext);
- PostmasterContext = NULL;
-
- StartBackgroundWorker();
-
- exit(1); /* should not get here */
- break;
-#endif
- default:
- /* in postmaster, fork successful ... */
- rw->rw_pid = worker_pid;
- rw->rw_backend->pid = rw->rw_pid;
- ReportBackgroundWorkerPID(rw);
- /* add new worker to lists of backends */
- dlist_push_head(&BackendList, &rw->rw_backend->elem);
-#ifdef EXEC_BACKEND
- ShmemBackendArrayAdd(rw->rw_backend);
-#endif
- return true;
- }
-
- return false;
-}
-
/*
* Does the current postmaster state require starting a worker with the
* specified start_time?
@@ -5918,67 +5747,6 @@ bgworker_should_start_now(BgWorkerStartTime start_time)
return false;
}
-/*
- * Allocate the Backend struct for a connected background worker, but don't
- * add it to the list of backends just yet.
- *
- * On failure, return false without changing any worker state.
- *
- * Some info from the Backend is copied into the passed rw.
- */
-static bool
-assign_backendlist_entry(RegisteredBgWorker *rw)
-{
- Backend *bn;
-
- /*
- * Check that database state allows another connection. Currently the
- * only possible failure is CAC_TOOMANY, so we just log an error message
- * based on that rather than checking the error code precisely.
- */
- if (canAcceptConnections(BACKEND_TYPE_BGWORKER) != CAC_OK)
- {
- ereport(LOG,
- (errcode(ERRCODE_CONFIGURATION_LIMIT_EXCEEDED),
- errmsg("no slot available for new worker process")));
- return false;
- }
-
- /*
- * Compute the cancel key that will be assigned to this session. We
- * probably don't need cancel keys for background workers, but we'd better
- * have something random in the field to prevent unfriendly people from
- * sending cancels to them.
- */
- if (!RandomCancelKey(&MyCancelKey))
- {
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("could not generate random cancel key")));
- return false;
- }
-
- bn = malloc(sizeof(Backend));
- if (bn == NULL)
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- return false;
- }
-
- bn->cancel_key = MyCancelKey;
- bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
- bn->bkend_type = BACKEND_TYPE_BGWORKER;
- bn->dead_end = false;
- bn->bgworker_notify = false;
-
- rw->rw_backend = bn;
- rw->rw_child_slot = bn->child_slot;
-
- return true;
-}
-
/*
* If the time is right, start background worker(s).
*
@@ -6083,7 +5851,8 @@ maybe_start_bgworkers(void)
* crashed, but there's no need because the next run of this
* function will do that.
*/
- if (!do_start_bgworker(rw))
+ MyRegisteredBgWorker = rw;
+ if (StartSubprocess(BgWorkerType) <= 0)
{
StartWorkerNeeded = true;
return;
@@ -6202,6 +5971,7 @@ save_backend_variables(BackendParameters *param, Port *port,
param->first_syslogger_file_time = first_syslogger_file_time;
param->MySubprocessType = MySubprocessType;
+ param->bgworker_shmem_slot = bgworker_shmem_slot;
param->redirection_done = redirection_done;
param->syslogFileNo = syslogFileNo;
@@ -6441,6 +6211,7 @@ restore_backend_variables(BackendParameters *param, Port *port)
first_syslogger_file_time = param->first_syslogger_file_time;
MySubprocessType = param->MySubprocessType;
+ bgworker_shmem_slot = param->bgworker_shmem_slot;
redirection_done = param->redirection_done;
syslogFileNo = param->syslogFileNo;
@@ -6498,7 +6269,7 @@ ShmemBackendArrayAllocation(void)
memset(ShmemBackendArray, 0, size);
}
-static void
+void
ShmemBackendArrayAdd(Backend *bn)
{
/* The array slot corresponding to my PMChildSlot should be free */
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 122eb31b41..95c12da3cd 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -14,6 +14,7 @@
#include "bootstrap/bootstrap.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
+#include "postmaster/bgworker_internals.h"
#include "postmaster/bgwriter.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
@@ -127,6 +128,17 @@ static PgSubprocess process_types[] = {
.fork_prep = SysLoggerPrep,
.entrypoint = SysLoggerMain,
.postmaster_main = SysLoggerPostmasterMain
+ },
+ {
+ .name = "bgworker",
+ .desc = "background worker",
+ .needs_aux_proc = false,
+ .needs_shmem = true,
+ .keep_postmaster_memcontext = true,
+ .fork_prep = BackgroundWorkerPrep,
+ .entrypoint = BackgroundWorkerMain,
+ .fork_failure = BackgroundWorkerForkFailure,
+ .postmaster_main = BackgroundWorkerPostmasterMain
}
};
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index eb19644419..b31ce82f75 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -38,6 +38,7 @@ volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
int MyProcPid;
+int MyChildProcPid;
pg_time_t MyStartTime;
TimestampTz MyStartTimestamp;
struct Port *MyProcPort;
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 14fa127ab1..541ca5145d 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -163,6 +163,7 @@ extern PGDLLIMPORT int max_worker_processes;
extern PGDLLIMPORT int max_parallel_workers;
extern PGDLLIMPORT int MyProcPid;
+extern PGDLLIMPORT int MyChildProcPid;
extern PGDLLIMPORT pg_time_t MyStartTime;
extern PGDLLIMPORT TimestampTz MyStartTimestamp;
extern PGDLLIMPORT struct Port *MyProcPort;
diff --git a/src/include/postmaster/bgworker_internals.h b/src/include/postmaster/bgworker_internals.h
index f7e24664d5..27cc19ddb2 100644
--- a/src/include/postmaster/bgworker_internals.h
+++ b/src/include/postmaster/bgworker_internals.h
@@ -43,6 +43,10 @@ typedef struct RegisteredBgWorker
} RegisteredBgWorker;
extern slist_head BackgroundWorkerList;
+extern RegisteredBgWorker *MyRegisteredBgWorker;
+
+/* Hold the shmem slot globally instead of shoving it into argv */
+extern int bgworker_shmem_slot;
extern Size BackgroundWorkerShmemSize(void);
extern void BackgroundWorkerShmemInit(void);
@@ -53,8 +57,11 @@ extern void ReportBackgroundWorkerExit(slist_mutable_iter *cur);
extern void BackgroundWorkerStopNotifications(pid_t pid);
extern void ResetBackgroundWorkerCrashTimes(void);
-/* Function to start a background worker, called from postmaster.c */
-extern void StartBackgroundWorker(void) pg_attribute_noreturn();
+/* BgWorkerType subprocess */
+extern int BackgroundWorkerPrep(int argc, char *argv[]);
+extern void BackgroundWorkerMain(int argc, char *argv[]);
+extern bool BackgroundWorkerForkFailure(int fork_errno);
+extern void BackgroundWorkerPostmasterMain(int argc, char *argv[]);
#ifdef EXEC_BACKEND
extern BackgroundWorker *BackgroundWorkerEntry(int slotno);
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index babc87dfc9..b1411c1ed2 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -13,6 +13,41 @@
#ifndef _POSTMASTER_H
#define _POSTMASTER_H
+#include "lib/ilist.h"
+
+/*
+ * List of active backends (or child processes anyway; we don't actually
+ * know whether a given child has become a backend or is still in the
+ * authorization phase). This is used mainly to keep track of how many
+ * children we have and send them appropriate signals when necessary.
+ *
+ * "Special" children such as the startup, bgwriter and autovacuum launcher
+ * tasks are not in this list. Autovacuum worker and walsender are in it.
+ * Also, "dead_end" children are in it: these are children launched just for
+ * the purpose of sending a friendly rejection message to a would-be client.
+ * We must track them because they are attached to shared memory, but we know
+ * they will never become live backends. dead_end children are not assigned a
+ * PMChildSlot.
+ *
+ * Background workers are in this list, too.
+ */
+typedef struct bkend
+{
+ pid_t pid; /* process id of backend */
+ int32 cancel_key; /* cancel key for cancels for this backend */
+ int child_slot; /* PMChildSlot for this backend, if any */
+
+ /*
+ * Flavor of backend or auxiliary process. Note that BACKEND_TYPE_WALSND
+ * backends initially announce themselves as BACKEND_TYPE_NORMAL, so if
+ * bkend_type is normal, you should check for a recent transition.
+ */
+ int bkend_type;
+ bool dead_end; /* is it going to send an error and quit? */
+ bool bgworker_notify; /* gets bgworker start/stop notifications */
+ dlist_node elem; /* list link in BackendList */
+} Backend;
+
/* GUC options */
extern bool EnableSSL;
extern int ReservedBackends;
@@ -54,14 +89,19 @@ extern int MaxLivePostmasterChildren(void);
extern bool PostmasterMarkPIDForWorkerNotify(int);
+extern bool RandomCancelKey(int32 *cancel_key);
+
#ifdef EXEC_BACKEND
extern pid_t postmaster_forkexec(int argc, char *argv[]);
extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
extern Size ShmemBackendArraySize(void);
extern void ShmemBackendArrayAllocation(void);
+extern void ShmemBackendArrayAdd(Backend *bn);
#endif
+extern PGDLLIMPORT dlist_head BackendList;
+
/*
* Note: MAX_BACKENDS is limited to 2^18-1 because that's the width reserved
* for buffer references in buf_internals.h. This limitation could be lifted
@@ -74,4 +114,17 @@ extern void ShmemBackendArrayAllocation(void);
*/
#define MAX_BACKENDS 0x3FFFF
+/*
+ * Possible types of a backend. Beyond being the possible bkend_type values in
+ * struct bkend, these are OR-able request flag bits for SignalSomeChildren()
+ * and CountChildren().
+ */
+#define BACKEND_TYPE_NORMAL 0x0001 /* normal backend */
+#define BACKEND_TYPE_AUTOVAC 0x0002 /* autovacuum worker process */
+#define BACKEND_TYPE_WALSND 0x0004 /* walsender process */
+#define BACKEND_TYPE_BGWORKER 0x0008 /* bgworker process */
+#define BACKEND_TYPE_ALL 0x000F /* OR of all the above */
+
+#define BACKEND_TYPE_WORKER (BACKEND_TYPE_AUTOVAC | BACKEND_TYPE_BGWORKER)
+
#endif /* _POSTMASTER_H */
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index d0d98a5d69..706d60e4d6 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -28,6 +28,7 @@ typedef enum
PgstatCollectorType,
PgArchiverType,
SysLoggerType,
+ BgWorkerType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.21.1
0009-Add-Backends-to-subprocess-struct.patch (text/x-patch, 15.3K) download
From 4bc68db1f2ad43e87e75ae8329fb1e8ecf80e023 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Mon, 16 Mar 2020 23:05:33 +0000
Subject: [PATCH 09/11] Add Backends to subprocess struct
---
src/backend/postmaster/postmaster.c | 370 ++++++++++++++--------------
src/backend/postmaster/subprocess.c | 11 +
src/include/postmaster/postmaster.h | 6 +
src/include/postmaster/subprocess.h | 1 +
4 files changed, 199 insertions(+), 189 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 2e15b69055..d61b8f805b 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -188,6 +188,8 @@ static char ExtraOptions[MAXPGPATH];
static bool Reinit = true;
static int SendStop = false;
+static Backend *MyBackend = NULL;
+
/* still more option variables */
bool EnableSSL = false;
@@ -363,7 +365,6 @@ static void BackendInitialize(Port *port);
static void BackendRun(Port *port) pg_attribute_noreturn();
static void ExitPostmaster(int status) pg_attribute_noreturn();
static int ServerLoop(void);
-static int BackendStartup(Port *port);
static int ProcessStartupPacket(Port *port, bool secure_done);
static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options);
static void processCancelRequest(Port *port, void *pkt);
@@ -413,7 +414,6 @@ typedef struct
} win32_deadchild_waitinfo;
#endif /* WIN32 */
-static pid_t backend_forkexec(Port *port);
static pid_t internal_forkexec(int argc, char *argv[], Port *port);
/* Type for a socket that can be inherited to a client process */
@@ -521,6 +521,8 @@ int postmaster_alive_fds[2] = {-1, -1};
HANDLE PostmasterHandle;
#endif
+static Port *ConnProcPort = NULL;
+
/*
* Postmaster main entry point
*/
@@ -1680,19 +1682,18 @@ ServerLoop(void)
break;
if (FD_ISSET(ListenSocket[i], &rmask))
{
- Port *port;
+ ConnProcPort = ConnCreate(ListenSocket[i]);
- port = ConnCreate(ListenSocket[i]);
- if (port)
+ if (ConnProcPort)
{
- BackendStartup(port);
+ StartSubprocess(ClientBackendType);
/*
* We no longer need the open socket or port structure
* in this process
*/
- StreamClose(port->sock);
- ConnFree(port);
+ StreamClose(ConnProcPort->sock);
+ ConnFree(ConnProcPort);
}
}
}
@@ -3224,6 +3225,137 @@ reaper(SIGNAL_ARGS)
errno = save_errno;
}
+/*
+ * BackendPrep
+ *
+ * Do all of the pre-fork setup for a new Backend.
+*/
+int
+BackendPrep(int argc, char *argv[])
+{
+ /*
+ * Create backend data structure. Better before the fork() so we can
+ * handle failure cleanly.
+ */
+ Backend *bn;
+
+ bn = (Backend *) malloc(sizeof(Backend));
+ if (!bn)
+ {
+ ereport(LOG,
+ (errcode(ERRCODE_OUT_OF_MEMORY),
+ errmsg("out of memory")));
+ }
+
+ /*
+ * Compute the cancel key that will be assigned to this backend. The
+ * backend will have its own copy in the forked-off process' value of
+ * MyCancelKey, so that it can transmit the key to the frontend.
+ */
+ if (!RandomCancelKey(&MyCancelKey))
+ {
+ free(bn);
+ ereport(LOG,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random cancel key")));
+ }
+
+ bn->cancel_key = MyCancelKey;
+
+ /* Pass down canAcceptConnections state */
+ ConnProcPort->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
+ bn->dead_end = (ConnProcPort->canAcceptConnections != CAC_OK &&
+ ConnProcPort->canAcceptConnections != CAC_WAITBACKUP);
+
+ /*
+ * Unless it's a dead_end child, assign it a child slot number
+ */
+ if (!bn->dead_end)
+ bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
+ else
+ bn->child_slot = 0;
+
+ /* Hasn't asked to be notified about any bgworkers yet */
+ bn->bgworker_notify = false;
+
+ MyBackend = bn;
+
+ return 0;
+}
+
+/*
+ * BackendMain
+ *
+ * Child code when forking a Backend.
+ */
+void BackendMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ /*
+ * Perform additional initialization and collect startup packet.
+ *
+ * We want to do this before InitProcess() for a couple of reasons: 1.
+ * so that we aren't eating up a PGPROC slot while waiting on the
+ * client. 2. so that if InitProcess() fails due to being out of
+ * PGPROC slots, we have already initialized libpq and are able to
+ * report the error to the client.
+ */
+#ifndef EXEC_BACKEND
+ free(MyBackend);
+ BackendInitialize(ConnProcPort);
+#endif
+
+ /* And run the backend */
+ BackendRun(ConnProcPort); /* does not return */
+}
+
+/*
+ * BackendPostmasterMain
+ *
+ * Parent code when forking a Backend.
+ */
+void BackendPostmasterMain(pg_attribute_unused() int argc, pg_attribute_unused() char *argv[])
+{
+ /* in parent, successful fork */
+ ereport(DEBUG2,
+ (errmsg_internal("forked new backend, pid=%d socket=%d",
+ (int) MyChildProcPid, (int) ConnProcPort->sock)));
+
+ /*
+ * Everything's been successful, it's safe to add this backend to our list
+ * of backends.
+ */
+ MyBackend->pid = MyChildProcPid;
+ MyBackend->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
+
+ dlist_push_head(&BackendList, &MyBackend->elem);
+
+#ifdef EXEC_BACKEND
+ if (!MyBackend->dead_end)
+ ShmemBackendArrayAdd(MyBackend);
+#endif
+
+}
+
+/*
+ * BackendForkFailure
+ *
+ * Backend cleanup in case a failure occurs forking a new Backend.
+ */
+bool
+BackendForkFailure(int child_errno)
+{
+ if (!MyBackend->dead_end)
+ (void) ReleasePostmasterChildSlot(MyBackend->child_slot);
+ free(MyBackend);
+ errno = child_errno;
+ ereport(LOG,
+ (errmsg("could not fork new process for connection: %m")));
+ report_fork_failure_to_client(ConnProcPort, child_errno);
+
+ /* Don't panic */
+ return false;
+}
+
/*
* Scan the bgworkers list and see if the given PID (which has just stopped
* or crashed) is in it. Handle its shutdown if so, and return true. If not a
@@ -4084,122 +4216,6 @@ TerminateChildren(int signal)
signal_child(PgStatPID, signal);
}
-/*
- * BackendStartup -- start backend process
- *
- * returns: STATUS_ERROR if the fork failed, STATUS_OK otherwise.
- *
- * Note: if you change this code, also consider StartAutovacuumWorker.
- */
-static int
-BackendStartup(Port *port)
-{
- Backend *bn; /* for backend cleanup */
- pid_t pid;
-
- /*
- * Create backend data structure. Better before the fork() so we can
- * handle failure cleanly.
- */
- bn = (Backend *) malloc(sizeof(Backend));
- if (!bn)
- {
- ereport(LOG,
- (errcode(ERRCODE_OUT_OF_MEMORY),
- errmsg("out of memory")));
- return STATUS_ERROR;
- }
-
- /*
- * Compute the cancel key that will be assigned to this backend. The
- * backend will have its own copy in the forked-off process' value of
- * MyCancelKey, so that it can transmit the key to the frontend.
- */
- if (!RandomCancelKey(&MyCancelKey))
- {
- free(bn);
- ereport(LOG,
- (errcode(ERRCODE_INTERNAL_ERROR),
- errmsg("could not generate random cancel key")));
- return STATUS_ERROR;
- }
-
- bn->cancel_key = MyCancelKey;
-
- /* Pass down canAcceptConnections state */
- port->canAcceptConnections = canAcceptConnections(BACKEND_TYPE_NORMAL);
- bn->dead_end = (port->canAcceptConnections != CAC_OK &&
- port->canAcceptConnections != CAC_WAITBACKUP);
-
- /*
- * Unless it's a dead_end child, assign it a child slot number
- */
- if (!bn->dead_end)
- bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
- else
- bn->child_slot = 0;
-
- /* Hasn't asked to be notified about any bgworkers yet */
- bn->bgworker_notify = false;
-
-#ifdef EXEC_BACKEND
- pid = backend_forkexec(port);
-#else /* !EXEC_BACKEND */
- pid = fork_process();
- if (pid == 0) /* child */
- {
- free(bn);
-
- /* Detangle from postmaster */
- InitPostmasterChild();
-
- /* Close the postmaster's sockets */
- ClosePostmasterPorts(false);
-
- /* Perform additional initialization and collect startup packet */
- BackendInitialize(port);
-
- /* And run the backend */
- BackendRun(port);
- }
-#endif /* EXEC_BACKEND */
-
- if (pid < 0)
- {
- /* in parent, fork failed */
- int save_errno = errno;
-
- if (!bn->dead_end)
- (void) ReleasePostmasterChildSlot(bn->child_slot);
- free(bn);
- errno = save_errno;
- ereport(LOG,
- (errmsg("could not fork new process for connection: %m")));
- report_fork_failure_to_client(port, save_errno);
- return STATUS_ERROR;
- }
-
- /* in parent, successful fork */
- ereport(DEBUG2,
- (errmsg_internal("forked new backend, pid=%d socket=%d",
- (int) pid, (int) port->sock)));
-
- /*
- * Everything's been successful, it's safe to add this backend to our list
- * of backends.
- */
- bn->pid = pid;
- bn->bkend_type = BACKEND_TYPE_NORMAL; /* Can change later to WALSND */
- dlist_push_head(&BackendList, &bn->elem);
-
-#ifdef EXEC_BACKEND
- if (!bn->dead_end)
- ShmemBackendArrayAdd(bn);
-#endif
-
- return STATUS_OK;
-}
-
/*
* Try to report backend fork() failure to client before we close the
* connection. Since we do not care to risk blocking the postmaster on
@@ -4492,33 +4508,12 @@ postmaster_forkexec(int argc, char *argv[])
Port port;
/* This entry point passes dummy values for the Port variables */
- memset(&port, 0, sizeof(port));
- return internal_forkexec(argc, argv, &port);
-}
-
-/*
- * backend_forkexec -- fork/exec off a backend process
- *
- * Some operating systems (WIN32) don't have fork() so we have to simulate
- * it by storing parameters that need to be passed to the child and
- * then create a new child process.
- *
- * returns the pid of the fork/exec'd process, or -1 on failure
- */
-static pid_t
-backend_forkexec(Port *port)
-{
- char *av[4];
- int ac = 0;
-
- av[ac++] = "postgres";
- av[ac++] = "--forkbackend";
- av[ac++] = NULL; /* filled in by internal_forkexec */
-
- av[ac] = NULL;
- Assert(ac < lengthof(av));
+ if (MyBackendType != B_BACKEND || !ConnProcPort)
+ memset(&port, 0, sizeof(port));
+ else
+ port = *ConnProcPort;
- return internal_forkexec(ac, av, port);
+ return internal_forkexec(argc, argv, &port);
}
#ifndef WIN32
@@ -4857,6 +4852,7 @@ SubPostmasterMain(int argc, char *argv[])
/* Read in the variables file */
memset(&port, 0, sizeof(Port));
read_backend_variables(argv[2], &port);
+ ConnProcPort = &port;
/* Close the postmaster's sockets (as soon as we know them) */
ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
@@ -4898,11 +4894,7 @@ SubPostmasterMain(int argc, char *argv[])
* sometimes impossible to attach to shared memory at the desired address.
* Return the setting to its old value (usually '1' or '2') when finished.
*/
- if (strcmp(argv[1], "--forkbackend") == 0 ||
- strcmp(argv[1], "--forkavlauncher") == 0 ||
- strcmp(argv[1], "--forkavworker") == 0 ||
- strcmp(argv[1], "--forkboot") == 0 ||
- strncmp(argv[1], "--forkbgworker=", 15) == 0)
+ if (MySubprocess->needs_shmem)
PGSharedMemoryReAttach();
else
PGSharedMemoryNoReAttach();
@@ -4951,11 +4943,23 @@ SubPostmasterMain(int argc, char *argv[])
*/
process_shared_preload_libraries();
- /* Run backend or appropriate child */
- if (strcmp(argv[1], "--forkbackend") == 0)
+ if (MySubprocess->needs_aux_proc)
{
- Assert(argc == 3); /* shouldn't be any more args */
+ /* Restore basic shared memory pointers */
+ InitShmemAccess(UsedShmemSegAddr);
+
+ /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
+ InitAuxiliaryProcess();
+ /* Attach process to shared data structures */
+ CreateSharedMemoryAndSemaphores();
+
+ AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
+ }
+
+ /* Run backend or appropriate child */
+ if (MySubprocessType == ClientBackendType)
+ {
/*
* Need to reinitialize the SSL library in the backend, since the
* context structures contain function pointers and cannot be passed
@@ -4989,32 +4993,6 @@ SubPostmasterMain(int argc, char *argv[])
* report the error to the client.
*/
BackendInitialize(&port);
-
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- /* And run the backend */
- BackendRun(&port); /* does not return */
- }
-
- if (MySubprocess->needs_aux_proc)
- {
- /* Restore basic shared memory pointers */
- InitShmemAccess(UsedShmemSegAddr);
-
- /* Need a PGPROC to run CreateSharedMemoryAndSemaphores */
- InitAuxiliaryProcess();
-
- /* Attach process to shared data structures */
- CreateSharedMemoryAndSemaphores();
-
- AuxiliaryProcessMain(argc - 2, argv + 2); /* does not return */
}
if (MySubprocessType == BgWorkerType)
@@ -5035,14 +5013,28 @@ SubPostmasterMain(int argc, char *argv[])
CreateSharedMemoryAndSemaphores();
}
- if (MySubprocessType == BgWorkerType)
+ switch (MySubprocessType)
{
- /* Fetch MyBgworkerEntry from shared memory */
- MyBgworkerEntry = BackgroundWorkerEntry(bgworker_shmem_slot);
+ case AutoVacuumLauncherType:
+ case AutoVacuumWorkerType:
+ MySubprocess->entrypoint(argc - 2, argv + 2);
+ break;
+ case BgWorkerType:
+ /* Fetch MyBgworkerEntry from shared memory */
+ MyBgworkerEntry = BackgroundWorkerEntry(bgworker_shmem_slot);
+ /* fallthrough */
+ case PgArchiverType:
+ case PgstatCollectorType:
+ case SysLoggerType:
+ case ClientBackendType:
+ MySubprocess->entrypoint(argc, argv);
+ break;
+ default:
+ ereport(LOG,
+ (errmsg("could not start unknown process type (%d) under postmaster",
+ MySubprocessType)));
}
- MySubprocess->entrypoint(argc - 2, argv + 2);
-
abort(); /* shouldn't get here */
}
#endif /* EXEC_BACKEND */
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 95c12da3cd..70bc4a4c0c 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -139,6 +139,17 @@ static PgSubprocess process_types[] = {
.entrypoint = BackgroundWorkerMain,
.fork_failure = BackgroundWorkerForkFailure,
.postmaster_main = BackgroundWorkerPostmasterMain
+ },
+ {
+ .name = "backend",
+ .desc = "backend",
+ .needs_aux_proc = false,
+ .needs_shmem = true,
+ .keep_postmaster_memcontext = true,
+ .fork_prep = BackendPrep,
+ .entrypoint = BackendMain,
+ .fork_failure = BackendForkFailure,
+ .postmaster_main = BackendPostmasterMain
}
};
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index b1411c1ed2..6bc0af7c2b 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -91,6 +91,12 @@ extern bool PostmasterMarkPIDForWorkerNotify(int);
extern bool RandomCancelKey(int32 *cancel_key);
+extern int BackendPrep(int argc, char *argv[]);
+extern void BackendMain(int argc, char *argv[]);
+extern void BackendPostmasterMain(int argc, char *argv[]);
+extern bool BackendForkFailure(int child_errno);
+extern void BackgroundWorkerMain(int argc, char *argv[]);
+
#ifdef EXEC_BACKEND
extern pid_t postmaster_forkexec(int argc, char *argv[]);
extern void SubPostmasterMain(int argc, char *argv[]) pg_attribute_noreturn();
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 706d60e4d6..06e67efecc 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -29,6 +29,7 @@ typedef enum
PgArchiverType,
SysLoggerType,
BgWorkerType,
+ ClientBackendType,
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.21.1
0010-Add-WalSenderType-to-subprocess-struct.patch (text/x-patch, 2.0K) download
From fda7e44079289d7127de7adfd5d9a40fdb0d3b95 Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 17 Mar 2020 18:39:12 +0000
Subject: [PATCH 10/11] Add WalSenderType to subprocess struct
---
src/backend/postmaster/postmaster.c | 1 +
src/backend/postmaster/subprocess.c | 11 +++++++++++
src/include/postmaster/subprocess.h | 1 +
3 files changed, 13 insertions(+)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index d61b8f805b..90fe8ef92a 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -5027,6 +5027,7 @@ SubPostmasterMain(int argc, char *argv[])
case PgstatCollectorType:
case SysLoggerType:
case ClientBackendType:
+ case WalSenderType:
MySubprocess->entrypoint(argc, argv);
break;
default:
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 70bc4a4c0c..82861b1d5c 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -150,6 +150,17 @@ static PgSubprocess process_types[] = {
.entrypoint = BackendMain,
.fork_failure = BackendForkFailure,
.postmaster_main = BackendPostmasterMain
+ },
+ { /* Wal Sender placeholder */
+ .name = "walsender",
+ .desc = "wal sender",
+ .needs_aux_proc = false,
+ .needs_shmem = true,
+ .keep_postmaster_memcontext = true,
+ .fork_prep = BackendPrep,
+ .entrypoint = BackendMain,
+ .fork_failure = BackendForkFailure,
+ .postmaster_main = BackendPostmasterMain
}
};
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 06e67efecc..867e1cb499 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -31,6 +31,7 @@ typedef enum
BgWorkerType,
ClientBackendType,
+ WalSenderType, /* placeholder for wal sender so it can be identified in pgstat */
NUMSUBPROCESSTYPES /* Must be last! */
} SubprocessType;
--
2.21.1
0011-Move-to-new-MyBackendType.patch (text/x-patch, 21.4K) download
From a78600a9edd949e5b65e147defb5592c1450af6a Mon Sep 17 00:00:00 2001
From: Mike Palmiotto <mike.palmiotto@crunchydata.com>
Date: Tue, 17 Mar 2020 23:37:05 +0000
Subject: [PATCH 11/11] Move to new MyBackendType
- Replace MySubprocessType with MyBackendType
- Reshuffle the order of BackendType enums and move to subprocess.h
- Move GetBackendTypeDesc to subprocess.c and use process_types struct
- Change InitMySubprocess to more appropriate SetMySubprocess
- Handle resetting of MyBackendType on StartSubprocess failure
---
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/postmaster/bgworker.c | 1 -
src/backend/postmaster/postmaster.c | 114 ++++++++++++++--------------
src/backend/postmaster/subprocess.c | 11 ++-
src/backend/utils/error/elog.c | 1 +
src/backend/utils/init/miscinit.c | 54 -------------
src/include/miscadmin.h | 22 ------
src/include/pgstat.h | 1 +
src/include/postmaster/subprocess.h | 45 +++++------
9 files changed, 91 insertions(+), 160 deletions(-)
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 38d45d9163..7c6673c0fc 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -316,7 +316,7 @@ AuxiliaryProcessMain(int argc, char *argv[])
/*
* We've read the arguments and know what backend type we are.
*/
- InitMySubprocess((SubprocessType)MyAuxProcType);
+ SetMySubprocess((BackendType)MyAuxProcType);
/*
* Identify myself via ps
diff --git a/src/backend/postmaster/bgworker.c b/src/backend/postmaster/bgworker.c
index 5a11adac0c..618317a9ab 100644
--- a/src/backend/postmaster/bgworker.c
+++ b/src/backend/postmaster/bgworker.c
@@ -795,7 +795,6 @@ BackgroundWorkerMain(pg_attribute_unused() int argc, pg_attribute_unused() char
IsBackgroundWorker = true;
- MyBackendType = B_BG_WORKER;
init_ps_display(worker->bgw_name);
/*
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 90fe8ef92a..c7707671cd 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -146,9 +146,6 @@ static Backend *ShmemBackendArray;
BackgroundWorker *MyBgworkerEntry = NULL;
RegisteredBgWorker *MyRegisteredBgWorker = NULL;
-/* Struct containing postmaster subprocess control info */
-SubprocessType MySubprocessType;
-
/* The socket number we are listening for connections on */
int PostPortNumber;
@@ -380,7 +377,7 @@ static void TerminateChildren(int signal);
static int CountChildren(int target);
static void maybe_start_bgworkers(void);
static bool CreateOptsFile(int argc, char *argv[], char *fullprogname);
-static pid_t StartSubprocess(SubprocessType type);
+static pid_t StartSubprocess(BackendType type);
static void StartAutovacuumWorker(void);
static void MaybeStartWalReceiver(void);
static void InitPostmasterDeathWatchHandle(void);
@@ -466,8 +463,8 @@ typedef struct
TimestampTz PgReloadTime;
pg_time_t first_syslogger_file_time;
- SubprocessType MySubprocessType;
int bgworker_shmem_slot;
+ BackendType MyBackendType;
bool redirection_done;
#ifdef EXEC_BACKEND
@@ -1046,7 +1043,7 @@ PostmasterMain(int argc, char *argv[])
/*
* If enabled, start up syslogger collection subprocess
*/
- SysLoggerPID = StartSubprocess(SysLoggerType);
+ SysLoggerPID = StartSubprocess(B_LOGGER);
/*
* Reset whereToSendOutput from DestDebug (its starting state) to
@@ -1352,7 +1349,7 @@ PostmasterMain(int argc, char *argv[])
/*
* We're ready to rock and roll...
*/
- StartupPID = StartSubprocess(StartupType);
+ StartupPID = StartSubprocess(B_STARTUP);
Assert(StartupPID != 0);
StartupStatus = STARTUP_RUNNING;
pmState = PM_STARTUP;
@@ -1686,7 +1683,7 @@ ServerLoop(void)
if (ConnProcPort)
{
- StartSubprocess(ClientBackendType);
+ StartSubprocess(B_BACKEND);
/*
* We no longer need the open socket or port structure
@@ -1701,7 +1698,7 @@ ServerLoop(void)
/* If we have lost the log collector, try to start a new one */
if (SysLoggerPID == 0 && Logging_collector)
- SysLoggerPID = StartSubprocess(SysLoggerType);
+ SysLoggerPID = StartSubprocess(B_LOGGER);
/*
* If no background writer process is running, and we are not in a
@@ -1712,9 +1709,9 @@ ServerLoop(void)
pmState == PM_HOT_STANDBY)
{
if (CheckpointerPID == 0)
- CheckpointerPID = StartSubprocess(CheckpointerType);
+ CheckpointerPID = StartSubprocess(B_CHECKPOINTER);
if (BgWriterPID == 0)
- BgWriterPID = StartSubprocess(BgWriterType);
+ BgWriterPID = StartSubprocess(B_BG_WRITER);
}
/*
@@ -1723,7 +1720,7 @@ ServerLoop(void)
* be writing any new WAL).
*/
if (WalWriterPID == 0 && pmState == PM_RUN)
- WalWriterPID = StartSubprocess(WalWriterType);
+ WalWriterPID = StartSubprocess(B_WAL_WRITER);
/*
* If we have lost the autovacuum launcher, try to start a new one. We
@@ -1735,7 +1732,7 @@ ServerLoop(void)
(AutoVacuumingActive() || start_autovac_launcher) &&
pmState == PM_RUN)
{
- AutoVacPID = StartSubprocess(AutoVacuumLauncherType);
+ AutoVacPID = StartSubprocess(B_AUTOVAC_LAUNCHER);
if (AutoVacPID != 0)
start_autovac_launcher = false; /* signal processed */
}
@@ -1743,11 +1740,11 @@ ServerLoop(void)
/* If we have lost the stats collector, try to start a new one */
if (PgStatPID == 0 &&
(pmState == PM_RUN || pmState == PM_HOT_STANDBY))
- PgStatPID = StartSubprocess(PgstatCollectorType);
+ PgStatPID = StartSubprocess(B_STATS_COLLECTOR);
/* If we have lost the archiver, try to start a new one. */
if (PgArchPID == 0 && PgArchStartupAllowed())
- PgArchPID = StartSubprocess(PgArchiverType);
+ PgArchPID = StartSubprocess(B_ARCHIVER);
/* If we need to signal the autovacuum launcher, do so now */
if (avlauncher_needs_signal)
@@ -2222,9 +2219,9 @@ retry1:
port->user_name[NAMEDATALEN - 1] = '\0';
if (am_walsender)
- MyBackendType = B_WAL_SENDER;
+ SetMySubprocess(B_WAL_SENDER);
else
- MyBackendType = B_BACKEND;
+ SetMySubprocess(B_BACKEND);
/*
* Normal walsender backends, e.g. for streaming replication, are not
@@ -3004,22 +3001,22 @@ reaper(SIGNAL_ARGS)
* if this fails, we'll just try again later.
*/
if (CheckpointerPID == 0)
- CheckpointerPID = StartSubprocess(CheckpointerType);
+ CheckpointerPID = StartSubprocess(B_CHECKPOINTER);
if (BgWriterPID == 0)
- BgWriterPID = StartSubprocess(BgWriterType);
+ BgWriterPID = StartSubprocess(B_BG_WRITER);
if (WalWriterPID == 0)
- WalWriterPID = StartSubprocess(WalWriterType);
+ WalWriterPID = StartSubprocess(B_WAL_WRITER);
/*
* Likewise, start other special children as needed. In a restart
* situation, some of them may be alive already.
*/
if (!IsBinaryUpgrade && AutoVacuumingActive() && AutoVacPID == 0)
- AutoVacPID = StartSubprocess(AutoVacuumLauncherType);
+ AutoVacPID = StartSubprocess(B_AUTOVAC_LAUNCHER);
if (PgArchStartupAllowed() && PgArchPID == 0)
- PgArchPID = StartSubprocess(PgArchiverType);
+ PgArchPID = StartSubprocess(B_ARCHIVER);
if (PgStatPID == 0)
- PgStatPID = StartSubprocess(PgstatCollectorType);
+ PgStatPID = StartSubprocess(B_STATS_COLLECTOR);
/* workers may be scheduled to start now */
maybe_start_bgworkers();
@@ -3165,7 +3162,7 @@ reaper(SIGNAL_ARGS)
LogChildExit(LOG, _("archiver process"),
pid, exitstatus);
if (PgArchStartupAllowed())
- PgArchPID = StartSubprocess(PgArchiverType);
+ PgArchPID = StartSubprocess(B_ARCHIVER);
continue;
}
@@ -3181,7 +3178,7 @@ reaper(SIGNAL_ARGS)
LogChildExit(LOG, _("statistics collector process"),
pid, exitstatus);
if (pmState == PM_RUN || pmState == PM_HOT_STANDBY)
- PgStatPID = StartSubprocess(PgstatCollectorType);
+ PgStatPID = StartSubprocess(B_STATS_COLLECTOR);
continue;
}
@@ -3190,7 +3187,7 @@ reaper(SIGNAL_ARGS)
{
SysLoggerPID = 0;
/* for safety's sake, launch new logger *first* */
- SysLoggerPID = StartSubprocess(SysLoggerType);
+ SysLoggerPID = StartSubprocess(B_LOGGER);
if (!EXIT_STATUS_0(exitstatus))
LogChildExit(LOG, _("system logger process"),
pid, exitstatus);
@@ -3953,7 +3950,7 @@ PostmasterStateMachine(void)
Assert(Shutdown > NoShutdown);
/* Start the checkpointer if not running */
if (CheckpointerPID == 0)
- CheckpointerPID = StartSubprocess(CheckpointerType);
+ CheckpointerPID = StartSubprocess(B_CHECKPOINTER);
/* And tell it to shut down */
if (CheckpointerPID != 0)
{
@@ -4094,7 +4091,7 @@ PostmasterStateMachine(void)
reset_shared();
- StartupPID = StartSubprocess(StartupType);
+ StartupPID = StartSubprocess(B_STARTUP);
Assert(StartupPID != 0);
StartupStatus = STARTUP_RUNNING;
pmState = PM_STARTUP;
@@ -4181,6 +4178,7 @@ SignalSomeChildren(int signal, int target)
(errmsg_internal("sending signal %d to process %d",
signal, (int) bp->pid)));
signal_child(bp->pid, signal);
+
signaled = true;
}
return signaled;
@@ -4508,7 +4506,7 @@ postmaster_forkexec(int argc, char *argv[])
Port port;
/* This entry point passes dummy values for the Port variables */
- if (MyBackendType != B_BACKEND || !ConnProcPort)
+ if ((MyBackendType != B_BACKEND && MyBackendType != B_WAL_SENDER) || !ConnProcPort)
memset(&port, 0, sizeof(port));
else
port = *ConnProcPort;
@@ -4855,7 +4853,7 @@ SubPostmasterMain(int argc, char *argv[])
ConnProcPort = &port;
/* Close the postmaster's sockets (as soon as we know them) */
- ClosePostmasterPorts(strcmp(argv[1], "--forklog") == 0);
+ ClosePostmasterPorts(MyBackendType == B_LOGGER);
/*
* Set reference point for stack-depth checking
@@ -4876,7 +4874,7 @@ SubPostmasterMain(int argc, char *argv[])
/* We should only be here once per fork */
Assert(!MySubprocess);
- InitMySubprocess(MySubprocessType);
+ SetMySubprocess(MyBackendType);
/*
* If appropriate, physically re-attach to shared memory segment. We want
@@ -4900,9 +4898,9 @@ SubPostmasterMain(int argc, char *argv[])
PGSharedMemoryNoReAttach();
/* autovacuum needs this set before calling InitProcess */
- if (strcmp(argv[1], "--forkavlauncher") == 0)
+ if (MyBackendType == B_AUTOVAC_LAUNCHER)
AutovacuumLauncherIAm();
- if (strcmp(argv[1], "--forkavworker") == 0)
+ if (MyBackendType == B_AUTOVAC_WORKER)
AutovacuumWorkerIAm();
/*
@@ -4958,7 +4956,7 @@ SubPostmasterMain(int argc, char *argv[])
}
/* Run backend or appropriate child */
- if (MySubprocessType == ClientBackendType)
+ if (MyBackendType == B_BACKEND)
{
/*
* Need to reinitialize the SSL library in the backend, since the
@@ -4995,7 +4993,7 @@ SubPostmasterMain(int argc, char *argv[])
BackendInitialize(&port);
}
- if (MySubprocessType == BgWorkerType)
+ if (MyBackendType == B_BG_WORKER)
{
/* do this as early as possible; in particular, before InitProcess() */
IsBackgroundWorker = true;
@@ -5013,27 +5011,27 @@ SubPostmasterMain(int argc, char *argv[])
CreateSharedMemoryAndSemaphores();
}
- switch (MySubprocessType)
+ switch (MyBackendType)
{
- case AutoVacuumLauncherType:
- case AutoVacuumWorkerType:
+ case B_AUTOVAC_LAUNCHER:
+ case B_AUTOVAC_WORKER:
MySubprocess->entrypoint(argc - 2, argv + 2);
break;
- case BgWorkerType:
+ case B_BG_WORKER:
/* Fetch MyBgworkerEntry from shared memory */
MyBgworkerEntry = BackgroundWorkerEntry(bgworker_shmem_slot);
/* fallthrough */
- case PgArchiverType:
- case PgstatCollectorType:
- case SysLoggerType:
- case ClientBackendType:
- case WalSenderType:
+ case B_ARCHIVER:
+ case B_STATS_COLLECTOR:
+ case B_LOGGER:
+ case B_BACKEND:
+ case B_WAL_SENDER:
MySubprocess->entrypoint(argc, argv);
break;
default:
ereport(LOG,
(errmsg("could not start unknown process type (%d) under postmaster",
- MySubprocessType)));
+ MyBackendType)));
}
abort(); /* shouldn't get here */
@@ -5117,9 +5115,9 @@ sigusr1_handler(SIGNAL_ARGS)
* we'll just try again later.
*/
Assert(CheckpointerPID == 0);
- CheckpointerPID = StartSubprocess(CheckpointerType);
+ CheckpointerPID = StartSubprocess(B_CHECKPOINTER);
Assert(BgWriterPID == 0);
- BgWriterPID = StartSubprocess(BgWriterType);
+ BgWriterPID = StartSubprocess(B_BG_WRITER);
/*
* Start the archiver if we're responsible for (re-)archiving received
@@ -5127,7 +5125,7 @@ sigusr1_handler(SIGNAL_ARGS)
*/
Assert(PgArchPID == 0);
if (XLogArchivingAlways())
- PgArchPID = StartSubprocess(PgArchiverType);
+ PgArchPID = StartSubprocess(B_ARCHIVER);
/*
* If we aren't planning to enter hot standby mode later, treat
@@ -5151,7 +5149,7 @@ sigusr1_handler(SIGNAL_ARGS)
* Likewise, start other special children as needed.
*/
Assert(PgStatPID == 0);
- PgStatPID = StartSubprocess(PgstatCollectorType);
+ PgStatPID = StartSubprocess(B_STATS_COLLECTOR);
ereport(LOG,
(errmsg("database system is ready to accept read only connections")));
@@ -5356,8 +5354,9 @@ CountChildren(int target)
* the subprocess cleanup function decides whether or not to panic.
*/
static pid_t
-StartSubprocess(SubprocessType type)
+StartSubprocess(BackendType type)
{
+ BackendType prev_backend_type = MyBackendType;
pid_t pid;
char *argv[10];
int argc = 0;
@@ -5369,7 +5368,7 @@ StartSubprocess(SubprocessType type)
/*
* Get new subprocess data every time we start a new subprocess.
*/
- InitMySubprocess(type);
+ SetMySubprocess(type);
argv[argc++] = "postgres";
@@ -5387,6 +5386,7 @@ StartSubprocess(SubprocessType type)
{
if (MySubprocess->fork_prep(argc, argv))
{
+ SetMySubprocess(prev_backend_type);
return 0;
}
}
@@ -5406,7 +5406,7 @@ StartSubprocess(SubprocessType type)
InitPostmasterChild();
/* Close the postmaster's sockets */
- ClosePostmasterPorts(MySubprocessType == SysLoggerType);
+ ClosePostmasterPorts(MyBackendType == B_LOGGER);
/*
* Release postmaster's working memory context if the subprocess doesn't
@@ -5520,7 +5520,7 @@ StartAutovacuumWorker(void)
bn->child_slot = MyPMChildSlot = AssignPostmasterChildSlot();
bn->bgworker_notify = false;
- bn->pid = StartSubprocess(AutoVacuumWorkerType);
+ bn->pid = StartSubprocess(B_AUTOVAC_WORKER);
if (bn->pid > 0)
{
bn->bkend_type = BACKEND_TYPE_AUTOVAC;
@@ -5581,7 +5581,7 @@ MaybeStartWalReceiver(void)
pmState == PM_HOT_STANDBY || pmState == PM_WAIT_READONLY) &&
Shutdown == NoShutdown)
{
- WalReceiverPID = StartSubprocess(WalReceiverType);
+ WalReceiverPID = StartSubprocess(B_WAL_RECEIVER);
if (WalReceiverPID != 0)
WalReceiverRequested = false;
/* else leave the flag set, so we'll try again later */
@@ -5845,7 +5845,7 @@ maybe_start_bgworkers(void)
* function will do that.
*/
MyRegisteredBgWorker = rw;
- if (StartSubprocess(BgWorkerType) <= 0)
+ if (StartSubprocess(B_BG_WORKER) <= 0)
{
StartWorkerNeeded = true;
return;
@@ -5963,8 +5963,8 @@ save_backend_variables(BackendParameters *param, Port *port,
param->PgReloadTime = PgReloadTime;
param->first_syslogger_file_time = first_syslogger_file_time;
- param->MySubprocessType = MySubprocessType;
param->bgworker_shmem_slot = bgworker_shmem_slot;
+ param->MyBackendType = MyBackendType;
param->redirection_done = redirection_done;
param->syslogFileNo = syslogFileNo;
@@ -6203,8 +6203,8 @@ restore_backend_variables(BackendParameters *param, Port *port)
PgReloadTime = param->PgReloadTime;
first_syslogger_file_time = param->first_syslogger_file_time;
- MySubprocessType = param->MySubprocessType;
bgworker_shmem_slot = param->bgworker_shmem_slot;
+ MyBackendType = param->MyBackendType;
redirection_done = param->redirection_done;
syslogFileNo = param->syslogFileNo;
diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
index 82861b1d5c..4eff90c93f 100644
--- a/src/backend/postmaster/subprocess.c
+++ b/src/backend/postmaster/subprocess.c
@@ -24,7 +24,6 @@
#include "postmaster/walwriter.h"
#include "replication/walreceiver.h"
-SubprocessType MySubprocessType;
PgSubprocess *MySubprocess;
static PgSubprocess process_types[] = {
@@ -165,9 +164,15 @@ static PgSubprocess process_types[] = {
};
void
-InitMySubprocess(SubprocessType type)
+SetMySubprocess(BackendType type)
{
- MySubprocessType = type;
+ MyBackendType = type;
MySubprocess = &process_types[type];
MySubprocess->desc = gettext(MySubprocess->desc);
}
+
+const char *
+GetBackendTypeDesc(BackendType type)
+{
+ return gettext((&process_types[type])->desc);
+}
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 4ec598a28d..6291775540 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -74,6 +74,7 @@
#include "miscadmin.h"
#include "postmaster/bgworker.h"
#include "postmaster/postmaster.h"
+#include "postmaster/subprocess.h"
#include "postmaster/syslogger.h"
#include "storage/ipc.h"
#include "storage/proc.h"
diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c
index a7b7b12249..3d36836413 100644
--- a/src/backend/utils/init/miscinit.c
+++ b/src/backend/utils/init/miscinit.c
@@ -189,60 +189,6 @@ SwitchBackToLocalLatch(void)
SetLatch(MyLatch);
}
-const char *
-GetBackendTypeDesc(BackendType backendType)
-{
- const char *backendDesc = "unknown process type";
-
- switch (backendType)
- {
- case B_INVALID:
- backendDesc = "not initialized";
- break;
- case B_AUTOVAC_LAUNCHER:
- backendDesc = "autovacuum launcher";
- break;
- case B_AUTOVAC_WORKER:
- backendDesc = "autovacuum worker";
- break;
- case B_BACKEND:
- backendDesc = "client backend";
- break;
- case B_BG_WORKER:
- backendDesc = "background worker";
- break;
- case B_BG_WRITER:
- backendDesc = "background writer";
- break;
- case B_CHECKPOINTER:
- backendDesc = "checkpointer";
- break;
- case B_STARTUP:
- backendDesc = "startup";
- break;
- case B_WAL_RECEIVER:
- backendDesc = "walreceiver";
- break;
- case B_WAL_SENDER:
- backendDesc = "walsender";
- break;
- case B_WAL_WRITER:
- backendDesc = "walwriter";
- break;
- case B_ARCHIVER:
- backendDesc = "archiver";
- break;
- case B_STATS_COLLECTOR:
- backendDesc = "stats collector";
- break;
- case B_LOGGER:
- backendDesc = "logger";
- break;
- }
-
- return backendDesc;
-}
-
/* ----------------------------------------------------------------
* database path / name support stuff
* ----------------------------------------------------------------
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 541ca5145d..36a65865ab 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -307,28 +307,6 @@ extern void InitStandaloneProcess(const char *argv0);
extern void SwitchToSharedLatch(void);
extern void SwitchBackToLocalLatch(void);
-typedef enum BackendType
-{
- B_INVALID = 0,
- B_AUTOVAC_LAUNCHER,
- B_AUTOVAC_WORKER,
- B_BACKEND,
- B_BG_WORKER,
- B_BG_WRITER,
- B_CHECKPOINTER,
- B_STARTUP,
- B_WAL_RECEIVER,
- B_WAL_SENDER,
- B_WAL_WRITER,
- B_ARCHIVER,
- B_STATS_COLLECTOR,
- B_LOGGER,
-} BackendType;
-
-extern BackendType MyBackendType;
-
-extern const char *GetBackendTypeDesc(BackendType backendType);
-
extern void SetDatabasePath(const char *path);
extern void checkDataDir(void);
extern void SetDataDir(const char *dir);
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 92f4e7824c..46e71f4af4 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -17,6 +17,7 @@
#include "port/atomics.h"
#include "portability/instr_time.h"
#include "postmaster/pgarch.h"
+#include "postmaster/subprocess.h"
#include "storage/proc.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
diff --git a/src/include/postmaster/subprocess.h b/src/include/postmaster/subprocess.h
index 867e1cb499..7affd48e3a 100644
--- a/src/include/postmaster/subprocess.h
+++ b/src/include/postmaster/subprocess.h
@@ -13,27 +13,27 @@
#include "postmaster.h"
-typedef enum
+typedef enum BackendType
{
- NoProcessType = -1,
- CheckerType = 0,
- BootstrapType,
- StartupType,
- BgWriterType,
- CheckpointerType,
- WalWriterType,
- WalReceiverType, /* end of Auxiliary Process Forks */
- AutoVacuumLauncherType,
- AutoVacuumWorkerType,
- PgstatCollectorType,
- PgArchiverType,
- SysLoggerType,
- BgWorkerType,
- ClientBackendType,
-
- WalSenderType, /* placeholder for wal sender so it can be identified in pgstat */
- NUMSUBPROCESSTYPES /* Must be last! */
-} SubprocessType;
+ B_INVALID = -1,
+ B_CHECKER = 0,
+ B_BOOTSTRAP,
+ B_STARTUP,
+ B_BG_WRITER,
+ B_CHECKPOINTER,
+ B_WAL_WRITER,
+ B_WAL_RECEIVER, /* end of Auxiliary Processes */
+ B_AUTOVAC_LAUNCHER,
+ B_AUTOVAC_WORKER,
+ B_STATS_COLLECTOR,
+ B_ARCHIVER,
+ B_LOGGER,
+ B_BG_WORKER,
+ B_BACKEND,
+
+ B_WAL_SENDER, /* placeholder for wal sender so it can be identified in pgstat */
+ NUMBACKENDTYPES /* Must be last! */
+} BackendType;
typedef int (*SubprocessPrep) (int argc, char *argv[]);
typedef void (*SubprocessEntryPoint) (int argc, char *argv[]);
@@ -41,7 +41,8 @@ typedef bool (*SubprocessForkFailure) (int fork_errno);
typedef void (*SubprocessPostmasterMain) (int argc, char *argv[]);
/* Current subprocess initializer */
-extern void InitMySubprocess(SubprocessType type);
+extern void SetMySubprocess(BackendType type);
+extern const char *GetBackendTypeDesc(BackendType type);
typedef struct PgSubprocess
{
@@ -56,7 +57,7 @@ typedef struct PgSubprocess
SubprocessPostmasterMain postmaster_main;
} PgSubprocess;
-extern SubprocessType MySubprocessType;
+extern BackendType MyBackendType;
extern PGDLLIMPORT PgSubprocess *MySubprocess;
#endif /* SUBPROCESS_H */
--
2.21.1
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-02 14:53 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-04 04:56 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 18:50 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 21:52 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 01:04 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2020-03-18 13:22 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 14:17 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 15:26 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 16:07 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-19 10:35 ` Re: Auxiliary Processes and MyAuxProc Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2020-03-19 20:57 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2020-03-20 22:17 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-26 23:30 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
@ 2020-07-02 14:26 ` Daniel Gustafsson <daniel@yesql.se>
2020-07-02 15:03 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
1 sibling, 1 reply; 38+ messages in thread
From: Daniel Gustafsson @ 2020-07-02 14:26 UTC (permalink / raw)
To: Mike Palmiotto <mike.palmiotto@crunchydata.com>; +Cc: Andres Freund <andres@anarazel.de>; Peter Eisentraut <peter.eisentraut@2ndquadrant.com>; Justin Pryzby <pryzby@telsasoft.com>; Alvaro Herrera <alvherre@2ndquadrant.com>; Michael Paquier <michael@paquier.xyz>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
> On 27 Mar 2020, at 00:30, Mike Palmiotto <mike.palmiotto@crunchydata.com> wrote:
> Are these pieces required to make this patchset committable? Is there
> anything else needed at this point to make it committable?
The submitted version no longer applies, the 0009 patch has conflicts in
postmaster.c. Can you please submit a new version of the patch?
cheers ./daniel
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-02 14:53 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-04 04:56 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 18:50 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 21:52 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 01:04 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2020-03-18 13:22 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 14:17 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 15:26 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 16:07 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-19 10:35 ` Re: Auxiliary Processes and MyAuxProc Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2020-03-19 20:57 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2020-03-20 22:17 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-26 23:30 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-07-02 14:26 ` Re: Auxiliary Processes and MyAuxProc Daniel Gustafsson <daniel@yesql.se>
@ 2020-07-02 15:03 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2020-07-02 15:08 ` Re: Auxiliary Processes and MyAuxProc Daniel Gustafsson <daniel@yesql.se>
0 siblings, 1 reply; 38+ messages in thread
From: Alvaro Herrera @ 2020-07-02 15:03 UTC (permalink / raw)
To: Daniel Gustafsson <daniel@yesql.se>; +Cc: Mike Palmiotto <mike.palmiotto@crunchydata.com>; Andres Freund <andres@anarazel.de>; Peter Eisentraut <peter.eisentraut@2ndquadrant.com>; Justin Pryzby <pryzby@telsasoft.com>; Michael Paquier <michael@paquier.xyz>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On 2020-Jul-02, Daniel Gustafsson wrote:
> > On 27 Mar 2020, at 00:30, Mike Palmiotto <mike.palmiotto@crunchydata.com> wrote:
>
> > Are these pieces required to make this patchset committable? Is there
> > anything else needed at this point to make it committable?
>
> The submitted version no longer applies, the 0009 patch has conflicts in
> postmaster.c. Can you please submit a new version of the patch?
If the first 8 patches still apply, please do keep it as needs-review
though, since we can still give advice on those first parts.
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-02 14:53 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-04 04:56 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 18:50 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 21:52 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 01:04 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2020-03-18 13:22 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 14:17 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 15:26 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 16:07 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-19 10:35 ` Re: Auxiliary Processes and MyAuxProc Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2020-03-19 20:57 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2020-03-20 22:17 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-26 23:30 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-07-02 14:26 ` Re: Auxiliary Processes and MyAuxProc Daniel Gustafsson <daniel@yesql.se>
2020-07-02 15:03 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
@ 2020-07-02 15:08 ` Daniel Gustafsson <daniel@yesql.se>
0 siblings, 0 replies; 38+ messages in thread
From: Daniel Gustafsson @ 2020-07-02 15:08 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@2ndquadrant.com>; +Cc: Mike Palmiotto <mike.palmiotto@crunchydata.com>; Andres Freund <andres@anarazel.de>; Peter Eisentraut <peter.eisentraut@2ndquadrant.com>; Justin Pryzby <pryzby@telsasoft.com>; Michael Paquier <michael@paquier.xyz>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
> On 2 Jul 2020, at 17:03, Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
>
> On 2020-Jul-02, Daniel Gustafsson wrote:
>
>>> On 27 Mar 2020, at 00:30, Mike Palmiotto <mike.palmiotto@crunchydata.com> wrote:
>>
>>> Are these pieces required to make this patchset committable? Is there
>>> anything else needed at this point to make it committable?
>>
>> The submitted version no longer applies, the 0009 patch has conflicts in
>> postmaster.c. Can you please submit a new version of the patch?
>
> If the first 8 patches still apply, please do keep it as needs-review
> though, since we can still give advice on those first parts.
Done.
cheers ./daniel
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-02 14:53 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-04 04:56 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 18:50 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 21:52 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 01:04 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2020-03-18 13:22 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 14:17 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 15:26 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 16:07 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-19 10:35 ` Re: Auxiliary Processes and MyAuxProc Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2020-03-19 20:57 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2020-03-20 22:17 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-26 23:30 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
@ 2020-07-02 15:11 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2020-07-09 20:17 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
1 sibling, 1 reply; 38+ messages in thread
From: Alvaro Herrera @ 2020-07-02 15:11 UTC (permalink / raw)
To: Mike Palmiotto <mike.palmiotto@crunchydata.com>; +Cc: Andres Freund <andres@anarazel.de>; Peter Eisentraut <peter.eisentraut@2ndquadrant.com>; Justin Pryzby <pryzby@telsasoft.com>; Michael Paquier <michael@paquier.xyz>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On 2020-Mar-26, Mike Palmiotto wrote:
Regarding 0001:
> diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
> new file mode 100644
> index 0000000000..3e7a45bf10
> --- /dev/null
> +++ b/src/backend/postmaster/subprocess.c
> @@ -0,0 +1,62 @@
> +/*-------------------------------------------------------------------------
> + *
> + * subprocess.c
> + *
> + * Copyright (c) 2004-2020, PostgreSQL Global Development Group
> + *
> + *
> + * IDENTIFICATION
> + * src/backend/postmaster/syslogger.c
Wrong file identification.
> +static PgSubprocess process_types[] = {
> + {
> + .desc = "checker",
> + .entrypoint = CheckerModeMain
> + },
> + {
...
This array has to match the order in subprocess.h:
> +typedef enum
> +{
> + NoProcessType = -1,
> + CheckerType = 0,
> + BootstrapType,
> + StartupType,
> + BgWriterType,
> + CheckpointerType,
> + WalWriterType,
> + WalReceiverType, /* end of Auxiliary Process Forks */
> +
> + NUMSUBPROCESSTYPES /* Must be last! */
> +} SubprocessType;
This sort of thing is messy and unfriendly to maintain. I suggest we
use the same trick as in cmdtaglist.h and rmgrlist.h; see commits
2f9661311b83 and 5a1cd89f8f4a for examples.
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-02 14:53 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-04 04:56 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 18:50 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 21:52 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 01:04 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2020-03-18 13:22 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 14:17 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 15:26 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 16:07 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-19 10:35 ` Re: Auxiliary Processes and MyAuxProc Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2020-03-19 20:57 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2020-03-20 22:17 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-26 23:30 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-07-02 15:11 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
@ 2020-07-09 20:17 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-09-17 05:26 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 38+ messages in thread
From: Mike Palmiotto @ 2020-07-09 20:17 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@2ndquadrant.com>; +Cc: Andres Freund <andres@anarazel.de>; Peter Eisentraut <peter.eisentraut@2ndquadrant.com>; Justin Pryzby <pryzby@telsasoft.com>; Michael Paquier <michael@paquier.xyz>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Thu, Jul 2, 2020 at 11:11 AM Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
>
> On 2020-Mar-26, Mike Palmiotto wrote:
>
> Regarding 0001:
>
> > diff --git a/src/backend/postmaster/subprocess.c b/src/backend/postmaster/subprocess.c
> > new file mode 100644
> > index 0000000000..3e7a45bf10
> > --- /dev/null
> > +++ b/src/backend/postmaster/subprocess.c
> > @@ -0,0 +1,62 @@
> > +/*-------------------------------------------------------------------------
> > + *
> > + * subprocess.c
> > + *
> > + * Copyright (c) 2004-2020, PostgreSQL Global Development Group
> > + *
> > + *
> > + * IDENTIFICATION
> > + * src/backend/postmaster/syslogger.c
>
> Wrong file identification.
Thanks, I'll fix that.
<snip>
> > + WalReceiverType, /* end of Auxiliary Process Forks */
> > +
> > + NUMSUBPROCESSTYPES /* Must be last! */
> > +} SubprocessType;
>
> This sort of thing is messy and unfriendly to maintain. I suggest we
> use the same trick as in cmdtaglist.h and rmgrlist.h; see commits
> 2f9661311b83 and 5a1cd89f8f4a for examples.
Thanks for the reviews. I'm hoping to get to this next week (hopefully
sooner). It was on my TODO list to use this approach (from the last
round of reviews), so I'll make sure to do it first.
Other dangling items were the subprocess pids array and obviously the
rebase. I've got a decent AppVeyor/tap test workflow now, so it
should be a little less painful this time around.
Regards,
--
Mike Palmiotto
https://crunchydata.com
^ permalink raw reply [nested|flat] 38+ messages in thread
* Re: Auxiliary Processes and MyAuxProc
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Re: Auxiliary Processes and MyAuxProc Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Re: Auxiliary Processes and MyAuxProc Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Re: Auxiliary Processes and MyAuxProc Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-02 14:53 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-04 04:56 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 18:50 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 21:52 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 01:04 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2020-03-18 13:22 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 14:17 ` Re: Auxiliary Processes and MyAuxProc Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 15:26 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 16:07 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-19 10:35 ` Re: Auxiliary Processes and MyAuxProc Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2020-03-19 20:57 ` Re: Auxiliary Processes and MyAuxProc Andres Freund <andres@anarazel.de>
2020-03-20 22:17 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-26 23:30 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-07-02 15:11 ` Re: Auxiliary Processes and MyAuxProc Alvaro Herrera <alvherre@2ndquadrant.com>
2020-07-09 20:17 ` Re: Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
@ 2020-09-17 05:26 ` Michael Paquier <michael@paquier.xyz>
0 siblings, 0 replies; 38+ messages in thread
From: Michael Paquier @ 2020-09-17 05:26 UTC (permalink / raw)
To: Mike Palmiotto <mike.palmiotto@crunchydata.com>; +Cc: Alvaro Herrera <alvherre@2ndquadrant.com>; Andres Freund <andres@anarazel.de>; Peter Eisentraut <peter.eisentraut@2ndquadrant.com>; Justin Pryzby <pryzby@telsasoft.com>; Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>; Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers
On Thu, Jul 09, 2020 at 04:17:59PM -0400, Mike Palmiotto wrote:
> Thanks for the reviews. I'm hoping to get to this next week (hopefully
> sooner). It was on my TODO list to use this approach (from the last
> round of reviews), so I'll make sure to do it first.
Nothing has happened here in two months, so I am marking this stuff as
returned with feedback.
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 38+ messages in thread
end of thread, other threads:[~2020-09-17 05:26 UTC | newest]
Thread overview: 38+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2019-02-25 17:58 Auxiliary Processes and MyAuxProc Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 18:29 ` Tom Lane <tgl@sss.pgh.pa.us>
2019-02-25 18:41 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-02-25 22:25 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-08-14 15:01 ` Yuli Khodorkovskiy <yuli.khodorkovskiy@crunchydata.com>
2019-09-26 13:49 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 14:11 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-26 14:56 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2019-09-26 22:03 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-09-30 19:13 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 17:31 ` Andres Freund <andres@anarazel.de>
2019-10-03 18:33 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2019-10-03 18:39 ` Andres Freund <andres@anarazel.de>
2019-12-01 02:15 ` Michael Paquier <michael@paquier.xyz>
2020-03-01 02:51 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-02 14:53 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-04 04:56 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 18:50 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-17 21:52 ` Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 01:04 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2020-03-18 13:22 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 14:17 ` Justin Pryzby <pryzby@telsasoft.com>
2020-03-18 15:26 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 16:07 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-18 16:54 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2020-03-18 16:59 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-19 10:35 ` Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2020-03-19 13:29 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-21 17:40 ` Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2020-03-19 20:57 ` Andres Freund <andres@anarazel.de>
2020-03-20 22:17 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-03-26 23:30 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-07-02 14:26 ` Daniel Gustafsson <daniel@yesql.se>
2020-07-02 15:03 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2020-07-02 15:08 ` Daniel Gustafsson <daniel@yesql.se>
2020-07-02 15:11 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2020-07-09 20:17 ` Mike Palmiotto <mike.palmiotto@crunchydata.com>
2020-09-17 05:26 ` Michael Paquier <michael@paquier.xyz>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox