From: Alexander Lakhin <exclusion@gmail.com>
To: Fujii Masao <masao.fujii@gmail.com>
To: Ayush Tiwari <ayushtiwari.slg01@gmail.com>
Cc: Michael Paquier <michael@paquier.xyz>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: 041_checkpoint_at_promote.pl might fail due to race condition on child kill
Date: Sat, 14 Nov 2026 16:00:01 +0200
Message-ID: <1c733765-62af-48b3-b0d7-1e744c8b8522@gmail.com> (raw)
In-Reply-To: <CAHGQGwG3NJ0UZO0-u+GFxqO6x7RPu408OO0v0mjMdmYEPSaC-A@mail.gmail.com>
References: <e7abd282-655d-495e-94c9-e38a58917ebf@gmail.com>
<CAHGQGwFvuGE5cJJfTFtaYY855=3pmuiczAd13JvcpTVWnUVgFg@mail.gmail.com>
<c011d010-d227-4f27-b229-e9e0f2b668e0@gmail.com>
<CAHGQGwFtmZAuG+VruhcHWQcmv4=YcQtXLc9jcN3J8JPF_p4o-w@mail.gmail.com>
<CAJTYsWXkw4OqxNKVrp8dkhJjG8huFyvCjxEvSL7imSFembaP_g@mail.gmail.com>
<CAHGQGwG3NJ0UZO0-u+GFxqO6x7RPu408OO0v0mjMdmYEPSaC-A@mail.gmail.com>
Hello Fujii-san and Ayush,
30.10.2026 13:55, Fujii Masao wrote:
> On Fri, Oct 30, 2026 at 3:59 PM Ayush Tiwari
> <ayushtiwari.slg01@gmail.com> wrote:
>> Did multiple runs with it, and the changes lgtm.
> I've pushed the patch. Thanks!
Please look at a new failure of 041_checkpoint_at_promote produced by
tamandua recently [1]:
[03:10:10.869](0.025s) ok 4 - psql query died successfully after SIGKILL
[03:13:15.804](184.935s) # die: timed out waiting for file
/home/bf/bf-build/tamandua/REL_19_STABLE/pgsql.build/testrun/recovery/041_checkpoint_at_promote/log/041_checkpoint_at_promote_standby1.log
contents to match: (?^:all server processes terminated; reinitializing) at
/home/bf/bf-build/tamandua/REL_19_STABLE/pgsql/src/test/recovery/t/041_checkpoint_at_promote.pl line 166.
[03:13:15.806](0.003s) 1..4
timed out waiting for file
/home/bf/bf-build/tamandua/REL_19_STABLE/pgsql.build/testrun/recovery/041_checkpoint_at_promote/log/041_checkpoint_at_promote_standby1.log
contents to match: (?^:all server processes terminated; reinitializing) at
/home/bf/bf-build/tamandua/REL_19_STABLE/pgsql/src/test/recovery/t/041_checkpoint_at_promote.pl line 166.
# No postmaster PID for node "master"
041_checkpoint_at_promote_standby1.log contains:
2026-11-12 03:10:10.826 CET [3578531][client backend][6/2:0] LOG: statement: SELECT pg_backend_pid();
2026-11-12 03:10:10.861 CET [3577203][postmaster][:0] LOG: client backend (PID 3578531) was terminated by signal 9: Killed
2026-11-12 03:10:10.861 CET [3577203][postmaster][:0] DETAIL: Failed process was running: SELECT pg_backend_pid();
2026-11-12 03:10:10.862 CET [3577203][postmaster][:0] LOG: terminating any other active server processes
2026-11-12 03:10:10.928 CET [3577203][postmaster][:0] LOG: database system is ready to accept connections
There is no "terminating any other active server processes" message
indeed, the server was not shut down.
I managed to reproduce this locally (with the patch attached and with the
same parallel loop as before) and discovered that this can happen when
PMState goes from PM_HOT_STANDBY to PM_RUN (via PM_WAIT_BACKENDS) instead
of PM_HOT_STANDBY -> PM_WAIT_BACKENDS -> PM_WAIT_DEAD_END ->
PM_NO_CHILDREN -> "all server processes terminated; reinitializing".
This can be explained by two race conditions, with one precondition: the
startup process doesn't exit before pg_ctl kill.
1) The startup process becomes zombie for the postmaster if it exits when
waitpid() doesn't wait.
...
2026-11-14 13:29:02.900 UTC [695404][startup][:0] DEBUG: proc_exit(-1): 0 callbacks to make
...
2026-11-14 13:29:03.088 UTC [695367][postmaster][:0] DEBUG: sending signal 3/SIGQUIT to startup process with pid 695404
2026-11-14 13:29:03.100 UTC [695367][postmaster][:0] LOG: !!!signal_child| delayed signal 3/SIGQUIT to startup process
with pid 695404
...
2026-11-14 13:29:03.170 UTC [695367][postmaster][:0] DEBUG: startup process 695404 is still running
...
2026-11-14 13:29:03.198 UTC [695367][postmaster][:0] DEBUG: updating PMState from PM_WAIT_BACKENDS to PM_RUN
2) SIGQUIT sent to the startup process when it's being executed inside
exit(0), after the point when signal handlers of a process are not called
anymore, so SIGQUIT is ignored:
...
2026-11-14 07:54:44.900 EST [1110043][startup][:0] LOG: !!!proc_exit| delayed exit(0) for startup process
...
2026-11-14 07:54:44.900 EST [1110043][startup][:0] DEBUG: proc_exit(-1): 0 callbacks to make
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] LOG: !!!signal_child| delayed signal 3/SIGQUIT to startup process
with pid 1110043
...
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] DEBUG: startup process 1110043 is still running
...
2026-11-14 07:54:44.903 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 1110043, status: 0
2026-11-14 07:54:44.903 EST [1110037][postmaster][:0] DEBUG: releasing pm child slot 89
2026-11-14 07:54:44.903 EST [1110037][postmaster][:0] DEBUG: updating PMState from PM_WAIT_BACKENDS to PM_RUN
Corresponding logs for these 2 cases and also a normal run's log are
attached for reference.
[1] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=tamandua&dt=2026-11-12%2002%3A04%3A37
Best regards,
Alexander
Attachments:recovery-041-debugging.patch (text/x-patch, 2.2K) downloaddiff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.cindex f53aa9c404d..d1db634644b 100644--- a/src/backend/postmaster/postmaster.c+++ b/src/backend/postmaster/postmaster.c@@ -68,6 +68,7 @@
#include <unistd.h>
#include <signal.h>
#include <time.h>
+#include <sys/time.h>
#include <sys/wait.h>
#include <ctype.h>
#include <sys/stat.h>
@@ -2253,6 +2254,16 @@ handle_pm_child_exit_signal(SIGNAL_ARGS)
SetLatch(MyLatch);
}
+static pid_t+_waitpid(pid_t pid, int *status, int options)+{+pid_t ret;+elog(LOG, "!!!waitpid| pid: %d", pid);+ret = waitpid(pid, status, options);+elog(LOG, "!!!waitpid| ret: %d, status: %d", ret, *status);+return ret;+}+
/*
* Cleanup after a child process dies.
*/
@@ -2267,7 +2278,7 @@ process_pm_child_exit(void)
ereport(DEBUG4,
(errmsg_internal("reaping dead processes")));
- while ((pid = waitpid(-1, &exitstatus, WNOHANG)) > 0)+ while ((pid = _waitpid(-1, &exitstatus, WNOHANG)) > 0)
{
PMChild *pmchild;
@@ -3519,6 +3530,13 @@ signal_child(PMChild *pmchild, int signal)
GetBackendTypeDesc(pmchild->bkend_type),
(int) pmchild->pid)));
+if (pmchild == StartupPMChild && signal == SIGQUIT)+{+struct timeval current_time;+do gettimeofday(¤t_time, NULL); while ((current_time.tv_usec / 1000) % 100 >= 2);+elog(LOG, "!!!signal_child| delayed signal %d/%s to startup process with pid %d", signal, pm_signame(signal), (int) pmchild->pid);+}+
if (kill(pid, signal) < 0)
elog(DEBUG3, "kill(%ld,%d) failed: %m", (long) pid, signal);
#ifdef HAVE_SETSID
diff --git a/src/backend/storage/ipc/ipc.c b/src/backend/storage/ipc/ipc.cindex cb944edd8df..e96b3c2a035 100644--- a/src/backend/storage/ipc/ipc.c+++ b/src/backend/storage/ipc/ipc.c@@ -22,6 +22,7 @@
#include <signal.h>
#include <unistd.h>
#include <sys/stat.h>
+#include <sys/time.h>
#include "miscadmin.h"
#ifdef PROFILE_PID_DIR
@@ -154,6 +155,13 @@ proc_exit(int code)
elog(DEBUG3, "exit(%d)", code);
+if (AmStartupProcess() && code == 0)+{+struct timeval current_time;+pg_usleep(250000);+do gettimeofday(¤t_time, NULL); while ((current_time.tv_usec / 1000) % 100 >= 2);+elog(LOG, "!!!proc_exit| delayed exit(0) for startup process");+}
exit(code);
}
041_checkpoint_at_promote_standby1-2.log (text/x-log, 43.4K) download
...
2026-11-14 07:54:44.636 EST [1110043][startup][37/0:0] LOG: received promote request
2026-11-14 07:54:44.636 EST [1110043][startup][37/0:0] LOG: redo done at 0/04000028 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.58 s
2026-11-14 07:54:44.636 EST [1110043][startup][37/0:0] LOG: last completed transaction was at log time 2026-11-14 07:54:44.226403-05
2026-11-14 07:54:44.636 EST [1110043][startup][37/0:0] DEBUG: resetting unlogged relations: cleanup 0 init 1
2026-11-14 07:54:44.637 EST [1110043][startup][37/0:0] LOG: selected new timeline ID: 2
2026-11-14 07:54:44.637 EST [1110043][startup][37/0:0] DEBUG: updated min recovery point to 0/040000B0 on timeline 1
2026-11-14 07:54:44.648 EST [1110043][startup][37/0:0] DEBUG: could not remove file "pg_wal/000000020000000000000004": No such file or directory
2026-11-14 07:54:44.648 EST [1110043][startup][37/0:0] LOG: archive recovery complete
2026-11-14 07:54:44.648 EST [1110043][startup][37/0:0] DEBUG: MultiXactId wrap limit is 2147483648, limited by database with OID 1
2026-11-14 07:54:44.648 EST [1110043][startup][37/0:0] DEBUG: attempting to remove WAL segments newer than log file 000000020000000000000004
2026-11-14 07:54:44.648 EST [1110043][startup][37/0:0] DEBUG: removing all KnownAssignedXids
2026-11-14 07:54:44.648 EST [1110043][startup][37/0:0] DEBUG: release all standby locks
2026-11-14 07:54:44.648 EST [1110043][startup][37/0:0] DEBUG: shmem_exit(0): 5 before_shmem_exit callbacks to make
2026-11-14 07:54:44.648 EST [1110043][startup][37/0:0] DEBUG: shmem_exit(0): 8 on_shmem_exit callbacks to make
2026-11-14 07:54:44.648 EST [1110043][startup][:0] DEBUG: proc_exit(0): 1 callbacks to make
2026-11-14 07:54:44.648 EST [1110043][startup][:0] DEBUG: exit(0)
2026-11-14 07:54:44.738 EST [1110037][postmaster][:0] DEBUG: assigned pm child slot 2 for client backend
2026-11-14 07:54:44.738 EST [1110037][postmaster][:0] DEBUG: forked new client backend, pid=1110667 socket=8
2026-11-14 07:54:44.738 EST [1110667][client backend][:0] LOG: connection received: host=[local]
2026-11-14 07:54:44.738 EST [1110667][client backend][5/0:0] DEBUG: InitPostgres
2026-11-14 07:54:44.739 EST [1110667][client backend][5/1:0] LOG: connection authenticated: ...
2026-11-14 07:54:44.739 EST [1110667][client backend][5/1:0] LOG: connection authorized: ...
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] LOG: statement: SELECT injection_points_wakeup('create-restart-point');
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op invalid|target invalid|state HANDED_OUT : adding cb #1, id 2/aio_shared_buffer_readv_cb at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op invalid|target smgr|state HANDED_OUT : adding cb #2, id 1/aio_md_readv_cb at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state DEFINED : calling cb #1 2/aio_shared_buffer_readv_cb->stage(8) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state STAGED : staged (synchronous: 1, in_batch: 0) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #2, id 1/aio_md_readv_cb->complete_shared(0) with distilled result: (status OK, id 0, error_data 0, result 8192) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_shared(8) with distilled result: (status OK, id 0, error_data 0, result 1) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : after shared completion: distilled result: (status OK, id 0, error_data: 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_local(8) with distilled result: status OK, id 0, error_data 0, result 1 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: after local completion: result: (status OK, id 0, error_data 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_LOCAL : reclaiming: distilled_result: (status OK, id 0, error_data 0), raw_result: 8192 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op invalid|target invalid|state HANDED_OUT : adding cb #1, id 2/aio_shared_buffer_readv_cb at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op invalid|target smgr|state HANDED_OUT : adding cb #2, id 1/aio_md_readv_cb at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state DEFINED : calling cb #1 2/aio_shared_buffer_readv_cb->stage(8) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state STAGED : staged (synchronous: 1, in_batch: 0) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #2, id 1/aio_md_readv_cb->complete_shared(0) with distilled result: (status OK, id 0, error_data 0, result 8192) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_shared(8) with distilled result: (status OK, id 0, error_data 0, result 1) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : after shared completion: distilled result: (status OK, id 0, error_data: 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_local(8) with distilled result: status OK, id 0, error_data 0, result 1 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: after local completion: result: (status OK, id 0, error_data 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_LOCAL : reclaiming: distilled_result: (status OK, id 0, error_data 0), raw_result: 8192 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op invalid|target invalid|state HANDED_OUT : adding cb #1, id 2/aio_shared_buffer_readv_cb at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op invalid|target smgr|state HANDED_OUT : adding cb #2, id 1/aio_md_readv_cb at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state DEFINED : calling cb #1 2/aio_shared_buffer_readv_cb->stage(8) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state STAGED : staged (synchronous: 1, in_batch: 0) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #2, id 1/aio_md_readv_cb->complete_shared(0) with distilled result: (status OK, id 0, error_data 0, result 8192) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_shared(8) with distilled result: (status OK, id 0, error_data 0, result 1) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : after shared completion: distilled result: (status OK, id 0, error_data: 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_local(8) with distilled result: status OK, id 0, error_data 0, result 1 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: after local completion: result: (status OK, id 0, error_data 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_LOCAL : reclaiming: distilled_result: (status OK, id 0, error_data 0), raw_result: 8192 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op invalid|target invalid|state HANDED_OUT : adding cb #1, id 2/aio_shared_buffer_readv_cb at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op invalid|target smgr|state HANDED_OUT : adding cb #2, id 1/aio_md_readv_cb at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state DEFINED : calling cb #1 2/aio_shared_buffer_readv_cb->stage(8) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state STAGED : staged (synchronous: 1, in_batch: 0) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #2, id 1/aio_md_readv_cb->complete_shared(0) with distilled result: (status OK, id 0, error_data 0, result 8192) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_shared(8) with distilled result: (status OK, id 0, error_data 0, result 1) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : after shared completion: distilled result: (status OK, id 0, error_data: 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_local(8) with distilled result: status OK, id 0, error_data 0, result 1 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: after local completion: result: (status OK, id 0, error_data 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_LOCAL : reclaiming: distilled_result: (status OK, id 0, error_data 0), raw_result: 8192 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op invalid|target invalid|state HANDED_OUT : adding cb #1, id 2/aio_shared_buffer_readv_cb at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op invalid|target smgr|state HANDED_OUT : adding cb #2, id 1/aio_md_readv_cb at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state DEFINED : calling cb #1 2/aio_shared_buffer_readv_cb->stage(8) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state STAGED : staged (synchronous: 1, in_batch: 0) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #2, id 1/aio_md_readv_cb->complete_shared(0) with distilled result: (status OK, id 0, error_data 0, result 8192) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_shared(8) with distilled result: (status OK, id 0, error_data 0, result 1) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : after shared completion: distilled result: (status OK, id 0, error_data: 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_local(8) with distilled result: status OK, id 0, error_data 0, result 1 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: after local completion: result: (status OK, id 0, error_data 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_LOCAL : reclaiming: distilled_result: (status OK, id 0, error_data 0), raw_result: 8192 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op invalid|target invalid|state HANDED_OUT : adding cb #1, id 2/aio_shared_buffer_readv_cb at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op invalid|target smgr|state HANDED_OUT : adding cb #2, id 1/aio_md_readv_cb at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state DEFINED : calling cb #1 2/aio_shared_buffer_readv_cb->stage(8) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state STAGED : staged (synchronous: 1, in_batch: 0) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #2, id 1/aio_md_readv_cb->complete_shared(0) with distilled result: (status OK, id 0, error_data 0, result 8192) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_shared(8) with distilled result: (status OK, id 0, error_data 0, result 1) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : after shared completion: distilled result: (status OK, id 0, error_data: 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_local(8) with distilled result: status OK, id 0, error_data 0, result 1 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: after local completion: result: (status OK, id 0, error_data 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_LOCAL : reclaiming: distilled_result: (status OK, id 0, error_data 0), raw_result: 8192 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op invalid|target invalid|state HANDED_OUT : adding cb #1, id 2/aio_shared_buffer_readv_cb at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op invalid|target smgr|state HANDED_OUT : adding cb #2, id 1/aio_md_readv_cb at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state DEFINED : calling cb #1 2/aio_shared_buffer_readv_cb->stage(8) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state STAGED : staged (synchronous: 1, in_batch: 0) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #2, id 1/aio_md_readv_cb->complete_shared(0) with distilled result: (status OK, id 0, error_data 0, result 8192) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_shared(8) with distilled result: (status OK, id 0, error_data 0, result 1) at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : after shared completion: distilled result: (status OK, id 0, error_data: 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_local(8) with distilled result: status OK, id 0, error_data 0, result 1 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: after local completion: result: (status OK, id 0, error_data 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_LOCAL : reclaiming: distilled_result: (status OK, id 0, error_data 0), raw_result: 8192 at character 8
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op invalid|target invalid|state HANDED_OUT : adding cb #1, id 2/aio_shared_buffer_readv_cb
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op invalid|target smgr|state HANDED_OUT : adding cb #2, id 1/aio_md_readv_cb
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state DEFINED : calling cb #1 2/aio_shared_buffer_readv_cb->stage(8)
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state STAGED : staged (synchronous: 1, in_batch: 0)
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #2, id 1/aio_md_readv_cb->complete_shared(0) with distilled result: (status OK, id 0, error_data 0, result 8192)
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_shared(8) with distilled result: (status OK, id 0, error_data 0, result 1)
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : after shared completion: distilled result: (status OK, id 0, error_data: 0, result 1), raw_result: 8192
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_local(8) with distilled result: status OK, id 0, error_data 0, result 1
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: after local completion: result: (status OK, id 0, error_data 0, result 1), raw_result: 8192
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_LOCAL : reclaiming: distilled_result: (status OK, id 0, error_data 0), raw_result: 8192
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: find_in_path: trying ".../tmp_install/usr/local/pgsql/lib/injection_points"
2026-11-14 07:54:44.740 EST [1110667][client backend][5/2:0] DEBUG: find_in_path: trying ".../tmp_install/usr/local/pgsql/lib/injection_points.so"
2026-11-14 07:54:44.741 EST [1110667][client backend][5/0:0] DEBUG: shmem_exit(0): 5 before_shmem_exit callbacks to make
2026-11-14 07:54:44.741 EST [1110667][client backend][5/0:0] DEBUG: shmem_exit(0): 7 on_shmem_exit callbacks to make
2026-11-14 07:54:44.741 EST [1110667][client backend][:0] DEBUG: proc_exit(0): 3 callbacks to make
2026-11-14 07:54:44.741 EST [1110667][client backend][:0] LOG: disconnection: session time: 0:00:00.002 ...
2026-11-14 07:54:44.741 EST [1110667][client backend][:0] DEBUG: exit(0)
2026-11-14 07:54:44.741 EST [1110667][client backend][:0] DEBUG: shmem_exit(-1): 0 before_shmem_exit callbacks to make
2026-11-14 07:54:44.741 EST [1110667][client backend][:0] DEBUG: shmem_exit(-1): 0 on_shmem_exit callbacks to make
2026-11-14 07:54:44.741 EST [1110667][client backend][:0] DEBUG: proc_exit(-1): 0 callbacks to make
2026-11-14 07:54:44.741 EST [1110037][postmaster][:0] DEBUG: reaping dead processes
2026-11-14 07:54:44.741 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.741 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 1110667, status: 0
2026-11-14 07:54:44.741 EST [1110037][postmaster][:0] DEBUG: releasing pm child slot 2
2026-11-14 07:54:44.741 EST [1110037][postmaster][:0] DEBUG: client backend (PID 1110667) exited with exit code 0
2026-11-14 07:54:44.741 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.741 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 0, status: 0
2026-11-14 07:54:44.749 EST [1110041][checkpointer][:0] DEBUG: attempting to remove WAL segments older than log file 000000000000000000000002
2026-11-14 07:54:44.749 EST [1110041][checkpointer][:0] DEBUG: recycled write-ahead log file "000000010000000000000002"
2026-11-14 07:54:44.749 EST [1110041][checkpointer][:0] DEBUG: SlruScanDirectory invoking callback on pg_subtrans/0000
2026-11-14 07:54:44.749 EST [1110041][checkpointer][:0] LOG: restartpoint complete: fast wait: wrote 38 buffers (29.7%), wrote 2 SLRU buffers; 0 WAL file(s) added, 0 removed, 1 recycled; write=0.001 s, sync=0.001 s, total=0.528 s; sync files=0, longest=0.000 s, average=0.000 s; distance=16384 kB, estimate=16384 kB; lsn=0/03000088, redo lsn=0/03000028
2026-11-14 07:54:44.749 EST [1110041][checkpointer][:0] LOG: recovery restart point at 0/03000028
2026-11-14 07:54:44.749 EST [1110041][checkpointer][:0] DETAIL: Last completed transaction was at log time 2026-11-14 07:54:44.226403-05.
2026-11-14 07:54:44.749 EST [1110041][checkpointer][:0] LOG: checkpoint starting: force
2026-11-14 07:54:44.843 EST [1110037][postmaster][:0] DEBUG: assigned pm child slot 2 for client backend
2026-11-14 07:54:44.843 EST [1110037][postmaster][:0] DEBUG: forked new client backend, pid=1110740 socket=8
2026-11-14 07:54:44.843 EST [1110740][client backend][:0] LOG: connection received: host=[local]
2026-11-14 07:54:44.843 EST [1110740][client backend][6/0:0] DEBUG: InitPostgres
2026-11-14 07:54:44.844 EST [1110740][client backend][6/1:0] LOG: connection authenticated: ...
2026-11-14 07:54:44.844 EST [1110740][client backend][6/1:0] LOG: connection authorized: ...
2026-11-14 07:54:44.844 EST [1110740][client backend][6/2:0] LOG: statement: SELECT pg_backend_pid();
2026-11-14 07:54:44.844 EST [1110740][client backend][6/2:0] DEBUG: io 6 |op invalid|target invalid|state HANDED_OUT : adding cb #1, id 2/aio_shared_buffer_readv_cb at character 8
2026-11-14 07:54:44.844 EST [1110740][client backend][6/2:0] DEBUG: io 6 |op invalid|target smgr|state HANDED_OUT : adding cb #2, id 1/aio_md_readv_cb at character 8
2026-11-14 07:54:44.844 EST [1110740][client backend][6/2:0] DEBUG: io 6 |op readv|target smgr|state DEFINED : calling cb #1 2/aio_shared_buffer_readv_cb->stage(8) at character 8
2026-11-14 07:54:44.844 EST [1110740][client backend][6/2:0] DEBUG: io 6 |op readv|target smgr|state STAGED : staged (synchronous: 1, in_batch: 0) at character 8
2026-11-14 07:54:44.844 EST [1110740][client backend][6/2:0] DEBUG: io 6 |op readv|target smgr|state COMPLETED_IO : calling cb #2, id 1/aio_md_readv_cb->complete_shared(0) with distilled result: (status OK, id 0, error_data 0, result 8192) at character 8
2026-11-14 07:54:44.844 EST [1110740][client backend][6/2:0] DEBUG: io 6 |op readv|target smgr|state COMPLETED_IO : calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_shared(8) with distilled result: (status OK, id 0, error_data 0, result 1) at character 8
2026-11-14 07:54:44.844 EST [1110740][client backend][6/2:0] DEBUG: io 6 |op readv|target smgr|state COMPLETED_IO : after shared completion: distilled result: (status OK, id 0, error_data: 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.844 EST [1110740][client backend][6/2:0] DEBUG: io 6 |op readv|target smgr|state COMPLETED_SHARED: calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_local(8) with distilled result: status OK, id 0, error_data 0, result 1 at character 8
2026-11-14 07:54:44.844 EST [1110740][client backend][6/2:0] DEBUG: io 6 |op readv|target smgr|state COMPLETED_SHARED: after local completion: result: (status OK, id 0, error_data 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.844 EST [1110740][client backend][6/2:0] DEBUG: io 6 |op readv|target smgr|state COMPLETED_LOCAL : reclaiming: distilled_result: (status OK, id 0, error_data 0), raw_result: 8192 at character 8
2026-11-14 07:54:44.845 EST [1110037][postmaster][:0] DEBUG: reaping dead processes
2026-11-14 07:54:44.845 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.845 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 1110740, status: 9
2026-11-14 07:54:44.845 EST [1110037][postmaster][:0] DEBUG: releasing pm child slot 2
2026-11-14 07:54:44.845 EST [1110037][postmaster][:0] LOG: client backend (PID 1110740) was terminated by signal 9: Killed
2026-11-14 07:54:44.845 EST [1110037][postmaster][:0] DETAIL: Failed process was running: SELECT pg_backend_pid();
2026-11-14 07:54:44.845 EST [1110037][postmaster][:0] LOG: terminating any other active server processes
2026-11-14 07:54:44.845 EST [1110037][postmaster][:0] DEBUG: sending signal 3/SIGQUIT to client backend process with pid 1110383
2026-11-14 07:54:44.845 EST [1110037][postmaster][:0] DEBUG: sending signal 3/SIGQUIT to startup process with pid 1110043
2026-11-14 07:54:44.849 EST [1110041][checkpointer][:0] DEBUG: performing replication slot checkpoint
2026-11-14 07:54:44.849 EST [1110041][checkpointer][:0] DEBUG: snapshot of 0+0 running transaction ids (lsn 0/04000148 oldest xid 667 latest complete 666 next xid 667)
2026-11-14 07:54:44.849 EST [1110041][checkpointer][:0] DEBUG: attempting to remove WAL segments older than log file 000000000000000000000003
2026-11-14 07:54:44.849 EST [1110041][checkpointer][:0] DEBUG: recycled write-ahead log file "000000010000000000000003"
2026-11-14 07:54:44.849 EST [1110041][checkpointer][:0] DEBUG: SlruScanDirectory invoking callback on pg_subtrans/0000
2026-11-14 07:54:44.849 EST [1110041][checkpointer][:0] LOG: checkpoint complete: force: wrote 1 buffers (0.8%), wrote 2 SLRU buffers; 0 WAL file(s) added, 0 removed, 1 recycled; write=0.101 s, sync=0.001 s, total=0.101 s; sync files=0, longest=0.000 s, average=0.000 s; distance=16384 kB, estimate=16384 kB; lsn=0/04000148, redo lsn=0/040000E8
2026-11-14 07:54:44.900 EST [1110043][startup][:0] LOG: !!!proc_exit| delayed exit(0) for startup process
2026-11-14 07:54:44.900 EST [1110043][startup][:0] DEBUG: shmem_exit(-1): 0 before_shmem_exit callbacks to make
2026-11-14 07:54:44.900 EST [1110043][startup][:0] DEBUG: shmem_exit(-1): 0 on_shmem_exit callbacks to make
2026-11-14 07:54:44.900 EST [1110043][startup][:0] DEBUG: proc_exit(-1): 0 callbacks to make
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] LOG: !!!signal_child| delayed signal 3/SIGQUIT to startup process with pid 1110043
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] DEBUG: sending signal 3/SIGQUIT to background writer process with pid 1110042
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] DEBUG: sending signal 3/SIGQUIT to checkpointer process with pid 1110041
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] DEBUG: sending signal 3/SIGQUIT to io worker process with pid 1110040
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] DEBUG: sending signal 3/SIGQUIT to io worker process with pid 1110039
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] DEBUG: updating PMState from PM_HOT_STANDBY to PM_WAIT_BACKENDS
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 1110383, status: 512
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] DEBUG: releasing pm child slot 1
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 0, status: 512
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] DEBUG: startup process 1110043 is still running
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] DEBUG: background writer process 1110042 is still running
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] DEBUG: checkpointer process 1110041 is still running
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] DEBUG: io worker process 1110040 is still running
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] DEBUG: io worker process 1110039 is still running
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] DEBUG: reaping dead processes
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 0, status: 43690
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] DEBUG: startup process 1110043 is still running
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] DEBUG: background writer process 1110042 is still running
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] DEBUG: checkpointer process 1110041 is still running
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] DEBUG: io worker process 1110040 is still running
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] DEBUG: io worker process 1110039 is still running
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] DEBUG: reaping dead processes
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 1110040, status: 512
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] DEBUG: releasing pm child slot 58
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 0, status: 512
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] DEBUG: startup process 1110043 is still running
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] DEBUG: background writer process 1110042 is still running
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] DEBUG: checkpointer process 1110041 is still running
2026-11-14 07:54:44.900 EST [1110037][postmaster][:0] DEBUG: io worker process 1110039 is still running
2026-11-14 07:54:44.902 EST [1110037][postmaster][:0] DEBUG: reaping dead processes
2026-11-14 07:54:44.902 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.902 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 1110039, status: 512
2026-11-14 07:54:44.902 EST [1110037][postmaster][:0] DEBUG: releasing pm child slot 57
2026-11-14 07:54:44.902 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.902 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 1110041, status: 512
2026-11-14 07:54:44.902 EST [1110037][postmaster][:0] DEBUG: releasing pm child slot 56
2026-11-14 07:54:44.902 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.902 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 1110042, status: 512
2026-11-14 07:54:44.902 EST [1110037][postmaster][:0] DEBUG: releasing pm child slot 55
2026-11-14 07:54:44.902 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.902 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 0, status: 512
2026-11-14 07:54:44.902 EST [1110037][postmaster][:0] DEBUG: startup process 1110043 is still running
2026-11-14 07:54:44.903 EST [1110037][postmaster][:0] DEBUG: reaping dead processes
2026-11-14 07:54:44.903 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.903 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 1110043, status: 0
2026-11-14 07:54:44.903 EST [1110037][postmaster][:0] DEBUG: releasing pm child slot 89
2026-11-14 07:54:44.903 EST [1110037][postmaster][:0] DEBUG: updating PMState from PM_WAIT_BACKENDS to PM_RUN
2026-11-14 07:54:44.903 EST [1110037][postmaster][:0] LOG: database system is ready to accept connections
2026-11-14 07:54:44.903 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.903 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: -1, status: 0
2026-11-14 07:54:44.903 EST [1110037][postmaster][:0] DEBUG: assigned pm child slot 57 for io worker
2026-11-14 07:54:44.903 EST [1110037][postmaster][:0] DEBUG: assigned pm child slot 58 for io worker
2026-11-14 07:54:44.903 EST [1110037][postmaster][:0] DEBUG: assigned pm child slot 56 for checkpointer
2026-11-14 07:54:44.903 EST [1110037][postmaster][:0] DEBUG: assigned pm child slot 55 for background writer
2026-11-14 07:54:44.904 EST [1110037][postmaster][:0] DEBUG: assigned pm child slot 92 for walwriter
2026-11-14 07:54:44.904 EST [1110037][postmaster][:0] DEBUG: assigned pm child slot 45 for background worker
2026-11-14 07:54:44.904 EST [1110037][postmaster][:0] DEBUG: starting background worker process "logical replication launcher"
2026-11-14 07:54:44.904 EST [1110788][checkpointer][:0] DEBUG: checkpointer updated shared memory configuration values
2026-11-14 07:54:44.904 EST [1110793][logical replication launcher][15/0:0] DEBUG: logical replication launcher started
2026-11-14 07:54:44.904 EST [1110793][logical replication launcher][15/0:0] DEBUG: InitPostgres
2026-11-14 07:55:00.050 EST [1110790][background writer][:0] DEBUG: snapshot of 0+0 running transaction ids (lsn 0/04000208 oldest xid 667 latest complete 666 next xid 667)
2026-11-14 07:55:44.967 EST [1110786][io worker][:0] DEBUG: shmem_exit(0): 4 before_shmem_exit callbacks to make
2026-11-14 07:55:44.967 EST [1110786][io worker][:0] DEBUG: shmem_exit(0): 7 on_shmem_exit callbacks to make
2026-11-14 07:55:44.967 EST [1110786][io worker][:0] DEBUG: proc_exit(0): 1 callbacks to make
2026-11-14 07:55:44.967 EST [1110786][io worker][:0] DEBUG: exit(0)
2026-11-14 07:55:44.967 EST [1110786][io worker][:0] DEBUG: shmem_exit(-1): 0 before_shmem_exit callbacks to make
2026-11-14 07:55:44.967 EST [1110786][io worker][:0] DEBUG: shmem_exit(-1): 0 on_shmem_exit callbacks to make
2026-11-14 07:55:44.967 EST [1110786][io worker][:0] DEBUG: proc_exit(-1): 0 callbacks to make
2026-11-14 07:55:44.968 EST [1110037][postmaster][:0] DEBUG: reaping dead processes
2026-11-14 07:55:44.968 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:55:44.968 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 1110786, status: 0
2026-11-14 07:55:44.968 EST [1110037][postmaster][:0] DEBUG: releasing pm child slot 58
2026-11-14 07:55:44.968 EST [1110037][postmaster][:0] DEBUG: assigned pm child slot 58 for io worker
2026-11-14 07:55:44.969 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:55:44.969 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 0, status: 0
2026-11-14 07:56:45.031 EST [1111045][io worker][:0] DEBUG: shmem_exit(0): 4 before_shmem_exit callbacks to make
2026-11-14 07:56:45.032 EST [1111045][io worker][:0] DEBUG: shmem_exit(0): 7 on_shmem_exit callbacks to make
2026-11-14 07:56:45.032 EST [1111045][io worker][:0] DEBUG: proc_exit(0): 1 callbacks to make
2026-11-14 07:56:45.032 EST [1111045][io worker][:0] DEBUG: exit(0)
2026-11-14 07:56:45.032 EST [1111045][io worker][:0] DEBUG: shmem_exit(-1): 0 before_shmem_exit callbacks to make
2026-11-14 07:56:45.032 EST [1111045][io worker][:0] DEBUG: shmem_exit(-1): 0 on_shmem_exit callbacks to make
2026-11-14 07:56:45.032 EST [1111045][io worker][:0] DEBUG: proc_exit(-1): 0 callbacks to make
2026-11-14 07:56:45.032 EST [1110037][postmaster][:0] DEBUG: reaping dead processes
2026-11-14 07:56:45.032 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:56:45.033 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 1111045, status: 0
2026-11-14 07:56:45.033 EST [1110037][postmaster][:0] DEBUG: releasing pm child slot 58
2026-11-14 07:56:45.033 EST [1110037][postmaster][:0] DEBUG: assigned pm child slot 58 for io worker
2026-11-14 07:56:45.033 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:56:45.033 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 0, status: 0
2026-11-14 07:57:45.098 EST [1111065][io worker][:0] DEBUG: shmem_exit(0): 4 before_shmem_exit callbacks to make
2026-11-14 07:57:45.098 EST [1111065][io worker][:0] DEBUG: shmem_exit(0): 7 on_shmem_exit callbacks to make
2026-11-14 07:57:45.098 EST [1111065][io worker][:0] DEBUG: proc_exit(0): 1 callbacks to make
2026-11-14 07:57:45.098 EST [1111065][io worker][:0] DEBUG: exit(0)
2026-11-14 07:57:45.098 EST [1111065][io worker][:0] DEBUG: shmem_exit(-1): 0 before_shmem_exit callbacks to make
2026-11-14 07:57:45.098 EST [1111065][io worker][:0] DEBUG: shmem_exit(-1): 0 on_shmem_exit callbacks to make
2026-11-14 07:57:45.098 EST [1111065][io worker][:0] DEBUG: proc_exit(-1): 0 callbacks to make
2026-11-14 07:57:45.099 EST [1110037][postmaster][:0] DEBUG: reaping dead processes
2026-11-14 07:57:45.099 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:57:45.099 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 1111065, status: 0
2026-11-14 07:57:45.099 EST [1110037][postmaster][:0] DEBUG: releasing pm child slot 58
2026-11-14 07:57:45.099 EST [1110037][postmaster][:0] DEBUG: assigned pm child slot 58 for io worker
2026-11-14 07:57:45.099 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:57:45.099 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 0, status: 0
2026-11-14 07:57:49.231 EST [1110037][postmaster][:0] DEBUG: postmaster received shutdown request signal
2026-11-14 07:57:49.231 EST [1110037][postmaster][:0] LOG: received immediate shutdown request
2026-11-14 07:57:49.231 EST [1110037][postmaster][:0] DEBUG: sending signal 3/SIGQUIT to io worker process with pid 1111082
2026-11-14 07:57:49.231 EST [1110037][postmaster][:0] DEBUG: sending signal 3/SIGQUIT to background worker process with pid 1110793
2026-11-14 07:57:49.231 EST [1110037][postmaster][:0] DEBUG: sending signal 3/SIGQUIT to walwriter process with pid 1110791
2026-11-14 07:57:49.231 EST [1110037][postmaster][:0] DEBUG: sending signal 3/SIGQUIT to background writer process with pid 1110790
2026-11-14 07:57:49.231 EST [1110037][postmaster][:0] DEBUG: sending signal 3/SIGQUIT to checkpointer process with pid 1110788
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] DEBUG: sending signal 3/SIGQUIT to io worker process with pid 1110784
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] DEBUG: updating PMState from PM_RUN to PM_WAIT_BACKENDS
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] DEBUG: io worker process 1111082 is still running
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] DEBUG: background worker process 1110793 is still running
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] DEBUG: walwriter process 1110791 is still running
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] DEBUG: background writer process 1110790 is still running
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] DEBUG: checkpointer process 1110788 is still running
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] DEBUG: io worker process 1110784 is still running
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] DEBUG: reaping dead processes
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 1110790, status: 512
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] DEBUG: releasing pm child slot 55
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 1110788, status: 512
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] DEBUG: releasing pm child slot 56
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 1110791, status: 512
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] DEBUG: releasing pm child slot 92
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 1110784, status: 512
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] DEBUG: releasing pm child slot 57
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 1110793, status: 512
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] DEBUG: releasing pm child slot 45
2026-11-14 07:57:49.232 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:57:49.233 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: 1111082, status: 512
2026-11-14 07:57:49.233 EST [1110037][postmaster][:0] DEBUG: releasing pm child slot 58
2026-11-14 07:57:49.233 EST [1110037][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:57:49.233 EST [1110037][postmaster][:0] LOG: !!!waitpid| ret: -1, status: 512
2026-11-14 07:57:49.233 EST [1110037][postmaster][:0] DEBUG: updating PMState from PM_WAIT_BACKENDS to PM_WAIT_DEAD_END
2026-11-14 07:57:49.233 EST [1110037][postmaster][:0] DEBUG: updating PMState from PM_WAIT_DEAD_END to PM_NO_CHILDREN
2026-11-14 07:57:49.233 EST [1110037][postmaster][:0] DEBUG: shmem_exit(0): 0 before_shmem_exit callbacks to make
2026-11-14 07:57:49.233 EST [1110037][postmaster][:0] DEBUG: shmem_exit(0): 5 on_shmem_exit callbacks to make
2026-11-14 07:57:49.233 EST [1110037][postmaster][:0] DEBUG: cleaning up orphaned dynamic shared memory with ID 1459897042
2026-11-14 07:57:49.233 EST [1110037][postmaster][:0] DEBUG: cleaning up orphaned dynamic shared memory with ID 3069910578
2026-11-14 07:57:49.233 EST [1110037][postmaster][:0] DEBUG: cleaning up orphaned dynamic shared memory with ID 3859456016
2026-11-14 07:57:49.233 EST [1110037][postmaster][:0] DEBUG: cleaning up dynamic shared memory control segment with ID 2869141000
2026-11-14 07:57:49.234 EST [1110037][postmaster][:0] DEBUG: proc_exit(0): 2 callbacks to make
2026-11-14 07:57:49.234 EST [1110037][postmaster][:0] LOG: database system is shut down
2026-11-14 07:57:49.234 EST [1110037][postmaster][:0] DEBUG: exit(0)
2026-11-14 07:57:49.234 EST [1110037][postmaster][:0] DEBUG: shmem_exit(-1): 0 before_shmem_exit callbacks to make
2026-11-14 07:57:49.234 EST [1110037][postmaster][:0] DEBUG: shmem_exit(-1): 0 on_shmem_exit callbacks to make
2026-11-14 07:57:49.234 EST [1110037][postmaster][:0] DEBUG: proc_exit(-1): 0 callbacks to make
041_checkpoint_at_promote_standby1-1.log (text/x-log, 71.0K) download041_checkpoint_at_promote_standby1-0.log (text/x-log, 44.4K) download
...
2026-11-14 07:54:44.154 EST [1109760][startup][37/0:0] LOG: received promote request
2026-11-14 07:54:44.154 EST [1109760][startup][37/0:0] LOG: redo done at 0/04000028 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.33 s
2026-11-14 07:54:44.154 EST [1109760][startup][37/0:0] LOG: last completed transaction was at log time 2026-11-14 07:54:44.016107-05
2026-11-14 07:54:44.154 EST [1109760][startup][37/0:0] DEBUG: resetting unlogged relations: cleanup 0 init 1
2026-11-14 07:54:44.154 EST [1109760][startup][37/0:0] LOG: selected new timeline ID: 2
2026-11-14 07:54:44.155 EST [1109760][startup][37/0:0] DEBUG: updated min recovery point to 0/040000B0 on timeline 1
2026-11-14 07:54:44.172 EST [1109760][startup][37/0:0] DEBUG: could not remove file "pg_wal/000000020000000000000004": No such file or directory
2026-11-14 07:54:44.172 EST [1109760][startup][37/0:0] LOG: archive recovery complete
2026-11-14 07:54:44.172 EST [1109760][startup][37/0:0] DEBUG: MultiXactId wrap limit is 2147483648, limited by database with OID 1
2026-11-14 07:54:44.172 EST [1109760][startup][37/0:0] DEBUG: attempting to remove WAL segments newer than log file 000000020000000000000004
2026-11-14 07:54:44.172 EST [1109760][startup][37/0:0] DEBUG: removing all KnownAssignedXids
2026-11-14 07:54:44.172 EST [1109760][startup][37/0:0] DEBUG: release all standby locks
2026-11-14 07:54:44.172 EST [1109760][startup][37/0:0] DEBUG: shmem_exit(0): 5 before_shmem_exit callbacks to make
2026-11-14 07:54:44.172 EST [1109760][startup][37/0:0] DEBUG: shmem_exit(0): 8 on_shmem_exit callbacks to make
2026-11-14 07:54:44.172 EST [1109760][startup][:0] DEBUG: proc_exit(0): 1 callbacks to make
2026-11-14 07:54:44.172 EST [1109760][startup][:0] DEBUG: exit(0)
2026-11-14 07:54:44.256 EST [1109746][postmaster][:0] DEBUG: assigned pm child slot 2 for client backend
2026-11-14 07:54:44.256 EST [1109746][postmaster][:0] DEBUG: forked new client backend, pid=1110418 socket=8
2026-11-14 07:54:44.256 EST [1110418][client backend][:0] LOG: connection received: host=[local]
2026-11-14 07:54:44.257 EST [1110418][client backend][5/0:0] DEBUG: InitPostgres
2026-11-14 07:54:44.257 EST [1110418][client backend][5/1:0] LOG: connection authenticated: ...
2026-11-14 07:54:44.257 EST [1110418][client backend][5/1:0] LOG: connection authorized: ...
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] LOG: statement: SELECT injection_points_wakeup('create-restart-point');
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op invalid|target invalid|state HANDED_OUT : adding cb #1, id 2/aio_shared_buffer_readv_cb at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op invalid|target smgr|state HANDED_OUT : adding cb #2, id 1/aio_md_readv_cb at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state DEFINED : calling cb #1 2/aio_shared_buffer_readv_cb->stage(8) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state STAGED : staged (synchronous: 1, in_batch: 0) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #2, id 1/aio_md_readv_cb->complete_shared(0) with distilled result: (status OK, id 0, error_data 0, result 8192) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_shared(8) with distilled result: (status OK, id 0, error_data 0, result 1) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : after shared completion: distilled result: (status OK, id 0, error_data: 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_local(8) with distilled result: status OK, id 0, error_data 0, result 1 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: after local completion: result: (status OK, id 0, error_data 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_LOCAL : reclaiming: distilled_result: (status OK, id 0, error_data 0), raw_result: 8192 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op invalid|target invalid|state HANDED_OUT : adding cb #1, id 2/aio_shared_buffer_readv_cb at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op invalid|target smgr|state HANDED_OUT : adding cb #2, id 1/aio_md_readv_cb at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state DEFINED : calling cb #1 2/aio_shared_buffer_readv_cb->stage(8) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state STAGED : staged (synchronous: 1, in_batch: 0) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #2, id 1/aio_md_readv_cb->complete_shared(0) with distilled result: (status OK, id 0, error_data 0, result 8192) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_shared(8) with distilled result: (status OK, id 0, error_data 0, result 1) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : after shared completion: distilled result: (status OK, id 0, error_data: 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_local(8) with distilled result: status OK, id 0, error_data 0, result 1 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: after local completion: result: (status OK, id 0, error_data 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_LOCAL : reclaiming: distilled_result: (status OK, id 0, error_data 0), raw_result: 8192 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op invalid|target invalid|state HANDED_OUT : adding cb #1, id 2/aio_shared_buffer_readv_cb at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op invalid|target smgr|state HANDED_OUT : adding cb #2, id 1/aio_md_readv_cb at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state DEFINED : calling cb #1 2/aio_shared_buffer_readv_cb->stage(8) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state STAGED : staged (synchronous: 1, in_batch: 0) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #2, id 1/aio_md_readv_cb->complete_shared(0) with distilled result: (status OK, id 0, error_data 0, result 8192) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_shared(8) with distilled result: (status OK, id 0, error_data 0, result 1) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : after shared completion: distilled result: (status OK, id 0, error_data: 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_local(8) with distilled result: status OK, id 0, error_data 0, result 1 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: after local completion: result: (status OK, id 0, error_data 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_LOCAL : reclaiming: distilled_result: (status OK, id 0, error_data 0), raw_result: 8192 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op invalid|target invalid|state HANDED_OUT : adding cb #1, id 2/aio_shared_buffer_readv_cb at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op invalid|target smgr|state HANDED_OUT : adding cb #2, id 1/aio_md_readv_cb at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state DEFINED : calling cb #1 2/aio_shared_buffer_readv_cb->stage(8) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state STAGED : staged (synchronous: 1, in_batch: 0) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #2, id 1/aio_md_readv_cb->complete_shared(0) with distilled result: (status OK, id 0, error_data 0, result 8192) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_shared(8) with distilled result: (status OK, id 0, error_data 0, result 1) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : after shared completion: distilled result: (status OK, id 0, error_data: 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_local(8) with distilled result: status OK, id 0, error_data 0, result 1 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: after local completion: result: (status OK, id 0, error_data 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_LOCAL : reclaiming: distilled_result: (status OK, id 0, error_data 0), raw_result: 8192 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op invalid|target invalid|state HANDED_OUT : adding cb #1, id 2/aio_shared_buffer_readv_cb at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op invalid|target smgr|state HANDED_OUT : adding cb #2, id 1/aio_md_readv_cb at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state DEFINED : calling cb #1 2/aio_shared_buffer_readv_cb->stage(8) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state STAGED : staged (synchronous: 1, in_batch: 0) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #2, id 1/aio_md_readv_cb->complete_shared(0) with distilled result: (status OK, id 0, error_data 0, result 8192) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_shared(8) with distilled result: (status OK, id 0, error_data 0, result 1) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : after shared completion: distilled result: (status OK, id 0, error_data: 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_local(8) with distilled result: status OK, id 0, error_data 0, result 1 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: after local completion: result: (status OK, id 0, error_data 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_LOCAL : reclaiming: distilled_result: (status OK, id 0, error_data 0), raw_result: 8192 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op invalid|target invalid|state HANDED_OUT : adding cb #1, id 2/aio_shared_buffer_readv_cb at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op invalid|target smgr|state HANDED_OUT : adding cb #2, id 1/aio_md_readv_cb at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state DEFINED : calling cb #1 2/aio_shared_buffer_readv_cb->stage(8) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state STAGED : staged (synchronous: 1, in_batch: 0) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #2, id 1/aio_md_readv_cb->complete_shared(0) with distilled result: (status OK, id 0, error_data 0, result 8192) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_shared(8) with distilled result: (status OK, id 0, error_data 0, result 1) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : after shared completion: distilled result: (status OK, id 0, error_data: 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_local(8) with distilled result: status OK, id 0, error_data 0, result 1 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: after local completion: result: (status OK, id 0, error_data 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_LOCAL : reclaiming: distilled_result: (status OK, id 0, error_data 0), raw_result: 8192 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op invalid|target invalid|state HANDED_OUT : adding cb #1, id 2/aio_shared_buffer_readv_cb at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op invalid|target smgr|state HANDED_OUT : adding cb #2, id 1/aio_md_readv_cb at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state DEFINED : calling cb #1 2/aio_shared_buffer_readv_cb->stage(8) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state STAGED : staged (synchronous: 1, in_batch: 0) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #2, id 1/aio_md_readv_cb->complete_shared(0) with distilled result: (status OK, id 0, error_data 0, result 8192) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_shared(8) with distilled result: (status OK, id 0, error_data 0, result 1) at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : after shared completion: distilled result: (status OK, id 0, error_data: 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_local(8) with distilled result: status OK, id 0, error_data 0, result 1 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: after local completion: result: (status OK, id 0, error_data 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_LOCAL : reclaiming: distilled_result: (status OK, id 0, error_data 0), raw_result: 8192 at character 8
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op invalid|target invalid|state HANDED_OUT : adding cb #1, id 2/aio_shared_buffer_readv_cb
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op invalid|target smgr|state HANDED_OUT : adding cb #2, id 1/aio_md_readv_cb
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state DEFINED : calling cb #1 2/aio_shared_buffer_readv_cb->stage(8)
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state STAGED : staged (synchronous: 1, in_batch: 0)
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #2, id 1/aio_md_readv_cb->complete_shared(0) with distilled result: (status OK, id 0, error_data 0, result 8192)
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_shared(8) with distilled result: (status OK, id 0, error_data 0, result 1)
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_IO : after shared completion: distilled result: (status OK, id 0, error_data: 0, result 1), raw_result: 8192
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_local(8) with distilled result: status OK, id 0, error_data 0, result 1
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_SHARED: after local completion: result: (status OK, id 0, error_data 0, result 1), raw_result: 8192
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: io 5 |op readv|target smgr|state COMPLETED_LOCAL : reclaiming: distilled_result: (status OK, id 0, error_data 0), raw_result: 8192
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: find_in_path: trying ".../tmp_install/usr/local/pgsql/lib/injection_points"
2026-11-14 07:54:44.258 EST [1110418][client backend][5/2:0] DEBUG: find_in_path: trying ".../tmp_install/usr/local/pgsql/lib/injection_points.so"
2026-11-14 07:54:44.259 EST [1110418][client backend][5/0:0] DEBUG: shmem_exit(0): 5 before_shmem_exit callbacks to make
2026-11-14 07:54:44.259 EST [1110418][client backend][5/0:0] DEBUG: shmem_exit(0): 7 on_shmem_exit callbacks to make
2026-11-14 07:54:44.259 EST [1110418][client backend][:0] DEBUG: proc_exit(0): 3 callbacks to make
2026-11-14 07:54:44.259 EST [1110418][client backend][:0] LOG: disconnection: session time: 0:00:00.002 ...
2026-11-14 07:54:44.259 EST [1110418][client backend][:0] DEBUG: exit(0)
2026-11-14 07:54:44.259 EST [1110418][client backend][:0] DEBUG: shmem_exit(-1): 0 before_shmem_exit callbacks to make
2026-11-14 07:54:44.259 EST [1110418][client backend][:0] DEBUG: shmem_exit(-1): 0 on_shmem_exit callbacks to make
2026-11-14 07:54:44.259 EST [1110418][client backend][:0] DEBUG: proc_exit(-1): 0 callbacks to make
2026-11-14 07:54:44.260 EST [1109746][postmaster][:0] DEBUG: reaping dead processes
2026-11-14 07:54:44.260 EST [1109746][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.260 EST [1109746][postmaster][:0] LOG: !!!waitpid| ret: 1110418, status: 0
2026-11-14 07:54:44.260 EST [1109746][postmaster][:0] DEBUG: releasing pm child slot 2
2026-11-14 07:54:44.260 EST [1109746][postmaster][:0] DEBUG: client backend (PID 1110418) exited with exit code 0
2026-11-14 07:54:44.260 EST [1109746][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.260 EST [1109746][postmaster][:0] LOG: !!!waitpid| ret: 0, status: 0
2026-11-14 07:54:44.273 EST [1109758][checkpointer][:0] DEBUG: attempting to remove WAL segments older than log file 000000000000000000000002
2026-11-14 07:54:44.273 EST [1109758][checkpointer][:0] DEBUG: recycled write-ahead log file "000000010000000000000002"
2026-11-14 07:54:44.273 EST [1109758][checkpointer][:0] DEBUG: SlruScanDirectory invoking callback on pg_subtrans/0000
2026-11-14 07:54:44.273 EST [1109758][checkpointer][:0] LOG: restartpoint complete: fast wait: wrote 16 buffers (12.5%), wrote 2 SLRU buffers; 0 WAL file(s) added, 0 removed, 1 recycled; write=0.001 s, sync=0.001 s, total=0.261 s; sync files=0, longest=0.000 s, average=0.000 s; distance=16384 kB, estimate=16384 kB; lsn=0/03000088, redo lsn=0/03000028
2026-11-14 07:54:44.273 EST [1109758][checkpointer][:0] LOG: recovery restart point at 0/03000028
2026-11-14 07:54:44.273 EST [1109758][checkpointer][:0] DETAIL: Last completed transaction was at log time 2026-11-14 07:54:44.016107-05.
2026-11-14 07:54:44.273 EST [1109758][checkpointer][:0] LOG: checkpoint starting: force
2026-11-14 07:54:44.361 EST [1109746][postmaster][:0] DEBUG: assigned pm child slot 2 for client backend
2026-11-14 07:54:44.361 EST [1109746][postmaster][:0] DEBUG: forked new client backend, pid=1110482 socket=8
2026-11-14 07:54:44.361 EST [1110482][client backend][:0] LOG: connection received: host=[local]
2026-11-14 07:54:44.362 EST [1110482][client backend][6/0:0] DEBUG: InitPostgres
2026-11-14 07:54:44.362 EST [1110482][client backend][6/1:0] LOG: connection authenticated: ...
2026-11-14 07:54:44.362 EST [1110482][client backend][6/1:0] LOG: connection authorized: ...
2026-11-14 07:54:44.362 EST [1110482][client backend][6/2:0] LOG: statement: SELECT pg_backend_pid();
2026-11-14 07:54:44.363 EST [1110482][client backend][6/2:0] DEBUG: io 6 |op invalid|target invalid|state HANDED_OUT : adding cb #1, id 2/aio_shared_buffer_readv_cb at character 8
2026-11-14 07:54:44.363 EST [1110482][client backend][6/2:0] DEBUG: io 6 |op invalid|target smgr|state HANDED_OUT : adding cb #2, id 1/aio_md_readv_cb at character 8
2026-11-14 07:54:44.363 EST [1110482][client backend][6/2:0] DEBUG: io 6 |op readv|target smgr|state DEFINED : calling cb #1 2/aio_shared_buffer_readv_cb->stage(8) at character 8
2026-11-14 07:54:44.363 EST [1110482][client backend][6/2:0] DEBUG: io 6 |op readv|target smgr|state STAGED : staged (synchronous: 1, in_batch: 0) at character 8
2026-11-14 07:54:44.363 EST [1110482][client backend][6/2:0] DEBUG: io 6 |op readv|target smgr|state COMPLETED_IO : calling cb #2, id 1/aio_md_readv_cb->complete_shared(0) with distilled result: (status OK, id 0, error_data 0, result 8192) at character 8
2026-11-14 07:54:44.363 EST [1110482][client backend][6/2:0] DEBUG: io 6 |op readv|target smgr|state COMPLETED_IO : calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_shared(8) with distilled result: (status OK, id 0, error_data 0, result 1) at character 8
2026-11-14 07:54:44.363 EST [1110482][client backend][6/2:0] DEBUG: io 6 |op readv|target smgr|state COMPLETED_IO : after shared completion: distilled result: (status OK, id 0, error_data: 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.363 EST [1110482][client backend][6/2:0] DEBUG: io 6 |op readv|target smgr|state COMPLETED_SHARED: calling cb #1, id 2/aio_shared_buffer_readv_cb->complete_local(8) with distilled result: status OK, id 0, error_data 0, result 1 at character 8
2026-11-14 07:54:44.363 EST [1110482][client backend][6/2:0] DEBUG: io 6 |op readv|target smgr|state COMPLETED_SHARED: after local completion: result: (status OK, id 0, error_data 0, result 1), raw_result: 8192 at character 8
2026-11-14 07:54:44.363 EST [1110482][client backend][6/2:0] DEBUG: io 6 |op readv|target smgr|state COMPLETED_LOCAL : reclaiming: distilled_result: (status OK, id 0, error_data 0), raw_result: 8192 at character 8
2026-11-14 07:54:44.364 EST [1109746][postmaster][:0] DEBUG: reaping dead processes
2026-11-14 07:54:44.364 EST [1109746][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.364 EST [1109746][postmaster][:0] LOG: !!!waitpid| ret: 1110482, status: 9
2026-11-14 07:54:44.364 EST [1109746][postmaster][:0] DEBUG: releasing pm child slot 2
2026-11-14 07:54:44.364 EST [1109746][postmaster][:0] LOG: client backend (PID 1110482) was terminated by signal 9: Killed
2026-11-14 07:54:44.364 EST [1109746][postmaster][:0] DETAIL: Failed process was running: SELECT pg_backend_pid();
2026-11-14 07:54:44.364 EST [1109746][postmaster][:0] LOG: terminating any other active server processes
2026-11-14 07:54:44.364 EST [1109746][postmaster][:0] DEBUG: sending signal 3/SIGQUIT to client backend process with pid 1109914
2026-11-14 07:54:44.364 EST [1109746][postmaster][:0] DEBUG: sending signal 3/SIGQUIT to startup process with pid 1109760
2026-11-14 07:54:44.375 EST [1109758][checkpointer][:0] DEBUG: performing replication slot checkpoint
2026-11-14 07:54:44.375 EST [1109758][checkpointer][:0] DEBUG: snapshot of 0+0 running transaction ids (lsn 0/04000148 oldest xid 667 latest complete 666 next xid 667)
2026-11-14 07:54:44.375 EST [1109758][checkpointer][:0] DEBUG: attempting to remove WAL segments older than log file 000000000000000000000003
2026-11-14 07:54:44.375 EST [1109758][checkpointer][:0] DEBUG: recycled write-ahead log file "000000010000000000000003"
2026-11-14 07:54:44.375 EST [1109758][checkpointer][:0] DEBUG: SlruScanDirectory invoking callback on pg_subtrans/0000
2026-11-14 07:54:44.375 EST [1109758][checkpointer][:0] LOG: checkpoint complete: force: wrote 1 buffers (0.8%), wrote 2 SLRU buffers; 0 WAL file(s) added, 0 removed, 1 recycled; write=0.101 s, sync=0.001 s, total=0.103 s; sync files=0, longest=0.000 s, average=0.000 s; distance=16384 kB, estimate=16384 kB; lsn=0/04000148, redo lsn=0/040000E8
2026-11-14 07:54:44.400 EST [1109746][postmaster][:0] LOG: !!!signal_child| delayed signal 3/SIGQUIT to startup process with pid 1109760
2026-11-14 07:54:44.500 EST [1109760][startup][:0] LOG: !!!proc_exit| delayed exit(0) for startup process
2026-11-14 07:54:44.539 EST [1109746][postmaster][:0] DEBUG: sending signal 3/SIGQUIT to background writer process with pid 1109759
2026-11-14 07:54:44.539 EST [1109746][postmaster][:0] DEBUG: sending signal 3/SIGQUIT to checkpointer process with pid 1109758
2026-11-14 07:54:44.539 EST [1109746][postmaster][:0] DEBUG: sending signal 3/SIGQUIT to io worker process with pid 1109757
2026-11-14 07:54:44.539 EST [1109746][postmaster][:0] DEBUG: sending signal 3/SIGQUIT to io worker process with pid 1109756
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] DEBUG: updating PMState from PM_HOT_STANDBY to PM_WAIT_BACKENDS
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] LOG: !!!waitpid| ret: 1109756, status: 512
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] DEBUG: releasing pm child slot 57
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] LOG: !!!waitpid| ret: 1109757, status: 512
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] DEBUG: releasing pm child slot 58
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] LOG: !!!waitpid| ret: 1109758, status: 512
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] DEBUG: releasing pm child slot 56
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] LOG: !!!waitpid| ret: 1109759, status: 512
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] DEBUG: releasing pm child slot 55
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] LOG: !!!waitpid| ret: 1109760, status: 512
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] DEBUG: releasing pm child slot 89
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] LOG: !!!waitpid| ret: 1109914, status: 512
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] DEBUG: releasing pm child slot 1
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] LOG: !!!waitpid| ret: -1, status: 512
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] DEBUG: updating PMState from PM_WAIT_BACKENDS to PM_WAIT_DEAD_END
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] DEBUG: updating PMState from PM_WAIT_DEAD_END to PM_NO_CHILDREN
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] LOG: all server processes terminated; reinitializing
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] DEBUG: shmem_exit(1): 0 before_shmem_exit callbacks to make
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] DEBUG: shmem_exit(1): 5 on_shmem_exit callbacks to make
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] DEBUG: cleaning up orphaned dynamic shared memory with ID 924198510
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] DEBUG: cleaning up orphaned dynamic shared memory with ID 1616401164
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] DEBUG: cleaning up orphaned dynamic shared memory with ID 3295781356
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] DEBUG: cleaning up dynamic shared memory control segment with ID 2272285588
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] DEBUG: invoking IpcMemoryCreate(size=5873664)
2026-11-14 07:54:44.540 EST [1109746][postmaster][:0] DEBUG: mmap(6291456) with MAP_HUGETLB failed, huge pages disabled: Cannot allocate memory
2026-11-14 07:54:44.541 EST [1109746][postmaster][:0] DEBUG: dynamic shared memory system will support 229 segments
2026-11-14 07:54:44.541 EST [1109746][postmaster][:0] DEBUG: created dynamic shared memory control segment 4225595752 (9176 bytes)
2026-11-14 07:54:44.541 EST [1109746][postmaster][:0] DEBUG: updating PMState from PM_NO_CHILDREN to PM_STARTUP
2026-11-14 07:54:44.541 EST [1109746][postmaster][:0] DEBUG: assigned pm child slot 58 for io worker
2026-11-14 07:54:44.541 EST [1109746][postmaster][:0] DEBUG: assigned pm child slot 57 for io worker
2026-11-14 07:54:44.541 EST [1109746][postmaster][:0] DEBUG: assigned pm child slot 89 for startup
2026-11-14 07:54:44.541 EST [1109746][postmaster][:0] DEBUG: assigned pm child slot 56 for checkpointer
2026-11-14 07:54:44.541 EST [1109746][postmaster][:0] DEBUG: assigned pm child slot 55 for background writer
2026-11-14 07:54:44.542 EST [1109746][postmaster][:0] DEBUG: reaping dead processes
2026-11-14 07:54:44.542 EST [1109746][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.542 EST [1109746][postmaster][:0] LOG: !!!waitpid| ret: 0, status: 43690
2026-11-14 07:54:44.542 EST [1110541][startup][:0] LOG: database system was interrupted; last known up at 2026-11-14 07:54:44 EST
2026-11-14 07:54:44.542 EST [1110541][startup][:0] DEBUG: removing all temporary WAL segments
2026-11-14 07:54:44.542 EST [1110541][startup][:0] DEBUG: checkpoint record is at 0/04000148
2026-11-14 07:54:44.542 EST [1110541][startup][:0] DEBUG: redo record is at 0/040000E8; shutdown false
2026-11-14 07:54:44.542 EST [1110541][startup][:0] DEBUG: next transaction ID: 667; next OID: 16398
2026-11-14 07:54:44.542 EST [1110541][startup][:0] DEBUG: next MultiXactId: 1; next MultiXactOffset: 1
2026-11-14 07:54:44.542 EST [1110541][startup][:0] DEBUG: oldest unfrozen transaction ID: 656, in database 1
2026-11-14 07:54:44.542 EST [1110541][startup][:0] DEBUG: oldest MultiXactId: 1, in database 1
2026-11-14 07:54:44.542 EST [1110541][startup][:0] DEBUG: commit timestamp Xid oldest/newest: 0/0
2026-11-14 07:54:44.542 EST [1110541][startup][:0] LOG: database system was not properly shut down; automatic recovery in progress
2026-11-14 07:54:44.542 EST [1110541][startup][:0] DEBUG: transaction ID wrap limit is 2147484303, limited by database with OID 1
2026-11-14 07:54:44.542 EST [1110541][startup][:0] DEBUG: MultiXactId wrap limit is 2147483648, limited by database with OID 1
2026-11-14 07:54:44.542 EST [1110542][checkpointer][:0] DEBUG: checkpointer updated shared memory configuration values
2026-11-14 07:54:44.542 EST [1110541][startup][:0] DEBUG: starting up replication slots
2026-11-14 07:54:44.542 EST [1110541][startup][:0] DEBUG: xmin required by slots: data 0, catalog 0
2026-11-14 07:54:44.542 EST [1110541][startup][:0] DEBUG: starting up replication origin progress state
2026-11-14 07:54:44.542 EST [1110541][startup][:0] DEBUG: didn't need to unlink permanent stats file "pg_stat/pgstat.stat" - didn't exist
2026-11-14 07:54:44.542 EST [1110541][startup][:0] DEBUG: resetting unlogged relations: cleanup 1 init 0
2026-11-14 07:54:44.542 EST [1110541][startup][:0] LOG: redo starts at 0/040000E8
2026-11-14 07:54:44.542 EST [1110541][startup][:0] LOG: invalid record length at 0/040001D0: expected at least 24, got 0
2026-11-14 07:54:44.542 EST [1110541][startup][:0] LOG: redo done at 0/04000148 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
2026-11-14 07:54:44.542 EST [1110541][startup][:0] DEBUG: resetting unlogged relations: cleanup 0 init 1
2026-11-14 07:54:44.542 EST [1109746][postmaster][:0] DEBUG: postmaster received pmsignal signal
2026-11-14 07:54:44.542 EST [1109746][postmaster][:0] DEBUG: updating PMState from PM_STARTUP to PM_RECOVERY
2026-11-14 07:54:44.542 EST [1110541][startup][:0] DEBUG: MultiXactId wrap limit is 2147483648, limited by database with OID 1
2026-11-14 07:54:44.543 EST [1110542][checkpointer][:0] LOG: checkpoint starting: end-of-recovery fast wait
2026-11-14 07:54:44.543 EST [1110542][checkpointer][:0] DEBUG: performing replication slot checkpoint
2026-11-14 07:54:44.543 EST [1110542][checkpointer][:0] DEBUG: attempting to remove WAL segments older than log file 000000000000000000000003
2026-11-14 07:54:44.544 EST [1110542][checkpointer][:0] LOG: checkpoint complete: end-of-recovery fast wait: wrote 0 buffers (0.0%), wrote 3 SLRU buffers; 0 WAL file(s) added, 0 removed, 0 recycled; write=0.001 s, sync=0.001 s, total=0.001 s; sync files=0, longest=0.000 s, average=0.000 s; distance=0 kB, estimate=0 kB; lsn=0/040001D0, redo lsn=0/040001D0
2026-11-14 07:54:44.544 EST [1110541][startup][:0] DEBUG: shmem_exit(0): 5 before_shmem_exit callbacks to make
2026-11-14 07:54:44.544 EST [1110541][startup][:0] DEBUG: shmem_exit(0): 7 on_shmem_exit callbacks to make
2026-11-14 07:54:44.544 EST [1110541][startup][:0] DEBUG: proc_exit(0): 1 callbacks to make
2026-11-14 07:54:44.544 EST [1110541][startup][:0] DEBUG: exit(0)
2026-11-14 07:54:44.569 EST [1109746][postmaster][:0] DEBUG: allocating dead-end child
2026-11-14 07:54:44.569 EST [1109746][postmaster][:0] DEBUG: forked new dead-end client backend, pid=1110598 socket=8
2026-11-14 07:54:44.569 EST [1110598][dead-end client backend][:0] LOG: connection received: host=[local]
2026-11-14 07:54:44.569 EST [1110598][dead-end client backend][:0] FATAL: the database system is not yet accepting connections
2026-11-14 07:54:44.569 EST [1110598][dead-end client backend][:0] DETAIL: Consistent recovery state has not been yet reached.
2026-11-14 07:54:44.569 EST [1110598][dead-end client backend][:0] DEBUG: shmem_exit(1): 0 before_shmem_exit callbacks to make
2026-11-14 07:54:44.569 EST [1110598][dead-end client backend][:0] DEBUG: shmem_exit(1): 0 on_shmem_exit callbacks to make
2026-11-14 07:54:44.569 EST [1110598][dead-end client backend][:0] DEBUG: proc_exit(1): 1 callbacks to make
2026-11-14 07:54:44.569 EST [1110598][dead-end client backend][:0] DEBUG: exit(1)
2026-11-14 07:54:44.569 EST [1110598][dead-end client backend][:0] DEBUG: shmem_exit(-1): 0 before_shmem_exit callbacks to make
2026-11-14 07:54:44.569 EST [1110598][dead-end client backend][:0] DEBUG: shmem_exit(-1): 0 on_shmem_exit callbacks to make
2026-11-14 07:54:44.569 EST [1110598][dead-end client backend][:0] DEBUG: proc_exit(-1): 0 callbacks to make
2026-11-14 07:54:44.570 EST [1109746][postmaster][:0] DEBUG: reaping dead processes
2026-11-14 07:54:44.570 EST [1109746][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.570 EST [1109746][postmaster][:0] LOG: !!!waitpid| ret: 1110598, status: 256
2026-11-14 07:54:44.570 EST [1109746][postmaster][:0] DEBUG: releasing dead-end backend
2026-11-14 07:54:44.570 EST [1109746][postmaster][:0] DEBUG: dead-end client backend (PID 1110598) exited with exit code 1
2026-11-14 07:54:44.570 EST [1109746][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.570 EST [1109746][postmaster][:0] LOG: !!!waitpid| ret: 0, status: 256
2026-11-14 07:54:44.672 EST [1109746][postmaster][:0] DEBUG: allocating dead-end child
2026-11-14 07:54:44.672 EST [1109746][postmaster][:0] DEBUG: forked new dead-end client backend, pid=1110640 socket=8
2026-11-14 07:54:44.673 EST [1110640][dead-end client backend][:0] LOG: connection received: host=[local]
2026-11-14 07:54:44.673 EST [1110640][dead-end client backend][:0] FATAL: the database system is not yet accepting connections
2026-11-14 07:54:44.673 EST [1110640][dead-end client backend][:0] DETAIL: Consistent recovery state has not been yet reached.
2026-11-14 07:54:44.673 EST [1110640][dead-end client backend][:0] DEBUG: shmem_exit(1): 0 before_shmem_exit callbacks to make
2026-11-14 07:54:44.673 EST [1110640][dead-end client backend][:0] DEBUG: shmem_exit(1): 0 on_shmem_exit callbacks to make
2026-11-14 07:54:44.673 EST [1110640][dead-end client backend][:0] DEBUG: proc_exit(1): 1 callbacks to make
2026-11-14 07:54:44.673 EST [1110640][dead-end client backend][:0] DEBUG: exit(1)
2026-11-14 07:54:44.673 EST [1110640][dead-end client backend][:0] DEBUG: shmem_exit(-1): 0 before_shmem_exit callbacks to make
2026-11-14 07:54:44.673 EST [1110640][dead-end client backend][:0] DEBUG: shmem_exit(-1): 0 on_shmem_exit callbacks to make
2026-11-14 07:54:44.673 EST [1110640][dead-end client backend][:0] DEBUG: proc_exit(-1): 0 callbacks to make
2026-11-14 07:54:44.673 EST [1109746][postmaster][:0] DEBUG: reaping dead processes
2026-11-14 07:54:44.673 EST [1109746][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.673 EST [1109746][postmaster][:0] LOG: !!!waitpid| ret: 1110640, status: 256
2026-11-14 07:54:44.673 EST [1109746][postmaster][:0] DEBUG: releasing dead-end backend
2026-11-14 07:54:44.673 EST [1109746][postmaster][:0] DEBUG: dead-end client backend (PID 1110640) exited with exit code 1
2026-11-14 07:54:44.673 EST [1109746][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.673 EST [1109746][postmaster][:0] LOG: !!!waitpid| ret: 0, status: 256
2026-11-14 07:54:44.775 EST [1109746][postmaster][:0] DEBUG: allocating dead-end child
2026-11-14 07:54:44.775 EST [1109746][postmaster][:0] DEBUG: forked new dead-end client backend, pid=1110689 socket=8
2026-11-14 07:54:44.775 EST [1110689][dead-end client backend][:0] LOG: connection received: host=[local]
2026-11-14 07:54:44.775 EST [1110689][dead-end client backend][:0] FATAL: the database system is not yet accepting connections
2026-11-14 07:54:44.775 EST [1110689][dead-end client backend][:0] DETAIL: Consistent recovery state has not been yet reached.
2026-11-14 07:54:44.775 EST [1110689][dead-end client backend][:0] DEBUG: shmem_exit(1): 0 before_shmem_exit callbacks to make
2026-11-14 07:54:44.775 EST [1110689][dead-end client backend][:0] DEBUG: shmem_exit(1): 0 on_shmem_exit callbacks to make
2026-11-14 07:54:44.775 EST [1110689][dead-end client backend][:0] DEBUG: proc_exit(1): 1 callbacks to make
2026-11-14 07:54:44.775 EST [1110689][dead-end client backend][:0] DEBUG: exit(1)
2026-11-14 07:54:44.775 EST [1110689][dead-end client backend][:0] DEBUG: shmem_exit(-1): 0 before_shmem_exit callbacks to make
2026-11-14 07:54:44.775 EST [1110689][dead-end client backend][:0] DEBUG: shmem_exit(-1): 0 on_shmem_exit callbacks to make
2026-11-14 07:54:44.775 EST [1110689][dead-end client backend][:0] DEBUG: proc_exit(-1): 0 callbacks to make
2026-11-14 07:54:44.776 EST [1109746][postmaster][:0] DEBUG: reaping dead processes
2026-11-14 07:54:44.776 EST [1109746][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.776 EST [1109746][postmaster][:0] LOG: !!!waitpid| ret: 1110689, status: 256
2026-11-14 07:54:44.776 EST [1109746][postmaster][:0] DEBUG: releasing dead-end backend
2026-11-14 07:54:44.776 EST [1109746][postmaster][:0] DEBUG: dead-end client backend (PID 1110689) exited with exit code 1
2026-11-14 07:54:44.776 EST [1109746][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.776 EST [1109746][postmaster][:0] LOG: !!!waitpid| ret: 0, status: 256
2026-11-14 07:54:44.800 EST [1110541][startup][:0] LOG: !!!proc_exit| delayed exit(0) for startup process
2026-11-14 07:54:44.800 EST [1110541][startup][:0] DEBUG: shmem_exit(-1): 0 before_shmem_exit callbacks to make
2026-11-14 07:54:44.800 EST [1110541][startup][:0] DEBUG: shmem_exit(-1): 0 on_shmem_exit callbacks to make
2026-11-14 07:54:44.800 EST [1110541][startup][:0] DEBUG: proc_exit(-1): 0 callbacks to make
2026-11-14 07:54:44.800 EST [1109746][postmaster][:0] DEBUG: reaping dead processes
2026-11-14 07:54:44.800 EST [1109746][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.800 EST [1109746][postmaster][:0] LOG: !!!waitpid| ret: 1110541, status: 0
2026-11-14 07:54:44.800 EST [1109746][postmaster][:0] DEBUG: releasing pm child slot 89
2026-11-14 07:54:44.800 EST [1109746][postmaster][:0] DEBUG: updating PMState from PM_RECOVERY to PM_RUN
2026-11-14 07:54:44.800 EST [1109746][postmaster][:0] LOG: database system is ready to accept connections
2026-11-14 07:54:44.800 EST [1109746][postmaster][:0] LOG: !!!waitpid| pid: -1
2026-11-14 07:54:44.800 EST [1109746][postmaster][:0] LOG: !!!waitpid| ret: 0, status: 0
2026-11-14 07:54:44.800 EST [1109746][postmaster][:0] DEBUG: assigned pm child slot 92 for walwriter
2026-11-14 07:54:44.801 EST [1109746][postmaster][:0] DEBUG: assigned pm child slot 45 for background worker
2026-11-14 07:54:44.801 EST [1109746][postmaster][:0] DEBUG: starting background worker process "logical replication launcher"
2026-11-14 07:54:44.801 EST [1110704][logical replication launcher][15/0:0] DEBUG: logical replication launcher started
2026-11-14 07:54:44.801 EST [1110704][logical replication launcher][15/0:0] DEBUG: InitPostgres
2026-11-14 07:54:44.884 EST [1109746][postmaster][:0] DEBUG: assigned pm child slot 1 for client backend
2026-11-14 07:54:44.884 EST [1109746][postmaster][:0] DEBUG: forked new client backend, pid=1110770 socket=8
2026-11-14 07:54:44.884 EST [1110770][client backend][:0] LOG: connection received: host=[local]
2026-11-14 07:54:44.884 EST [1110770][client backend][0/0:0] DEBUG: InitPostgres
...
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-hackers@postgresql.org
Cc: exclusion@gmail.com, masao.fujii@gmail.com, ayushtiwari.slg01@gmail.com, michael@paquier.xyz
Subject: Re: 041_checkpoint_at_promote.pl might fail due to race condition on child kill
In-Reply-To: <1c733765-62af-48b3-b0d7-1e744c8b8522@gmail.com>
References: <e7abd282-655d-495e-94c9-e38a58917ebf@gmail.com> <CAHGQGwFvuGE5cJJfTFtaYY855=3pmuiczAd13JvcpTVWnUVgFg@mail.gmail.com> <c011d010-d227-4f27-b229-e9e0f2b668e0@gmail.com> <CAHGQGwFtmZAuG+VruhcHWQcmv4=YcQtXLc9jcN3J8JPF_p4o-w@mail.gmail.com> <CAJTYsWXkw4OqxNKVrp8dkhJjG8huFyvCjxEvSL7imSFembaP_g@mail.gmail.com> <CAHGQGwG3NJ0UZO0-u+GFxqO6x7RPu408OO0v0mjMdmYEPSaC-A@mail.gmail.com> <1c733765-62af-48b3-b0d7-1e744c8b8522@gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox