pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Make pg_ctl stop/restart/promote recheck postmaster aliveness. 6+ messages / 1 participants [nested] [flat]
* pgsql: Make pg_ctl stop/restart/promote recheck postmaster aliveness. @ 2022-02-10 21:49 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2022-02-10 21:49 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Make pg_ctl stop/restart/promote recheck postmaster aliveness. "pg_ctl stop/restart" checked that the postmaster PID is valid just once, as a side-effect of sending the stop signal, and then would wait-till-timeout for the postmaster.pid file to go away. This neglects the case wherein the postmaster dies uncleanly after we signal it. Similarly, once "pg_ctl promote" has sent the signal, it'd wait for the corresponding on-disk state change to occur even if the postmaster dies. I'm not sure how we've managed not to notice this problem, but it seems to explain slow execution of the 017_shm.pl test script on AIX since commit 4fdbf9af5, which added a speculative "pg_ctl stop" with the idea of making real sure that the postmaster isn't there. In the test steps that kill-9 and then restart the postmaster, it's possible to get past the initial signal attempt before kill() stops working for the doomed postmaster. If that happens, pg_ctl waited till PGCTLTIMEOUT before giving up ... and the buildfarm's AIX members have that set very high. To fix, include a "kill(pid, 0)" test (similar to what postmaster_is_alive uses) in these wait loops, so that we'll give up immediately if the postmaster PID disappears. While here, I chose to refactor those loops out of where they were. do_stop() and do_restart() can perfectly well share one copy of the wait-for-stop loop, and it seems desirable to put a similar function beside that for wait-for-promote. Back-patch to all supported versions, since pg_ctl's wait logic is substantially identical in all, and we're seeing the slow test behavior in all branches. Discussion: https://postgr.es/m/20220210023537.GA3222837@rfd.leadboat.com Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/a745b936507f57334e2c0108dcea5965848e3e35 Modified Files -------------- src/bin/pg_ctl/pg_ctl.c | 128 ++++++++++++++++++++++++++++++------------------ 1 file changed, 80 insertions(+), 48 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Make pg_ctl stop/restart/promote recheck postmaster aliveness. @ 2022-02-10 21:49 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2022-02-10 21:49 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Make pg_ctl stop/restart/promote recheck postmaster aliveness. "pg_ctl stop/restart" checked that the postmaster PID is valid just once, as a side-effect of sending the stop signal, and then would wait-till-timeout for the postmaster.pid file to go away. This neglects the case wherein the postmaster dies uncleanly after we signal it. Similarly, once "pg_ctl promote" has sent the signal, it'd wait for the corresponding on-disk state change to occur even if the postmaster dies. I'm not sure how we've managed not to notice this problem, but it seems to explain slow execution of the 017_shm.pl test script on AIX since commit 4fdbf9af5, which added a speculative "pg_ctl stop" with the idea of making real sure that the postmaster isn't there. In the test steps that kill-9 and then restart the postmaster, it's possible to get past the initial signal attempt before kill() stops working for the doomed postmaster. If that happens, pg_ctl waited till PGCTLTIMEOUT before giving up ... and the buildfarm's AIX members have that set very high. To fix, include a "kill(pid, 0)" test (similar to what postmaster_is_alive uses) in these wait loops, so that we'll give up immediately if the postmaster PID disappears. While here, I chose to refactor those loops out of where they were. do_stop() and do_restart() can perfectly well share one copy of the wait-for-stop loop, and it seems desirable to put a similar function beside that for wait-for-promote. Back-patch to all supported versions, since pg_ctl's wait logic is substantially identical in all, and we're seeing the slow test behavior in all branches. Discussion: https://postgr.es/m/20220210023537.GA3222837@rfd.leadboat.com Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/1e8c5cf7c6c2668da69b77cea91f74ca75e67c56 Modified Files -------------- src/bin/pg_ctl/pg_ctl.c | 128 ++++++++++++++++++++++++++++++------------------ 1 file changed, 80 insertions(+), 48 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Make pg_ctl stop/restart/promote recheck postmaster aliveness. @ 2022-02-10 21:49 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2022-02-10 21:49 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Make pg_ctl stop/restart/promote recheck postmaster aliveness. "pg_ctl stop/restart" checked that the postmaster PID is valid just once, as a side-effect of sending the stop signal, and then would wait-till-timeout for the postmaster.pid file to go away. This neglects the case wherein the postmaster dies uncleanly after we signal it. Similarly, once "pg_ctl promote" has sent the signal, it'd wait for the corresponding on-disk state change to occur even if the postmaster dies. I'm not sure how we've managed not to notice this problem, but it seems to explain slow execution of the 017_shm.pl test script on AIX since commit 4fdbf9af5, which added a speculative "pg_ctl stop" with the idea of making real sure that the postmaster isn't there. In the test steps that kill-9 and then restart the postmaster, it's possible to get past the initial signal attempt before kill() stops working for the doomed postmaster. If that happens, pg_ctl waited till PGCTLTIMEOUT before giving up ... and the buildfarm's AIX members have that set very high. To fix, include a "kill(pid, 0)" test (similar to what postmaster_is_alive uses) in these wait loops, so that we'll give up immediately if the postmaster PID disappears. While here, I chose to refactor those loops out of where they were. do_stop() and do_restart() can perfectly well share one copy of the wait-for-stop loop, and it seems desirable to put a similar function beside that for wait-for-promote. Back-patch to all supported versions, since pg_ctl's wait logic is substantially identical in all, and we're seeing the slow test behavior in all branches. Discussion: https://postgr.es/m/20220210023537.GA3222837@rfd.leadboat.com Branch ------ REL_13_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/d0e1fd9587b3445904076fb7ba11b37138f0ac94 Modified Files -------------- src/bin/pg_ctl/pg_ctl.c | 128 ++++++++++++++++++++++++++++++------------------ 1 file changed, 80 insertions(+), 48 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Make pg_ctl stop/restart/promote recheck postmaster aliveness. @ 2022-02-10 21:49 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2022-02-10 21:49 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Make pg_ctl stop/restart/promote recheck postmaster aliveness. "pg_ctl stop/restart" checked that the postmaster PID is valid just once, as a side-effect of sending the stop signal, and then would wait-till-timeout for the postmaster.pid file to go away. This neglects the case wherein the postmaster dies uncleanly after we signal it. Similarly, once "pg_ctl promote" has sent the signal, it'd wait for the corresponding on-disk state change to occur even if the postmaster dies. I'm not sure how we've managed not to notice this problem, but it seems to explain slow execution of the 017_shm.pl test script on AIX since commit 4fdbf9af5, which added a speculative "pg_ctl stop" with the idea of making real sure that the postmaster isn't there. In the test steps that kill-9 and then restart the postmaster, it's possible to get past the initial signal attempt before kill() stops working for the doomed postmaster. If that happens, pg_ctl waited till PGCTLTIMEOUT before giving up ... and the buildfarm's AIX members have that set very high. To fix, include a "kill(pid, 0)" test (similar to what postmaster_is_alive uses) in these wait loops, so that we'll give up immediately if the postmaster PID disappears. While here, I chose to refactor those loops out of where they were. do_stop() and do_restart() can perfectly well share one copy of the wait-for-stop loop, and it seems desirable to put a similar function beside that for wait-for-promote. Back-patch to all supported versions, since pg_ctl's wait logic is substantially identical in all, and we're seeing the slow test behavior in all branches. Discussion: https://postgr.es/m/20220210023537.GA3222837@rfd.leadboat.com Branch ------ REL_12_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/61c20a5eaac3a77ff52a3a47ceb64ff238192281 Modified Files -------------- src/bin/pg_ctl/pg_ctl.c | 128 ++++++++++++++++++++++++++++++------------------ 1 file changed, 80 insertions(+), 48 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Make pg_ctl stop/restart/promote recheck postmaster aliveness. @ 2022-02-10 21:49 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2022-02-10 21:49 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Make pg_ctl stop/restart/promote recheck postmaster aliveness. "pg_ctl stop/restart" checked that the postmaster PID is valid just once, as a side-effect of sending the stop signal, and then would wait-till-timeout for the postmaster.pid file to go away. This neglects the case wherein the postmaster dies uncleanly after we signal it. Similarly, once "pg_ctl promote" has sent the signal, it'd wait for the corresponding on-disk state change to occur even if the postmaster dies. I'm not sure how we've managed not to notice this problem, but it seems to explain slow execution of the 017_shm.pl test script on AIX since commit 4fdbf9af5, which added a speculative "pg_ctl stop" with the idea of making real sure that the postmaster isn't there. In the test steps that kill-9 and then restart the postmaster, it's possible to get past the initial signal attempt before kill() stops working for the doomed postmaster. If that happens, pg_ctl waited till PGCTLTIMEOUT before giving up ... and the buildfarm's AIX members have that set very high. To fix, include a "kill(pid, 0)" test (similar to what postmaster_is_alive uses) in these wait loops, so that we'll give up immediately if the postmaster PID disappears. While here, I chose to refactor those loops out of where they were. do_stop() and do_restart() can perfectly well share one copy of the wait-for-stop loop, and it seems desirable to put a similar function beside that for wait-for-promote. Back-patch to all supported versions, since pg_ctl's wait logic is substantially identical in all, and we're seeing the slow test behavior in all branches. Discussion: https://postgr.es/m/20220210023537.GA3222837@rfd.leadboat.com Branch ------ REL_11_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/69cc15c316b8a0421cd5a8baa2a3041b22fa979c Modified Files -------------- src/bin/pg_ctl/pg_ctl.c | 128 ++++++++++++++++++++++++++++++------------------ 1 file changed, 80 insertions(+), 48 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Make pg_ctl stop/restart/promote recheck postmaster aliveness. @ 2022-02-10 21:49 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2022-02-10 21:49 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Make pg_ctl stop/restart/promote recheck postmaster aliveness. "pg_ctl stop/restart" checked that the postmaster PID is valid just once, as a side-effect of sending the stop signal, and then would wait-till-timeout for the postmaster.pid file to go away. This neglects the case wherein the postmaster dies uncleanly after we signal it. Similarly, once "pg_ctl promote" has sent the signal, it'd wait for the corresponding on-disk state change to occur even if the postmaster dies. I'm not sure how we've managed not to notice this problem, but it seems to explain slow execution of the 017_shm.pl test script on AIX since commit 4fdbf9af5, which added a speculative "pg_ctl stop" with the idea of making real sure that the postmaster isn't there. In the test steps that kill-9 and then restart the postmaster, it's possible to get past the initial signal attempt before kill() stops working for the doomed postmaster. If that happens, pg_ctl waited till PGCTLTIMEOUT before giving up ... and the buildfarm's AIX members have that set very high. To fix, include a "kill(pid, 0)" test (similar to what postmaster_is_alive uses) in these wait loops, so that we'll give up immediately if the postmaster PID disappears. While here, I chose to refactor those loops out of where they were. do_stop() and do_restart() can perfectly well share one copy of the wait-for-stop loop, and it seems desirable to put a similar function beside that for wait-for-promote. Back-patch to all supported versions, since pg_ctl's wait logic is substantially identical in all, and we're seeing the slow test behavior in all branches. Discussion: https://postgr.es/m/20220210023537.GA3222837@rfd.leadboat.com Branch ------ REL_10_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/34523933b8dc118da0693d55a3b01e9d02be0949 Modified Files -------------- src/bin/pg_ctl/pg_ctl.c | 128 ++++++++++++++++++++++++++++++------------------ 1 file changed, 80 insertions(+), 48 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2022-02-10 21:49 UTC | newest] Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2022-02-10 21:49 pgsql: Make pg_ctl stop/restart/promote recheck postmaster aliveness. Tom Lane <tgl@sss.pgh.pa.us> 2022-02-10 21:49 pgsql: Make pg_ctl stop/restart/promote recheck postmaster aliveness. Tom Lane <tgl@sss.pgh.pa.us> 2022-02-10 21:49 pgsql: Make pg_ctl stop/restart/promote recheck postmaster aliveness. Tom Lane <tgl@sss.pgh.pa.us> 2022-02-10 21:49 pgsql: Make pg_ctl stop/restart/promote recheck postmaster aliveness. Tom Lane <tgl@sss.pgh.pa.us> 2022-02-10 21:49 pgsql: Make pg_ctl stop/restart/promote recheck postmaster aliveness. Tom Lane <tgl@sss.pgh.pa.us> 2022-02-10 21:49 pgsql: Make pg_ctl stop/restart/promote recheck postmaster aliveness. Tom Lane <tgl@sss.pgh.pa.us>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox