MIME-Version: 1.0
References: 
 <CAEze2WgAJmWReDN7Chtba8Er2YBvKCoa0KVN25-1evnTrHsLyA@mail.gmail.com>
 <ojnnvj553eber32njg5nphybcf6zvb3bzexfkh3obylsll67pc@jk7jmtzeb66k>
In-Reply-To: <ojnnvj553eber32njg5nphybcf6zvb3bzexfkh3obylsll67pc@jk7jmtzeb66k>
From: Masahiko Sawada <sawada.mshk@gmail.com>
Date: Fri, 24 Apr 2026 10:52:31 -0700
Message-ID: 
 <CAD21AoBj+zKvgw_Q8gjr4YbKccW_uMe3OFQ5+KT246FHUuNXSQ@mail.gmail.com>
Subject: Re: Startup process deadlock: WaitForProcSignalBarriers vs aux
 process
To: Andres Freund <andres@anarazel.de>
Cc: Matthias van de Meent <boekewurm+postgres@gmail.com>,
 Thomas Munro <thomas.munro@gmail.com>,
	PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
 Heikki Linnakangas <hlinnaka@iki.fi>
Content-Type: multipart/mixed; boundary="00000000000043c53e0650387054"
Archived-At: 
 <https://www.postgresql.org/message-id/CAD21AoBj%2BzKvgw_Q8gjr4YbKccW_uMe3OFQ5%2BKT246FHUuNXSQ%40mail.gmail.com>
Precedence: bulk

--00000000000043c53e0650387054
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, Apr 22, 2026 at 12:05=E2=80=AFPM Andres Freund <andres@anarazel.de>=
 wrote:
>
> Hi,
>
> On 2026-04-22 13:21:02 +0200, Matthias van de Meent wrote:
> > If the PSB is emitted (and signaled to checkpointer) before the
> > checkpointer has registered its SIGUSR1 handler, then the checkpointer
> > won't receive the notice to check its procsignal slots, it won't
> > notice the updated procsignal flags, and it won't process the PSB; not
> > until it receives a new SIGUSR1.
> >
> > Signals are sent to all processes that have their procsignal pss_pid
> > set, which is true for every process which has called ProcSignalInit,
> > which for the checkpointer (like other aux processes) happens in
> > AuxiliaryProcessMainCommon. However, checkpointer (also like other aux
> > processes) calls AuxiliaryProcessMainCommon before registering its
> > signal handlers, creating a small window in time where signals are
> > sent, but not handled.
>
> Hm. Have we confirmed this happens?
>
> CheckpointerMain() is called with all signals masked, so it should be ok =
for
> the signal handler to only be set up after AuxiliaryProcessMainCommon(), =
as
> long as it happens before
>
>         /*
>          * Unblock signals (they were blocked when the postmaster forked =
us)
>          */
>         sigprocmask(SIG_SETMASK, &UnBlockSig, NULL);
>
> as the signal delivery should be held until after unblocking signals.

Right. The postmaster blocks all signals before starting child process
as the following comment explains:

     /*
      * We start postmaster children with signals blocked.  This allows the=
m to
      * install their own handlers before unblocking, to avoid races where =
they
      * might run the postmaster's handler and miss an important control
      * signal. With more analysis this could potentially be relaxed.
      */
     sigprocmask(SIG_SETMASK, &BlockSig, &save_mask);

Investigating the issue, I found there is a race condition between the
procsignal initialization and emitting signal barrier that could be
the cause of this issue. Imagine the following scenario:

1. In ProcSignalInit(), the checkpointer initializes its
slot->pss_barrierGeneration with the global generation.
2. In EmitProcSignalBarrier(), the startup checks the checkpointer's
procsignal slot but it skips emitting the signal as slot->pss_pid is
still 0. It can happen even though the checkpointer holds a spinlock
on its slot during the initialization because the first pid check is
done without a spinlock acquisition.
3. The checkpointer sets its pid to slot->pss_pid and releases the spin loc=
k.
4. In WaitForProcSignalBarrier(), the startup checks the
checkpointer's procsignal slot that has already initialized the
pss_barrierGeneration, and waits for it to be updated. However, the
checkpointer never updates its barrier generation as it doesn't get
the signal.


Another similar issue I found would be that child processes could miss
the PROCSIGNAL_BARRIER_UPDATE_XLOG_LOGICAL_INFO signal during the
initialization and end up in an inconsistent state because
InitializeProcessXLogLogicalInfo() is called (in BaseInit()) before
ProcSignalInit(). If the startup emits the signal to a process who is
between two steps, the process would not reflect the latest
XLogLogicalInfo state. I think we should move
InitializeProcessXLogLogicalInfo() after ProcSignalInit() like we do
so for InitLocalDataChecksumState().

I've attached the patch for fixing the latter problem as the fix is
straightforward.

Regards,

--=20
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

--00000000000043c53e0650387054
Content-Type: text/x-patch; charset="US-ASCII";
	name="0001-Fix-race-condition-in-XLogLogicalInfo-and-ProcSignal.patch"
Content-Disposition: attachment;
	filename="0001-Fix-race-condition-in-XLogLogicalInfo-and-ProcSignal.patch"
Content-Transfer-Encoding: base64
Content-ID: <f_mod7i4k00>
X-Attachment-Id: f_mod7i4k00

RnJvbSAwMTM3MDg3OWJiMGZlNDA2NWQwNmQ5MmVmYjAxNTgyZDFiMWRmOTk2IE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBNYXNhaGlrbyBTYXdhZGEgPHNhd2FkYS5tc2hrQGdtYWlsLmNv
bT4KRGF0ZTogRnJpLCAyNCBBcHIgMjAyNiAxMDozNjo1NSAtMDcwMApTdWJqZWN0OiBbUEFUQ0hd
IEZpeCByYWNlIGNvbmRpdGlvbiBpbiBYTG9nTG9naWNhbEluZm8gYW5kIFByb2NTaWduYWwKIGlu
aXRpYWxpemF0aW9uCgpQcmV2aW91c2x5LCBJbml0aWFsaXplUHJvY2Vzc1hMb2dMb2dpY2FsSW5m
bygpIHdhcyBjYWxsZWQgYmVmb3JlClByb2NTaWduYWxJbml0KCkuIFRoaXMgY3JlYXRlZCBhIHdp
bmRvdyB3aGVyZSBhIHByb2Nlc3MgY291bGQgbWlzcyBhCnNpZ25hbCBiYXJyaWVyIGlmIGl0IHdh
cyBpc3N1ZWQgYmV0d2VlbiB0aGVzZSB0d28gY2FsbHMuIEFzIGEgcmVzdWx0LAp0aGUgcHJvY2Vz
cyBjb3VsZCBmYWlsIHRvIHVwZGF0ZSBpdHMgbG9jYWwgWExvZ0xvZ2ljYWxJbmZvIGNhY2hlLAps
ZWFkaW5nIHRvIGFuIGluY29uc2lzdGVudCBsb2dpY2FsIGRlY29kaW5nIHN0YXRlLgoKVGhpcyBj
b21taXQgZml4ZXMgdGhpcyBieSBtb3ZpbmcgSW5pdGlhbGl6ZVByb2Nlc3NYTG9nTG9naWNhbElu
Zm8oKQphZnRlciBQcm9jU2lnbmFsSW5pdCgpLiBUaGlzIGVuc3VyZXMgdGhhdCB0aGUgcHJvY2Vz
cyBpcyByZWdpc3RlcmVkIHRvCnBhcnRpY2lwYXRlIGluIHNpZ25hbCBiYXJyaWVycyBiZWZvcmUg
aXRzIHN0YXRlIGlzIGluaXRpYWxpemVkLApwcmV2ZW50aW5nIGl0IGZyb20gbWlzc2luZyBhbnkg
c3RhdGUgY2hhbmdlIHByb3BhZ2F0ZWQgZHVyaW5nIHRoZQpzdGFydHVwIHNlcXVlbmNlLgoKRGlz
Y3Vzc2lvbjogaHR0cHM6Ly9wb3N0Z3IuZXMvbS8KLS0tCiBzcmMvYmFja2VuZC9wb3N0bWFzdGVy
L2F1eHByb2Nlc3MuYyB8IDE3ICsrKysrKysrKysrKy0tLS0tCiBzcmMvYmFja2VuZC91dGlscy9p
bml0L3Bvc3Rpbml0LmMgICB8IDIwICsrKysrKysrKysrKy0tLS0tLS0tCiAyIGZpbGVzIGNoYW5n
ZWQsIDI0IGluc2VydGlvbnMoKyksIDEzIGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL3NyYy9i
YWNrZW5kL3Bvc3RtYXN0ZXIvYXV4cHJvY2Vzcy5jIGIvc3JjL2JhY2tlbmQvcG9zdG1hc3Rlci9h
dXhwcm9jZXNzLmMKaW5kZXggOGZkYzUxOGIzYTEuLjAxY2NlZDYxNDkyIDEwMDY0NAotLS0gYS9z
cmMvYmFja2VuZC9wb3N0bWFzdGVyL2F1eHByb2Nlc3MuYworKysgYi9zcmMvYmFja2VuZC9wb3N0
bWFzdGVyL2F1eHByb2Nlc3MuYwpAQCAtNzEsMTIgKzcxLDE2IEBAIEF1eGlsaWFyeVByb2Nlc3NN
YWluQ29tbW9uKHZvaWQpCiAJUHJvY1NpZ25hbEluaXQoTlVMTCwgMCk7CiAKIAkvKgotCSAqIElu
aXRpYWxpemUgYSBsb2NhbCBjYWNoZSBvZiB0aGUgZGF0YV9jaGVja3N1bV92ZXJzaW9uLCB0byBi
ZSB1cGRhdGVkIGJ5Ci0JICogdGhlIHByb2NzaWduYWwtYmFzZWQgYmFycmllcnMuCisJICogSW5p
dGlhbGl6ZSBsb2NhbCBzdGF0ZXMsIHRvIGJlIHVwZGF0ZWQgYnkgdGhlIHByb2NzaWduYWwtYmFz
ZWQKKwkgKiBiYXJyaWVycy4KIAkgKgotCSAqIFRoaXMgaW50ZW50aW9uYWxseSBoYXBwZW5zIGFm
dGVyIGluaXRpYWxpemluZyB0aGUgcHJvY3NpZ25hbCwgb3RoZXJ3aXNlCi0JICogd2UgbWlnaHQg
bWlzcyBhIHN0YXRlIGNoYW5nZS4gVGhpcyBtZWFucyB3ZSBjYW4gZ2V0IGEgYmFycmllciBmb3Ig
dGhlCi0JICogc3RhdGUgd2UndmUganVzdCBpbml0aWFsaXplZCAtIGJ1dCBpdCBjYW4gaGFwcGVu
IG9ubHkgb25jZS4KKwkgKiBUaGVzZSBpbml0aWFsaXphdGlvbiBpbnRlbnRpb25hbGx5IGhhcHBl
bnMgYWZhdGVyIGluaXRpYWxpemluZyB0aGUKKwkgKiBwcm9jc2lnbmFsLCBvdGhlcndpc2Ugd2Ug
bWlnaHQgbWlzcyBhIHN0YXRlIGNoYW5nZS4gVGhpcyBtZWFucyB3ZSBjYW4KKwkgKiBnZXQgYSBi
YXJyaWVyIGZvciB0aGUgc3RhdGUgd2UndmUganVzdCBpbml0aWFsaXplZC4KKwkgKi8KKworCS8q
CisJICogSW5pdGlhbGl6ZSBhIGxvY2FsIGNhY2hlIG9mIHRoZSBkYXRhX2NoZWNrc3VtX3ZlcnNp
b24uCiAJICoKIAkgKiBUaGUgcG9zdG1hc3RlciAod2hpY2ggaXMgd2hhdCBnZXRzIGZvcmtlZCBp
bnRvIHRoZSBuZXcgY2hpbGQgcHJvY2VzcykKIAkgKiBkb2VzIG5vdCBoYW5kbGUgYmFycmllcnMs
IHRoZXJlZm9yZSBpdCBtYXkgbm90IGhhdmUgdGhlIGN1cnJlbnQgdmFsdWUKQEAgLTg4LDYgKzky
LDkgQEAgQXV4aWxpYXJ5UHJvY2Vzc01haW5Db21tb24odm9pZCkKIAkgKi8KIAlJbml0TG9jYWxE
YXRhQ2hlY2tzdW1TdGF0ZSgpOwogCisJLyogSW5pdGlhbGl6ZSBsb2dpY2FsIGluZm8gV0FMIGxv
Z2dpbmcgc3RhdGUgKi8KKwlJbml0aWFsaXplUHJvY2Vzc1hMb2dMb2dpY2FsSW5mbygpOworCiAJ
LyoKIAkgKiBBdXhpbGlhcnkgcHJvY2Vzc2VzIGRvbid0IHJ1biB0cmFuc2FjdGlvbnMsIGJ1dCB0
aGV5IG1heSBuZWVkIGEKIAkgKiByZXNvdXJjZSBvd25lciBhbnl3YXkgdG8gbWFuYWdlIGJ1ZmZl
ciBwaW5zIGFjcXVpcmVkIG91dHNpZGUKZGlmZiAtLWdpdCBhL3NyYy9iYWNrZW5kL3V0aWxzL2lu
aXQvcG9zdGluaXQuYyBiL3NyYy9iYWNrZW5kL3V0aWxzL2luaXQvcG9zdGluaXQuYwppbmRleCA2
ZjA3NDAxM2FhOS4uOTZiMDZlNDQ0ZWMgMTAwNjQ0Ci0tLSBhL3NyYy9iYWNrZW5kL3V0aWxzL2lu
aXQvcG9zdGluaXQuYworKysgYi9zcmMvYmFja2VuZC91dGlscy9pbml0L3Bvc3Rpbml0LmMKQEAg
LTY2Miw5ICs2NjIsNiBAQCBCYXNlSW5pdCh2b2lkKQogCS8qIEluaXRpYWxpemUgbG9jayBtYW5h
Z2VyJ3MgbG9jYWwgc3RydWN0cyAqLwogCUluaXRMb2NrTWFuYWdlckFjY2VzcygpOwogCi0JLyog
SW5pdGlhbGl6ZSBsb2dpY2FsIGluZm8gV0FMIGxvZ2dpbmcgc3RhdGUgKi8KLQlJbml0aWFsaXpl
UHJvY2Vzc1hMb2dMb2dpY2FsSW5mbygpOwotCiAJLyoKIAkgKiBJbml0aWFsaXplIHJlcGxpY2F0
aW9uIHNsb3RzIGFmdGVyIHBnc3RhdC4gVGhlIGV4aXQgaG9vayBtaWdodCBuZWVkIHRvCiAJICog
ZHJvcCBlcGhlbWVyYWwgc2xvdHMsIHdoaWNoIGluIHR1cm4gdHJpZ2dlcnMgc3RhdHMgcmVwb3J0
aW5nLgpAQCAtNzU5LDEyICs3NTYsMTYgQEAgSW5pdFBvc3RncmVzKGNvbnN0IGNoYXIgKmluX2Ri
bmFtZSwgT2lkIGRib2lkLAogCVByb2NTaWduYWxJbml0KE15Q2FuY2VsS2V5LCBNeUNhbmNlbEtl
eUxlbmd0aCk7CiAKIAkvKgotCSAqIEluaXRpYWxpemUgYSBsb2NhbCBjYWNoZSBvZiB0aGUgZGF0
YV9jaGVja3N1bV92ZXJzaW9uLCB0byBiZSB1cGRhdGVkIGJ5Ci0JICogdGhlIHByb2NzaWduYWwt
YmFzZWQgYmFycmllcnMuCisJICogSW5pdGlhbGl6ZSBsb2NhbCBzdGF0ZXMsIHRvIGJlIHVwZGF0
ZWQgYnkgdGhlIHByb2NzaWduYWwtYmFzZWQKKwkgKiBiYXJyaWVycy4KIAkgKgotCSAqIFRoaXMg
aW50ZW50aW9uYWxseSBoYXBwZW5zIGFmdGVyIGluaXRpYWxpemluZyB0aGUgcHJvY3NpZ25hbCwg
b3RoZXJ3aXNlCi0JICogd2UgbWlnaHQgbWlzcyBhIHN0YXRlIGNoYW5nZS4gVGhpcyBtZWFucyB3
ZSBjYW4gZ2V0IGEgYmFycmllciBmb3IgdGhlCi0JICogc3RhdGUgd2UndmUganVzdCBpbml0aWFs
aXplZC4KKwkgKiBUaGVzZSBpbml0aWFsaXphdGlvbiBpbnRlbnRpb25hbGx5IGhhcHBlbnMgYWZh
dGVyIGluaXRpYWxpemluZyB0aGUKKwkgKiBwcm9jc2lnbmFsLCBvdGhlcndpc2Ugd2UgbWlnaHQg
bWlzcyBhIHN0YXRlIGNoYW5nZS4gVGhpcyBtZWFucyB3ZSBjYW4KKwkgKiBnZXQgYSBiYXJyaWVy
IGZvciB0aGUgc3RhdGUgd2UndmUganVzdCBpbml0aWFsaXplZC4KKwkgKi8KKworCS8qCisJICog
SW5pdGlhbGl6ZSBhIGxvY2FsIGNhY2hlIG9mIHRoZSBkYXRhX2NoZWNrc3VtX3ZlcnNpb24uCiAJ
ICoKIAkgKiBUaGUgcG9zdG1hc3RlciAod2hpY2ggaXMgd2hhdCBnZXRzIGZvcmtlZCBpbnRvIHRo
ZSBuZXcgY2hpbGQgcHJvY2VzcykKIAkgKiBkb2VzIG5vdCBoYW5kbGUgYmFycmllcnMsIHRoZXJl
Zm9yZSBpdCBtYXkgbm90IGhhdmUgdGhlIGN1cnJlbnQgdmFsdWUKQEAgLTc3Niw2ICs3NzcsOSBA
QCBJbml0UG9zdGdyZXMoY29uc3QgY2hhciAqaW5fZGJuYW1lLCBPaWQgZGJvaWQsCiAJICovCiAJ
SW5pdExvY2FsRGF0YUNoZWNrc3VtU3RhdGUoKTsKIAorCS8qIEluaXRpYWxpemUgbG9naWNhbCBp
bmZvIFdBTCBsb2dnaW5nIHN0YXRlICovCisJSW5pdGlhbGl6ZVByb2Nlc3NYTG9nTG9naWNhbElu
Zm8oKTsKKwogCS8qCiAJICogQWxzbyBzZXQgdXAgdGltZW91dCBoYW5kbGVycyBuZWVkZWQgZm9y
IGJhY2tlbmQgb3BlcmF0aW9uLiAgV2UgbmVlZAogCSAqIHRoZXNlIGluIGV2ZXJ5IGNhc2UgZXhj
ZXB0IGJvb3RzdHJhcC4KLS0gCjIuNTMuMAoK
--00000000000043c53e0650387054--