Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1izDTk-00028I-0x for pgsql-hackers@arkaria.postgresql.org; Wed, 05 Feb 2020 05:44:00 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1izDTi-0004iM-3F for pgsql-hackers@arkaria.postgresql.org; Wed, 05 Feb 2020 05:43:58 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1izDTh-0004iE-QI for pgsql-hackers@lists.postgresql.org; Wed, 05 Feb 2020 05:43:57 +0000 Received: from oss.nttdata.com ([49.212.34.109]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1izDTe-0003bT-6U for pgsql-hackers@postgresql.org; Wed, 05 Feb 2020 05:43:57 +0000 Received: from hnk.local (sp49-98-213-66.msd.spmode.ne.jp [49.98.213.66]) by oss.nttdata.com (Postfix) with ESMTPSA id 49B4260343 for ; Wed, 5 Feb 2020 14:43:50 +0900 (JST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.101.5 at oss.nttdata.com To: pgsql-hackers@postgresql.org From: Fujii Masao Subject: A bug in LWLOCK_STATS Message-ID: <26359edb-798a-568f-d93a-6aafac49752d@oss.nttdata.com> Date: Wed, 5 Feb 2020 14:43:49 +0900 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:68.0) Gecko/20100101 Thunderbird/68.4.2 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="------------78151BB5396440A24C3A4FD6" Content-Language: en-US List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk This is a multi-part message in MIME format. --------------78151BB5396440A24C3A4FD6 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Hi, When I compiled PostgreSQL with -DLWLOCK_STATS and tried to check the statistics of light-weight locks, I observed that more than one statistics entries were output *for the same backend process and the same lwlock*. For example, I got the following four statistics when I checked how the process with PID 81141 processed ProcArrayLock. This is strange, and IMO only one statistics should be output for the same backend process and lwlock. $ grep "PID 81141 lwlock ProcArrayLock" data/log/postgresql-2020-02-05_141842.log PID 81141 lwlock ProcArrayLock 0x111e87780: shacq 4000 exacq 0 blk 0 spindelay 0 dequeue self 0 PID 81141 lwlock ProcArrayLock 0x111e87780: shacq 2 exacq 0 blk 0 spindelay 0 dequeue self 0 PID 81141 lwlock ProcArrayLock 0x111e87780: shacq 6001 exacq 1 blk 0 spindelay 0 dequeue self 0 PID 81141 lwlock ProcArrayLock 0x111e87780: shacq 5 exacq 1 blk 0 spindelay 0 dequeue self 0 The cause of this issue is that the key variable used for lwlock hash table was not fully initialized. The key consists of two fields and they are initialized as follows. But the following 4 bytes allocated for the alignment was not initialized. So even if the same key was specified, hash_search(HASH_ENTER) could not find the existing entry for that key and created new one. key.tranche = lock->tranche; key.instance = lock; Attached is the patch fixing this issue by initializing the key variable with zero. In the patched version, I confirmed that only one statistics is output for the same process and the same lwlock. Also this patch would reduce the volume of lwlock statistics very much. This issue was introduced by commit 3761fe3c20. So the patch needs to be back-patch to v10. Regards, -- Fujii Masao NTT DATA CORPORATION Advanced Platform Technology Group Research and Development Headquarters --------------78151BB5396440A24C3A4FD6 Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0"; name="lwlock_stats.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="lwlock_stats.patch" ZGlmZiAtLWdpdCBhL3NyYy9iYWNrZW5kL3N0b3JhZ2UvbG1nci9sd2xvY2suYyBiL3NyYy9i YWNrZW5kL3N0b3JhZ2UvbG1nci9sd2xvY2suYwppbmRleCBkMDdjZTYwOWQ0Li40YzE0ZTUx YzY3IDEwMDY0NAotLS0gYS9zcmMvYmFja2VuZC9zdG9yYWdlL2xtZ3IvbHdsb2NrLmMKKysr IGIvc3JjL2JhY2tlbmQvc3RvcmFnZS9sbWdyL2x3bG9jay5jCkBAIC0zMTAsNiArMzEwLDcg QEAgZ2V0X2x3bG9ja19zdGF0c19lbnRyeShMV0xvY2sgKmxvY2spCiAJCXJldHVybiAmbHds b2NrX3N0YXRzX2R1bW15OwogCiAJLyogRmV0Y2ggb3IgY3JlYXRlIHRoZSBlbnRyeS4gKi8K KwlNZW1TZXQoJmtleSwgMCwgc2l6ZW9mKGtleSkpOwogCWtleS50cmFuY2hlID0gbG9jay0+ dHJhbmNoZTsKIAlrZXkuaW5zdGFuY2UgPSBsb2NrOwogCWx3c3RhdHMgPSBoYXNoX3NlYXJj aChsd2xvY2tfc3RhdHNfaHRhYiwgJmtleSwgSEFTSF9FTlRFUiwgJmZvdW5kKTsK --------------78151BB5396440A24C3A4FD6--