Content-Type: multipart/mixed; boundary="------------k93Wv7BdXkBaEMlI9jjs835l"
Message-ID: <b54df21b-6595-4031-baf8-7c87851794c8@vondra.me>
Date: Tue, 24 Jun 2025 22:32:25 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: pgsql: Introduce pg_shmem_allocations_numa view
To: Christoph Berg <myon@debian.org>
Cc: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>,
 Andres Freund <andres@anarazel.de>,
 Tomas Vondra <tomas.vondra@postgresql.org>,
 pgsql-hackers@lists.postgresql.org
References: <aFnGQ2lfb8cfukim@msg.df7cb.de>
 <cd16c4a3-2860-46de-a74e-5532d1b2ee39@vondra.me>
 <6342f601-77de-4ee0-8c2a-3deb50ceac5b@vondra.me>
 <aFpg1de9ZfS1QgUt@ip-10-97-1-34.eu-west-3.compute.internal>
 <a3a4fe3d-1a80-4e03-aa8e-150ee15f6c35@vondra.me>
 <aFqHoNXQ/uWKXJ4U@ip-10-97-1-34.eu-west-3.compute.internal>
 <8649a4e3-c60d-4f37-aa6f-e7e7c14c581e@vondra.me>
 <aFqnM/iiL+MB62dG@ip-10-97-1-34.eu-west-3.compute.internal>
 <aFq5HQj016rVS2lm@msg.df7cb.de>
 <8961c087-e49b-4b16-9437-31331625215c@vondra.me>
 <aFrEesaN9YlV0RrJ@msg.df7cb.de>
Content-Language: en-US
From: Tomas Vondra <tomas@vondra.me>
In-Reply-To: <aFrEesaN9YlV0RrJ@msg.df7cb.de>
Archived-At: 
 <https://www.postgresql.org/message-id/b54df21b-6595-4031-baf8-7c87851794c8%40vondra.me>
Precedence: bulk

This is a multi-part message in MIME format.
--------------k93Wv7BdXkBaEMlI9jjs835l
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 6/24/25 17:30, Christoph Berg wrote:
> Re: Tomas Vondra
>> If it's a reliable fix, then I guess we can do it like this. But won't
>> that be a performance penalty on everyone? Or does the system split the
>> array into 16-element chunks anyway, so this makes no difference?
> 
> There's still the overhead of the syscall itself. But no idea how
> costly it is to have this 16-step loop in user or kernel space.
> 
> We could claim that on 32-bit systems, shared_buffers would be smaller
> anyway, so there the overhead isn't that big. And the step size should
> be larger (if at all) on 64-bit.
> 
>> Anyway, maybe we should start by reporting this to the kernel people. Do
>> you want me to do that, or shall one of you take care of that? I suppose
>> that'd be better, as you already wrote a fix / know the code better.
> 
> Submitted: https://marc.info/?l=linux-mm&m=175077821909222&w=2
> 

Thanks! Now we wait ...

Attached is a minor tweak of the valgrind suppresion rules, to add the
two places touching the memory. I was hoping I could add a single rule
for pg_numa_touch_mem_if_required, but that does not work - it's a
macro, not a function. So I had to add one rule for both functions,
querying the NUMA. That's a bit disappointing, because it means it'll
hide all other failues (of Memcheck:Addr8 type) in those functions.

Perhaps it'd be be better to turn pg_numa_touch_mem_if_required into a
proper (inlined) function, at least with USE_VALGRIND defined. Something
like the v2 patch - needs more testing to ensure the inlined function
doesn't break the touching or something silly like that.

regards

-- 
Tomas Vondra

--------------k93Wv7BdXkBaEMlI9jjs835l
Content-Type: text/x-patch; charset=UTF-8; name="fix-valgrind-for-numa.patch"
Content-Disposition: attachment; filename="fix-valgrind-for-numa.patch"
Content-Transfer-Encoding: base64

ZGlmZiAtLWdpdCBhL3NyYy90b29scy92YWxncmluZC5zdXBwIGIvc3JjL3Rvb2xzL3ZhbGdy
aW5kLnN1cHAKaW5kZXggN2VhNDY0YzgwOTQuLjM2YmYzMjUzZjc2IDEwMDY0NAotLS0gYS9z
cmMvdG9vbHMvdmFsZ3JpbmQuc3VwcAorKysgYi9zcmMvdG9vbHMvdmFsZ3JpbmQuc3VwcApA
QCAtMTgwLDMgKzE4MCwyMiBAQAogICAgTWVtY2hlY2s6Q29uZAogICAgZnVuOlB5T2JqZWN0
X1JlYWxsb2MKIH0KKworIyBRdWVyeWluZyBOVU1BIG5vZGUgZm9yIHNoYXJlZCBtZW1vcnkg
cmVxdWlyZXMgdG91Y2hpbmcgdGhlIG1lbW9yeSBzbworIyB0aGF0IGl0IGdldHMgYWxsb2Nh
dGVkIGluIHRoZSBwcm9jZXNzLiBCdXQgd2UnbGwgdG91Y2ggbWVtb3J5IGJhY2tpbmcKKyMg
YnVmZmVycywgYnV0IHRoYXQgbWVtb3J5IG1heSBiZSBtYXJrZWQgYXMgbm9hY2Nlc3MgZm9y
IGJ1ZmZlcnMgdGhhdAorIyBhcmUgbm90IHBpbm5lZC4gU28ganVzdCBpZ25vcmUgdGhhdCwg
d2UncmUgbm90IHJlYWxseSBhY2Nlc3NpbmcgdGhlCisjIGJ1ZmZlcnMsIGZvciBib3RoIHBs
YWNlcyBxdWVyeWluZyB0aGUgTlVNQSBzdGF0dXMuCit7CisgICBwZ19idWZmZXJjYWNoZV9u
dW1hX3BhZ2VzCisgICBNZW1jaGVjazpBZGRyOAorICAgZnVuOnBnX2J1ZmZlcmNhY2hlX251
bWFfcGFnZXMKKyAgIGZ1bjpFeGVjTWFrZVRhYmxlRnVuY3Rpb25SZXN1bHQKK30KKworewor
ICAgcGdfZ2V0X3NobWVtX2FsbG9jYXRpb25zX251bWEKKyAgIE1lbWNoZWNrOkFkZHI4Cisg
ICBmdW46cGdfZ2V0X3NobWVtX2FsbG9jYXRpb25zX251bWEKKyAgIGZ1bjpFeGVjTWFrZVRh
YmxlRnVuY3Rpb25SZXN1bHQKK30K
--------------k93Wv7BdXkBaEMlI9jjs835l
Content-Type: text/x-patch; charset=UTF-8;
 name="fix-valgrind-for-numa-v2.patch"
Content-Disposition: attachment; filename="fix-valgrind-for-numa-v2.patch"
Content-Transfer-Encoding: base64

ZGlmZiAtLWdpdCBhL3NyYy9pbmNsdWRlL3BvcnQvcGdfbnVtYS5oIGIvc3JjL2luY2x1ZGUv
cG9ydC9wZ19udW1hLmgKaW5kZXggNDBmMWQzMjRkY2YuLjNiOWE1YjQyODk4IDEwMDY0NAot
LS0gYS9zcmMvaW5jbHVkZS9wb3J0L3BnX251bWEuaAorKysgYi9zcmMvaW5jbHVkZS9wb3J0
L3BnX251bWEuaApAQCAtMjQsOSArMjQsMjIgQEAgZXh0ZXJuIFBHRExMSU1QT1JUIGludCBw
Z19udW1hX2dldF9tYXhfbm9kZSh2b2lkKTsKICAqIFRoaXMgaXMgcmVxdWlyZWQgb24gTGlu
dXgsIGJlZm9yZSBwZ19udW1hX3F1ZXJ5X3BhZ2VzKCkgYXMgd2UKICAqIG5lZWQgdG8gcGFn
ZS1mYXVsdCBiZWZvcmUgbW92ZV9wYWdlcygyKSBzeXNjYWxsIHJldHVybnMgdmFsaWQgcmVz
dWx0cy4KICAqLworI2lmZGVmIFVTRV9WQUxHUklORAorCitzdGF0aWMgaW5saW5lIHZvaWQK
K3BnX251bWFfdG91Y2hfbWVtX2lmX3JlcXVpcmVkKHVpbnQ2NCB0bXAsIGNoYXIgKnB0cikK
K3sKKwl2b2xhdGlsZSB1aW50NjQgcm9fdm9sYXRpbGVfdmFyIHBnX2F0dHJpYnV0ZV91bnVz
ZWQoKTsKKwlyb192b2xhdGlsZV92YXIgPSAqKHZvbGF0aWxlIHVpbnQ2NCAqKSBwdHI7Cit9
CisKKyNlbHNlCisKICNkZWZpbmUgcGdfbnVtYV90b3VjaF9tZW1faWZfcmVxdWlyZWQocm9f
dm9sYXRpbGVfdmFyLCBwdHIpIFwKIAlyb192b2xhdGlsZV92YXIgPSAqKHZvbGF0aWxlIHVp
bnQ2NCAqKSBwdHIKIAorI2VuZGlmCisKICNlbHNlCiAKICNkZWZpbmUgcGdfbnVtYV90b3Vj
aF9tZW1faWZfcmVxdWlyZWQocm9fdm9sYXRpbGVfdmFyLCBwdHIpIFwKZGlmZiAtLWdpdCBh
L3NyYy90b29scy92YWxncmluZC5zdXBwIGIvc3JjL3Rvb2xzL3ZhbGdyaW5kLnN1cHAKaW5k
ZXggN2VhNDY0YzgwOTQuLjZiOWE4OTk4ZjgyIDEwMDY0NAotLS0gYS9zcmMvdG9vbHMvdmFs
Z3JpbmQuc3VwcAorKysgYi9zcmMvdG9vbHMvdmFsZ3JpbmQuc3VwcApAQCAtMTgwLDMgKzE4
MCwxNCBAQAogICAgTWVtY2hlY2s6Q29uZAogICAgZnVuOlB5T2JqZWN0X1JlYWxsb2MKIH0K
KworIyBRdWVyeWluZyBOVU1BIG5vZGUgZm9yIHNoYXJlZCBtZW1vcnkgcmVxdWlyZXMgdG91
Y2hpbmcgdGhlIG1lbW9yeSBzbworIyB0aGF0IGl0IGdldHMgYWxsb2NhdGVkIGluIHRoZSBw
cm9jZXNzLiBCdXQgd2UnbGwgdG91Y2ggbWVtb3J5IGJhY2tpbmcKKyMgYnVmZmVycywgYnV0
IHRoYXQgbWVtb3J5IG1heSBiZSBtYXJrZWQgYXMgbm9hY2Nlc3MgZm9yIGJ1ZmZlcnMgdGhh
dAorIyBhcmUgbm90IHBpbm5lZC4gU28ganVzdCBpZ25vcmUgdGhhdCwgd2UncmUgbm90IHJl
YWxseSBhY2Nlc3NpbmcgdGhlCisjIGJ1ZmZlcnMsIGZvciBhbGwgcGxhY2VzIHF1ZXJ5aW5n
IHRoZSBOVU1BIHN0YXR1cy4KK3sKKyAgIHBnX251bWFfdG91Y2hfbWVtX2lmX3JlcXVpcmVk
CisgICBNZW1jaGVjazpBZGRyOAorICAgZnVuOnBnX251bWFfdG91Y2hfbWVtX2lmX3JlcXVp
cmVkCit9Cg==

--------------k93Wv7BdXkBaEMlI9jjs835l--