Content-Type: multipart/mixed; boundary="------------50Dzy99YlzMzZg9svkfxMJxS"
Message-ID: <403c813e-7d63-4ee8-a95f-4e5c1e310f4d@vondra.me>
Date: Mon, 8 Dec 2025 21:02:27 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Adding basic NUMA awareness
To: Jakub Wartak <jakub.wartak@enterprisedb.com>
Cc: Alexey Makhmutov <a.makhmutov@postgrespro.ru>,
 PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
 Andres Freund <andres@anarazel.de>
References: 
 <CAKZiRmy7VW5b0OKwHXtRSudCBFB48-kYnOEXeqrLfbTQ=5qQrA@mail.gmail.com>
 <659c44a5-f616-492c-ab81-60273d2fe7f6@vondra.me>
 <bf95094a-77c2-46cf-913a-443f7419bc79@postgrespro.ru>
 <9c31717c-fe38-4824-9bb7-e4f0ac4f1991@vondra.me>
 <92e23c85-f646-4bab-b5e0-df30d8ddf4bd@postgrespro.ru>
 <72506e5a-92ac-47fb-9d3a-3b7b420af985@vondra.me>
 <e4d7e6fc-b5c5-4288-991c-56219db2edd5@vondra.me>
 <CAKZiRmzQ=jzWSz7NjHggyqpnMkZUaeO00t7rUovs+zvi_YY48w@mail.gmail.com>
 <ae031f63-4f0b-47a0-ad15-134e4ec677b6@vondra.me>
 <CAKZiRmx=0C5k3Qs0DdHZw9cL+72sX_ZH_RXdUW-7U1-978Kvnw@mail.gmail.com>
 <7b824e42-02de-4f4a-a81d-5acc89d417ea@vondra.me>
 <CAKZiRmww2P6QAzu6W+vxB89i5Ha-YRSHMeyr6ax2Lymcu3LUcw@mail.gmail.com>
 <33d0da22-1400-4b7a-b4c8-1867b8a2cae0@vondra.me>
 <5bc6d309-61fe-4d8b-9e1f-a2961564a559@vondra.me>
 <CAKZiRmwPVxi1H23pNZ4_Vc=mtMaNgY1z79s6SwjuUZD3EaOPeA@mail.gmail.com>
 <05df16f8-025a-43cd-9636-3194464012ed@vondra.me>
 <CAKZiRmxwN+qMpbijCLPix_y6mwSjgus2CPPj=1+uFo9fQG-Knw@mail.gmail.com>
Content-Language: en-US
From: Tomas Vondra <tomas@vondra.me>
In-Reply-To: 
 <CAKZiRmxwN+qMpbijCLPix_y6mwSjgus2CPPj=1+uFo9fQG-Knw@mail.gmail.com>
Archived-At: 
 <https://www.postgresql.org/message-id/403c813e-7d63-4ee8-a95f-4e5c1e310f4d%40vondra.me>
Precedence: bulk

This is a multi-part message in MIME format.
--------------50Dzy99YlzMzZg9svkfxMJxS
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Hi,

I've spent the last couple days considering what to do about this patch
series in this thread. The approach assumes partitioning shared memory
in a NUMA-aware way has enough benefits to justify the extra complexity.
I'm getting somewhat skeptical about that being a good trade off.

We've been able to demonstrate some significant benefits of the patches.
See for example the results from about a month ago [1], showing that in
some cases the throughput almost doubles.

But I'm skeptical about this for two reasons:

* Most of the benefit comes from patches unrelated to NUMA. The initial
patches partition clockweep, in a NUMA oblivious way. In fact, applying
the NUMA patches often *reduces* the throughput. So if we're concerned
about contention on the clocksweep hand, we could apply just these first
patches. That way we wouldn't have to deal with huge pages.

* Furthermore, I'm not quite sure clocksweep really is a bottleneck in
realistic cases. The benchmark used in this thread does many concurrent
sequential scans, on data that exceeds shared buffers / fits into RAM.
Perhaps that happens, but I doubt it's all that common.

I've been unable to demonstrate any benefits on other workloads, even if
there's a lot of buffer misses / reads into shared buffers. As soon as
the query starts doing something else, the clocksweep contention becomes
a non-issue. Consider for example read-only pgbench with database much
larger than shared buffers (but still within RAM). The cost of the index
scans (and other nodes) seems to reduce the pressure on clocksweep.

So I'm skeptical about clocksweep pressure being a serious issue, except
for some very narrow benchmarks (like the concurrent seqscan test). And
even if this happened for some realistic cases, partitioning the buffers
in a NUMA-oblivious way seems to do the trick.


When discussing this stuff off list, it was suggested this might help
with the scenario Andres presented in [3], where the throughput improves
a lot with multiple databases. I've not observed that in practice, and I
don't think these patches really can help with that. That scenario is
about buffer lock contention, not clocksweep contention.

With a single database there's nothing to do - there's one contended
page, located on a single node. There'll be contention, the no matter
which node it ends up on. With multiple databases (or multiple root
pages), it either happens to work by chance (if the buffers happen to be
from different nodes), or it would require figuring out which buffers
are busy, and place them on different nodes. But the patches did not
even try to do anything like that. So it still was a matter of chance.


That does not mean we don't need to worry about NUMA. There's still the
issue of misbalancing the allocation - with memory coming from just one
node, etc. Which is an issue because it "overloads" the memory subsystem
on that particular node. But that can be solved simply by interleaving
the shared memory segment.

That sidesteps most of the complexity - it does not need to figure out
how to partition buffers, does not need to worry about huge pages, etc.
This is not a new idea - it's more or less what Jakub Wartak initially
proposed in [2], before I hijacked the thread into the more complex
approach.

Attached is a tiny patch doing mostly what Jakub did, except that it
does two things. First, it allows interleaving the shared memory on all
relevant NUMA nodes (per numa_get_mems_allowed). Second, it allows
populating all memory by setting MAP_POPULATE in mmap(). There's a new
GUC to enable each of these.


I think we should try this (much simpler) approach first, or something
close to it. Sorry for dragging everyone into a much more complex
approach, which now seems to be a dead end.


regards

[1]
https://www.postgresql.org/message-id/e4d7e6fc-b5c5-4288-991c-56219db2edd5@vondra.me

[2]
https://www.postgresql.org/message-id/CAKZiRmw6i1W1AwXxa-Asrn8wrVcVH3TO715g_MCoowTS9rkGyw@mail.gmail.com

[3] https://youtu.be/V75KpACdl6E?t=2120
-- 
Tomas Vondra

--------------50Dzy99YlzMzZg9svkfxMJxS
Content-Type: text/x-patch; charset=UTF-8;
 name="v20251208-0001-numa-Simple-interleaving-and-MAP_POPULATE.patch"
Content-Disposition: attachment;
 filename*0="v20251208-0001-numa-Simple-interleaving-and-MAP_POPULATE.pat";
 filename*1="ch"
Content-Transfer-Encoding: base64

RnJvbSA3MmIwYTliODZkYTdlMWE2ZTIzOWY1ZmRlYWQyNGFlYWEzZDk5ZjI3IE1vbiBTZXAg
MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBUb21hcyBWb25kcmEgPHRvbWFzQHZvbmRyYS5tZT4K
RGF0ZTogRnJpLCA1IERlYyAyMDI1IDE0OjUzOjQ3ICswMTAwClN1YmplY3Q6IFtQQVRDSCB2
MjAyNTEyMDhdIG51bWE6IFNpbXBsZSBpbnRlcmxlYXZpbmcgYW5kIE1BUF9QT1BVTEFURQoK
QWxsb3dzIE5VTUEgaW50ZXJsZWF2aW5nIG9uIHRoZSBzaGFyZWQgbWVtb3J5IHNlZ21lbnQs
IHRvIGVuc3VyZSBtZW1vcnkKaXMgYmFsYW5jZWQgYmV0d2VlbiB0aGUgTlVNQSBub2Rlcy4g
VGhlIHBhdGNoIGFsc28gYWxsb3dzIHByZWZhdWx0aW5nCnRoZSBzaGFyZWQgbWVtb3J5IChi
eSBzZXR0aW5nIE1BUF9QT1BVTEFURSBmbGFnKSwgdG8gYWN0dWFsbHkgYWxsb2NhdGUKdGhl
IHBhZ2VzIG9uIG5vZGVzLgoKVGhlIGNvbW1pdCBhZGRlcyB0d28gR1VDIHBhcmFtZXRlcnMs
IGJvdGggc2V0IHRvICdvZmYnIGJ5IGRlZmF1bHQ6CgotIHNoYXJlZF9tZW1vcnlfaW50ZXJs
ZWF2ZSAoZW5hYmxlcyBOVU1BIGludGVybGVhdmluZykKCi0gc2hhcmVkX21lbW9yeV9wb3B1
bGF0ZSAoc2V0cyBNQVBfUE9QVUxBVEUpCgpUaGUgbWVtb3J5IGlzIGludGVybGVhdmVkIG9u
IGFsbCBub2RlcyBlbmFibGVkIGluIHRoZSBjcHVzZXQsIGFzCnJldHVybmVkIGJ5IG51bWFf
Z2V0X21lbXNfYWxsb3dlZCgpLiBUaGUgaW50ZXJsZWF2aW5nIGlzIGFwcGxpZWQgYXQgdGhl
Cm1lbW9yeSBwYWdlIGdyYW51bGFyaXR5LCBhbmQgaXMgb2JsaXZpb3VzIHRvIHdoYXQncyBz
dG9yZWQgaW4gaXQuCi0tLQogc3JjL2JhY2tlbmQvcG9ydC9zeXN2X3NobWVtLmMgICAgICAg
ICAgICAgICAgIHwgNTEgKysrKysrKysrKysrKysrKysrLQogc3JjL2JhY2tlbmQvdXRpbHMv
bWlzYy9ndWNfcGFyYW1ldGVycy5kYXQgICAgIHwgMTIgKysrKysKIHNyYy9iYWNrZW5kL3V0
aWxzL21pc2MvZ3VjX3RhYmxlcy5jICAgICAgICAgICB8ICAyICsKIHNyYy9iYWNrZW5kL3V0
aWxzL21pc2MvcG9zdGdyZXNxbC5jb25mLnNhbXBsZSB8ICAyICsKIHNyYy9pbmNsdWRlL3Bv
cnQvcGdfbnVtYS5oICAgICAgICAgICAgICAgICAgICB8ICA0ICsrCiBzcmMvaW5jbHVkZS9z
dG9yYWdlL3BnX3NobWVtLmggICAgICAgICAgICAgICAgfCAgMyArKwogc3JjL3BvcnQvcGdf
bnVtYS5jICAgICAgICAgICAgICAgICAgICAgICAgICAgIHwgNDIgKysrKysrKysrKysrKysr
CiA3IGZpbGVzIGNoYW5nZWQsIDExNSBpbnNlcnRpb25zKCspLCAxIGRlbGV0aW9uKC0pCgpk
aWZmIC0tZ2l0IGEvc3JjL2JhY2tlbmQvcG9ydC9zeXN2X3NobWVtLmMgYi9zcmMvYmFja2Vu
ZC9wb3J0L3N5c3Zfc2htZW0uYwppbmRleCAyOThjZWIzZTIxOC4uYjMwMGY1ZWY0YTcgMTAw
NjQ0Ci0tLSBhL3NyYy9iYWNrZW5kL3BvcnQvc3lzdl9zaG1lbS5jCisrKyBiL3NyYy9iYWNr
ZW5kL3BvcnQvc3lzdl9zaG1lbS5jCkBAIC0xOSw2ICsxOSw3IEBACiAgKi8KICNpbmNsdWRl
ICJwb3N0Z3Jlcy5oIgogCisjaW5jbHVkZSA8bnVtYS5oPgogI2luY2x1ZGUgPHNpZ25hbC5o
PgogI2luY2x1ZGUgPHVuaXN0ZC5oPgogI2luY2x1ZGUgPHN5cy9maWxlLmg+CkBAIC0yOSw2
ICszMCw3IEBACiAKICNpbmNsdWRlICJtaXNjYWRtaW4uaCIKICNpbmNsdWRlICJwb3J0L3Bn
X2JpdHV0aWxzLmgiCisjaW5jbHVkZSAicG9ydC9wZ19udW1hLmgiCiAjaW5jbHVkZSAicG9y
dGFiaWxpdHkvbWVtLmgiCiAjaW5jbHVkZSAic3RvcmFnZS9kc20uaCIKICNpbmNsdWRlICJz
dG9yYWdlL2ZkLmgiCkBAIC02MDIsNiArNjA0LDI4IEBAIENyZWF0ZUFub255bW91c1NlZ21l
bnQoU2l6ZSAqc2l6ZSkKIAl2b2lkCSAgICpwdHIgPSBNQVBfRkFJTEVEOwogCWludAkJCW1t
YXBfZXJybm8gPSAwOwogCisJLyoKKwkgKiBXaGVuIGFza2VkIHRvIHVzZSBOVU1BLWludGVy
bGVhdmluZywgd2UgbmVlZCB0byBzZXQgdGhlIHBvbGljeSBiZWZvcmUKKwkgKiB0b3VjaGlu
ZyB0aGUgbWVtb3J5LiBCeSBkZWZhdWx0IHRoYXQnbGwgaGFwcGVuIGxhdGVyLCB3aGVuIGVp
dGhlcgorCSAqIGluaXRpYWxpemluZyBzb21lIG9mIHRoZSBzaG1lbSBzdHJ1Y3R1cmVzIChl
LmcuIGJ1ZmZlciBkZXNjcmlwdG9ycyksIG9yCisJICogd2hlbiBydW5uaW5nIHF1ZXJpZXMu
IEluIHRoYXQgY2FzZSBpdCdzIGVub3VnaCB0byBzZXQgdGhlIHBvbGljeSBhZnRlcgorCSAq
IHRoZSBtbWFwKCkgY2FsbCwgYW5kIHdlIGRvbid0IG5lZWQgdG8gZG8gYW55dGhpbmcgaGVy
ZS4KKwkgKgorCSAqIFdpdGggTUFQX1BPUFVMQVRFLCB0aGUgbW1hcCgpIGl0c2VsZiB3aWxs
IHByZWZhdWx0IHRoZSBwYWdlcywgc28gd2UKKwkgKiBuZWVkIHRvIHNldCB0aGUgcG9saWN5
IHRvIGludGVybGVhdmUgYmVmb3JlIHRoZSBtbWFwKCkgY2FsbCwgYW5kIHRoZW4KKwkgKiBy
ZXZlcnQgdG8gbG9jYWxhbGxvYyAoc28gdGhhdCBwcml2YXRlIG1lbW9yeSBpcyBhbGxvY2F0
ZWQgbG9jYWxseSkuCisJICoKKwkgKiBYWFggSXQgcHJvYmFibHkgaXMgbm90IGEgZ29vZCBp
ZGVhIHRvIGVuYWJsZSBpbnRlcmxlYXZpbmcgd2l0aCByZWd1bGFyCisJICogbWVtb3J5IHBh
Z2VzLCBiZWNhdXNlIHRoZW4gZWFjaCBidWZmZXIgd2lsbCBnZXQgc3BsaXQgb24gdHdvIG5v
ZGVzLCBhbmQKKwkgKiB0aGUgc3lzdGVtIHdvbid0IGJlIGFibGUgdG8gZml4IHRoYXQgYnkg
bWlncmF0aW5nIG9uZSBvZiB0aGUgcGFnZXMuIEJ1dAorCSAqIHdlIGxlYXZlIHRoYXQgdXAg
dG8gdGhlIGFkbWluLCBpbnN0ZWFkIG9mIGZvcmJpZGRpbmcgaXQuCisJICovCisJaWYgKHNo
YXJlZF9tZW1vcnlfaW50ZXJsZWF2ZSAmJiBzaGFyZWRfbWVtb3J5X3BvcHVsYXRlKQorCXsK
KwkJLyogc2V0IHRoZSBhbGxvY2F0aW9uIHRvIGludGVybGVhdmUgb24gbm9kZXMgYWxsb3dl
ZCBieSB0aGUgY3B1c2V0ICovCisJCXBnX251bWFfc2V0X2ludGVybGVhdmUoKTsKKwl9CisK
ICNpZm5kZWYgTUFQX0hVR0VUTEIKIAkvKiBQR1NoYXJlZE1lbW9yeUNyZWF0ZSBzaG91bGQg
aGF2ZSBkZWFsdCB3aXRoIHRoaXMgY2FzZSAqLwogCUFzc2VydChodWdlX3BhZ2VzICE9IEhV
R0VfUEFHRVNfT04pOwpAQCAtNjE5LDYgKzY0MywxMCBAQCBDcmVhdGVBbm9ueW1vdXNTZWdt
ZW50KFNpemUgKnNpemUpCiAJCWlmIChhbGxvY3NpemUgJSBodWdlcGFnZXNpemUgIT0gMCkK
IAkJCWFsbG9jc2l6ZSArPSBodWdlcGFnZXNpemUgLSAoYWxsb2NzaXplICUgaHVnZXBhZ2Vz
aXplKTsKIAorCQkvKiBwb3B1bGF0ZSB0aGUgc2hhcmVkIG1lbW9yeSBpZiByZXF1ZXN0ZWQg
Ki8KKwkJaWYgKHNoYXJlZF9tZW1vcnlfcG9wdWxhdGUpCisJCQltbWFwX2ZsYWdzIHw9IE1B
UF9QT1BVTEFURTsKKwogCQlwdHIgPSBtbWFwKE5VTEwsIGFsbG9jc2l6ZSwgUFJPVF9SRUFE
IHwgUFJPVF9XUklURSwKIAkJCQkgICBQR19NTUFQX0ZMQUdTIHwgbW1hcF9mbGFncywgLTEs
IDApOwogCQltbWFwX2Vycm5vID0gZXJybm87CkBAIC02MzgsMTMgKzY2NiwxOSBAQCBDcmVh
dGVBbm9ueW1vdXNTZWdtZW50KFNpemUgKnNpemUpCiAKIAlpZiAocHRyID09IE1BUF9GQUlM
RUQgJiYgaHVnZV9wYWdlcyAhPSBIVUdFX1BBR0VTX09OKQogCXsKKwkJaW50CQkJbW1hcF9m
bGFncyA9IDA7CisKKwkJLyogcG9wdWxhdGUgdGhlIHNoYXJlZCBtZW1vcnkgaWYgcmVxdWVz
dGVkICovCisJCWlmIChzaGFyZWRfbWVtb3J5X3BvcHVsYXRlKQorCQkJbW1hcF9mbGFncyB8
PSBNQVBfUE9QVUxBVEU7CisKIAkJLyoKIAkJICogVXNlIHRoZSBvcmlnaW5hbCBzaXplLCBu
b3QgdGhlIHJvdW5kZWQtdXAgdmFsdWUsIHdoZW4gZmFsbGluZyBiYWNrCiAJCSAqIHRvIG5v
bi1odWdlIHBhZ2VzLgogCQkgKi8KIAkJYWxsb2NzaXplID0gKnNpemU7CiAJCXB0ciA9IG1t
YXAoTlVMTCwgYWxsb2NzaXplLCBQUk9UX1JFQUQgfCBQUk9UX1dSSVRFLAotCQkJCSAgIFBH
X01NQVBfRkxBR1MsIC0xLCAwKTsKKwkJCQkgICBQR19NTUFQX0ZMQUdTIHwgbW1hcF9mbGFn
cywgLTEsIDApOwogCQltbWFwX2Vycm5vID0gZXJybm87CiAJfQogCkBAIC02NjMsNiArNjk3
LDIxIEBAIENyZWF0ZUFub255bW91c1NlZ21lbnQoU2l6ZSAqc2l6ZSkKIAkJCQkJCSBhbGxv
Y3NpemUpIDogMCkpOwogCX0KIAorCS8qCisJICogV2l0aCBOVU1BIGludGVybGVhdmluZywg
d2UgbmVlZCB0byBlaXRoZXIgYXBwbHkgaW50ZXJsZWF2aW5nIGZvciB0aGUKKwkgKiBzaG1l
bSBzZWdtZW50IHdlIGp1c3QgYWxsb2NhdGVkLCBvciByZXNldCB0aGUgbWVtb3J5IHBvbGlj
eSB0byBsb2NhbAorCSAqIGFsbG9jYXRpb24gKHdoZW4gdXNpbmcgTUFQX1BPUFVMQVRFKS4K
KwkgKi8KKwlpZiAoc2hhcmVkX21lbW9yeV9pbnRlcmxlYXZlKQorCXsKKwkJaWYgKHNoYXJl
ZF9tZW1vcnlfcG9wdWxhdGUpCisJCQkvKiByZXZlcnQgYmFjayB0byB1c2luZyB0aGUgbG9j
YWwgbm9kZSAqLworCQkJcGdfbnVtYV9zZXRfbG9jYWxhbGxvYygpOworCQllbHNlCisJCQkv
KiBhcHBseSBpbnRlcmxlYXZpbmcgdG8gdGhlIG5ldyBtZW1vcnkgc2VnbWVudCAqLworCQkJ
cGdfbnVtYV9pbnRlcmxlYXZlX21lbW9yeShwdHIsIGFsbG9jc2l6ZSk7CisJfQorCiAJKnNp
emUgPSBhbGxvY3NpemU7CiAJcmV0dXJuIHB0cjsKIH0KZGlmZiAtLWdpdCBhL3NyYy9iYWNr
ZW5kL3V0aWxzL21pc2MvZ3VjX3BhcmFtZXRlcnMuZGF0IGIvc3JjL2JhY2tlbmQvdXRpbHMv
bWlzYy9ndWNfcGFyYW1ldGVycy5kYXQKaW5kZXggM2I5ZDgzNDkwNzguLmNlMWMwYzQzMjdm
IDEwMDY0NAotLS0gYS9zcmMvYmFja2VuZC91dGlscy9taXNjL2d1Y19wYXJhbWV0ZXJzLmRh
dAorKysgYi9zcmMvYmFja2VuZC91dGlscy9taXNjL2d1Y19wYXJhbWV0ZXJzLmRhdApAQCAt
MjU5Nyw2ICsyNTk3LDE4IEBACiAgIG1heCA9PiAnSU5UX01BWCAvIDInLAogfSwKIAoreyBu
YW1lID0+ICdzaGFyZWRfbWVtb3J5X2ludGVybGVhdmUnLCB0eXBlID0+ICdib29sJywgY29u
dGV4dCA9PiAnUEdDX1BPU1RNQVNURVInLCBncm91cCA9PiAnUkVTT1VSQ0VTX01FTScsCisg
IHNob3J0X2Rlc2MgPT4gJ0VuYWJsZXMgTlVNQSBpbnRlcmxlYXZpbmcgb2Ygc2hhcmVkIG1l
bW9yeS4nLAorICB2YXJpYWJsZSA9PiAnc2hhcmVkX21lbW9yeV9pbnRlcmxlYXZlJywKKyAg
Ym9vdF92YWwgPT4gJ2ZhbHNlJywKK30sCisKK3sgbmFtZSA9PiAnc2hhcmVkX21lbW9yeV9w
b3B1bGF0ZScsIHR5cGUgPT4gJ2Jvb2wnLCBjb250ZXh0ID0+ICdQR0NfUE9TVE1BU1RFUics
IGdyb3VwID0+ICdSRVNPVVJDRVNfTUVNJywKKyAgc2hvcnRfZGVzYyA9PiAnUG9wdWxhdGVz
IHNoYXJlZCBtZW1vcnkgYXQgc3RhcnQuJywKKyAgdmFyaWFibGUgPT4gJ3NoYXJlZF9tZW1v
cnlfcG9wdWxhdGUnLAorICBib290X3ZhbCA9PiAnZmFsc2UnLAorfSwKKwogeyBuYW1lID0+
ICdzaGFyZWRfbWVtb3J5X3NpemUnLCB0eXBlID0+ICdpbnQnLCBjb250ZXh0ID0+ICdQR0Nf
SU5URVJOQUwnLCBncm91cCA9PiAnUFJFU0VUX09QVElPTlMnLAogICBzaG9ydF9kZXNjID0+
ICdTaG93cyB0aGUgc2l6ZSBvZiB0aGUgc2VydmVyXCdzIG1haW4gc2hhcmVkIG1lbW9yeSBh
cmVhIChyb3VuZGVkIHVwIHRvIHRoZSBuZWFyZXN0IE1CKS4nLAogICBmbGFncyA9PiAnR1VD
X05PVF9JTl9TQU1QTEUgfCBHVUNfRElTQUxMT1dfSU5fRklMRSB8IEdVQ19VTklUX01CIHwg
R1VDX1JVTlRJTUVfQ09NUFVURUQnLApkaWZmIC0tZ2l0IGEvc3JjL2JhY2tlbmQvdXRpbHMv
bWlzYy9ndWNfdGFibGVzLmMgYi9zcmMvYmFja2VuZC91dGlscy9taXNjL2d1Y190YWJsZXMu
YwppbmRleCBmODdiNTU4YzJjNi4uY2ZlZTBkZjk4N2YgMTAwNjQ0Ci0tLSBhL3NyYy9iYWNr
ZW5kL3V0aWxzL21pc2MvZ3VjX3RhYmxlcy5jCisrKyBiL3NyYy9iYWNrZW5kL3V0aWxzL21p
c2MvZ3VjX3RhYmxlcy5jCkBAIC01ODAsNiArNTgwLDggQEAgc3RhdGljIGludAlzc2xfcmVu
ZWdvdGlhdGlvbl9saW1pdDsKIGludAkJCWh1Z2VfcGFnZXMgPSBIVUdFX1BBR0VTX1RSWTsK
IGludAkJCWh1Z2VfcGFnZV9zaXplOwogaW50CQkJaHVnZV9wYWdlc19zdGF0dXMgPSBIVUdF
X1BBR0VTX1VOS05PV047Citib29sCQlzaGFyZWRfbWVtb3J5X2ludGVybGVhdmUgPSBmYWxz
ZTsKK2Jvb2wJCXNoYXJlZF9tZW1vcnlfcG9wdWxhdGUgPSBmYWxzZTsKIAogLyoKICAqIFRo
ZXNlIHZhcmlhYmxlcyBhcmUgYWxsIGR1bW1pZXMgdGhhdCBkb24ndCBkbyBhbnl0aGluZywg
ZXhjZXB0IGluIHNvbWUKZGlmZiAtLWdpdCBhL3NyYy9iYWNrZW5kL3V0aWxzL21pc2MvcG9z
dGdyZXNxbC5jb25mLnNhbXBsZSBiL3NyYy9iYWNrZW5kL3V0aWxzL21pc2MvcG9zdGdyZXNx
bC5jb25mLnNhbXBsZQppbmRleCBkYzllMjI1NWY4YS4uZGUxMjc2ZjY4OTcgMTAwNjQ0Ci0t
LSBhL3NyYy9iYWNrZW5kL3V0aWxzL21pc2MvcG9zdGdyZXNxbC5jb25mLnNhbXBsZQorKysg
Yi9zcmMvYmFja2VuZC91dGlscy9taXNjL3Bvc3RncmVzcWwuY29uZi5zYW1wbGUKQEAgLTE1
Miw2ICsxNTIsOCBAQAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICMgICBzeXN2CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIyAg
IHdpbmRvd3MKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAjIChj
aGFuZ2UgcmVxdWlyZXMgcmVzdGFydCkKKyNzaGFyZWRfbWVtb3J5X2ludGVybGVhdmUgPSBv
ZmYgICAgICAgICAjIGludGVybGVhdmUgYWxsIG1lbW9yeSBvbiBhdmFpbGFibGUgTlVNQSBu
b2RlcworI3NoYXJlZF9tZW1vcnlfcG9wdWxhdGUgPSBvZmYgICAgICAgICAgICMgcHJlZmF1
bHQgc2hhcmVkIG1lbW9yeSBvbiBzdGFydAogI2R5bmFtaWNfc2hhcmVkX21lbW9yeV90eXBl
ID0gcG9zaXggICAgICMgdGhlIGRlZmF1bHQgaXMgdXN1YWxseSB0aGUgZmlyc3Qgb3B0aW9u
CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIyBzdXBwb3J0ZWQg
YnkgdGhlIG9wZXJhdGluZyBzeXN0ZW06CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIyAgIHBvc2l4CmRpZmYgLS1naXQgYS9zcmMvaW5jbHVkZS9wb3J0L3Bn
X251bWEuaCBiL3NyYy9pbmNsdWRlL3BvcnQvcGdfbnVtYS5oCmluZGV4IDlkMWVhNmQwZGI4
Li5kYzlmMTNkOWZhMCAxMDA2NDQKLS0tIGEvc3JjL2luY2x1ZGUvcG9ydC9wZ19udW1hLmgK
KysrIGIvc3JjL2luY2x1ZGUvcG9ydC9wZ19udW1hLmgKQEAgLTE4LDYgKzE4LDEwIEBAIGV4
dGVybiBQR0RMTElNUE9SVCBpbnQgcGdfbnVtYV9pbml0KHZvaWQpOwogZXh0ZXJuIFBHRExM
SU1QT1JUIGludCBwZ19udW1hX3F1ZXJ5X3BhZ2VzKGludCBwaWQsIHVuc2lnbmVkIGxvbmcg
Y291bnQsIHZvaWQgKipwYWdlcywgaW50ICpzdGF0dXMpOwogZXh0ZXJuIFBHRExMSU1QT1JU
IGludCBwZ19udW1hX2dldF9tYXhfbm9kZSh2b2lkKTsKIAorZXh0ZXJuIFBHRExMSU1QT1JU
IHZvaWQgcGdfbnVtYV9zZXRfaW50ZXJsZWF2ZSh2b2lkKTsKK2V4dGVybiBQR0RMTElNUE9S
VCB2b2lkIHBnX251bWFfc2V0X2xvY2FsYWxsb2Modm9pZCk7CitleHRlcm4gUEdETExJTVBP
UlQgdm9pZCBwZ19udW1hX2ludGVybGVhdmVfbWVtb3J5KHZvaWQgKnB0ciwgU2l6ZSBzaXpl
KTsKKwogI2lmZGVmIFVTRV9MSUJOVU1BCiAKIC8qCmRpZmYgLS1naXQgYS9zcmMvaW5jbHVk
ZS9zdG9yYWdlL3BnX3NobWVtLmggYi9zcmMvaW5jbHVkZS9zdG9yYWdlL3BnX3NobWVtLmgK
aW5kZXggNWY3ZDRiODNhNjAuLjdiNTZiZDViNDRmIDEwMDY0NAotLS0gYS9zcmMvaW5jbHVk
ZS9zdG9yYWdlL3BnX3NobWVtLmgKKysrIGIvc3JjL2luY2x1ZGUvc3RvcmFnZS9wZ19zaG1l
bS5oCkBAIC00Nyw2ICs0Nyw5IEBAIGV4dGVybiBQR0RMTElNUE9SVCBpbnQgaHVnZV9wYWdl
czsKIGV4dGVybiBQR0RMTElNUE9SVCBpbnQgaHVnZV9wYWdlX3NpemU7CiBleHRlcm4gUEdE
TExJTVBPUlQgaW50IGh1Z2VfcGFnZXNfc3RhdHVzOwogCitleHRlcm4gUEdETExJTVBPUlQg
Ym9vbCBzaGFyZWRfbWVtb3J5X2ludGVybGVhdmU7CitleHRlcm4gUEdETExJTVBPUlQgYm9v
bCBzaGFyZWRfbWVtb3J5X3BvcHVsYXRlOworCiAvKiBQb3NzaWJsZSB2YWx1ZXMgZm9yIGh1
Z2VfcGFnZXMgYW5kIGh1Z2VfcGFnZXNfc3RhdHVzICovCiB0eXBlZGVmIGVudW0KIHsKZGlm
ZiAtLWdpdCBhL3NyYy9wb3J0L3BnX251bWEuYyBiL3NyYy9wb3J0L3BnX251bWEuYwppbmRl
eCA1NDBhZGEzZjhlZi4uYTkxZDMzOTAzM2YgMTAwNjQ0Ci0tLSBhL3NyYy9wb3J0L3BnX251
bWEuYworKysgYi9zcmMvcG9ydC9wZ19udW1hLmMKQEAgLTExNiw2ICsxMTYsMzMgQEAgcGdf
bnVtYV9nZXRfbWF4X25vZGUodm9pZCkKIAlyZXR1cm4gbnVtYV9tYXhfbm9kZSgpOwogfQog
CisvKgorICogU2V0IGFsbG9jYXRpb24gbWVtb3J5IHRvIGludGVybGVhdmUgb24gYWxsIG1l
bW9yeSBub2RlcyBpbiB0aGUgY3B1c2V0LgorICovCit2b2lkCitwZ19udW1hX3NldF9pbnRl
cmxlYXZlKHZvaWQpCit7CisJbnVtYV9zZXRfbWVtYmluZChudW1hX2dldF9tZW1zX2FsbG93
ZWQoKSk7Cit9CisKKy8qCisgKiBTZXQgYWxsb2NhdGlvbiBtZW1vcnkgdG8gbG9jYWxhbGxv
Yy4KKyAqLwordm9pZAorcGdfbnVtYV9zZXRfbG9jYWxhbGxvYyh2b2lkKQoreworCW51bWFf
c2V0X2xvY2FsYWxsb2MoKTsKK30KKworLyoKKyAqIFNldCBwb2xpY3kgZm9yIG1lbW9yeSB0
byBpbnRlcmxlYXZpbmcgKG9uIGFsbCBub2RlcyBwZXIgY3B1c2V0KS4KKyAqLwordm9pZAor
cGdfbnVtYV9pbnRlcmxlYXZlX21lbW9yeSh2b2lkICpwdHIsIFNpemUgc2l6ZSkKK3sKKwlu
dW1hX2ludGVybGVhdmVfbWVtb3J5KHB0ciwgc2l6ZSwgbnVtYV9nZXRfbWVtc19hbGxvd2Vk
KCkpOworfQorCiAjZWxzZQogCiAvKiBFbXB0eSB3cmFwcGVycyAqLwpAQCAtMTM4LDQgKzE2
NSwxOSBAQCBwZ19udW1hX2dldF9tYXhfbm9kZSh2b2lkKQogCXJldHVybiAwOwogfQogCit2
b2lkCitwZ19udW1hX3NldF9pbnRlcmxlYXZlKHZvaWQpCit7Cit9CisKK3ZvaWQKK3BnX251
bWFfc2V0X2xvY2FsYWxsb2Modm9pZCkKK3sKK30KKwordm9pZAorcGdfbnVtYV9pbnRlcmxl
YXZlX21lbW9yeSh2b2lkICpwdHIsIFNpemUgc2l6ZSkKK3sKK30KKwogI2VuZGlmCi0tIAoy
LjUxLjEKCg==

--------------50Dzy99YlzMzZg9svkfxMJxS--