Content-Type: multipart/mixed; boundary="------------OuUKBTjMM9m79d4fHIH3ziSQ"
Message-ID: <10ec0bc3-5933-1189-6bb8-5dec4114558e@gmail.com>
Date: Thu, 31 Oct 2024 17:00:02 +0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.4.2
Subject: Re: Inval reliability, especially for inplace updates
Content-Language: en-US
To: Noah Misch <noah@leadboat.com>
Cc: Nitin Motiani <nitinmotiani@google.com>,
 Andres Freund <andres@anarazel.de>, pgsql-hackers@postgresql.org
References: <20240618015730.jijhox2a3w3t26en@awork3.anarazel.de>
 <20240618152349.7f.nmisch@google.com> <20240831010711.67.nmisch@google.com>
 <20240930193350.04.nmisch@google.com>
 <CAH5HC9508Xvz4BVca8OCoDOv7pgHqAf85EqZkWMZ_Q2hAkebzg@mail.gmail.com>
 <20241013004511.9c.nmisch@google.com>
 <CAH5HC94r2ZKy_MhNaPUjn7NHkOFgbyBAgV2jaYSRt35BNWuOAA@mail.gmail.com>
 <CAH5HC95rn4gON8zNrFXFY-Cn4z0NZuDU_1TDwKnafX_25_C0UA@mail.gmail.com>
 <20241024025357.a8.nmisch@google.com>
 <CAH5HC97fbvr63wJO23k6N0YmQh8BK3QE8ch3cQC=dWJJBxaCjw@mail.gmail.com>
 <20241031013924.cd.nmisch@google.com>
From: Alexander Lakhin <exclusion@gmail.com>
In-Reply-To: <20241031013924.cd.nmisch@google.com>
Archived-At: 
 <https://www.postgresql.org/message-id/10ec0bc3-5933-1189-6bb8-5dec4114558e%40gmail.com>
Precedence: bulk

This is a multi-part message in MIME format.
--------------OuUKBTjMM9m79d4fHIH3ziSQ
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hello Noah,

31.10.2024 04:39, Noah Misch wrote:
> I had pushed this during the indicated week, before your mail.  Reverting it
> is an option.  Let's see if more opinions arrive.

I've accidentally discovered an incorrect behaviour caused by commit
4eac5a1fa. Running this script:
for ((j=1; j<=100; j++)); do
echo "iteration $j"

cat << 'EOF' | timeout 60 psql >>psql-$SID.log || { res=1; echo "hanged on iteration $j"; break; }
SELECT format('CREATE TABLE t%s (a int, b text);', g) FROM generate_series(1, 50) g
\gexec

SELECT format('DROP TABLE t%s;', g) FROM generate_series(1, 50) g
\gexec
EOF
done

with
autovacuum = on
autovacuum_naptime = 1s
autovacuum_vacuum_threshold = 1
autovacuum_analyze_threshold = 1

in parallel using separate servers (the full script is attached), like:
parallel -j40 --linebuffer --tag .../reproi.sh ::: `seq 40`

I can catch the following:
...
3       hanged on iteration 51
...
19      hanged on iteration 64
...
39      hanged on iteration 99

And after the script run, I see the server processes hanging:
law      1081433       1  0 16:22 ?        00:00:00 .../usr/local/pgsql/bin/postgres
law      1081452 1081433  0 16:22 ?        00:00:00 postgres: checkpointer
law      1081453 1081433  0 16:22 ?        00:00:00 postgres: background writer
law      1081460 1081433  0 16:22 ?        00:00:00 postgres: walwriter
law      1081462 1081433  0 16:22 ?        00:00:00 postgres: autovacuum launcher
law      1081464 1081433  0 16:22 ?        00:00:00 postgres: logical replication launcher
law      1143065 1081433  0 16:32 ?        00:00:00 postgres: postgres postgres [local] CREATE TABLE
law      1143263 1081433  0 16:32 ?        00:00:00 postgres: autovacuum worker postgres
law      1143320 1081433  0 16:32 ?        00:00:00 postgres: autovacuum worker postgres
law      1143403 1081433  0 16:32 ?        00:00:00 postgres: autovacuum worker


Attaching to process 1143065
...
(gdb) bt
#0  __futex_abstimed_wait_common64 (private=<optimized out>, cancel=true, abstime=0x0, op=265, expected=0, 
futex_word=0x7fed9a8171b8) at ./nptl/futex-internal.c:57
#1  __futex_abstimed_wait_common (cancel=true, private=<optimized out>, abstime=0x0, clockid=0, expected=0, 
futex_word=0x7fed9a8171b8) at ./nptl/futex-internal.c:87
#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x7fed9a8171b8, expected=expected@entry=0, 
clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=<optimized out>) at ./nptl/futex-internal.c:139
#3  0x00007feda4674c5f in do_futex_wait (sem=sem@entry=0x7fed9a8171b8, abstime=0x0, clockid=0) at 
./nptl/sem_waitcommon.c:111
#4  0x00007feda4674cf8 in __new_sem_wait_slow64 (sem=0x7fed9a8171b8, abstime=0x0, clockid=0) at ./nptl/sem_waitcommon.c:183
#5  0x0000561dd715f1fe in PGSemaphoreLock (sema=0x7fed9a8171b8) at pg_sema.c:327
#6  0x0000561dd722fe02 in LWLockAcquire (lock=0x7fed9ad9b4e4, mode=LW_SHARED) at lwlock.c:1318
#7  0x0000561dd71f8423 in LockBuffer (buffer=36, mode=1) at bufmgr.c:4182
#8  0x0000561dd6d4f506 in heapam_index_fetch_tuple (scan=0x561dd8cb6588, tid=0x561dd8cb64d0, snapshot=0x561dd8bfee28, 
slot=0x561dd8cb75a0, call_again=0x561dd8cb64d6, all_dead=0x7ffdd63842c6) at heapam_handler.c:146
...
(the full backtrace is attached)

All three autovacuum workers (1143263, 1143320, 1143403) are also waiting
for the same buffer lock:
#5  0x0000561dd715f1fe in PGSemaphoreLock (sema=0x7fed9a817338) at pg_sema.c:327
#6  0x0000561dd722fe02 in LWLockAcquire (lock=0x7fed9ad9b4e4, mode=LW_SHARED) at lwlock.c:1318
#7  0x0000561dd71f8423 in LockBuffer (buffer=36, mode=1) at bufmgr.c:4182

Probably, this can be reproduced with VACUUM pg_class/pg_type/..., but I
haven't found out the exact combination needed yet.

Also as a side note, these processes can't be terminated with SIGTERM, I
have to kill them.

Initially I saw this on a slowed down VM, but with the attached patch
applied I could reproduce it on my workstation too.

Best regards,
Alexander
--------------OuUKBTjMM9m79d4fHIH3ziSQ
Content-Type: application/x-shellscript; name="reproi.sh"
Content-Disposition: attachment; filename="reproi.sh"
Content-Transfer-Encoding: base64

IyEvYmluL2Jhc2gKCnNldCAtZQoKWyAtbiAiJDEiIF0gJiYgU0lEPSQxIHx8IFNJRD0xCgpQ
R0RBVEE9YHB3ZGAvdG1wZGIkU0lECgpQR1JPT1Q9YHB3ZGAvdG1wX2luc3RhbGwKZXhwb3J0
IFBHREFUQQoKZXhwb3J0IFBBVEg9IiRQR1JPT1QvdXNyL2xvY2FsL3Bnc3FsL2JpbjokUEFU
SCIKZXhwb3J0IExEX0xJQlJBUllfUEFUSD0iJFBHUk9PVC91c3IvbG9jYWwvcGdzcWwvbGli
IgoKcGdfY3RsIC13IC10IDUgc3RvcCAtbSBpbW1lZGlhdGUgPi9kZXYvbnVsbCB8fCB0cnVl
CgpbIC1kICIkUEdEQVRBIiBdICYmIHJtIC1yZiAiJFBHREFUQSIKTENfQUxMPSdlbl9VUy5V
VEYtOCcgaW5pdGRiIC0tdXNlcm5hbWU9cG9zdGdyZXMgPmluaXRkYiRTSUQubG9nIDI+JjEg
fHwgeyBlY2hvICJpbml0ZGIgZmFpbGVkIjsgZXhpdCAxOyB9CgpleHBvcnQgUEdQT1JUPSQo
KDE1NDMxICsgMTAjJFNJRCkpCmVjaG8gIgpwb3J0ID0gJFBHUE9SVAoKYXV0b3ZhY3V1bSA9
IG9uCmF1dG92YWN1dW1fbmFwdGltZSA9IDFzCmF1dG92YWN1dW1fdmFjdXVtX3RocmVzaG9s
ZCA9IDEKYXV0b3ZhY3V1bV9hbmFseXplX3RocmVzaG9sZCA9IDEKbG9nX2F1dG92YWN1dW1f
bWluX2R1cmF0aW9uID0gMAoKbG9nX2xpbmVfcHJlZml4ID0gJyVtIFslZF0gWyVwOiVsXVsl
Yl0gJXFbJWFdWyV2OiV4XSAnCmxvZ19jb25uZWN0aW9ucyA9IHRydWUKbG9nX2Rpc2Nvbm5l
Y3Rpb25zID0gdHJ1ZQpsb2dfc3RhdGVtZW50ID0gYWxsCmxvZ19taW5fbWVzc2FnZXMgPSBE
RUJVRzMKCmZzeW5jID0gb2ZmCiIgPj4gJFBHREFUQS9wb3N0Z3Jlc3FsLmF1dG8uY29uZgoK
WyAtZiBzZXJ2ZXIkU0lELmxvZyBdICYmIHJtIHNlcnZlciRTSUQubG9nCnBnX2N0bCAtbCBz
ZXJ2ZXIkU0lELmxvZyBzdGFydAoKZXhwb3J0IFBHREFUQUJBU0U9cG9zdGdyZXMKZXhwb3J0
IFBHVVNFUj1wb3N0Z3JlcwoKWyAtZiBwc3FsLSRTSUQubG9nIF0gJiYgcm0gcHNxbC0kU0lE
LmxvZwpyZXM9MApmb3IgKChpPTE7IGk8PTEwMDsgaSsrKSk7IGRvCmVjaG8gIml0ZXJhdGlv
biAkaSIKCmNhdCA8PCAnRU9GJyB8IHRpbWVvdXQgNjAgcHNxbCA+PnBzcWwtJFNJRC5sb2cg
fHwgeyByZXM9MTsgZWNobyAiaGFuZ2VkIG9uIGl0ZXJhdGlvbiAkaSI7IGJyZWFrOyB9ClNF
TEVDVCBmb3JtYXQoJ0NSRUFURSBUQUJMRSB0JXMgKGEgaW50LCBiIHRleHQpOycsIGcpIEZS
T00gZ2VuZXJhdGVfc2VyaWVzKDEsIDgwKSBnClxnZXhlYwoKU0VMRUNUIGZvcm1hdCgnRFJP
UCBUQUJMRSB0JXM7JywgZykgRlJPTSBnZW5lcmF0ZV9zZXJpZXMoMSwgODApIGcKXGdleGVj
CkVPRgp3YWl0CmRvbmUKClsgJHJlcyA9IDAgXSAmJiBwZ19jdGwgLXcgLXQgMTIwIHN0b3Ag
Pi9kZXYvbnVsbApleGl0ICRyZXMK
--------------OuUKBTjMM9m79d4fHIH3ziSQ
Content-Type: text/x-patch; charset=UTF-8;
 name="delay-before-lockbuffer.patch"
Content-Disposition: attachment; filename="delay-before-lockbuffer.patch"
Content-Transfer-Encoding: base64

ZGlmZiAtLWdpdCBhL3NyYy9iYWNrZW5kL2FjY2Vzcy9oZWFwL2hlYXBhbV9oYW5kbGVyLmMg
Yi9zcmMvYmFja2VuZC9hY2Nlc3MvaGVhcC9oZWFwYW1faGFuZGxlci5jCmluZGV4IDIxNjgy
NTkyNDcuLjczMGVmOWI1YTIgMTAwNjQ0Ci0tLSBhL3NyYy9iYWNrZW5kL2FjY2Vzcy9oZWFw
L2hlYXBhbV9oYW5kbGVyLmMKKysrIGIvc3JjL2JhY2tlbmQvYWNjZXNzL2hlYXAvaGVhcGFt
X2hhbmRsZXIuYwpAQCAtMTM4LDYgKzEzOCwxMCBAQCBoZWFwYW1faW5kZXhfZmV0Y2hfdHVw
bGUoc3RydWN0IEluZGV4RmV0Y2hUYWJsZURhdGEgKnNjYW4sCiAJCQloZWFwX3BhZ2VfcHJ1
bmVfb3B0KGhzY2FuLT54c19iYXNlLnJlbCwgaHNjYW4tPnhzX2NidWYpOwogCX0KIAordWlu
dDY0IHIgPSAwOworZm9yICh1aW50NjQgaSA9IDA7IGkgPCAxMDAwMDA7IGkrKykgciArPSAo
ciBeIGkpOworQXNzZXJ0KHIgIT0gMCk7CisKIAkvKiBPYnRhaW4gc2hhcmUtbG9jayBvbiB0
aGUgYnVmZmVyIHNvIHdlIGNhbiBleGFtaW5lIHZpc2liaWxpdHkgKi8KIAlMb2NrQnVmZmVy
KGhzY2FuLT54c19jYnVmLCBCVUZGRVJfTE9DS19TSEFSRSk7CiAJZ290X2hlYXBfdHVwbGUg
PSBoZWFwX2hvdF9zZWFyY2hfYnVmZmVyKHRpZCwK
--------------OuUKBTjMM9m79d4fHIH3ziSQ
Content-Type: text/plain; charset=UTF-8; name="1143065-bt.txt"
Content-Disposition: attachment; filename="1143065-bt.txt"
Content-Transfer-Encoding: base64

KGdkYikgYnQKIzAgIF9fZnV0ZXhfYWJzdGltZWRfd2FpdF9jb21tb242NCAocHJpdmF0ZT08
b3B0aW1pemVkIG91dD4sIGNhbmNlbD10cnVlLCBhYnN0aW1lPTB4MCwgb3A9MjY1LCBleHBl
Y3RlZD0wLCBmdXRleF93b3JkPTB4N2ZlZDlhODE3MWI4KSBhdCAuL25wdGwvZnV0ZXgtaW50
ZXJuYWwuYzo1NwojMSAgX19mdXRleF9hYnN0aW1lZF93YWl0X2NvbW1vbiAoY2FuY2VsPXRy
dWUsIHByaXZhdGU9PG9wdGltaXplZCBvdXQ+LCBhYnN0aW1lPTB4MCwgY2xvY2tpZD0wLCBl
eHBlY3RlZD0wLCBmdXRleF93b3JkPTB4N2ZlZDlhODE3MWI4KSBhdCAuL25wdGwvZnV0ZXgt
aW50ZXJuYWwuYzo4NwojMiAgX19HSV9fX2Z1dGV4X2Fic3RpbWVkX3dhaXRfY2FuY2VsYWJs
ZTY0IChmdXRleF93b3JkPWZ1dGV4X3dvcmRAZW50cnk9MHg3ZmVkOWE4MTcxYjgsIGV4cGVj
dGVkPWV4cGVjdGVkQGVudHJ5PTAsIGNsb2NraWQ9Y2xvY2tpZEBlbnRyeT0wLCBhYnN0aW1l
PWFic3RpbWVAZW50cnk9MHgwLCBwcml2YXRlPTxvcHRpbWl6ZWQgb3V0PikgYXQgLi9ucHRs
L2Z1dGV4LWludGVybmFsLmM6MTM5CiMzICAweDAwMDA3ZmVkYTQ2NzRjNWYgaW4gZG9fZnV0
ZXhfd2FpdCAoc2VtPXNlbUBlbnRyeT0weDdmZWQ5YTgxNzFiOCwgYWJzdGltZT0weDAsIGNs
b2NraWQ9MCkgYXQgLi9ucHRsL3NlbV93YWl0Y29tbW9uLmM6MTExCiM0ICAweDAwMDA3ZmVk
YTQ2NzRjZjggaW4gX19uZXdfc2VtX3dhaXRfc2xvdzY0IChzZW09MHg3ZmVkOWE4MTcxYjgs
IGFic3RpbWU9MHgwLCBjbG9ja2lkPTApIGF0IC4vbnB0bC9zZW1fd2FpdGNvbW1vbi5jOjE4
MwojNSAgMHgwMDAwNTYxZGQ3MTVmMWZlIGluIFBHU2VtYXBob3JlTG9jayAoc2VtYT0weDdm
ZWQ5YTgxNzFiOCkgYXQgcGdfc2VtYS5jOjMyNwojNiAgMHgwMDAwNTYxZGQ3MjJmZTAyIGlu
IExXTG9ja0FjcXVpcmUgKGxvY2s9MHg3ZmVkOWFkOWI0ZTQsIG1vZGU9TFdfU0hBUkVEKSBh
dCBsd2xvY2suYzoxMzE4CiM3ICAweDAwMDA1NjFkZDcxZjg0MjMgaW4gTG9ja0J1ZmZlciAo
YnVmZmVyPTM2LCBtb2RlPTEpIGF0IGJ1Zm1nci5jOjQxODIKIzggIDB4MDAwMDU2MWRkNmQ0
ZjUwNiBpbiBoZWFwYW1faW5kZXhfZmV0Y2hfdHVwbGUgKHNjYW49MHg1NjFkZDhjYjY1ODgs
IHRpZD0weDU2MWRkOGNiNjRkMCwgc25hcHNob3Q9MHg1NjFkZDhiZmVlMjgsIHNsb3Q9MHg1
NjFkZDhjYjc1YTAsIGNhbGxfYWdhaW49MHg1NjFkZDhjYjY0ZDYsIGFsbF9kZWFkPTB4N2Zm
ZGQ2Mzg0MmM2KSBhdCBoZWFwYW1faGFuZGxlci5jOjE0NgojOSAgMHgwMDAwNTYxZGQ2ZDZh
N2RhIGluIHRhYmxlX2luZGV4X2ZldGNoX3R1cGxlIChzY2FuPTB4NTYxZGQ4Y2I2NTg4LCB0
aWQ9MHg1NjFkZDhjYjY0ZDAsIHNuYXBzaG90PTB4NTYxZGQ4YmZlZTI4LCBzbG90PTB4NTYx
ZGQ4Y2I3NWEwLCBjYWxsX2FnYWluPTB4NTYxZGQ4Y2I2NGQ2LCBhbGxfZGVhZD0weDdmZmRk
NjM4NDJjNikKICAgIGF0IC4uLy4uLy4uLy4uL3NyYy9pbmNsdWRlL2FjY2Vzcy90YWJsZWFt
Lmg6MTIyNgojMTAgMHgwMDAwNTYxZGQ2ZDZiZDEyIGluIGluZGV4X2ZldGNoX2hlYXAgKHNj
YW49MHg1NjFkZDhjYjY0NzAsIHNsb3Q9MHg1NjFkZDhjYjc1YTApIGF0IGluZGV4YW0uYzo2
MjIKIzExIDB4MDAwMDU2MWRkNmQ2YmViOCBpbiBpbmRleF9nZXRuZXh0X3Nsb3QgKHNjYW49
MHg1NjFkZDhjYjY0NzAsIGRpcmVjdGlvbj1Gb3J3YXJkU2NhbkRpcmVjdGlvbiwgc2xvdD0w
eDU2MWRkOGNiNzVhMCkgYXQgaW5kZXhhbS5jOjY4MgojMTIgMHgwMDAwNTYxZGQ2ZDY5YjQ5
IGluIHN5c3RhYmxlX2dldG5leHQgKHN5c3NjYW49MHg1NjFkZDhjYjYzNTgpIGF0IGdlbmFt
LmM6NTEyCiMxMyAweDAwMDA1NjFkZDc0MDAyMjkgaW4gU2NhblBnUmVsYXRpb24gKHRhcmdl
dFJlbElkPTEyNTksIGluZGV4T0s9dHJ1ZSwgZm9yY2Vfbm9uX2hpc3RvcmljPWZhbHNlKSBh
dCByZWxjYWNoZS5jOjM4NQojMTQgMHgwMDAwNTYxZGQ3NDAzYzFmIGluIFJlbGF0aW9uUmVs
b2FkTmFpbGVkIChyZWxhdGlvbj0weDdmZWRhNGMwNWI4MCkgYXQgcmVsY2FjaGUuYzoyMzgx
CiMxNSAweDAwMDA1NjFkZDc0MDQxMjIgaW4gUmVsYXRpb25DbGVhclJlbGF0aW9uIChyZWxh
dGlvbj0weDdmZWRhNGMwNWI4MCwgcmVidWlsZD10cnVlKSBhdCByZWxjYWNoZS5jOjI1MjcK
IzE2IDB4MDAwMDU2MWRkNzQwMzU1NiBpbiBSZWxhdGlvbklkR2V0UmVsYXRpb24gKHJlbGF0
aW9uSWQ9MTI1OSkgYXQgcmVsY2FjaGUuYzoyMDg4CiMxNyAweDAwMDA1NjFkZDZjZDgxOWUg
aW4gcmVsYXRpb25fb3BlbiAocmVsYXRpb25JZD0xMjU5LCBsb2NrbW9kZT0xKSBhdCByZWxh
dGlvbi5jOjU5CiMxOCAweDAwMDA1NjFkZDZkYmFhNjAgaW4gdGFibGVfb3BlbiAocmVsYXRp
b25JZD0xMjU5LCBsb2NrbW9kZT0xKSBhdCB0YWJsZS5jOjQzCiMxOSAweDAwMDA1NjFkZDc0
MDAxYmMgaW4gU2NhblBnUmVsYXRpb24gKHRhcmdldFJlbElkPTU2MjM3LCBpbmRleE9LPXRy
dWUsIGZvcmNlX25vbl9oaXN0b3JpYz1mYWxzZSkgYXQgcmVsY2FjaGUuYzozNjgKIzIwIDB4
MDAwMDU2MWRkNzQwMzg4NCBpbiBSZWxhdGlvblJlbG9hZEluZGV4SW5mbyAocmVsYXRpb249
MHg3ZmVkYTRhM2U5NzgpIGF0IHJlbGNhY2hlLmM6MjI1NwojMjEgMHgwMDAwNTYxZGQ3NDA0
MWE3IGluIFJlbGF0aW9uQ2xlYXJSZWxhdGlvbiAocmVsYXRpb249MHg3ZmVkYTRhM2U5Nzgs
IHJlYnVpbGQ9dHJ1ZSkgYXQgcmVsY2FjaGUuYzoyNTUxCiMyMiAweDAwMDA1NjFkZDc0MDRh
NzYgaW4gUmVsYXRpb25GbHVzaFJlbGF0aW9uIChyZWxhdGlvbj0weDdmZWRhNGEzZTk3OCkg
YXQgcmVsY2FjaGUuYzoyODI5CiMyMyAweDAwMDA1NjFkZDc0MDRjMzggaW4gUmVsYXRpb25D
YWNoZUludmFsaWRhdGVFbnRyeSAocmVsYXRpb25JZD01NjIzNykgYXQgcmVsY2FjaGUuYzoy
OTAwCiMyNCAweDAwMDA1NjFkZDczZjQ5NzEgaW4gTG9jYWxFeGVjdXRlSW52YWxpZGF0aW9u
TWVzc2FnZSAobXNnPTB4NTYxZGQ4YmZlNTk4KSBhdCBpbnZhbC5jOjY2NgojMjUgMHgwMDAw
NTYxZGQ3M2Y0NmM2IGluIFByb2Nlc3NJbnZhbGlkYXRpb25NZXNzYWdlcyAoZ3JvdXA9MHg1
NjFkZDhiZmUyYjgsIGZ1bmM9MHg1NjFkZDczZjQ4NjEgPExvY2FsRXhlY3V0ZUludmFsaWRh
dGlvbk1lc3NhZ2U+KSBhdCBpbnZhbC5jOjUzNwojMjYgMHgwMDAwNTYxZGQ3M2Y1ODFlIGlu
IENvbW1hbmRFbmRJbnZhbGlkYXRpb25NZXNzYWdlcyAoKSBhdCBpbnZhbC5jOjEyOTEKIzI3
IDB4MDAwMDU2MWRkNmRkNmQ0ZSBpbiBBdENDSV9Mb2NhbENhY2hlICgpIGF0IHhhY3QuYzox
NTYwCiMyOCAweDAwMDA1NjFkZDZkZDY2MGIgaW4gQ29tbWFuZENvdW50ZXJJbmNyZW1lbnQg
KCkgYXQgeGFjdC5jOjExMTEKIzI5IDB4MDAwMDU2MWRkNmUzMzNhMiBpbiBpbmRleF9idWls
ZCAoaGVhcFJlbGF0aW9uPTB4N2ZlZGE0YTNhMmIwLCBpbmRleFJlbGF0aW9uPTB4N2ZlZGE0
YTNlOTc4LCBpbmRleEluZm89MHg1NjFkZDhjYjUxYjAsIGlzcmVpbmRleD1mYWxzZSwgcGFy
YWxsZWw9dHJ1ZSkgYXQgaW5kZXguYzozMTA4CiMzMCAweDAwMDA1NjFkZDZlMmZiMGYgaW4g
aW5kZXhfY3JlYXRlIChoZWFwUmVsYXRpb249MHg3ZmVkYTRhM2EyYjAsIGluZGV4UmVsYXRp
b25OYW1lPTB4N2ZmZGQ2Mzg0ZjAwICJwZ190b2FzdF81NjIzM19pbmRleCIsIGluZGV4UmVs
YXRpb25JZD01NjIzNywgcGFyZW50SW5kZXhSZWxpZD0wLCBwYXJlbnRDb25zdHJhaW50SWQ9
MCwgcmVsRmlsZU5vZGU9MCwgaW5kZXhJbmZvPTB4NTYxZGQ4Y2I1MWIwLCAKICAgIGluZGV4
Q29sTmFtZXM9MHg1NjFkZDhjYTEwMTAsIGFjY2Vzc01ldGhvZE9iamVjdElkPTQwMywgdGFi
bGVTcGFjZUlkPTAsIGNvbGxhdGlvbk9iamVjdElkPTB4N2ZmZGQ2Mzg0ZTYwLCBjbGFzc09i
amVjdElkPTB4N2ZmZGQ2Mzg0ZTY4LCBjb2xvcHRpb25zPTB4N2ZmZGQ2Mzg0ZTVjLCByZWxv
cHRpb25zPTAsIGZsYWdzPTEsIGNvbnN0cl9mbGFncz0wLCAKICAgIGFsbG93X3N5c3RlbV90
YWJsZV9tb2RzPXRydWUsIGlzX2ludGVybmFsPXRydWUsIGNvbnN0cmFpbnRJZD0weDApIGF0
IGluZGV4LmM6MTI1NAojMzEgMHgwMDAwNTYxZGQ2ZTY3NmI3IGluIGNyZWF0ZV90b2FzdF90
YWJsZSAocmVsPTB4N2ZlZGE0YzFiMTQwLCB0b2FzdE9pZD0wLCB0b2FzdEluZGV4T2lkPTAs
IHJlbG9wdGlvbnM9MCwgbG9ja21vZGU9OCwgY2hlY2s9ZmFsc2UsIE9JRE9sZFRvYXN0PTAp
IGF0IHRvYXN0aW5nLmM6MzI2CiMzMiAweDAwMDA1NjFkZDZlNjZlZWMgaW4gQ2hlY2tBbmRD
cmVhdGVUb2FzdFRhYmxlIChyZWxPaWQ9NTYyMzMsIHJlbG9wdGlvbnM9MCwgbG9ja21vZGU9
OCwgY2hlY2s9ZmFsc2UsIE9JRE9sZFRvYXN0PTApIGF0IHRvYXN0aW5nLmM6OTAKIzMzIDB4
MDAwMDU2MWRkNmU2NmU4NSBpbiBOZXdSZWxhdGlvbkNyZWF0ZVRvYXN0VGFibGUgKHJlbE9p
ZD01NjIzMywgcmVsb3B0aW9ucz0wKSBhdCB0b2FzdGluZy5jOjc3CiMzNCAweDAwMDA1NjFk
ZDcyNTFkYjkgaW4gUHJvY2Vzc1V0aWxpdHlTbG93IChwc3RhdGU9MHg1NjFkZDhjOTE2MTAs
IHBzdG10PTB4NTYxZGQ4YmQ1YzQwLCBxdWVyeVN0cmluZz0weDU2MWRkOGJkNGE0MCAiQ1JF
QVRFIFRBQkxFIHQ3MCAoYSBpbnQsIGIgdGV4dCk7IiwgY29udGV4dD1QUk9DRVNTX1VUSUxJ
VFlfVE9QTEVWRUwsIHBhcmFtcz0weDAsIHF1ZXJ5RW52PTB4MCwgCiAgICBkZXN0PTB4NTYx
ZGQ4YmQ1ZDMwLCBxYz0weDdmZmRkNjM4NTdiMCkgYXQgdXRpbGl0eS5jOjExOTkKIzM1IDB4
MDAwMDU2MWRkNzI1MWE1NCBpbiBzdGFuZGFyZF9Qcm9jZXNzVXRpbGl0eSAocHN0bXQ9MHg1
NjFkZDhiZDVjNDAsIHF1ZXJ5U3RyaW5nPTB4NTYxZGQ4YmQ0YTQwICJDUkVBVEUgVEFCTEUg
dDcwIChhIGludCwgYiB0ZXh0KTsiLCByZWFkT25seVRyZWU9ZmFsc2UsIGNvbnRleHQ9UFJP
Q0VTU19VVElMSVRZX1RPUExFVkVMLCBwYXJhbXM9MHgwLCBxdWVyeUVudj0weDAsIAogICAg
ZGVzdD0weDU2MWRkOGJkNWQzMCwgcWM9MHg3ZmZkZDYzODU3YjApIGF0IHV0aWxpdHkuYzox
MDc0CiMzNiAweDAwMDA1NjFkZDcyNTA5OTIgaW4gUHJvY2Vzc1V0aWxpdHkgKHBzdG10PTB4
NTYxZGQ4YmQ1YzQwLCBxdWVyeVN0cmluZz0weDU2MWRkOGJkNGE0MCAiQ1JFQVRFIFRBQkxF
IHQ3MCAoYSBpbnQsIGIgdGV4dCk7IiwgcmVhZE9ubHlUcmVlPWZhbHNlLCBjb250ZXh0PVBS
T0NFU1NfVVRJTElUWV9UT1BMRVZFTCwgcGFyYW1zPTB4MCwgcXVlcnlFbnY9MHgwLCBkZXN0
PTB4NTYxZGQ4YmQ1ZDMwLCAKICAgIHFjPTB4N2ZmZGQ2Mzg1N2IwKSBhdCB1dGlsaXR5LmM6
NTMwCiMzNyAweDAwMDA1NjFkZDcyNGYyM2UgaW4gUG9ydGFsUnVuVXRpbGl0eSAocG9ydGFs
PTB4NTYxZGQ4YzQ1YzYwLCBwc3RtdD0weDU2MWRkOGJkNWM0MCwgaXNUb3BMZXZlbD10cnVl
LCBzZXRIb2xkU25hcHNob3Q9ZmFsc2UsIGRlc3Q9MHg1NjFkZDhiZDVkMzAsIHFjPTB4N2Zm
ZGQ2Mzg1N2IwKSBhdCBwcXVlcnkuYzoxMTU4CiMzOCAweDAwMDA1NjFkZDcyNGY0YzkgaW4g
UG9ydGFsUnVuTXVsdGkgKHBvcnRhbD0weDU2MWRkOGM0NWM2MCwgaXNUb3BMZXZlbD10cnVl
LCBzZXRIb2xkU25hcHNob3Q9ZmFsc2UsIGRlc3Q9MHg1NjFkZDhiZDVkMzAsIGFsdGRlc3Q9
MHg1NjFkZDhiZDVkMzAsIHFjPTB4N2ZmZGQ2Mzg1N2IwKSBhdCBwcXVlcnkuYzoxMzE1CiMz
OSAweDAwMDA1NjFkZDcyNGU4ZjUgaW4gUG9ydGFsUnVuIChwb3J0YWw9MHg1NjFkZDhjNDVj
NjAsIGNvdW50PTkyMjMzNzIwMzY4NTQ3NzU4MDcsIGlzVG9wTGV2ZWw9dHJ1ZSwgcnVuX29u
Y2U9dHJ1ZSwgZGVzdD0weDU2MWRkOGJkNWQzMCwgYWx0ZGVzdD0weDU2MWRkOGJkNWQzMCwg
cWM9MHg3ZmZkZDYzODU3YjApIGF0IHBxdWVyeS5jOjc5MQojNDAgMHgwMDAwNTYxZGQ3MjQ3
MzE0IGluIGV4ZWNfc2ltcGxlX3F1ZXJ5IChxdWVyeV9zdHJpbmc9MHg1NjFkZDhiZDRhNDAg
IkNSRUFURSBUQUJMRSB0NzAgKGEgaW50LCBiIHRleHQpOyIpIGF0IHBvc3RncmVzLmM6MTI1
NAojNDEgMHgwMDAwNTYxZGQ3MjRjNWVmIGluIFBvc3RncmVzTWFpbiAoZGJuYW1lPTB4NTYx
ZGQ4YzAzYWUwICJwb3N0Z3JlcyIsIHVzZXJuYW1lPTB4NTYxZGQ4YzAzYWI4ICJwb3N0Z3Jl
cyIpIGF0IHBvc3RncmVzLmM6NDY4NgojNDIgMHgwMDAwNTYxZGQ3MTcwYWM3IGluIEJhY2tl
bmRSdW4gKHBvcnQ9MHg1NjFkZDhiZmM1YjApIGF0IHBvc3RtYXN0ZXIuYzo0NTE0CiM0MyAw
eDAwMDA1NjFkZDcxNzAzNGUgaW4gQmFja2VuZFN0YXJ0dXAgKHBvcnQ9MHg1NjFkZDhiZmM1
YjApIGF0IHBvc3RtYXN0ZXIuYzo0MjQyCiM0NCAweDAwMDA1NjFkZDcxNmMyZGIgaW4gU2Vy
dmVyTG9vcCAoKSBhdCBwb3N0bWFzdGVyLmM6MTgwOQojNDUgMHgwMDAwNTYxZGQ3MTZiYTM4
IGluIFBvc3RtYXN0ZXJNYWluIChhcmdjPTEsIGFyZ3Y9MHg1NjFkZDhiY2VjMTApIGF0IHBv
c3RtYXN0ZXIuYzoxNDgxCiM0NiAweDAwMDA1NjFkZDcwNWU5MjkgaW4gbWFpbiAoYXJnYz0x
LCBhcmd2PTB4NTYxZGQ4YmNlYzEwKSBhdCBtYWluLmM6MjAyCg==

--------------OuUKBTjMM9m79d4fHIH3ziSQ--