Subject: Re: Add Information during standby recovery conflicts
From: "Drouvot, Bertrand" <bdrouvot@amazon.com>
To: Masahiko Sawada <masahiko.sawada@2ndquadrant.com>
CC: PostgreSQL-development <pgsql-hackers@postgresql.org>
References: <9a60178c-a853-1440-2cdc-c3af916cff59@amazon.com>
 <CA+fd4k6oBCoF+4sAgWqsC5+B5+EWQfT=FyPg9zVo4wh+zTXhUg@mail.gmail.com>
 <daeecc9d-567a-cad8-298a-6f12ed7cb665@amazon.com>
 <CA+fd4k45sOD63XV-fcpi-ACS12PQa4G19_qf+cCVUcKK2nfKJw@mail.gmail.com>
 <3aededb3-8eb7-bfd6-6fd4-b6487f8f1d74@amazon.com>
 <CA+fd4k5ftD-OZPA2K0gPJxDv12mxAknAM2LVTjMRFbjZCC2FhQ@mail.gmail.com>
 <85c21e8e-4b12-9ff6-92bf-24fac2166632@amazon.com>
 <CA+fd4k7Y2H+NT-yNr1NTBKnefyWbnOP5+1cfaACTNL8WXpwhNA@mail.gmail.com>
 <45fba139-8415-ae3e-0637-f9c698432f00@amazon.com>
 <CA+fd4k6msoKMGBxuOvZ8+iC2LAdzWbGZbkYY9r0+z56FVca-AA@mail.gmail.com>
 <05621ec8-6e62-25db-14c4-50079304617d@amazon.com>
 <CA+fd4k4kqcSxuWpLOESwGsEnKLvdEpEKbspLCD5uf9SsE8bEAw@mail.gmail.com>
 <0fb96369-cc7c-cddb-a474-ca3d29904123@amazon.com>
Message-ID: <29da248a-e21c-a3eb-a051-f1ec79b13d31@amazon.com>
Date: Sun, 4 Oct 2020 13:48:22 +0200
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:78.0)
 Gecko/20100101 Thunderbird/78.3.1
MIME-Version: 1.0
In-Reply-To: <0fb96369-cc7c-cddb-a474-ca3d29904123@amazon.com>
Content-Type: multipart/mixed;
	boundary="------------9A04667B6EEAC4D4DE776B88"
Content-Language: en-US
Precedence: bulk

--------------9A04667B6EEAC4D4DE776B88
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Transfer-Encoding: 8bit


On 8/28/20 4:14 PM, Drouvot, Bertrand wrote:
>
> On 8/28/20 7:03 AM, Masahiko Sawada wrote:
>> CAUTION: This email originated from outside of the organization. Do 
>> not click links or open attachments unless you can confirm the sender 
>> and know the content is safe.
>>
>>
>>
>> On Thu, 27 Aug 2020 at 20:58, Drouvot, Bertrand <bdrouvot@amazon.com> 
>> wrote:
>>>
>>> On 8/27/20 10:16 AM, Masahiko Sawada wrote
>>>> On Mon, 10 Aug 2020 at 23:43, Drouvot, Bertrand 
>>>> <bdrouvot@amazon.com> wrote:
>>>>> Hi,
>>>>>
>>>>> On 7/31/20 7:12 AM, Masahiko Sawada wrote:
>>>>>> +   tmpWaitlist = waitlist;
>>>>>> +   while (VirtualTransactionIdIsValid(*tmpWaitlist))
>>>>>> +   {
>>>>>> +       tmpWaitlist++;
>>>>>> +   }
>>>>>> +
>>>>>> +   num_waitlist_entries = (tmpWaitlist - waitlist);
>>>>>> +
>>>>>> +   /* display wal record information */
>>>>>> +   if (log_recovery_conflicts &&
>>>>>> (TimestampDifferenceExceeds(recovery_conflicts_log_time,
>>>>>> GetCurrentTimestamp(),
>>>>>> +                                   DeadlockTimeout))) {
>>>>>> +       LogBlockedWalRecordInfo(num_waitlist_entries, reason);
>>>>>> +       recovery_conflicts_log_time = GetCurrentTimestamp();
>>>>>> +   }
>>>>>>
>>>>>> recovery_conflicts_log_time is not initialized. And shouldn't we
>>>>>> compare the current timestamp to the timestamp when the startup
>>>>>> process started waiting?
>>>>>>
>>>>>> I think we should call LogBlockedWalRecordInfo() inside of the inner
>>>>>> while loop rather than at the beginning of
>>>>>> ResolveRecoveryConflictWithVirtualXIDs(). In lock conflict cases, 
>>>>>> the
>>>>>> startup process waits until 'ltime', then enters
>>>>>> ResolveRecoveryConflictWithVirtualXIDs() after reaching 'ltime'.
>>>>>> Therefore, it makes sense to call LogBlockedWalRecordInfo() at the
>>>>>> beginning of ResolveRecoveryConflictWithVirtualXIDs(). However, in
>>>>>> snapshot and tablespace conflict cases (i.g.
>>>>>> ResolveRecoveryConflictWithSnapshot() and
>>>>>> ResolveRecoveryConflictWithTablespace()), it enters
>>>>>> ResolveRecoveryConflictWithVirtualXIDs() without waits and waits for
>>>>>> reaching ‘ltime’ inside of the inner while look. So the above
>>>>>> condition could always be false.
>>>>> That would make the information being displayed after
>>>>> max_standby_streaming_delay is reached for the multiple cases you 
>>>>> just
>>>>> described.
>>>> Sorry, it should be deadlock_timeout, not max_standby_streaming_delay.
>>>> Otherwise, the recovery conflict log message is printed when
>>>> resolution, which seems not to achieve the original purpose. Am I
>>>> missing something?
>>> Ok, I understand where the confusion is coming from.
>>>
>>> Indeed the new version is now printing the recovery conflict log 
>>> message
>>> during the conflict resolution (while the initial intention was to be
>>> warned as soon as the replay had to wait).
>>>
>>> The advantage of the new version is that it would be consistent across
>>> all the conflicts scenarios (if not, we would get messages during the
>>> resolution or when the replay started waiting, depending of the 
>>> conflict
>>> scenario).
>>>
>>> On the other hand, the cons of the new version is that we would miss
>>> messages when no resolution is needed (replay wait duration <
>>> max_standby_streaming_delay), but is that really annoying? (As no
>>> cancellation would occur)
>>>
>>> Thinking about it, i like the new version (being warned during the
>>> resolution) as we would get messages only when cancelation will occur
>>> (which is what the user might want to avoid, so the extra info would be
>>> useful).
>>>
>>> What do you think?
>> Hmm, I think we print the reason why backends are canceled even of as
>> now by ProcessInterrupts(). With this patch and related patches you
>> proposed on another thread, the startup process reports virtual xids
>> being interrupted, the reason, and LSN of blocked WAL, then processes
>> will also report its virtual xid and reason. Therefore, the new
>> information added by these patches is only the LSN of blocked WAL.
>
> That's completely right, let's come back to the original intention of 
> this patch (means, don't wait for the conflict resolution to log 
> messages).
>
> I'll submit a new version once updated.

Please find attached the new patch.

It provides the following outcomes depending on the conflict:

2020-10-04 09:08:51.923 UTC [30788] LOG:  recovery is waiting recovery 
conflict on buffer pin

OR

2020-10-04 09:52:25.832 UTC [1249] LOG:  recovery is waiting recovery 
conflict on snapshot
2020-10-04 09:52:25.832 UTC [1249] DETAIL:  Conflicting virtual 
transaction ids: 3/2, 2/4.

OR

2020-10-04 09:11:51.717 UTC [30788] LOG:  recovery is waiting recovery 
conflict on lock
2020-10-04 09:11:51.717 UTC [30788] DETAIL:  Conflicting virtual 
transaction id: 2/5.

OR

2020-10-04 09:13:04.104 UTC [30788] LOG:  recovery is resolving recovery 
conflict on database

Thanks

Bertrand


--------------9A04667B6EEAC4D4DE776B88
Content-Type: text/plain; charset="UTF-8"; x-mac-type=0; x-mac-creator=0;
	name="v1-0004-Add-Standby-Recovery-Conflicts-BlockedOnly-Info.patch"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
	filename*0="v1-0004-Add-Standby-Recovery-Conflicts-BlockedOnly-Info.patc";
	filename*1="h"

IGRvYy9zcmMvc2dtbC9jb25maWcuc2dtbCAgICAgICAgICAgICAgICAgICAgICB8IDIyICsr
KysrKysrKysKIHNyYy9iYWNrZW5kL3N0b3JhZ2UvYnVmZmVyL2J1Zm1nci5jICAgICAgICAg
ICB8IDEzICsrKysrKwogc3JjL2JhY2tlbmQvc3RvcmFnZS9pcGMvcHJvY3NpZ25hbC5jICAg
ICAgICAgIHwgMzYgKysrKysrKysrKysrKysrKwogc3JjL2JhY2tlbmQvc3RvcmFnZS9pcGMv
c3RhbmRieS5jICAgICAgICAgICAgIHwgNjEgKysrKysrKysrKysrKysrKysrKysrKysrKysr
CiBzcmMvYmFja2VuZC9zdG9yYWdlL2xtZ3IvcHJvYy5jICAgICAgICAgICAgICAgfCAgOSAr
KysrCiBzcmMvYmFja2VuZC91dGlscy9taXNjL2d1Yy5jICAgICAgICAgICAgICAgICAgfCAx
MCArKysrLQogc3JjL2JhY2tlbmQvdXRpbHMvbWlzYy9wb3N0Z3Jlc3FsLmNvbmYuc2FtcGxl
IHwgIDEgKwogc3JjL2luY2x1ZGUvc3RvcmFnZS9wcm9jc2lnbmFsLmggICAgICAgICAgICAg
IHwgIDEgKwogc3JjL2luY2x1ZGUvc3RvcmFnZS9zdGFuZGJ5LmggICAgICAgICAgICAgICAg
IHwgIDIgKwogOSBmaWxlcyBjaGFuZ2VkLCAxNTQgaW5zZXJ0aW9ucygrKSwgMSBkZWxldGlv
bigtKQpkaWZmIC0tZ2l0IGEvZG9jL3NyYy9zZ21sL2NvbmZpZy5zZ21sIGIvZG9jL3NyYy9z
Z21sL2NvbmZpZy5zZ21sCmluZGV4IGVlOTE0NzQwY2MuLjYwMjNjN2Q4YzAgMTAwNjQ0Ci0t
LSBhL2RvYy9zcmMvc2dtbC9jb25maWcuc2dtbAorKysgYi9kb2Mvc3JjL3NnbWwvY29uZmln
LnNnbWwKQEAgLTY4MzksNiArNjgzOSwyOCBAQCBsb2dfbGluZV9wcmVmaXggPSAnJW0gWyVw
XSAlcSV1QCVkLyVhICcKICAgICAgIDwvbGlzdGl0ZW0+CiAgICAgIDwvdmFybGlzdGVudHJ5
PgogCisgICAgIDx2YXJsaXN0ZW50cnkgaWQ9Imd1Yy1sb2ctcmVjb3ZlcnktY29uZmxpY3Qt
d2FpdHMiIHhyZWZsYWJlbD0ibG9nX3JlY292ZXJ5X2NvbmZsaWN0X3dhaXRzIj4KKyAgICAg
IDx0ZXJtPjx2YXJuYW1lPmxvZ19yZWNvdmVyeV9jb25mbGljdF93YWl0czwvdmFybmFtZT4g
KDx0eXBlPmJvb2xlYW48L3R5cGU+KQorICAgICAgPGluZGV4dGVybT4KKyAgICAgICA8cHJp
bWFyeT48dmFybmFtZT5sb2dfcmVjb3ZlcnlfY29uZmxpY3Rfd2FpdHM8L3Zhcm5hbWU+IGNv
bmZpZ3VyYXRpb24gcGFyYW1ldGVyPC9wcmltYXJ5PgorICAgICAgPC9pbmRleHRlcm0+Cisg
ICAgICA8L3Rlcm0+CisgICAgICA8bGlzdGl0ZW0+CisgICAgICAgPHBhcmE+CisgICAgICAg
IENvbnRyb2xzIHdoZXRoZXIgZGV0YWlsZWQgaW5mb3JtYXRpb24gaXMgcHJvZHVjZWQgd2hl
biB0aGUgCisgICAgICAgIHN0YXJ0dXAgcHJvY2VzcyBpcyByZXNvbHZpbmcgb3Igd2FpdGlu
ZyBmb3IgcmVjb3ZlcnkgY29uZmxpY3RzLgorICAgICAgICBJdCBsb2dzIG9ubHkgd2hlbiBp
dCB3YWl0cyBmb3IgbG9uZ2VyIHRoYW4gPHZhcm5hbWU+ZGVhZGxvY2tfdGltZW91dDwvdmFy
bmFtZT4uCisgICAgICAgIEZvciB0aGUgbG9jayBjb25mbGljdCBjYXNlLCBpdCBkb2VzIG5v
dCB3YWl0IGZvciA8dmFybmFtZT5kZWFkbG9ja190aW1lb3V0PC92YXJuYW1lPiB0byBiZSBy
ZWFjaGVkLgorICAgICAgIDwvcGFyYT4KKworICAgICAgIDxwYXJhPgorICAgICAgICBUaGlz
IHBhcmFtZXRlciBjYW4gb25seSBiZSBzZXQgaW4gdGhlIDxmaWxlbmFtZT5wb3N0Z3Jlc3Fs
LmNvbmY8L2ZpbGVuYW1lPgorICAgICAgICBmaWxlIG9yIG9uIHRoZSBzZXJ2ZXIgY29tbWFu
ZCBsaW5lLgorICAgICAgICBUaGUgZGVmYXVsdCBpcyA8bGl0ZXJhbD5vZmY8L2xpdGVyYWw+
LgorICAgICAgIDwvcGFyYT4KKyAgICAgIDwvbGlzdGl0ZW0+CisgICAgIDwvdmFybGlzdGVu
dHJ5PgorCiAgICAgIDx2YXJsaXN0ZW50cnkgaWQ9Imd1Yy1sb2ctcGFyYW1ldGVyLW1heC1s
ZW5ndGgiIHhyZWZsYWJlbD0ibG9nX3BhcmFtZXRlcl9tYXhfbGVuZ3RoIj4KICAgICAgIDx0
ZXJtPjx2YXJuYW1lPmxvZ19wYXJhbWV0ZXJfbWF4X2xlbmd0aDwvdmFybmFtZT4gKDx0eXBl
PmludGVnZXI8L3R5cGU+KQogICAgICAgPGluZGV4dGVybT4KZGlmZiAtLWdpdCBhL3NyYy9i
YWNrZW5kL3N0b3JhZ2UvYnVmZmVyL2J1Zm1nci5jIGIvc3JjL2JhY2tlbmQvc3RvcmFnZS9i
dWZmZXIvYnVmbWdyLmMKaW5kZXggZTU0OWZhMWQzMC4uMTUyYzIyOTJkOCAxMDA2NDQKLS0t
IGEvc3JjL2JhY2tlbmQvc3RvcmFnZS9idWZmZXIvYnVmbWdyLmMKKysrIGIvc3JjL2JhY2tl
bmQvc3RvcmFnZS9idWZmZXIvYnVmbWdyLmMKQEAgLTM4MDgsNiArMzgwOCw4IEBAIExvY2tC
dWZmZXJGb3JDbGVhbnVwKEJ1ZmZlciBidWZmZXIpCiB7CiAJQnVmZmVyRGVzYyAqYnVmSGRy
OwogCWNoYXIJICAgKm5ld19zdGF0dXMgPSBOVUxMOworCVRpbWVzdGFtcFR6IHdhaXRTdGFy
dCA9IDA7CisJYm9vbCBsb2dnZWQ7CiAKIAlBc3NlcnQoQnVmZmVySXNQaW5uZWQoYnVmZmVy
KSk7CiAJQXNzZXJ0KFBpbkNvdW50V2FpdEJ1ZiA9PSBOVUxMKTsKQEAgLTM4MjgsNiArMzgz
MCw4IEBAIExvY2tCdWZmZXJGb3JDbGVhbnVwKEJ1ZmZlciBidWZmZXIpCiAJCQkgR2V0UHJp
dmF0ZVJlZkNvdW50KGJ1ZmZlcikpOwogCiAJYnVmSGRyID0gR2V0QnVmZmVyRGVzY3JpcHRv
cihidWZmZXIgLSAxKTsKKwl3YWl0U3RhcnQgPSBHZXRDdXJyZW50VGltZXN0YW1wKCk7CisJ
bG9nZ2VkID0gZmFsc2U7CiAKIAlmb3IgKDs7KQogCXsKQEAgLTM4ODEsNiArMzg4NSwxNSBA
QCBMb2NrQnVmZmVyRm9yQ2xlYW51cChCdWZmZXIgYnVmZmVyKQogCQkJCW5ld19zdGF0dXNb
bGVuXSA9ICdcMCc7IC8qIHRydW5jYXRlIG9mZiAiIHdhaXRpbmciICovCiAJCQl9CiAKKwkJ
CWlmIChsb2dfcmVjb3ZlcnlfY29uZmxpY3Rfd2FpdHMgJiYgIWxvZ2dlZAorCQkJCQkmJiBU
aW1lc3RhbXBEaWZmZXJlbmNlRXhjZWVkcyh3YWl0U3RhcnQsCisJCQkJCQkJR2V0Q3VycmVu
dFRpbWVzdGFtcCgpLCBEZWFkbG9ja1RpbWVvdXQpKQorCQkJeworCQkJCUxvZ1JlY292ZXJ5
Q29uZmxpY3QoTlVMTCwKKwkJCQkJCVBST0NTSUdfUkVDT1ZFUllfQ09ORkxJQ1RfQlVGRkVS
UElOLCB0cnVlKTsKKwkJCQlsb2dnZWQgPSB0cnVlOworCQkJfQorCiAJCQkvKiBQdWJsaXNo
IHRoZSBidWZpZCB0aGF0IFN0YXJ0dXAgcHJvY2VzcyB3YWl0cyBvbiAqLwogCQkJU2V0U3Rh
cnR1cEJ1ZmZlclBpbldhaXRCdWZJZChidWZmZXIgLSAxKTsKIAkJCS8qIFNldCBhbGFybSBh
bmQgdGhlbiB3YWl0IHRvIGJlIHNpZ25hbGVkIGJ5IFVucGluQnVmZmVyKCkgKi8KZGlmZiAt
LWdpdCBhL3NyYy9iYWNrZW5kL3N0b3JhZ2UvaXBjL3Byb2NzaWduYWwuYyBiL3NyYy9iYWNr
ZW5kL3N0b3JhZ2UvaXBjL3Byb2NzaWduYWwuYwppbmRleCBmZmU2N2FjZWExLi41NzVlZTc5
Njg3IDEwMDY0NAotLS0gYS9zcmMvYmFja2VuZC9zdG9yYWdlL2lwYy9wcm9jc2lnbmFsLmMK
KysrIGIvc3JjL2JhY2tlbmQvc3RvcmFnZS9pcGMvcHJvY3NpZ25hbC5jCkBAIC01OTEsMyAr
NTkxLDM5IEBAIHByb2NzaWduYWxfc2lndXNyMV9oYW5kbGVyKFNJR05BTF9BUkdTKQogCiAJ
ZXJybm8gPSBzYXZlX2Vycm5vOwogfQorCitleHRlcm4gY29uc3QgY2hhciAqZ2V0X3Byb2Nz
aWduYWxfcmVhc29uX2Rlc2MoUHJvY1NpZ25hbFJlYXNvbiByZWFzb24pCisJeworCQljb25z
dCBjaGFyICpyZWFzb25EZXNjID0gInVua25vd24gcmVhc29uIjsKKworCQlzd2l0Y2ggKHJl
YXNvbikKKwkJeworCQkJY2FzZSBQUk9DU0lHX1JFQ09WRVJZX0NPTkZMSUNUX0JVRkZFUlBJ
TjoKKwkJCQlyZWFzb25EZXNjID0gImJ1ZmZlciBwaW4iOworCQkJCWJyZWFrOworCQkJY2Fz
ZSBQUk9DU0lHX1JFQ09WRVJZX0NPTkZMSUNUX0xPQ0s6CisJCQkJcmVhc29uRGVzYyA9ICJs
b2NrIjsKKwkJCQlicmVhazsKKwkJCWNhc2UgUFJPQ1NJR19SRUNPVkVSWV9DT05GTElDVF9U
QUJMRVNQQUNFOgorCQkJCXJlYXNvbkRlc2MgPSAidGFibGVzcGFjZSI7CisJCQkJYnJlYWs7
CisJCQljYXNlIFBST0NTSUdfUkVDT1ZFUllfQ09ORkxJQ1RfU05BUFNIT1Q6CisJCQkJcmVh
c29uRGVzYyA9ICJzbmFwc2hvdCI7CisJCQkJYnJlYWs7CisJCQljYXNlIFBST0NTSUdfUkVD
T1ZFUllfQ09ORkxJQ1RfU1RBUlRVUF9ERUFETE9DSzoKKwkJCQlyZWFzb25EZXNjID0gImRl
YWRsb2NrIjsKKwkJCQlicmVhazsKKwkJCWNhc2UgUFJPQ1NJR19SRUNPVkVSWV9DT05GTElD
VF9EQVRBQkFTRToKKwkJCQlyZWFzb25EZXNjID0gImRhdGFiYXNlIjsKKwkJCQlicmVhazsK
KwkJCWNhc2UgUFJPQ1NJR19DQVRDSFVQX0lOVEVSUlVQVDoKKwkJCWNhc2UgUFJPQ1NJR19O
T1RJRllfSU5URVJSVVBUOgorCQkJY2FzZSBQUk9DU0lHX1BBUkFMTEVMX01FU1NBR0U6CisJ
CQljYXNlIFBST0NTSUdfV0FMU05EX0lOSVRfU1RPUFBJTkc6CisJCQljYXNlIE5VTV9QUk9D
U0lHTkFMUzoKKwkJCQlicmVhazsKKwkJCWRlZmF1bHQ6CisJCQkJYnJlYWs7CisJCX0KKwly
ZXR1cm4gcmVhc29uRGVzYzsKK30KZGlmZiAtLWdpdCBhL3NyYy9iYWNrZW5kL3N0b3JhZ2Uv
aXBjL3N0YW5kYnkuYyBiL3NyYy9iYWNrZW5kL3N0b3JhZ2UvaXBjL3N0YW5kYnkuYwppbmRl
eCA1MmIyODA5ZGFjLi43ZDQyNDFhNGUxIDEwMDY0NAotLS0gYS9zcmMvYmFja2VuZC9zdG9y
YWdlL2lwYy9zdGFuZGJ5LmMKKysrIGIvc3JjL2JhY2tlbmQvc3RvcmFnZS9pcGMvc3RhbmRi
eS5jCkBAIC0zOSw2ICszOSw3IEBACiBpbnQJCQl2YWN1dW1fZGVmZXJfY2xlYW51cF9hZ2U7
CiBpbnQJCQltYXhfc3RhbmRieV9hcmNoaXZlX2RlbGF5ID0gMzAgKiAxMDAwOwogaW50CQkJ
bWF4X3N0YW5kYnlfc3RyZWFtaW5nX2RlbGF5ID0gMzAgKiAxMDAwOworYm9vbAkJCWxvZ19y
ZWNvdmVyeV9jb25mbGljdF93YWl0cyA9IGZhbHNlOwogCiBzdGF0aWMgSFRBQiAqUmVjb3Zl
cnlMb2NrTGlzdHM7CiAKQEAgLTIxNSw2ICsyMTYsNTIgQEAgV2FpdEV4Y2VlZHNNYXhTdGFu
ZGJ5RGVsYXkodWludDMyIHdhaXRfZXZlbnRfaW5mbykKIAlyZXR1cm4gZmFsc2U7CiB9CiAK
Ky8qCisgKiBMb2cgdGhlIHJlY292ZXJ5IGNvbmZsaWN0CisgKi8KK3ZvaWQKK0xvZ1JlY292
ZXJ5Q29uZmxpY3QoVmlydHVhbFRyYW5zYWN0aW9uSWQgKndhaXRsaXN0LCBQcm9jU2lnbmFs
UmVhc29uIHJlYXNvbiwgYm9vbCB3YWl0aW5nKQoreworCisJVmlydHVhbFRyYW5zYWN0aW9u
SWQgKnZ4aWRzOworCVN0cmluZ0luZm9EYXRhIGJ1ZjsKKwlpbnQgY291bnQ7CisKKwlpZiAo
d2FpdGxpc3QgJiYgd2FpdGluZykgeworCQl2eGlkcyA9IHdhaXRsaXN0OworCQljb3VudCA9
IDA7CisJCWluaXRTdHJpbmdJbmZvKCZidWYpOworCisJCXdoaWxlIChWaXJ0dWFsVHJhbnNh
Y3Rpb25JZElzVmFsaWQoKnZ4aWRzKSkKKwkJeworCQkJY291bnQrKzsKKwkJCWlmIChjb3Vu
dCA+IDEpCisJCQkJYXBwZW5kU3RyaW5nSW5mbygmYnVmLCIsICIpOworCisJCQlhcHBlbmRT
dHJpbmdJbmZvKCZidWYsIiVkLyV1Iix2eGlkcy0+YmFja2VuZElkLAorCQkJCQkJdnhpZHMt
PmxvY2FsVHJhbnNhY3Rpb25JZCk7CisJCQl2eGlkcysrOworCQl9CisJCWVyZXBvcnQoTE9H
LAorCQkJKGVycm1zZygicmVjb3ZlcnkgaXMgd2FpdGluZyByZWNvdmVyeSBjb25mbGljdCBv
biAlcyIsCisJCQkJCQlnZXRfcHJvY3NpZ25hbF9yZWFzb25fZGVzYyhyZWFzb24pKSwKKwkJ
CShlcnJkZXRhaWxfbG9nX3BsdXJhbCgiQ29uZmxpY3RpbmcgdmlydHVhbCB0cmFuc2FjdGlv
biBpZDogJXMuIiwKKwkJCQkJCSJDb25mbGljdGluZyB2aXJ0dWFsIHRyYW5zYWN0aW9uIGlk
czogJXMuIiwKKwkJCQkJCWNvdW50LCBidWYuZGF0YSkpKSk7CisJCXBmcmVlKGJ1Zi5kYXRh
KTsKKwl9CisJZWxzZSBpZiAoIXdhaXRsaXN0KSB7CisJCWlmICghd2FpdGluZykKKwkJCWVy
ZXBvcnQoTE9HLAorCQkJCShlcnJtc2coInJlY292ZXJ5IGlzIHJlc29sdmluZyByZWNvdmVy
eSBjb25mbGljdCBvbiAlcyIsCisJCQkJCQlnZXRfcHJvY3NpZ25hbF9yZWFzb25fZGVzYyhy
ZWFzb24pKSkpOworCQllbHNlCisJCQllcmVwb3J0KExPRywKKwkJCQkoZXJybXNnKCJyZWNv
dmVyeSBpcyB3YWl0aW5nIHJlY292ZXJ5IGNvbmZsaWN0IG9uICVzIiwKKwkJCQkJCWdldF9w
cm9jc2lnbmFsX3JlYXNvbl9kZXNjKHJlYXNvbikpKSk7CisJfQorfQorCiAvKgogICogVGhp
cyBpcyB0aGUgbWFpbiBleGVjdXRpb25lciBmb3IgYW55IHF1ZXJ5IGJhY2tlbmQgdGhhdCBj
b25mbGljdHMgd2l0aAogICogcmVjb3ZlcnkgcHJvY2Vzc2luZy4gSnVkZ2VtZW50IGhhcyBh
bHJlYWR5IGJlZW4gcGFzc2VkIG9uIGl0IHdpdGhpbgpAQCAtMjMyLDYgKzI3OSw3IEBAIFJl
c29sdmVSZWNvdmVyeUNvbmZsaWN0V2l0aFZpcnR1YWxYSURzKFZpcnR1YWxUcmFuc2FjdGlv
bklkICp3YWl0bGlzdCwKIHsKIAlUaW1lc3RhbXBUeiB3YWl0U3RhcnQgPSAwOwogCWNoYXIJ
ICAgKm5ld19zdGF0dXM7CisJYm9vbCBsb2dnZWQgPSBmYWxzZTsKIAogCS8qIEZhc3QgZXhp
dCwgdG8gYXZvaWQgYSBrZXJuZWwgY2FsbCBpZiB0aGVyZSdzIG5vIHdvcmsgdG8gYmUgZG9u
ZS4gKi8KIAlpZiAoIVZpcnR1YWxUcmFuc2FjdGlvbklkSXNWYWxpZCgqd2FpdGxpc3QpKQpA
QCAtMjQ5LDYgKzI5NywxNCBAQCBSZXNvbHZlUmVjb3ZlcnlDb25mbGljdFdpdGhWaXJ0dWFs
WElEcyhWaXJ0dWFsVHJhbnNhY3Rpb25JZCAqd2FpdGxpc3QsCiAJCS8qIHdhaXQgdW50aWwg
dGhlIHZpcnR1YWwgeGlkIGlzIGdvbmUgKi8KIAkJd2hpbGUgKCFWaXJ0dWFsWGFjdExvY2so
KndhaXRsaXN0LCBmYWxzZSkpCiAJCXsKKwkJCS8qIExvZyB0aGUgcmVjb3ZlcnkgY29uZmxp
Y3QgKi8KKwkJCWlmIChsb2dfcmVjb3ZlcnlfY29uZmxpY3Rfd2FpdHMgJiYgIWxvZ2dlZAor
CQkJCQkmJiBUaW1lc3RhbXBEaWZmZXJlbmNlRXhjZWVkcyh3YWl0U3RhcnQsCisJCQkJCQkJ
R2V0Q3VycmVudFRpbWVzdGFtcCgpLCBEZWFkbG9ja1RpbWVvdXQpKQorCQkJeworCQkJCUxv
Z1JlY292ZXJ5Q29uZmxpY3Qod2FpdGxpc3QsIHJlYXNvbiwgcmVwb3J0X3dhaXRpbmcpOwor
CQkJCWxvZ2dlZCA9IHRydWU7CisJCQl9CiAJCQkvKgogCQkJICogUmVwb3J0IHZpYSBwcyBp
ZiB3ZSBoYXZlIGJlZW4gd2FpdGluZyBmb3IgbW9yZSB0aGFuIDUwMCBtc2VjCiAJCQkgKiAo
c2hvdWxkIHRoYXQgYmUgY29uZmlndXJhYmxlPykKQEAgLTM3MCw2ICs0MjYsMTEgQEAgUmVz
b2x2ZVJlY292ZXJ5Q29uZmxpY3RXaXRoRGF0YWJhc2UoT2lkIGRiaWQpCiAJICogYmxvY2sg
ZHVyaW5nIEluaXRQb3N0Z3JlcygpIGFuZCB0aGVuIGRpc2Nvbm5lY3Qgd2hlbiB0aGV5IHNl
ZSB0aGUKIAkgKiBkYXRhYmFzZSBoYXMgYmVlbiByZW1vdmVkLgogCSAqLworCisJLyogTG9n
IHRoZSByZWNvdmVyeSBjb25mbGljdCAqLworCWlmIChsb2dfcmVjb3ZlcnlfY29uZmxpY3Rf
d2FpdHMpCisJCUxvZ1JlY292ZXJ5Q29uZmxpY3QoTlVMTCwgUFJPQ1NJR19SRUNPVkVSWV9D
T05GTElDVF9EQVRBQkFTRSwgZmFsc2UpOworCiAJd2hpbGUgKENvdW50REJCYWNrZW5kcyhk
YmlkKSA+IDApCiAJewogCQlDYW5jZWxEQkJhY2tlbmRzKGRiaWQsIFBST0NTSUdfUkVDT1ZF
UllfQ09ORkxJQ1RfREFUQUJBU0UsIHRydWUpOwpkaWZmIC0tZ2l0IGEvc3JjL2JhY2tlbmQv
c3RvcmFnZS9sbWdyL3Byb2MuYyBiL3NyYy9iYWNrZW5kL3N0b3JhZ2UvbG1nci9wcm9jLmMK
aW5kZXggODg1NjZiZDlmYS4uOGNiZDNiMzlhMyAxMDA2NDQKLS0tIGEvc3JjL2JhY2tlbmQv
c3RvcmFnZS9sbWdyL3Byb2MuYworKysgYi9zcmMvYmFja2VuZC9zdG9yYWdlL2xtZ3IvcHJv
Yy5jCkBAIC0xMDY5LDYgKzEwNjksNyBAQCBQcm9jU2xlZXAoTE9DQUxMT0NLICpsb2NhbGxv
Y2ssIExvY2tNZXRob2QgbG9ja01ldGhvZFRhYmxlKQogCVBHUFJPQwkgICAqcHJvYzsKIAlQ
R1BST0MJICAgKmxlYWRlciA9IE15UHJvYy0+bG9ja0dyb3VwTGVhZGVyOwogCWludAkJCWk7
CisJYm9vbCBsb2dnZWQ7CiAKIAkvKgogCSAqIElmIGdyb3VwIGxvY2tpbmcgaXMgaW4gdXNl
LCBsb2NrcyBoZWxkIGJ5IG1lbWJlcnMgb2YgbXkgbG9ja2luZyBncm91cApAQCAtMTI3NSwx
MSArMTI3NiwxOSBAQCBQcm9jU2xlZXAoTE9DQUxMT0NLICpsb2NhbGxvY2ssIExvY2tNZXRo
b2QgbG9ja01ldGhvZFRhYmxlKQogCSAqIHRpbWVvdXQgYW5kIHVwZGF0aW5nIHRoZSBsb2Nh
bGxvY2sgdGFibGUsIGJ1dCBpZiB3ZSBsb3NlIGNvbnRyb2wgdG8gYW4KIAkgKiBlcnJvciwg
TG9ja0Vycm9yQ2xlYW51cCB3aWxsIGZpeCB0aGF0IHVwLgogCSAqLworCWxvZ2dlZCA9IGZh
bHNlOwogCWRvCiAJewogCQlpZiAoSW5Ib3RTdGFuZGJ5KQogCQl7CiAJCQkvKiBTZXQgYSB0
aW1lciBhbmQgd2FpdCBmb3IgdGhhdCBvciBmb3IgdGhlIExvY2sgdG8gYmUgZ3JhbnRlZCAq
LworCQkJVmlydHVhbFRyYW5zYWN0aW9uSWQgKmJhY2tlbmRzOworCQkJYmFja2VuZHMgPSBH
ZXRMb2NrQ29uZmxpY3RzKCZsb2NhbGxvY2stPnRhZy5sb2NrLCBBY2Nlc3NFeGNsdXNpdmVM
b2NrLCBOVUxMKTsKKwkJCWlmIChsb2dfcmVjb3ZlcnlfY29uZmxpY3Rfd2FpdHMgJiYgIWxv
Z2dlZCkgeworCQkJCUxvZ1JlY292ZXJ5Q29uZmxpY3QoYmFja2VuZHMsIFBST0NTSUdfUkVD
T1ZFUllfQ09ORkxJQ1RfTE9DSywgdHJ1ZSk7CisJCQkJbG9nZ2VkID0gdHJ1ZTsKKwkJCX0K
KwogCQkJUmVzb2x2ZVJlY292ZXJ5Q29uZmxpY3RXaXRoTG9jayhsb2NhbGxvY2stPnRhZy5s
b2NrKTsKIAkJfQogCQllbHNlCmRpZmYgLS1naXQgYS9zcmMvYmFja2VuZC91dGlscy9taXNj
L2d1Yy5jIGIvc3JjL2JhY2tlbmQvdXRpbHMvbWlzYy9ndWMuYwppbmRleCA1OTZiY2I3Yjg0
Li5lOTRhYTc5NDE4IDEwMDY0NAotLS0gYS9zcmMvYmFja2VuZC91dGlscy9taXNjL2d1Yy5j
CisrKyBiL3NyYy9iYWNrZW5kL3V0aWxzL21pc2MvZ3VjLmMKQEAgLTE1NzAsNyArMTU3MCwx
NSBAQCBzdGF0aWMgc3RydWN0IGNvbmZpZ19ib29sIENvbmZpZ3VyZU5hbWVzQm9vbFtdID0K
IAkJZmFsc2UsCiAJCU5VTEwsIE5VTEwsIE5VTEwKIAl9LAotCisJeworCQl7ImxvZ19yZWNv
dmVyeV9jb25mbGljdF93YWl0cyIsIFBHQ19TSUdIVVAsIExPR0dJTkdfV0hBVCwKKwkJCWdl
dHRleHRfbm9vcCgiTG9ncyBzdGFuZGJ5IHJlY292ZXJ5IGNvbmZsaWN0cyByZXNvbHV0aW9u
LiIpLAorCQkJTlVMTAorCQl9LAorCQkmbG9nX3JlY292ZXJ5X2NvbmZsaWN0X3dhaXRzLAor
CQlmYWxzZSwKKwkJTlVMTCwgTlVMTCwgTlVMTAorCX0sCiAJewogCQl7ImxvZ19ob3N0bmFt
ZSIsIFBHQ19TSUdIVVAsIExPR0dJTkdfV0hBVCwKIAkJCWdldHRleHRfbm9vcCgiTG9ncyB0
aGUgaG9zdCBuYW1lIGluIHRoZSBjb25uZWN0aW9uIGxvZ3MuIiksCmRpZmYgLS1naXQgYS9z
cmMvYmFja2VuZC91dGlscy9taXNjL3Bvc3RncmVzcWwuY29uZi5zYW1wbGUgYi9zcmMvYmFj
a2VuZC91dGlscy9taXNjL3Bvc3RncmVzcWwuY29uZi5zYW1wbGUKaW5kZXggOWNiNTcxZjdj
Yy4uYzJlMWFmNzE0ZCAxMDA2NDQKLS0tIGEvc3JjL2JhY2tlbmQvdXRpbHMvbWlzYy9wb3N0
Z3Jlc3FsLmNvbmYuc2FtcGxlCisrKyBiL3NyYy9iYWNrZW5kL3V0aWxzL21pc2MvcG9zdGdy
ZXNxbC5jb25mLnNhbXBsZQpAQCAtNTUzLDYgKzU1Myw3IEBACiAJCQkJCSMgICAlJSA9ICcl
JwogCQkJCQkjIGUuZy4gJzwldSUlJWQ+ICcKICNsb2dfbG9ja193YWl0cyA9IG9mZgkJCSMg
bG9nIGxvY2sgd2FpdHMgPj0gZGVhZGxvY2tfdGltZW91dAorI2xvZ19yZWNvdmVyeV9jb25m
bGljdF93YWl0cyA9IG9mZgkjIGxvZyBzdGFuZGJ5IHJlY292ZXJ5IGNvbmZsaWN0cwogI2xv
Z19wYXJhbWV0ZXJfbWF4X2xlbmd0aCA9IC0xCQkjIHdoZW4gbG9nZ2luZyBzdGF0ZW1lbnRz
LCBsaW1pdCBsb2dnZWQKIAkJCQkJIyBiaW5kLXBhcmFtZXRlciB2YWx1ZXMgdG8gTiBieXRl
czsKIAkJCQkJIyAtMSBtZWFucyBwcmludCBpbiBmdWxsLCAwIGRpc2FibGVzCmRpZmYgLS1n
aXQgYS9zcmMvaW5jbHVkZS9zdG9yYWdlL3Byb2NzaWduYWwuaCBiL3NyYy9pbmNsdWRlL3N0
b3JhZ2UvcHJvY3NpZ25hbC5oCmluZGV4IDVjYjM5Njk3ZjMuLjdjZDk4ODI2M2QgMTAwNjQ0
Ci0tLSBhL3NyYy9pbmNsdWRlL3N0b3JhZ2UvcHJvY3NpZ25hbC5oCisrKyBiL3NyYy9pbmNs
dWRlL3N0b3JhZ2UvcHJvY3NpZ25hbC5oCkBAIC03MSw1ICs3MSw2IEBAIGV4dGVybiB2b2lk
IFdhaXRGb3JQcm9jU2lnbmFsQmFycmllcih1aW50NjQgZ2VuZXJhdGlvbik7CiBleHRlcm4g
dm9pZCBQcm9jZXNzUHJvY1NpZ25hbEJhcnJpZXIodm9pZCk7CiAKIGV4dGVybiB2b2lkIHBy
b2NzaWduYWxfc2lndXNyMV9oYW5kbGVyKFNJR05BTF9BUkdTKTsKK2V4dGVybiBjb25zdCBj
aGFyICpnZXRfcHJvY3NpZ25hbF9yZWFzb25fZGVzYyhQcm9jU2lnbmFsUmVhc29uIHJlYXNv
bik7CiAKICNlbmRpZgkJCQkJCQkvKiBQUk9DU0lHTkFMX0ggKi8KZGlmZiAtLWdpdCBhL3Ny
Yy9pbmNsdWRlL3N0b3JhZ2Uvc3RhbmRieS5oIGIvc3JjL2luY2x1ZGUvc3RvcmFnZS9zdGFu
ZGJ5LmgKaW5kZXggZmFhZjFkMzgxNy4uOGUzYjAyZjdiZSAxMDA2NDQKLS0tIGEvc3JjL2lu
Y2x1ZGUvc3RvcmFnZS9zdGFuZGJ5LmgKKysrIGIvc3JjL2luY2x1ZGUvc3RvcmFnZS9zdGFu
ZGJ5LmgKQEAgLTIzLDYgKzIzLDcgQEAKIGV4dGVybiBpbnQJdmFjdXVtX2RlZmVyX2NsZWFu
dXBfYWdlOwogZXh0ZXJuIGludAltYXhfc3RhbmRieV9hcmNoaXZlX2RlbGF5OwogZXh0ZXJu
IGludAltYXhfc3RhbmRieV9zdHJlYW1pbmdfZGVsYXk7CitleHRlcm4gYm9vbAlsb2dfcmVj
b3ZlcnlfY29uZmxpY3Rfd2FpdHM7CiAKIGV4dGVybiB2b2lkIEluaXRSZWNvdmVyeVRyYW5z
YWN0aW9uRW52aXJvbm1lbnQodm9pZCk7CiBleHRlcm4gdm9pZCBTaHV0ZG93blJlY292ZXJ5
VHJhbnNhY3Rpb25FbnZpcm9ubWVudCh2b2lkKTsKQEAgLTM4LDYgKzM5LDcgQEAgZXh0ZXJu
IHZvaWQgQ2hlY2tSZWNvdmVyeUNvbmZsaWN0RGVhZGxvY2sodm9pZCk7CiBleHRlcm4gdm9p
ZCBTdGFuZGJ5RGVhZExvY2tIYW5kbGVyKHZvaWQpOwogZXh0ZXJuIHZvaWQgU3RhbmRieVRp
bWVvdXRIYW5kbGVyKHZvaWQpOwogZXh0ZXJuIHZvaWQgU3RhbmRieUxvY2tUaW1lb3V0SGFu
ZGxlcih2b2lkKTsKK2V4dGVybiB2b2lkIExvZ1JlY292ZXJ5Q29uZmxpY3QoVmlydHVhbFRy
YW5zYWN0aW9uSWQgKndhaXRsaXN0LCBQcm9jU2lnbmFsUmVhc29uIHJlYXNvbiwgYm9vbCB3
YWl0aW5nKTsKIAogLyoKICAqIFN0YW5kYnkgUm1nciAoUk1fU1RBTkRCWV9JRCkK
--------------9A04667B6EEAC4D4DE776B88--