Subject: Re: Minimal logical decoding on standbys
From: "Drouvot, Bertrand" <bdrouvot@amazon.com>
To: Andres Freund <andres@anarazel.de>
CC: Ibrar Ahmed <ibrar.ahmad@gmail.com>, Amit Khandekar
	<amitdkhan.pg@gmail.com>, <fabriziomello@gmail.com>, Alvaro Herrera
	<alvherre@2ndquadrant.com>, tushar <tushar.ahuja@enterprisedb.com>, "[pgdg]
 Robert Haas" <robertmhaas@gmail.com>, Rahila Syed
	<rahila.syed@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
References: <20210407170954.xbovgh3xdrnogg7w@alap3.anarazel.de>
 <20210407203218.pestfxu7watrnog4@alap3.anarazel.de>
 <20210408034713.auf7oaxyymu2kc7n@alap3.anarazel.de>
 <26c6f320-98f0-253c-f8b5-df1e7c1f6315@amazon.com>
 <8d263a6b-a99f-0f9b-43ba-e8967c386f2b@amazon.com>
 <2feee395-959b-6d53-369a-c00f8f2fa6c5@amazon.com>
 <CALtqXTfXsyq-G7x3Ti3bm4KgpepqZA_-ZukzJ5W5vFLjRrhGSg@mail.gmail.com>
 <fa90206a-6bf1-2121-cf0c-15b0734a2731@amazon.com>
 <20210727172257.ekvkspdqaeh2etfg@alap3.anarazel.de>
 <69aad0bf-697a-04e1-df6c-0920ec8fa528@amazon.com>
 <20210802160133.uugcce5ql4m5mv5m@alap3.anarazel.de>
 <7094ac62-340f-9b23-3af6-e24fe08c8697@amazon.com>
Message-ID: <09dd62cd-6f07-e15b-164c-4ffb1569f837@amazon.com>
Date: Wed, 8 Sep 2021 12:08:43 +0200
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <7094ac62-340f-9b23-3af6-e24fe08c8697@amazon.com>
Content-Type: multipart/mixed;
	boundary="------------393460E7D763AE325A9CCC4F"
Content-Language: en-US
Archived-At: 
 <https://www.postgresql.org/message-id/09dd62cd-6f07-e15b-164c-4ffb1569f837%40amazon.com>
Precedence: bulk

--------------393460E7D763AE325A9CCC4F
Content-Type: multipart/alternative;
	boundary="------------B95E7255F0818A232FAA81EE"

--------------B95E7255F0818A232FAA81EE
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Transfer-Encoding: 8bit

Hi Andres,

On 8/6/21 1:27 PM, Drouvot, Bertrand wrote:
> Hi,
>
> On 8/2/21 6:01 PM, Andres Freund wrote:
>> While working on this I found a, somewhat substantial, issue:
>>> If so, do you already have in mind a way to handle this? (I thought you
>>> already had in mind a way to handle it so the question)
>> Yes. I think we need to add a condition variable to be able to wait for
>> WAL positions to change. Either multiple condition variables (one for
>> the flush position, one for the replay position), or one that just
>> changes more often. That way one can wait for apply without a race
>> condition.
>>
> Thanks for the feedback.
>
> Wouldn't a condition variable on the replay position be enough? I 
> don't get why the proposed one on the flush position is needed.

Please find enclosed a patch proposal to address those corner cases.

I think (but may be wrong) that the condition variable on the flush 
position would be needed only for the walsender(s) on non Standby node, 
that's why:

  * I made use of a condition variable on the replay position only.

  * The walsender waits on it in WalSndWaitForWal() only if recovery is
    in progress.

For simplicity to discuss those corner cases, this is a dedicated patch 
that can be applied on top of v23 patches shared previously.

Thanks

Bertrand

>
>>> if not, what kind of additional  tests would you like to see?
>> A few catalog rows being removed (e.g. due to DELETE and then VACUUM
>> *without* full) and a standby without hot_standby_feedback catching
>> that.
>
> Test added in v23 attached.
>
> Thanks
>
> Bertrand
>

--------------B95E7255F0818A232FAA81EE
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: 8bit

<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <p>Hi Andres,<br>
    </p>
    <div class="moz-cite-prefix">On 8/6/21 1:27 PM, Drouvot, Bertrand
      wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:7094ac62-340f-9b23-3af6-e24fe08c8697@amazon.com">Hi,
      <br>
      <br>
      On 8/2/21 6:01 PM, Andres Freund wrote:
      <br>
      <blockquote type="cite">While working on this I found a, somewhat
        substantial, issue:
        <br>
        <blockquote type="cite">If so, do you already have in mind a way
          to handle this? (I thought you
          <br>
          already had in mind a way to handle it so the question)
          <br>
        </blockquote>
        Yes. I think we need to add a condition variable to be able to
        wait for
        <br>
        WAL positions to change. Either multiple condition variables
        (one for
        <br>
        the flush position, one for the replay position), or one that
        just
        <br>
        changes more often. That way one can wait for apply without a
        race
        <br>
        condition.
        <br>
        <br>
      </blockquote>
      Thanks for the feedback.
      <br>
      <br>
      Wouldn't a condition variable on the replay position be enough? I
      don't get why the proposed one on the flush position is needed.
      <br>
    </blockquote>
    <p>Please find enclosed a patch proposal to address those corner
      cases. <br>
    </p>
    <p>I think (but may be wrong) that the condition variable on the
      flush position would be needed only for the walsender(s) on non
      Standby node, that's why:<br>
    </p>
    <ul>
      <li>I made use of a condition variable on the replay position
        only.</li>
    </ul>
    <ul>
      <li>The walsender waits on it in WalSndWaitForWal() only if
        recovery is in progress.</li>
    </ul>
    <p>For simplicity to discuss those corner cases, this is a dedicated
      patch that can be applied on top of v23 patches shared previously.</p>
    <p>Thanks</p>
    <p>Bertrand<br>
    </p>
    <blockquote type="cite"
      cite="mid:7094ac62-340f-9b23-3af6-e24fe08c8697@amazon.com">
      <br>
      <blockquote type="cite">
        <blockquote type="cite">if not, what kind of additional  tests
          would you like to see?
          <br>
        </blockquote>
        A few catalog rows being removed (e.g. due to DELETE and then
        VACUUM
        <br>
        *without* full) and a standby without hot_standby_feedback
        catching
        <br>
        that.
        <br>
      </blockquote>
      <br>
      Test added in v23 attached.
      <br>
      <br>
      Thanks
      <br>
      <br>
      Bertrand
      <br>
      <br>
    </blockquote>
  </body>
</html>

--------------B95E7255F0818A232FAA81EE--

--------------393460E7D763AE325A9CCC4F
Content-Type: text/plain; charset="UTF-8"; x-mac-type=0; x-mac-creator=0;
	name="v23-0006-Logical-Decoding-On-Standby-WalSender-Corner-Case.patch"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
	filename*0="v23-0006-Logical-Decoding-On-Standby-WalSender-Corner-Case.p";
	filename*1="atch"

ZGlmZiAtLWdpdCBhL3NyYy9iYWNrZW5kL2FjY2Vzcy90cmFuc2FtL3hsb2cuYyBiL3NyYy9i
YWNrZW5kL2FjY2Vzcy90cmFuc2FtL3hsb2cuYwppbmRleCA2MzMyY2E1ZDUzLi5jZTUzYTIz
NmNjIDEwMDY0NAotLS0gYS9zcmMvYmFja2VuZC9hY2Nlc3MvdHJhbnNhbS94bG9nLmMKKysr
IGIvc3JjL2JhY2tlbmQvYWNjZXNzL3RyYW5zYW0veGxvZy5jCkBAIC03MjcsNiArNzI3LDcg
QEAgdHlwZWRlZiBzdHJ1Y3QgWExvZ0N0bERhdGEKIH0gWExvZ0N0bERhdGE7CiAKIHN0YXRp
YyBYTG9nQ3RsRGF0YSAqWExvZ0N0bCA9IE5VTEw7CitYTG9nQ3RsQ3ZEYXRhICpYTG9nQ3Rs
Q3YgPSBOVUxMOwogCiAvKiBhIHByaXZhdGUgY29weSBvZiBYTG9nQ3RsLT5JbnNlcnQuV0FM
SW5zZXJ0TG9ja3MsIGZvciBjb252ZW5pZW5jZSAqLwogc3RhdGljIFdBTEluc2VydExvY2tQ
YWRkZWQgKldBTEluc2VydExvY2tzID0gTlVMTDsKQEAgLTUxNDEsNyArNTE0Miw4IEBAIHZv
aWQKIFhMT0dTaG1lbUluaXQodm9pZCkKIHsKIAlib29sCQlmb3VuZENGaWxlLAotCQkJCWZv
dW5kWExvZzsKKwkJCQlmb3VuZFhMb2csCisJCQkJZm91bmRYTG9nQ3Y7CiAJY2hhcgkgICAq
YWxsb2NwdHI7CiAJaW50CQkJaTsKIAlDb250cm9sRmlsZURhdGEgKmxvY2FsQ29udHJvbEZp
bGU7CkBAIC01MTY2LDE0ICs1MTY4LDE3IEBAIFhMT0dTaG1lbUluaXQodm9pZCkKIAlYTG9n
Q3RsID0gKFhMb2dDdGxEYXRhICopCiAJCVNobWVtSW5pdFN0cnVjdCgiWExPRyBDdGwiLCBY
TE9HU2htZW1TaXplKCksICZmb3VuZFhMb2cpOwogCisJWExvZ0N0bEN2ID0gKFhMb2dDdGxD
dkRhdGEgKikKKwkJU2htZW1Jbml0U3RydWN0KCJYTE9HIEN2IEN0bCIsIHNpemVvZihYTG9n
Q3RsQ3ZEYXRhKSwgJmZvdW5kWExvZ0N2KTsKKwogCWxvY2FsQ29udHJvbEZpbGUgPSBDb250
cm9sRmlsZTsKIAlDb250cm9sRmlsZSA9IChDb250cm9sRmlsZURhdGEgKikKIAkJU2htZW1J
bml0U3RydWN0KCJDb250cm9sIEZpbGUiLCBzaXplb2YoQ29udHJvbEZpbGVEYXRhKSwgJmZv
dW5kQ0ZpbGUpOwogCi0JaWYgKGZvdW5kQ0ZpbGUgfHwgZm91bmRYTG9nKQorCWlmIChmb3Vu
ZENGaWxlIHx8IGZvdW5kWExvZyB8fCBmb3VuZFhMb2dDdikKIAl7Ci0JCS8qIGJvdGggc2hv
dWxkIGJlIHByZXNlbnQgb3IgbmVpdGhlciAqLwotCQlBc3NlcnQoZm91bmRDRmlsZSAmJiBm
b3VuZFhMb2cpOworCQkvKiBBbGwgc2hvdWxkIGJlIHByZXNlbnQgb3IgbmVpdGhlciAqLwor
CQlBc3NlcnQoZm91bmRDRmlsZSAmJiBmb3VuZFhMb2cgJiYgZm91bmRYTG9nQ3YpOwogCiAJ
CS8qIEluaXRpYWxpemUgbG9jYWwgY29weSBvZiBXQUxJbnNlcnRMb2NrcyAqLwogCQlXQUxJ
bnNlcnRMb2NrcyA9IFhMb2dDdGwtPkluc2VydC5XQUxJbnNlcnRMb2NrczsKQEAgLTUxODMs
NiArNTE4OCw3IEBAIFhMT0dTaG1lbUluaXQodm9pZCkKIAkJcmV0dXJuOwogCX0KIAltZW1z
ZXQoWExvZ0N0bCwgMCwgc2l6ZW9mKFhMb2dDdGxEYXRhKSk7CisJbWVtc2V0KFhMb2dDdGxD
diwgMCwgc2l6ZW9mKFhMb2dDdGxDdkRhdGEpKTsKIAogCS8qCiAJICogQWxyZWFkeSBoYXZl
IHJlYWQgY29udHJvbCBmaWxlIGxvY2FsbHksIHVubGVzcyBpbiBib290c3RyYXAgbW9kZS4g
TW92ZQpAQCAtNTI0NCw2ICs1MjUwLDcgQEAgWExPR1NobWVtSW5pdCh2b2lkKQogCVNwaW5M
b2NrSW5pdCgmWExvZ0N0bC0+dWxzbl9sY2spOwogCUluaXRTaGFyZWRMYXRjaCgmWExvZ0N0
bC0+cmVjb3ZlcnlXYWtldXBMYXRjaCk7CiAJQ29uZGl0aW9uVmFyaWFibGVJbml0KCZYTG9n
Q3RsLT5yZWNvdmVyeU5vdFBhdXNlZENWKTsKKwlDb25kaXRpb25WYXJpYWJsZUluaXQoJlhM
b2dDdGxDdi0+cmVwbGF5ZWRDVik7CiB9CiAKIC8qCkBAIC03NTMzLDYgKzc1NDAsMTEgQEAg
U3RhcnR1cFhMT0codm9pZCkKIAkJCQlYTG9nQ3RsLT5sYXN0UmVwbGF5ZWRUTEkgPSBUaGlz
VGltZUxpbmVJRDsKIAkJCQlTcGluTG9ja1JlbGVhc2UoJlhMb2dDdGwtPmluZm9fbGNrKTsK
IAorCQkJCS8qCisJCQkJICogd2FrZSB1cCB3YWxzZW5kZXIocykgdXNlZCBieSBsb2dpY2Fs
IGRlY29kaW5nIG9uIHN0YW5kYnkuCisJCQkJICovCisJCQkJQ29uZGl0aW9uVmFyaWFibGVC
cm9hZGNhc3QoJlhMb2dDdGxDdi0+cmVwbGF5ZWRDVik7CisKIAkJCQkvKgogCQkJCSAqIElm
IHJtX3JlZG8gY2FsbGVkIFhMb2dSZXF1ZXN0V2FsUmVjZWl2ZXJSZXBseSwgdGhlbiB3ZSB3
YWtlCiAJCQkJICogdXAgdGhlIHJlY2VpdmVyIHNvIHRoYXQgaXQgbm90aWNlcyB0aGUgdXBk
YXRlZApkaWZmIC0tZ2l0IGEvc3JjL2JhY2tlbmQvcmVwbGljYXRpb24vd2Fsc2VuZGVyLmMg
Yi9zcmMvYmFja2VuZC9yZXBsaWNhdGlvbi93YWxzZW5kZXIuYwppbmRleCBkYTI1MzNlMWM5
Li40ZDA3ZDI4YTMxIDEwMDY0NAotLS0gYS9zcmMvYmFja2VuZC9yZXBsaWNhdGlvbi93YWxz
ZW5kZXIuYworKysgYi9zcmMvYmFja2VuZC9yZXBsaWNhdGlvbi93YWxzZW5kZXIuYwpAQCAt
MTM5NCw2ICsxMzk0LDcgQEAgV2FsU25kV2FpdEZvcldhbChYTG9nUmVjUHRyIGxvYykKIHsK
IAlpbnQJCQl3YWtlRXZlbnRzOwogCXN0YXRpYyBYTG9nUmVjUHRyIFJlY2VudEZsdXNoUHRy
ID0gSW52YWxpZFhMb2dSZWNQdHI7CisJWExvZ0N0bEN2RGF0YSAqeGxvZ2N0bGN2ID0gWExv
Z0N0bEN2OwogCiAJLyoKIAkgKiBGYXN0IHBhdGggdG8gYXZvaWQgYWNxdWlyaW5nIHRoZSBz
cGlubG9jayBpbiBjYXNlIHdlIGFscmVhZHkga25vdyB3ZQpAQCAtMTQxMiw3ICsxNDEzLDYg
QEAgV2FsU25kV2FpdEZvcldhbChYTG9nUmVjUHRyIGxvYykKIAogCWZvciAoOzspCiAJewot
CQlsb25nCQlzbGVlcHRpbWU7CiAKIAkJLyogQ2xlYXIgYW55IGFscmVhZHktcGVuZGluZyB3
YWtldXBzICovCiAJCVJlc2V0TGF0Y2goTXlMYXRjaCk7CkBAIC0xNDk2LDIwICsxNDk2LDMz
IEBAIFdhbFNuZFdhaXRGb3JXYWwoWExvZ1JlY1B0ciBsb2MpCiAJCVdhbFNuZEtlZXBhbGl2
ZUlmTmVjZXNzYXJ5KCk7CiAKIAkJLyoKLQkJICogU2xlZXAgdW50aWwgc29tZXRoaW5nIGhh
cHBlbnMgb3Igd2UgdGltZSBvdXQuICBBbHNvIHdhaXQgZm9yIHRoZQotCQkgKiBzb2NrZXQg
YmVjb21pbmcgd3JpdGFibGUsIGlmIHRoZXJlJ3Mgc3RpbGwgcGVuZGluZyBvdXRwdXQuCisJ
CSAqIFdoZW4gbm90IGluIHJlY292ZXJ5LCBzbGVlcCB1bnRpbCBzb21ldGhpbmcgaGFwcGVu
cyBvciB3ZSB0aW1lIG91dC4KKwkJICogQWxzbyB3YWl0IGZvciB0aGUgc29ja2V0IGJlY29t
aW5nIHdyaXRhYmxlLCBpZiB0aGVyZSdzIHN0aWxsIHBlbmRpbmcgb3V0cHV0LgogCQkgKiBP
dGhlcndpc2Ugd2UgbWlnaHQgc2l0IG9uIHNlbmRhYmxlIG91dHB1dCBkYXRhIHdoaWxlIHdh
aXRpbmcgZm9yCiAJCSAqIG5ldyBXQUwgdG8gYmUgZ2VuZXJhdGVkLiAgKEJ1dCBpZiB3ZSBo
YXZlIG5vdGhpbmcgdG8gc2VuZCwgd2UgZG9uJ3QKIAkJICogd2FudCB0byB3YWtlIG9uIHNv
Y2tldC13cml0YWJsZS4pCiAJCSAqLwotCQlzbGVlcHRpbWUgPSBXYWxTbmRDb21wdXRlU2xl
ZXB0aW1lKEdldEN1cnJlbnRUaW1lc3RhbXAoKSk7CisJCWlmICghUmVjb3ZlcnlJblByb2dy
ZXNzKCkpCisJCXsKKwkJCWxvbmcJCXNsZWVwdGltZTsKKwkJCXNsZWVwdGltZSA9IFdhbFNu
ZENvbXB1dGVTbGVlcHRpbWUoR2V0Q3VycmVudFRpbWVzdGFtcCgpKTsKIAotCQl3YWtlRXZl
bnRzID0gV0xfU09DS0VUX1JFQURBQkxFOworCQkJd2FrZUV2ZW50cyA9IFdMX1NPQ0tFVF9S
RUFEQUJMRTsKIAotCQlpZiAocHFfaXNfc2VuZF9wZW5kaW5nKCkpCi0JCQl3YWtlRXZlbnRz
IHw9IFdMX1NPQ0tFVF9XUklURUFCTEU7CisJCQlpZiAocHFfaXNfc2VuZF9wZW5kaW5nKCkp
CisJCQkJd2FrZUV2ZW50cyB8PSBXTF9TT0NLRVRfV1JJVEVBQkxFOwogCi0JCVdhbFNuZFdh
aXQod2FrZUV2ZW50cywgc2xlZXB0aW1lLCBXQUlUX0VWRU5UX1dBTF9TRU5ERVJfV0FJVF9X
QUwpOworCQkJV2FsU25kV2FpdCh3YWtlRXZlbnRzLCBzbGVlcHRpbWUgKiAxMCwgV0FJVF9F
VkVOVF9XQUxfU0VOREVSX1dBSVRfV0FMKTsKKwkJfQorCQllbHNlCisJCS8qCisJCSAqIFdl
IGFyZSBpbiB0aGUgbG9naWNhbCBkZWNvZGluZyBvbiBzdGFuZGJ5IGNhc2UuCisJCSAqIFdl
IGFyZSB3YWl0aW5nIGZvciB0aGUgc3RhcnR1cCBwcm9jZXNzIHRvIHJlcGxheSB3YWwgcmVj
b3JkKHMpLgorCQkgKi8KKwkJeworCQkJQ29uZGl0aW9uVmFyaWFibGVQcmVwYXJlVG9TbGVl
cCgmeGxvZ2N0bGN2LT5yZXBsYXllZENWKTsKKwkJCUNvbmRpdGlvblZhcmlhYmxlU2xlZXAo
Jnhsb2djdGxjdi0+cmVwbGF5ZWRDViwgV0FJVF9FVkVOVF9XQUxfU0VOREVSX1dBSVRfUkVQ
TEFZKTsKKwkJfQogCX0KIAogCS8qIHJlYWN0aXZhdGUgbGF0Y2ggc28gV2FsU25kTG9vcCBr
bm93cyB0byBjb250aW51ZSAqLwpkaWZmIC0tZ2l0IGEvc3JjL2JhY2tlbmQvdXRpbHMvYWN0
aXZpdHkvd2FpdF9ldmVudC5jIGIvc3JjL2JhY2tlbmQvdXRpbHMvYWN0aXZpdHkvd2FpdF9l
dmVudC5jCmluZGV4IGVmN2U2YmZiNzcuLjZlNzRlNjA2MzAgMTAwNjQ0Ci0tLSBhL3NyYy9i
YWNrZW5kL3V0aWxzL2FjdGl2aXR5L3dhaXRfZXZlbnQuYworKysgYi9zcmMvYmFja2VuZC91
dGlscy9hY3Rpdml0eS93YWl0X2V2ZW50LmMKQEAgLTQ0OCw2ICs0NDgsOSBAQCBwZ3N0YXRf
Z2V0X3dhaXRfaXBjKFdhaXRFdmVudElQQyB3KQogCQljYXNlIFdBSVRfRVZFTlRfV0FMX1JF
Q0VJVkVSX1dBSVRfU1RBUlQ6CiAJCQlldmVudF9uYW1lID0gIldhbFJlY2VpdmVyV2FpdFN0
YXJ0IjsKIAkJCWJyZWFrOworCQljYXNlIFdBSVRfRVZFTlRfV0FMX1NFTkRFUl9XQUlUX1JF
UExBWToKKwkJCWV2ZW50X25hbWUgPSAiV2FsUmVjZWl2ZXJXYWl0UmVwbGF5IjsKKwkJCWJy
ZWFrOwogCQljYXNlIFdBSVRfRVZFTlRfWEFDVF9HUk9VUF9VUERBVEU6CiAJCQlldmVudF9u
YW1lID0gIlhhY3RHcm91cFVwZGF0ZSI7CiAJCQlicmVhazsKZGlmZiAtLWdpdCBhL3NyYy9p
bmNsdWRlL3JlcGxpY2F0aW9uL3dhbHNlbmRlci5oIGIvc3JjL2luY2x1ZGUvcmVwbGljYXRp
b24vd2Fsc2VuZGVyLmgKaW5kZXggODI4MTA2OTMzYy4uN2EyYzA0YzkzNyAxMDA2NDQKLS0t
IGEvc3JjL2luY2x1ZGUvcmVwbGljYXRpb24vd2Fsc2VuZGVyLmgKKysrIGIvc3JjL2luY2x1
ZGUvcmVwbGljYXRpb24vd2Fsc2VuZGVyLmgKQEAgLTEzLDYgKzEzLDcgQEAKICNkZWZpbmUg
X1dBTFNFTkRFUl9ICiAKICNpbmNsdWRlIDxzaWduYWwuaD4KKyNpbmNsdWRlICJzdG9yYWdl
L2NvbmRpdGlvbl92YXJpYWJsZS5oIgogCiAvKgogICogV2hhdCB0byBkbyB3aXRoIGEgc25h
cHNob3QgaW4gY3JlYXRlIHJlcGxpY2F0aW9uIHNsb3QgY29tbWFuZC4KQEAgLTQ4LDYgKzQ5
LDE3IEBAIGV4dGVybiB2b2lkIFdhbFNuZFdhaXRTdG9wcGluZyh2b2lkKTsKIGV4dGVybiB2
b2lkIEhhbmRsZVdhbFNuZEluaXRTdG9wcGluZyh2b2lkKTsKIGV4dGVybiB2b2lkIFdhbFNu
ZFJxc3RGaWxlUmVsb2FkKHZvaWQpOwogCisvKgorICogc2hhcmVkLW1lbW9yeSBzdGF0ZSBm
b3IgQ29uZGl0aW9uIFZhcmlhYmxlKHMpCisgKiBiZXR3ZWVuIHRoZSBzdGFydHVwIHByb2Nl
c3MgYW5kIHRoZSB3YWxzZW5kZXIuCisgKi8KK3R5cGVkZWYgc3RydWN0IFhMb2dDdGxDdkRh
dGEKK3sKKwlDb25kaXRpb25WYXJpYWJsZSByZXBsYXllZENWOworfSBYTG9nQ3RsQ3ZEYXRh
OworCitleHRlcm4gWExvZ0N0bEN2RGF0YSAqWExvZ0N0bEN2OworCiAvKgogICogUmVtZW1i
ZXIgdGhhdCB3ZSB3YW50IHRvIHdha2V1cCB3YWxzZW5kZXJzIGxhdGVyCiAgKgpkaWZmIC0t
Z2l0IGEvc3JjL2luY2x1ZGUvdXRpbHMvd2FpdF9ldmVudC5oIGIvc3JjL2luY2x1ZGUvdXRp
bHMvd2FpdF9ldmVudC5oCmluZGV4IDYwMDc4MjdiNDQuLjUzZDhjZTg1YzUgMTAwNjQ0Ci0t
LSBhL3NyYy9pbmNsdWRlL3V0aWxzL3dhaXRfZXZlbnQuaAorKysgYi9zcmMvaW5jbHVkZS91
dGlscy93YWl0X2V2ZW50LmgKQEAgLTEyNSw2ICsxMjUsNyBAQCB0eXBlZGVmIGVudW0KIAlX
QUlUX0VWRU5UX1NZTkNfUkVQLAogCVdBSVRfRVZFTlRfV0FMX1JFQ0VJVkVSX0VYSVQsCiAJ
V0FJVF9FVkVOVF9XQUxfUkVDRUlWRVJfV0FJVF9TVEFSVCwKKwlXQUlUX0VWRU5UX1dBTF9T
RU5ERVJfV0FJVF9SRVBMQVksCiAJV0FJVF9FVkVOVF9YQUNUX0dST1VQX1VQREFURQogfSBX
YWl0RXZlbnRJUEM7CiAK
--------------393460E7D763AE325A9CCC4F--