Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1odAXm-0002D6-MX for pgsql-hackers@arkaria.postgresql.org; Tue, 27 Sep 2022 13:22:38 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1odAXl-0006r6-HC for pgsql-hackers@arkaria.postgresql.org; Tue, 27 Sep 2022 13:22:37 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1odAXl-0006ph-3I for pgsql-hackers@lists.postgresql.org; Tue, 27 Sep 2022 13:22:37 +0000 Received: from mail-wr1-x435.google.com ([2a00:1450:4864:20::435]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1odAXi-0003PN-Ak for pgsql-hackers@lists.postgresql.org; Tue, 27 Sep 2022 13:22:36 +0000 Received: by mail-wr1-x435.google.com with SMTP id cc5so14989773wrb.6 for ; Tue, 27 Sep 2022 06:22:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date; bh=xOskQOhLq6Ed8lz21MMXBXjuGAFVsLJScfY+HAkX0Qw=; b=bfZEeEWhDVEH0bL+H7d3PifP4uHwFKwdCgj6zJ7yZZHv/z3qYUTARcJEHr8QIgxkHu Ulz5ka/VwdGGjKwtkuYdgxUjiq4R8kkOawlvN1s0viR7dMHx4Iv3cMiKwOPE+m7PRaTs C1khUVUsv3QGuY8GYEWQ55BFPWLzHqFzJshm9E8U8G5hygEndcrm6P8HZobQ8iOiV8oc ZmN1YiRf+cAuyNQXJHTu7usl95+osYda4gSQ9oFKUXuIfpIH5eqKgVDTitF6nYJyqgRr oPIeZnIF1NkouQThKcC94yS+PtwyL7v0ByS1DaoDyk3f7VuarrHrgvJNMfpjo373Kw5z zMKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date; bh=xOskQOhLq6Ed8lz21MMXBXjuGAFVsLJScfY+HAkX0Qw=; b=EbaOyQidu1n7xoF8EnlZn/IzoXc0ufm0zfY8/GWUnXMujTs3XVsuu79DQeHFR1nDXV 2uVDRHaJYkYePe2cPftTs1Ge4Pt8dRALX9TT5oD6s5t6gOaVtuR0bpjnzCkT9l3ualwW vryGYpHsWoNrEjz7eqlwHnYb/JO/wi9tDoaJj5WkMwmoC1uqcG4KByUFj9jgo6x3xfxv kAPux+65uZ7h07NoFnMKRw4EYbzpmmbHddicBXm2ygwMsMTP454PxSgvSsWBzGORZvpr lq5laa+B5J9iapzEwaQhEpA6ciUwHYobBs3Aulm7CFdRb6EBTjpAmRDAmLIoZp9+ExS/ FxIQ== X-Gm-Message-State: ACrzQf25X5EUDELk0cGwtvJ4PIQm+Ph6BsA9xt7yIsAgn+90Ezppx8lu HJyrAdg8F9OEbPCdgGlXSRjkg1/3C7a3oUFjt70= X-Google-Smtp-Source: AMsMyM6DaA3sd1rlGtxM743WUtqcdbe0mMouRs6Lz+T1CtkGXGPLbWSZvA1/N6PFLbCdr76rdPepvXLp1G2G3d3Z4b8= X-Received: by 2002:a05:6000:18a1:b0:22a:f4c2:c9e6 with SMTP id b1-20020a05600018a100b0022af4c2c9e6mr17829733wri.71.1664284952869; Tue, 27 Sep 2022 06:22:32 -0700 (PDT) MIME-Version: 1.0 References: <9290b55b6ae2b04e002ca9dadadd1cca09461482.camel@cybertec.at> <20220805.114916.994654810780821553.horikyota.ntt@gmail.com> <20220809.161236.1486509314201074910.horikyota.ntt@gmail.com> In-Reply-To: From: Bharath Rupireddy Date: Tue, 27 Sep 2022 18:52:21 +0530 Message-ID: Subject: Re: An attempt to avoid locally-committed-but-not-replicated-to-standby-transactions in synchronous replication To: Kyotaro Horiguchi Cc: Laurenz Albe , PostgreSQL Hackers , SATYANARAYANA NARLAPURAM Content-Type: multipart/mixed; boundary="0000000000006bd79c05e9a88924" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --0000000000006bd79c05e9a88924 Content-Type: text/plain; charset="UTF-8" On Tue, Aug 9, 2022 at 2:16 PM Bharath Rupireddy wrote: > > I've explained the problem with the current HA setup with synchronous > replication upthread at [1]. Let me reiterate it here once again. > > When a query is cancelled (a simple stroke of CTRL+C or > pg_cancel_backend() call) while the txn is waiting for ack in > SyncRepWaitForLSN(); for the client, the txn is actually committed > (locally-committed-but-not-yet-replicated to all of sync standbys) > like any other txn, a warning is emitted into server logs but it is of > no use for the client (think of client as applications). There can be > many such txns waiting for ack in SyncRepWaitForLSN() and query cancel > can be issued on all of those sessions. The problem is that the > subsequent reads will then be able to read all of those > locally-committed-but-not-yet-replicated to all of sync standbys txns > data - this is what I call an inconsistency (can we call this a > read-after-write inconsistency?) because of lack of proper query > cancel handling. And if the sync standbys are down or unable to come > up for some reason, until then, the primary will be serving clients > with the inconsistent data. BTW, I found a report of this problem here > [2]. > > The solution proposed for the above problem is to just 'not honor > query cancels at all while waiting for ack in SyncRepWaitForLSN()'. > > When a proc die is pending, then also, there can be > locally-committed-but-not-yet-replicated to all of sync standbys txns. > Typically, there are two choices for the clients 1) reuse the primary > instance after restart 2) failover to one of sync standbys. For case > (1), there might be read-after-write inconsistency as explained above. > For case (2), those txns might get lost completely if the failover > target sync standby or the new primary didn't receive them and the > other sync standbys that have received them are now ahead and need a > special treatment (run pg_rewind) for them to be able to connect to > new primary. > > The solution proposed for case (1) of the above problem is to 'process > the ProcDiePending immediately and upon restart the first backend can > wait until the sync standbys are caught up to ensure no inconsistent > reads'. > The solution proposed for case (2) of the above problem is to 'either > run pg_rewind for the sync standbys that are ahead or use the idea > proposed at [3]'. > > I hope the above explanation helps. > > [1] https://www.postgresql.org/message-id/flat/CALj2ACUrOB59QaE6%3DjF2cFAyv1MR7fzD8tr4YM5%2BOwEYG1SNzA%40mail.gmail.com > [2] https://stackoverflow.com/questions/42686097/how-to-disable-uncommited-reads-in-postgres-synchronous-replication > [3] https://www.postgresql.org/message-id/CALj2ACX-xO-ZenQt1MWazj0Z3ziSXBMr24N_X2c0dYysPQghrw%40mail.gmail.com I'm attaching the v2 patch rebased on the latest HEAD. Please note that there are still some open points, I'm yet to find time to think more about them. Meanwhile, I'm posting the v2 patch for making cfbot happy. Any further thoughts on the overall design of the patch are most welcome. Thanks. -- Bharath Rupireddy PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com --0000000000006bd79c05e9a88924 Content-Type: application/x-patch; name="v2-0001-Wait-specified-amount-of-time-before-cancelling-s.patch" Content-Disposition: attachment; filename="v2-0001-Wait-specified-amount-of-time-before-cancelling-s.patch" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_l8k7gs7o0 RnJvbSBiZTY3MzRjODNkNzIzMzNjYzRlYzFiMmJlMWY0ZDU0Yjg3NWI3NGM4IE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBCaGFyYXRoIFJ1cGlyZWRkeSA8YmhhcmF0aC5ydXBpcmVkZHlm b3Jwb3N0Z3Jlc0BnbWFpbC5jb20+CkRhdGU6IFR1ZSwgMjcgU2VwIDIwMjIgMTI6NTU6MzQgKzAw MDAKU3ViamVjdDogW1BBVENIIHYyXSBXYWl0IHNwZWNpZmllZCBhbW91bnQgb2YgdGltZSBiZWZv cmUgY2FuY2VsbGluZyBzeW5jCiByZXBsaWNhdGlvbgoKSW4gUG9zdGdyZVNRTCBoaWdoIGF2YWls YWJpbGl0eSBzZXR1cCB3aXRoIHN5bmNocm9ub3VzIHJlcGxpY2F0aW9uLAp0eXBpY2FsbHkgYWxs IHRoZSB0cmFuc2FjdGlvbnMgZmlyc3QgbG9jYWxseSBnZXQgY29tbWl0dGVkLCB0aGVuCnN0cmVh bWVkIHRvIHRoZSBzeW5jaHJvbm91cyBzdGFuZGJ5cyBhbmQgdGhlIGJhY2tlbmRzIHRoYXQgZ2Vu ZXJhdGVkCnRoZSB0cmFuc2FjdGlvbiB3aWxsIHdhaXQgZm9yIGFja25vd2xlZGdlbWVudCBmcm9t IHN5bmNocm9ub3VzCnN0YW5kYnlzLiBXaGlsZSB3YWl0aW5nIGZvciBhY2tub3dsZWRnZW1lbnQs IGl0IG1heSBoYXBwZW4gdGhhdCB0aGUKcXVlcnkgb3IgdGhlIHRyYW5zYWN0aW9uIGdldHMgY2Fu Y2VsZWQgb3IgdGhlIGJhY2tlbmQgdGhhdCdzIHdhaXRpbmcKZm9yIGFja25vd2xlZGdlbWVudCBp cyBhc2tlZCB0byBleGl0LiBJbiBlaXRoZXIgb2YgdGhlc2UgY2FzZXMsIHRoZQp3YWl0IGZvciBh Y2tub3dsZWRnZW1lbnQgZ2V0cyBjYW5jZWxlZCBhbmQgbGVhdmVzIHRyYW5zYWN0aW9uIGluIGFu CmluY29uc2lzdGVudCBzdGF0ZSBhcyBpdCBnb3QgY29tbWl0dGVkIGxvY2FsbHkgYnV0IG5vdCBv biB0aGUKc3RhbmRieXMuIFdoZW4gc2V0IHRoZSBHVUMgc3luY2hyb25vdXNfcmVwbGljYXRpb25f bmFwdGltZV9iZWZvcmVfY2FuY2VsCmludHJvZHVjZWQgaW4gdGhpcyBwYXRjaCwgaXQgd2lsbCBs ZXQgdGhlIGJhY2tlbmRzIHdhaXQgZm9yIHRoZQphY2tub3dsZWRnZW1lbnQgYmVmb3JlIGNhbmNl bGluZyB0aGUgd2FpdCBmb3IgYWNrbm93bGVkZ2VtZW50IHNvCnRoYXQgdGhlIHRyYW5zYWN0aW9u IGNhbiBiZSBhdmFpbGFibGUgaW4gc3luY2hyb25vdXMgc3RhbmRieXMgYXMKd2VsbC4KLS0tCiBk b2Mvc3JjL3NnbWwvY29uZmlnLnNnbWwgICAgICAgICAgICAgICAgICAgICAgfCAzMCArKysrKysr KysrKwogc3JjL2JhY2tlbmQvcmVwbGljYXRpb24vc3luY3JlcC5jICAgICAgICAgICAgIHwgNTAg KysrKysrKysrKysrKysrKysrKwogc3JjL2JhY2tlbmQvdXRpbHMvbWlzYy9ndWNfdGFibGVzLmMg ICAgICAgICAgIHwgMTIgKysrKysKIHNyYy9iYWNrZW5kL3V0aWxzL21pc2MvcG9zdGdyZXNxbC5j b25mLnNhbXBsZSB8ICAyICsKIHNyYy9pbmNsdWRlL3JlcGxpY2F0aW9uL3N5bmNyZXAuaCAgICAg ICAgICAgICB8ICAzICsrCiA1IGZpbGVzIGNoYW5nZWQsIDk3IGluc2VydGlvbnMoKykKCmRpZmYg LS1naXQgYS9kb2Mvc3JjL3NnbWwvY29uZmlnLnNnbWwgYi9kb2Mvc3JjL3NnbWwvY29uZmlnLnNn bWwKaW5kZXggZDg4NDhiYzc3NC4uYmFlZWY0OTAxMiAxMDA2NDQKLS0tIGEvZG9jL3NyYy9zZ21s L2NvbmZpZy5zZ21sCisrKyBiL2RvYy9zcmMvc2dtbC9jb25maWcuc2dtbApAQCAtNDUyNCw2ICs0 NTI0LDM2IEBAIEFOWSA8cmVwbGFjZWFibGUgY2xhc3M9InBhcmFtZXRlciI+bnVtX3N5bmM8L3Jl cGxhY2VhYmxlPiAoIDxyZXBsYWNlYWJsZSBjbGFzcz0iCiAgICAgICA8L2xpc3RpdGVtPgogICAg ICA8L3Zhcmxpc3RlbnRyeT4KIAorICAgICA8dmFybGlzdGVudHJ5IGlkPSJndWMtc3luY2hyb25v dXMtcmVwbGljYXRpb24tbmFwdGltZS1iZWZvcmUtY2FuY2VsIiB4cmVmbGFiZWw9InN5bmNocm9u b3VzX3JlcGxpY2F0aW9uX25hcHRpbWVfYmVmb3JlX2NhbmNlbCI+CisgICAgICA8dGVybT48dmFy bmFtZT5zeW5jaHJvbm91c19yZXBsaWNhdGlvbl9uYXB0aW1lX2JlZm9yZV9jYW5jZWw8L3Zhcm5h bWU+ICg8dHlwZT5pbnRlZ2VyPC90eXBlPikKKyAgICAgIDxpbmRleHRlcm0+CisgICAgICAgPHBy aW1hcnk+PHZhcm5hbWU+c3luY2hyb25vdXNfcmVwbGljYXRpb25fbmFwdGltZV9iZWZvcmVfY2Fu Y2VsPC92YXJuYW1lPiBjb25maWd1cmF0aW9uIHBhcmFtZXRlcjwvcHJpbWFyeT4KKyAgICAgIDwv aW5kZXh0ZXJtPgorICAgICAgPC90ZXJtPgorICAgICAgPGxpc3RpdGVtPgorICAgICAgIDxwYXJh PgorICAgICAgICBTcGVjaWZpZXMgdGhlIGFtb3VudCBvZiB0aW1lIGluIG1pbGxpc2Vjb25kcyB0 byB3YWl0IGZvciBzeW5jaHJvbm91cworICAgICAgICByZXBsaWNhdGlvbiBiZWZvcmUgY2FuY2Vs bGluZy4gRGVmYXVsdCB2YWx1ZSBpcyAwLCBhIHZhbHVlIG9mIC0xIG9yIDAKKyAgICAgICAgZGlz YWJsZXMgdGhpcyBmZWF0dXJlLiBJbiA8cHJvZHVjdG5hbWU+UG9zdGdyZVNRTDwvcHJvZHVjdG5h bWU+IGhpZ2gKKyAgICAgICAgYXZhaWxhYmlsaXR5IHNldHVwIHdpdGggc3luY2hyb25vdXMgcmVw bGljYXRpb24sIHR5cGljYWxseSBhbGwgdGhlCisgICAgICAgIHRyYW5zYWN0aW9ucyBmaXJzdCBs b2NhbGx5IGdldCBjb21taXR0ZWQsIHRoZW4gc3RyZWFtZWQgdG8gdGhlCisgICAgICAgIHN5bmNo cm9ub3VzIHN0YW5kYnlzIGFuZCB0aGUgYmFja2VuZHMgdGhhdCBnZW5lcmF0ZWQgdGhlIHRyYW5z YWN0aW9uCisgICAgICAgIHdpbGwgd2FpdCBmb3IgYWNrbm93bGVkZ2VtZW50IGZyb20gc3luY2hy b25vdXMgc3RhbmRieXMuIFdoaWxlIHdhaXRpbmcKKyAgICAgICAgZm9yIGFja25vd2xlZGdlbWVu dCwgaXQgbWF5IGhhcHBlbiB0aGF0IHRoZSBxdWVyeSBvciB0aGUgdHJhbnNhY3Rpb24KKyAgICAg ICAgZ2V0cyBjYW5jZWxlZCBvciB0aGUgYmFja2VuZCB0aGF0J3Mgd2FpdGluZyBmb3IgYWNrbm93 bGVkZ2VtZW50IGlzCisgICAgICAgIGFza2VkIHRvIGV4aXQuIEluIGVpdGhlciBvZiB0aGVzZSBj YXNlcywgdGhlIHdhaXQgZm9yIGFja25vd2xlZGdlbWVudAorICAgICAgICBnZXRzIGNhbmNlbGVk IGFuZCBsZWF2ZXMgdHJhbnNhY3Rpb24gaW4gYW4gaW5jb25zaXN0ZW50IHN0YXRlIGFzIGl0IGdv dAorICAgICAgICBjb21taXR0ZWQgbG9jYWxseSBidXQgbm90IG9uIHRoZSBzdGFuZGJ5cy4gV2hl biBzZXQgdGhlCisgICAgICAgIDx2YXJuYW1lPnN5bmNocm9ub3VzX3JlcGxpY2F0aW9uX25hcHRp bWVfYmVmb3JlX2NhbmNlbDwvdmFybmFtZT4KKyAgICAgICAgcGFyYW1ldGVyLCBpdCB3aWxsIGxl dCB0aGUgYmFja2VuZHMgd2FpdCBmb3IgdGhlIGFja25vd2xlZGdlbWVudAorICAgICAgICBiZWZv cmUgY2FuY2VsaW5nIHRoZSB3YWl0IGZvciBhY2tub3dsZWRnZW1lbnQgc28gdGhhdCB0aGUgdHJh bnNhY3Rpb24KKyAgICAgICAgY2FuIGJlIGF2YWlsYWJsZSBpbiBzeW5jaHJvbm91cyBzdGFuZGJ5 cyBhcyB3ZWxsLiBUaGlzIHBhcmFtZXRlciBjYW4KKyAgICAgICAgb25seSBiZSBzZXQgaW4gdGhl IDxmaWxlbmFtZT5wb3N0Z3Jlc3FsLmNvbmY8L2ZpbGVuYW1lPiBmaWxlIG9yIG9uIHRoZQorICAg ICAgICBzZXJ2ZXIgY29tbWFuZCBsaW5lLgorICAgICAgICA8L3BhcmE+CisgICAgICA8L2xpc3Rp dGVtPgorICAgICA8L3Zhcmxpc3RlbnRyeT4KKwogICAgICA8L3ZhcmlhYmxlbGlzdD4KICAgICA8 L3NlY3QyPgogCmRpZmYgLS1naXQgYS9zcmMvYmFja2VuZC9yZXBsaWNhdGlvbi9zeW5jcmVwLmMg Yi9zcmMvYmFja2VuZC9yZXBsaWNhdGlvbi9zeW5jcmVwLmMKaW5kZXggZTM2MGQ5MjViMC4uNjBi NmE1ZTQ3MSAxMDA2NDQKLS0tIGEvc3JjL2JhY2tlbmQvcmVwbGljYXRpb24vc3luY3JlcC5jCisr KyBiL3NyYy9iYWNrZW5kL3JlcGxpY2F0aW9uL3N5bmNyZXAuYwpAQCAtODksNiArODksNyBAQAog CiAvKiBVc2VyLXNldHRhYmxlIHBhcmFtZXRlcnMgZm9yIHN5bmMgcmVwICovCiBjaGFyCSAgICpT eW5jUmVwU3RhbmRieU5hbWVzOworaW50CQkJU3luY1JlcE5hcFRpbWVCZWZvcmVDYW5jZWwgPSAw OwogCiAjZGVmaW5lIFN5bmNTdGFuZGJ5c0RlZmluZWQoKSBcCiAJKFN5bmNSZXBTdGFuZGJ5TmFt ZXMgIT0gTlVMTCAmJiBTeW5jUmVwU3RhbmRieU5hbWVzWzBdICE9ICdcMCcpCkBAIC0xMjAsNiAr MTIxLDcgQEAgc3RhdGljIHZvaWQgU3luY1JlcEdldE50aExhdGVzdFN5bmNSZWNQdHIoWExvZ1Jl Y1B0ciAqd3JpdGVQdHIsCiBzdGF0aWMgaW50CVN5bmNSZXBHZXRTdGFuZGJ5UHJpb3JpdHkodm9p ZCk7CiBzdGF0aWMgaW50CXN0YW5kYnlfcHJpb3JpdHlfY29tcGFyYXRvcihjb25zdCB2b2lkICph LCBjb25zdCB2b2lkICpiKTsKIHN0YXRpYyBpbnQJY21wX2xzbihjb25zdCB2b2lkICphLCBjb25z dCB2b2lkICpiKTsKK3N0YXRpYyBib29sIFN5bmNSZXBOYXBCZWZvcmVDYW5jZWwodm9pZCk7CiAK ICNpZmRlZiBVU0VfQVNTRVJUX0NIRUNLSU5HCiBzdGF0aWMgYm9vbCBTeW5jUmVwUXVldWVJc09y ZGVyZWRCeUxTTihpbnQgbW9kZSk7CkBAIC0xMzEsNiArMTMzLDQyIEBAIHN0YXRpYyBib29sIFN5 bmNSZXBRdWV1ZUlzT3JkZXJlZEJ5TFNOKGludCBtb2RlKTsKICAqID09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09CiAgKi8KIAorLyoKKyAq IFdhaXQgZm9yIHN5bmNocm9ub3VzIHJlcGxpY2F0aW9uIGJlZm9yZSBjYW5jZWxsaW5nLCBpZiBy ZXF1ZXN0ZWQgYnkgdXNlci4KKyAqLworc3RhdGljIGJvb2wKK1N5bmNSZXBOYXBCZWZvcmVDYW5j ZWwodm9pZCkKK3sKKwlpbnQgd2FpdF90aW1lOworCisJaWYgKFN5bmNSZXBOYXBUaW1lQmVmb3Jl Q2FuY2VsIDw9IDApCisJCXJldHVybiBmYWxzZTsKKworCWVyZXBvcnQoV0FSTklORywKKwkJCShl cnJtc2dfcGx1cmFsKCJ3YWl0aW5nICVkIG1pbGxpc2Vjb25kIGZvciBzeW5jaHJvbm91cyByZXBs aWNhdGlvbiBiZWZvcmUgY2FuY2VsbGluZyIsCisJCQkJCQkgICAid2FpdGluZyAlZCBtaWxsaXNl Y29uZHMgZm9yIHN5bmNocm9ub3VzIHJlcGxpY2F0aW9uIGJlZm9yZSBjYW5jZWxsaW5nIiwKKwkJ CQkJCQlTeW5jUmVwTmFwVGltZUJlZm9yZUNhbmNlbCwKKwkJCQkJCQlTeW5jUmVwTmFwVGltZUJl Zm9yZUNhbmNlbCkpKTsKKworCXdhaXRfdGltZSA9IFN5bmNSZXBOYXBUaW1lQmVmb3JlQ2FuY2Vs OworCisJd2hpbGUgKHdhaXRfdGltZSA+IDApCisJeworCQkvKgorCQkgKiBXYWl0IGluIGludGVy dmFscyBvZiAxIG1pbGxpc2Vjb25kIHNvIHRoYXQgd2UgY2FuIGZyZXF1ZW50bHkgY2hlY2sKKwkJ ICogZm9yIHRoZSBhY2tub3dsZWRnZW1lbnQuCisJCSAqLworCQlwZ191c2xlZXAoMSAqIDEwMDBM KTsKKworCQl3YWl0X3RpbWUtLTsKKworCQlpZiAoTXlQcm9jLT5zeW5jUmVwU3RhdGUgPT0gU1lO Q19SRVBfV0FJVF9DT01QTEVURSkKKwkJCXJldHVybiB0cnVlOworCX0KKworCXJldHVybiB0cnVl OworfQorCiAvKgogICogV2FpdCBmb3Igc3luY2hyb25vdXMgcmVwbGljYXRpb24sIGlmIHJlcXVl c3RlZCBieSB1c2VyLgogICoKQEAgLTI2NCw2ICszMDIsMTIgQEAgU3luY1JlcFdhaXRGb3JMU04o WExvZ1JlY1B0ciBsc24sIGJvb2wgY29tbWl0KQogCQkgKi8KIAkJaWYgKFByb2NEaWVQZW5kaW5n KQogCQl7CisJCQlpZiAoU3luY1JlcE5hcEJlZm9yZUNhbmNlbCgpKQorCQkJeworCQkJCWlmIChN eVByb2MtPnN5bmNSZXBTdGF0ZSA9PSBTWU5DX1JFUF9XQUlUX0NPTVBMRVRFKQorCQkJCQlicmVh azsKKwkJCX0KKwogCQkJZXJlcG9ydChXQVJOSU5HLAogCQkJCQkoZXJyY29kZShFUlJDT0RFX0FE TUlOX1NIVVRET1dOKSwKIAkJCQkJIGVycm1zZygiY2FuY2VsaW5nIHRoZSB3YWl0IGZvciBzeW5j aHJvbm91cyByZXBsaWNhdGlvbiBhbmQgdGVybWluYXRpbmcgY29ubmVjdGlvbiBkdWUgdG8gYWRt aW5pc3RyYXRvciBjb21tYW5kIiksCkBAIC0yODEsNiArMzI1LDEyIEBAIFN5bmNSZXBXYWl0Rm9y TFNOKFhMb2dSZWNQdHIgbHNuLCBib29sIGNvbW1pdCkKIAkJICovCiAJCWlmIChRdWVyeUNhbmNl bFBlbmRpbmcpCiAJCXsKKwkJCWlmIChTeW5jUmVwTmFwQmVmb3JlQ2FuY2VsKCkpCisJCQl7CisJ CQkJaWYgKE15UHJvYy0+c3luY1JlcFN0YXRlID09IFNZTkNfUkVQX1dBSVRfQ09NUExFVEUpCisJ CQkJCWJyZWFrOworCQkJfQorCiAJCQlRdWVyeUNhbmNlbFBlbmRpbmcgPSBmYWxzZTsKIAkJCWVy ZXBvcnQoV0FSTklORywKIAkJCQkJKGVycm1zZygiY2FuY2VsaW5nIHdhaXQgZm9yIHN5bmNocm9u b3VzIHJlcGxpY2F0aW9uIGR1ZSB0byB1c2VyIHJlcXVlc3QiKSwKZGlmZiAtLWdpdCBhL3NyYy9i YWNrZW5kL3V0aWxzL21pc2MvZ3VjX3RhYmxlcy5jIGIvc3JjL2JhY2tlbmQvdXRpbHMvbWlzYy9n dWNfdGFibGVzLmMKaW5kZXggYWIzMTQwZmYzYS4uYmIxNGU0N2M0NSAxMDA2NDQKLS0tIGEvc3Jj L2JhY2tlbmQvdXRpbHMvbWlzYy9ndWNfdGFibGVzLmMKKysrIGIvc3JjL2JhY2tlbmQvdXRpbHMv bWlzYy9ndWNfdGFibGVzLmMKQEAgLTI0OTYsNiArMjQ5NiwxOCBAQCBzdHJ1Y3QgY29uZmlnX2lu dCBDb25maWd1cmVOYW1lc0ludFtdID0KIAkJMCwgMCwgMTAwMDAwMCwJCQkvKiBzZWUgQ29tcHV0 ZVhpZEhvcml6b25zICovCiAJCU5VTEwsIE5VTEwsIE5VTEwKIAl9LAorCisJeworCQl7InN5bmNo cm9ub3VzX3JlcGxpY2F0aW9uX25hcHRpbWVfYmVmb3JlX2NhbmNlbCIsIFBHQ19TSUdIVVAsIFJF UExJQ0FUSU9OX1BSSU1BUlksCisJCQlnZXR0ZXh0X25vb3AoIlNldHMgdGhlIGFtb3VudCBvZiB0 aW1lIHRvIHdhaXQgZm9yIHN5bmNocm9ub3VzIHJlcGxpY3RhaW9uIGJlZm9yZSBjYW5jZWxsaW5n LiIpLAorCQkJZ2V0dGV4dF9ub29wKCJBIHZhbHVlIG9mIC0xIG9yIDAgZGlzYWJsZXMgdGhpcyBm ZWF0dXJlLiIpLAorCQkJR1VDX1VOSVRfTVMKKwkJfSwKKwkJJlN5bmNSZXBOYXBUaW1lQmVmb3Jl Q2FuY2VsLAorCQkwLCAwLCBJTlRfTUFYLAorCQlOVUxMLCBOVUxMLCBOVUxMCisJfSwKKwogCXsK IAkJeyJ2YWN1dW1fZmFpbHNhZmVfYWdlIiwgUEdDX1VTRVJTRVQsIENMSUVOVF9DT05OX1NUQVRF TUVOVCwKIAkJCWdldHRleHRfbm9vcCgiQWdlIGF0IHdoaWNoIFZBQ1VVTSBzaG91bGQgdHJpZ2dl ciBmYWlsc2FmZSB0byBhdm9pZCBhIHdyYXBhcm91bmQgb3V0YWdlLiIpLApkaWZmIC0tZ2l0IGEv c3JjL2JhY2tlbmQvdXRpbHMvbWlzYy9wb3N0Z3Jlc3FsLmNvbmYuc2FtcGxlIGIvc3JjL2JhY2tl bmQvdXRpbHMvbWlzYy9wb3N0Z3Jlc3FsLmNvbmYuc2FtcGxlCmluZGV4IDJhZTc2ZTVjZmIuLjc4 NDllM2M1YjMgMTAwNjQ0Ci0tLSBhL3NyYy9iYWNrZW5kL3V0aWxzL21pc2MvcG9zdGdyZXNxbC5j b25mLnNhbXBsZQorKysgYi9zcmMvYmFja2VuZC91dGlscy9taXNjL3Bvc3RncmVzcWwuY29uZi5z YW1wbGUKQEAgLTMyNCw2ICszMjQsOCBAQAogCQkJCSMgYW5kIGNvbW1hLXNlcGFyYXRlZCBsaXN0 IG9mIGFwcGxpY2F0aW9uX25hbWUKIAkJCQkjIGZyb20gc3RhbmRieShzKTsgJyonID0gYWxsCiAj dmFjdXVtX2RlZmVyX2NsZWFudXBfYWdlID0gMAkjIG51bWJlciBvZiB4YWN0cyBieSB3aGljaCBj bGVhbnVwIGlzIGRlbGF5ZWQKKyNzeW5jaHJvbm91c19yZXBsaWNhdGlvbl9uYXB0aW1lX2JlZm9y ZV9jYW5jZWwgPSAwICMgYW1vdW50IG9mIHRpbWUgdG8gd2FpdCBmb3IKKwkJCQkjIHN5bmNocm9u b3VzIHJlcGxpY3RhaW9uIGJlZm9yZSBjYW5jZWxsaW5nOyAwIG9yIC0xIGRpc2FibGVzCiAKICMg LSBTdGFuZGJ5IFNlcnZlcnMgLQogCmRpZmYgLS1naXQgYS9zcmMvaW5jbHVkZS9yZXBsaWNhdGlv bi9zeW5jcmVwLmggYi9zcmMvaW5jbHVkZS9yZXBsaWNhdGlvbi9zeW5jcmVwLmgKaW5kZXggMGYz YjNhNzk1NS4uOWI2NjhlOWE2MSAxMDA2NDQKLS0tIGEvc3JjL2luY2x1ZGUvcmVwbGljYXRpb24v c3luY3JlcC5oCisrKyBiL3NyYy9pbmNsdWRlL3JlcGxpY2F0aW9uL3N5bmNyZXAuaApAQCAtODAs NiArODAsOSBAQCBleHRlcm4gUEdETExJTVBPUlQgY2hhciAqc3luY3JlcF9wYXJzZV9lcnJvcl9t c2c7CiAvKiB1c2VyLXNldHRhYmxlIHBhcmFtZXRlcnMgZm9yIHN5bmNocm9ub3VzIHJlcGxpY2F0 aW9uICovCiBleHRlcm4gUEdETExJTVBPUlQgY2hhciAqU3luY1JlcFN0YW5kYnlOYW1lczsKIAor LyogdXNlci1zZXR0YWJsZSBuYXAgdGltZSBmb3Igc3luY2hyb25vdXMgcmVwbGljdGFpb24gYmVm b3JlIGNhbmNlbGxpbmcgKi8KK2V4dGVybiBQR0RMTElNUE9SVCBpbnQgU3luY1JlcE5hcFRpbWVC ZWZvcmVDYW5jZWw7CisKIC8qIGNhbGxlZCBieSB1c2VyIGJhY2tlbmQgKi8KIGV4dGVybiB2b2lk IFN5bmNSZXBXYWl0Rm9yTFNOKFhMb2dSZWNQdHIgbHNuLCBib29sIGNvbW1pdCk7CiAKLS0gCjIu MzQuMQoK --0000000000006bd79c05e9a88924--