Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eSsDD-0005qC-T8 for pgsql-hackers@arkaria.postgresql.org; Sat, 23 Dec 2017 22:24:12 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eSsDC-0000t3-0B for pgsql-hackers@arkaria.postgresql.org; Sat, 23 Dec 2017 22:24:10 +0000 Received: from makus.postgresql.org ([2001:4800:1501:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1eSsDB-0000st-Ff for pgsql-hackers@lists.postgresql.org; Sat, 23 Dec 2017 22:24:09 +0000 Received: from lb2-smtp-cloud9.xs4all.net ([194.109.24.26]) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1eSsD3-00032l-0s for pgsql-hackers@postgresql.org; Sat, 23 Dec 2017 22:24:08 +0000 Received: from webmailclassic.xs4all.nl ([IPv6:2001:888:0:22:194:109:20:213]) by smtp-cloud9.xs4all.net with ESMTPA id SsCzeR77gnIXbSsCzejD3W; Sat, 23 Dec 2017 23:23:57 +0100 Received: from nat-pool.ips-0000-nat0x.kabeltex.nl ([82.151.32.170]) by webmailclassic.xs4all.nl with HTTP (HTTP/1.1 POST); Sat, 23 Dec 2017 23:23:57 +0100 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=_30b6744fa9638ee9c96be3ccd19d943f" Date: Sat, 23 Dec 2017 23:23:57 +0100 From: Erik Rijkers To: Tomas Vondra Cc: PostgreSQL Hackers Subject: Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions In-Reply-To: <1cc7daa2-027e-d490-2396-96e16d328c45@2ndquadrant.com> References: <688b0b7f-2f6c-d827-c27b-216a8e3ea700@2ndquadrant.com> <1cc7daa2-027e-d490-2396-96e16d328c45@2ndquadrant.com> Message-ID: X-Sender: er@xs4all.nl User-Agent: XS4ALL Webmail X-CMAE-Envelope: MS4wfFElK3IVN2QSaCCkREuAihlwRPT8/yKrNZDalgNOlN57stJUblH2aR2+qmmYSrxAEGKkR2qtI2R5As/L4/4QvJ58cf2TWQ0qqJGBYMagduxG8ZjFmVN8 n0g0dDIbXR8mluVTCk8NrCeRnBQK7i3Eqrwq+LK2wSLH/hG2iP7TfJjVC65QERNlLgLeCyKK2eboDANyndZjJJHzRaE5ml8hfo8+VsrZtj+gywN7GFSQiLWY KiF0AhKYzokiUzATYNqjVJpPhBikyRsPwfkqZXdpGfQ= List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --=_30b6744fa9638ee9c96be3ccd19d943f Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed On 2017-12-23 21:06, Tomas Vondra wrote: > On 12/23/2017 03:03 PM, Erikjan Rijkers wrote: >> On 2017-12-23 05:57, Tomas Vondra wrote: >>> Hi all, >>> >>> Attached is a patch series that implements two features to the >>> logical >>> replication - ability to define a memory limit for the reorderbuffer >>> (responsible for building the decoded transactions), and ability to >>> stream large in-progress transactions (exceeding the memory limit). >>> >> >> logical replication of 2 instances is OK but 3 and up fail with: >> >> TRAP: FailedAssertion("!(last_lsn < change->lsn)", File: >> "reorderbuffer.c", Line: 1773) >> >> I can cobble up a script but I hope you have enough from the assertion >> to see what's going wrong... > > The assertion says that the iterator produces changes in order that > does > not correlate with LSN. But I have a hard time understanding how that > could happen, particularly because according to the line number this > happens in ReorderBufferCommit(), i.e. the current (non-streaming) > case. > > So instructions to reproduce the issue would be very helpful. Using: 0001-Introduce-logical_work_mem-to-limit-ReorderBuffer-v2.patch 0002-Issue-XLOG_XACT_ASSIGNMENT-with-wal_level-logical-v2.patch 0003-Issue-individual-invalidations-with-wal_level-log-v2.patch 0004-Extend-the-output-plugin-API-with-stream-methods-v2.patch 0005-Implement-streaming-mode-in-ReorderBuffer-v2.patch 0006-Add-support-for-streaming-to-built-in-replication-v2.patch As you expected the problem is the same with these new patches. I have now tested more, and seen that it not always fails. I guess that it here fails 3 times out of 4. But the laptop I'm using at the moment is old and slow -- it may well be a factor as we've seen before [1]. Attached is the bash that I put together. I tested with NUM_INSTANCES=2, which yields success, and NUM_INSTANCES=3, which fails often. This same program run with HEAD never seems to fail (I tried a few dozen times). thanks, Erik Rijkers [1] https://www.postgresql.org/message-id/3897361c7010c4ac03f358173adbcd60%40xs4all.nl --=_30b6744fa9638ee9c96be3ccd19d943f Content-Transfer-Encoding: base64 Content-Type: text/x-shellscript; name=test.sh Content-Disposition: attachment; filename=test.sh; size=7491 IyEvYmluL2Jhc2gKdW5zZXQgUEdTRVJWSUNFIFBHU0VSVklDRUZJTEUgUEdEQVRBIFBHUE9SVCBQ R0RBVEFCQVNFCiMgUEdQQVNTRklMRSBtdXN0IGJlIHNldCBhbmQgaGF2ZSB0aGUgYXBwcm9wcmlh dGUgZW50cmllcwoKZW52IHwgZ3JlcCBeUEcKCiAgUFJPSkVDVD1sYXJnZV9sb2dpY2FsCiMgUFJP SkVDVD1IRUFECgpCSU5fRElSPSRIT01FL3BnX3N0dWZmL3BnX2luc3RhbGxhdGlvbnMvcGdzcWwu JFBST0pFQ1QvYmluClBPU1RHUkVTPSRCSU5fRElSL3Bvc3RncmVzCklOSVREQj0kQklOX0RJUi9p bml0ZGIKVE1QX0RJUj0kSE9NRScvdG1wLyckUFJPSkVDVApkZXZlbF9maWxlPSR7VE1QX0RJUn0n Ly5kZXZlbCcKTlVNX0lOU1RBTkNFUz0zCkJBU0VfUE9SVD02MDE1ICAjICAgcG9ydHMgNjAxNSwg NjAxNiwgNjAxNyAKcG9ydDE9JCgoICRCQVNFX1BPUlQgKyAwICkpCnBvcnQyPSQoKCAkcG9ydDEg KyAxICkpCnBvcnQzPSQoKCAkcG9ydDEgKyAyICkpCnNjYWxlPTEgIGRibmFtZT1wb3N0Z3JlcyAg cHVibmFtZT1wdWIxICBzdWJuYW1lPXN1YjEKaWYgW1sgISAtZCAkVE1QX0RJUiBdXTsgdGhlbiBt a2RpciAkVE1QX0RJUjsgZmkKZWNobyAnczNrcjF0JyA+ICRkZXZlbF9maWxlCiAgICAgICAgICAg ICAgICAgIG1heF93YWxfc2VuZGVycz0xMCAgIyBwdWJsaWNhdGlvbiBzaWRlCiAgICAgICAgICAg IG1heF9yZXBsaWNhdGlvbl9zbG90cz0xMCAgIyBwdWJsaWNhdGlvbiBzaWRlIGFuZCBzdWJzY3Jp cHRpb24gc2lkZQogICAgICAgICAgICAgbWF4X3dvcmtlcl9wcm9jZXNzZXM9MTIgICMgc3Vic2Ny aXB0aW9uIHNpZGUKICBtYXhfbG9naWNhbF9yZXBsaWNhdGlvbl93b3JrZXJzPTEwICAjIHN1YnNj cmlwdGlvbiBzaWRlCm1heF9zeW5jX3dvcmtlcnNfcGVyX3N1YnNjcmlwdGlvbj00ICAgIyBzdWJz Y3JpcHRpb24gc2lkZQpmb3IgbiBpbiBgc2VxIDEgJE5VTV9JTlNUQU5DRVNgOyBkbwogIHBvcnQ9 JCgoICRCQVNFX1BPUlQgKyAkbiAtMSApKQogICAgZGF0YV9kaXI9JFRNUF9ESVIvcGdzcWwuaW5z dGFuY2Uke259L2RhdGEKICBzZXJ2ZXJfZGlyPSRUTVBfRElSL3Bnc3FsLmluc3RhbmNlJHtufQog ICRJTklUREIgLS1wZ2RhdGE9JGRhdGFfZGlyIC0tZW5jb2Rpbmc9VVRGOCAtLWF1dGg9c2NyYW0t c2hhLTI1NiAtLXB3ZmlsZT0kZGV2ZWxfZmlsZSAgIyAtLXdhbGRpcj0keGxvZ19kaXIKICggJFBP U1RHUkVTIC1EICRkYXRhX2RpciAtcCAkcG9ydCBcCiAgICAtLXdhbF9sZXZlbD1sb2dpY2FsIFwK ICAgIC0tbWF4X3JlcGxpY2F0aW9uX3Nsb3RzPSRtYXhfcmVwbGljYXRpb25fc2xvdHMgXAogICAg LS1tYXhfd29ya2VyX3Byb2Nlc3Nlcz0kbWF4X3dvcmtlcl9wcm9jZXNzZXMgXAogICAgLS1tYXhf bG9naWNhbF9yZXBsaWNhdGlvbl93b3JrZXJzPSRtYXhfbG9naWNhbF9yZXBsaWNhdGlvbl93b3Jr ZXJzIFwKICAgIC0tbWF4X3dhbF9zZW5kZXJzPSRtYXhfd2FsX3NlbmRlcnMgXAogICAgLS1tYXhf c3luY193b3JrZXJzX3Blcl9zdWJzY3JpcHRpb249JG1heF9zeW5jX3dvcmtlcnNfcGVyX3N1YnNj cmlwdGlvbiBcCiAgICAtLWxvZ2dpbmdfY29sbGVjdG9yPW9uIFwKICAgIC0tbG9nX2RpcmVjdG9y eT0ke3NlcnZlcl9kaXJ9IFwKICAgIC0tbG9nX2ZpbGVuYW1lPWxvZ2ZpbGUuJHtwb3J0fSBcCiAg ICAtLWxvZ19yZXBsaWNhdGlvbl9jb21tYW5kcz1vbiBcCiAgICAtLWF1dG92YWN1dW09b2ZmICYg KQojICAgLS1sb2dpY2FsX3dvcmtfbWVtPTEyOE1CICYgKQojICAgcGdfaXNyZWFkeSAtZCAkZGJu YW1lIC0tdGltZW91dD02MCAtcCAkcG9ydApkb25lCiNzbGVlcCAkTlVNX0lOU1RBTkNFUwojcGdf aXNyZWFkeSAtZCAkZGJuYW1lIC1xcCA2MDE1IC0tdGltZW91dD02MAojcGdfaXNyZWFkeSAtZCAk ZGJuYW1lIC1xcCA2MDE2IC0tdGltZW91dD02MApudW1fbG9vcD0kKCggJE5VTV9JTlNUQU5DRVMg LSAxICkpCiRCSU5fRElSL3BnYmVuY2ggLS1wb3J0PSRCQVNFX1BPUlQgLS1xdWlldCAtLWluaXRp YWxpemUgLS1zY2FsZT0kc2NhbGUgJGRibmFtZQplY2hvICJhbHRlciB0YWJsZSBwZ2JlbmNoX2hp c3RvcnkgYWRkIGNvbHVtbiBoaWQgc2VyaWFsIHByaW1hcnkga2V5IiB8ICRCSU5fRElSL3BzcWwg LWQgJGRibmFtZSAtcCAkQkFTRV9QT1JUIC1YCiNwZ19pc3JlYWR5IC1kICRkYm5hbWUgLXFwIDYw MTUgLS10aW1lb3V0PTYwCiNwZ19pc3JlYWR5IC1kICRkYm5hbWUgLXFwIDYwMTYgLS10aW1lb3V0 PTYwCmZvciBuIGluIGBzZXEgMSAkbnVtX2xvb3BgOyBkbwogIHRhcmdldF9wb3J0PSQoKCAkQkFT RV9QT1JUICsgJG4gKSkKICBwZ19kdW1wIC1GYyAtcCAkQkFTRV9QT1JUIFwKICAgIC0tZXhjbHVk ZS10YWJsZS1kYXRhPXBnYmVuY2hfaGlzdG9yeSAgLS1leGNsdWRlLXRhYmxlLWRhdGE9cGdiZW5j aF9hY2NvdW50cyBcCiAgICAtLWV4Y2x1ZGUtdGFibGUtZGF0YT1wZ2JlbmNoX2JyYW5jaGVzIC0t ZXhjbHVkZS10YWJsZS1kYXRhPXBnYmVuY2hfdGVsbGVycyBcCiAgICAtdHBnYmVuY2hfaGlzdG9y eSAtdHBnYmVuY2hfYWNjb3VudHMgXAogICAgLXRwZ2JlbmNoX2JyYW5jaGVzIC10cGdiZW5jaF90 ZWxsZXJzIFwKICAgICAgICAkZGJuYW1lIHwgcGdfcmVzdG9yZSAtMSAtcCAkdGFyZ2V0X3BvcnQg LWQgJGRibmFtZQpkb25lCgojZWNobyAic2xlZXAgMiAoYWZ0ZXIgZHVtcC9yZXN0b3JlKSI7IHNs ZWVwIDIKCmZvciBuIGluIGBzZXEgMSAkbnVtX2xvb3BgOyBkbwogIHB1YnBvcnQ9JCgoICRCQVNF X1BPUlQgKyAkbiAtIDEgKSkKICBzdWJwb3J0PSQoKCAkQkFTRV9QT1JUICsgJG4gICAgICkpCiAg YXBwbmFtZT0nY2FzYzonJHtzdWJwb3J0fSc8JyR7cHVicG9ydH0KICBlY2hvICJjcmVhdGUgcHVi bGljYXRpb24gICRwdWJuYW1lIGZvciBhbGwgdGFibGVzIiB8IHBzcWwgLWQgJGRibmFtZSAtcCAk cHVicG9ydCAtWAogIGVjaG8gImNyZWF0ZSBzdWJzY3JpcHRpb24gJHN1Ym5hbWUKICAgICAgICBj b25uZWN0aW9uICdwb3J0PSR7cHVicG9ydH0gZGJuYW1lPSR7ZGJuYW1lfSBhcHBsaWNhdGlvbl9u YW1lPSR7YXBwbmFtZX0nCiAgICAgICAgcHVibGljYXRpb24gJHB1Ym5hbWUgd2l0aCAoZW5hYmxl ZD1mYWxzZSwgc2xvdF9uYW1lPSR7c3VibmFtZX1fJHtzdWJwb3J0fSk7IiB8IHBzcWwgLWQgJGRi bmFtZSAtcCAkc3VicG9ydCAtWAogIGVjaG8gImFsdGVyIHN1YnNjcmlwdGlvbiAkc3VibmFtZSBl bmFibGU7ICIgfCBwc3FsIC1kICRkYm5hbWUgLXAgJHN1YnBvcnQgLVgKZG9uZQpjX2ExPSQoIGVj aG8gInNlbGVjdCBjb3VudCgqKSBmcm9tIHBnYmVuY2hfYWNjb3VudHMifHBzcWwgLWQkZGJuYW1l IC1xdEFYIC1wJHBvcnQxKSAKY19iMT0kKCBlY2hvICJzZWxlY3QgY291bnQoKikgZnJvbSBwZ2Jl bmNoX2JyYW5jaGVzInxwc3FsIC1kJGRibmFtZSAtcXRBWCAtcCRwb3J0MSkgCmNfdDE9JCggZWNo byAic2VsZWN0IGNvdW50KCopIGZyb20gcGdiZW5jaF90ZWxsZXJzICJ8cHNxbCAtZCRkYm5hbWUg LXF0QVggLXAkcG9ydDEpIApjX2gxPSQoIGVjaG8gInNlbGVjdCBjb3VudCgqKSBmcm9tIHBnYmVu Y2hfaGlzdG9yeSAifHBzcWwgLWQkZGJuYW1lIC1xdEFYIC1wJHBvcnQxKSAKbWRhMT0kKCBlY2hv ICJzZWxlY3QgYWlkLGJpZCxhYmFsYW5jZSxmaWxsZXIgICAgICAgICAgICBmcm9tIHBnYmVuY2hf YWNjb3VudHMgb3JkZXIgYnkgYWlkInxwc3FsIC1kICRkYm5hbWUgLXF0QVggLXAkcG9ydDF8bWQ1 c3VtfGN1dCAtYiAxLTcgKSAKbWRiMT0kKCBlY2hvICJzZWxlY3QgYmlkLGJiYWxhbmNlLGZpbGxl ciAgICAgICAgICAgICAgICBmcm9tIHBnYmVuY2hfYnJhbmNoZXMgb3JkZXIgYnkgYmlkInxwc3Fs IC1kICRkYm5hbWUgLXF0QVggLXAkcG9ydDF8bWQ1c3VtfGN1dCAtYiAxLTcgKQptZHQxPSQoIGVj aG8gInNlbGVjdCB0aWQsYmlkLHRiYWxhbmNlLGZpbGxlciAgICAgICAgICAgIGZyb20gcGdiZW5j aF90ZWxsZXJzICBvcmRlciBieSB0aWQifHBzcWwgLWQgJGRibmFtZSAtcXRBWCAtcCRwb3J0MXxt ZDVzdW18Y3V0IC1iIDEtNyApCm1kaDE9JCggZWNobyAic2VsZWN0IGhpZCxiaWQsYWlkLGRlbHRh LG10aW1lLGZpbGxlcixoaWQgZnJvbSBwZ2JlbmNoX2hpc3RvcnkgIG9yZGVyIGJ5IGhpZCJ8cHNx bCAtZCAkZGJuYW1lIC1xdEFYIC1wJHBvcnQxfG1kNXN1bXxjdXQgLWIgMS03ICkKbWQ1XzE9JChl Y2hvICIkbWRhMSAgJG1kYjEgICRtZHQxICAkbWRoMSIgfCBtZDVzdW0gfCBjdXQgLWIgMS03ICkK cHJpbnRmICJhLGIsdCxoICAkcG9ydDEgICAgJTZkICAlNmQgICU2ZCAgJTZkICAgJG1kYTEgICRt ZGIxICAkbWR0MSAgJG1kaDEgICAgICRtZDVfMVxuIiAkY19hMSAgJGNfYjEgICRjX3QxICAkY19o MQp2ZXIxPSQoIGVjaG8gInNlbGVjdCBzdWJzdHJpbmcodmVyc2lvbigpLDEsNzApIiB8IHBzcWwg LWQgJGRibmFtZSAtcXRBWHAgJHBvcnQxICkKdmVyMj0kKCBlY2hvICJzZWxlY3Qgc3Vic3RyaW5n KHZlcnNpb24oKSwxLDcwKSIgfCBwc3FsIC1kICRkYm5hbWUgLXF0QVhwICRwb3J0MiApCmlmIFtb ICROVU1fSU5TVEFOQ0VTIC1ndCAyIF1dOyB0aGVuCnZlcjM9JCggZWNobyAic2VsZWN0IHN1YnN0 cmluZyh2ZXJzaW9uKCksMSw3MCkiIHwgcHNxbCAtZCAkZGJuYW1lIC1xdEFYcCAkcG9ydDMgKQpm aQplY2hvIApyYz0wCndoaWxlIFtbICRyYyAtZXEgMCBdXQpkbwogIG1kYTI9JChlY2hvICJzZWxl Y3QgYWlkLGJpZCxhYmFsYW5jZSxmaWxsZXIgICAgICAgICAgICBmcm9tIHBnYmVuY2hfYWNjb3Vu dHMgb3JkZXIgYnkgYWlkInxwc3FsIC1kJGRibmFtZSAtcXRBWHAkcG9ydDJ8bWQ1c3VtfGN1dCAt YiAxLTcpIAogIG1kYjI9JChlY2hvICJzZWxlY3QgYmlkLGJiYWxhbmNlLGZpbGxlciAgICAgICAg ICAgICAgICBmcm9tIHBnYmVuY2hfYnJhbmNoZXMgb3JkZXIgYnkgYmlkInxwc3FsIC1kJGRibmFt ZSAtcXRBWHAkcG9ydDJ8bWQ1c3VtfGN1dCAtYiAxLTcpCiAgbWR0Mj0kKGVjaG8gInNlbGVjdCB0 aWQsYmlkLHRiYWxhbmNlLGZpbGxlciAgICAgICAgICAgIGZyb20gcGdiZW5jaF90ZWxsZXJzICBv cmRlciBieSB0aWQifHBzcWwgLWQkZGJuYW1lIC1xdEFYcCRwb3J0MnxtZDVzdW18Y3V0IC1iIDEt NykKICBtZGgyPSQoZWNobyAic2VsZWN0IGhpZCxiaWQsYWlkLGRlbHRhLG10aW1lLGZpbGxlcixo aWQgZnJvbSBwZ2JlbmNoX2hpc3RvcnkgIG9yZGVyIGJ5IGhpZCJ8cHNxbCAtZCRkYm5hbWUgLXF0 QVhwJHBvcnQyfG1kNXN1bXxjdXQgLWIgMS03KQogIGNfYTI9JChlY2hvICJzZWxlY3QgY291bnQo KikgZnJvbSBwZ2JlbmNoX2FjY291bnRzInxwc3FsIC1kJGRibmFtZSAtcXRBWCAtcCRwb3J0Mikg CiAgY19iMj0kKGVjaG8gInNlbGVjdCBjb3VudCgqKSBmcm9tIHBnYmVuY2hfYnJhbmNoZXMifHBz cWwgLWQkZGJuYW1lIC1xdEFYIC1wJHBvcnQyKSAKICBjX3QyPSQoZWNobyAic2VsZWN0IGNvdW50 KCopIGZyb20gcGdiZW5jaF90ZWxsZXJzICJ8cHNxbCAtZCRkYm5hbWUgLXF0QVggLXAkcG9ydDIp IAogIGNfaDI9JChlY2hvICJzZWxlY3QgY291bnQoKikgZnJvbSBwZ2JlbmNoX2hpc3RvcnkgInxw c3FsIC1kJGRibmFtZSAtcXRBWCAtcCRwb3J0MikgCiAgbWQ1XzI9JChlY2hvICIkbWRhMiAgJG1k YjIgICRtZHQyICAkbWRoMiIgfCBtZDVzdW0gfCBjdXQgLWIgMS03ICkKICBpZiBbWyAkTlVNX0lO U1RBTkNFUyAtZ3QgMiBdXTsgdGhlbgogIG1kYTM9JChlY2hvICJzZWxlY3QgYWlkLGJpZCxhYmFs YW5jZSxmaWxsZXIgICAgICAgICAgICBmcm9tIHBnYmVuY2hfYWNjb3VudHMgb3JkZXIgYnkgYWlk Inxwc3FsIC1kJGRibmFtZSAtcXRBWHAkcG9ydDN8bWQ1c3VtfGN1dCAtYiAxLTcpIAogIG1kYjM9 JChlY2hvICJzZWxlY3QgYmlkLGJiYWxhbmNlLGZpbGxlciAgICAgICAgICAgICAgICBmcm9tIHBn YmVuY2hfYnJhbmNoZXMgb3JkZXIgYnkgYmlkInxwc3FsIC1kJGRibmFtZSAtcXRBWHAkcG9ydDN8 bWQ1c3VtfGN1dCAtYiAxLTcpCiAgbWR0Mz0kKGVjaG8gInNlbGVjdCB0aWQsYmlkLHRiYWxhbmNl LGZpbGxlciAgICAgICAgICAgIGZyb20gcGdiZW5jaF90ZWxsZXJzICBvcmRlciBieSB0aWQifHBz cWwgLWQkZGJuYW1lIC1xdEFYcCRwb3J0M3xtZDVzdW18Y3V0IC1iIDEtNykKICBtZGgzPSQoZWNo byAic2VsZWN0IGhpZCxiaWQsYWlkLGRlbHRhLG10aW1lLGZpbGxlcixoaWQgZnJvbSBwZ2JlbmNo X2hpc3RvcnkgIG9yZGVyIGJ5IGhpZCJ8cHNxbCAtZCRkYm5hbWUgLXF0QVhwJHBvcnQzfG1kNXN1 bXxjdXQgLWIgMS03KQogIGNfYTM9JChlY2hvICJzZWxlY3QgY291bnQoKikgZnJvbSBwZ2JlbmNo X2FjY291bnRzInxwc3FsIC1kJGRibmFtZSAtcXRBWCAtcCRwb3J0MykgCiAgY19iMz0kKGVjaG8g InNlbGVjdCBjb3VudCgqKSBmcm9tIHBnYmVuY2hfYnJhbmNoZXMifHBzcWwgLWQkZGJuYW1lIC1x dEFYIC1wJHBvcnQzKSAKICBjX3QzPSQoZWNobyAic2VsZWN0IGNvdW50KCopIGZyb20gcGdiZW5j aF90ZWxsZXJzICJ8cHNxbCAtZCRkYm5hbWUgLXF0QVggLXAkcG9ydDMpIAogIGNfaDM9JChlY2hv ICJzZWxlY3QgY291bnQoKikgZnJvbSBwZ2JlbmNoX2hpc3RvcnkgInxwc3FsIC1kJGRibmFtZSAt cXRBWCAtcCRwb3J0MykgCiAgbWQ1XzM9JChlY2hvICIkbWRhMyAgJG1kYjMgICRtZHQzICAkbWRo MyIgfCBtZDVzdW0gfCBjdXQgLWIgMS03ICkKICBmaQogIGVjaG8gIi0tICRQT1NUR1JFUyIKICBw cmludGYgImEsYix0LGggICRwb3J0MSAgICAlN2QgJTZkICU2ZCAlNmQgICAgJG1kYTEgJG1kYjEg JG1kdDEgJG1kaDEgICAkbWQ1XzEgICAiICRjX2ExICRjX2IxICRjX3QxICRjX2gxOyBlY2hvICIk dmVyMSIKICBwcmludGYgImEsYix0LGggICRwb3J0MiAgICAlN2QgJTZkICU2ZCAlNmQgICAgJG1k YTIgJG1kYjIgJG1kdDIgJG1kaDIgICAkbWQ1XzIgICAiICRjX2EyICRjX2IyICRjX3QyICRjX2gy OyBlY2hvICIkdmVyMiIKICBpZiBbWyAkTlVNX0lOU1RBTkNFUyAtZ3QgMiBdXTsgdGhlbgogIHBy aW50ZiAiYSxiLHQsaCAgJHBvcnQzICAgICU3ZCAlNmQgJTZkICU2ZCAgICAkbWRhMyAkbWRiMyAk bWR0MyAkbWRoMyAgICRtZDVfMyAgICIgJGNfYTMgJGNfYjMgJGNfdDMgJGNfaDM7IGVjaG8gIiR2 ZXIzIgogIGZpCiAgaWYgICBbWyAkTlVNX0lOU1RBTkNFUyAtZXEgMiBdXTsgdGhlbgogICAgICAg aWYgW1sgIiRtZDVfMSIgPT0gIiRtZDVfMiIgXV0gOyB0aGVuIGVjaG8gIk9LIC0gZG9uZS4iOyBi cmVhazsgZmkKICBlbGlmIFtbICROVU1fSU5TVEFOQ0VTIC1lcSAzIF1dOyB0aGVuCiAgICAgICBp ZiBbWyAiJG1kNV8xIiA9PSAiJG1kNV8zIiBdXSA7IHRoZW4gZWNobyAiT0sgLSBkb25lLiI7IGJy ZWFrOyBmaQogIGZpCiAgc2xlZXAgMQogIHJjPSQ/CmRvbmUKZm9yIG4gaW4gYHNlcSAxICROVU1f SU5TVEFOQ0VTYDsgZG8gCiAgcG9ydD0kKCggJEJBU0VfUE9SVCArICRuIC0xICkpCiAgZGF0YV9k aXI9JFRNUF9ESVIvcGdzcWwuaW5zdGFuY2Uke259L2RhdGEKICAkQklOX0RJUi9wZ19jdGwgc3Rv cCAtdyAtRCAkZGF0YV9kaXIKZG9uZQoK --=_30b6744fa9638ee9c96be3ccd19d943f--