Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1t1RAv-00GJ4x-R5 for pgsql-general@arkaria.postgresql.org; Thu, 17 Oct 2024 14:08:26 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1t1RAu-005MFU-2U for pgsql-general@arkaria.postgresql.org; Thu, 17 Oct 2024 14:08:24 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1t1RAt-005MFL-LT for pgsql-general@lists.postgresql.org; Thu, 17 Oct 2024 14:08:24 +0000 Received: from mail-ed1-x52e.google.com ([2a00:1450:4864:20::52e]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1t1RAr-001Xyq-OD for pgsql-general@lists.postgresql.org; Thu, 17 Oct 2024 14:08:23 +0000 Received: by mail-ed1-x52e.google.com with SMTP id 4fb4d7f45d1cf-5c942815197so530752a12.1 for ; Thu, 17 Oct 2024 07:08:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729174099; x=1729778899; darn=lists.postgresql.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=MBGIefYenAsW5mvOTHXc+8vwCM+MqZ+Nq5cIO43n+IQ=; b=CeZF6+7bORe9zjAAuX7uClquB+EVrSEDbCxFlfSJpeYUpmnYggf3ChvYk4EJxPNRB/ qBMdMxQyDkDRbqGDj8/gVx5iyALMEJUcMznvNHEJln5NkmfkGh1pg6ru8XdRPZmNzJ1q HViRz4EqCvx6wRz7zQ+ZIsrQfAdSHcrZmgL4kGcOXa9kV01y9iqGuVERVYOQcjuvlfyC baf3rji6R6F6gOabdHLSmZOIGI/lB9TbYPkqNnoLRzx2XkA2lCe8j8KI3un4U7aUd7da 3M3aawtbquABfKzVfkUx7dUpVpupfdYzSBJEaUxX3mdfKaIpg6NEfw1sOxjAVlJUq3ab ornw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729174099; x=1729778899; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=MBGIefYenAsW5mvOTHXc+8vwCM+MqZ+Nq5cIO43n+IQ=; b=mge7l1PeGvVBPMJuty7jIkrsGpG9ncb/fEEIpqP2TbWD93520R/V+T5ZGhQK5llC3L v5eY8ZNRKJAtFYvr590boBw8AF0t34EL4VbLbGJm+ZcgTbZGx8kIuwldi7vKg6nK05Qp epaOxCfFWu7BwRFNYA2ZIic77VA3iWqv8BlDl+8fVaZHj9p7wZsbogIAyAWZVdHytJcb FKs+4S//p7dfbLw5HKAipvFNQ4GV1Eq6jdX5hL/c4Kxpi+TuFBt9bsADs3R/oqBX1YBB IjwUAlrdmDcJ2z5ofRyqlluRgsTNf49gpebML3IoHEiTX/0TqnzkRPkWFwnCiAtScmFN aoYQ== X-Gm-Message-State: AOJu0YxQAVRL1sl0554BnPR13NwJki3t/A2FtGJP9JLEYORFvJ/4VQD2 J1/UarHFe0KFOHfL7mWjBo83uamg+vaeXh6VBRu9CqhMjD076bGsMqiWG0Umh92o4oO+S2OBKz9 CWHlcMVrf9AwwZiwfoEijuchkdvUtTZZJ X-Google-Smtp-Source: AGHT+IEsY2M/1LGKyff2hwybFL1GGXNwvfaSJbYydRC0OSGPr58mc6miMZdDJIa1h72fAn2McLoNGNpZessMF3Rx6P8= X-Received: by 2002:a05:6402:8c3:b0:5c9:55aa:4285 with SMTP id 4fb4d7f45d1cf-5c95ac63c95mr21039665a12.34.1729174098912; Thu, 17 Oct 2024 07:08:18 -0700 (PDT) MIME-Version: 1.0 From: Siraj G Date: Thu, 17 Oct 2024 19:38:06 +0530 Message-ID: Subject: Performance difference between Primary & Secondary in the query execution To: pgsql-general@lists.postgresql.org Content-Type: multipart/mixed; boundary="000000000000ec145b0624acb71f" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --000000000000ec145b0624acb71f Content-Type: multipart/alternative; boundary="000000000000ec14590624acb71d" --000000000000ec14590624acb71d Content-Type: text/plain; charset="UTF-8" Hello Experts! We have a PgSQL instance running with HA (secondary is being in sync with streaming replication). Both the ends, we have same version, but not sure a few SQLs behave badly in the secondary: Primary: PostgreSQL 12.20 (Ubuntu 12.20-0ubuntu0.20.04.1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0, 64-bit OS: Ubuntu 20.04.6 LTS \n \l Secondary: ostgreSQL 12.20 (Ubuntu 12.20-0ubuntu0.20.04.1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0, 64-bit OS: Ubuntu 20.04.6 LTS \n \l The application consumes more data from secondary, hence the server has extra vCPUs. Can you please advise what needs to be checked. FYI, I am attaching the query with the different execution plans. Regards Siraj --000000000000ec14590624acb71d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hello Experts!

We have a PgSQL instance= running with HA (secondary is being in sync with streaming replication). B= oth the ends, we have same version, but not sure a few SQLs behave badly in= the secondary:

Primary:
PostgreSQL 12.2= 0 (Ubuntu 12.20-0ubuntu0.20.04.1) on x86_64-pc-linux-gnu, compiled by gcc (= Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0, 64-bit

OS:= =C2=A0Ubuntu 20.04.6 LTS \n \l

Secondary:
ostg= reSQL 12.20 (Ubuntu 12.20-0ubuntu0.20.04.1) on x86_64-pc-linux-gnu, compile= d by gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0, 64-bit

OS:=C2=A0Ubuntu 20.04.6 LTS \n \l

T= he application consumes more data from secondary, hence the server has extr= a vCPUs.

Can you please advise what needs to be ch= ecked.

FYI, I am attaching the query with the diff= erent execution plans.

Regards
Siraj
--000000000000ec14590624acb71d-- --000000000000ec145b0624acb71f Content-Type: application/octet-stream; name=bad-explain-secondary Content-Disposition: attachment; filename=bad-explain-secondary Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_m2ddljqj0 QWdncmVnYXRlICAoY29zdD0yNDkyMjI1Ljk4Li4yNDkyMjI1Ljk5IHJvd3M9MSB3aWR0aD04KSAo YWN0dWFsIHRpbWU9MjI4MTIwLjM1OC4uMjI4MTIwLjM2NyByb3dzPTEgbG9vcHM9MSkNCiAgIC0+ ICBVbmlxdWUgIChjb3N0PTIyNjM4MDYuNDQuLjI0NTk1OTQuNjEgcm93cz0yNjEwNTA5IHdpZHRo PTIwMSkgKGFjdHVhbCB0aW1lPTIyODAyMi4zODMuLjIyODEyMC4zMjUgcm93cz0xMiBsb29wcz0x KQ0KICAgICAgICAgLT4gIFNvcnQgIChjb3N0PTIyNjM4MDYuNDQuLjIyNzAzMzIuNzEgcm93cz0y NjEwNTA5IHdpZHRoPTIwMSkgKGFjdHVhbCB0aW1lPTIyODAyMi4zODEuLjIyODA0MS42NDMgcm93 cz02MTYxNyBsb29wcz0xKQ0KICAgICAgICAgICAgICAgU29ydCBLZXk6IHRlc3QuaWQsIHRlc3Qu cXVlc3Rpb25fcGFwZXJfaWQsIHRlc3QuZ3JhZGVfaWQsIHRlc3QuaGFzX3N1Yl9ncm91cHMsIHRl c3QuZ3JvdXBfaWQsIHRlc3QuY2VudHJhbF9xcF9pZCwgdGVzdC5pc19lcnBfcXAsIHRlc3QuaXNf cmV2aWV3X2VuYWJsZWQsIHRlc3QudGVzdF9uYW1lLCB0ZXN0LnRlc3RfaWQsIHRlc3QudGVzdF90 eXBlX2lkLCB0ZXN0LnRlYWNoZXJfaWQsIHRlc3QudGVzdF9tb2RlLCB0ZXN0LnRvdGFsX21hcmss IHRlc3QuaXNfcXVlc3Rpb25fd2lzZSwgdGVzdC50ZXN0X2RhdGUsIHRlc3QudGVzdF9kdXJhdGlv biwgdGVzdC50ZXN0X2VuZF9kYXRldGltZSwgdGVzdC5pbnN0cnVjdGlvbnMsIHRlc3QuZGVzY3Jp cHRpb25zLCB0ZXN0LnRvdGFsX3F1ZXN0aW9uLCB0ZXN0LmlzX2RlbGV0ZSwgdGVzdC5pc19jb21w bGV0ZWQsIHRlc3QuY3JlYXRlZF9hdCwgdGVzdC51cGRhdGVkX2F0LCB0ZXN0LmNyZWF0ZWRfYnlf aWQsIHRlc3QudXBkYXRlZF9ieV9pZCwgdGVzdC5pc19sb2NrLCB0ZXN0LmlzX29tcg0KICAgICAg ICAgICAgICAgU29ydCBNZXRob2Q6IHF1aWNrc29ydCAgTWVtb3J5OiAxNzkwNGtCDQogICAgICAg ICAgICAgICAtPiAgSGFzaCBSaWdodCBKb2luICAoY29zdD03NTg4Ny40NS4uMTcyNjgyMS4yMCBy b3dzPTI2MTA1MDkgd2lkdGg9MjAxKSAoYWN0dWFsIHRpbWU9Mzg5Ny40NjIuLjIyNzAzMy43NjAg cm93cz02MTYxNyBsb29wcz0xKQ0KICAgICAgICAgICAgICAgICAgICAgSGFzaCBDb25kOiAodXNl cl9yZXNwb25zZS50ZXN0X2lkID0gdGVzdC5pZCkNCiAgICAgICAgICAgICAgICAgICAgIEZpbHRl cjogKCh0ZXN0LnRlc3RfZW5kX2RhdGV0aW1lIDw9ICcyMDI0LTEwLTE3IDE4OjQ4OjE3LjEwNTA0 Myc6OnRpbWVzdGFtcCB3aXRob3V0IHRpbWUgem9uZSkgT1IgKHVzZXJfcmVzcG9uc2Uuc3VibWl0 dGVkX2J5X2lkID0gMTA4MzE5KSBPUiAoKHRlc3QudGVzdF9lbmRfZGF0ZXRpbWUgSVMgTlVMTCkg QU5EICh0ZXN0LnRlc3RfdHlwZV9pZCA9IEFOWSAoJ3sxNSwxNH0nOjppbnRlZ2VyW10pKSBBTkQg KHVzZXJfcmVzcG9uc2Uuc3VibWl0dGVkX2J5X2lkID0gMTA4MzE5KSkpDQogICAgICAgICAgICAg ICAgICAgICAtPiAgU2VxIFNjYW4gb24gdXNlcl9yZXNwb25zZSAgKGNvc3Q9MC4wMC4uMTU3MDg4 My42MiByb3dzPTYzNDAwNjIgd2lkdGg9OCkgKGFjdHVhbCB0aW1lPTAuMDI1Li4yMjIyMDcuNDc2 IHJvd3M9NjM0Mjc4MiBsb29wcz0xKQ0KICAgICAgICAgICAgICAgICAgICAgLT4gIEhhc2ggIChj b3N0PTc1ODA5Ljc4Li43NTgwOS43OCByb3dzPTYyMTMgd2lkdGg9MjAxKSAoYWN0dWFsIHRpbWU9 MzcxOC4zMzYuLjM3MTguMzQyIHJvd3M9MTIgbG9vcHM9MSkNCiAgICAgICAgICAgICAgICAgICAg ICAgICAgIEJ1Y2tldHM6IDgxOTIgIEJhdGNoZXM6IDEgIE1lbW9yeSBVc2FnZTogNjdrQg0KICAg ICAgICAgICAgICAgICAgICAgICAgICAgLT4gIEhhc2ggSm9pbiAgKGNvc3Q9MjI3Mi4xOS4uNzU4 MDkuNzggcm93cz02MjEzIHdpZHRoPTIwMSkgKGFjdHVhbCB0aW1lPTMzOTAuMzk4Li4zNzE4LjE3 MiByb3dzPTEyIGxvb3BzPTEpDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBIYXNo IENvbmQ6ICh0ZXN0LmlkID0gdGVzdF9zdWJqZWN0cy50ZXN0X2lkKQ0KICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgLT4gIEhhc2ggTGVmdCBKb2luICAoY29zdD0xMzQyLjI1Li43NDQy My43MCByb3dzPTYzMTQgd2lkdGg9MjAxKSAoYWN0dWFsIHRpbWU9MjI5OS41OTcuLjI2MjcuMzQ1 IHJvd3M9MTIgbG9vcHM9MSkNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg IEhhc2ggQ29uZDogKHRlc3QuZ3JvdXBfaWQgPSBhc3Nlc3NtZW50X2dyb3VwdXNlcnMuY3JlYXRl ZHVzZXJzaW5mb19wdHJfaWQpDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICBGaWx0ZXI6ICgoKHRlc3QuZ3JvdXBfaWQgSVMgTlVMTCkgQU5EICh0ZXN0X3NlY3Rpb25fbWFw cGluZy5zZWN0aW9ubWFwcGluZ19pZCA9IDEzNzk5OSkpIE9SIChhc3Nlc3NtZW50X2dyb3VwdXNl cnNfZXJwdXNlcnMuZXJwdXNlcnNfaWQgPSA0NjUxMykpDQogICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICBSb3dzIFJlbW92ZWQgYnkgRmlsdGVyOiAxMTYzOTMyDQogICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAtPiAgSGFzaCBSaWdodCBKb2luICAoY29z dD05NTguOTAuLjMxMjcyLjA3IHJvd3M9MTM4Nzc5MSB3aWR0aD0yMDUpIChhY3R1YWwgdGltZT05 MS4yNDcuLjI0MjUuNzU0IHJvd3M9MTE2MzY0MyBsb29wcz0xKQ0KICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgSGFzaCBDb25kOiAodGVzdF9zZWN0aW9uX21hcHBp bmcudGVzdF9pZCA9IHRlc3QuaWQpDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAtPiAgU2VxIFNjYW4gb24gdGVzdF9zZWN0aW9uX21hcHBpbmcgIChjb3N0PTAu MDAuLjI1ODk4Ljk2IHJvd3M9MTY4MTA5NiB3aWR0aD04KSAoYWN0dWFsIHRpbWU9MC4wMTEuLjE3 NTkuMjA0IHJvd3M9MTY4MTEyNyBsb29wcz0xKQ0KICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgLT4gIEhhc2ggIChjb3N0PTgxNC41OS4uODE0LjU5IHJvd3M9MTE1 NDUgd2lkdGg9MjAxKSAoYWN0dWFsIHRpbWU9OTEuMTkyLi45MS4xOTIgcm93cz0xMTQ3MiBsb29w cz0xKQ0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg QnVja2V0czogMTYzODQgIEJhdGNoZXM6IDEgIE1lbW9yeSBVc2FnZTogMjg1M2tCDQogICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAtPiAgU2VxIFNjYW4g b24gdGVzdCAgKGNvc3Q9MC4wMC4uODE0LjU5IHJvd3M9MTE1NDUgd2lkdGg9MjAxKSAoYWN0dWFs IHRpbWU9MC4wMjYuLjgyLjE3MiByb3dzPTExNDcyIGxvb3BzPTEpDQogICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBGaWx0ZXI6ICgoTk9UIGlz X2RlbGV0ZSkgQU5EICgoTk9UIGlzX2xvY2spIE9SIChpc19sb2NrIElTIE5VTEwpKSBBTkQgKHRl c3RfdHlwZV9pZCA9IEFOWSAoJ3sxNSwxNCwxMywxMiwxMSwxMCw5LDgsNyw2LDUsNCwzLDF9Jzo6 aW50ZWdlcltdKSkpDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICBSb3dzIFJlbW92ZWQgYnkgRmlsdGVyOiAyNTI5DQogICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAtPiAgSGFzaCAgKGNvc3Q9MjM1LjAzLi4yMzUuMDMg cm93cz0xMTg2NSB3aWR0aD04KSAoYWN0dWFsIHRpbWU9MzcuODIzLi4zNy44MjYgcm93cz0xMjI3 NyBsb29wcz0xKQ0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg QnVja2V0czogMTYzODQgIEJhdGNoZXM6IDEgIE1lbW9yeSBVc2FnZTogNjA4a0INCiAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC0+ICBIYXNoIFJpZ2h0IEpvaW4g IChjb3N0PTE1Ljk5Li4yMzUuMDMgcm93cz0xMTg2NSB3aWR0aD04KSAoYWN0dWFsIHRpbWU9MC4x NTAuLjM1LjE0MSByb3dzPTEyMjc3IGxvb3BzPTEpDQogICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICBIYXNoIENvbmQ6IChhc3Nlc3NtZW50X2dyb3VwdXNl cnNfZXJwdXNlcnMuZ3JvdXB1c2Vyc19pZCA9IGFzc2Vzc21lbnRfZ3JvdXB1c2Vycy5jcmVhdGVk dXNlcnNpbmZvX3B0cl9pZCkNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgIC0+ICBTZXEgU2NhbiBvbiBhc3Nlc3NtZW50X2dyb3VwdXNlcnNfZXJwdXNl cnMgIChjb3N0PTAuMDAuLjE4Ny42NSByb3dzPTExODY1IHdpZHRoPTgpIChhY3R1YWwgdGltZT0w LjAwNy4uMzIuMzY4IHJvd3M9MTIyNzcgbG9vcHM9MSkNCiAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgIC0+ICBIYXNoICAoY29zdD05LjMzLi45LjMzIHJv d3M9NTMzIHdpZHRoPTQpIChhY3R1YWwgdGltZT0wLjEzMy4uMC4xMzMgcm93cz01MzYgbG9vcHM9 MSkNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgIEJ1Y2tldHM6IDEwMjQgIEJhdGNoZXM6IDEgIE1lbW9yeSBVc2FnZTogMjdrQg0KICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgLT4gIFNl cSBTY2FuIG9uIGFzc2Vzc21lbnRfZ3JvdXB1c2VycyAgKGNvc3Q9MC4wMC4uOS4zMyByb3dzPTUz MyB3aWR0aD00KSAoYWN0dWFsIHRpbWU9MC4wMTIuLjAuMDY1IHJvd3M9NTM2IGxvb3BzPTEpDQog ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAtPiAgSGFzaCAgKGNvc3Q9NzU3LjkzLi43 NTcuOTMgcm93cz0xMzc2MSB3aWR0aD00KSAoYWN0dWFsIHRpbWU9MTA5MC43ODMuLjEwOTAuNzgz IHJvd3M9MTM1OTYgbG9vcHM9MSkNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgIEJ1Y2tldHM6IDE2Mzg0ICBCYXRjaGVzOiAxICBNZW1vcnkgVXNhZ2U6IDYwNmtCDQogICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAtPiAgSW5kZXggU2NhbiB1c2luZyB0 ZXN0X3N1YmplY3RzX3N1YmplY3RfaWRfZWUwNzBjZjYgb24gdGVzdF9zdWJqZWN0cyAgKGNvc3Q9 MC4yOS4uNzU3LjkzIHJvd3M9MTM3NjEgd2lkdGg9NCkgKGFjdHVhbCB0aW1lPTk3NS43NDAuLjEw ODYuNTA5IHJvd3M9MTM1DQo5NiBsb29wcz0xKQ0KICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgSW5kZXggQ29uZDogKHN1YmplY3RfaWQgPSBBTlkgKCd7MTIzNSwx MjMxLDExODgsMTE4NiwxMTg0LDExODMsMTE3MiwxMTcxLDExNjMsMTE1OSwxMTU4LDExNTIsMTE1 MSwxMTQ3LDExNDYsMTE0NSwxMTQ0LDExNDMsMTE0MiwxMTM4LDExMzEsMTEzMCwxMQ0KMjUsMTEy NCwxMTIzLDEwOTksMTA5NCwxMDU0LDEwNTMsMTA0NiwxMDQzLDEwNDEsMTAzNCwxMDMxLDEwMzAs MTAyOSwxMDE2LDk5MSw5MDgsODk0LDg5MCw4ODQsODgwLDg3Niw4NzUsODY3LDg2MSw4NjAsODU4 LDg1Nyw4NTUsODU0LDg1MSw4NDksODQ4LDg0Niw4NDUsNzg3LDcxLDcwLDY5LDYwLDU5LDUzLDUy LDUxLDM0LDMzLDMyLA0KMzEsMzAsMjksMjcsMTksMTcsMTYsMTUsMTQsMTMsMTIsMTAsOCw3LDYs NCwzLDIsMX0nOjppbnRlZ2VyW10pKQ0KIFBsYW5uaW5nIFRpbWU6IDE1LjU5OCBtcw== --000000000000ec145b0624acb71f Content-Type: application/octet-stream; name=explain-primary Content-Disposition: attachment; filename=explain-primary Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_m2ddlmr61 QWdncmVnYXRlICAoY29zdD0xNTg1MDMxLjAyLi4xNTg1MDMxLjAzIHJvd3M9MSB3aWR0aD04KSAo YWN0dWFsIHRpbWU9MTI2Mi4wMjAuLjEyNjIuMDI2IHJvd3M9MSBsb29wcz0xKQ0KICAgLT4gIEhh c2hBZ2dyZWdhdGUgIChjb3N0PTE1MjU5MDIuMjMuLjE1NTIxODEuNjkgcm93cz0yNjI3OTQ2IHdp ZHRoPTIwMSkgKGFjdHVhbCB0aW1lPTEyNTUuNTQ1Li4xMjYyLjAxNCByb3dzPTEyIGxvb3BzPTEp DQogICAgICAgICBHcm91cCBLZXk6IHRlc3QuaWQsIHRlc3QucXVlc3Rpb25fcGFwZXJfaWQsIHRl c3QuZ3JhZGVfaWQsIHRlc3QuaGFzX3N1Yl9ncm91cHMsIHRlc3QuZ3JvdXBfaWQsIHRlc3QuY2Vu dHJhbF9xcF9pZCwgdGVzdC5pc19lcnBfcXAsIHRlc3QuaXNfcmV2aWV3X2VuYWJsZWQsIHRlc3Qu dGVzdF9uYW1lLCB0ZXN0LnRlc3RfaWQsIHRlc3QudGVzdF90eXBlX2lkLCB0ZXN0LnRlYWNoZXJf aWQsIHRlc3QudGVzdF9tb2RlLCB0ZXN0LnRvdGFsX21hcmssIHRlc3QuaXNfcXVlc3Rpb25fd2lz ZSwgdGVzdC50ZXN0X2RhdGUsIHRlc3QudGVzdF9kdXJhdGlvbiwgdGVzdC50ZXN0X2VuZF9kYXRl dGltZSwgdGVzdC5pbnN0cnVjdGlvbnMsIHRlc3QuZGVzY3JpcHRpb25zLCB0ZXN0LnRvdGFsX3F1 ZXN0aW9uLCB0ZXN0LmlzX2RlbGV0ZSwgdGVzdC5pc19jb21wbGV0ZWQsIHRlc3QuY3JlYXRlZF9h dCwgdGVzdC51cGRhdGVkX2F0LCB0ZXN0LmNyZWF0ZWRfYnlfaWQsIHRlc3QudXBkYXRlZF9ieV9p ZCwgdGVzdC5pc19sb2NrLCB0ZXN0LmlzX29tcg0KICAgICAgICAgLT4gIE5lc3RlZCBMb29wIExl ZnQgSm9pbiAgKGNvc3Q9MTc3Ni44NS4uMTMzNTM3Ni4xNSByb3dzPTI2Mjc5NDYgd2lkdGg9MjAx KSAoYWN0dWFsIHRpbWU9MTA4NC45MDcuLjEyMjEuNDk0IHJvd3M9NjE2MTcgbG9vcHM9MSkNCiAg ICAgICAgICAgICAgIEZpbHRlcjogKCh0ZXN0LnRlc3RfZW5kX2RhdGV0aW1lIDw9ICcyMDI0LTEw LTE3IDE4OjQ4OjE3LjEwNTA0Myc6OnRpbWVzdGFtcCB3aXRob3V0IHRpbWUgem9uZSkgT1IgKHVz ZXJfcmVzcG9uc2Uuc3VibWl0dGVkX2J5X2lkID0gMTA4MzE5KSBPUiAoKHRlc3QudGVzdF9lbmRf ZGF0ZXRpbWUgSVMgTlVMTCkgQU5EICh0ZXN0LnRlc3RfdHlwZV9pZCA9IEFOWSAoJ3sxNSwxNH0n OjppbnRlZ2VyW10pKSBBTkQgKHVzZXJfcmVzcG9uc2Uuc3VibWl0dGVkX2J5X2lkID0gMTA4MzE5 KSkpDQogICAgICAgICAgICAgICAtPiAgSGFzaCBKb2luICAoY29zdD0xNzc2LjQyLi43NTMxNC4w MSByb3dzPTYyMTMgd2lkdGg9MjAxKSAoYWN0dWFsIHRpbWU9MTA4NC44ODEuLjExNTguMDkzIHJv d3M9MTIgbG9vcHM9MSkNCiAgICAgICAgICAgICAgICAgICAgIEhhc2ggQ29uZDogKHRlc3QuaWQg PSB0ZXN0X3N1YmplY3RzLnRlc3RfaWQpDQogICAgICAgICAgICAgICAgICAgICAtPiAgSGFzaCBM ZWZ0IEpvaW4gIChjb3N0PTEyMDguNDEuLjc0Mjg5Ljg3IHJvd3M9NjMxNCB3aWR0aD0yMDEpIChh Y3R1YWwgdGltZT0zNzkuODk3Li40NTMuMDk4IHJvd3M9MTIgbG9vcHM9MSkNCiAgICAgICAgICAg ICAgICAgICAgICAgICAgIEhhc2ggQ29uZDogKHRlc3QuZ3JvdXBfaWQgPSBhc3Nlc3NtZW50X2dy b3VwdXNlcnMuY3JlYXRlZHVzZXJzaW5mb19wdHJfaWQpDQogICAgICAgICAgICAgICAgICAgICAg ICAgICBGaWx0ZXI6ICgoKHRlc3QuZ3JvdXBfaWQgSVMgTlVMTCkgQU5EICh0ZXN0X3NlY3Rpb25f bWFwcGluZy5zZWN0aW9ubWFwcGluZ19pZCA9IDEzNzk5OSkpIE9SIChhc3Nlc3NtZW50X2dyb3Vw dXNlcnNfZXJwdXNlcnMuZXJwdXNlcnNfaWQgPSA0NjUxMykpDQogICAgICAgICAgICAgICAgICAg ICAgICAgICBSb3dzIFJlbW92ZWQgYnkgRmlsdGVyOiAxMTYzOTMyDQogICAgICAgICAgICAgICAg ICAgICAgICAgICAtPiAgSGFzaCBSaWdodCBKb2luICAoY29zdD04MjUuMDcuLjMxMTM4LjI0IHJv d3M9MTM4Nzc5MSB3aWR0aD0yMDUpIChhY3R1YWwgdGltZT03LjM0MC4uMzc2LjY1OSByb3dzPTEx NjM2NDMgbG9vcHM9MSkNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIEhhc2ggQ29u ZDogKHRlc3Rfc2VjdGlvbl9tYXBwaW5nLnRlc3RfaWQgPSB0ZXN0LmlkKQ0KICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgLT4gIFNlcSBTY2FuIG9uIHRlc3Rfc2VjdGlvbl9tYXBwaW5n ICAoY29zdD0wLjAwLi4yNTg5OC45NiByb3dzPTE2ODEwOTYgd2lkdGg9OCkgKGFjdHVhbCB0aW1l PTAuMDA5Li4xMDYuNDYyIHJvd3M9MTY4MTEyNyBsb29wcz0xKQ0KICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgLT4gIEhhc2ggIChjb3N0PTY4MC43NS4uNjgwLjc1IHJvd3M9MTE1NDUg d2lkdGg9MjAxKSAoYWN0dWFsIHRpbWU9Ny4zMTUuLjcuMzE3IHJvd3M9MTE0NzIgbG9vcHM9MSkN CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIEJ1Y2tldHM6IDE2Mzg0ICBC YXRjaGVzOiAxICBNZW1vcnkgVXNhZ2U6IDI4NTNrQg0KICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgLT4gIEluZGV4IFNjYW4gdXNpbmcgdGVzdF90ZXN0X3R5cGVfaWRfNDUz NzNhOTYgb24gdGVzdCAgKGNvc3Q9MC4yOS4uNjgwLjc1IHJvd3M9MTE1NDUgd2lkdGg9MjAxKSAo YWN0dWFsIHRpbWU9MC4wMTcuLjQuMzc4IHJvd3M9MTE0NzIgbG9vcHM9MSkNCiAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIEluZGV4IENvbmQ6ICh0ZXN0X3R5cGVf aWQgPSBBTlkgKCd7MTUsMTQsMTMsMTIsMTEsMTAsOSw4LDcsNiw1LDQsMywxfSc6OmludGVnZXJb XSkpDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBGaWx0ZXI6 ICgoTk9UIGlzX2RlbGV0ZSkgQU5EICgoTk9UIGlzX2xvY2spIE9SIChpc19sb2NrIElTIE5VTEwp KSkNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIFJvd3MgUmVt b3ZlZCBieSBGaWx0ZXI6IDk0OQ0KICAgICAgICAgICAgICAgICAgICAgICAgICAgLT4gIEhhc2gg IChjb3N0PTIzNS4wMy4uMjM1LjAzIHJvd3M9MTE4NjUgd2lkdGg9OCkgKGFjdHVhbCB0aW1lPTMu NDM0Li4zLjQzNiByb3dzPTEyMjc3IGxvb3BzPTEpDQogICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICBCdWNrZXRzOiAxNjM4NCAgQmF0Y2hlczogMSAgTWVtb3J5IFVzYWdlOiA2MDhrQg0K ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgLT4gIEhhc2ggUmlnaHQgSm9pbiAgKGNv c3Q9MTUuOTkuLjIzNS4wMyByb3dzPTExODY1IHdpZHRoPTgpIChhY3R1YWwgdGltZT0wLjA5NC4u Mi4zOTggcm93cz0xMjI3NyBsb29wcz0xKQ0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgSGFzaCBDb25kOiAoYXNzZXNzbWVudF9ncm91cHVzZXJzX2VycHVzZXJzLmdyb3Vw dXNlcnNfaWQgPSBhc3Nlc3NtZW50X2dyb3VwdXNlcnMuY3JlYXRlZHVzZXJzaW5mb19wdHJfaWQp DQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAtPiAgU2VxIFNjYW4gb24g YXNzZXNzbWVudF9ncm91cHVzZXJzX2VycHVzZXJzICAoY29zdD0wLjAwLi4xODcuNjUgcm93cz0x MTg2NSB3aWR0aD04KSAoYWN0dWFsIHRpbWU9MC4wMDUuLjAuNzE0IHJvd3M9MTIyNzcgbG9vcHM9 MSkNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC0+ICBIYXNoICAoY29z dD05LjMzLi45LjMzIHJvd3M9NTMzIHdpZHRoPTQpIChhY3R1YWwgdGltZT0wLjA4NC4uMC4wODQg cm93cz01MzYgbG9vcHM9MSkNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgIEJ1Y2tldHM6IDEwMjQgIEJhdGNoZXM6IDEgIE1lbW9yeSBVc2FnZTogMjdrQg0KICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgLT4gIFNlcSBTY2FuIG9u IGFzc2Vzc21lbnRfZ3JvdXB1c2VycyAgKGNvc3Q9MC4wMC4uOS4zMyByb3dzPTUzMyB3aWR0aD00 KSAoYWN0dWFsIHRpbWU9MC4wMDcuLjAuMDQzIHJvd3M9NTM2IGxvb3BzPTEpDQogICAgICAgICAg ICAgICAgICAgICAtPiAgSGFzaCAgKGNvc3Q9Mzk1Ljk5Li4zOTUuOTkgcm93cz0xMzc2MSB3aWR0 aD00KSAoYWN0dWFsIHRpbWU9NzA0Ljk2NC4uNzA0Ljk2NSByb3dzPTEzNTk2IGxvb3BzPTEpDQog ICAgICAgICAgICAgICAgICAgICAgICAgICBCdWNrZXRzOiAxNjM4NCAgQmF0Y2hlczogMSAgTWVt b3J5IFVzYWdlOiA2MDZrQg0KICAgICAgICAgICAgICAgICAgICAgICAgICAgLT4gIEluZGV4IFNj YW4gdXNpbmcgdGVzdF9zdWJqZWN0c19zdWJqZWN0X2lkX2VlMDcwY2Y2IG9uIHRlc3Rfc3ViamVj dHMgIChjb3N0PTAuMjkuLjM5NS45OSByb3dzPTEzNzYxIHdpZHRoPTQpIChhY3R1YWwgdGltZT03 MDEuNTM3Li43MDMuNzA5IHJvd3M9MTM1OTYgbG9vcHM9MSkNCiAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgIEluZGV4IENvbmQ6IChzdWJqZWN0X2lkID0gQU5ZICgnezEyMzUsMTIzMSwx MTg4LDExODYsMTE4NCwxMTgzLDExNzIsMTE3MSwxMTYzLDExNTksMTE1OCwxMTUyLDExNTEsMTE0 NywxMTQ2LDExNDUsMTE0NCwxMTQzLDExNDIsMTEzOCwxMTMxLDExMzAsMTEyNSwxMTI0LDExMjMs MTA5OSwxMDk0LDEwNTQsMTA1MywxMDQ2LDEwNDMsMTA0MSwxMDM0LDEwMzEsMTAzMCwxMDI5LDEw MTYsOTkxLDkwOCw4OTQsODkwLDg4NCw4ODAsODc2LDg3NSw4NjcsODYxLDg2MCw4NTgsODU3LDg1 NSw4NTQsODUxLDg0OSw4NDgsODQ2LDg0NSw3ODcsNzEsNzAsNjksNjAsNTksNTMsNTIsNTEsMzQs MzMsMzIsMzEsMzAsMjksMjcsMTksMTcsMTYsMTUsMTQsMTMsMTIsMTAsOCw3LDYsNCwzLDIsMX0n OjppbnRlZ2VyW10pKQ0KICAgICAgICAgICAgICAgLT4gIEluZGV4IFNjYW4gdXNpbmcgdXNlcl9y ZXNwb25zZV90ZXN0X2lkXzI2MTAwYjE4IG9uIHVzZXJfcmVzcG9uc2UgIChjb3N0PTAuNDMuLjE2 NS45OSByb3dzPTE4NDEgd2lkdGg9OCkgKGFjdHVhbCB0aW1lPTAuMDEwLi40LjUzMyByb3dzPTUx MzUgbG9vcHM9MTIpDQogICAgICAgICAgICAgICAgICAgICBJbmRleCBDb25kOiAodGVzdF9pZCA9 IHRlc3QuaWQpDQogUGxhbm5pbmcgVGltZTogMS4yOTIgbXM= --000000000000ec145b0624acb71f Content-Type: application/octet-stream; name=query Content-Disposition: attachment; filename=query Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_m2ddlpef2 U0VMRUNUIENPVU5UKCopIEZST00gKFNFTEVDVCBESVNUSU5DVCAidGVzdCIuImlkIiBBUyBDb2wx LCAidGVzdCIuInF1ZXN0aW9uX3BhcGVyX2lkIiBBUyBDb2wyLCAidGVzdCIuImdyYWRlX2lkIiBB UyBDb2wzLCAidGVzdCIuImhhc19zdWJfZ3JvdXBzIiBBUyBDb2w0LCAidGVzdCIuImdyb3VwX2lk IiBBUyBDb2w1LCAidGVzdCIuImNlbnRyYWxfcXBfaWQiIEFTIENvbDYsICJ0ZXN0Ii4iaXNfZXJw X3FwIiBBUyBDb2w3LCAidGVzdCIuImlzX3Jldmlld19lbmFibGVkIiBBUyBDb2w4LCAidGVzdCIu InRlc3RfbmFtZSIgQVMgQ29sOSwgInRlc3QiLiJ0ZXN0X2lkIiBBUyBDb2wxMCwgInRlc3QiLiJ0 ZXN0X3R5cGVfaWQiIEFTIENvbDExLCAidGVzdCIuInRlYWNoZXJfaWQiIEFTIENvbDEyLCAidGVz dCIuInRlc3RfbW9kZSIgQVMgQ29sMTMsICJ0ZXN0Ii4idG90YWxfbWFyayIgQVMgQ29sMTQsICJ0 ZXN0Ii4iaXNfcXVlc3Rpb25fd2lzZSIgQVMgQ29sMTUsICJ0ZXN0Ii4idGVzdF9kYXRlIiBBUyBD b2wxNiwgInRlc3QiLiJ0ZXN0X2R1cmF0aW9uIiBBUyBDb2wxNywgInRlc3QiLiJ0ZXN0X2VuZF9k YXRldGltZSIgQVMgQ29sMTgsICJ0ZXN0Ii4iaW5zdHJ1Y3Rpb25zIiBBUyBDb2wxOSwgInRlc3Qi LiJkZXNjcmlwdGlvbnMiIEFTIENvbDIwLCAidGVzdCIuInRvdGFsX3F1ZXN0aW9uIiBBUyBDb2wy MSwgInRlc3QiLiJpc19kZWxldGUiIEFTIENvbDIyLCAidGVzdCIuImlzX2NvbXBsZXRlZCIgQVMg Q29sMjMsICJ0ZXN0Ii4iY3JlYXRlZF9hdCIgQVMgQ29sMjQsICJ0ZXN0Ii4idXBkYXRlZF9hdCIg QVMgQ29sMjUsICJ0ZXN0Ii4iY3JlYXRlZF9ieV9pZCIgQVMgQ29sMjYsICJ0ZXN0Ii4idXBkYXRl ZF9ieV9pZCIgQVMgQ29sMjcsICJ0ZXN0Ii4iaXNfbG9jayIgQVMgQ29sMjgsICJ0ZXN0Ii4iaXNf b21yIiBBUyBDb2wyOSBGUk9NICJ0ZXN0IiBJTk5FUiBKT0lOICJ0ZXN0X3N1YmplY3RzIiBPTiAo InRlc3QiLiJpZCIgPSAidGVzdF9zdWJqZWN0cyIuInRlc3RfaWQiKSBMRUZUIE9VVEVSIEpPSU4g InVzZXJfcmVzcG9uc2UiIE9OICgidGVzdCIuImlkIiA9ICJ1c2VyX3Jlc3BvbnNlIi4idGVzdF9p ZCIpIExFRlQgT1VURVIgSk9JTiAiYXNzZXNzbWVudF9ncm91cHVzZXJzIiBPTiAoInRlc3QiLiJn cm91cF9pZCIgPSAiYXNzZXNzbWVudF9ncm91cHVzZXJzIi4iY3JlYXRlZHVzZXJzaW5mb19wdHJf aWQiKSBMRUZUIE9VVEVSIEpPSU4gInRlc3Rfc2VjdGlvbl9tYXBwaW5nIiBPTiAoInRlc3QiLiJp ZCIgPSAidGVzdF9zZWN0aW9uX21hcHBpbmciLiJ0ZXN0X2lkIikgTEVGVCBPVVRFUiBKT0lOICJh c3Nlc3NtZW50X2dyb3VwdXNlcnNfZXJwdXNlcnMiIE9OICgiYXNzZXNzbWVudF9ncm91cHVzZXJz Ii4iY3JlYXRlZHVzZXJzaW5mb19wdHJfaWQiID0gImFzc2Vzc21lbnRfZ3JvdXB1c2Vyc19lcnB1 c2VycyIuImdyb3VwdXNlcnNfaWQiKSBXSEVSRSAoTk9UICJ0ZXN0Ii4iaXNfZGVsZXRlIiBBTkQg InRlc3Rfc3ViamVjdHMiLiJzdWJqZWN0X2lkIiBJTiAoMTIzNSwgMTIzMSwgMTE4OCwgMTE4Niwg MTE4NCwgMTE4MywgMTE3MiwgMTE3MSwgMTE2MywgMTE1OSwgMTE1OCwgMTE1MiwgMTE1MSwgMTE0 NywgMTE0NiwgMTE0NSwgMTE0NCwgMTE0MywgMTE0MiwgMTEzOCwgMTEzMSwgMTEzMCwgMTEyNSwg MTEyNCwgMTEyMywgMTA5OSwgMTA5NCwgMTA1NCwgMTA1MywgMTA0NiwgMTA0MywgMTA0MSwgMTAz NCwgMTAzMSwgMTAzMCwgMTAyOSwgMTAxNiwgOTkxLCA5MDgsIDg5NCwgODkwLCA4ODQsIDg4MCwg ODc2LCA4NzUsIDg2NywgODYxLCA4NjAsIDg1OCwgODU3LCA4NTUsIDg1NCwgODUxLCA4NDksIDg0 OCwgODQ2LCA4NDUsIDc4NywgNzEsIDcwLCA2OSwgNjAsIDU5LCA1MywgNTIsIDUxLCAzNCwgMzMs IDMyLCAzMSwgMzAsIDI5LCAyNywgMTksIDE3LCAxNiwgMTUsIDE0LCAxMywgMTIsIDEwLCA4LCA3 LCA2LCA0LCAzLCAyLCAxKSBBTkQgInRlc3QiLiJ0ZXN0X3R5cGVfaWQiIElOICgxNSwgMTQsIDEz LCAxMiwgMTEsIDEwLCA5LCA4LCA3LCA2LCA1LCA0LCAzLCAxKSBBTkQgKCJ0ZXN0Ii4idGVzdF9l bmRfZGF0ZXRpbWUiIDw9ICcyMDI0LTEwLTE3VDE4OjQ4OjE3LjEwNTA0Myc6OnRpbWVzdGFtcCBP UiAidXNlcl9yZXNwb25zZSIuInN1Ym1pdHRlZF9ieV9pZCIgPSAxMDgzMTkgT1IgKCJ0ZXN0Ii4i dGVzdF9lbmRfZGF0ZXRpbWUiIElTIE5VTEwgQU5EICJ0ZXN0Ii4idGVzdF90eXBlX2lkIiBJTiAo MTUsIDE0KSBBTkQgInVzZXJfcmVzcG9uc2UiLiJzdWJtaXR0ZWRfYnlfaWQiID0gMTA4MzE5KSkg QU5EICgoInRlc3QiLiJncm91cF9pZCIgSVMgTlVMTCBBTkQgInRlc3Rfc2VjdGlvbl9tYXBwaW5n Ii4ic2VjdGlvbm1hcHBpbmdfaWQiIElOICgxMzc5OTkpKSBPUiAiYXNzZXNzbWVudF9ncm91cHVz ZXJzX2VycHVzZXJzIi4iZXJwdXNlcnNfaWQiIElOICg0NjUxMykpIEFORCBOT1QgKCJ0ZXN0Ii4i aXNfbG9jayIgQU5EICJ0ZXN0Ii4iaXNfbG9jayIgSVMgTk9UIE5VTEwpKSkgc3VicXVlcnk= --000000000000ec145b0624acb71f--