Message-ID: <566EF984.5070404@rwa-net.co.uk>
Date: Mon, 14 Dec 2015 17:16:52 +0000
From: Matthew Lunnon <mlunnon@rwa-net.co.uk>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64;
 rv:31.0) Gecko/20100101 Thunderbird/31.3.0
MIME-Version: 1.0
To: pgsql-performance@postgresql.org
Subject: Performance difference between Slon master and slave
Content-Type: multipart/alternative;
 boundary="------------060801060007070805090403"
Precedence: bulk
Sender: pgsql-performance-owner@postgresql.org

This is a multi-part message in MIME format.
--------------060801060007070805090403
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit

Hello all, I hope someone can help me with this.

Postgres 9.4.4
Slon 2.2.4
Linux

I am using slony-i to replicate a production database which is in the 
order of 70GB. I have a reasonably complex select query that runs in 40 
seconds on the master but takes in the region of 30-40 minutes on the 
slave. The postgres configurations are identical and the machines are a 
similar specifications (12 core hyper threaded HP server and the slave 
has slightly less RAM: 132GB vs 148GB) The server running the slave 
database has a higher load than the one running the master though the 
load average on the slave machine was low (1-2) when running the test 
and the postgres process on the slave machine runs at 100% of a CPU with 
very little iowait on the server.

Inspecting the execution plan shows that there are some differences, for 
example, the slave is using a HashAggregate when the master is simply 
grouping. There also seems to be a difference with the ordering of the 
sub plans. Armed with this knowledge I have set enable_hashagg to off 
and run the query again and it now takes 53 seconds on the slave which 
is a more acceptable difference and the execution plans now look very 
similar (one difference being that there is another HashAggregate in the 
master which is now missing on the slave and may account for the 13 
seconds). I have isolated a much simpler query which I have detailed 
below with their execution plans which shows the difference on line 4. I 
would rather not disable hash aggregation on the slave as this might 
have other consequences so this raises a number of questions. Firstly Is 
there anything that I can do to stop this feature? Why is the slave 
behaving differently to the master?

Thanks in advance for any help.

Cheers
Matthew

explain
with my_view_booking_pax_breakdown as (
SELECT bev.booking_id,
     ( SELECT count(*) AS count
            FROM passenger_version
           WHERE passenger_version.current_version = 'T'::bpchar AND 
passenger_version.deleted = 'F'::bpchar AND 
passenger_version.indicative_pax_type = 'A'::bpchar AND 
passenger_version.booking_id = bev.booking_id) AS adult_count,
     ( SELECT count(*) AS count
            FROM passenger_version
           WHERE passenger_version.current_version = 'T'::bpchar AND 
passenger_version.deleted = 'F'::bpchar AND 
passenger_version.indicative_pax_type = 'C'::bpchar AND 
passenger_version.booking_id = bev.booking_id) AS child_count,
     ( SELECT count(*) AS count
            FROM passenger_version
           WHERE passenger_version.current_version = 'T'::bpchar AND 
passenger_version.deleted = 'F'::bpchar AND 
passenger_version.indicative_pax_type = 'I'::bpchar AND 
passenger_version.booking_id = bev.booking_id) AS infant_count
    FROM booking_expanded_version bev
   GROUP BY bev.booking_id
)
select * from "my_view_booking_pax_breakdown" "view_booking_pax_breakdown"
     INNER JOIN "booking"."booking_expanded_version" 
"booking_expanded_version" ON 
"view_booking_pax_breakdown"."booking_id"="booking_expanded_version"."booking_id" 


Master

"Merge Join  (cost=5569138.32..6158794.12 rows=2461265 width=1375)"
"  Merge Cond: (booking_expanded_version.booking_id = 
view_booking_pax_breakdown.booking_id)"
"  CTE my_view_booking_pax_breakdown"
*"    ->  Group  (cost=0.43..5545692.19 rows=215891 width=4)"*
"          Group Key: bev.booking_id"
"          ->  Index Only Scan using 
booking_expanded_version_booking_idx on booking_expanded_version bev  
(cost=0.43..64607.40 rows=2461265 width=4)"
"          SubPlan 1"
"            ->  Aggregate  (cost=8.57..8.58 rows=1 width=0)"
"                  ->  Index Scan using passenger_version_idx_4 on 
passenger_version  (cost=0.43..8.55 rows=5 width=0)"
"                        Index Cond: (booking_id = bev.booking_id)"
"          SubPlan 2"
"            ->  Aggregate  (cost=8.45..8.46 rows=1 width=0)"
"                  ->  Index Scan using passenger_version_idx_3 on 
passenger_version passenger_version_1  (cost=0.42..8.45 rows=1 width=0)"
"                        Index Cond: (booking_id = bev.booking_id)"
"          SubPlan 3"
"            ->  Aggregate  (cost=8.31..8.32 rows=1 width=0)"
"                  ->  Index Scan using passenger_version_idx_2 on 
passenger_version passenger_version_2  (cost=0.29..8.31 rows=1 width=0)"
"                        Index Cond: (booking_id = bev.booking_id)"
"  ->  Index Scan using booking_expanded_version_booking_idx on 
booking_expanded_version  (cost=0.43..546584.09 rows=2461265 width=1347)"
"  ->  Sort  (cost=23445.70..23985.43 rows=215891 width=28)"
"        Sort Key: view_booking_pax_breakdown.booking_id"
"        ->  CTE Scan on my_view_booking_pax_breakdown 
view_booking_pax_breakdown  (cost=0.00..4317.82 rows=215891 width=28)"

Slave

"Merge Join  (cost=6168518.91..6764756.86 rows=2505042 width=1299)"
"  Merge Cond: (booking_expanded_version.booking_id = 
view_booking_pax_breakdown.booking_id)"
"  CTE my_view_booking_pax_breakdown"
*"    ->  HashAggregate  (cost=212185.03..6142965.53 rows=234040 width=4)"*
"          Group Key: bev.booking_id"
"          ->  Seq Scan on booking_expanded_version bev 
(cost=0.00..205922.42 rows=2505042 width=4)"
"          SubPlan 1"
"            ->  Aggregate  (cost=8.54..8.55 rows=1 width=0)"
"                  ->  Index Scan using passenger_version_idx_4 on 
passenger_version  (cost=0.43..8.53 rows=4 width=0)"
"                        Index Cond: (booking_id = bev.booking_id)"
"          SubPlan 2"
"            ->  Aggregate  (cost=8.45..8.46 rows=1 width=0)"
"                  ->  Index Scan using passenger_version_idx_3 on 
passenger_version passenger_version_1  (cost=0.42..8.45 rows=1 width=0)"
"                        Index Cond: (booking_id = bev.booking_id)"
"          SubPlan 3"
"            ->  Aggregate  (cost=8.31..8.32 rows=1 width=0)"
"                  ->  Index Scan using passenger_version_idx_2 on 
passenger_version passenger_version_2  (cost=0.29..8.31 rows=1 width=0)"
"                        Index Cond: (booking_id = bev.booking_id)"
"  ->  Index Scan using booking_expanded_version_booking_idx on 
booking_expanded_version  (cost=0.43..552400.15 rows=2505042 width=1271)"
"  ->  Sort  (cost=25552.95..26138.05 rows=234040 width=28)"
"        Sort Key: view_booking_pax_breakdown.booking_id"
"        ->  CTE Scan on my_view_booking_pax_breakdown 
view_booking_pax_breakdown  (cost=0.00..4680.80 rows=234040 width=28)"


This message has been scanned for malware by Websense. www.websense.com

--------------060801060007070805090403
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: base64

PGh0bWw+CiAgPGhlYWQ+CgogICAgPG1ldGEgaHR0cC1lcXVpdj0iY29udGVu
dC10eXBlIiBjb250ZW50PSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgiPgog
IDwvaGVhZD4KICA8Ym9keSBiZ2NvbG9yPSIjRkZGRkZGIiB0ZXh0PSIjMDAw
MDAwIj4KICAgIEhlbGxvIGFsbCwgSSBob3BlIHNvbWVvbmUgY2FuIGhlbHAg
bWUgd2l0aCB0aGlzLjxicj4KICAgIDxicj4KICAgIFBvc3RncmVzIDkuNC40
PGJyPgogICAgU2xvbiAyLjIuNDxicj4KICAgIExpbnV4IDxicj4KICAgIDxi
cj4KICAgIEkgYW0gdXNpbmcgc2xvbnktaSB0byByZXBsaWNhdGUgYSBwcm9k
dWN0aW9uIGRhdGFiYXNlIHdoaWNoIGlzIGluCiAgICB0aGUgb3JkZXIgb2Yg
NzBHQi4gSSBoYXZlIGEgcmVhc29uYWJseSBjb21wbGV4IHNlbGVjdCBxdWVy
eSB0aGF0CiAgICBydW5zIGluIDQwIHNlY29uZHMgb24gdGhlIG1hc3RlciBi
dXQgdGFrZXMgaW4gdGhlIHJlZ2lvbiBvZiAzMC00MAogICAgbWludXRlcyBv
biB0aGUgc2xhdmUuIFRoZSBwb3N0Z3JlcyBjb25maWd1cmF0aW9ucyBhcmUg
aWRlbnRpY2FsIGFuZAogICAgdGhlIG1hY2hpbmVzIGFyZSBhIHNpbWlsYXIg
c3BlY2lmaWNhdGlvbnMgKDEyIGNvcmUgaHlwZXIgdGhyZWFkZWQgSFAKICAg
IHNlcnZlciBhbmQgdGhlIHNsYXZlIGhhcyBzbGlnaHRseSBsZXNzIFJBTTog
MTMyR0IgdnMgMTQ4R0IpIFRoZQogICAgc2VydmVyIHJ1bm5pbmcgdGhlIHNs
YXZlIGRhdGFiYXNlIGhhcyBhIGhpZ2hlciBsb2FkIHRoYW4gdGhlIG9uZQog
ICAgcnVubmluZyB0aGUgbWFzdGVyIHRob3VnaCB0aGUgbG9hZCBhdmVyYWdl
IG9uIHRoZSBzbGF2ZSBtYWNoaW5lIHdhcwogICAgbG93ICgxLTIpIHdoZW4g
cnVubmluZyB0aGUgdGVzdCBhbmQgdGhlIHBvc3RncmVzIHByb2Nlc3Mgb24g
dGhlCiAgICBzbGF2ZSBtYWNoaW5lIHJ1bnMgYXQgMTAwJSBvZiBhIENQVSB3
aXRoIHZlcnkgbGl0dGxlIGlvd2FpdCBvbiB0aGUKICAgIHNlcnZlci48YnI+
CiAgICA8YnI+CiAgICBJbnNwZWN0aW5nIHRoZSBleGVjdXRpb24gcGxhbiBz
aG93cyB0aGF0IHRoZXJlIGFyZSBzb21lIGRpZmZlcmVuY2VzLAogICAgZm9y
IGV4YW1wbGUsIHRoZSBzbGF2ZSBpcyB1c2luZyBhIEhhc2hBZ2dyZWdhdGUg
d2hlbiB0aGUgbWFzdGVyIGlzCiAgICBzaW1wbHkgZ3JvdXBpbmcuIFRoZXJl
IGFsc28gc2VlbXMgdG8gYmUgYSBkaWZmZXJlbmNlIHdpdGggdGhlCiAgICBv
cmRlcmluZyBvZiB0aGUgc3ViIHBsYW5zLiBBcm1lZCB3aXRoIHRoaXMga25v
d2xlZGdlIEkgaGF2ZSBzZXQKICAgIGVuYWJsZV9oYXNoYWdnIHRvIG9mZiBh
bmQgcnVuIHRoZSBxdWVyeSBhZ2FpbiBhbmQgaXQgbm93IHRha2VzIDUzCiAg
ICBzZWNvbmRzIG9uIHRoZSBzbGF2ZSB3aGljaCBpcyBhIG1vcmUgYWNjZXB0
YWJsZSBkaWZmZXJlbmNlIGFuZCB0aGUKICAgIGV4ZWN1dGlvbiBwbGFucyBu
b3cgbG9vayB2ZXJ5IHNpbWlsYXIgKG9uZSBkaWZmZXJlbmNlIGJlaW5nIHRo
YXQKICAgIHRoZXJlIGlzIGFub3RoZXIgSGFzaEFnZ3JlZ2F0ZSBpbiB0aGUg
bWFzdGVyIHdoaWNoIGlzIG5vdyBtaXNzaW5nIG9uCiAgICB0aGUgc2xhdmUg
YW5kIG1heSBhY2NvdW50IGZvciB0aGUgMTMgc2Vjb25kcykuIEkgaGF2ZSBp
c29sYXRlZCBhCiAgICBtdWNoIHNpbXBsZXIgcXVlcnkgd2hpY2ggSSBoYXZl
IGRldGFpbGVkIGJlbG93IHdpdGggdGhlaXIgZXhlY3V0aW9uCiAgICBwbGFu
cyB3aGljaCBzaG93cyB0aGUgZGlmZmVyZW5jZSBvbiBsaW5lIDQuIEkgd291
bGQgcmF0aGVyIG5vdAogICAgZGlzYWJsZSBoYXNoIGFnZ3JlZ2F0aW9uIG9u
IHRoZSBzbGF2ZSBhcyB0aGlzIG1pZ2h0IGhhdmUgb3RoZXIKICAgIGNvbnNl
cXVlbmNlcyBzbyB0aGlzIHJhaXNlcyBhIG51bWJlciBvZiBxdWVzdGlvbnMu
IEZpcnN0bHkgSXMgdGhlcmUKICAgIGFueXRoaW5nIHRoYXQgSSBjYW4gZG8g
dG8gc3RvcCB0aGlzIGZlYXR1cmU/IFdoeSBpcyB0aGUgc2xhdmUKICAgIGJl
aGF2aW5nIGRpZmZlcmVudGx5IHRvIHRoZSBtYXN0ZXI/IDxicj4KICAgIDxi
cj4KICAgIFRoYW5rcyBpbiBhZHZhbmNlIGZvciBhbnkgaGVscC48YnI+CiAg
ICA8YnI+CiAgICBDaGVlcnM8YnI+CiAgICBNYXR0aGV3PGJyPgogICAgPGJy
PgogICAgZXhwbGFpbjxicj4KICAgIHdpdGggbXlfdmlld19ib29raW5nX3Bh
eF9icmVha2Rvd24gYXMgKDxicj4KICAgIFNFTEVDVCBiZXYuYm9va2luZ19p
ZCw8YnI+CiAgICDCoMKgwqAgKCBTRUxFQ1QgY291bnQoKikgQVMgY291bnQ8
YnI+CiAgICDCoMKgwqDCoMKgwqDCoMKgwqDCoCBGUk9NIHBhc3Nlbmdlcl92
ZXJzaW9uPGJyPgogICAgwqDCoMKgwqDCoMKgwqDCoMKgIFdIRVJFIHBhc3Nl
bmdlcl92ZXJzaW9uLmN1cnJlbnRfdmVyc2lvbiA9ICdUJzo6YnBjaGFyIEFO
RAogICAgcGFzc2VuZ2VyX3ZlcnNpb24uZGVsZXRlZCA9ICdGJzo6YnBjaGFy
IEFORAogICAgcGFzc2VuZ2VyX3ZlcnNpb24uaW5kaWNhdGl2ZV9wYXhfdHlw
ZSA9ICdBJzo6YnBjaGFyIEFORAogICAgcGFzc2VuZ2VyX3ZlcnNpb24uYm9v
a2luZ19pZCA9IGJldi5ib29raW5nX2lkKSBBUyBhZHVsdF9jb3VudCw8YnI+
CiAgICDCoMKgwqAgKCBTRUxFQ1QgY291bnQoKikgQVMgY291bnQ8YnI+CiAg
ICDCoMKgwqDCoMKgwqDCoMKgwqDCoCBGUk9NIHBhc3Nlbmdlcl92ZXJzaW9u
PGJyPgogICAgwqDCoMKgwqDCoMKgwqDCoMKgIFdIRVJFIHBhc3Nlbmdlcl92
ZXJzaW9uLmN1cnJlbnRfdmVyc2lvbiA9ICdUJzo6YnBjaGFyIEFORAogICAg
cGFzc2VuZ2VyX3ZlcnNpb24uZGVsZXRlZCA9ICdGJzo6YnBjaGFyIEFORAog
ICAgcGFzc2VuZ2VyX3ZlcnNpb24uaW5kaWNhdGl2ZV9wYXhfdHlwZSA9ICdD
Jzo6YnBjaGFyIEFORAogICAgcGFzc2VuZ2VyX3ZlcnNpb24uYm9va2luZ19p
ZCA9IGJldi5ib29raW5nX2lkKSBBUyBjaGlsZF9jb3VudCw8YnI+CiAgICDC
oMKgwqAgKCBTRUxFQ1QgY291bnQoKikgQVMgY291bnQ8YnI+CiAgICDCoMKg
wqDCoMKgwqDCoMKgwqDCoCBGUk9NIHBhc3Nlbmdlcl92ZXJzaW9uPGJyPgog
ICAgwqDCoMKgwqDCoMKgwqDCoMKgIFdIRVJFIHBhc3Nlbmdlcl92ZXJzaW9u
LmN1cnJlbnRfdmVyc2lvbiA9ICdUJzo6YnBjaGFyIEFORAogICAgcGFzc2Vu
Z2VyX3ZlcnNpb24uZGVsZXRlZCA9ICdGJzo6YnBjaGFyIEFORAogICAgcGFz
c2VuZ2VyX3ZlcnNpb24uaW5kaWNhdGl2ZV9wYXhfdHlwZSA9ICdJJzo6YnBj
aGFyIEFORAogICAgcGFzc2VuZ2VyX3ZlcnNpb24uYm9va2luZ19pZCA9IGJl
di5ib29raW5nX2lkKSBBUyBpbmZhbnRfY291bnQ8YnI+CiAgICDCoMKgIEZS
T00gYm9va2luZ19leHBhbmRlZF92ZXJzaW9uIGJldjxicj4KICAgIMKgIEdS
T1VQIEJZIGJldi5ib29raW5nX2lkPGJyPgogICAgKTxicj4KICAgIHNlbGVj
dCAqIGZyb20gIm15X3ZpZXdfYm9va2luZ19wYXhfYnJlYWtkb3duIgogICAg
InZpZXdfYm9va2luZ19wYXhfYnJlYWtkb3duIjxicj4KICAgIMKgwqDCoCBJ
Tk5FUiBKT0lOICJib29raW5nIi4iYm9va2luZ19leHBhbmRlZF92ZXJzaW9u
IgogICAgImJvb2tpbmdfZXhwYW5kZWRfdmVyc2lvbiIgT04KICAgICJ2aWV3
X2Jvb2tpbmdfcGF4X2JyZWFrZG93biIuImJvb2tpbmdfaWQiPSJib29raW5n
X2V4cGFuZGVkX3ZlcnNpb24iLiJib29raW5nX2lkIgogICAgPGJyPgogICAg
wqDCoCDCoDxicj4KICAgIE1hc3Rlcjxicj4KICAgIDxicj4KICAgICJNZXJn
ZSBKb2luwqAgKGNvc3Q9NTU2OTEzOC4zMi4uNjE1ODc5NC4xMiByb3dzPTI0
NjEyNjUgd2lkdGg9MTM3NSkiPGJyPgogICAgIsKgIE1lcmdlIENvbmQ6IChi
b29raW5nX2V4cGFuZGVkX3ZlcnNpb24uYm9va2luZ19pZCA9CiAgICB2aWV3
X2Jvb2tpbmdfcGF4X2JyZWFrZG93bi5ib29raW5nX2lkKSI8YnI+CiAgICAi
wqAgQ1RFIG15X3ZpZXdfYm9va2luZ19wYXhfYnJlYWtkb3duIjxicj4KICAg
IDxiPiLCoMKgwqAgLSZndDvCoCBHcm91cMKgIChjb3N0PTAuNDMuLjU1NDU2
OTIuMTkgcm93cz0yMTU4OTEgd2lkdGg9NCkiPC9iPjxicj4KICAgICLCoMKg
wqDCoMKgwqDCoMKgwqAgR3JvdXAgS2V5OiBiZXYuYm9va2luZ19pZCI8YnI+
CiAgICAiwqDCoMKgwqDCoMKgwqDCoMKgIC0mZ3Q7wqAgSW5kZXggT25seSBT
Y2FuIHVzaW5nCiAgICBib29raW5nX2V4cGFuZGVkX3ZlcnNpb25fYm9va2lu
Z19pZHggb24gYm9va2luZ19leHBhbmRlZF92ZXJzaW9uCiAgICBiZXbCoCAo
Y29zdD0wLjQzLi42NDYwNy40MCByb3dzPTI0NjEyNjUgd2lkdGg9NCkiPGJy
PgogICAgIsKgwqDCoMKgwqDCoMKgwqDCoCBTdWJQbGFuIDEiPGJyPgogICAg
IsKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgLSZndDvCoCBBZ2dyZWdhdGXCoCAo
Y29zdD04LjU3Li44LjU4IHJvd3M9MSB3aWR0aD0wKSI8YnI+CiAgICAiwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAtJmd0O8KgIEluZGV4
IFNjYW4gdXNpbmcgcGFzc2VuZ2VyX3ZlcnNpb25faWR4XzQKICAgIG9uIHBh
c3Nlbmdlcl92ZXJzaW9uwqAgKGNvc3Q9MC40My4uOC41NSByb3dzPTUgd2lk
dGg9MCkiPGJyPgogICAgIsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqAgSW5kZXggQ29uZDogKGJvb2tpbmdfaWQgPSBi
ZXYuYm9va2luZ19pZCkiPGJyPgogICAgIsKgwqDCoMKgwqDCoMKgwqDCoCBT
dWJQbGFuIDIiPGJyPgogICAgIsKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgLSZn
dDvCoCBBZ2dyZWdhdGXCoCAoY29zdD04LjQ1Li44LjQ2IHJvd3M9MSB3aWR0
aD0wKSI8YnI+CiAgICAiwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoCAtJmd0O8KgIEluZGV4IFNjYW4gdXNpbmcgcGFzc2VuZ2VyX3ZlcnNp
b25faWR4XzMKICAgIG9uIHBhc3Nlbmdlcl92ZXJzaW9uIHBhc3Nlbmdlcl92
ZXJzaW9uXzHCoCAoY29zdD0wLjQyLi44LjQ1IHJvd3M9MQogICAgd2lkdGg9
MCkiPGJyPgogICAgIsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqAgSW5kZXggQ29uZDogKGJvb2tpbmdfaWQgPSBiZXYu
Ym9va2luZ19pZCkiPGJyPgogICAgIsKgwqDCoMKgwqDCoMKgwqDCoCBTdWJQ
bGFuIDMiPGJyPgogICAgIsKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgLSZndDvC
oCBBZ2dyZWdhdGXCoCAoY29zdD04LjMxLi44LjMyIHJvd3M9MSB3aWR0aD0w
KSI8YnI+CiAgICAiwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oCAtJmd0O8KgIEluZGV4IFNjYW4gdXNpbmcgcGFzc2VuZ2VyX3ZlcnNpb25f
aWR4XzIKICAgIG9uIHBhc3Nlbmdlcl92ZXJzaW9uIHBhc3Nlbmdlcl92ZXJz
aW9uXzLCoCAoY29zdD0wLjI5Li44LjMxIHJvd3M9MQogICAgd2lkdGg9MCki
PGJyPgogICAgIsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqAgSW5kZXggQ29uZDogKGJvb2tpbmdfaWQgPSBiZXYuYm9v
a2luZ19pZCkiPGJyPgogICAgIsKgIC0mZ3Q7wqAgSW5kZXggU2NhbiB1c2lu
ZyBib29raW5nX2V4cGFuZGVkX3ZlcnNpb25fYm9va2luZ19pZHggb24KICAg
IGJvb2tpbmdfZXhwYW5kZWRfdmVyc2lvbsKgIChjb3N0PTAuNDMuLjU0NjU4
NC4wOSByb3dzPTI0NjEyNjUKICAgIHdpZHRoPTEzNDcpIjxicj4KICAgICLC
oCAtJmd0O8KgIFNvcnTCoCAoY29zdD0yMzQ0NS43MC4uMjM5ODUuNDMgcm93
cz0yMTU4OTEgd2lkdGg9MjgpIjxicj4KICAgICLCoMKgwqDCoMKgwqDCoCBT
b3J0IEtleTogdmlld19ib29raW5nX3BheF9icmVha2Rvd24uYm9va2luZ19p
ZCI8YnI+CiAgICAiwqDCoMKgwqDCoMKgwqAgLSZndDvCoCBDVEUgU2NhbiBv
biBteV92aWV3X2Jvb2tpbmdfcGF4X2JyZWFrZG93bgogICAgdmlld19ib29r
aW5nX3BheF9icmVha2Rvd27CoCAoY29zdD0wLjAwLi40MzE3LjgyIHJvd3M9
MjE1ODkxCiAgICB3aWR0aD0yOCkiPGJyPgogICAgPGJyPgogICAgU2xhdmU8
YnI+CiAgICA8YnI+CiAgICAiTWVyZ2UgSm9pbsKgIChjb3N0PTYxNjg1MTgu
OTEuLjY3NjQ3NTYuODYgcm93cz0yNTA1MDQyIHdpZHRoPTEyOTkpIjxicj4K
ICAgICLCoCBNZXJnZSBDb25kOiAoYm9va2luZ19leHBhbmRlZF92ZXJzaW9u
LmJvb2tpbmdfaWQgPQogICAgdmlld19ib29raW5nX3BheF9icmVha2Rvd24u
Ym9va2luZ19pZCkiPGJyPgogICAgIsKgIENURSBteV92aWV3X2Jvb2tpbmdf
cGF4X2JyZWFrZG93biI8YnI+CiAgICA8Yj4iwqDCoMKgIC0mZ3Q7wqAgSGFz
aEFnZ3JlZ2F0ZcKgIChjb3N0PTIxMjE4NS4wMy4uNjE0Mjk2NS41MwogICAg
ICByb3dzPTIzNDA0MCB3aWR0aD00KSI8L2I+PGJyPgogICAgIsKgwqDCoMKg
wqDCoMKgwqDCoCBHcm91cCBLZXk6IGJldi5ib29raW5nX2lkIjxicj4KICAg
ICLCoMKgwqDCoMKgwqDCoMKgwqAgLSZndDvCoCBTZXEgU2NhbiBvbiBib29r
aW5nX2V4cGFuZGVkX3ZlcnNpb24gYmV2wqAKICAgIChjb3N0PTAuMDAuLjIw
NTkyMi40MiByb3dzPTI1MDUwNDIgd2lkdGg9NCkiPGJyPgogICAgIsKgwqDC
oMKgwqDCoMKgwqDCoCBTdWJQbGFuIDEiPGJyPgogICAgIsKgwqDCoMKgwqDC
oMKgwqDCoMKgwqAgLSZndDvCoCBBZ2dyZWdhdGXCoCAoY29zdD04LjU0Li44
LjU1IHJvd3M9MSB3aWR0aD0wKSI8YnI+CiAgICAiwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoCAtJmd0O8KgIEluZGV4IFNjYW4gdXNpbmcg
cGFzc2VuZ2VyX3ZlcnNpb25faWR4XzQKICAgIG9uIHBhc3Nlbmdlcl92ZXJz
aW9uwqAgKGNvc3Q9MC40My4uOC41MyByb3dzPTQgd2lkdGg9MCkiPGJyPgog
ICAgIsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqAgSW5kZXggQ29uZDogKGJvb2tpbmdfaWQgPSBiZXYuYm9va2luZ19p
ZCkiPGJyPgogICAgIsKgwqDCoMKgwqDCoMKgwqDCoCBTdWJQbGFuIDIiPGJy
PgogICAgIsKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgLSZndDvCoCBBZ2dyZWdh
dGXCoCAoY29zdD04LjQ1Li44LjQ2IHJvd3M9MSB3aWR0aD0wKSI8YnI+CiAg
ICAiwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAtJmd0O8Kg
IEluZGV4IFNjYW4gdXNpbmcgcGFzc2VuZ2VyX3ZlcnNpb25faWR4XzMKICAg
IG9uIHBhc3Nlbmdlcl92ZXJzaW9uIHBhc3Nlbmdlcl92ZXJzaW9uXzHCoCAo
Y29zdD0wLjQyLi44LjQ1IHJvd3M9MQogICAgd2lkdGg9MCkiPGJyPgogICAg
IsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqAgSW5kZXggQ29uZDogKGJvb2tpbmdfaWQgPSBiZXYuYm9va2luZ19pZCki
PGJyPgogICAgIsKgwqDCoMKgwqDCoMKgwqDCoCBTdWJQbGFuIDMiPGJyPgog
ICAgIsKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgLSZndDvCoCBBZ2dyZWdhdGXC
oCAoY29zdD04LjMxLi44LjMyIHJvd3M9MSB3aWR0aD0wKSI8YnI+CiAgICAi
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAtJmd0O8KgIElu
ZGV4IFNjYW4gdXNpbmcgcGFzc2VuZ2VyX3ZlcnNpb25faWR4XzIKICAgIG9u
IHBhc3Nlbmdlcl92ZXJzaW9uIHBhc3Nlbmdlcl92ZXJzaW9uXzLCoCAoY29z
dD0wLjI5Li44LjMxIHJvd3M9MQogICAgd2lkdGg9MCkiPGJyPgogICAgIsKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAg
SW5kZXggQ29uZDogKGJvb2tpbmdfaWQgPSBiZXYuYm9va2luZ19pZCkiPGJy
PgogICAgIsKgIC0mZ3Q7wqAgSW5kZXggU2NhbiB1c2luZyBib29raW5nX2V4
cGFuZGVkX3ZlcnNpb25fYm9va2luZ19pZHggb24KICAgIGJvb2tpbmdfZXhw
YW5kZWRfdmVyc2lvbsKgIChjb3N0PTAuNDMuLjU1MjQwMC4xNSByb3dzPTI1
MDUwNDIKICAgIHdpZHRoPTEyNzEpIjxicj4KICAgICLCoCAtJmd0O8KgIFNv
cnTCoCAoY29zdD0yNTU1Mi45NS4uMjYxMzguMDUgcm93cz0yMzQwNDAgd2lk
dGg9MjgpIjxicj4KICAgICLCoMKgwqDCoMKgwqDCoCBTb3J0IEtleTogdmll
d19ib29raW5nX3BheF9icmVha2Rvd24uYm9va2luZ19pZCI8YnI+CiAgICAi
wqDCoMKgwqDCoMKgwqAgLSZndDvCoCBDVEUgU2NhbiBvbiBteV92aWV3X2Jv
b2tpbmdfcGF4X2JyZWFrZG93bgogICAgdmlld19ib29raW5nX3BheF9icmVh
a2Rvd27CoCAoY29zdD0wLjAwLi40NjgwLjgwIHJvd3M9MjM0MDQwCiAgICB3
aWR0aD0yOCkiPGJyPgogICAgPGJyPgogIDxicj48YnI+CjxQIGFsaWduPWNl
bnRlcj48Rk9OVCBzdHlsZT0iQkFDS0dST1VORC1DT0xPUjogI2ZmZmZmZiI+
VGhpcyBtZXNzYWdlIGhhcyBiZWVuIHNjYW5uZWQgZm9yIG1hbHdhcmUgYnkg
V2Vic2Vuc2UuICA8L0ZPTlQ+PEEgaHJlZj0iaHR0cDovL3d3dy53ZWJzZW5z
ZS5jb20vIj48Rk9OVCBzdHlsZT0iQkFDS0dST1VORC1DT0xPUjogI2ZmZmZm
ZiIgY29sb3I9IzAwMDAwMD53d3cud2Vic2Vuc2UuY29tPC9GT05UPjwvQT48
L1A+CjwvYm9keT4KPC9odG1sPgo=

--------------060801060007070805090403--