Date: Mon, 11 Feb 2019 08:05:50 +0000 (UTC)
From: keith anderson <kwj.anderson@yahoo.co.uk>
To: "pgsql-performance@postgresql.org" <pgsql-performance@postgresql.org>
Message-ID: <1270348147.2230544.1549872350935@mail.yahoo.com>
In-Reply-To: <20190208130444.GC29720@telsasoft.com>
References: <1233297695.414916.1549624431535.ref@mail.yahoo.com>
 <1233297695.414916.1549624431535@mail.yahoo.com>
 <20190208130444.GC29720@telsasoft.com>
Subject: Re: Partitioning Optimizer Questions and Issues
MIME-Version: 1.0
Content-Type: multipart/mixed;
	boundary="----=_Part_2230543_275789688.1549872350935"
Content-Length: 25379
Precedence: bulk

------=_Part_2230543_275789688.1549872350935
Content-Type: multipart/alternative;
	boundary="----=_Part_2230542_1821760923.1549872350895"

------=_Part_2230542_1821760923.1549872350895
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

 Thanks for the feedback Justin.

You are right, the most-common-values list is empty for my test case and so=
 it is using n_distinct for the 'id IN()' scenario.And I can see that with =
the pg_class.reltuples and the pg_stats.histogram_bounds values how the opt=
imizer can conclude with my range query that only 1 in 5 entries in my rang=
e query are in each individual partition.
However, I can also see that the pg_stats.n_distinct value for tablea shows=
 -1, as do all the individual child partitions. In my opinion it makes sens=
e for the optimizer when using n_distinct on partitioned tables to use the =
n_distinct value of the parent table level when estimating row counts rathe=
r than a sum of the partition level statistics. Or can someone come up with=
 a good reason to avoid this?=C2=A0
A couple of examples of different data in a partitioned table:
- Unique identifier -> if providing a single value in a query -> using n_di=
stinct from parent will estimate 1, using child tables will be 1 * (number =
of partitions). Use of parent table would be correct.- Date of activity, wi=
th=C2=A0 1000 records per day -> if providing a single day to the query -> =
using n_distinct from parent would show 1000 rows returned, using child tab=
les will be 1000 * (number of partitions). Use of parent table n_distinct i=
s correct.
Perhaps when querying on columns that are part of the partition logic you c=
ould use the partition level stats, but I think the vast majority of the ti=
me, using the parent statistics would be much more reliable/accurate than s=
umming across partitions.
In terms of the partition strategy, I agree that it should be done with a v=
iew to helping performance improve. I will look into more detail at your su=
ggestions, but in general it is very hard to use effectively as there are c=
ompeting priorities:
- I would like to not have to manage massive numbers of partitions- I would=
 like to be able to archive data easily using date (a big plus point to the=
 existing date partitioning strategy)- It is hard in most cases to come up =
with a partition strategy that allows for partition elimination e.g. consid=
er a common 'transaction record' table with a primary key, an account ident=
ifier, and a date -> it is natural to want to be able to query on any one o=
f these, but as things stand it cannot be achieved performantly with partit=
ioning.
Global index support feels like it has potential to resolve many of the iss=
ues I have with partitioning (beyond the optimizer concern above). I assume=
 this has been discussed and rejected though by the community?
I've attached as a file the original test script.
Keith


    On Friday, 8 February 2019, 13:05:04 GMT, Justin Pryzby <pryzby@telsaso=
ft.com> wrote: =20
=20
 On Fri, Feb 08, 2019 at 11:13:51AM +0000, keith anderson wrote:
> So to summarise the findings/questions from above:
> - It seems like the Postgres optimizer sometimes uses the partition level=
 statistics, and sometimes the global table level statistics? Or is it usin=
g something else?- With partitioning tables with unique identifier and retr=
ieving explicitly on those identifiers, at present the optimizer will alway=
s understimate the selectivity and overestimate the rows returned. This ina=
ccuracy increases in proportion to the number of partitions.- As a result, =
when joining to other tables, you are liable to hitting sequential scans. T=
his becomes more likely as you have more partitions or if join to more part=
itioned tables (note I am aware I could try and tune random_page_cost to tr=
y and prevent this).- To me in the examples queries described above, it mak=
es sense to use the partition statistics for the partition level access str=
ategy, but the global statistics when estimating the actual rows returned b=
y all the individual partition queries. Is there a reason not to do this? O=
r do others believe the optimizer is doing the right thing here?
> And then some general questions:
> - How do other people use partitioning but without a significant performa=
nce disadvantage on reading the data? Is there something else I should be d=
oing here to achieve the same thing without the overhead? At present my rea=
ds have increased optimization cost (as it needs to optimize access to each=
 partition) and also execution cost (access the index on every partition). =
Even without the optimizer issues described above, the cost of reading simp=
le data is extremely high relative to non-partitioned data (unless you use =
the partition key as a filter for each table to eliminate those partitions)=
.- Is there any chance/plan to add global indexes to postgres? If so would =
that impact significantly the cost of the partition drop e.g. to clean up t=
he index.
> Thanks in advance for any feedback/support,

An equality or IN() query will use the pg_stats most-common-values list,
whereas a range query will use the histogram.

The tables probably doesn't have complete MCV list.=C2=A0 By default, that'=
s limited
to 100 entries.=C2=A0 Since the maximum allowed by ALTER..SET STATISTICS is=
 10k, I
don't think it'll help to change it (at least for your production use case)=
.
Each partition's rowcount appears to be estimated from its ndistinct, and n=
ot
from its content, so each is estimated as having about the same rowcount.

Your partitions are sharing a sequence for their ID column, which causes th=
e
DEFAULT IDs to be unique...but their global uniqueness isn't enforced nor
guaranteed.

Note, in postgres11, it's possible to create an index on the parent table.
It's NOT a global index, but it can be unique if it includes the partition =
key.
I don't know how closely your example describes your real use case, but I d=
on't
think that helps with your example; it doesn't seems useful to partition on=
 a
serial column.

You seem to be adding unnecessary CHECK constraints that duplicate the
partition bounds.=C2=A0 Note, it's still useful to include CHECK constraint=
s on key
column if you're planning on DETACHing and re-ATTACHing the partitions, in
order to avoid seqscan to verify tuples don't violate specified bounds.

You might need to rethink your partitioning scheme - you should choose one =
that
causes performance to improve, and probably naturally includes the partitio=
n
key in most queries.

Perhaps you'd use 2 levels of partitioning: a RANGE partition by date, whic=
h
allows for archiving, and a HASH partition by ID, which allows for partitio=
n
pruning.=C2=A0 Note that it's also possible to partition on multiple keys, =
like
RANGE(id,date) - I don't think that's useful here, though.=C2=A0 PG11 also =
allows a
"default" partition.

Or perhaps you could partition by RANGE(date) but add CHECK constraints on =
ID,
after the table is fully populated, to optimize queries by allowing for
partition pruning.

Or you could maybe change the ID column to include the timestamp (like BIND
zonesfiles YYYYMMDDNNNNNNNN).=C2=A0 You'd set a bigint sequence on each par=
tition's
ID as default to the beginning of the month.=C2=A0 A bigint is enough to ha=
ndle
5*10^4 times your volume: 20190401000020111222.=C2=A0 (I think this is tryi=
ng to be
unnecessarily clever, unless there's some reason the other two ideas don't
work.)

Justin
 =20
------=_Part_2230542_1821760923.1549872350895
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<html><head></head><body><div class=3D"ydpa2645a0yahoo-style-wrap" style=3D=
"font-family: Helvetica Neue, Helvetica, Arial, sans-serif; font-size: 13px=
;"><div></div>
        <div><span><div>Thanks for the feedback Justin.<br></div><div><br><=
/div><div>You are right, the most-common-values list is empty for my test c=
ase and so it is using n_distinct for the 'id IN()' scenario.</div><div>And=
 I can see that with the pg_class.reltuples and the pg_stats.histogram_boun=
ds values how the optimizer can conclude with my range query that only 1 in=
 5 entries in my range query are in each individual partition.</div><div><b=
r></div><div>However, I can also see that the pg_stats.n_distinct value for=
 tablea shows -1, as do all the individual child partitions. In my opinion =
it makes sense for the optimizer when using n_distinct on partitioned table=
s to use the n_distinct value of the parent table level when estimating row=
 counts rather than a sum of the partition level statistics. Or can someone=
 come up with a good reason to avoid this?&nbsp;</div><div><br></div><div>A=
 couple of examples of different data in a partitioned table:</div><div><br=
></div><div>- Unique identifier -&gt; if providing a single value in a quer=
y -&gt; using n_distinct from parent will estimate 1, using child tables wi=
ll be 1 * (number of partitions). Use of parent table would be correct.</di=
v><div>- Date of activity, with&nbsp; 1000 records per day -&gt; if providi=
ng a single day to the query -&gt; using n_distinct from parent would show =
1000 rows returned, using child tables will be 1000 * (number of partitions=
). Use of parent table n_distinct is correct.</div><div><br></div><div>Perh=
aps when querying on columns that are part of the partition logic you could=
 use the partition level stats, but I think the vast majority of the time, =
using the parent statistics would be much more reliable/accurate than summi=
ng across partitions.</div><div><br></div><div>In terms of the partition st=
rategy, I agree that it should be done with a view to helping performance i=
mprove. I will look into more detail at your suggestions, but in general it=
 is very hard to use effectively as there are competing priorities:</div><d=
iv><br></div><div>- I would like to not have to manage massive numbers of p=
artitions</div><div>- I would like to be able to archive data easily using =
date (a big plus point to the existing date partitioning strategy)</div><di=
v>- It is hard in most cases to come up with a partition strategy that allo=
ws for partition elimination e.g. consider a common 'transaction record' ta=
ble with a primary key, an account identifier, and a date -&gt; it is natur=
al to want to be able to query on any one of these, but as things stand it =
cannot be achieved performantly with partitioning.</div><div><br></div><div=
>Global index support feels like it has potential to resolve many of the is=
sues I have with partitioning (beyond the optimizer concern above). I assum=
e this has been discussed and rejected though by the community?</div><div><=
br></div><div>I've attached as a file the original test script.</div><div><=
br></div><div>Keith</div><div><br></div></span><br></div><div><br></div>
       =20
        </div><div id=3D"ydpfbf7e68ayahoo_quoted_0592464605" class=3D"ydpfb=
f7e68ayahoo_quoted">
            <div style=3D"font-family:'Helvetica Neue', Helvetica, Arial, s=
ans-serif;font-size:13px;color:#26282a;">
               =20
                <div>
                    On Friday, 8 February 2019, 13:05:04 GMT, Justin Pryzby=
 &lt;pryzby@telsasoft.com&gt; wrote:
                </div>
                <div><br></div>
                <div><br></div>
                <div>On Fri, Feb 08, 2019 at 11:13:51AM +0000, keith anders=
on wrote:<div class=3D"ydpfbf7e68ayqt7254353245" id=3D"ydpfbf7e68ayqtfd7988=
5"><br clear=3D"none">&gt; So to summarise the findings/questions from abov=
e:<br clear=3D"none">&gt; - It seems like the Postgres optimizer sometimes =
uses the partition level statistics, and sometimes the global table level s=
tatistics? Or is it using something else?- With partitioning tables with un=
ique identifier and retrieving explicitly on those identifiers, at present =
the optimizer will always understimate the selectivity and overestimate the=
 rows returned. This inaccuracy increases in proportion to the number of pa=
rtitions.- As a result, when joining to other tables, you are liable to hit=
ting sequential scans. This becomes more likely as you have more partitions=
 or if join to more partitioned tables (note I am aware I could try and tun=
e random_page_cost to try and prevent this).- To me in the examples queries=
 described above, it makes sense to use the partition statistics for the pa=
rtition level access strategy, but the global statistics when estimating th=
e actual rows returned by all the individual partition queries. Is there a =
reason not to do this? Or do others believe the optimizer is doing the righ=
t thing here?<br clear=3D"none">&gt; And then some general questions:<br cl=
ear=3D"none">&gt; - How do other people use partitioning but without a sign=
ificant performance disadvantage on reading the data? Is there something el=
se I should be doing here to achieve the same thing without the overhead? A=
t present my reads have increased optimization cost (as it needs to optimiz=
e access to each partition) and also execution cost (access the index on ev=
ery partition). Even without the optimizer issues described above, the cost=
 of reading simple data is extremely high relative to non-partitioned data =
(unless you use the partition key as a filter for each table to eliminate t=
hose partitions).- Is there any chance/plan to add global indexes to postgr=
es? If so would that impact significantly the cost of the partition drop e.=
g. to clean up the index.<br clear=3D"none">&gt; Thanks in advance for any =
feedback/support,</div><br clear=3D"none"><br clear=3D"none">An equality or=
 IN() query will use the pg_stats most-common-values list,<br clear=3D"none=
">whereas a range query will use the histogram.<br clear=3D"none"><br clear=
=3D"none">The tables probably doesn't have complete MCV list.&nbsp; By defa=
ult, that's limited<br clear=3D"none">to 100 entries.&nbsp; Since the maxim=
um allowed by ALTER..SET STATISTICS is 10k, I<br clear=3D"none">don't think=
 it'll help to change it (at least for your production use case).<br clear=
=3D"none">Each partition's rowcount appears to be estimated from its ndisti=
nct, and not<br clear=3D"none">from its content, so each is estimated as ha=
ving about the same rowcount.<br clear=3D"none"><br clear=3D"none">Your par=
titions are sharing a sequence for their ID column, which causes the<br cle=
ar=3D"none">DEFAULT IDs to be unique...but their global uniqueness isn't en=
forced nor<br clear=3D"none">guaranteed.<br clear=3D"none"><br clear=3D"non=
e">Note, in postgres11, it's possible to create an index on the parent tabl=
e.<br clear=3D"none">It's NOT a global index, but it can be unique if it in=
cludes the partition key.<br clear=3D"none">I don't know how closely your e=
xample describes your real use case, but I don't<br clear=3D"none">think th=
at helps with your example; it doesn't seems useful to partition on a<br cl=
ear=3D"none">serial column.<br clear=3D"none"><br clear=3D"none">You seem t=
o be adding unnecessary CHECK constraints that duplicate the<br clear=3D"no=
ne">partition bounds.&nbsp; Note, it's still useful to include CHECK constr=
aints on key<br clear=3D"none">column if you're planning on DETACHing and r=
e-ATTACHing the partitions, in<br clear=3D"none">order to avoid seqscan to =
verify tuples don't violate specified bounds.<br clear=3D"none"><br clear=
=3D"none">You might need to rethink your partitioning scheme - you should c=
hoose one that<br clear=3D"none">causes performance to improve, and probabl=
y naturally includes the partition<br clear=3D"none">key in most queries.<b=
r clear=3D"none"><br clear=3D"none">Perhaps you'd use 2 levels of partition=
ing: a RANGE partition by date, which<br clear=3D"none">allows for archivin=
g, and a HASH partition by ID, which allows for partition<br clear=3D"none"=
>pruning.&nbsp; Note that it's also possible to partition on multiple keys,=
 like<br clear=3D"none">RANGE(id,date) - I don't think that's useful here, =
though.&nbsp; PG11 also allows a<br clear=3D"none">"default" partition.<br =
clear=3D"none"><br clear=3D"none">Or perhaps you could partition by RANGE(d=
ate) but add CHECK constraints on ID,<br clear=3D"none">after the table is =
fully populated, to optimize queries by allowing for<br clear=3D"none">part=
ition pruning.<br clear=3D"none"><br clear=3D"none">Or you could maybe chan=
ge the ID column to include the timestamp (like BIND<br clear=3D"none">zone=
sfiles YYYYMMDDNNNNNNNN).&nbsp; You'd set a bigint sequence on each partiti=
on's<br clear=3D"none">ID as default to the beginning of the month.&nbsp; A=
 bigint is enough to handle<br clear=3D"none">5*10^4 times your volume: 201=
90401000020111222.&nbsp; (I think this is trying to be<br clear=3D"none">un=
necessarily clever, unless there's some reason the other two ideas don't<br=
 clear=3D"none">work.)<br clear=3D"none"><br clear=3D"none">Justin<div clas=
s=3D"ydpfbf7e68ayqt7254353245" id=3D"ydpfbf7e68ayqtfd92861"><br clear=3D"no=
ne"></div></div>
            </div>
        </div></body></html>
------=_Part_2230542_1821760923.1549872350895--

------=_Part_2230543_275789688.1549872350935
Content-Type: application/sql
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="partitionExample.sql"
Content-ID: <7d74ded7-4117-9f99-876c-4b90f2f12e87@yahoo.com>

ZHJvcCB0YWJsZSBpZiBleGlzdHMgdGFibGVhIGNhc2NhZGU7CmRyb3AgdGFibGUgaWYgZXhpc3Rz
IHRhYmxlYiBjYXNjYWRlOwoKQ1JFQVRFIFRBQkxFIHRhYmxlYSAoCiAgICBpZCAgICAgICAgICAg
IHNlcmlhbCwKICAgIHJlZmVyZW5jZSAgICAgaW50IG5vdCBudWxsLAogICAgY3JlYXRlZCAgICAg
ICBkYXRlIG5vdCBudWxsCikgIFBBUlRJVElPTiBCWSBSQU5HRSAoY3JlYXRlZCk7CgoKQ1JFQVRF
IFRBQkxFIHRhYmxlYV9wYXJ0MSBQQVJUSVRJT04gT0YgdGFibGVhCiAgICBGT1IgVkFMVUVTIEZS
T00gKCcyMDE4LTAxLTAxJykgVE8gKCcyMDE4LTAxLTAyJyk7CkNSRUFURSBUQUJMRSB0YWJsZWFf
cGFydDIgUEFSVElUSU9OIE9GIHRhYmxlYQogICAgRk9SIFZBTFVFUyBGUk9NICgnMjAxOC0wMS0w
MicpIFRPICgnMjAxOC0wMS0wMycpOwpDUkVBVEUgVEFCTEUgdGFibGVhX3BhcnQzIFBBUlRJVElP
TiBPRiB0YWJsZWEKICAgIEZPUiBWQUxVRVMgRlJPTSAoJzIwMTgtMDEtMDMnKSBUTyAoJzIwMTgt
MDEtMDQnKTsKQ1JFQVRFIFRBQkxFIHRhYmxlYV9wYXJ0NCBQQVJUSVRJT04gT0YgdGFibGVhCiAg
ICBGT1IgVkFMVUVTIEZST00gKCcyMDE4LTAxLTA0JykgVE8gKCcyMDE4LTAxLTA1Jyk7CkNSRUFU
RSBUQUJMRSB0YWJsZWFfcGFydDUgUEFSVElUSU9OIE9GIHRhYmxlYQogICAgRk9SIFZBTFVFUyBG
Uk9NICgnMjAxOC0wMS0wNScpIFRPICgnMjAxOC0wMS0wNicpOwoKQ1JFQVRFIElOREVYIHRhYmxl
YV9pZF8xIE9OIHRhYmxlYV9wYXJ0MSAoaWQpOwpDUkVBVEUgSU5ERVggdGFibGVhX2lkXzIgT04g
dGFibGVhX3BhcnQyIChpZCk7CkNSRUFURSBJTkRFWCB0YWJsZWFfaWRfMyBPTiB0YWJsZWFfcGFy
dDMgKGlkKTsKQ1JFQVRFIElOREVYIHRhYmxlYV9pZF80IE9OIHRhYmxlYV9wYXJ0NCAoaWQpOwpD
UkVBVEUgSU5ERVggdGFibGVhX2lkXzUgT04gdGFibGVhX3BhcnQ1IChpZCk7CkNSRUFURSBJTkRF
WCB0YWJsZWFfcmVmZXJlbmNlXzEgT04gdGFibGVhX3BhcnQxIChyZWZlcmVuY2UpOwpDUkVBVEUg
SU5ERVggdGFibGVhX3JlZmVyZW5jZV8yIE9OIHRhYmxlYV9wYXJ0MiAocmVmZXJlbmNlKTsKQ1JF
QVRFIElOREVYIHRhYmxlYV9yZWZlcmVuY2VfMyBPTiB0YWJsZWFfcGFydDMgKHJlZmVyZW5jZSk7
CkNSRUFURSBJTkRFWCB0YWJsZWFfcmVmZXJlbmNlXzQgT04gdGFibGVhX3BhcnQ0IChyZWZlcmVu
Y2UpOwpDUkVBVEUgSU5ERVggdGFibGVhX3JlZmVyZW5jZV81IE9OIHRhYmxlYV9wYXJ0NSAocmVm
ZXJlbmNlKTsKQ1JFQVRFIElOREVYIHRhYmxlYV9jcmVhdGVkXzEgT04gdGFibGVhX3BhcnQxIChj
cmVhdGVkKTsKQ1JFQVRFIElOREVYIHRhYmxlYV9jcmVhdGVkXzIgT04gdGFibGVhX3BhcnQyIChj
cmVhdGVkKTsKQ1JFQVRFIElOREVYIHRhYmxlYV9jcmVhdGVkXzMgT04gdGFibGVhX3BhcnQzIChj
cmVhdGVkKTsKQ1JFQVRFIElOREVYIHRhYmxlYV9jcmVhdGVkXzQgT04gdGFibGVhX3BhcnQ0IChj
cmVhdGVkKTsKQ1JFQVRFIElOREVYIHRhYmxlYV9jcmVhdGVkXzUgT04gdGFibGVhX3BhcnQ1IChj
cmVhdGVkKTsKYWx0ZXIgdGFibGUgdGFibGVhX3BhcnQxIGFkZCBDSEVDSyAoIGNyZWF0ZWQgPj0g
REFURSAnMjAxOC0wMS0wMScgQU5EIGNyZWF0ZWQgPCBEQVRFICcyMDE4LTAxLTAyJyk7CmFsdGVy
IHRhYmxlIHRhYmxlYV9wYXJ0MiBhZGQgQ0hFQ0sgKCBjcmVhdGVkID49IERBVEUgJzIwMTgtMDEt
MDInIEFORCBjcmVhdGVkIDwgREFURSAnMjAxOC0wMS0wMycpOwphbHRlciB0YWJsZSB0YWJsZWFf
cGFydDMgYWRkIENIRUNLICggY3JlYXRlZCA+PSBEQVRFICcyMDE4LTAxLTAzJyBBTkQgY3JlYXRl
ZCA8IERBVEUgJzIwMTgtMDEtMDQnKTsKYWx0ZXIgdGFibGUgdGFibGVhX3BhcnQ0IGFkZCBDSEVD
SyAoIGNyZWF0ZWQgPj0gREFURSAnMjAxOC0wMS0wNCcgQU5EIGNyZWF0ZWQgPCBEQVRFICcyMDE4
LTAxLTA1Jyk7CmFsdGVyIHRhYmxlIHRhYmxlYV9wYXJ0NSBhZGQgQ0hFQ0sgKCBjcmVhdGVkID49
IERBVEUgJzIwMTgtMDEtMDUnIEFORCBjcmVhdGVkIDwgREFURSAnMjAxOC0wMS0wNicpOwoKY3Jl
YXRlIG9yIHJlcGxhY2UgZnVuY3Rpb24gcG9wdWxhdGVfdGFibGVhKCkKIFJFVFVSTlMgaW50ZWdl
ciBBUwokQk9EWSQgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgCkRFQ0xBUkUgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIAogIGkgaW50ZWdlcjsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogIHZf
Y3JlYXRlZCBkYXRlOwpCRUdJTiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAKICAgIGkgOj0gMDsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAK
ICAgIFdISUxFIChpIDwgNTAwMDApCiAgICBsb29wCiAgICAgICAgaSA6PSBpICsgMTsgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgSUYgKG1vZChpLDUpID0g
MSkgVEhFTgogICAgICAgICAgICB2X2NyZWF0ZWQgPSAnMjAxOC0wMS0wMSc7CiAgICAgICAgRUxT
SUYgKG1vZChpLDUpID0gMikgVEhFTgogICAgICAgICAgICB2X2NyZWF0ZWQgPSAnMjAxOC0wMS0w
Mic7CiAgICAgICAgIEVMU0lGIChtb2QoaSw1KSA9IDMpIFRIRU4KICAgICAgICAgICAgdl9jcmVh
dGVkID0gJzIwMTgtMDEtMDMnOwogICAgICAgIEVMU0lGIChtb2QoaSw1KSA9IDQpIFRIRU4KICAg
ICAgICAgICAgdl9jcmVhdGVkID0gJzIwMTgtMDEtMDQnOwogICAgICAgIEVMU0lGIChtb2QoaSw1
KSA9IDApIFRIRU4KICAgICAgICAgICAgdl9jcmVhdGVkID0gJzIwMTgtMDEtMDUnOwogICAgICAg
IEVORCBJRjsgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgaW5zZXJ0IGludG8g
dGFibGVhIHZhbHVlcyAoaSwgaSwgdl9jcmVhdGVkKTsKCiAgICBlbmQgbG9vcDsgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIAogICAgICAgICAgIAogICAgUkVUVVJOIGk7CkVORDsKJEJPRFkkCkxB
TkdVQUdFIHBscGdzcWwgVk9MQVRJTEUKICAgIENPU1QgMTAwOwoKQ1JFQVRFIFRBQkxFIHRhYmxl
YigKICAgIGlkICAgICAgICAgc2VyaWFsLAogICAgcmVmZXJlbmNlICBpbnQgbm90IG51bGwsCiAg
ICBjcmVhdGVkICAgIGRhdGUgbm90IG51bGwKKSBQQVJUSVRJT04gQlkgUkFOR0UgKGNyZWF0ZWQp
OwoKCkNSRUFURSBUQUJMRSB0YWJsZWJfcGFydDEgUEFSVElUSU9OIE9GIHRhYmxlYgogICAgRk9S
IFZBTFVFUyBGUk9NICgnMjAxOC0wMS0wMScpIFRPICgnMjAxOC0wMS0wMicpOwpDUkVBVEUgVEFC
TEUgdGFibGViX3BhcnQyIFBBUlRJVElPTiBPRiB0YWJsZWIKICAgIEZPUiBWQUxVRVMgRlJPTSAo
JzIwMTgtMDEtMDInKSBUTyAoJzIwMTgtMDEtMDMnKTsKQ1JFQVRFIFRBQkxFIHRhYmxlYl9wYXJ0
MyBQQVJUSVRJT04gT0YgdGFibGViCiAgICBGT1IgVkFMVUVTIEZST00gKCcyMDE4LTAxLTAzJykg
VE8gKCcyMDE4LTAxLTA0Jyk7CkNSRUFURSBUQUJMRSB0YWJsZWJfcGFydDQgUEFSVElUSU9OIE9G
IHRhYmxlYgogICAgRk9SIFZBTFVFUyBGUk9NICgnMjAxOC0wMS0wNCcpIFRPICgnMjAxOC0wMS0w
NScpOwpDUkVBVEUgVEFCTEUgdGFibGViX3BhcnQ1IFBBUlRJVElPTiBPRiB0YWJsZWIKICAgIEZP
UiBWQUxVRVMgRlJPTSAoJzIwMTgtMDEtMDUnKSBUTyAoJzIwMTgtMDEtMDYnKTsKCgpDUkVBVEUg
SU5ERVggdGFibGViX2lkXzEgT04gdGFibGViX3BhcnQxIChpZCk7CkNSRUFURSBJTkRFWCB0YWJs
ZWJfaWRfMiBPTiB0YWJsZWJfcGFydDIgKGlkKTsKQ1JFQVRFIElOREVYIHRhYmxlYl9pZF8zIE9O
IHRhYmxlYl9wYXJ0MyAoaWQpOwpDUkVBVEUgSU5ERVggdGFibGViX2lkXzQgT04gdGFibGViX3Bh
cnQ0IChpZCk7CkNSRUFURSBJTkRFWCB0YWJsZWJfaWRfNSBPTiB0YWJsZWJfcGFydDUgKGlkKTsK
Q1JFQVRFIElOREVYIHRhYmxlYl9yZWZlcmVuY2VfMSBPTiB0YWJsZWJfcGFydDEgKHJlZmVyZW5j
ZSk7CkNSRUFURSBJTkRFWCB0YWJsZWJfcmVmZXJlbmNlXzIgT04gdGFibGViX3BhcnQyIChyZWZl
cmVuY2UpOwpDUkVBVEUgSU5ERVggdGFibGViX3JlZmVyZW5jZV8zIE9OIHRhYmxlYl9wYXJ0MyAo
cmVmZXJlbmNlKTsKQ1JFQVRFIElOREVYIHRhYmxlYl9yZWZlcmVuY2VfNCBPTiB0YWJsZWJfcGFy
dDQgKHJlZmVyZW5jZSk7CkNSRUFURSBJTkRFWCB0YWJsZWJfcmVmZXJlbmNlXzUgT04gdGFibGVi
X3BhcnQ1IChyZWZlcmVuY2UpOwpDUkVBVEUgSU5ERVggdGFibGViX2NyZWF0ZWRfMSBPTiB0YWJs
ZWJfcGFydDEgKGNyZWF0ZWQpOwpDUkVBVEUgSU5ERVggdGFibGViX2NyZWF0ZWRfMiBPTiB0YWJs
ZWJfcGFydDIgKGNyZWF0ZWQpOwpDUkVBVEUgSU5ERVggdGFibGViX2NyZWF0ZWRfMyBPTiB0YWJs
ZWJfcGFydDMgKGNyZWF0ZWQpOwpDUkVBVEUgSU5ERVggdGFibGViX2NyZWF0ZWRfNCBPTiB0YWJs
ZWJfcGFydDQgKGNyZWF0ZWQpOwpDUkVBVEUgSU5ERVggdGFibGViX2NyZWF0ZWRfNSBPTiB0YWJs
ZWJfcGFydDUgKGNyZWF0ZWQpOwphbHRlciB0YWJsZSB0YWJsZWJfcGFydDEgYWRkIENIRUNLICgg
Y3JlYXRlZCA+PSBEQVRFICcyMDE4LTAxLTAxJyBBTkQgY3JlYXRlZCA8IERBVEUgJzIwMTgtMDEt
MDInKTsKYWx0ZXIgdGFibGUgdGFibGViX3BhcnQyIGFkZCBDSEVDSyAoIGNyZWF0ZWQgPj0gREFU
RSAnMjAxOC0wMS0wMicgQU5EIGNyZWF0ZWQgPCBEQVRFICcyMDE4LTAxLTAzJyk7CmFsdGVyIHRh
YmxlIHRhYmxlYl9wYXJ0MyBhZGQgQ0hFQ0sgKCBjcmVhdGVkID49IERBVEUgJzIwMTgtMDEtMDMn
IEFORCBjcmVhdGVkIDwgREFURSAnMjAxOC0wMS0wNCcpOwphbHRlciB0YWJsZSB0YWJsZWJfcGFy
dDQgYWRkIENIRUNLICggY3JlYXRlZCA+PSBEQVRFICcyMDE4LTAxLTA0JyBBTkQgY3JlYXRlZCA8
IERBVEUgJzIwMTgtMDEtMDUnKTsKYWx0ZXIgdGFibGUgdGFibGViX3BhcnQ1IGFkZCBDSEVDSyAo
IGNyZWF0ZWQgPj0gREFURSAnMjAxOC0wMS0wNScgQU5EIGNyZWF0ZWQgPCBEQVRFICcyMDE4LTAx
LTA2Jyk7CgoKY3JlYXRlIG9yIHJlcGxhY2UgZnVuY3Rpb24gcG9wdWxhdGVfdGFibGViKCkKIFJF
VFVSTlMgaW50ZWdlciBBUwokQk9EWSQKREVDTEFSRQogIGkgaW50ZWdlcjsKICB2X2NyZWF0ZWQg
ZGF0ZTsKQkVHSU4KICAgIGkgOj0gMDsKICAgIFdISUxFIChpIDwgNTAwMDApCiAgICBsb29wCiAg
ICAgICAgaSA6PSBpICsgMTsKICAgICAgICBJRiAobW9kKGksNSkgPSAwKSBUSEVOCiAgICAgICAg
ICAgIHZfY3JlYXRlZCA9ICcyMDE4LTAxLTAxJzsKICAgICAgICBFTFNJRiAobW9kKGksNSkgPSAx
KSBUSEVOCiAgICAgICAgICAgIHZfY3JlYXRlZCA9ICcyMDE4LTAxLTAyJzsKICAgICAgICAgRUxT
SUYgKG1vZChpLDUpID0gMikgVEhFTgogICAgICAgICAgICB2X2NyZWF0ZWQgPSAnMjAxOC0wMS0w
Myc7CiAgICAgICAgRUxTSUYgKG1vZChpLDUpID0gMykgVEhFTgogICAgICAgICAgICB2X2NyZWF0
ZWQgPSAnMjAxOC0wMS0wNCc7CiAgICAgICAgRUxTSUYgKG1vZChpLDUpID0gNCkgVEhFTgogICAg
ICAgICAgICB2X2NyZWF0ZWQgPSAnMjAxOC0wMS0wNSc7ICAgIAogICAgICAgIEVORCBJRjsKICAg
ICAgICBpbnNlcnQgaW50byB0YWJsZWIgdmFsdWVzIChpLCBpLCB2X2NyZWF0ZWQpOwogICAgZW5k
IGxvb3A7CiAgICBSRVRVUk4gaTsKRU5EOwokQk9EWSQKTEFOR1VBR0UgcGxwZ3NxbCBWT0xBVElM
RQogICAgQ09TVCAxMDA7CgpzZWxlY3QgcG9wdWxhdGVfdGFibGVhKCk7CnNlbGVjdCBwb3B1bGF0
ZV90YWJsZWIoKTsKdmFjdXVtIGFuYWx5emU7

------=_Part_2230543_275789688.1549872350935--