Content-Type: multipart/mixed; boundary="------------Uv0HWsTjG1eJvqJ2SB01oE2e"
Message-ID: <6dce55f4-e8f6-8e8e-b2af-b45ccff13598@enterprisedb.com>
Date: Sat, 24 Jun 2023 13:23:23 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.12.0
Subject: Re: Problems with estimating OR conditions, IS NULL on LEFT JOINs
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
References: <52b55b53-420a-722c-adbd-706922fc059b@enterprisedb.com>
 <104950.1687565314@sss.pgh.pa.us>
Content-Language: en-US
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
In-Reply-To: <104950.1687565314@sss.pgh.pa.us>
Archived-At: <https://www.postgresql.org/message-id/6dce55f4-e8f6-8e8e-b2af-b45ccff13598%40enterprisedb.com>
Precedence: bulk

This is a multi-part message in MIME format.
--------------Uv0HWsTjG1eJvqJ2SB01oE2e
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 6/24/23 02:08, Tom Lane wrote:
> Tomas Vondra <tomas.vondra@enterprisedb.com> writes:
>> The problem is that the selectivity for "IS NULL" is estimated using the
>> table-level statistics. But the LEFT JOIN entirely breaks the idea that
>> the null_frac has anything to do with NULLs in the join result.
> 
> Right.
> 
>> I wonder how to improve this, say by adjusting the IS NULL selectivity
>> when we know to operate on the outer side of the join. We're able to
>> do this for antijoins, so maybe we could do that here, somehow?
> 
> This mess is part of the long-term plan around the work I've been doing
> on outer-join-aware Vars.  We now have infrastructure that can let
> the estimator routines see "oh, this Var isn't directly from a scan
> of its table, it's been passed through a potentially-nulling outer
> join --- and I can see which one".  I don't have more than vague ideas
> about what happens next, but that is clearly an essential step on the
> road to doing better.
> 

I was wondering if that work on outer-join-aware Vars could help with
this, but I wasn't following it very closely. I agree the ability to
check if the Var could be NULL due to an outer join seems useful, as it
says whether applying raw attribute statistics makes sense or not.

I was thinking about what to do for the case when that's not possible,
i.e. when the Var refers to nullable side of the join. Knowing that this
is happening is clearly not enough - we need to know how many new NULLs
are "injected" into the join result, and "communicate" that to the
estimation routines.

Attached is a very ugly experimental patch doing that, and with it the
estimate changes to this:

                                 QUERY PLAN
  ----------------------------------------------------------------------
   Hash Left Join  (cost=3.25..18179.88 rows=999900 width=16)
                   (actual time=0.528..596.151 rows=999900 loops=1)
     Hash Cond: (large.id = small.id)
     Filter: ((small.id IS NULL) OR
              (large.a = ANY ('{1000,2000,3000,4000,5000}'::integer[])))
     Rows Removed by Filter: 100
     ->  Seq Scan on large  (cost=0.00..14425.00 rows=1000000 width=8)
                     (actual time=0.069..176.138 rows=1000000 loops=1)
     ->  Hash  (cost=2.00..2.00 rows=100 width=8)
               (actual time=0.371..0.373 rows=100 loops=1)
           Buckets: 1024  Batches: 1  Memory Usage: 12kB
           ->  Seq Scan on small  (cost=0.00..2.00 rows=100 width=8)
                         (actual time=0.032..0.146 rows=100 loops=1)
   Planning Time: 3.845 ms
   Execution Time: 712.405 ms
  (10 rows)

Seems nice, but. The patch is pretty ugly, I don't claim it works for
other queries or that this is exactly what we should do. It calculates
"unmatched frequency" next to eqjoinsel_inner, stashes that info into
sjinfo and the estimator (nulltestsel) then uses that to adjust the
nullfrac it gets from the statistics.

The good thing is this helps even for IS NULL checks on non-join-key
columns (where we don't switch to an antijoin), but there's a couple
things that I dislike ...

1) It's not restricted to outer joins or anything like that (this is
mostly just my laziness / interest in one particular query, but also
something the outer-join-aware patch might help with).

2) We probably don't want to pass this kind of information through
sjinfo. That was the simplest thing for an experimental patch, but I
suspect it's not the only piece of information we may need to pass to
the lower levels of estimation code.

3) I kinda doubt we actually want to move this responsibility (to
consider fraction of unmatched rows) to the low-level estimation
routines (e.g. nulltestsel and various others). AFAICS this just
"introduces NULLs" into the relation, so maybe we could "adjust" the
attribute statistics (in examine_variable?) by inflating null_frac and
modifying the other frequencies in MCV/histogram.

4) But I'm not sure we actually want to do that in these low-level
selectivity functions. The outer join essentially produces output with
two subsets - one with matches on the outer side, one without them. But
the side without matches has NULLs in all columns. In a way, we know
exactly how are these columns correlated - if we do the usual estimation
(even with the null_frac adjusted), we just throw this information away.
And when there's a lot of rows without a match, that seems bad.

So maybe we should split the join estimate into two parts, one for each
subset of the join result. One for the rows with a match (and then we
can just do what we do now, with the attribute stats we already have).
And one for the "unmatched part" where we know the values on the outer
side are NULL (and then we can easily "fake" stats with null_frac=1.0).


I really hope what I just wrote makes at least a little bit of sense.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--------------Uv0HWsTjG1eJvqJ2SB01oE2e
Content-Type: text/x-patch; charset=UTF-8; name="left-join-estimates.patch"
Content-Disposition: attachment; filename="left-join-estimates.patch"
Content-Transfer-Encoding: base64

ZGlmZiAtLWdpdCBhL3NyYy9iYWNrZW5kL3V0aWxzL2FkdC9zZWxmdW5jcy5jIGIvc3JjL2Jh
Y2tlbmQvdXRpbHMvYWR0L3NlbGZ1bmNzLmMKaW5kZXggYzRmY2QwMDc2ZS4uYTAzNWUyMTNh
MCAxMDA2NDQKLS0tIGEvc3JjL2JhY2tlbmQvdXRpbHMvYWR0L3NlbGZ1bmNzLmMKKysrIGIv
c3JjL2JhY2tlbmQvdXRpbHMvYWR0L3NlbGZ1bmNzLmMKQEAgLTE1NCw2ICsxNTQsMTMgQEAg
c3RhdGljIGRvdWJsZSBlcWpvaW5zZWxfaW5uZXIoT2lkIG9wZnVuY29pZCwgT2lkIGNvbGxh
dGlvbiwKIAkJCQkJCQkgIEF0dFN0YXRzU2xvdCAqc3Nsb3QxLCBBdHRTdGF0c1Nsb3QgKnNz
bG90MiwKIAkJCQkJCQkgIEZvcm1fcGdfc3RhdGlzdGljIHN0YXRzMSwgRm9ybV9wZ19zdGF0
aXN0aWMgc3RhdHMyLAogCQkJCQkJCSAgYm9vbCBoYXZlX21jdnMxLCBib29sIGhhdmVfbWN2
czIpOworc3RhdGljIGRvdWJsZSBlcWpvaW5zZWxfdW5tYXRjaF9sZWZ0KE9pZCBvcGZ1bmNv
aWQsIE9pZCBjb2xsYXRpb24sCisJCQkJCQkJCQkgVmFyaWFibGVTdGF0RGF0YSAqdmFyZGF0
YTEsIFZhcmlhYmxlU3RhdERhdGEgKnZhcmRhdGEyLAorCQkJCQkJCQkJIGRvdWJsZSBuZDEs
IGRvdWJsZSBuZDIsCisJCQkJCQkJCQkgYm9vbCBpc2RlZmF1bHQxLCBib29sIGlzZGVmYXVs
dDIsCisJCQkJCQkJCQkgQXR0U3RhdHNTbG90ICpzc2xvdDEsIEF0dFN0YXRzU2xvdCAqc3Ns
b3QyLAorCQkJCQkJCQkJIEZvcm1fcGdfc3RhdGlzdGljIHN0YXRzMSwgRm9ybV9wZ19zdGF0
aXN0aWMgc3RhdHMyLAorCQkJCQkJCQkJIGJvb2wgaGF2ZV9tY3ZzMSwgYm9vbCBoYXZlX21j
dnMyKTsKIHN0YXRpYyBkb3VibGUgZXFqb2luc2VsX3NlbWkoT2lkIG9wZnVuY29pZCwgT2lk
IGNvbGxhdGlvbiwKIAkJCQkJCQkgVmFyaWFibGVTdGF0RGF0YSAqdmFyZGF0YTEsIFZhcmlh
YmxlU3RhdERhdGEgKnZhcmRhdGEyLAogCQkJCQkJCSBkb3VibGUgbmQxLCBkb3VibGUgbmQy
LApAQCAtMTcxMCw2ICsxNzE3LDkgQEAgbnVsbHRlc3RzZWwoUGxhbm5lckluZm8gKnJvb3Qs
IE51bGxUZXN0VHlwZSBudWxsdGVzdHR5cGUsIE5vZGUgKmFyZywKIAkJc3RhdHMgPSAoRm9y
bV9wZ19zdGF0aXN0aWMpIEdFVFNUUlVDVCh2YXJkYXRhLnN0YXRzVHVwbGUpOwogCQlmcmVx
X251bGwgPSBzdGF0cy0+c3RhbnVsbGZyYWM7CiAKKwkJaWYgKHNqaW5mbykKKwkJCWZyZXFf
bnVsbCA9IGZyZXFfbnVsbCArIHNqaW5mby0+dW5tYXRjaGVkX2ZyYWMgLSBmcmVxX251bGwg
KiBzamluZm8tPnVubWF0Y2hlZF9mcmFjOworCiAJCXN3aXRjaCAobnVsbHRlc3R0eXBlKQog
CQl7CiAJCQljYXNlIElTX05VTEw6CkBAIC0yMjQ5LDYgKzIyNTksNyBAQCBlcWpvaW5zZWwo
UEdfRlVOQ1RJT05fQVJHUykKIAlPaWQJCQljb2xsYXRpb24gPSBQR19HRVRfQ09MTEFUSU9O
KCk7CiAJZG91YmxlCQlzZWxlYzsKIAlkb3VibGUJCXNlbGVjX2lubmVyOworCWRvdWJsZQkJ
dW5tYXRjaGVkX2ZyYWM7CiAJVmFyaWFibGVTdGF0RGF0YSB2YXJkYXRhMTsKIAlWYXJpYWJs
ZVN0YXREYXRhIHZhcmRhdGEyOwogCWRvdWJsZQkJbmQxOwpAQCAtMjMyMSw2ICsyMzMyLDI2
IEBAIGVxam9pbnNlbChQR19GVU5DVElPTl9BUkdTKQogCQkJCQkJCQkgIHN0YXRzMSwgc3Rh
dHMyLAogCQkJCQkJCQkgIGhhdmVfbWN2czEsIGhhdmVfbWN2czIpOwogCisJLyoKKwkgKiBj
YWxjdWxhdGUgZnJhY3Rpb24gb2YgcmlnaHQgd2l0aG91dCBvZiBtYXRjaGluZyByb3cgb24g
bGVmdAorCSAqCisJICogRklYTUUgU2hvdWxkIGJlIHJlc3RyaWN0ZWQgdG8gSk9JTl9MRUZU
LCB3ZSBzaG91bGQgaGF2ZSBzaW1pbGFyIGxvZ2ljCisJICogZm9yIEpPSU5fRlVMTC4KKwkg
KgorCSAqIFhYWCBQcm9iYWJseSBzaG91bGQgY2FsY3VsYXRlIHVubWF0Y2hlZCBhcyBmcmFj
dGlvbiBvZiB0aGUgam9pbiByZXN1bHQsCisJICogbm90IG9mIHRoZSByZWxhdGlvbiBvbiB0
aGUgcmlnaHQgKGJlY2F1c2UgdGhlIG1hdGNoZWQgcGFydCBjYW4gaGF2ZSBtb3JlCisJICog
bWF0Y2hlcyBwZXIgcm93IGFuZCB0aHVzIGdyb3cpLiBOb3Qgc3VyZS4gRGVwZW5kcyBvbiBo
b3cgaXQncyB1c2VkIGxhdGVyLgorCSAqLworCXVubWF0Y2hlZF9mcmFjID0gZXFqb2luc2Vs
X3VubWF0Y2hfbGVmdChvcGZ1bmNvaWQsIGNvbGxhdGlvbiwKKwkJCQkJCQkJCQkJJnZhcmRh
dGExLCAmdmFyZGF0YTIsCisJCQkJCQkJCQkJCW5kMSwgbmQyLAorCQkJCQkJCQkJCQlpc2Rl
ZmF1bHQxLCBpc2RlZmF1bHQyLAorCQkJCQkJCQkJCQkmc3Nsb3QxLCAmc3Nsb3QyLAorCQkJ
CQkJCQkJCQlzdGF0czEsIHN0YXRzMiwKKwkJCQkJCQkJCQkJaGF2ZV9tY3ZzMSwgaGF2ZV9t
Y3ZzMik7CisKKwlzamluZm8tPnVubWF0Y2hlZF9mcmFjID0gdW5tYXRjaGVkX2ZyYWM7CisK
IAlzd2l0Y2ggKHNqaW5mby0+am9pbnR5cGUpCiAJewogCQljYXNlIEpPSU5fSU5ORVI6CkBA
IC0yNTkwLDYgKzI2MjEsMTE3IEBAIGVxam9pbnNlbF9pbm5lcihPaWQgb3BmdW5jb2lkLCBP
aWQgY29sbGF0aW9uLAogCXJldHVybiBzZWxlYzsKIH0KIAorLyoKKyAqIGVxam9pbnNlbF91
bm1hdGNoX2xlZnQKKyAqCisgKiBYWFggTW9zdGx5IGNvcHkgcGFzdGUgb2YgZXFqb2luc2Vs
X2lubmVyLCBidXQgY2FsY3VsYXRlcyB1bm1hdGNoZWQgZnJhY3Rpb24KKyAqIG9mIHRoZSBy
ZWxhdGlvbiBvbiB0aGUgbGVmdC4gU2VlIGVxam9pbnNlbF9pbm5lciBmb3IgbW9yZSBjb21t
ZW50cy4KKyAqCisgKiBYWFggTWF5YmUgd2UgY291bGQvc2hvdWxkIGludGVncmF0ZSB0aGlz
IGludG8gZXFqb2luc2VsX2lubmVyLCBzbyB0aGF0IHdlCisgKiBkb24ndCBoYXZlIHRvIHdh
bGsgdGhlIE1DViBsaXN0cyB0d2ljZT8KKyAqLworc3RhdGljIGRvdWJsZQorZXFqb2luc2Vs
X3VubWF0Y2hfbGVmdChPaWQgb3BmdW5jb2lkLCBPaWQgY29sbGF0aW9uLAorCQkJCQkgICBW
YXJpYWJsZVN0YXREYXRhICp2YXJkYXRhMSwgVmFyaWFibGVTdGF0RGF0YSAqdmFyZGF0YTIs
CisJCQkJCSAgIGRvdWJsZSBuZDEsIGRvdWJsZSBuZDIsCisJCQkJCSAgIGJvb2wgaXNkZWZh
dWx0MSwgYm9vbCBpc2RlZmF1bHQyLAorCQkJCQkgICBBdHRTdGF0c1Nsb3QgKnNzbG90MSwg
QXR0U3RhdHNTbG90ICpzc2xvdDIsCisJCQkJCSAgIEZvcm1fcGdfc3RhdGlzdGljIHN0YXRz
MSwgRm9ybV9wZ19zdGF0aXN0aWMgc3RhdHMyLAorCQkJCQkgICBib29sIGhhdmVfbWN2czEs
IGJvb2wgaGF2ZV9tY3ZzMikKK3sKKwlkb3VibGUJCXVubWF0Y2hmcmVxOworCisJaWYgKGhh
dmVfbWN2czEgJiYgaGF2ZV9tY3ZzMikKKwl7CisJCUxPQ0FMX0ZDSU5GTyhmY2luZm8sIDIp
OworCQlGbWdySW5mbwllcXByb2M7CisJCWJvb2wJICAgKmhhc21hdGNoMTsKKwkJYm9vbAkg
ICAqaGFzbWF0Y2gyOworCQlkb3VibGUJCW1hdGNocHJvZGZyZXEsCisJCQkJCW1hdGNoZnJl
cTEsCisJCQkJCXVubWF0Y2hmcmVxMTsKKwkJaW50CQkJaSwKKwkJCQkJbm1hdGNoZXM7CisK
KwkJZm1ncl9pbmZvKG9wZnVuY29pZCwgJmVxcHJvYyk7CisKKwkJLyoKKwkJICogU2F2ZSBh
IGZldyBjeWNsZXMgYnkgc2V0dGluZyB1cCB0aGUgZmNpbmZvIHN0cnVjdCBqdXN0IG9uY2Uu
IFVzaW5nCisJCSAqIEZ1bmN0aW9uQ2FsbEludm9rZSBkaXJlY3RseSBhbHNvIGF2b2lkcyBm
YWlsdXJlIGlmIHRoZSBlcXByb2MKKwkJICogcmV0dXJucyBOVUxMLCB0aG91Z2ggcmVhbGx5
IGVxdWFsaXR5IGZ1bmN0aW9ucyBzaG91bGQgbmV2ZXIgZG8KKwkJICogdGhhdC4KKwkJICov
CisJCUluaXRGdW5jdGlvbkNhbGxJbmZvRGF0YSgqZmNpbmZvLCAmZXFwcm9jLCAyLCBjb2xs
YXRpb24sCisJCQkJCQkJCSBOVUxMLCBOVUxMKTsKKwkJZmNpbmZvLT5hcmdzWzBdLmlzbnVs
bCA9IGZhbHNlOworCQlmY2luZm8tPmFyZ3NbMV0uaXNudWxsID0gZmFsc2U7CisKKwkJaGFz
bWF0Y2gxID0gKGJvb2wgKikgcGFsbG9jMChzc2xvdDEtPm52YWx1ZXMgKiBzaXplb2YoYm9v
bCkpOworCQloYXNtYXRjaDIgPSAoYm9vbCAqKSBwYWxsb2MwKHNzbG90Mi0+bnZhbHVlcyAq
IHNpemVvZihib29sKSk7CisKKwkJLyoKKwkJICogTm90ZSB3ZSBhc3N1bWUgdGhhdCBlYWNo
IE1DViB3aWxsIG1hdGNoIGF0IG1vc3Qgb25lIG1lbWJlciBvZiB0aGUKKwkJICogb3RoZXIg
TUNWIGxpc3QuICBJZiB0aGUgb3BlcmF0b3IgaXNuJ3QgcmVhbGx5IGVxdWFsaXR5LCB0aGVy
ZSBjb3VsZAorCQkgKiBiZSBtdWx0aXBsZSBtYXRjaGVzIC0tLSBidXQgd2UgZG9uJ3QgbG9v
ayBmb3IgdGhlbSwgYm90aCBmb3Igc3BlZWQKKwkJICogYW5kIGJlY2F1c2UgdGhlIG1hdGgg
d291bGRuJ3QgYWRkIHVwLi4uCisJCSAqLworCQltYXRjaHByb2RmcmVxID0gMC4wOworCQlu
bWF0Y2hlcyA9IDA7CisJCWZvciAoaSA9IDA7IGkgPCBzc2xvdDEtPm52YWx1ZXM7IGkrKykK
KwkJeworCQkJaW50CQkJajsKKworCQkJZmNpbmZvLT5hcmdzWzBdLnZhbHVlID0gc3Nsb3Qx
LT52YWx1ZXNbaV07CisKKwkJCWZvciAoaiA9IDA7IGogPCBzc2xvdDItPm52YWx1ZXM7IGor
KykKKwkJCXsKKwkJCQlEYXR1bQkJZnJlc3VsdDsKKworCQkJCWlmIChoYXNtYXRjaDJbal0p
CisJCQkJCWNvbnRpbnVlOworCQkJCWZjaW5mby0+YXJnc1sxXS52YWx1ZSA9IHNzbG90Mi0+
dmFsdWVzW2pdOworCQkJCWZjaW5mby0+aXNudWxsID0gZmFsc2U7CisJCQkJZnJlc3VsdCA9
IEZ1bmN0aW9uQ2FsbEludm9rZShmY2luZm8pOworCQkJCWlmICghZmNpbmZvLT5pc251bGwg
JiYgRGF0dW1HZXRCb29sKGZyZXN1bHQpKQorCQkJCXsKKwkJCQkJaGFzbWF0Y2gxW2ldID0g
aGFzbWF0Y2gyW2pdID0gdHJ1ZTsKKwkJCQkJbWF0Y2hwcm9kZnJlcSArPSBzc2xvdDEtPm51
bWJlcnNbaV0gKiBzc2xvdDItPm51bWJlcnNbal07CisJCQkJCW5tYXRjaGVzKys7CisJCQkJ
CWJyZWFrOworCQkJCX0KKwkJCX0KKwkJfQorCQlDTEFNUF9QUk9CQUJJTElUWShtYXRjaHBy
b2RmcmVxKTsKKwkJLyogU3VtIHVwIGZyZXF1ZW5jaWVzIG9mIG1hdGNoZWQgYW5kIHVubWF0
Y2hlZCBNQ1ZzICovCisJCW1hdGNoZnJlcTEgPSB1bm1hdGNoZnJlcTEgPSAwLjA7CisJCWZv
ciAoaSA9IDA7IGkgPCBzc2xvdDEtPm52YWx1ZXM7IGkrKykKKwkJeworCQkJaWYgKGhhc21h
dGNoMVtpXSkKKwkJCQltYXRjaGZyZXExICs9IHNzbG90MS0+bnVtYmVyc1tpXTsKKwkJCWVs
c2UKKwkJCQl1bm1hdGNoZnJlcTEgKz0gc3Nsb3QxLT5udW1iZXJzW2ldOworCQl9CisJCUNM
QU1QX1BST0JBQklMSVRZKG1hdGNoZnJlcTEpOworCQlDTEFNUF9QUk9CQUJJTElUWSh1bm1h
dGNoZnJlcTEpOworCisJCXVubWF0Y2hmcmVxID0gdW5tYXRjaGZyZXExOworCX0KKwllbHNl
CisJeworCQkvKgorCQkgKiBYWFggU2hvdWxkIHRoaXMgbG9vayBhdCBudWxsZnJhYyBvbiBl
aXRoZXIgc2lkZT8gUHJvYmFibHkgZGVwZW5kcyBvbgorCQkgKiBpZiB3ZSdyZSBjYWxjdWxh
dGluZyBmcmFjdGlvbiBvZiBOVUxMcyBvciBmcmFjdGlvbiBvZiB1bm1hdGNoZWQgcm93cy4K
KwkJICovCisJCS8vIHVubWF0Y2hmcmVxID0gKDEuMCAtIG51bGxmcmFjMSkgKiAoMS4wIC0g
bnVsbGZyYWMyKTsKKwkJaWYgKG5kMSA+IG5kMikKKwkJCXVubWF0Y2hmcmVxID0gKG5kMSAt
IG5kMikgKiAxLjAgLyBuZDE7CisJCWVsc2UKKwkJCXVubWF0Y2hmcmVxID0gMC4wOworCX0K
KworCXJldHVybiB1bm1hdGNoZnJlcTsKK30KKwogLyoKICAqIGVxam9pbnNlbF9zZW1pIC0t
LSBlcWpvaW5zZWwgZm9yIHNlbWkgam9pbgogICoKZGlmZiAtLWdpdCBhL3NyYy9pbmNsdWRl
L25vZGVzL3BhdGhub2Rlcy5oIGIvc3JjL2luY2x1ZGUvbm9kZXMvcGF0aG5vZGVzLmgKaW5k
ZXggYzE3YjUzZjdhZC4uNmJjNjNlNjQ4ZSAxMDA2NDQKLS0tIGEvc3JjL2luY2x1ZGUvbm9k
ZXMvcGF0aG5vZGVzLmgKKysrIGIvc3JjL2luY2x1ZGUvbm9kZXMvcGF0aG5vZGVzLmgKQEAg
LTI4NTMsNiArMjg1Myw5IEBAIHN0cnVjdCBTcGVjaWFsSm9pbkluZm8KIAlib29sCQlzZW1p
X2Nhbl9oYXNoOwkvKiB0cnVlIGlmIHNlbWlfb3BlcmF0b3JzIGFyZSBhbGwgaGFzaCAqLwog
CUxpc3QJICAgKnNlbWlfb3BlcmF0b3JzOyAvKiBPSURzIG9mIGVxdWFsaXR5IGpvaW4gb3Bl
cmF0b3JzICovCiAJTGlzdAkgICAqc2VtaV9yaHNfZXhwcnM7IC8qIHJpZ2h0aGFuZC1zaWRl
IGV4cHJlc3Npb25zIG9mIHRoZXNlIG9wcyAqLworCisJLyogRm9yIG91dGVyIGpvaW4sIGZy
YWN0aW9uIG9mIHJvd3Mgd2l0aG91dCBhIG1hdGNoLiAqLworCVNlbGVjdGl2aXR5CXVubWF0
Y2hlZF9mcmFjOwogfTsKIAogLyoK

--------------Uv0HWsTjG1eJvqJ2SB01oE2e--