MIME-Version: 1.0
References: <b3d80961-c2e5-38cc-6a32-61886cdf766d@gmail.com>
 <CAFiTN-v1yDvU=X+hwfJ+55=sbgDj=_kuvbduEG-F7=BjpWcnuw@mail.gmail.com> <efcb213c-c87c-41f0-85b7-7fa8079df807@enterprisedb.com>
In-Reply-To: <efcb213c-c87c-41f0-85b7-7fa8079df807@enterprisedb.com>
From: Donghang Lin <donghanglin@gmail.com>
Date: Sun, 24 Mar 2024 23:28:52 -0700
Message-ID: <CAA=D8a1ZGWTk0RDH-LyU==RsE-217T29eprS-NBhyj+eLHyRrQ@mail.gmail.com>
Subject: Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE
To: Tomas Vondra <tomas.vondra@enterprisedb.com>
Cc: Dilip Kumar <dilipbalaut@gmail.com>, David Geier <geidav.pg@gmail.com>, 
	PostgreSQL Developers <pgsql-hackers@lists.postgresql.org>, hlinnaka@iki.fi, 
	melanieplageman@gmail.com
Content-Type: multipart/mixed; boundary="000000000000c0f0c20614764bff"
Archived-At: <https://www.postgresql.org/message-id/CAA%3DD8a1ZGWTk0RDH-LyU%3D%3DRsE-217T29eprS-NBhyj%2BeLHyRrQ%40mail.gmail.com>
Precedence: bulk

--000000000000c0f0c20614764bff
Content-Type: multipart/alternative; boundary="000000000000c0f0c00614764bfd"

--000000000000c0f0c00614764bfd
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Tomas,
>
> On Sat, Feb 17, 2024 at 2:31=E2=80=AFPM Tomas Vondra <
tomas.vondra@enterprisedb.com> wrote:
> Hi David,
>
> Do you plan to work continue working on this patch? I did take a look,
> and on the whole it looks reasonable - it modifies the right places etc.
>
> I think there are two things that may need an improvement:
>
> 1) Storing variable-length data in ParallelBitmapHeapState
>
> I agree with Robert the snapshot_and_stats name is not great. I see
> Dmitry mentioned phs_snapshot_off as used by ParallelTableScanDescData -
> the reasons are somewhat different (phs_snapshot_off exists because we
> don't know which exact struct will be allocated), while here we simply
> need to allocate two variable-length pieces of memory. But it seems like
> it would work nicely for this. That is, we could try adding an offset
> for each of those pieces of memory:
>
>  - snapshot_off
>  - stats_off
>
> I don't like the GetSharedSnapshotData name very much, it seems very
> close to GetSnapshotData - quite confusing, I think.
>
> Dmitry also suggested we might add a separate piece of shared memory. I
> don't quite see how that would work for ParallelBitmapHeapState, but I
> doubt it'd be simpler than having two offsets. I don't think the extra
> complexity (paid by everyone) would be worth it just to make EXPLAIN
> ANALYZE work.

This issue is now gone after Heikki's fix.

> 2) Leader vs. worker counters
>
> It seems to me this does nothing to add the per-worker values from "Heap
> Blocks" into the leader, which means we get stuff like this:
>
>     Heap Blocks: exact=3D102 lossy=3D10995
>     Worker 0:  actual time=3D50.559..209.773 rows=3D215253 loops=3D1
>        Heap Blocks: exact=3D207 lossy=3D19354
>     Worker 1:  actual time=3D50.543..211.387 rows=3D162934 loops=3D1
>        Heap Blocks: exact=3D161 lossy=3D14636
>
> I think this is wrong / confusing, and inconsistent with what we do for
> other nodes. It's also inconsistent with how we deal e.g. with BUFFERS,
> where we *do* add the values to the leader:
>
>     Heap Blocks: exact=3D125 lossy=3D10789
>     Buffers: shared hit=3D11 read=3D45420
>     Worker 0:  actual time=3D51.419..221.904 rows=3D150437 loops=3D1
>       Heap Blocks: exact=3D136 lossy=3D13541
>       Buffers: shared hit=3D4 read=3D13541
>     Worker 1:  actual time=3D56.610..222.469 rows=3D229738 loops=3D1
>       Heap Blocks: exact=3D209 lossy=3D20655
>       Buffers: shared hit=3D4 read=3D20655
>
> Here it's not entirely obvious, because leader participates in the
> execution, but once we disable leader participation, it's clearer:
>
>     Buffers: shared hit=3D7 read=3D45421
>     Worker 0:  actual time=3D28.540..247.683 rows=3D309112 loops=3D1
>       Heap Blocks: exact=3D282 lossy=3D27806
>       Buffers: shared hit=3D4 read=3D28241
>     Worker 1:  actual time=3D24.290..251.993 rows=3D190815 loops=3D1
>       Heap Blocks: exact=3D188 lossy=3D17179
>       Buffers: shared hit=3D3 read=3D17180
>
> Not only is "Buffers" clearly a sum of per-worker stats, but the "Heap
> Blocks" simply disappeared because the leader does nothing and we don't
> print zeros.

Heap Blocks is specific to Bitmap Heap Scan. It seems that node specific
stats
do not aggregate workers' stats into leaders for some existing nodes. For
example,
Memorize node for Hits, Misses, etc

   ->  Nested Loop (actual rows=3D166667 loops=3D3)
         ->  Parallel Seq Scan on t (actual rows=3D33333 loops=3D3)
         ->  Memoize (actual rows=3D5 loops=3D100000)
               Cache Key: t.j
               Cache Mode: logical
               Hits: 32991  Misses: 5  Evictions: 0  Overflows: 0  Memory
Usage: 2kB
               Worker 0:  Hits: 33551  Misses: 5  Evictions: 0  Overflows:
0  Memory Usage: 2kB
               Worker 1:  Hits: 33443  Misses: 5  Evictions: 0  Overflows:
0  Memory Usage: 2kB
               ->  Index Scan using uj on u (actual rows=3D5 loops=3D15)
                     Index Cond: (j =3D t.j)

Sort, HashAggregate also do the same stuff.

> 3) I'm not sure dealing with various EXPLAIN flags may not be entirely
> correct. Consider this:
>
> EXPLAIN (ANALYZE):
>
>    ->  Parallel Bitmap Heap Scan on t  (...)
>          Recheck Cond: (a < 5000)
>          Rows Removed by Index Recheck: 246882
>          Worker 0:  Heap Blocks: exact=3D168 lossy=3D15648
>          Worker 1:  Heap Blocks: exact=3D302 lossy=3D29337
>
> EXPLAIN (ANALYZE, VERBOSE):
>
>    ->  Parallel Bitmap Heap Scan on public.t  (...)
>          Recheck Cond: (t.a < 5000)
>          Rows Removed by Index Recheck: 246882
>          Worker 0:  actual time=3D35.067..300.882 rows=3D282108 loops=3D1
>            Heap Blocks: exact=3D257 lossy=3D25358
>          Worker 1:  actual time=3D32.827..302.224 rows=3D217819 loops=3D1
>            Heap Blocks: exact=3D213 lossy=3D19627
>
> EXPLAIN (ANALYZE, BUFFERS):
>
>    ->  Parallel Bitmap Heap Scan on t  (...)
>          Recheck Cond: (a < 5000)
>          Rows Removed by Index Recheck: 246882
>          Buffers: shared hit=3D7 read=3D45421
>          Worker 0:  Heap Blocks: exact=3D236 lossy=3D21870
>          Worker 1:  Heap Blocks: exact=3D234 lossy=3D23115
>
> EXPLAIN (ANALYZE, VERBOSE, BUFFERS):
>
>    ->  Parallel Bitmap Heap Scan on public.t  (...)
>          Recheck Cond: (t.a < 5000)
>          Rows Removed by Index Recheck: 246882
>          Buffers: shared hit=3D7 read=3D45421
>          Worker 0:  actual time=3D28.265..260.381 rows=3D261264 loops=3D1
>            Heap Blocks: exact=3D260 lossy=3D23477
>            Buffers: shared hit=3D3 read=3D23478
>          Worker 1:  actual time=3D28.224..261.627 rows=3D238663 loops=3D1
>            Heap Blocks: exact=3D210 lossy=3D21508
>            Buffers: shared hit=3D4 read=3D21943
>
> Why should the per-worker buffer info be shown when combined with the
> VERBOSE flag, and not just with BUFFERS, when the patch shows the
> per-worker info always?
>

It seems that the general explain print framework requires verbose mode to
show per worker stats.
For example, how Buffers hits, JIT are printed. While in some specific
nodes which involves parallelism,
they always show worker blocks. This is why we see that some worker blocks
don't have buffers
stats in non verbose mode. There are several existing nodes have the same
issue as what this
patch does: memorize, sort, hashaggregate.

> 4) Now that I think about this, isn't the *main* problem really that we
> don't display the sum of the per-worker stats (which I think is wrong)?
> I mean, we already can get the worker details VERBOSEm right? So the
> only reason to display that by default seems to be that it the values in
> "Heap Blocks" are from the leader only.

If we print aggregate Heap Blocks in the 'leader' block and show worker
stats only in
verbose mode, does it look better? In this way, it matches better with how
the
general framework prints workers stats, but it differs from some existing
nodes which
also print worker stats. As mentioned above, they are memorize, sort, and
hashaggregate nodes.
By the way, is that also a problem for these nodes?

> BTW doesn't this also suggest some of the code added to explain.c may
> not be quite necessary? Wouldn't it be enough to just "extend" the
> existing code printing per-worker stats. (I haven't tried, so maybe I'm
> wrong and we need the new code.)

We need the new code as they are node specific stats and we do call the
worker print function
to lay out the explain plan. I think the problem is in which mode we should
show worker blocks. This
is discussed in the above section.

v3 failed on master, attached a rebased version.

Regards,
Donghang Lin
(ServiceNow)

--000000000000c0f0c00614764bfd
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><br></div><div>Hi Tomas, <br>&gt; <br>&gt; On Sat, Fe=
b 17, 2024 at 2:31=E2=80=AFPM Tomas Vondra &lt;<a href=3D"mailto:tomas.vond=
ra@enterprisedb.com">tomas.vondra@enterprisedb.com</a>&gt; wrote:<br>&gt; H=
i David,<br>&gt; <br>&gt; Do you plan to work continue working on this patc=
h? I did take a look,<br>&gt; and on the whole it looks reasonable - it mod=
ifies the right places etc.<br>&gt; <br>&gt; I think there are two things t=
hat may need an improvement:<br>&gt; <br>&gt; 1) Storing variable-length da=
ta in ParallelBitmapHeapState<br>&gt; <br>&gt; I agree with Robert the snap=
shot_and_stats name is not great. I see<br>&gt; Dmitry mentioned phs_snapsh=
ot_off as used by ParallelTableScanDescData -<br>&gt; the reasons are somew=
hat different (phs_snapshot_off exists because we<br>&gt; don&#39;t know wh=
ich exact struct will be allocated), while here we simply<br>&gt; need to a=
llocate two variable-length pieces of memory. But it seems like<br>&gt; it =
would work nicely for this. That is, we could try adding an offset<br>&gt; =
for each of those pieces of memory:<br>&gt; <br>&gt; =C2=A0- snapshot_off<b=
r>&gt; =C2=A0- stats_off<br>&gt; <br>&gt; I don&#39;t like the GetSharedSna=
pshotData name very much, it seems very<br>&gt; close to GetSnapshotData - =
quite confusing, I think.<br>&gt; <br>&gt; Dmitry also suggested we might a=
dd a separate piece of shared memory. I<br>&gt; don&#39;t quite see how tha=
t would work for ParallelBitmapHeapState, but I<br>&gt; doubt it&#39;d be s=
impler than having two offsets. I don&#39;t think the extra<br>&gt; complex=
ity (paid by everyone) would be worth it just to make EXPLAIN<br>&gt; ANALY=
ZE work.<br><br>This issue is now gone after Heikki&#39;s fix. =C2=A0<br><b=
r>&gt; 2) Leader vs. worker counters<br>&gt; <br>&gt; It seems to me this d=
oes nothing to add the per-worker values from &quot;Heap<br>&gt; Blocks&quo=
t; into the leader, which means we get stuff like this:<br>&gt; <br>&gt; =
=C2=A0 =C2=A0 Heap Blocks: exact=3D102 lossy=3D10995<br>&gt; =C2=A0 =C2=A0 =
Worker 0: =C2=A0actual time=3D50.559..209.773 rows=3D215253 loops=3D1<br>&g=
t; =C2=A0 =C2=A0 =C2=A0 =C2=A0Heap Blocks: exact=3D207 lossy=3D19354<br>&gt=
; =C2=A0 =C2=A0 Worker 1: =C2=A0actual time=3D50.543..211.387 rows=3D162934=
 loops=3D1<br>&gt; =C2=A0 =C2=A0 =C2=A0 =C2=A0Heap Blocks: exact=3D161 loss=
y=3D14636<br>&gt; <br>&gt; I think this is wrong / confusing, and inconsist=
ent with what we do for<br>&gt; other nodes. It&#39;s also inconsistent wit=
h how we deal e.g. with BUFFERS,<br>&gt; where we *do* add the values to th=
e leader:<br>&gt; <br>&gt; =C2=A0 =C2=A0 Heap Blocks: exact=3D125 lossy=3D1=
0789<br>&gt; =C2=A0 =C2=A0 Buffers: shared hit=3D11 read=3D45420<br>&gt; =
=C2=A0 =C2=A0 Worker 0: =C2=A0actual time=3D51.419..221.904 rows=3D150437 l=
oops=3D1<br>&gt; =C2=A0 =C2=A0 =C2=A0 Heap Blocks: exact=3D136 lossy=3D1354=
1<br>&gt; =C2=A0 =C2=A0 =C2=A0 Buffers: shared hit=3D4 read=3D13541<br>&gt;=
 =C2=A0 =C2=A0 Worker 1: =C2=A0actual time=3D56.610..222.469 rows=3D229738 =
loops=3D1<br>&gt; =C2=A0 =C2=A0 =C2=A0 Heap Blocks: exact=3D209 lossy=3D206=
55<br>&gt; =C2=A0 =C2=A0 =C2=A0 Buffers: shared hit=3D4 read=3D20655<br>&gt=
; <br>&gt; Here it&#39;s not entirely obvious, because leader participates =
in the<br>&gt; execution, but once we disable leader participation, it&#39;=
s clearer:<br>&gt; <br>&gt; =C2=A0 =C2=A0 Buffers: shared hit=3D7 read=3D45=
421<br>&gt; =C2=A0 =C2=A0 Worker 0: =C2=A0actual time=3D28.540..247.683 row=
s=3D309112 loops=3D1<br>&gt; =C2=A0 =C2=A0 =C2=A0 Heap Blocks: exact=3D282 =
lossy=3D27806<br>&gt; =C2=A0 =C2=A0 =C2=A0 Buffers: shared hit=3D4 read=3D2=
8241<br>&gt; =C2=A0 =C2=A0 Worker 1: =C2=A0actual time=3D24.290..251.993 ro=
ws=3D190815 loops=3D1<br>&gt; =C2=A0 =C2=A0 =C2=A0 Heap Blocks: exact=3D188=
 lossy=3D17179<br>&gt; =C2=A0 =C2=A0 =C2=A0 Buffers: shared hit=3D3 read=3D=
17180<br>&gt; <br>&gt; Not only is &quot;Buffers&quot; clearly a sum of per=
-worker stats, but the &quot;Heap<br>&gt; Blocks&quot; simply disappeared b=
ecause the leader does nothing and we don&#39;t<br>&gt; print zeros.<br><br=
>Heap Blocks is specific to Bitmap Heap Scan. It seems that node specific s=
tats <br>do not aggregate workers&#39; stats into leaders for some existing=
 nodes. For example, <br>Memorize node for Hits, Misses, etc<br><br>=C2=A0 =
=C2=A0-&gt; =C2=A0Nested Loop (actual rows=3D166667 loops=3D3)<br>=C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0-&gt; =C2=A0Parallel Seq Scan on t (actual rows=
=3D33333 loops=3D3)<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-&gt; =C2=A0Memoiz=
e (actual rows=3D5 loops=3D100000)<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0Cache Key: t.j<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0Cache Mode: logical<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0Hits: 32991 =C2=A0Misses: 5 =C2=A0Evictions: 0 =C2=
=A0Overflows: 0 =C2=A0Memory Usage: 2kB<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0Worker 0: =C2=A0Hits: 33551 =C2=A0Misses: 5 =C2=A0E=
victions: 0 =C2=A0Overflows: 0 =C2=A0Memory Usage: 2kB<br>=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Worker 1: =C2=A0Hits: 33443 =C2=A0Mis=
ses: 5 =C2=A0Evictions: 0 =C2=A0Overflows: 0 =C2=A0Memory Usage: 2kB<br>=C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0-&gt; =C2=A0Index Scan =
using uj on u (actual rows=3D5 loops=3D15)<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Index Cond: (j =3D t.j)<br>=
<br>Sort, HashAggregate also do the same stuff. <br><br>&gt; 3) I&#39;m not=
 sure dealing with various EXPLAIN flags may not be entirely<br>&gt; correc=
t. Consider this:<br>&gt; <br>&gt; EXPLAIN (ANALYZE):<br>&gt; <br>&gt; =C2=
=A0 =C2=A0-&gt; =C2=A0Parallel Bitmap Heap Scan on t =C2=A0(...)<br>&gt; =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Recheck Cond: (a &lt; 5000)<br>&gt; =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Rows Removed by Index Recheck: 246882<br>&gt=
; =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Worker 0: =C2=A0Heap Blocks: exact=3D16=
8 lossy=3D15648<br>&gt; =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Worker 1: =C2=A0H=
eap Blocks: exact=3D302 lossy=3D29337<br>&gt; <br>&gt; EXPLAIN (ANALYZE, VE=
RBOSE):<br>&gt; <br>&gt; =C2=A0 =C2=A0-&gt; =C2=A0Parallel Bitmap Heap Scan=
 on public.t =C2=A0(...)<br>&gt; =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Recheck =
Cond: (t.a &lt; 5000)<br>&gt; =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Rows Remove=
d by Index Recheck: 246882<br>&gt; =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Worker=
 0: =C2=A0actual time=3D35.067..300.882 rows=3D282108 loops=3D1<br>&gt; =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Heap Blocks: exact=3D257 lossy=3D2535=
8<br>&gt; =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Worker 1: =C2=A0actual time=3D3=
2.827..302.224 rows=3D217819 loops=3D1<br>&gt; =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0Heap Blocks: exact=3D213 lossy=3D19627<br>&gt; <br>&gt; EXPLAI=
N (ANALYZE, BUFFERS):<br>&gt; <br>&gt; =C2=A0 =C2=A0-&gt; =C2=A0Parallel Bi=
tmap Heap Scan on t =C2=A0(...)<br>&gt; =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0R=
echeck Cond: (a &lt; 5000)<br>&gt; =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Rows R=
emoved by Index Recheck: 246882<br>&gt; =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0B=
uffers: shared hit=3D7 read=3D45421<br>&gt; =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0Worker 0: =C2=A0Heap Blocks: exact=3D236 lossy=3D21870<br>&gt; =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0Worker 1: =C2=A0Heap Blocks: exact=3D234 lossy=
=3D23115<br>&gt; <br>&gt; EXPLAIN (ANALYZE, VERBOSE, BUFFERS):<br>&gt; <br>=
&gt; =C2=A0 =C2=A0-&gt; =C2=A0Parallel Bitmap Heap Scan on public.t =C2=A0(=
...)<br>&gt; =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Recheck Cond: (t.a &lt; 5000=
)<br>&gt; =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Rows Removed by Index Recheck: =
246882<br>&gt; =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Buffers: shared hit=3D7 re=
ad=3D45421<br>&gt; =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Worker 0: =C2=A0actual=
 time=3D28.265..260.381 rows=3D261264 loops=3D1<br>&gt; =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0Heap Blocks: exact=3D260 lossy=3D23477<br>&gt; =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Buffers: shared hit=3D3 read=3D23478<=
br>&gt; =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Worker 1: =C2=A0actual time=3D28.=
224..261.627 rows=3D238663 loops=3D1<br>&gt; =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0Heap Blocks: exact=3D210 lossy=3D21508<br>&gt; =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0Buffers: shared hit=3D4 read=3D21943<br>&gt; <br=
>&gt; Why should the per-worker buffer info be shown when combined with the=
<br>&gt; VERBOSE flag, and not just with BUFFERS, when the patch shows the<=
br>&gt; per-worker info always?<br>&gt; <br><br>It seems that the general e=
xplain print framework requires verbose mode to show per worker stats.<br>F=
or example, how Buffers hits, JIT are printed. While in some specific nodes=
 which involves parallelism,<br>they always show worker blocks. This is why=
 we see that some worker blocks don&#39;t have buffers <br>stats in non ver=
bose mode. There are several existing nodes have the same issue as what thi=
s<br>patch does: memorize, sort, hashaggregate.<br><br>&gt; 4) Now that I t=
hink about this, isn&#39;t the *main* problem really that we<br>&gt; don=
9;t display the sum of the per-worker stats (which I think is wrong)?<br>&g=
t; I mean, we already can get the worker details VERBOSEm right? So the<br>=
&gt; only reason to display that by default seems to be that it the values =
in<br>&gt; &quot;Heap Blocks&quot; are from the leader only.<br><br>If we p=
rint aggregate Heap Blocks in the &#39;leader&#39; block and show worker st=
ats only in <br>verbose mode, does it look better? In this way, it matches =
better with how the <br>general framework prints workers stats, but it diff=
ers from some existing nodes which <br>also print worker stats. As mentione=
d above, they are memorize, sort, and hashaggregate nodes. <br>By the way, =
is that also a problem for these nodes? <br><br>&gt; BTW doesn&#39;t this a=
lso suggest some of the code added to explain.c may<br>&gt; not be quite ne=
cessary? Wouldn&#39;t it be enough to just &quot;extend&quot; the<br>&gt; e=
xisting code printing per-worker stats. (I haven&#39;t tried, so maybe I=
9;m<br>&gt; wrong and we need the new code.)<br>=C2=A0<br>We need the new c=
ode as they are node specific stats and we do call the worker print functio=
n <br>to lay out the explain plan. I think the problem is in which mode we =
should show worker blocks. This<br>is discussed in the above section. =C2=
=A0 =C2=A0</div><div><br></div><div>v3 failed on master, attached a rebased=
 version.=C2=A0 =C2=A0=C2=A0<br><br>Regards,<br>Donghang Lin<br>(ServiceNow=
)<br></div></div>

--000000000000c0f0c00614764bfd--
--000000000000c0f0c20614764bff
Content-Type: application/octet-stream; 
	name="v4-0001-Parallel-Bitmap-Heap-Scan-reports-per-worker-stat.patch"
Content-Disposition: attachment; 
	filename="v4-0001-Parallel-Bitmap-Heap-Scan-reports-per-worker-stat.patch"
Content-Transfer-Encoding: base64
Content-ID: <f_lu6kgklo0>
X-Attachment-Id: f_lu6kgklo0

RnJvbSBjYWRjY2E1MjhmYjI1OTg2ZmQ5OTViNzFlODU5Y2Q2OWQ2ZjQ4M2U0IE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBEYXZpZCBHZWllciA8Z2VpZGF2LnBnQGdtYWlsLmNvbT4KRGF0
ZTogVHVlLCA4IE5vdiAyMDIyIDE5OjQwOjMxICswMTAwClN1YmplY3Q6IFtQQVRDSCB2NF0gUGFy
YWxsZWwgQml0bWFwIEhlYXAgU2NhbiByZXBvcnRzIHBlci13b3JrZXIgc3RhdHMKClNpbWlsYXJs
eSB0byBvdGhlciBub2RlcyAoZS5nLiBoYXNoIGpvaW4sIHNvcnQsIG1lbW9pemUpLApCaXRtYXAg
SGVhcCBTY2FuIG5vdyByZXBvcnRzIHBlci13b3JrZXIgc3RhdHMgaW4gdGhlIEVYUExBSU4KQU5B
TFlaRSBvdXRwdXQuIFByZXZpb3VzbHkgb25seSB0aGUgaGVhcCBibG9ja3Mgc3RhdHMgZm9yIHRo
ZQpsZWFkZXIgd2VyZSByZXBvcnRlZCB3aGljaCB3YXMgaW5jb21wbGV0ZSBpbiBwYXJhbGxlbCBz
Y2Fucy4KCkRpc2N1c3Npb246IGh0dHBzOi8vd3d3LnBvc3RncmVzcWwub3JnL21lc3NhZ2UtaWQv
ZmxhdC9iM2Q4MDk2MS1jMmU1LTM4Y2MtNmEzMi02MTg4NmNkZjc2NmQlNDBnbWFpbC5jb20KLS0t
CiBzcmMvYmFja2VuZC9jb21tYW5kcy9leHBsYWluLmMgICAgICAgICAgICB8IDQ1ICsrKysrKysr
Ky0tCiBzcmMvYmFja2VuZC9leGVjdXRvci9leGVjUGFyYWxsZWwuYyAgICAgICB8ICAzICsKIHNy
Yy9iYWNrZW5kL2V4ZWN1dG9yL25vZGVCaXRtYXBIZWFwc2Nhbi5jIHwgOTMgKysrKysrKysrKysr
KysrKysrKystLS0KIHNyYy9pbmNsdWRlL2V4ZWN1dG9yL25vZGVCaXRtYXBIZWFwc2Nhbi5oIHwg
IDEgKwogc3JjL2luY2x1ZGUvbm9kZXMvZXhlY25vZGVzLmggICAgICAgICAgICAgfCAzMyArKysr
KysrLQogNSBmaWxlcyBjaGFuZ2VkLCAxNTcgaW5zZXJ0aW9ucygrKSwgMTggZGVsZXRpb25zKC0p
CgpkaWZmIC0tZ2l0IGEvc3JjL2JhY2tlbmQvY29tbWFuZHMvZXhwbGFpbi5jIGIvc3JjL2JhY2tl
bmQvY29tbWFuZHMvZXhwbGFpbi5jCmluZGV4IDkyNmQ3MGFmYWYuLjAyMjUxOTk0YzYgMTAwNjQ0
Ci0tLSBhL3NyYy9iYWNrZW5kL2NvbW1hbmRzL2V4cGxhaW4uYworKysgYi9zcmMvYmFja2VuZC9j
b21tYW5kcy9leHBsYWluLmMKQEAgLTM0NjcsMjYgKzM0NjcsNTcgQEAgc2hvd19oYXNoYWdnX2lu
Zm8oQWdnU3RhdGUgKmFnZ3N0YXRlLCBFeHBsYWluU3RhdGUgKmVzKQogc3RhdGljIHZvaWQKIHNo
b3dfdGlkYml0bWFwX2luZm8oQml0bWFwSGVhcFNjYW5TdGF0ZSAqcGxhbnN0YXRlLCBFeHBsYWlu
U3RhdGUgKmVzKQogeworCUFzc2VydChlcy0+YW5hbHl6ZSk7CisKIAlpZiAoZXMtPmZvcm1hdCAh
PSBFWFBMQUlOX0ZPUk1BVF9URVhUKQogCXsKIAkJRXhwbGFpblByb3BlcnR5SW50ZWdlcigiRXhh
Y3QgSGVhcCBCbG9ja3MiLCBOVUxMLAotCQkJCQkJCSAgIHBsYW5zdGF0ZS0+ZXhhY3RfcGFnZXMs
IGVzKTsKKwkJCQkJCQkgICBwbGFuc3RhdGUtPnN0YXRzLmV4YWN0X3BhZ2VzLCBlcyk7CiAJCUV4
cGxhaW5Qcm9wZXJ0eUludGVnZXIoIkxvc3N5IEhlYXAgQmxvY2tzIiwgTlVMTCwKLQkJCQkJCQkg
ICBwbGFuc3RhdGUtPmxvc3N5X3BhZ2VzLCBlcyk7CisJCQkJCQkJICAgcGxhbnN0YXRlLT5zdGF0
cy5sb3NzeV9wYWdlcywgZXMpOwogCX0KIAllbHNlCiAJewotCQlpZiAocGxhbnN0YXRlLT5leGFj
dF9wYWdlcyA+IDAgfHwgcGxhbnN0YXRlLT5sb3NzeV9wYWdlcyA+IDApCisJCWlmIChwbGFuc3Rh
dGUtPnN0YXRzLmV4YWN0X3BhZ2VzID4gMCB8fCBwbGFuc3RhdGUtPnN0YXRzLmxvc3N5X3BhZ2Vz
ID4gMCkKIAkJewogCQkJRXhwbGFpbkluZGVudFRleHQoZXMpOwogCQkJYXBwZW5kU3RyaW5nSW5m
b1N0cmluZyhlcy0+c3RyLCAiSGVhcCBCbG9ja3M6Iik7Ci0JCQlpZiAocGxhbnN0YXRlLT5leGFj
dF9wYWdlcyA+IDApCi0JCQkJYXBwZW5kU3RyaW5nSW5mbyhlcy0+c3RyLCAiIGV4YWN0PSVsZCIs
IHBsYW5zdGF0ZS0+ZXhhY3RfcGFnZXMpOwotCQkJaWYgKHBsYW5zdGF0ZS0+bG9zc3lfcGFnZXMg
PiAwKQotCQkJCWFwcGVuZFN0cmluZ0luZm8oZXMtPnN0ciwgIiBsb3NzeT0lbGQiLCBwbGFuc3Rh
dGUtPmxvc3N5X3BhZ2VzKTsKKwkJCWlmIChwbGFuc3RhdGUtPnN0YXRzLmV4YWN0X3BhZ2VzID4g
MCkKKwkJCQlhcHBlbmRTdHJpbmdJbmZvKGVzLT5zdHIsICIgZXhhY3Q9JWxkIiwgcGxhbnN0YXRl
LT5zdGF0cy5leGFjdF9wYWdlcyk7CisJCQlpZiAocGxhbnN0YXRlLT5zdGF0cy5sb3NzeV9wYWdl
cyA+IDApCisJCQkJYXBwZW5kU3RyaW5nSW5mbyhlcy0+c3RyLCAiIGxvc3N5PSVsZCIsIHBsYW5z
dGF0ZS0+c3RhdHMubG9zc3lfcGFnZXMpOwogCQkJYXBwZW5kU3RyaW5nSW5mb0NoYXIoZXMtPnN0
ciwgJ1xuJyk7CiAJCX0KIAl9CisKKwlpZiAocGxhbnN0YXRlLT5wc3RhdGUgIT0gTlVMTCkKKwl7
CisJCWZvciAoaW50IG4gPSAwOyBuIDwgcGxhbnN0YXRlLT5zaW5zdHJ1bWVudC0+bnVtX3dvcmtl
cnM7IG4rKykKKwkJeworCQkJQml0bWFwSGVhcFNjYW5JbnN0cnVtZW50YXRpb24gKnNpID0gJnBs
YW5zdGF0ZS0+c2luc3RydW1lbnQtPnNpbnN0cnVtZW50W25dOworCisJCQlpZiAoc2ktPmV4YWN0
X3BhZ2VzID09IDAgJiYgc2ktPmxvc3N5X3BhZ2VzID09IDApCisJCQkJY29udGludWU7CisKKwkJ
CWlmIChlcy0+d29ya2Vyc19zdGF0ZSkKKwkJCQlFeHBsYWluT3BlbldvcmtlcihuLCBlcyk7CisK
KwkJCWlmIChlcy0+Zm9ybWF0ID09IEVYUExBSU5fRk9STUFUX1RFWFQpCisJCQl7CisJCQkJRXhw
bGFpbkluZGVudFRleHQoZXMpOworCQkJCWFwcGVuZFN0cmluZ0luZm8oZXMtPnN0ciwgIkhlYXAg
QmxvY2tzOiBleGFjdD0lbGQgbG9zc3k9JWxkXG4iLAorCQkJCQkJIHNpLT5leGFjdF9wYWdlcywg
c2ktPmxvc3N5X3BhZ2VzKTsKKwkJCX0KKwkJCWVsc2UKKwkJCXsKKwkJCQlFeHBsYWluUHJvcGVy
dHlJbnRlZ2VyKCJFeGFjdCBIZWFwIEJsb2NrcyIsIE5VTEwsIHNpLT5leGFjdF9wYWdlcywgZXMp
OworCQkJCUV4cGxhaW5Qcm9wZXJ0eUludGVnZXIoIkxvc3N5IEhlYXAgQmxvY2tzIiwgTlVMTCwg
c2ktPmxvc3N5X3BhZ2VzLCBlcyk7CisJCQl9CisKKwkJCWlmIChlcy0+d29ya2Vyc19zdGF0ZSkK
KwkJCQlFeHBsYWluQ2xvc2VXb3JrZXIobiwgZXMpOworCQl9CisJfQogfQogCiAvKgpkaWZmIC0t
Z2l0IGEvc3JjL2JhY2tlbmQvZXhlY3V0b3IvZXhlY1BhcmFsbGVsLmMgYi9zcmMvYmFja2VuZC9l
eGVjdXRvci9leGVjUGFyYWxsZWwuYwppbmRleCA4YzUzZDE4MzRlLi5iZmIzNDE5ZWZiIDEwMDY0
NAotLS0gYS9zcmMvYmFja2VuZC9leGVjdXRvci9leGVjUGFyYWxsZWwuYworKysgYi9zcmMvYmFj
a2VuZC9leGVjdXRvci9leGVjUGFyYWxsZWwuYwpAQCAtMTA3Niw2ICsxMDc2LDkgQEAgRXhlY1Bh
cmFsbGVsUmV0cmlldmVJbnN0cnVtZW50YXRpb24oUGxhblN0YXRlICpwbGFuc3RhdGUsCiAJCWNh
c2UgVF9NZW1vaXplU3RhdGU6CiAJCQlFeGVjTWVtb2l6ZVJldHJpZXZlSW5zdHJ1bWVudGF0aW9u
KChNZW1vaXplU3RhdGUgKikgcGxhbnN0YXRlKTsKIAkJCWJyZWFrOworCQljYXNlIFRfQml0bWFw
SGVhcFNjYW5TdGF0ZToKKwkJCUV4ZWNCaXRtYXBIZWFwUmV0cmlldmVJbnN0cnVtZW50YXRpb24o
KEJpdG1hcEhlYXBTY2FuU3RhdGUgKikgcGxhbnN0YXRlKTsKKwkJCWJyZWFrOwogCQlkZWZhdWx0
OgogCQkJYnJlYWs7CiAJfQpkaWZmIC0tZ2l0IGEvc3JjL2JhY2tlbmQvZXhlY3V0b3Ivbm9kZUJp
dG1hcEhlYXBzY2FuLmMgYi9zcmMvYmFja2VuZC9leGVjdXRvci9ub2RlQml0bWFwSGVhcHNjYW4u
YwppbmRleCBjZWU3ZjQ1YWFiLi4wZDdkZDFmMTY1IDEwMDY0NAotLS0gYS9zcmMvYmFja2VuZC9l
eGVjdXRvci9ub2RlQml0bWFwSGVhcHNjYW4uYworKysgYi9zcmMvYmFja2VuZC9leGVjdXRvci9u
b2RlQml0bWFwSGVhcHNjYW4uYwpAQCAtMjA1LDkgKzIwNSw5IEBAIEJpdG1hcEhlYXBOZXh0KEJp
dG1hcEhlYXBTY2FuU3RhdGUgKm5vZGUpCiAJCQlCaXRtYXBBZGp1c3RQcmVmZXRjaEl0ZXJhdG9y
KG5vZGUsIHRibXJlcyk7CiAKIAkJCWlmICh0Ym1yZXMtPm50dXBsZXMgPj0gMCkKLQkJCQlub2Rl
LT5leGFjdF9wYWdlcysrOworCQkJCW5vZGUtPnN0YXRzLmV4YWN0X3BhZ2VzKys7CiAJCQllbHNl
Ci0JCQkJbm9kZS0+bG9zc3lfcGFnZXMrKzsKKwkJCQlub2RlLT5zdGF0cy5sb3NzeV9wYWdlcysr
OwogCiAJCQkvKgogCQkJICogV2UgY2FuIHNraXAgZmV0Y2hpbmcgdGhlIGhlYXAgcGFnZSBpZiB3
ZSBkb24ndCBuZWVkIGFueSBmaWVsZHMKQEAgLTY0Nyw2ICs2NDcsMTcgQEAgRXhlY0VuZEJpdG1h
cEhlYXBTY2FuKEJpdG1hcEhlYXBTY2FuU3RhdGUgKm5vZGUpCiB7CiAJVGFibGVTY2FuRGVzYyBz
Y2FuRGVzYzsKIAorCS8qCisJICogV2hlbiBlbmRpbmcgYSBwYXJhbGxlbCB3b3JrZXIsIGNvcHkg
dGhlIHN0YXRpc3RpY3MgZ2F0aGVyZWQgYnkgdGhlCisJICogd29ya2VyIGJhY2sgaW50byBzaGFy
ZWQgbWVtb3J5IHNvIHRoYXQgaXQgY2FuIGJlIHBpY2tlZCB1cCBieSB0aGUgbWFpbgorCSAqIHBy
b2Nlc3MgdG8gcmVwb3J0IGluIEVYUExBSU4gQU5BTFlaRS4KKwkgKi8KKwlpZiAobm9kZS0+c2lu
c3RydW1lbnQgIT0gTlVMTCAmJiBJc1BhcmFsbGVsV29ya2VyKCkpCisJeworCQlBc3NlcnQoUGFy
YWxsZWxXb3JrZXJOdW1iZXIgPD0gbm9kZS0+c2luc3RydW1lbnQtPm51bV93b3JrZXJzKTsKKwkJ
bm9kZS0+c2luc3RydW1lbnQtPnNpbnN0cnVtZW50W1BhcmFsbGVsV29ya2VyTnVtYmVyXSA9IG5v
ZGUtPnN0YXRzOworCX0KKwogCS8qCiAJICogZXh0cmFjdCBpbmZvcm1hdGlvbiBmcm9tIHRoZSBu
b2RlCiAJICovCkBAIC03MTYsOCArNzI3LDggQEAgRXhlY0luaXRCaXRtYXBIZWFwU2NhbihCaXRt
YXBIZWFwU2NhbiAqbm9kZSwgRVN0YXRlICplc3RhdGUsIGludCBlZmxhZ3MpCiAJc2NhbnN0YXRl
LT5yZXR1cm5fZW1wdHlfdHVwbGVzID0gMDsKIAlzY2Fuc3RhdGUtPnZtYnVmZmVyID0gSW52YWxp
ZEJ1ZmZlcjsKIAlzY2Fuc3RhdGUtPnB2bWJ1ZmZlciA9IEludmFsaWRCdWZmZXI7Ci0Jc2NhbnN0
YXRlLT5leGFjdF9wYWdlcyA9IDA7Ci0Jc2NhbnN0YXRlLT5sb3NzeV9wYWdlcyA9IDA7CisJc2Nh
bnN0YXRlLT5zdGF0cy5leGFjdF9wYWdlcyA9IDA7CisJc2NhbnN0YXRlLT5zdGF0cy5sb3NzeV9w
YWdlcyA9IDA7CiAJc2NhbnN0YXRlLT5wcmVmZXRjaF9pdGVyYXRvciA9IE5VTEw7CiAJc2NhbnN0
YXRlLT5wcmVmZXRjaF9wYWdlcyA9IDA7CiAJc2NhbnN0YXRlLT5wcmVmZXRjaF90YXJnZXQgPSAw
OwpAQCAtODQwLDcgKzg1MSwyMiBAQCB2b2lkCiBFeGVjQml0bWFwSGVhcEVzdGltYXRlKEJpdG1h
cEhlYXBTY2FuU3RhdGUgKm5vZGUsCiAJCQkJCSAgIFBhcmFsbGVsQ29udGV4dCAqcGN4dCkKIHsK
LQlzaG1fdG9jX2VzdGltYXRlX2NodW5rKCZwY3h0LT5lc3RpbWF0b3IsIHNpemVvZihQYXJhbGxl
bEJpdG1hcEhlYXBTdGF0ZSkpOworCVNpemUJCXNpemU7CisKKwkvKgorCSAqIFdlIHN0b3JlIFBh
cmFsbGVsQml0bWFwSGVhcFN0YXRlIGZvbGxvd2VkIGJ5CisJICogU2hhcmVkQml0bWFwSGVhcElu
c3RydW1lbnRhdGlvbkluZm8gaW4gdGhlIHNhbWUgc2htZW0gY2h1bmsuCisJICovCisJc2l6ZSA9
IE1BWEFMSUdOKHNpemVvZihQYXJhbGxlbEJpdG1hcEhlYXBTdGF0ZSkpOworCisJLyogZG9uJ3Qg
bmVlZCB0aGlzIGlmIG5vdCBpbnN0cnVtZW50aW5nIG9yIG5vIHdvcmtlcnMgKi8KKwlpZiAobm9k
ZS0+c3MucHMuaW5zdHJ1bWVudCAmJiBwY3h0LT5ud29ya2VycyA+IDApCisJeworCQlzaXplID0g
YWRkX3NpemUoc2l6ZSwgb2Zmc2V0b2YoU2hhcmVkQml0bWFwSGVhcEluc3RydW1lbnRhdGlvbiwg
c2luc3RydW1lbnQpKTsKKwkJc2l6ZSA9IGFkZF9zaXplKHNpemUsIG11bF9zaXplKHBjeHQtPm53
b3JrZXJzLCBzaXplb2YoQml0bWFwSGVhcFNjYW5JbnN0cnVtZW50YXRpb24pKSk7CisJfQorCisJ
c2htX3RvY19lc3RpbWF0ZV9jaHVuaygmcGN4dC0+ZXN0aW1hdG9yLCBzaXplKTsKIAlzaG1fdG9j
X2VzdGltYXRlX2tleXMoJnBjeHQtPmVzdGltYXRvciwgMSk7CiB9CiAKQEAgLTg1NSwxMyArODgx
LDI3IEBAIEV4ZWNCaXRtYXBIZWFwSW5pdGlhbGl6ZURTTShCaXRtYXBIZWFwU2NhblN0YXRlICpu
b2RlLAogCQkJCQkJCVBhcmFsbGVsQ29udGV4dCAqcGN4dCkKIHsKIAlQYXJhbGxlbEJpdG1hcEhl
YXBTdGF0ZSAqcHN0YXRlOworCVNoYXJlZEJpdG1hcEhlYXBJbnN0cnVtZW50YXRpb24gKnNpbnN0
cnVtZW50ID0gTlVMTDsKIAlkc2FfYXJlYSAgICpkc2EgPSBub2RlLT5zcy5wcy5zdGF0ZS0+ZXNf
cXVlcnlfZHNhOworCWNoYXIJICAgKnB0cjsKKwlTaXplCQlzaXplOwogCiAJLyogSWYgdGhlcmUn
cyBubyBEU0EsIHRoZXJlIGFyZSBubyB3b3JrZXJzOyBpbml0aWFsaXplIG5vdGhpbmcuICovCiAJ
aWYgKGRzYSA9PSBOVUxMKQogCQlyZXR1cm47CiAKLQlwc3RhdGUgPSBzaG1fdG9jX2FsbG9jYXRl
KHBjeHQtPnRvYywgc2l6ZW9mKFBhcmFsbGVsQml0bWFwSGVhcFN0YXRlKSk7CisJc2l6ZSA9IE1B
WEFMSUdOKHNpemVvZihQYXJhbGxlbEJpdG1hcEhlYXBTdGF0ZSkpOworCWlmIChub2RlLT5zcy5w
cy5pbnN0cnVtZW50ICYmIHBjeHQtPm53b3JrZXJzID4gMCkKKwl7CisJCXNpemUgPSBhZGRfc2l6
ZShzaXplLCBvZmZzZXRvZihTaGFyZWRCaXRtYXBIZWFwSW5zdHJ1bWVudGF0aW9uLCBzaW5zdHJ1
bWVudCkpOworCQlzaXplID0gYWRkX3NpemUoc2l6ZSwgbXVsX3NpemUocGN4dC0+bndvcmtlcnMs
IHNpemVvZihCaXRtYXBIZWFwU2Nhbkluc3RydW1lbnRhdGlvbikpKTsKKwl9CisKKwlwdHIgPSBz
aG1fdG9jX2FsbG9jYXRlKHBjeHQtPnRvYywgc2l6ZSk7CisJcHN0YXRlID0gKFBhcmFsbGVsQml0
bWFwSGVhcFN0YXRlICopIHB0cjsKKwlwdHIgKz0gTUFYQUxJR04oc2l6ZW9mKFBhcmFsbGVsQml0
bWFwSGVhcFN0YXRlKSk7CisJaWYgKG5vZGUtPnNzLnBzLmluc3RydW1lbnQgJiYgcGN4dC0+bndv
cmtlcnMgPiAwKQorCQlzaW5zdHJ1bWVudCA9IChTaGFyZWRCaXRtYXBIZWFwSW5zdHJ1bWVudGF0
aW9uICopIHB0cjsKIAogCXBzdGF0ZS0+dGJtaXRlcmF0b3IgPSAwOwogCXBzdGF0ZS0+cHJlZmV0
Y2hfaXRlcmF0b3IgPSAwOwpAQCAtODc0LDggKzkxNCwxNiBAQCBFeGVjQml0bWFwSGVhcEluaXRp
YWxpemVEU00oQml0bWFwSGVhcFNjYW5TdGF0ZSAqbm9kZSwKIAogCUNvbmRpdGlvblZhcmlhYmxl
SW5pdCgmcHN0YXRlLT5jdik7CiAKKwkvKiBlbnN1cmUgYW55IHVuZmlsbGVkIHNsb3RzIHdpbGwg
Y29udGFpbiB6ZXJvZXMgKi8KKwlpZiAoc2luc3RydW1lbnQpCisJeworCQlzaW5zdHJ1bWVudC0+
bnVtX3dvcmtlcnMgPSBwY3h0LT5ud29ya2VyczsKKwkJbWVtc2V0KHNpbnN0cnVtZW50LT5zaW5z
dHJ1bWVudCwgMCwgcGN4dC0+bndvcmtlcnMgKiBzaXplb2YoQml0bWFwSGVhcFNjYW5JbnN0cnVt
ZW50YXRpb24pKTsKKwl9CisKIAlzaG1fdG9jX2luc2VydChwY3h0LT50b2MsIG5vZGUtPnNzLnBz
LnBsYW4tPnBsYW5fbm9kZV9pZCwgcHN0YXRlKTsKIAlub2RlLT5wc3RhdGUgPSBwc3RhdGU7CisJ
bm9kZS0+c2luc3RydW1lbnQgPSBzaW5zdHJ1bWVudDsKIH0KIAogLyogLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQpAQCAtOTE3
LDEwICs5NjUsMzcgQEAgdm9pZAogRXhlY0JpdG1hcEhlYXBJbml0aWFsaXplV29ya2VyKEJpdG1h
cEhlYXBTY2FuU3RhdGUgKm5vZGUsCiAJCQkJCQkJICAgUGFyYWxsZWxXb3JrZXJDb250ZXh0ICpw
d2N4dCkKIHsKLQlQYXJhbGxlbEJpdG1hcEhlYXBTdGF0ZSAqcHN0YXRlOworCWNoYXIJICAgKnB0
cjsKIAogCUFzc2VydChub2RlLT5zcy5wcy5zdGF0ZS0+ZXNfcXVlcnlfZHNhICE9IE5VTEwpOwog
Ci0JcHN0YXRlID0gc2htX3RvY19sb29rdXAocHdjeHQtPnRvYywgbm9kZS0+c3MucHMucGxhbi0+
cGxhbl9ub2RlX2lkLCBmYWxzZSk7Ci0Jbm9kZS0+cHN0YXRlID0gcHN0YXRlOworCXB0ciA9IHNo
bV90b2NfbG9va3VwKHB3Y3h0LT50b2MsIG5vZGUtPnNzLnBzLnBsYW4tPnBsYW5fbm9kZV9pZCwg
ZmFsc2UpOworCisJbm9kZS0+cHN0YXRlID0gKFBhcmFsbGVsQml0bWFwSGVhcFN0YXRlICopIHB0
cjsKKwlwdHIgKz0gTUFYQUxJR04oc2l6ZW9mKFBhcmFsbGVsQml0bWFwSGVhcFN0YXRlKSk7CisK
KwlpZiAobm9kZS0+c3MucHMuaW5zdHJ1bWVudCkKKwkJbm9kZS0+c2luc3RydW1lbnQgPSAoU2hh
cmVkQml0bWFwSGVhcEluc3RydW1lbnRhdGlvbiAqKSBwdHI7Cit9CisKKy8qIC0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KKyAq
CQlFeGVjQml0bWFwSGVhcFJldHJpZXZlSW5zdHJ1bWVudGF0aW9uCisgKgorICoJCVRyYW5zZmVy
IGJpdG1hcCBoZWFwIHNjYW4gc3RhdGlzdGljcyBmcm9tIERTTSB0byBwcml2YXRlIG1lbW9yeS4K
KyAqIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0KKyAqLwordm9pZAorRXhlY0JpdG1hcEhlYXBSZXRyaWV2ZUluc3RydW1lbnRh
dGlvbihCaXRtYXBIZWFwU2NhblN0YXRlICpub2RlKQoreworCVNoYXJlZEJpdG1hcEhlYXBJbnN0
cnVtZW50YXRpb24gKnNpbnN0cnVtZW50ID0gbm9kZS0+c2luc3RydW1lbnQ7CisJU2l6ZQkJc2l6
ZTsKKworCWlmIChzaW5zdHJ1bWVudCA9PSBOVUxMKQorCQlyZXR1cm47CisKKwlzaXplID0gb2Zm
c2V0b2YoU2hhcmVkQml0bWFwSGVhcEluc3RydW1lbnRhdGlvbiwgc2luc3RydW1lbnQpCisJCSsg
c2luc3RydW1lbnQtPm51bV93b3JrZXJzICogc2l6ZW9mKEJpdG1hcEhlYXBTY2FuSW5zdHJ1bWVu
dGF0aW9uKTsKKworCW5vZGUtPnNpbnN0cnVtZW50ID0gcGFsbG9jKHNpemUpOworCW1lbWNweShu
b2RlLT5zaW5zdHJ1bWVudCwgc2luc3RydW1lbnQsIHNpemUpOwogfQpkaWZmIC0tZ2l0IGEvc3Jj
L2luY2x1ZGUvZXhlY3V0b3Ivbm9kZUJpdG1hcEhlYXBzY2FuLmggYi9zcmMvaW5jbHVkZS9leGVj
dXRvci9ub2RlQml0bWFwSGVhcHNjYW4uaAppbmRleCBlYTAwM2E5Y2FhLi40NDZhNjY0NTkwIDEw
MDY0NAotLS0gYS9zcmMvaW5jbHVkZS9leGVjdXRvci9ub2RlQml0bWFwSGVhcHNjYW4uaAorKysg
Yi9zcmMvaW5jbHVkZS9leGVjdXRvci9ub2RlQml0bWFwSGVhcHNjYW4uaApAQCAtMjgsNSArMjgs
NiBAQCBleHRlcm4gdm9pZCBFeGVjQml0bWFwSGVhcFJlSW5pdGlhbGl6ZURTTShCaXRtYXBIZWFw
U2NhblN0YXRlICpub2RlLAogCQkJCQkJCQkJCSAgUGFyYWxsZWxDb250ZXh0ICpwY3h0KTsKIGV4
dGVybiB2b2lkIEV4ZWNCaXRtYXBIZWFwSW5pdGlhbGl6ZVdvcmtlcihCaXRtYXBIZWFwU2NhblN0
YXRlICpub2RlLAogCQkJCQkJCQkJCSAgIFBhcmFsbGVsV29ya2VyQ29udGV4dCAqcHdjeHQpOwor
ZXh0ZXJuIHZvaWQgRXhlY0JpdG1hcEhlYXBSZXRyaWV2ZUluc3RydW1lbnRhdGlvbihCaXRtYXBI
ZWFwU2NhblN0YXRlICpub2RlKTsKIAogI2VuZGlmCQkJCQkJCS8qIE5PREVCSVRNQVBIRUFQU0NB
Tl9IICovCmRpZmYgLS1naXQgYS9zcmMvaW5jbHVkZS9ub2Rlcy9leGVjbm9kZXMuaCBiL3NyYy9p
bmNsdWRlL25vZGVzL2V4ZWNub2Rlcy5oCmluZGV4IDE3NzRjNTZhZTMuLjgzMDZlMGM0N2IgMTAw
NjQ0Ci0tLSBhL3NyYy9pbmNsdWRlL25vZGVzL2V4ZWNub2Rlcy5oCisrKyBiL3NyYy9pbmNsdWRl
L25vZGVzL2V4ZWNub2Rlcy5oCkBAIC0xNzM1LDYgKzE3MzUsMTIgQEAgdHlwZWRlZiBzdHJ1Y3Qg
Qml0bWFwSW5kZXhTY2FuU3RhdGUKIAlzdHJ1Y3QgSW5kZXhTY2FuRGVzY0RhdGEgKmJpc3NfU2Nh
bkRlc2M7CiB9IEJpdG1hcEluZGV4U2NhblN0YXRlOwogCit0eXBlZGVmIHN0cnVjdCBCaXRtYXBI
ZWFwU2Nhbkluc3RydW1lbnRhdGlvbgoreworCWxvbmcgbG9zc3lfcGFnZXM7CisJbG9uZyBleGFj
dF9wYWdlczsKK30gQml0bWFwSGVhcFNjYW5JbnN0cnVtZW50YXRpb247CisKIC8qIC0tLS0tLS0t
LS0tLS0tLS0KICAqCSBTaGFyZWRCaXRtYXBTdGF0ZSBpbmZvcm1hdGlvbgogICoKQEAgLTE3Nzgs
NiArMTc4NCwyMCBAQCB0eXBlZGVmIHN0cnVjdCBQYXJhbGxlbEJpdG1hcEhlYXBTdGF0ZQogCUNv
bmRpdGlvblZhcmlhYmxlIGN2OwogfSBQYXJhbGxlbEJpdG1hcEhlYXBTdGF0ZTsKIAorLyogLS0t
LS0tLS0tLS0tLS0tLQorICoJIEluc3RydW1lbnRhdGlvbiBkYXRhIGZvciBhIHBhcmFsbGVsIGJp
dG1hcCBoZWFwIHNjYW4uCisgKgorICogRHVyaW5nIGEgcGFyYWxsZWwgYml0bWFwIGhlYXAgc2Nh
biwgdGhpcyBsaXZlcyBpbiBzaGFyZWQgbWVtb3J5LiAgQXQgdGhlCisgKiBlbmQsIGVhY2ggd29y
a2VyIGNvcGllcyB0aGVpciBvd24gc3RhdHMgdG8gdGhlIHJpZ2h0IHNsb3QuICBGaW5hbGx5LCB0
aGUKKyAqIGxlYWRlciBjb3BpZXMgdGhlIGRhdGEgdG8gaXRzIGxvY2FsIG1lbW9yeQorICAqIC0t
LS0tLS0tLS0tLS0tLS0KKyAqLwordHlwZWRlZiBzdHJ1Y3QgU2hhcmVkQml0bWFwSGVhcEluc3Ry
dW1lbnRhdGlvbgoreworCWludAkJCW51bV93b3JrZXJzOworCUJpdG1hcEhlYXBTY2FuSW5zdHJ1
bWVudGF0aW9uIHNpbnN0cnVtZW50W0ZMRVhJQkxFX0FSUkFZX01FTUJFUl07Cit9IFNoYXJlZEJp
dG1hcEhlYXBJbnN0cnVtZW50YXRpb247CisKIC8qIC0tLS0tLS0tLS0tLS0tLS0KICAqCSBCaXRt
YXBIZWFwU2NhblN0YXRlIGluZm9ybWF0aW9uCiAgKgpAQCAtMTgxMiw4ICsxODMyLDYgQEAgdHlw
ZWRlZiBzdHJ1Y3QgQml0bWFwSGVhcFNjYW5TdGF0ZQogCWludAkJCXJldHVybl9lbXB0eV90dXBs
ZXM7CiAJQnVmZmVyCQl2bWJ1ZmZlcjsKIAlCdWZmZXIJCXB2bWJ1ZmZlcjsKLQlsb25nCQlleGFj
dF9wYWdlczsKLQlsb25nCQlsb3NzeV9wYWdlczsKIAlUQk1JdGVyYXRvciAqcHJlZmV0Y2hfaXRl
cmF0b3I7CiAJaW50CQkJcHJlZmV0Y2hfcGFnZXM7CiAJaW50CQkJcHJlZmV0Y2hfdGFyZ2V0OwpA
QCAtMTgyMiw2ICsxODQwLDE3IEBAIHR5cGVkZWYgc3RydWN0IEJpdG1hcEhlYXBTY2FuU3RhdGUK
IAlUQk1TaGFyZWRJdGVyYXRvciAqc2hhcmVkX3RibWl0ZXJhdG9yOwogCVRCTVNoYXJlZEl0ZXJh
dG9yICpzaGFyZWRfcHJlZmV0Y2hfaXRlcmF0b3I7CiAJUGFyYWxsZWxCaXRtYXBIZWFwU3RhdGUg
KnBzdGF0ZTsKKworCS8qCisJICogSW4gYSBwYXJhbGxlbGl6ZWQgYml0bWFwIGhlYXAgc2Nhbiwg
ZWFjaCB3b3JrZXIgc2V0cyB0aGVpcgorCSAqIGluc3RydW1lbnRhdG9uIGRhdGEgaW4gcHN0YXRl
LT5zaW5zdHJ1bWVudCBhdCB0aGUgZW5kLiAgVGhlIGxlYWRlcgorCSAqIGNvcGllcyB0aGUgc2hh
cmVkLW1lbW9yeSBpbmZvIGJhY2sgdG8gbG9jYWwgc3RvcmFnZSBiZWZvcmUgRFNNCisJICogc2h1
dGRvd24sIGFuZCBzdG9yZXMgaXQgaGVyZSBpbiAnaW5zdHJ1bWVudCcuICBJdCByZW1haW5zIE5V
TEwgaW4KKwkgKiB3b3JrZXJzIGFuZCBpbiBub24tcGFyYWxsZWwgc2NhbnMuCisJICovCisJU2hh
cmVkQml0bWFwSGVhcEluc3RydW1lbnRhdGlvbiAqc2luc3RydW1lbnQ7CisKKwlCaXRtYXBIZWFw
U2Nhbkluc3RydW1lbnRhdGlvbiBzdGF0czsKIH0gQml0bWFwSGVhcFNjYW5TdGF0ZTsKIAogLyog
LS0tLS0tLS0tLS0tLS0tLQotLSAKMi40MC4xCgo=
--000000000000c0f0c20614764bff--