MIME-Version: 1.0
References: <b3d80961-c2e5-38cc-6a32-61886cdf766d@gmail.com>
 <CAFiTN-v1yDvU=X+hwfJ+55=sbgDj=_kuvbduEG-F7=BjpWcnuw@mail.gmail.com>
 <efcb213c-c87c-41f0-85b7-7fa8079df807@enterprisedb.com> <CAA=D8a1ZGWTk0RDH-LyU==RsE-217T29eprS-NBhyj+eLHyRrQ@mail.gmail.com>
 <CAAKRu_YjBPfGp85ehY1t9NN=R9pB9k=6rztaeVkAm-OeTqUK4g@mail.gmail.com>
 <CAA=D8a1G5CgvdWLd5_VCEc5Sa3rgJftcGfTEvYqG1HnjMBQmpA@mail.gmail.com>
 <13bd913f-94b6-43cf-b849-4d762e5297d8@yandex.ru> <TYWPR01MB10982C38F6BD6BE4D1D5CCC5EB1D62@TYWPR01MB10982.jpnprd01.prod.outlook.com>
 <CAApHDvqsTB-zNw_SpcujnYCi6FYN3M16kJn9QX7pHtVq5wP_Lw@mail.gmail.com> <CAApHDvpAz_wVNXeCEhjyxcXM6yXMhs+wG+3vywDaJptj6w7dRA@mail.gmail.com>
In-Reply-To: <CAApHDvpAz_wVNXeCEhjyxcXM6yXMhs+wG+3vywDaJptj6w7dRA@mail.gmail.com>
From: David Rowley <dgrowleyml@gmail.com>
Date: Mon, 8 Jul 2024 12:19:58 +1200
Message-ID: <CAApHDvpS_97TU+jWPc=T83WPp7vJa1dTw3mojEtAVEZOWh9bjQ@mail.gmail.com>
Subject: Re: Parallel Bitmap Heap Scan reports per-worker stats in EXPLAIN ANALYZE
To: Masahiro.Ikeda@nttdata.com
Cc: lena.ribackina@yandex.ru, donghanglin@gmail.com, geidav.pg@gmail.com, 
	melanieplageman@gmail.com, tomas.vondra@enterprisedb.com, 
	dilipbalaut@gmail.com, pgsql-hackers@lists.postgresql.org, hlinnaka@iki.fi
Content-Type: text/plain; charset="UTF-8"
Archived-At: <https://www.postgresql.org/message-id/CAApHDvpS_97TU%2BjWPc%3DT83WPp7vJa1dTw3mojEtAVEZOWh9bjQ%40mail.gmail.com>
Precedence: bulk

On Fri, 5 Jul 2024 at 12:52, David Rowley <dgrowleyml@gmail.com> wrote:
> I propose we change these to uint64 while causing churn in this area,
> probably as a follow-on patch.  I think a uint32 isn't wide enough as
> you could exceed the limit with rescans.

I wondered how large a query it would take to cause this problem.  I tried:

create table a (a int);
insert into a select x%1000 from generate_Series(1,1500000)x;
create index on a(a);
vacuum freeze analyze a;

set enable_hashjoin=0;
set enable_mergejoin=0;
set enable_indexscan=0;
set max_parallel_workers_per_gather=0;

explain (analyze, costs off, timing off, summary off)
select count(*) from a a1 inner join a a2 on a1.a=a2.a;

After about 15 mins, the trimmed output from Linux is:

 Aggregate (actual rows=1 loops=1)
   ->  Nested Loop (actual rows=2250000000 loops=1)
         ->  Seq Scan on a a1 (actual rows=1500000 loops=1)
         ->  Bitmap Heap Scan on a a2 (actual rows=1500 loops=1500000)
               Recheck Cond: (a1.a = a)
               Heap Blocks: exact=2250000000
               ->  Bitmap Index Scan on a_a_idx (actual rows=1500 loops=1500000)
                     Index Cond: (a = a1.a)

Whereas, on MSVC, due to sizeof(long) == 4, it's:

 Aggregate  (actual rows=1 loops=1)
   ->  Nested Loop  (actual rows=2250000000 loops=1)
         ->  Seq Scan on a a1  (actual rows=1500000 loops=1)
         ->  Bitmap Heap Scan on a a2  (actual rows=1500 loops=1500000)
               Recheck Cond: (a1.a = a)
               ->  Bitmap Index Scan on a_a_idx (actual rows=1500 loops=1500000)
                     Index Cond: (a = a1.a)

Notice the "Heap Blocks: exact=2250000000" is missing on Windows.
This is because it wrapped around to a negative value and
show_tidbitmap_info() only shows > 0 values.

I feel this is a good enough justification to increase the width of
those counters to uint64, so I'll do that too.

David