From: Tom Lane <tgl@sss.pgh.pa.us>
To: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
cc: Alvaro Herrera <alvherre@alvh.no-ip.org>,
        Stephen Frost <sfrost@snowman.net>,
        Magnus Hagander <magnus@hagander.net>,
        PostgreSQL WWW <pgsql-www@lists.postgresql.org>
Subject: Re: Bogus reports from coverage.postgresql.org
In-reply-to: <4710.1521044117@sss.pgh.pa.us>
References: <20180314154352.ao4psdohgxd4pw65@alvherre.pgsql>
 <4710.1521044117@sss.pgh.pa.us>
Comments: In-reply-to Tom Lane <tgl@sss.pgh.pa.us>
	message dated "Wed, 14 Mar 2018 12:15:17 -0400"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <5743.1521045105.1@sss.pgh.pa.us>
Content-Transfer-Encoding: quoted-printable
Date: Wed, 14 Mar 2018 12:31:45 -0400
Message-ID: <5744.1521045105@sss.pgh.pa.us>
Precedence: bulk

I wrote:
> I also notice an interesting warning with either version:

> $ make coverage-html
> /usr/bin/lcov --gcov-tool /usr/bin/gcov -q --no-external -c -i -d . -d .=
 -o lcov_base.info
> geninfo: Note: --initial does not generate branch coverage data
> /usr/bin/lcov --gcov-tool /usr/bin/gcov -q --no-external -c -d . -d . -o=
 lcov_test.info
> rm -rf coverage
> /usr/bin/genhtml -q --legend -o coverage --title=3D'PostgreSQL 11devel' =
--num-spaces=3D4 --prefix=3D'/home/tgl/pgsql' lcov_base.info lcov_test.inf=
o
> touch coverage-html-stamp

> No idea what "--initial" refers to, but it suggests that we're
> misusing the tool somehow with this configuration.

So I went and read the lcov man page, and according to it we're doing
this entirely wrong.  The correct process, saith lcov, is

       -i
       --initial
              Capture initial zero coverage data.

              Run  lcov  with -c and this option on the directories contai=
ning
              .bb, .bbg or .gcno files  before  running  any  test  case. =
 The
              result  is  a  "baseline"  coverage data file that contains =
zero
              coverage for every instrumented line.  Combine  this  data  =
file
              (using  lcov  -a) with coverage data files captured after a =
test
              run to ensure that the percentage of total lines covered is =
cor-
              rect  even when not all source code files were loaded during=
 the
              test.

              Recommended procedure when capturing data for a test case:

              1. create baseline coverage data file
                     # lcov -c -i -d appdir -o app_base.info

              2. perform test
                     # appdir/test

              3. create test coverage data file
                     # lcov -c -d appdir -o app_test.info

              4. combine baseline and test coverage data
                     # lcov -a app_base.info -a app_test.info -o
                     app_total.info

Our process swaps steps 1 and 2.  What I now suspect is that
we accidentally get away with that for basic mode, but it
doesn't work with lcov_branch_coverage.

			regards, tom lane