From: Alexander Lakhin <exclusion@gmail.com>
To: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: gcov coverage data not full with immediate stop
Date: Mon, 11 May 2020 12:00:00 +0300
Message-ID: <445c1dd9-bb8a-3153-04b2-40ba97be87b1@gmail.com> (raw)
In-Reply-To: <20200511034243.GA5671@alvherre.pgsql>
References: <20200511034243.GA5671@alvherre.pgsql>
Hello Alvaro,
11.05.2020 06:42, Alvaro Herrera wrote:
> (Strangely, I was just thinking about these branches of mine as I
> closed my week last Friday...)
>
> On 2020-May-10, Alexander Lakhin wrote:
>
>> So if we want to make the coverage reports more precise, I see the three
>> ways:
>> 1. Change the stop mode in teardown_node to fast (probably only when
>> configured with --enable-coverage);
>> 2. Explicitly stop nodes in TAP tests (where it's important) -- seems
>> too tedious and troublesome;
>> 3. Explicitly call __gcov_flush in SIGQUIT handler (quickdie)?
> I tried your idea 3 a long time ago and my experiments didn't show an
> increase in coverage [1]. But I like this idea the best, and maybe I
> did something wrong. Attached is the patch I had (on top of
> fc115d0f9fc6), but I don't know if it still applies.
Thanks for the reference to that discussion and your patch.
As I see the issue with that patch is that quickdie() is not the only
SIGQUIT handler. When a backend is interrupted with SIGQUIT, it's
exiting in SignalHandlerForCrashExit().
In fact if I only add __gcov_flush() in SignalHandlerForCrashExit(), it
raises test coverage for `make check -C src/test/recovery/` from
106198 lines/6319 functions
to
106420 lines/6328 functions
It's not yet clear to me what happens when __gcov_flush() called inside
__gcov_flush().
The test coverage changes to:
108432 lines/5417 functions
(number of function calls decreased)
And for example in coverage/src/backend/utils/cache/catcache.c.gcov.html
I see
147 8 : int2eqfast(Datum a, Datum b)
...
153 0 : int2hashfast(Datum datum)
but without __gcov_flush in quickdie() we have:
147 78038 : int2eqfast(Datum a, Datum b)
...
153 255470 : int2hashfast(Datum datum)
So it needs more investigation.
But I can confirm that calling __gcov_flush() in
SignalHandlerForCrashExit() really improves a code coverage report.
I tried to develop a test to elevate a coverage for gist:
https://coverage.postgresql.org/src/backend/access/gist/gistxlog.c.gcov.html
(Please look at the attached test if it could be interesting.)
and came to this issue with a coverage. I tried to play with
GCOV_PREFIX, but without luck.
Yesterday I found the more recent discussion:
https://www.postgresql.org/message-id/flat/44ecae53-9861-71b7-1d43-4658acc52519%402ndquadrant.com#d0...
(where probably the same problem came out).
Finally I've managed to get an expected coverage when I performed
$node_standby->stop() (but __gcov_flush() in SignalHandlerForCrashExit()
helps too).
Best regards,
Alexander
Attachments:021_indexes-test.patch (text/x-patch, 3.6K) downloaddiff --git a/src/test/recovery/t/021_indexes.pl b/src/test/recovery/t/021_indexes.plnew file mode 100644index 0000000000..902133d55a--- /dev/null+++ b/src/test/recovery/t/021_indexes.pl@@ -0,0 +1,102 @@+# Test testing indexes replication+use strict;+use warnings;+use PostgresNode;+use TestLib;+use Test::More tests => 1;++# Initialize master node+my $node_master = get_new_node('master');+$node_master->init(allows_streaming => 1);+$node_master->start;++# Add a table for gist index check+$node_master->safe_psql('postgres',+ qq{+create table gist_point_tbl(id int, p point);+insert into gist_point_tbl(id, p)+ select g, point(g*10, g*10) from generate_series(10000, 11000) g;+create index gist_pointidx on gist_point_tbl using gist(p);});++# Take backup+my $backup_name = 'my_backup';+$node_master->backup($backup_name);++# Create streaming standby from backup+my $node_standby = get_new_node('standby');+$node_standby->init_from_backup($node_master, $backup_name,+ has_streaming => 1);+$node_standby->start;++$node_master->safe_psql('postgres', qq{+create temp table gist_point_tbl_t(id int, p point);+create index gist_pointidx_t on gist_point_tbl_t using gist(p);+insert into gist_point_tbl_t(id, p)+ select g, point(g*10, g*10) from generate_series(1, 1000) g;+set enable_seqscan=off;+set enable_bitmapscan=off;+explain (costs off)+select p from gist_point_tbl_t where p <@ box(point(0,0), point(100, 100));+});++$node_master->safe_psql('postgres', qq{+create unlogged table gist_point_tbl_u(id int, p point);+create index gist_pointidx_u on gist_point_tbl_u using gist(p);+insert into gist_point_tbl_u(id, p)+ select g, point(g*10, g*10) from generate_series(1, 1000) g;+set enable_seqscan=off;+set enable_bitmapscan=off;+explain (costs off)+select p from gist_point_tbl_u where p <@ box(point(0,0), point(100, 100));+});++$node_master->safe_psql('postgres', qq{+insert into gist_point_tbl (id, p)+ select g, point(g*10, g*10) from generate_series(1, 1000) g;});++$node_master->safe_psql('postgres', "delete from gist_point_tbl where id < 500");++$node_master->safe_psql('postgres', qq{+create table test as (select x, box(point(x, x),point(x, x))+ from generate_series(1,4000000) as x);++create index test_idx on test using gist (box);++set enable_seqscan TO false;+set enable_bitmapscan TO false;++delete from test where box && box(point(0,0), point(100000,100000));++-- This query invokes gistkillitems()+select count(box) from test where box && box(point(0,0), point(100000,100000));+});++$node_master->safe_psql('postgres', qq{+insert into test+ select x, box(point(x, x),point(x*3, x*3))+ from generate_series(1,200000) as x;});++$node_master->safe_psql('postgres', qq{+update test set box = box where x<2000000; vacuum test;});+$node_master->safe_psql('postgres', qq{+delete from test where x<1000000; vacuum test;});+$node_master->safe_psql('postgres', qq{+insert into test+ select x, box(point(x, x),point(x, x))+ from generate_series(1,1000000) as x;});++$node_master->safe_psql('postgres', qq{+insert into gist_point_tbl (id, p)+ select g, point(g*5, g*5) from generate_series(1, 10000) g;});++$node_master->wait_for_catchup($node_standby, 'replay');++my $result = $node_standby->safe_psql('postgres', qq{+set enable_seqscan=off;+set enable_bitmapscan=off;+explain (costs off)+select p from gist_point_tbl where p <@ box(point(0,0), point(100, 100));});+ok($result =~ /^Index Only Scan using gist_pointidx/, "gist index used on a standby");++#$node_standby->stop();+#$node_master->stop();
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: exclusion@gmail.com, alvherre@2ndquadrant.com
Subject: Re: gcov coverage data not full with immediate stop
In-Reply-To: <445c1dd9-bb8a-3153-04b2-40ba97be87b1@gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox