From: Justin Pryzby <pryzby@telsasoft.com>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Andrew Dunstan <andrew@dunslane.net>
Cc: Anastasia Lubennikova <lubennikovaav@gmail.com>
Cc: Andres Freund <andres@anarazel.de>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: A test for replay of regression tests
Date: Wed, 3 Aug 2022 20:13:00 -0500
Message-ID: <20220804011300.GJ19644@telsasoft.com> (raw)
In-Reply-To: <CA+hUKGKDsx=kpVqj_s5QcYgGtrNqci75e5cp-C4b8z52VoEaPQ@mail.gmail.com>
References: <CA+hUKGKwH-mDeReafALoR5Ui5MVM_HKjyskbkMh+U=6oKk+E3w@mail.gmail.com>
<CA+hUKG+stUD8qE6s5uJPpFaTW-5ZADp+Bq2Bhiu+-00j3xz7gg@mail.gmail.com>
<CA+hUKGK=FdtCWjx5eqJ8DVtOoAT2W9os+DYNT3Z9MjX-iaa8zA@mail.gmail.com>
<1f2ea615-2c26-633d-5018-19e4315f84d3@dunslane.net>
<e05b213c-1257-84d4-f079-5c4d8c79e3ad@dunslane.net>
<CA+hUKG+nk_WCqi_y+9TentqwXvy=1dW0DgBB-8yTtgdTTLtP0Q@mail.gmail.com>
<3e9d1044-ddb2-91a4-917b-8cab3ea93b69@dunslane.net>
<CA+hUKGLnJoeSqtp0tNyFj2GUYaNgbCQ3KTZqCfFH7QY2+Ojcdg@mail.gmail.com>
<20220803153050.GC19644@telsasoft.com>
<CA+hUKGKDsx=kpVqj_s5QcYgGtrNqci75e5cp-C4b8z52VoEaPQ@mail.gmail.com>
On Thu, Aug 04, 2022 at 09:24:24AM +1200, Thomas Munro wrote:
> On Thu, Aug 4, 2022 at 3:30 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
> > On Thu, Dec 09, 2021 at 12:10:23PM +1300, Thomas Munro wrote:
> > > This adds 2 whole minutes to the recovery check, when running with the
> > > Windows serial-only scripts on Cirrus CI (using Andres's CI patches).
> > > For Linux it adds ~20 seconds to the total of -j8 check-world.
> > > Hopefully that's time well spent, because it adds test coverage for
> > > all the redo routines, and hopefully soon we won't have to run 'em in
> > > series on Windows.
> >
> > Should 027-stream-regress be renamed to something that starts earlier ?
> > Off-list earlier this year, Andres referred to 000.
See also:
https://www.postgresql.org/message-id/20220213220709.vjz5rziuhfdpqxrg@alap3.anarazel.de> Do you have any data on improved times from doing that?
>
> I have wondered about moving it into 001_stream_rep.pl.
The immediate motive for raising the question is due to working on your cygwin
patch (where I've set PROVE_FLAGS=-j3). The last invocation I have opened ends
like:
[20:46:47.577] [13:46:47] t/026_overwrite_contrecord.pl ........ ok 10264 ms ( 0.02 usr 0.02 sys + 11.25 cusr 35.95 csys = 47.24 CPU)
[20:47:08.087] [13:47:08] t/028_pitr_timelines.pl .............. ok 13153 ms ( 0.00 usr 0.00 sys + 4.03 cusr 14.79 csys = 18.82 CPU)
[20:47:08.999] [13:47:09] t/029_stats_restart.pl ............... ok 12631 ms ( 0.00 usr 0.02 sys + 7.40 cusr 23.30 csys = 30.71 CPU)
[20:47:34.353] [13:47:34] t/031_recovery_conflict.pl ........... ok 11337 ms ( 0.00 usr 0.00 sys + 3.84 cusr 11.82 csys = 15.66 CPU)
[20:47:35.070] [13:47:35] t/030_stats_cleanup_replica.pl ....... ok 14054 ms ( 0.02 usr 0.00 sys + 7.64 cusr 25.02 csys = 32.68 CPU)
[20:48:04.887] [13:48:04] t/032_relfilenode_reuse.pl ........... ok 12755 ms ( 0.00 usr 0.00 sys + 3.36 cusr 11.57 csys = 14.93 CPU)
[20:48:42.055] [13:48:42] t/033_replay_tsp_drops.pl ............ ok 43529 ms ( 0.00 usr 0.00 sys + 12.29 cusr 41.43 csys = 53.71 CPU)
[20:50:02.770] [13:50:02] t/027_stream_regress.pl .............. ok 198408 ms ( 0.02 usr 0.06 sys + 44.92 cusr 142.42 csys = 187.42 CPU)
[20:50:02.771] [13:50:02]
[20:50:02.771] All tests successful.
[20:50:02.771] Files=33, Tests=411, 402 wallclock secs ( 0.16 usr 0.27 sys + 138.03 cusr 441.56 csys = 580.01 CPU)
If 027 had been started sooner, this test might have finished up to 78sec
earlier. If lots of tests are added in the future, maybe it won't matter, but
it seems like it does now.
As I understand, checks are usually parallelized by "make -j" and not by
"prove". In that case, starting a slow test later doesn't matter. But it'd be
better for anyone who runs tap tests manually, and (I think) for meson.
As a one-off test on localhost:
time make check -C src/test/recovery
=> 11m42,790s
time make check -C src/test/recovery PROVE_FLAGS=-j2
=> 7m56,315s
After renaming it to 001:
time make check -C src/test/recovery
=> 11m33,887s (~same)
time make check -C src/test/recovery PROVE_FLAGS=-j2
=> 6m59,969s
I don't know how it affect the buildfarm (but I think that's not optimized
primarily for speed anyway).
--
Justin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: pryzby@telsasoft.com, thomas.munro@gmail.com, andrew@dunslane.net, lubennikovaav@gmail.com, andres@anarazel.de, tgl@sss.pgh.pa.us
Subject: Re: A test for replay of regression tests
In-Reply-To: <20220804011300.GJ19644@telsasoft.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox