From: Tom Lane <tgl@sss.pgh.pa.us>
To: Andres Freund <andres@anarazel.de>
cc: Thomas Munro <thomas.munro@gmail.com>,
        pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: A test for replay of regression tests
In-reply-to: <20210423152031.unkv7mmvhw4pbjuc@alap3.anarazel.de>
References: 
 <CA+hUKGKpRWQ9SxdxxDmTBCJoR0YnFpMBe7kyzY8SUQk+Heskxg@mail.gmail.com>
 <20210423152031.unkv7mmvhw4pbjuc@alap3.anarazel.de>
Comments: In-reply-to Andres Freund <andres@anarazel.de>
	message dated "Fri, 23 Apr 2021 08:20:31 -0700"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <1356861.1619193215.1@sss.pgh.pa.us>
Date: Fri, 23 Apr 2021 11:53:35 -0400
Message-ID: <1356862.1619193215@sss.pgh.pa.us>
Archived-At: 
 <https://www.postgresql.org/message-id/1356862.1619193215%40sss.pgh.pa.us>
Precedence: bulk

Andres Freund <andres@anarazel.de> writes:
> On 2021-04-23 17:37:48 +1200, Thomas Munro wrote:
>> We have automated tests for many specific replication and recovery
>> scenarios, but nothing that tests replay of a wide range of records.

> Yay.

+1

>> Add a new TAP test under src/test/recovery that runs the regression
>> tests with wal_consistency_checking=all.

> Hm. I wonder if running with wal_consistency_checking=all doesn't also
> reduce coverage of some aspects of recovery, by increasing record sizes
> etc.

Yeah.  I found out earlier that wal_consistency_checking=all is an
absolute PIG.  Running the regression tests that way requires tens of
gigabytes of disk space, and a significant amount of time if your
disk is not very speedy.  If we put this into the buildfarm at all,
it would have to be opt-in, not run-by-default, because a lot of BF
animals simply don't have the horsepower.  I think I'd vote against
adding it to check-world, too; the cost/benefit ratio is not good
unless you are specifically working on replay functions.

And as you say, it alters the behavior enough to make it a little
questionable whether we're exercising the "normal" code paths.

The two things I'd say about this are (1) Whether to use
wal_consistency_checking, and with what value, needs to be
easily adjustable. (2) People will want to run other test suites
than the core regression tests, eg contrib modules.

			regards, tom lane