Received: from malur.postgresql.org ([2a02:16a8:dc51::56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1fxjQ5-0007Xh-7u for pgsql-hackers@arkaria.postgresql.org; Thu, 06 Sep 2018 01:49:18 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1fxjPx-0005Iy-Fa for pgsql-hackers@arkaria.postgresql.org; Thu, 06 Sep 2018 01:49:09 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1fxjPv-0005In-6G for pgsql-hackers@lists.postgresql.org; Thu, 06 Sep 2018 01:49:09 +0000 Received: from out1-smtp.messagingengine.com ([66.111.4.25]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1fxjPi-0005Ot-NX for pgsql-hackers@postgresql.org; Thu, 06 Sep 2018 01:49:06 +0000 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 9C1A821FDE; Wed, 5 Sep 2018 21:48:52 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Wed, 05 Sep 2018 21:48:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=paquier.xyz; h= cc:content-type:date:from:message-id:mime-version:subject:to :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=M2uttJfAKpJsVKED4 lsqz3CEIBoKUcx+1iohLOeYo5U=; b=N/ldfOjvHIh+C74dntMtB02TOcUiaRmfv UCW10RGbLvgPAYmOo0eioE9GxktvQMo9tfkTLijhOrHmPbzFlKbrdb0xETTuHZMq hB/Yb0m5LGd2iIWr8UvpoK6f0cfto/nYPLlKHsmyDJ07LL9SdflmM+djHZj0q41Z l3HbB9FrI5QZGPK/KmbBHRUHkuiqsQ4fAlkFvmK0MAYINBqdmT+4MppoALObX31b wk2/DdwIsscFWUdlX6qb0f81bTTA/Q1DPkqbDJVvZCuGfmqIE53oI0qgnUgnD+YK C7ISD9A2g+5q385rjufAYAqQ4vEm3m5m2KDTbWFBqWSBPVjVSc7tA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:message-id :mime-version:subject:to:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; bh=M2uttJfAKpJsVKED4lsqz3CEIBoKUcx+1iohLOeYo5U=; b=dCL4n0sd osFThOF74ynloxgHYF2bOXKePiCNDlVnlsindLR1YRQo7Tlo7m5I2liQGU2ZQ/Ds u8Fp/UsDRRobyOZDeftULCHHhuwrj8YcgfSlF83Mg5+7vK9R0fXszO70SBerTdXa tjrAjLHsrm0cM0H2k10585ARjr7X8n1Ce4bbskwcT/AsCXxJvp/9W9HB7cgXofCp IGDncRLjNItWC3Vl2ODwCAgSkKP1MhfbKF6hxBpYmzuUl6Zb7TCj+ysK2IIQO1+0 icOj+8e6oc5GZKlof14uJNjFZEBRKa+fJP2eFIO0oiqcE5kRn7lY7jW3VrIwTaD/ OBLKsaf4OoD7GA== X-ME-Proxy: X-ME-Sender: Received: from paquier.xyz (unknown [50.233.106.125]) by mail.messagingengine.com (Postfix) with ESMTPA id 7458F1029C; Wed, 5 Sep 2018 21:48:51 -0400 (EDT) Date: Wed, 5 Sep 2018 18:48:49 -0700 From: Michael Paquier To: Postgres hackers Cc: Alvaro Herrera , Tom Lane Subject: Add extension options to control TAP and isolation tests Message-ID: <20180906014849.GG2726@paquier.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="pfTAc8Cvt8L6I27a" Content-Disposition: inline List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --pfTAc8Cvt8L6I27a Content-Type: multipart/mixed; boundary="nhYGnrYv1PEJ5gA2" Content-Disposition: inline --nhYGnrYv1PEJ5gA2 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi all, On a recent thread of pgsql-committers has been discussed the fact that we lacked a bit of infrastructure to allow extensions to control isolation and TAP tests: https://www.postgresql.org/message-id/20180905174527.GA2726@paquier.xyz Attached is a patch which is the result of the previous thread, where a couple of variables are added for extension authors: - ISOLATION, similar to REGRESS for pg_regress, which lists isolation tests. - ISOLATION_OPTS, which can be used to pass an option set to pg_isolation_regress. - TAP_TESTS, a switch to enable running TAP tests. This results in a bit of cleanup in the Makefile of modules we ship with upstream: - modules/brin - modules/commit_ts - modules/test_pg_dump - contrib/bloom, where the TAP tests were actually not running. - contrib/oid2name - contrib/test_decoding, which keeps the same feature set, and is heavily cleaned up (it missed for example the use of the existing NO_INSTALLCHECK). - contrib/vacuumlo Thoughts? -- Michael --nhYGnrYv1PEJ5gA2 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="pgxs-isolation-tap.patch" Content-Transfer-Encoding: quoted-printable diff --git a/contrib/bloom/Makefile b/contrib/bloom/Makefile index 13bd397b70..da9553a2d0 100644 --- a/contrib/bloom/Makefile +++ b/contrib/bloom/Makefile @@ -8,6 +8,7 @@ DATA =3D bloom--1.0.sql PGFILEDESC =3D "bloom access method - signature file based index" =20 REGRESS =3D bloom +TAP_TESTS =3D 1 =20 ifdef USE_PGXS PG_CONFIG =3D pg_config @@ -19,6 +20,3 @@ top_builddir =3D ../.. include $(top_builddir)/src/Makefile.global include $(top_srcdir)/contrib/contrib-global.mk endif - -wal-check: temp-install - $(prove_check) diff --git a/contrib/oid2name/Makefile b/contrib/oid2name/Makefile index 908e078714..361a80a7a1 100644 --- a/contrib/oid2name/Makefile +++ b/contrib/oid2name/Makefile @@ -6,11 +6,11 @@ PGAPPICON =3D win32 PROGRAM =3D oid2name OBJS =3D oid2name.o $(WIN32RES) =20 +TAP_TESTS =3D 1 + PG_CPPFLAGS =3D -I$(libpq_srcdir) PG_LIBS_INTERNAL =3D $(libpq_pgport) =20 -EXTRA_CLEAN =3D tmp_check - ifdef USE_PGXS PG_CONFIG =3D pg_config PGXS :=3D $(shell $(PG_CONFIG) --pgxs) @@ -21,9 +21,3 @@ top_builddir =3D ../.. include $(top_builddir)/src/Makefile.global include $(top_srcdir)/contrib/contrib-global.mk endif - -check: - $(prove_check) - -installcheck: - $(prove_installcheck) diff --git a/contrib/test_decoding/Makefile b/contrib/test_decoding/Makefile index afcab930f7..8cd83a763f 100644 --- a/contrib/test_decoding/Makefile +++ b/contrib/test_decoding/Makefile @@ -3,9 +3,20 @@ MODULES =3D test_decoding PGFILEDESC =3D "test_decoding - example of a logical decoding output plugi= n" =20 -# Note: because we don't tell the Makefile there are any regression tests, -# we have to clean those result files explicitly -EXTRA_CLEAN =3D $(pg_regress_clean_files) +EXTRA_INSTALL=3Dcontrib/test_decoding + +REGRESS =3D ddl xact rewrite toast permissions decoding_in_xact \ + decoding_into_rel binary prepared replorigin time messages \ + spill slot truncate +ISOLATION =3D mxact delayed_startup ondisk_startup concurrent_ddl_dml \ + oldest_xmin snapshot_transfer + +REGRESS_OPTS =3D --temp-config $(top_srcdir)/contrib/test_decoding/logical= =2Econf +ISOLATION_OPTS =3D --temp-config $(top_srcdir)/contrib/test_decoding/logic= al.conf + +# Disabled because these tests require "wal_level=3Dlogical", which +# typical installcheck users do not have (e.g. buildfarm clients). +NO_INSTALLCHECK =3D 1 =20 ifdef USE_PGXS PG_CONFIG =3D pg_config @@ -18,52 +29,8 @@ include $(top_builddir)/src/Makefile.global include $(top_srcdir)/contrib/contrib-global.mk endif =20 -# Disabled because these tests require "wal_level=3Dlogical", which -# typical installcheck users do not have (e.g. buildfarm clients). -installcheck:; - # But it can nonetheless be very helpful to run tests on preexisting # installation, allow to do so, but only if requested explicitly. -installcheck-force: regresscheck-install-force isolationcheck-install-force - -check: regresscheck isolationcheck - -submake-regress: - $(MAKE) -C $(top_builddir)/src/test/regress all - -submake-isolation: - $(MAKE) -C $(top_builddir)/src/test/isolation all - -submake-test_decoding: - $(MAKE) -C $(top_builddir)/contrib/test_decoding - -REGRESSCHECKS=3Dddl xact rewrite toast permissions decoding_in_xact \ - decoding_into_rel binary prepared replorigin time messages \ - spill slot truncate - -regresscheck: | submake-regress submake-test_decoding temp-install - $(pg_regress_check) \ - --temp-config $(top_srcdir)/contrib/test_decoding/logical.conf \ - $(REGRESSCHECKS) - -regresscheck-install-force: | submake-regress submake-test_decoding temp-i= nstall - $(pg_regress_installcheck) \ - $(REGRESSCHECKS) - -ISOLATIONCHECKS=3Dmxact delayed_startup ondisk_startup concurrent_ddl_dml \ - oldest_xmin snapshot_transfer - -isolationcheck: | submake-isolation submake-test_decoding temp-install - $(pg_isolation_regress_check) \ - --temp-config $(top_srcdir)/contrib/test_decoding/logical.conf \ - $(ISOLATIONCHECKS) - -isolationcheck-install-force: all | submake-isolation submake-test_decodin= g temp-install - $(pg_isolation_regress_installcheck) \ - $(ISOLATIONCHECKS) - -.PHONY: submake-test_decoding submake-regress check \ - regresscheck regresscheck-install-force \ - isolationcheck isolationcheck-install-force - -temp-install: EXTRA_INSTALL=3Dcontrib/test_decoding +installcheck-force: + $(pg_regress_installcheck) $(REGRESS) + $(pg_isolation_regress_installcheck) $(ISOLATION) diff --git a/contrib/vacuumlo/Makefile b/contrib/vacuumlo/Makefile index 5de506151e..3efcb46735 100644 --- a/contrib/vacuumlo/Makefile +++ b/contrib/vacuumlo/Makefile @@ -6,11 +6,11 @@ PGAPPICON =3D win32 PROGRAM =3D vacuumlo OBJS =3D vacuumlo.o $(WIN32RES) =20 +TAP_TESTS =3D 1 + PG_CPPFLAGS =3D -I$(libpq_srcdir) PG_LIBS_INTERNAL =3D $(libpq_pgport) =20 -EXTRA_CLEAN =3D tmp_check - ifdef USE_PGXS PG_CONFIG =3D pg_config PGXS :=3D $(shell $(PG_CONFIG) --pgxs) @@ -21,9 +21,3 @@ top_builddir =3D ../.. include $(top_builddir)/src/Makefile.global include $(top_srcdir)/contrib/contrib-global.mk endif - -check: - $(prove_check) - -installcheck: - $(prove_installcheck) diff --git a/doc/src/sgml/extend.sgml b/doc/src/sgml/extend.sgml index 1b1adae1a6..701e8593a9 100644 --- a/doc/src/sgml/extend.sgml +++ b/doc/src/sgml/extend.sgml @@ -1293,6 +1293,34 @@ include $(PGXS) =20 + + ISOLATION + + + list of isolation test cases + + + + + + ISOLATION_OPTS + + + additional switches to pass to + pg_isolation_regress + + + + + + TAP_TESTS + + + switch defining if TAP tests need to be run + + + + NO_INSTALLCHECK diff --git a/src/makefiles/pgxs.mk b/src/makefiles/pgxs.mk index fe7d899c5c..06cda28829 100644 --- a/src/makefiles/pgxs.mk +++ b/src/makefiles/pgxs.mk @@ -46,6 +46,9 @@ # HEADERS_built_$(MODULE) -- as above but built first # REGRESS -- list of regression test cases (without suffix) # REGRESS_OPTS -- additional switches to pass to pg_regress +# TAP_TESTS -- switch to enable TAP tests +# ISOLATION -- list of isolation test cases +# ISOLATION_OPTS -- additional switches to pass to pg_isolation_regress # NO_INSTALLCHECK -- don't define an installcheck target, useful e.g. if # tests require special configuration, or don't use pg_regress # EXTRA_CLEAN -- extra files to remove in 'make clean' @@ -336,6 +339,12 @@ ifeq ($(PORTNAME), win) rm -f regress.def endif endif # REGRESS +ifdef TAP_TESTS + rm -rf tmp_check/ +endif +ifdef ISOLATION + rm -rf output_iso/ tmp_check_iso/ +endif =20 ifdef MODULE_big clean: clean-lib @@ -370,28 +379,51 @@ $(test_files_build): $(abs_builddir)/%: $(srcdir)/% $(MKDIR_P) $(dir $@) ln -s $< $@ endif # VPATH +endif # REGRESS =20 .PHONY: submake -submake: ifndef PGXS +submake: +ifdef REGRESS $(MAKE) -C $(top_builddir)/src/test/regress pg_regress$(X) endif +ifdef ISOLATION + $(MAKE) -C $(top_builddir)/src/test/isolation all +endif +endif # PGXS =20 -# against installed postmaster +# Standard rules to run regression tests including multiple test suites. +# Runs against an installed postmaster ifndef NO_INSTALLCHECK installcheck: submake $(REGRESS_PREP) +ifdef REGRESS $(pg_regress_installcheck) $(REGRESS_OPTS) $(REGRESS) endif +ifdef ISOLATION + $(pg_isolation_regress_installcheck) $(ISOLATION_OPTS) $(ISOLATION) +endif +ifdef TAP_TESTS + $(prove_installcheck) +endif +endif # NO_INSTALLCHECK =20 +# Runs independently of any installation ifdef PGXS check: @echo '"$(MAKE) check" is not supported.' @echo 'Do "$(MAKE) install", then "$(MAKE) installcheck" instead.' else check: submake $(REGRESS_PREP) +ifdef REGRESS $(pg_regress_check) $(REGRESS_OPTS) $(REGRESS) endif -endif # REGRESS +ifdef ISOLATION + $(pg_isolation_regress_check) $(ISOLATION_OPTS) $(ISOLATION) +endif +ifdef TAP_TESTS + $(prove_check) +endif +endif # PGXS =20 ifndef NO_TEMP_INSTALL temp-install: EXTRA_INSTALL+=3D$(subdir) diff --git a/src/test/modules/brin/.gitignore b/src/test/modules/brin/.giti= gnore index 62bbe8f6b1..44f600cb6c 100644 --- a/src/test/modules/brin/.gitignore +++ b/src/test/modules/brin/.gitignore @@ -1,3 +1,3 @@ # Generated subdirectories -/isolation_output/ +/output_iso/ /tmp_check/ diff --git a/src/test/modules/brin/Makefile b/src/test/modules/brin/Makefile index 566655cd61..c871593906 100644 --- a/src/test/modules/brin/Makefile +++ b/src/test/modules/brin/Makefile @@ -1,12 +1,9 @@ # src/test/modules/brin/Makefile =20 -# Note: because we don't tell the Makefile there are any regression tests, -# we have to clean those result files explicitly -EXTRA_CLEAN =3D $(pg_regress_clean_files) ./isolation_output +EXTRA_INSTALL =3D contrib/pageinspect =20 -EXTRA_INSTALL=3Dcontrib/pageinspect - -ISOLATIONCHECKS=3Dsummarization-and-inprogress-insertion +ISOLATION =3D summarization-and-inprogress-insertion +TAP_TESTS =3D 1 =20 ifdef USE_PGXS PG_CONFIG =3D pg_config @@ -18,19 +15,3 @@ top_builddir =3D ../../../.. include $(top_builddir)/src/Makefile.global include $(top_srcdir)/contrib/contrib-global.mk endif - -check: isolation-check prove-check - -isolation-check: | submake-isolation temp-install - $(MKDIR_P) isolation_output - $(pg_isolation_regress_check) \ - --outputdir=3D./isolation_output \ - $(ISOLATIONCHECKS) - -prove-check: | temp-install - $(prove_check) - -.PHONY: check isolation-check prove-check - -submake-isolation: - $(MAKE) -C $(top_builddir)/src/test/isolation all diff --git a/src/test/modules/commit_ts/Makefile b/src/test/modules/commit_= ts/Makefile index 6d4f3be358..7a24bb3c6d 100644 --- a/src/test/modules/commit_ts/Makefile +++ b/src/test/modules/commit_ts/Makefile @@ -2,6 +2,7 @@ =20 REGRESS =3D commit_timestamp REGRESS_OPTS =3D --temp-config=3D$(top_srcdir)/src/test/modules/commit_ts/= commit_ts.conf +TAP_TESTS =3D 1 =20 ifdef USE_PGXS PG_CONFIG =3D pg_config @@ -13,8 +14,3 @@ top_builddir =3D ../../../.. include $(top_builddir)/src/Makefile.global include $(top_srcdir)/contrib/contrib-global.mk endif - -check: prove-check - -prove-check: | temp-install - $(prove_check) diff --git a/src/test/modules/test_pg_dump/Makefile b/src/test/modules/test= _pg_dump/Makefile index c64b353707..6123b994f6 100644 --- a/src/test/modules/test_pg_dump/Makefile +++ b/src/test/modules/test_pg_dump/Makefile @@ -7,6 +7,7 @@ EXTENSION =3D test_pg_dump DATA =3D test_pg_dump--1.0.sql =20 REGRESS =3D test_pg_dump +TAP_TESTS =3D 1 =20 ifdef USE_PGXS PG_CONFIG =3D pg_config @@ -18,8 +19,3 @@ top_builddir =3D ../../../.. include $(top_builddir)/src/Makefile.global include $(top_srcdir)/contrib/contrib-global.mk endif - -check: prove-check - -prove-check: | temp-install - $(prove_check) --nhYGnrYv1PEJ5gA2-- --pfTAc8Cvt8L6I27a Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAluQh4EACgkQnvQgOdby QH02hg/+MGlVy866dpKQyyBieC0gV5NZrG4SkQuRzCi0a8k/z82C/QQt5e+qWUOW crfEgf9emA1ZWvQ5phx/FE5n0I7Gfixz8UOJvx7NY5EzNtKy8AgcetX7cHq7xQi4 WKDwsG3t4TTTcapvisHa7pOgT3PG+xnbY6y1J9C6w9Ol7RXhpgRfTy+vIG4q8Mwb XNMSOf4+C1rI9Yhcphg2iHHAL/O7lOhjeDThDyX4gd/wmMABVUM4xwubF+D4VtOB jv6Z3bfxHq1IejscIcNSfvwCKPonNCIQXNJ+gF08eZho10E00/GjXLEdJtFEDJBl +NigbO21c5PnVbIfeGPTW2TzZQImBIy5G5ZDHD+2X4on6G/hfhTf/BYNgj7BQr8P /iCnkrTFZeJ4usj2ZILzUAjRAQuwiO9/vbqmsLoW8X7a+hewA290iYK8gccmQQYT FSwknCDHiVoL68PC/FdQyhAbnJrYfWiGA67Bs+S4gF4loSLt34/oPjhKLJCfj4ZG GM1GvGGzRuMA/u71PUsWXja2uCkN7CUzKae6+xpda0i7xTv4/cMfKir7So2RTlQC m5vXUk3ubRaTVUg+3Tiw+nST6jf2RFskha9NDGoDTwEUK5nVI2o3yYpQC8Y6Dywe lrnBask0iB9t33F/Ex1TpIy76QMlXRneidDsLFDZfk5/maK3hC4= =R3ee -----END PGP SIGNATURE----- --pfTAc8Cvt8L6I27a--