Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1gSyVo-00019n-7E for pgsql-hackers@arkaria.postgresql.org; Sat, 01 Dec 2018 06:12:20 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1gSyVm-0004xH-FA for pgsql-hackers@arkaria.postgresql.org; Sat, 01 Dec 2018 06:12:18 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1gSyVl-0004xA-TI for pgsql-hackers@lists.postgresql.org; Sat, 01 Dec 2018 06:12:18 +0000 Received: from mail-lj1-x244.google.com ([2a00:1450:4864:20::244]) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1gSyVd-00061s-VE for pgsql-hackers@postgresql.org; Sat, 01 Dec 2018 06:12:16 +0000 Received: by mail-lj1-x244.google.com with SMTP id g11-v6so6896127ljk.3 for ; Fri, 30 Nov 2018 22:12:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=a5uHhwEQCK8qW4EyoU2oC3ym4I4qr4Uhmbu+RjSjdJU=; b=qHJnKndLoPjDdwisBalQO/UvpCMf1iwszms9Yl0NKxPmg5IS9I9GAMzjp3w1C9TzeJ pPOm/Cl+tuexSxEHjbXKCTLM/dSP3H4pgzk/Ok3u6QGPU2rwdydZC9VgQ9PEzpgZy1aX mnU3qIovpEONdnPs+mQqA8r/g1zOBn05E2hnTJ9P6iFWBEQTUzVC9DvuAzh++e6p3yGK +ChAgt8/naW2C7raaRyZwcg+4NKUBHNxV9VwPSg4bYOtfai2ussDIMSlQrMOZ56FQQ2x Zj8qR7OgqIhQgenuxb4Dw+jBVx4OMTt8PcVlElQhP2uVXKzfz6YflRKKMEYXjxvVg769 ssZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=a5uHhwEQCK8qW4EyoU2oC3ym4I4qr4Uhmbu+RjSjdJU=; b=pVkFHDX0zgucprPv+8w/d9C2Xy9YCUSujxzdwqQ6LSy9t2q04w6OxU4lfvdVuLiJi0 vlyubnMvoy92QOSKlRYhtXSC1dyiDZL5r6bNB5AgdSFYEbqA+JIYx4tukUt3ogijasIP 8xw+5X9qFSLxY73FK4ingJrlczsqPfJcKtsGlQrmzrlA0UxmcGcMZg59eLqdonxahtOx ClUUkbarPrID223DL49ts5NOfdSA2ga3ArFdUTYGAj0PRrWIFvBcSwLffo++4u99UNta p3JaiZS2K7fm/jjNORPMVi9BMxqUCzSbYinKu2hGJfjwjfQbuASkbUNd56lzldS7BpPu +qsw== X-Gm-Message-State: AA+aEWZO6SMccwT5voti7FjZZVMKFCH1TdFH9fxmhMiy+ZW0xbNGHBp/ AJgHglp7qw9amID1sVq38wKzp3QE X-Google-Smtp-Source: AFSGD/UNrWHmXZ1XnjNfKkTGecoXZwlFOdEjNiSnyZ9/oHRG0TUILsovnIFRX6TAggeK2TOH1xr8mg== X-Received: by 2002:a2e:568b:: with SMTP id k11-v6mr5589208lje.105.1543644727791; Fri, 30 Nov 2018 22:12:07 -0800 (PST) Received: from [1.0.0.7] ([178.155.4.177]) by smtp.gmail.com with ESMTPSA id x11sm1287552lfd.81.2018.11.30.22.12.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 30 Nov 2018 22:12:06 -0800 (PST) Subject: Re: make installcheck-world in a clean environment To: Dmitry Dolgov <9erthalion6@gmail.com> Cc: Tom Lane , Michael Paquier , Robert Haas , PostgreSQL-development References: <24834.1525712835@sss.pgh.pa.us> <20180912072049.GL25160@paquier.xyz> <171724bb-0436-3665-6738-cbd42bdf9783@gmail.com> <8351.1542331145@sss.pgh.pa.us> <6f7a6818-78d8-48ee-1a0f-e4ad30615af3@gmail.com> From: Alexander Lakhin Message-ID: <502e87c5-da7d-fc75-6597-94dffc387fd9@gmail.com> Date: Sat, 1 Dec 2018 09:12:05 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/mixed; boundary="------------0B89BFA128936D6F40DA4C61" Content-Language: ru-RU List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk This is a multi-part message in MIME format. --------------0B89BFA128936D6F40DA4C61 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Hello, 30.11.2018 23:59, Dmitry Dolgov wrote: >> On Sun, Nov 18, 2018 at 8:31 AM Alexander Lakhin wrote: >> >> I've modified the patch to use the installed version of pg_regress. It >> simplifies a lot. The main idea of the change is to not build pg_regress. > Hi, > > I've noticed that for this patch cfbot show strange error > > USE_INSTALLED_ASSETS=1 make all > ... > make[1]: Leaving directory > `/home/travis/build/postgresql-cfbot/postgresql/src/test/regress' > make: *** [installcheck-parallel] Error 2 > make: Leaving directory `/home/travis/build/postgresql-cfbot/postgresql' > > The same error I've got when tried to run it on my laptop, is it something > expected? Thank you for bringing this to my attention. I've fixed the patch. (pg_upgrade/test.sh passes custom DESTDIR to `make` but the same DESTDIR is needed for `make install`) Best regards, Alexander --------------0B89BFA128936D6F40DA4C61 Content-Type: text/x-patch; name="make-installcheck-v4.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="make-installcheck-v4.patch" diff --git a/contrib/test_decoding/Makefile b/contrib/test_decoding/Makefile index afcab930f7..fd0e0eedb0 100644 --- a/contrib/test_decoding/Makefile +++ b/contrib/test_decoding/Makefile @@ -46,7 +46,7 @@ regresscheck: | submake-regress submake-test_decoding temp-install --temp-config $(top_srcdir)/contrib/test_decoding/logical.conf \ $(REGRESSCHECKS) -regresscheck-install-force: | submake-regress submake-test_decoding temp-install +regresscheck-install-force: | temp-install $(pg_regress_installcheck) \ $(REGRESSCHECKS) @@ -58,7 +58,7 @@ isolationcheck: | submake-isolation submake-test_decoding temp-install --temp-config $(top_srcdir)/contrib/test_decoding/logical.conf \ $(ISOLATIONCHECKS) -isolationcheck-install-force: all | submake-isolation submake-test_decoding temp-install +isolationcheck-install-force: | submake-isolation temp-install $(pg_isolation_regress_installcheck) \ $(ISOLATIONCHECKS) diff --git a/doc/src/sgml/regress.sgml b/doc/src/sgml/regress.sgml index 673a8c2164..9444dee351 100644 --- a/doc/src/sgml/regress.sgml +++ b/doc/src/sgml/regress.sgml @@ -111,7 +111,12 @@ make installcheck-parallel default port number, unless directed otherwise by PGHOST and PGPORT environment variables. The tests will be run in a database named regression; any existing database by this name - will be dropped. + will be dropped. The tests in the installcheck mode will use the executables, + libraries, and headers, which are already present in the installation tree, + not their copies in the source tree (if any). This mode is suitable for + testing distribution packages of PostgreSQL installed on user systems as + it allows to check the installation in its entirety without rebuilding + any parts of it. diff --git a/src/Makefile.global.in b/src/Makefile.global.in index 956fd274cd..3f5b779823 100644 --- a/src/Makefile.global.in +++ b/src/Makefile.global.in @@ -245,14 +245,16 @@ PG_SYSROOT = @PG_SYSROOT@ override CPPFLAGS := $(ICU_CFLAGS) $(CPPFLAGS) -ifdef PGXS +# For PGXS and in the installcheck mode (when we use the installed assets) +# we need to target already installed headers and libraries +ifneq (,$(PGXS)$(USE_INSTALLED_ASSETS)) override CPPFLAGS := -I$(includedir_server) -I$(includedir_internal) $(CPPFLAGS) -else # not PGXS +else # not PGXS and not USE_INSTALLED_ASSETS override CPPFLAGS := -I$(top_srcdir)/src/include $(CPPFLAGS) ifdef VPATH override CPPFLAGS := -I$(top_builddir)/src/include $(CPPFLAGS) endif -endif # not PGXS +endif # not PGXS and USE_INSTALLED_ASSETS CC = @CC@ GCC = @GCC@ @@ -306,7 +308,7 @@ with_gnu_ld = @with_gnu_ld@ # "LDFLAGS := something" anywhere, ditto for LDFLAGS_INTERNAL. # These initial assignments must be "=" type, and elsewhere we must only do # "LDFLAGS += something" or "LDFLAGS_INTERNAL += something". -ifdef PGXS +ifneq (,$(PGXS)$(USE_INSTALLED_ASSETS)) LDFLAGS_INTERNAL = -L$(libdir) else LDFLAGS_INTERNAL = -L$(top_builddir)/src/port -L$(top_builddir)/src/common @@ -379,6 +381,9 @@ endif # install tree just once in any recursive "make check". The additional test # on abs_top_builddir prevents doing anything foolish to the root directory. +# In the installcheck mode we should use what is already installed +# in the DESTDIR (namely, pg_regress). + check: temp-install .PHONY: temp-install @@ -421,7 +426,7 @@ ifeq ($(enable_tap_tests),yes) define prove_installcheck rm -rf '$(CURDIR)'/tmp_check $(MKDIR_P) '$(CURDIR)'/tmp_check -cd $(srcdir) && TESTDIR='$(CURDIR)' PATH="$(bindir):$$PATH" PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl) +cd $(srcdir) && TESTDIR='$(CURDIR)' PATH="$(bindir):$$PATH" PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' PG_REGRESS='$(DESTDIR)$(pgxsdir)/src/test/regress/pg_regress' $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl) endef define prove_check @@ -564,6 +569,8 @@ endif # # Commonly used submake targets +# build these libraries only when we are not going to use the installed ones +ifndef USE_INSTALLED_ASSETS submake-libpq: | submake-generated-headers $(MAKE) -C $(libpq_builddir) all @@ -575,6 +582,7 @@ submake-libpgfeutils: | submake-generated-headers $(MAKE) -C $(top_builddir)/src/port all $(MAKE) -C $(top_builddir)/src/common all $(MAKE) -C $(top_builddir)/src/fe_utils all +endif .PHONY: submake-libpq submake-libpgport submake-libpgfeutils @@ -617,7 +625,7 @@ pg_regress_check = \ $(TEMP_CONF) \ $(pg_regress_locale_flags) $(EXTRA_REGRESS_OPTS) pg_regress_installcheck = \ - $(top_builddir)/src/test/regress/pg_regress \ + '$(DESTDIR)$(pgxsdir)/src/test/regress/pg_regress' \ --inputdir=$(srcdir) \ --bindir='$(bindir)' \ $(pg_regress_locale_flags) $(EXTRA_REGRESS_OPTS) diff --git a/src/bin/pg_upgrade/test.sh b/src/bin/pg_upgrade/test.sh index 1d3272dabf..0ec38ef929 100644 --- a/src/bin/pg_upgrade/test.sh +++ b/src/bin/pg_upgrade/test.sh @@ -169,7 +169,7 @@ createdb "$dbname1" || createdb_status=$? createdb "$dbname2" || createdb_status=$? createdb "$dbname3" || createdb_status=$? -if "$MAKE" -C "$oldsrc" installcheck-parallel; then +if "$MAKE" -C "$oldsrc" installcheck-parallel DESTDIR="$temp_install"; then oldpgversion=`psql -X -A -t -d regression -c "SHOW server_version_num"` # before dumping, get rid of objects not existing in later versions diff --git a/src/interfaces/ecpg/test/Makefile b/src/interfaces/ecpg/test/Makefile index c761a4dcb0..0b12d1b8ad 100644 --- a/src/interfaces/ecpg/test/Makefile +++ b/src/interfaces/ecpg/test/Makefile @@ -86,7 +86,11 @@ check: all checktcp: all | temp-install $(with_temp_install) ./pg_regress $(REGRESS_OPTS) --temp-instance=./tmp_check $(TEMP_CONF) --bindir= $(pg_regress_locale_flags) $(THREAD) --schedule=$(srcdir)/ecpg_schedule --host=localhost sql/twophase connect/test1 -installcheck: all +.PHONY: all-installcheck +all-installcheck: + USE_INSTALLED_ASSETS=1 $(MAKE) all + +installcheck: all-installcheck ./pg_regress $(REGRESS_OPTS) --bindir='$(bindir)' $(pg_regress_locale_flags) $(THREAD) --schedule=$(srcdir)/ecpg_schedule # Versions of the check tests that include the twophase commit test. diff --git a/src/interfaces/ecpg/test/Makefile.regress b/src/interfaces/ecpg/test/Makefile.regress index 4da1bb8a03..99fb6fdcc7 100644 --- a/src/interfaces/ecpg/test/Makefile.regress +++ b/src/interfaces/ecpg/test/Makefile.regress @@ -1,19 +1,29 @@ # This file is included into the Makefiles of subdirectories of ecpg/test/, # so the file references have one more level of .. than you might expect. +ifdef USE_INSTALLED_ASSETS +# Standard way to invoke the installed ecpg preprocessor +ECPG = '$(DESTDIR)$(bindir)/ecpg' --regression -I'$(DESTDIR)$(includedir)' \ + -I'$(DESTDIR)$(pkgincludedir)/informix/esql' +# Use installed headers and libs +override CPPFLAGS := -I'$(DESTDIR)$(includedir)' \ + -I'$(DESTDIR)$(pkgincludedir)/informix/esql' $(CPPFLAGS) +LDFLAGS_INTERNAL += -lecpg -lpgtypes $(libpq) +else +# Standard way to invoke the ecpg preprocessor +ECPG = ../../preproc/ecpg --regression -I$(srcdir)/../../include -I$(srcdir) + +ECPG_DEP = ../../preproc/ecpg$(X) override CPPFLAGS := -I../../include -I$(top_srcdir)/src/interfaces/ecpg/include \ -I$(libpq_srcdir) $(CPPFLAGS) -override CFLAGS += $(PTHREAD_CFLAGS) - LDFLAGS_INTERNAL += -L../../ecpglib -lecpg -L../../pgtypeslib -lpgtypes $(libpq) +endif +override CFLAGS += $(PTHREAD_CFLAGS) override LIBS += $(PTHREAD_LIBS) -# Standard way to invoke the ecpg preprocessor -ECPG = ../../preproc/ecpg --regression -I$(srcdir)/../../include -I$(srcdir) - # Files that most or all ecpg preprocessor test outputs depend on -ECPG_TEST_DEPENDENCIES = ../../preproc/ecpg$(X) \ +ECPG_TEST_DEPENDENCIES = $(ECPG_DEP) \ $(srcdir)/../regression.h \ $(srcdir)/../printf_hack.h \ $(srcdir)/../../include/sqlca.h \ diff --git a/src/makefiles/pgxs.mk b/src/makefiles/pgxs.mk index 070d151018..1edef064e6 100644 --- a/src/makefiles/pgxs.mk +++ b/src/makefiles/pgxs.mk @@ -392,7 +392,7 @@ endif # against installed postmaster ifndef NO_INSTALLCHECK -installcheck: submake $(REGRESS_PREP) +installcheck: $(REGRESS_PREP) $(pg_regress_installcheck) $(REGRESS_OPTS) $(REGRESS) endif diff --git a/src/pl/plperl/GNUmakefile b/src/pl/plperl/GNUmakefile index 9b1c514101..da605f8abd 100644 --- a/src/pl/plperl/GNUmakefile +++ b/src/pl/plperl/GNUmakefile @@ -113,7 +113,7 @@ uninstall-data: check: submake $(pg_regress_check) $(REGRESS_OPTS) $(REGRESS) -installcheck: submake +installcheck: $(pg_regress_installcheck) $(REGRESS_OPTS) $(REGRESS) .PHONY: submake diff --git a/src/pl/plpgsql/src/Makefile b/src/pl/plpgsql/src/Makefile index 25a5a9d448..5285eda129 100644 --- a/src/pl/plpgsql/src/Makefile +++ b/src/pl/plpgsql/src/Makefile @@ -76,7 +76,7 @@ plerrcodes.h: $(top_srcdir)/src/backend/utils/errcodes.txt generate-plerrcodes.p check: submake $(pg_regress_check) $(REGRESS_OPTS) $(REGRESS) -installcheck: submake +installcheck: $(pg_regress_installcheck) $(REGRESS_OPTS) $(REGRESS) .PHONY: submake diff --git a/src/pl/plpython/Makefile b/src/pl/plpython/Makefile index 667a74469e..839f57d8e3 100644 --- a/src/pl/plpython/Makefile +++ b/src/pl/plpython/Makefile @@ -142,7 +142,7 @@ include $(srcdir)/regress-python3-mangle.mk check: submake-pg-regress $(pg_regress_check) $(REGRESS_OPTS) $(REGRESS) -installcheck: submake-pg-regress +installcheck: $(pg_regress_installcheck) $(REGRESS_OPTS) $(REGRESS) diff --git a/src/pl/tcl/Makefile b/src/pl/tcl/Makefile index ef61ee596e..cc6216df04 100644 --- a/src/pl/tcl/Makefile +++ b/src/pl/tcl/Makefile @@ -82,7 +82,7 @@ uninstall-data: check: submake $(pg_regress_check) $(REGRESS_OPTS) $(REGRESS) -installcheck: submake +installcheck: $(pg_regress_installcheck) $(REGRESS_OPTS) $(REGRESS) .PHONY: submake diff --git a/src/test/isolation/Makefile b/src/test/isolation/Makefile index c3c8280ea2..4c3a937c2f 100644 --- a/src/test/isolation/Makefile +++ b/src/test/isolation/Makefile @@ -15,6 +15,9 @@ OBJS = specparse.o isolationtester.o $(WIN32RES) all: isolationtester$(X) pg_isolation_regress$(X) +all-installcheck: + USE_INSTALLED_ASSETS=1 $(MAKE) all + # Though we don't install these binaries, build them during installation # (including temp-install). Otherwise, "make -j check-world" and "make -j # installcheck-world" would spawn multiple, concurrent builds in this @@ -49,7 +52,7 @@ clean distclean: maintainer-clean: distclean rm -f specparse.c specscanner.c -installcheck: all +installcheck: all-installcheck $(pg_isolation_regress_installcheck) --schedule=$(srcdir)/isolation_schedule check: all @@ -59,7 +62,7 @@ check: all # It only makes sense to run these if set up to use prepared transactions, # via TEMP_CONFIG for the check case, or via the postgresql.conf for the # installcheck case. -installcheck-prepared-txns: all temp-install +installcheck-prepared-txns: all-installcheck temp-install $(pg_isolation_regress_installcheck) --schedule=$(srcdir)/isolation_schedule prepared-transactions check-prepared-txns: all temp-install diff --git a/src/test/regress/GNUmakefile b/src/test/regress/GNUmakefile index 378bd01753..595ffc853f 100644 --- a/src/test/regress/GNUmakefile +++ b/src/test/regress/GNUmakefile @@ -34,11 +34,17 @@ EXTRADEFS = '-DHOST_TUPLE="$(host_tuple)"' \ # Build regression test driver +ifndef USE_INSTALLED_ASSETS all: pg_regress$(X) +endif pg_regress$(X): pg_regress.o pg_regress_main.o $(WIN32RES) | submake-libpgport $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@ +.PHONY: all-installcheck +all-installcheck: + USE_INSTALLED_ASSETS=1 $(MAKE) all + # dependencies ensure that path changes propagate pg_regress.o: pg_regress.c $(top_builddir)/src/port/pg_config_paths.h pg_regress.o: override CPPFLAGS += -I$(top_builddir)/src/port $(EXTRADEFS) @@ -134,16 +140,16 @@ check: all tablespace-setup check-tests: all tablespace-setup | temp-install $(pg_regress_check) $(REGRESS_OPTS) $(MAXCONNOPT) $(TESTS) $(EXTRA_TESTS) -installcheck: all tablespace-setup +installcheck: all-installcheck tablespace-setup $(pg_regress_installcheck) $(REGRESS_OPTS) --schedule=$(srcdir)/serial_schedule $(EXTRA_TESTS) -installcheck-parallel: all tablespace-setup +installcheck-parallel: all-installcheck tablespace-setup $(pg_regress_installcheck) $(REGRESS_OPTS) --schedule=$(srcdir)/parallel_schedule $(MAXCONNOPT) $(EXTRA_TESTS) -installcheck-tests: all tablespace-setup +installcheck-tests: all-installcheck tablespace-setup $(pg_regress_installcheck) $(REGRESS_OPTS) $(TESTS) $(EXTRA_TESTS) -standbycheck: all +standbycheck: all-installcheck $(pg_regress_installcheck) $(REGRESS_OPTS) --schedule=$(srcdir)/standby_schedule --use-existing # old interfaces follow... @@ -152,7 +158,7 @@ runcheck: check runtest: installcheck runtest-parallel: installcheck-parallel -bigtest: all tablespace-setup +bigtest: all-installcheck tablespace-setup $(pg_regress_installcheck) $(REGRESS_OPTS) --schedule=$(srcdir)/serial_schedule numeric_big bigcheck: all tablespace-setup | temp-install --------------0B89BFA128936D6F40DA4C61--