Subject: Add support for EXTRA_REGRESS_OPTS for meson

From: Andreas Karlsson <[email-masked]>
Date: Thu, 27 Feb 2025 13:53:17 +0100

Hi,

We use EXTRA_REGRESS_OPTS to make sure the whole test suite passes with 
our extension loaded and since I prefer develop in meson over using 
autotools and make the lack of support for EXTRA_REGRESS_OPTS in meson
has bugged me for a while.

I have implemented support for it as an environment variable we read in 
the testwrap script instead of adding it as a configuration option to 
meson.build. The reason for this is that I do not like having to run 
"meson reconfigure" all the time plus that for the PG_TEST_EXTRA we 
ended up having to add an environment variable anyway.

To use this run e.g. the following:

   EXTRA_REGRESS_OPTS="--load-extension=pgcrypto" meson test

Question: Would it make sense to rename it to PG_REGRESS_EXTRA_OPTS or 
something similar while we already touch touch this code to make the 
various options easier to remember?

Andreas
From 87ce622a19315b679bbd5691e01c96261bc0c4c8 Mon Sep 17 00:00:00 2001
From: Andreas Karlsson <[email-masked]>
Date: Thu, 27 Feb 2025 00:23:44 +0100
Subject: [PATCH] meson: Add support for EXTRA_REGRESS_OPTS

Add support for the EXTRA_REGRESS_OPTS environment variable in meson
which works just like with make and applies to all regress, ecpg and
isolation tests. TAP tests which support it under make will continue
to support the option.

Run it with e.g:

    EXTRA_REGRESS_OPTS="--temp-config=test.conf" meson test
---
 src/tools/testwrap | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/src/tools/testwrap b/src/tools/testwrap
index 8ae8fb79ba7..f52ca376da1 100755
--- a/src/tools/testwrap
+++ b/src/tools/testwrap
@@ -2,6 +2,7 @@
 
 import argparse
 import shutil
+import shlex
 import subprocess
 import os
 import sys
@@ -51,7 +52,12 @@ env_dict = {**os.environ,
 if "PG_TEST_EXTRA" not in env_dict and args.pg_test_extra:
     env_dict["PG_TEST_EXTRA"] = args.pg_test_extra
 
-sp = subprocess.Popen(args.test_command, env=env_dict, stdout=subprocess.PIPE)
+if args.testname in ['regress', 'isolation', 'ecpg'] and 'EXTRA_REGRESS_OPTS' in env_dict:
+    test_command = args.test_command + shlex.split(env_dict['EXTRA_REGRESS_OPTS'])
+else:
+    test_command = args.test_command
+
+sp = subprocess.Popen(test_command, env=env_dict, stdout=subprocess.PIPE)
 # Meson categorizes a passing TODO test point as bad
 # (https://github.com/mesonbuild/meson/issues/13183).  Remove the TODO
 # directive, so Meson computes the file result like Perl does.  This could
-- 
2.47.2

---

From: Andres Freund <[email-masked]>
Date: Thu, 27 Feb 2025 08:21:22 -0500

Hi,

On 2025-02-27 13:53:17 +0100, Andreas Karlsson wrote:
> We use EXTRA_REGRESS_OPTS to make sure the whole test suite passes with our
> extension loaded and since I prefer develop in meson over using autotools
> and make the lack of support for EXTRA_REGRESS_OPTS in meson

Yep, we should add support for that.  TEMP_CONFIG probably too.

> something similar while we already touch touch this code to make the various
> options easier to remember?

I'd not tackle that at the same time personally.

>  # directive, so Meson computes the file result like Perl does.  This could

I hacked up something similar before, for TEMP_CONFIG, and found that I needed
to do something like this:

+if 'TEMP_CONFIG' in os.environ and \
+    args.testname in ['regress', 'isolation', 'ecpg']:
+        # be careful to insert before non-option args, otherwise it'll fail
+        # e.g. on windows
+        args.test_command.insert(1, '--temp-config='+os.environ['TEMP_CONFIG'])

Greetings,

Andres Freund

---

From: vignesh C <[email-masked]>
Date: Thu, 13 Mar 2025 20:11:49 +0530

> +        args.test_command.insert(1, '--temp-config='+os.environ['TEMP_CONFIG'])

Could you please upload a v2 to address this? In the meantime, I’ve
updated the commitfest entry status to 'Waiting on Author.' Kindly
update the status once the new version is posted.

Regards,
Vignesh

---

From: Rustam ALLAKOV <[email-masked]>
Date: Wed, 19 Mar 2025 22:25:46 +0000

The following review has been posted through the commitfest application:
make installcheck-world:  not tested
Implements feature:       tested, failed
Spec compliant:           tested, failed
Documentation:            tested, failed

Hello everyone, 
for v2 patch I suggest to refactor from

> +sp = subprocess.Popen(test_command, env=env_dict, stdout=subprocess.PIPE)
 
to

-sp = subprocess.Popen(args.test_command, env=env_dict, stdout=subprocess.PIPE)
+if args.testname in ['regress', 'isolation', 'ecpg']:
+    test_command = args.test_command[:]
+    if 'TEMP_CONFIG' in env_dict:
+        test_command.insert(1, '--temp-config=' + env_dict['TEMP_CONFIG'])
+    if 'EXTRA_REGRESS_OPTS' in env_dict:
+        test_command += shlex.split(env_dict['EXTRA_REGRESS_OPTS'])
+else:
+    test_command = args.test_command
+
+sp = subprocess.Popen(test_command, env=env_dict, stdout=subprocess.PIPE)

I double checked whether shlex module was built in Python, and yes it is, 
so no need for additional requirement.txt input for pip to install.

in addition to the above, might be worth to add some documentation like

Environment Variables Supported:  
EXTRA_REGRESS_OPTS: Additional options to pass to regression, isolation, or ecpg tests.
TEMP_CONFIG: Specify a temporary configuration file for testing purposes.
Example Usage:
# Use EXTRA_REGRESS_OPTS to load an extension
  EXTRA_REGRESS_OPTS="--load-extension=pgcrypto" meson test
# Use TEMP_CONFIG to specify a temporary configuration file
  TEMP_CONFIG="path/to/test.conf" meson test
# Use both EXTRA_REGRESS_OPTS and TEMP_CONFIG together
  TEMP_CONFIG="path/to/test.conf" EXTRA_REGRESS_OPTS="--load-extension=pgcrypto" meson test

Should we cover these new lines with test? Asking, because I see EXTRA_REGRESS_OPTS
being tested for autotools in src/interfaces/ecpg/test/makefile

Regards.

---

From: Andreas Karlsson <[email-masked]>
Date: Wed, 31 Dec 2025 02:01:05 +0100

> +sp = subprocess.Popen(test_command, env=env_dict, stdout=subprocess.PIPE)

Since commit 51da766494dcc84b6f8d793ecaa064363a9243c2 it is possible 
that we no longer need to use .insert() to make sure the code works on 
Windows but I need to think a bit more on it.

But added support for TEMP_CONFIG.

> so no need for additional requirement.txt input for pip to install.

Thanks!

>    TEMP_CONFIG="path/to/test.conf" EXTRA_REGRESS_OPTS="--load-extension=pgcrypto" meson test

Yeah, we probably should. But not sure where, maybe at 
https://www.postgresql.org/docs/current/install-meson.html or did you 
imagine somewhere else?

> being tested for autotools in src/interfaces/ecpg/test/makefile

As far as I can see they are not tested there, but maybe we should test 
them.

Attached version 2 of the patch.

Andreas
From 0e73ab29af1ccab9c4dd7453f07b41a4527b0340 Mon Sep 17 00:00:00 2001
From: Andreas Karlsson <[email-masked]>
Date: Wed, 31 Dec 2025 01:48:56 +0100
Subject: [PATCH v2 2/2] meson: Add support for EXTRA_REGRESS_OPTS and
 TEMP_CONFIG

Add support for the EXTRA_REGRESS_OPTS and TEMP_CONFIG environment
variables in our Meson build which work just like with make and
apply to all regress, ecpg and isolation tests.
---
 src/tools/testwrap | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/src/tools/testwrap b/src/tools/testwrap
index e91296ecd15..165d35fba98 100755
--- a/src/tools/testwrap
+++ b/src/tools/testwrap
@@ -4,6 +4,7 @@ import argparse
 import shutil
 import subprocess
 import os
+import shlex
 import sys
 
 parser = argparse.ArgumentParser()
@@ -53,6 +54,14 @@ env_dict = {**os.environ,
 if "PG_TEST_EXTRA" not in env_dict and args.pg_test_extra:
     env_dict["PG_TEST_EXTRA"] = args.pg_test_extra
 
+# Add extra regress arguments before we add non-option arguments
+if args.testname in ['regress', 'isolation', 'ecpg']:
+    if 'TEMP_CONFIG' in env_dict:
+        args.test_command += ['--temp-config=' + env_dict['TEMP_CONFIG']]
+
+    if 'EXTRA_REGRESS_OPTS' in env_dict:
+        args.test_command += shlex.split(env_dict['EXTRA_REGRESS_OPTS'])
+
 if "TESTS" in env_dict:
     args.test_command += env_dict["TESTS"].split()
 else:
-- 
2.47.3

---

From: Rustam ALLAKOV <[email-masked]>
Date: Tue, 13 Jan 2026 22:06:04 +0000

> https://www.postgresql.org/docs/current/install-meson.html or did you 

Either at docs or/and directly in file /src/tools/testwrap as comments
Regards.

---

From: Andreas Karlsson <[email-masked]>
Date: Wed, 14 Jan 2026 02:17:30 +0100

> Either at docs or/and directly in file /src/tools/testwrap as comments

If we add it to the documentation the question is if we should also 
document environment variables for builds with make. But maybe that is 
for a separate patch.

As for documenting it in testwrap: that does not feel very useful to me 
since people who know that this is where they should look likely already 
knows about these. It is a pretty hidden place. Hmm, not sure where such 
information belongs.

Andreas

---

From: Michael Paquier <[email-masked]>
Date: Tue, 10 Mar 2026 15:18:58 +0900

On Wed, Jan 14, 2026 at 02:17:30AM +0100, Andreas Karlsson wrote:
> If we add it to the documentation the question is if we should also document
> environment variables for builds with make. But maybe that is for a separate
> patch.
> 

On point is that this also make the gap in the documentation bigger,
and if we are just doing something, why not tackle it now.  It is true
that TESTS is not documented as something that can work in meson, and
that it is not the fault of this patch, but I think that we should
spend a bit of time on the shape of an extra effort for:
1) closing the gap with the existing variables before adding more
stuff.
2) adding documentation for the new things added to testwrap, and
require that documentation is added for everything in the future.

If somebody sends a patch doing 1) for the existing things, and adds
the documentation of the new variables, satisfying 2), I would be OK
to merge the proposal (after checking it of course).

One issue for me with meson is that too many things feel implied, like
the regression test facilities that we make the effort to document for
configure/make.
--
Michael

---

From: Michael Paquier <[email-masked]>
Date: Tue, 31 Mar 2026 19:46:35 +0900

On Tue, Mar 31, 2026 at 10:45:32AM +0200, Andreas Karlsson wrote:
> Agreed, the question is just where we should document them.
> 
> I would say one of these two pages:
> 
> https://www.postgresql.org/docs/current/install-meson.html
> https://www.postgresql.org/docs/current/regress-run.html

regress-run would be a nicer fit in the long run, where it seems to me
that the existing documentation is wrong with its decision to document
the regression tests that can be run in meson in a section for the
build and install:
https://www.postgresql.org/docs/current/install-meson.html#INSTALL-PROCEDURE-MESON

The format that we use for the coverage code is something I find
pretty clear:
https://www.postgresql.org/docs/current/regress-coverage.html

We have two sub-sections, one for configure/make and one meson, which
is nice.  Perhaps we don't need to have new sub-sections, limiting
ourselves to mention some of the equivalents that can be used with
meson?
--
Michael