public inbox for [email protected]help / color / mirror / Atom feed
Environment variable to disable diffs file output 4+ messages / 4 participants [nested] [flat]
* Environment variable to disable diffs file output @ 2026-04-06 07:55 Илья Чердаков <[email protected]> 2026-04-06 09:47 ` Re: Environment variable to disable diffs file output Jelte Fennema-Nio <[email protected]> 2026-04-06 14:11 ` Re: Environment variable to disable diffs file output Daniel Gustafsson <[email protected]> 0 siblings, 2 replies; 4+ messages in thread From: Илья Чердаков @ 2026-04-06 07:55 UTC (permalink / raw) To: pgsql-hackers; +Cc: [email protected] Greetings! I'd like to add a environment variable to the patch to disable diff output. Sometimes you have to be creative with exploratory testing, modifying test files, etc., and multiple diff outputs can become a nuisance. This doesn't change the default behavior, but it does give you control over diff output in tests for specific scenarios where a large number of failures are expected (mass runs or exploratory testing with changing tests). The env usage and implementation are somewhat similar to PG_REGRESS_DIFF_OPTS PG_REGRESS_DISABLE_DIFFS_OUTPUT=1 make check -s. For now, I suggest to simply disabling it, but I'm also considering passing the number of lines to output via env or via flag pg_regress instead of completely disabling it. Something like PG_REGRESS_DIFFS_NUMBER_LINES_OUTPUT=20 make check and then when an error occurs, 20 lines will be output instead of the default 80. Setting it to 0 disables diffs output. I think this small addition will be useful for many developers, especially testers. --- This is my first patch. I welcome critique and comments. Best regards, Ilya Cherdakov, PostgresPro From aece3527f29dcaf2731ef99a4d95c792facdf63d Mon Sep 17 00:00:00 2001 From: Ilya Cherdakov <[email protected]> Date: Mon, 6 Apr 2026 10:18:22 +0300 Subject: [PATCH v1] Adding-env-disables-diff-output --- src/test/regress/pg_regress.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/src/test/regress/pg_regress.c b/src/test/regress/pg_regress.c index 9a918156437..06af24699b4 100644 --- a/src/test/regress/pg_regress.c +++ b/src/test/regress/pg_regress.c @@ -1550,6 +1550,13 @@ results_differ(const char *testname, const char *resultsfile, const char *defaul pretty_diff_opts, best_expect_file, resultsfile, difffilename); run_diff(cmd, difffilename); + /* Disabling difffile output via an environment variable */ + if (getenv("PG_REGRESS_DISABLE_DIFFS_OUTPUT") != NULL) + { + unlink(diff); + return true; + } + /* * Reopen the file for reading to emit the diff as TAP diagnostics. We * can't keep the file open while diff appends to it, because on -- 2.47.3 Attachments: [text/plain] v1-0001-Adding-env-disables-diff-output.patch (1013B, 2-v1-0001-Adding-env-disables-diff-output.patch) download | inline diff: From aece3527f29dcaf2731ef99a4d95c792facdf63d Mon Sep 17 00:00:00 2001 From: Ilya Cherdakov <[email protected]> Date: Mon, 6 Apr 2026 10:18:22 +0300 Subject: [PATCH v1] Adding-env-disables-diff-output --- src/test/regress/pg_regress.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/src/test/regress/pg_regress.c b/src/test/regress/pg_regress.c index 9a918156437..06af24699b4 100644 --- a/src/test/regress/pg_regress.c +++ b/src/test/regress/pg_regress.c @@ -1550,6 +1550,13 @@ results_differ(const char *testname, const char *resultsfile, const char *defaul pretty_diff_opts, best_expect_file, resultsfile, difffilename); run_diff(cmd, difffilename); + /* Disabling difffile output via an environment variable */ + if (getenv("PG_REGRESS_DISABLE_DIFFS_OUTPUT") != NULL) + { + unlink(diff); + return true; + } + /* * Reopen the file for reading to emit the diff as TAP diagnostics. We * can't keep the file open while diff appends to it, because on -- 2.47.3 ^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Environment variable to disable diffs file output 2026-04-06 07:55 Environment variable to disable diffs file output Илья Чердаков <[email protected]> @ 2026-04-06 09:47 ` Jelte Fennema-Nio <[email protected]> 1 sibling, 0 replies; 4+ messages in thread From: Jelte Fennema-Nio @ 2026-04-06 09:47 UTC (permalink / raw) To: Илья Чердаков <[email protected]>; +Cc: pgsql-hackers On Mon, 6 Apr 2026 at 09:56, Илья Чердаков <[email protected]> wrote: > I'd like to add a environment variable to the patch to > disable diff output. Sometimes you have to be creative > with exploratory testing, modifying test files, etc., > and multiple diff outputs can become a nuisance. This > doesn't change the default behavior, but it does give > you control over diff output in tests for specific > scenarios where a large number of failures are expected > (mass runs or exploratory testing with changing tests). I'm fine with adding the abilitity to configure what pg_regress should print, but it's unclear to me what's special about the diff output compared to all the other output? i.e. what is the output you would actually like to see for your use case? Piping everything to /dev/null would silence all output except for the exit code, but I guess you want some output. i.e. what do you do with the output that you get? Do you only want to know which tests failed? If so, do you care about which tests are passing? ^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Environment variable to disable diffs file output 2026-04-06 07:55 Environment variable to disable diffs file output Илья Чердаков <[email protected]> @ 2026-04-06 14:11 ` Daniel Gustafsson <[email protected]> 2026-04-07 08:31 ` Re: Environment variable to disable diffs file output Ilya Cherdakov <[email protected]> 1 sibling, 1 reply; 4+ messages in thread From: Daniel Gustafsson @ 2026-04-06 14:11 UTC (permalink / raw) To: Илья Чердаков <[email protected]>; +Cc: pgsql-hackers; [email protected] > On 6 Apr 2026, at 09:55, Илья Чердаков <[email protected]> wrote: > I'd like to add a environment variable to the patch to > disable diff output. Sometimes you have to be creative > with exploratory testing, modifying test files, etc., > and multiple diff outputs can become a nuisance. This > doesn't change the default behavior, but it does give > you control over diff output in tests for specific > scenarios where a large number of failures are expected > (mass runs or exploratory testing with changing tests). > > The env usage and implementation are somewhat similar > to PG_REGRESS_DIFF_OPTS > PG_REGRESS_DISABLE_DIFFS_OUTPUT=1 make check -s. > > For now, I suggest to simply disabling it, but I'm also > considering passing the number of lines to output via env > or via flag pg_regress instead of completely disabling it. > Something like > PG_REGRESS_DIFFS_NUMBER_LINES_OUTPUT=20 make check > and then when an error occurs, 20 lines will be output > instead of the default 80. Setting it to 0 disables > diffs output. I'm not sure I entirely understand the problem. If you expect lots of failures, but also don't want to see the test failures, what is the use of running the tests? Why not run the subset you actually care about and expand that set as testing fixes bugs/issues? -- Daniel Gustafsson ^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Environment variable to disable diffs file output 2026-04-06 07:55 Environment variable to disable diffs file output Илья Чердаков <[email protected]> 2026-04-06 14:11 ` Re: Environment variable to disable diffs file output Daniel Gustafsson <[email protected]> @ 2026-04-07 08:31 ` Ilya Cherdakov <[email protected]> 0 siblings, 0 replies; 4+ messages in thread From: Ilya Cherdakov @ 2026-04-07 08:31 UTC (permalink / raw) To: Daniel Gustafsson <[email protected]>; +Cc: pgsql-hackers; [email protected] 06.04.2026 16:47, Jelte Fennema-Nio wrote: > I'm fine with adding the abilitity to configure what pg_regress should > print, but it's unclear to me what's special about the diff output > compared to all the other output? i.e. what is the output you would > actually like to see for your use case? Piping everything to /dev/null > would silence all output except for the exit code, but I guess you > want some output. i.e. what do you do with the output that you get? Do > you only want to know which tests failed? If so, do you care about > which tests are passing? Yes, redirecting all output to /dev/null isn't exactly what I need. Take a specific case, for example: When making numerous changes to tests as part of extensive stability testing (specifically, investigating crashes), the majority of test files may be modified. This results in diffs output for practically every single test, which turns into an unreadable jumble during the test run. In this case, I am not interested in all the diffs the ones associated with a crash. I can inspect those specific cases manually later, without needing the diffs output. Therefore, a flag like `PG_REGRESS_DIFF_OPTS='q'` does not suit my needs. And to find out which tests caused the crash (exit code 1 or 2) I need the entire output. 06.04.2026 21:11, Daniel Gustafsson wrote: > I'm not sure I entirely understand the problem. If you expect lots of > failures, but also don't want to see the test failures, what is the use of > running the tests? Why not run the subset you actually care about and expand > that set as testing fixes bugs/issues? As part of exploratory testing: 1. Output mismatches are considered normal behavior. 2. Exploratory testing almost always affects all tests. Cases where it is necessary to run a small subset of tests are rare. Another case where multiple tests may fail is testing a patch that changes cluster parameters.For example, a patch for the scheduler has been released, and I need to ensure that it doesn't cause crashes when changing the 'random_page_cost' parameter. This parameter significantly changes the plan output, leading to multiple crashes and, consequently, the diffs output. While I'm not interested in the diffs themselves, I do want to know that there haven't been any crashes. In this case, we need to run not just a small group of tests, but everything we have. These are just a couple of examples where disabling diffs output would be useful. --- Also, if the diffs within a single directory run are very large, the output starts to become cluttered with lines like: #(diff output truncated and silencing output for further failing tests...) Which also creates confusion and interferes with output analysis. Example #<long diffs output> not ok 1 - test_setup 133 ms # parallel group (20 tests, in groups of 1): boolean char name ... # (diff output truncated and silencing output for further failing tests...) not ok 2 + boolean 14 ms # (diff output truncated and silencing output for further failing tests...) not ok 3 + char 8 ms # (diff output truncated and silencing output for further failing tests...) not ok 4 + name 8 ms # (diff output truncated and silencing output for further failing tests...) <etc.> --- Best regards, Ilya Cherdakov Postgres Professional: https://postgrespro.com ^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2026-04-07 08:31 UTC | newest] Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2026-04-06 07:55 Environment variable to disable diffs file output Илья Чердаков <[email protected]> 2026-04-06 09:47 ` Jelte Fennema-Nio <[email protected]> 2026-04-06 14:11 ` Daniel Gustafsson <[email protected]> 2026-04-07 08:31 ` Ilya Cherdakov <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox