06.04.2026 16:47, Jelte Fennema-Nio wrote:
> I'm fine with adding the abilitity to
configure what pg_regress should
> print, but it's unclear to me what's special about the diff
output
> compared to all the other output? i.e. what is the output you
would
> actually like to see for your use case? Piping everything to
/dev/null
> would silence all output except for the exit code, but I
guess you
> want some output. i.e. what do you do with the output that
you get? Do
> you only want to know which tests failed? If so, do you care
about
> which tests are passing?
Yes, redirecting all output to /dev/null isn't exactly what I
need.
Take a specific case, for example:
When making numerous changes to tests as part
of extensive stability
testing (specifically, investigating
crashes), the majority of test
files may be modified. This results in diffs
output for practically
every single test, which turns into an
unreadable jumble during the
test run. In this case, I am not interested
in all the diffs
the ones associated with a crash. I can
inspect those specific cases
manually later, without needing the diffs
output. Therefore, a flag
like `PG_REGRESS_DIFF_OPTS='q'` does not suit
my needs. And to find
out which tests caused the crash (exit code 1
or 2) I need the
entire output.
06.04.2026
21:11, Daniel Gustafsson wrote:
> I'm not
sure I entirely understand the problem. If you expect lots of
> failures,
but also don't want to see the test failures, what is the use of
> running the
tests? Why not run the subset you actually care about and expand
> that set as
testing fixes bugs/issues?
As part of exploratory testing:
1. Output mismatches are considered normal behavior.
2. Exploratory testing almost always affects all tests. Cases
where
it is necessary to run a small subset of tests are rare.
Another case where multiple tests may fail is testing a patch
that
changes cluster parameters.For example, a patch for the
scheduler
has been released, and I need to ensure that it doesn't cause
crashes
when changing the 'random_page_cost' parameter. This parameter
significantly changes the plan output, leading to multiple
crashes and,
consequently, the diffs output. While I'm not interested in the
diffs
themselves, I do want to know that there haven't been any
crashes.
In this case, we need to run not just a small group of tests,
but everything we have.
These are just a couple of examples where disabling diffs
output
would be useful.
---
Also, if the
diffs within a single directory run are very large,
the output starts to become cluttered with lines like:
#(diff output truncated and silencing output for
further failing tests...)
Which also creates confusion and interferes with output
analysis.
Example
#<long diffs output>
not ok 1 - test_setup 133 ms
# parallel group (20 tests, in groups of 1): boolean char name
...
# (diff output truncated and silencing output for further
failing tests...)
not ok 2 + boolean 14 ms
# (diff output truncated and silencing output for further
failing tests...)
not ok 3 + char 8 ms
# (diff output truncated and silencing output for further
failing tests...)
not ok 4 + name 8 ms
# (diff output truncated and silencing output for further
failing tests...)
<etc.>
---
Best regards,
Ilya Cherdakov