MIME-Version: 1.0
In-Reply-To: 
 <CANxYE3KnRLfGxVgj2+ePbjhgwmoLPXzk0XLskUj0cTABLo25Pg@mail.gmail.com>
References: 
 <CAG8BBZM0gapFd0ZvMq7euCYamxKEBNqewC4YX4bYddyc_6AatA@mail.gmail.com>
 <CA+OCxoyhMprhxCMss5oESTMcVVnVcyBOP_AuntNyZsRyeZu5rg@mail.gmail.com>
 <CAG8BBZOkQ8FPzXzJAwe5+EgAKEhbpcx+_=XQh3UQhWQE620xqg@mail.gmail.com>
 <CA+OCxozBpu020t4+tagcvMStjHakp7aEduGVOFgM8quaRvmxCg@mail.gmail.com>
 <CAE+jjam92a4rwYM+VxekLTRjLoWdAiPw4MuiCP1dUT_aWc5-6g@mail.gmail.com>
 <CA+OCxowt6nQrdNj-tKupZ_ohJeGyH20G3VfNVjKa85RgVe+A1g@mail.gmail.com>
 <CAE+jjakFfSjDWyyQEwQrAsveSSWvJ_cpcNESeC6F4TShZqfP8Q@mail.gmail.com>
 <CAE+jjakW15xMbHyMBUu6X_7ndi07ymrwyE4cV_=txSSXBcSNuA@mail.gmail.com>
 <CA+OCxowq+5CSjtQNnmy_vf4m3=kd6XGYyFkh=nevQ6qF+5RxPg@mail.gmail.com>
 <CAE+jjanMmCJzHbMgtxB-6D8+Dvp9bbqEkiSE=oDuVxUU=AOzrA@mail.gmail.com>
 <CA+OCxoy6=pT+Fe2qh11aPzZ-psBC1mTu=Rj-dqdUaV2tXy4NwA@mail.gmail.com>
 <CANxYE3KnRLfGxVgj2+ePbjhgwmoLPXzk0XLskUj0cTABLo25Pg@mail.gmail.com>
From: Dave Page <dpage@pgadmin.org>
Date: Wed, 13 Jun 2018 13:37:48 +0100
Message-ID: 
 <CA+OCxoxerf5iC3YiF2urmguyaXVc=_q2kPiwbg+ZrfBYOrAduQ@mail.gmail.com>
Subject: Re: [pgadmin4][patch] Use pytest test runner for unit tests
To: Victoria Henry <vhenry@pivotal.io>
Cc: Joao De Almeida Pereira <jdealmeidapereira@pivotal.io>,
	pgadmin-hackers <pgadmin-hackers@postgresql.org>,
 Anthony Emengo <aemengo@pivotal.io>
Content-Type: multipart/alternative; boundary="0000000000001d48b7056e8540d0"
Precedence: bulk

--0000000000001d48b7056e8540d0
Content-Type: text/plain; charset="UTF-8"

Hi

On Tue, Jun 12, 2018 at 3:41 PM, Victoria Henry <vhenry@pivotal.io> wrote:

> Hi Dave,
>
> Attached are four different patches that should be applied in order.
> Included is a fix for the failing test above.
>

This is no longer failing :-)

However, I'm still seeing problems that need to be resolved:

- Passwords are still being written to the log file. This is a complete
show-stopper.

- The regression.log file is empty. I'm now only getting the JSON summary.
We really need both - the regression log for diagnostics, and the JSON file
for feeding into monitoring systems.

- There is still no summary list of tests that failed or were skipped
output at the end of the run. This is important because it highlights which
tests failed (obviously!) and need investigation, and knowing which ones
are skipped at a glance is very useful as they can be skipped due to
mis-configuration; e.g. running against a different server than intended,
or failing to specify something like a tablespace path which we can then
easily see to fix.

- I cannot run against servers other than the first one in the config file.
We need to run against all that are enabled, and have the output show which
one was used for each one as we have now. This may seem trivial, but it's
really not - to the point that for some of us it negates the whole point of
the patch by giving us more control over which tests are run but removing
all control over which database servers they will run against. In my case,
this will almost certainly cost even more time; right now I just have to
run more tests than I may need but I can do other things at the same time.
With this patch I'll have to keep re-running the tests, modifying the
config between each run.

Thanks.

-- 
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--0000000000001d48b7056e8540d0
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi<br><div class=3D"gmail_extra"><br><div class=3D"gmail_q=
uote">On Tue, Jun 12, 2018 at 3:41 PM, Victoria Henry <span dir=3D"ltr">&lt=
;<a href=3D"mailto:vhenry@pivotal.io" target=3D"_blank">vhenry@pivotal.io</=
a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0=
 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr">Hi =
Dave,<div><br></div><div>Attached are four different patches that should be=
 applied in order.=C2=A0 Included is a fix for the failing test above.</div=
></div></blockquote><div><br></div><div>This is no longer failing :-)</div>=
<div><br></div><div>However, I&#39;m still seeing problems that need to be =
resolved:</div><div><br></div><div>- Passwords are still being written to t=
he log file. This is a complete show-stopper.</div><div><br></div><div>- Th=
e regression.log file is empty. I&#39;m now only getting the JSON summary. =
We really need both - the regression log for diagnostics, and the JSON file=
 for feeding into monitoring systems.</div><div><br></div><div>- There is s=
till no summary list of tests that failed or were skipped output at the end=
 of the run. This is important because it highlights which tests failed (ob=
viously!) and need investigation, and knowing which ones are skipped at a g=
lance is very useful as they can be skipped due to mis-configuration; e.g. =
running against a different server than intended, or failing to specify som=
ething like a tablespace path which we can then easily see to fix.</div><di=
v><br></div><div>- I cannot run against servers other than the first one in=
 the config file. We need to run against all that are enabled, and have the=
 output show which one was used for each one as we have now. This may seem =
trivial, but it&#39;s really not - to the point that for some of us it nega=
tes the whole point of the patch by giving us more control over which tests=
 are run but removing all control over which database servers they will run=
 against. In my case, this will almost certainly cost even more time; right=
 now I just have to run more tests than I may need but I can do other thing=
s at the same time. With this patch I&#39;ll have to keep re-running the te=
sts, modifying the config between each run.</div><div><br></div><div>Thanks=
.</div></div><div><br></div>-- <br><div class=3D"gmail_signature" data-smar=
tmail=3D"gmail_signature">Dave Page<br>Blog: <a href=3D"http://pgsnake.blog=
spot.com" target=3D"_blank">http://pgsnake.blogspot.com</a><br>Twitter: @pg=
snake<br><br>EnterpriseDB UK: <a href=3D"http://www.enterprisedb.com" targe=
t=3D"_blank">http://www.enterprisedb.com</a><br>The Enterprise PostgreSQL C=
ompany<br></div>
</div></div>

--0000000000001d48b7056e8540d0--