Subject: [pgAdmin 4 - Housekeeping #5255] Implement Selenium Grid using multi-threading & solenoid using current test framework

From: Yogesh Mahajan <[email-masked]>
Date: Mon, 13 Apr 2020 14:40:02 +0530

Hi,

Please find the attached patch for running *features tests* using
solenoid(selenium grid + docker).
KIndly review.
To sun feature tests in parallel, required prerequisites can be checked in
'~/web/regression/README' file.
Also detailed instructions are added in the same file.
After applying the patch, any existing process for execution of
API/Features tests remains the same.

Thanks,
Yogesh Mahajan
QA - Team
EnterpriseDB Corporation

Phone: +91-9741705709

---

From: Akshay Joshi <[email-masked]>
Date: Mon, 13 Apr 2020 14:49:16 +0530

Hi Navnath

Can you please review it?

-- 
*Thanks & Regards*
*Akshay Joshi*

*Sr. Software Architect*
*EnterpriseDB Software India Private Limited*
*Mobile: +91 976-788-8246*

---

From: navnath gadakh <[email-masked]>
Date: Thu, 16 Apr 2020 14:40:46 +0530

Hi,
I think I am not the right person to review this patch now as I already
reviewed this code offline in the last week. I know the approached Yogesh
has followed, also given some review comments on it.
Someone else please review it.

Thanks!

On Mon, Apr 13, 2020 at 2:49 PM Akshay Joshi <[email-masked]>
wrote:

>> To sun feature tests in parallel, required prerequisites can be checked
>> in '~/web/regression/README' file.

-- 
Regards,
Navnath Gadakh

---

From: Shubham Agarwal <[email-masked]>
Date: Tue, 21 Apr 2020 13:18:29 +0530

Hi Yogesh,
Below are the review comments-

1. runtests.py
    a. The exception traceback logic at line number 653 in runtests.py is
not correct since it is particular to the thread
but there is much more code in that block which can throw some exception.
b. line number 447 -> The drop_database function will only try to drop the
database with the name which is newly created
at 431 line number, its probability is 1% instead of this you can write a
logic so that it will drop all the database which starts with name
‘acceptance_test_db'.
  c. line 584 - Why we are including resql test case execution in GUI
execution logic.
    d. Change the function name run_test as script name is also runtests.py

2. test_utils.py
    a. Remove the headless chrome code from get_remote_webdriver() in
test_utils.py since we are using solenoid and it is not required anymore.
    b. Create separate functions to instantiate the firefox driver and
chrome driver logic since the same code is used in multiple files.
c. launch_url_in_browser() -> you can simplify the definition of the
function like:
    retry = 60
        *while *retry > 0:
            try:
                driver.get(url)
            except WebDriverException:
                 retry -= 1
3. Execution logs are not printing as per the logic some time, I ran the
suite for two servers and attached are the execution logs.
4. Readme -
Please provide the Valid selenoid URL to be provided in the
test_config.json, with all the steps mentioned in the readme it is not
clear.
Revisit the readme and write the missing steps.
5. copy_selected_query_results_feature_tests.py-
Create the function to avoid duplicate code. The code for pasting the
values is repeating 8 times in the test code.
6. Provide the valid docstring in newly introduced functions and also valid
comments while calling it. for ex.- _update_preference() function is
introduced in pg_utilities_backup_restrore_test.py but from the function
name, it is not clear what preferences are going to update in it.
7. test_index_constraint_add test case is failing due to the latest change,
please merge and update this test case

On Thu, Apr 16, 2020 at 2:41 PM navnath gadakh <
[email-masked]> wrote:

> On Mon, Apr 13, 2020 at 2:49 PM Akshay Joshi <
> [email-masked]> wrote:
>
>> Hi Navnath
>>

-- 
Thanks & Regards,
Shubham Agarwal
EnterpriseDB Corporation

The Postgres Database Company

---

From: Yogesh Mahajan <[email-masked]>
Date: Fri, 1 May 2020 12:27:44 +0530

Hi,

Please find updated patch modified according to review comments -
Patch implements below things -
1.Enable the current framework to provide option to execute Feature tests
in parallel  on selenium grid set up.
   - Addition of new switch to start parallel features tests.
   - New parameters with respect to selenoid in test_config.json.in
   - Addition of new script to check solenoid updates.

Thanks,
Yogesh Mahajan
QA - Team
EnterpriseDB Corporation

Phone: +91-9741705709

On Tue, Apr 21, 2020 at 1:18 PM Shubham Agarwal <
[email-masked]> wrote:

> b. line number 447 -> The drop_database function will only try to drop
> the database with the name which is newly created
> 6. Provide the valid docstring in newly introduced functions and also
> valid comments while calling it. for ex.- _update_preference() function is
> 7. test_index_constraint_add test case is failing due to the latest
> change, please merge and update this test case
>

---

From: Akshay Joshi <[email-masked]>
Date: Mon, 4 May 2020 14:51:31 +0530

Hi Yogesh

The patch is not applied to the master branch. Can you please rebase and
send the patch again.

On Fri, May 1, 2020 at 12:28 PM Yogesh Mahajan <
[email-masked]> wrote:

>>     a. The exception traceback logic at line number 653 in runtests.py
>> is not correct since it is particular to the thread
>>>>> To sun feature tests in parallel, required prerequisites can be
>>>>> checked in '~/web/regression/README' file.

-- 
*Thanks & Regards*
*Akshay Joshi*

*Sr. Software Architect*

---

From: Yogesh Mahajan <[email-masked]>
Date: Mon, 4 May 2020 16:27:10 +0530

Hi Akshay,

Please find the updated patch.

Thanks,
Yogesh Mahajan
QA - Team
EnterpriseDB Corporation

Phone: +91-9741705709

On Mon, May 4, 2020 at 2:51 PM Akshay Joshi <[email-masked]>
wrote:

>>> at 431 line number, its probability is 1% instead of this you can write
>>> a logic so that it will drop all the database which starts with name
>>> ‘acceptance_test_db'.

---

From: Akshay Joshi <[email-masked]>
Date: Tue, 5 May 2020 15:58:10 +0530

Hi Yogesh

Following are the review comments:

   - *pyjq* package is not required as we used it only in one place. A
   result is a normal dictionary that can be easily looped through.
   - Remove "*if (SUPPORT_SSH_TUNNEL is True and ...*" logic from
   config.py, we have already removed that.
   - Remove yarn.lock file.
   - Remove *pyperclip *from the regression/requirements.txt as we are not
   using it.
   - Please mentioned the value of *pgAdmin_default_server *should not be '
   *127.0.0.1*' in the README file even though everything runs on the same
   machine.
   - Please mentioned that if we set the value of the browser version is
   *null* then selenoid will take the latest available browser version.
   - Got the below error if selenoid_url is not provided:
      - list index out of range
      Unable to find Selenoid Status

*test_config.json.in <http://test_config.json.in>*:

   - "selenoid_info" should be renamed to "selenoid_config". Proper
   alignment is required.
   - "cross_Browsers" should be renamed to "cross_browsers" or
   "run_on_browsers" or "run_tests_on_browsers". Provide entries for supported
   browsers with version set to null so that it will run on the latest browser
   version.
   - "selenoid_url": "Selenoid Url" should be changed to "selenoid_url":
   "http://<IP address of Selenoid Installed machine>:4444/wd/hub".

If you change the names in test_config.json.in then please update the same
in README as well.

On Mon, May 4, 2020 at 4:27 PM Yogesh Mahajan <
[email-masked]> wrote:

>>> 1.Enable the current framework to provide option to execute Feature
>>> tests in parallel  on selenium grid set up.
>>>> 3. Execution logs are not printing as per the logic some time, I ran
>>>> the suite for two servers and attached are the execution logs.
>>>> 4. Readme -
>>>> introduced in pg_utilities_backup_restrore_test.py but from the
>>>> function name, it is not clear what preferences are going to update in it.
>>>>> I think I am not the right person to review this patch now as I
>>>>> already reviewed this code offline in the last week. I know the approached
>>>>> Yogesh has followed, also given some review comments on it.

-- 
*Thanks & Regards*
*Akshay Joshi*

*Sr. Software Architect*

---

From: Yogesh Mahajan <[email-masked]>
Date: Fri, 8 May 2020 13:35:23 +0530

Hi,

Please find the updates patch with above review comments.
Patch adds below functionality to existing framework -
1.Ability to run features in parallel using solenoid(selenium + docker).
     a.Selenoid setup steps are included in /regression/README
     b.'python runtests.py  --pkg feature_tests --parallel' will trigger
parallel feature tests.
2.Removes dependency for pyperclip python module.
3.New script in ../tools/update_selenoid_browsers.py updates browser
images at local selneoid server setup.

Thanks,
Yogesh Mahajan
QA - Team
EnterpriseDB Corporation

Phone: +91-9741705709

On Tue, May 5, 2020 at 3:58 PM Akshay Joshi <[email-masked]>
wrote:

>    - Remove *pyperclip *from the regression/requirements.txt as we are
>    not using it.
>    - Please mentioned the value of *pgAdmin_default_server *should not be
>    '*127.0.0.1*' in the README file even though everything runs on the
>    same machine.
> If you change the names in test_config.json.in then please update the
> same in README as well.
>
>
>> On Mon, May 4, 2020 at 2:51 PM Akshay Joshi <
>>>>>     a. The exception traceback logic at line number 653 in
>>>>> runtests.py is not correct since it is particular to the thread
>>>>> b. line number 447 -> The drop_database function will only try to
>>>>> drop the database with the name which is newly created
>>>>> at 431 line number, its probability is 1% instead of this you can
>>>>> write a logic so that it will drop all the database which starts with name
>>>>> ‘acceptance_test_db'.
>>>>>     b. Create separate functions to instantiate the firefox driver
>>>>> and chrome driver logic since the same code is used in multiple files.

---

From: Akshay Joshi <[email-masked]>
Date: Fri, 8 May 2020 13:57:44 +0530

Hi Yogesh

You forgot to attach patch :)

On Fri, May 8, 2020 at 1:35 PM Yogesh Mahajan <
[email-masked]> wrote:

>>    - Please mentioned the value of *pgAdmin_default_server *should not
>>    be '*127.0.0.1*' in the README file even though everything runs on
>>    the same machine.
>>>> The patch is not applied to the master branch. Can you please rebase
>>>> and send the patch again.
>>>>
>>>>>>   c. line 584 - Why we are including resql test case execution in
>>>>>> GUI execution logic.
>>>>>>     a. Remove the headless chrome code from get_remote_webdriver()
>>>>>> in test_utils.py since we are using solenoid and it is not required
>>>>>> anymore.

-- 
*Thanks & Regards*
*Akshay Joshi*

*Sr. Software Architect*

---

From: Yogesh Mahajan <[email-masked]>
Date: Fri, 8 May 2020 14:05:18 +0530

Attaching patch

Thanks,
Yogesh Mahajan
QA - Team
EnterpriseDB Corporation

Phone: +91-9741705709

On Fri, May 8, 2020 at 1:57 PM Akshay Joshi <[email-masked]>
wrote:

>> On Tue, May 5, 2020 at 3:58 PM Akshay Joshi <
>>>    - Please mentioned that if we set the value of the browser version
>>>    is *null* then selenoid will take the latest available browser
>>>    version.
>>>    - "selenoid_url": "Selenoid Url" should be changed
>>>    to "selenoid_url": "http://<IP address of Selenoid Installed
>>>    machine>:4444/wd/hub".
>>>
>>>>>>> c. launch_url_in_browser() -> you can simplify the definition of
>>>>>>> the function like:
>>>>>>>     retry = 60
>>>>>>>         *while *retry > 0:
>>>>>>>             try:
>>>>>>>                 driver.get(url)
>>>>>>> Create the function to avoid duplicate code. The code for pasting
>>>>>>> the values is repeating 8 times in the test code.
>>>>>>> 6. Provide the valid docstring in newly introduced functions and
>>>>>>> also valid comments while calling it. for ex.- _update_preference()
>>>>>>> function is
>>>>>>>>>> Please find the attached patch for running *features tests*
>>>>>>>>>> using solenoid(selenium grid + docker).
>>>>>>>>>> KIndly review.

---

From: Akshay Joshi <[email-masked]>
Date: Fri, 8 May 2020 14:35:14 +0530

Hi Yogesh

Unable to apply the patch on the latest code. Please rebase and send it
again.

On Fri, May 8, 2020 at 2:05 PM Yogesh Mahajan <
[email-masked]> wrote:

>>>>>>>> in test_utils.py since we are using solenoid and it is not
>>>>>>>> required anymore.
>>>>>>>> 3. Execution logs are not printing as per the logic some time, I
>>>>>>>> ran the suite for two servers and attached are the execution logs.
>>>>>>>> 4. Readme -
>>>>>>>> 7. test_index_constraint_add test case is failing due to the
>>>>>>>> latest change, please merge and update this test case
>>>>>>>>

-- 
*Thanks & Regards*
*Akshay Joshi*

*Sr. Software Architect*

---

From: Yogesh Mahajan <[email-masked]>
Date: Fri, 8 May 2020 15:12:50 +0530

Hi Akshay,

Please find the attached rebase patch. Looks like there were some commits
after my today's git pull.

Thanks,
Yogesh Mahajan
QA - Team
EnterpriseDB Corporation

Phone: +91-9741705709

On Fri, May 8, 2020 at 2:35 PM Akshay Joshi <[email-masked]>
wrote:

>> On Fri, May 8, 2020 at 1:57 PM Akshay Joshi <
>>>>      b.'python runtests.py  --pkg feature_tests --parallel' will
>>>> trigger parallel feature tests.
>>>>>    - *pyjq* package is not required as we used it only in one place.
>>>>>    A result is a normal dictionary that can be easily looped through.
>>>>>    - Remove *pyperclip *from the regression/requirements.txt as we
>>>>>    are not using it.
>>>>>    - Please mentioned the value of *pgAdmin_default_server *should
>>>>>    not be '*127.0.0.1*' in the README file even though everything
>>>>>    runs on the same machine.
>>>>>>>>>     a. Remove the headless chrome code from
>>>>>>>>> get_remote_webdriver() in test_utils.py since we are using
>>>>>>>>> solenoid and it is not required anymore.
>>>>>>>>>     b. Create separate functions to instantiate the firefox
>>>>>>>>> driver and chrome driver logic since the same code is used in multiple
>>>>>>>>> files.

---

From: Akshay Joshi <[email-masked]>
Date: Mon, 11 May 2020 12:12:08 +0530

Thanks, patch applied.

On Fri, May 8, 2020 at 3:13 PM Yogesh Mahajan <
[email-masked]> wrote:

>>>>>>    - Please mentioned that if we set the value of the browser
>>>>>>    version is *null* then selenoid will take the latest available
>>>>>>    browser version.
>>>>>> If you change the names in test_config.json.in then please update
>>>>>> the same in README as well.
>>>>>>
>>>>>>
>>>>>>>> The patch is not applied to the master branch. Can you please
>>>>>>>> rebase and send the patch again.
>>>>>>>>
>>>>>>>>> 1.Enable the current framework to provide option to execute
>>>>>>>>> Feature tests in parallel  on selenium grid set up.
>>>>>>>>>> b. line number 447 -> The drop_database function will only try
>>>>>>>>>> to drop the database with the name which is newly created
>>>>>>>>>>   c. line 584 - Why we are including resql test case execution
>>>>>>>>>> in GUI execution logic.
>>>>>>>>>>>>> To sun feature tests in parallel, required prerequisites can
>>>>>>>>>>>>> be checked in '~/web/regression/README' file.
>>>>>>>>>>>>> After applying the patch, any existing process for execution
>>>>>>>>>>>>> of API/Features tests remains the same.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Yogesh Mahajan
>>>>>>>>>>>>> QA - Team
>>>>>>>>>>>>> EnterpriseDB Corporation
>>>>>>>>>>>>>
>>>>>>>>>>>>> Phone: +91-9741705709
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> *Thanks & Regards*
>>>>>>>>>>>> *Akshay Joshi*
>>>>>>>>>>>>
>>>>>>>>>>>> *Sr. Software Architect*

-- 
*Thanks & Regards*
*Akshay Joshi*

*Sr. Software Architect*