Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jWyMV-0006QZ-2V for pgadmin-hackers@arkaria.postgresql.org; Fri, 08 May 2020 08:28:03 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1jWyMT-0005wz-US for pgadmin-hackers@arkaria.postgresql.org; Fri, 08 May 2020 08:28:01 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jWyMT-0005vX-H0 for pgadmin-hackers@lists.postgresql.org; Fri, 08 May 2020 08:28:01 +0000 Received: from mail-il1-x12f.google.com ([2607:f8b0:4864:20::12f]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1jWyMP-0004bp-P3 for pgadmin-hackers@postgresql.org; Fri, 08 May 2020 08:28:00 +0000 Received: by mail-il1-x12f.google.com with SMTP id f82so665373ilh.8 for ; Fri, 08 May 2020 01:27:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=yAvGsbGQ0H6YFR9hwYw0UmO+36fKY7PT+A/PmXLb4S4=; b=w/63yro1gmikCrNOivWRfpYeeENVsNcZ85EBmCUROKUBhguCWDUj16nedHn6LxIsRb bRVFA6/NXNUaXsZRfb3vLPeznVDqRcKuE8gc/CVme3D3MKKChS4yTCtqpCn2z/46ccKp 8ghyH9xw3LyJ/NDf18yonHbAnEhjJaY478dUwA7tFUFkz3EolP8OS4UPBinIDI1EoGNL PSGOhJwgOJTdLGsNqkzNRF2WWyaYF6+3LJPucSLGrYV3FP7SLvWFii+Bu5szUJ40Q70W d+K6llWf34kwiapmy7S81PYuQXssYznN/3SOZyJzQ4mpWBkemJocjw+8+guJD5GxmWQg Jq2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=yAvGsbGQ0H6YFR9hwYw0UmO+36fKY7PT+A/PmXLb4S4=; b=RcrW6pZE0TX0wPKt8PMowfOy3tfQoiqDQcWlujHEYCQAWvOLzM7KNIjd4qMGEVjpOY 3rGlPJNDTtWI0ecSHCp4Z3xAIrONJ8IHLrPdr+U9XlINos2+8E2pBSPNX05byMiXHqcb PfzqJB0jNtRIou0+xENdHK+vAVowN5ScfShdDCrY03ICyrstjUPPOXywEzZ+yD/UmbPV 0gITmHdfDsq8q5g2gQygi+Ym5O62ZSWiEIrp3rF5NBcxeVaGy4wVYwq+VJuqiQ5YmFgh L0c9hsafdkAKQSfhrUcGVJpmH/ZDMN/rHnaIObl2xY+gJC9M8bqY9Ch9PYOw1P2AtNac v3yw== X-Gm-Message-State: AGi0PubXIbcZuLY4VBtb2HiS1ErIw0iEtru2aX0BiWMsTCOj0EXQIaAO P6RcgBItZBWnIoFj1Ktrd87KMpC7sue6ktcGBEff7MrA9i0NWffyHqqWGvdsGOSSMmie2EUW4rY YMjYQOQDyQeD+hpOx3PegQdls5uLCoUq9/UYfVJIl4c6xclteStt0HGldpFoJPTDLvSSdKFILo1 bhFYTEHxyqJRqodQycBYIY4gc8ywaQ6/oiZgMbUmhdgxGc14jEcP+2ZUyNwQ== X-Google-Smtp-Source: APiQypJv7PoRksEiyR86HSaiNgWCfOSrOPLNYAMsfOMdpp/vsuILHfKVulUMx60iqdwz4NBfo0Zl3DIMq82F3ctJPO8= X-Received: by 2002:a05:6e02:80e:: with SMTP id u14mr1707328ilm.176.1588926475447; Fri, 08 May 2020 01:27:55 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Akshay Joshi Date: Fri, 8 May 2020 13:57:44 +0530 Message-ID: Subject: Re: [pgAdmin 4 - Housekeeping #5255] Implement Selenium Grid using multi-threading & solenoid using current test framework To: Yogesh Mahajan Cc: pgadmin-hackers , Shubham Agarwal , navnath gadakh Content-Type: multipart/alternative; boundary="00000000000024cdd205a51ec6de" X-CLOUD-SEC-AV-Info: enterprisedb,google_mail,monitor X-CLOUD-SEC-AV-Sent: true X-Gm-Spam: 0 X-Gm-Phishy: 0 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --00000000000024cdd205a51ec6de Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Yogesh You forgot to attach patch :) On Fri, May 8, 2020 at 1:35 PM Yogesh Mahajan < yogesh.mahajan@enterprisedb.com> wrote: > Hi, > > Please find the updates patch with above review comments. > Patch adds below functionality to existing framework - > 1.Ability to run features in parallel using solenoid(selenium + docker). > a.Selenoid setup steps are included in /regression/README > b.'python runtests.py --pkg feature_tests --parallel' will trigger > parallel feature tests. > 2.Removes dependency for pyperclip python module. > 3.New script in ../tools/update_selenoid_browsers.py updates browser > images at local selneoid server setup. > > > Thanks, > Yogesh Mahajan > QA - Team > EnterpriseDB Corporation > > Phone: +91-9741705709 > > > On Tue, May 5, 2020 at 3:58 PM Akshay Joshi > wrote: > >> Hi Yogesh >> >> Following are the review comments: >> >> - *pyjq* package is not required as we used it only in one place. A >> result is a normal dictionary that can be easily looped through. >> - Remove "*if (SUPPORT_SSH_TUNNEL is True and ...*" logic from >> config.py, we have already removed that. >> - Remove yarn.lock file. >> - Remove *pyperclip *from the regression/requirements.txt as we are >> not using it. >> - Please mentioned the value of *pgAdmin_default_server *should not >> be '*127.0.0.1*' in the README file even though everything runs on >> the same machine. >> - Please mentioned that if we set the value of the browser version is >> *null* then selenoid will take the latest available browser version. >> - Got the below error if selenoid_url is not provided: >> - list index out of range >> Unable to find Selenoid Status >> >> *test_config.json.in *: >> >> - "selenoid_info" should be renamed to "selenoid_config". Proper >> alignment is required. >> - "cross_Browsers" should be renamed to "cross_browsers" or >> "run_on_browsers" or "run_tests_on_browsers". Provide entries for sup= ported >> browsers with version set to null so that it will run on the latest b= rowser >> version. >> - "selenoid_url": "Selenoid Url" should be changed to "selenoid_url": >> "http://:4444/wd/hub". >> >> If you change the names in test_config.json.in then please update the >> same in README as well. >> >> >> On Mon, May 4, 2020 at 4:27 PM Yogesh Mahajan < >> yogesh.mahajan@enterprisedb.com> wrote: >> >>> Hi Akshay, >>> >>> Please find the updated patch. >>> >>> Thanks, >>> Yogesh Mahajan >>> QA - Team >>> EnterpriseDB Corporation >>> >>> Phone: +91-9741705709 >>> >>> >>> On Mon, May 4, 2020 at 2:51 PM Akshay Joshi < >>> akshay.joshi@enterprisedb.com> wrote: >>> >>>> Hi Yogesh >>>> >>>> The patch is not applied to the master branch. Can you please rebase >>>> and send the patch again. >>>> >>>> On Fri, May 1, 2020 at 12:28 PM Yogesh Mahajan < >>>> yogesh.mahajan@enterprisedb.com> wrote: >>>> >>>>> Hi, >>>>> >>>>> Please find updated patch modified according to review comments - >>>>> Patch implements below things - >>>>> 1.Enable the current framework to provide option to execute Feature >>>>> tests in parallel on selenium grid set up. >>>>> - Addition of new switch to start parallel features tests. >>>>> - New parameters with respect to selenoid in test_config.json.in >>>>> - Addition of new script to check solenoid updates. >>>>> >>>>> >>>>> >>>>> Thanks, >>>>> Yogesh Mahajan >>>>> QA - Team >>>>> EnterpriseDB Corporation >>>>> >>>>> Phone: +91-9741705709 >>>>> >>>>> >>>>> On Tue, Apr 21, 2020 at 1:18 PM Shubham Agarwal < >>>>> shubham.agarwal@enterprisedb.com> wrote: >>>>> >>>>>> Hi Yogesh, >>>>>> Below are the review comments- >>>>>> >>>>>> 1. runtests.py >>>>>> a. The exception traceback logic at line number 653 in >>>>>> runtests.py is not correct since it is particular to the thread >>>>>> but there is much more code in that block which can throw some >>>>>> exception. >>>>>> b. line number 447 -> The drop_database function will only try to >>>>>> drop the database with the name which is newly created >>>>>> at 431 line number, its probability is 1% instead of this you can >>>>>> write a logic so that it will drop all the database which starts wit= h name >>>>>> =E2=80=98acceptance_test_db'. >>>>>> c. line 584 - Why we are including resql test case execution in >>>>>> GUI execution logic. >>>>>> d. Change the function name run_test as script name is also >>>>>> runtests.py >>>>>> >>>>>> 2. test_utils.py >>>>>> a. Remove the headless chrome code from get_remote_webdriver() >>>>>> in test_utils.py since we are using solenoid and it is not required >>>>>> anymore. >>>>>> b. Create separate functions to instantiate the firefox driver >>>>>> and chrome driver logic since the same code is used in multiple file= s. >>>>>> c. launch_url_in_browser() -> you can simplify the definition of the >>>>>> function like: >>>>>> retry =3D 60 >>>>>> =EF=BB=BF *while *retry > 0: >>>>>> try: >>>>>> driver.get(url) >>>>>> except WebDriverException: >>>>>> retry -=3D 1 >>>>>> 3. Execution logs are not printing as per the logic some time, I ran >>>>>> the suite for two servers and attached are the execution logs. >>>>>> 4. Readme - >>>>>> Please provide the Valid selenoid URL to be provided in the >>>>>> test_config.json, with all the steps mentioned in the readme it is n= ot >>>>>> clear. >>>>>> Revisit the readme and write the missing steps. >>>>>> 5. copy_selected_query_results_feature_tests.py- >>>>>> Create the function to avoid duplicate code. The code for pasting th= e >>>>>> values is repeating 8 times in the test code. >>>>>> 6. Provide the valid docstring in newly introduced functions and als= o >>>>>> valid comments while calling it. for ex.- _update_preference() funct= ion is >>>>>> introduced in pg_utilities_backup_restrore_test.py but from the >>>>>> function name, it is not clear what preferences are going to update = in it. >>>>>> 7. test_index_constraint_add test case is failing due to the latest >>>>>> change, please merge and update this test case >>>>>> >>>>>> On Thu, Apr 16, 2020 at 2:41 PM navnath gadakh < >>>>>> navnath.gadakh@enterprisedb.com> wrote: >>>>>> >>>>>>> Hi, >>>>>>> I think I am not the right person to review this patch now as I >>>>>>> already reviewed this code offline in the last week. I know the app= roached >>>>>>> Yogesh has followed, also given some review comments on it. >>>>>>> Someone else please review it. >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> On Mon, Apr 13, 2020 at 2:49 PM Akshay Joshi < >>>>>>> akshay.joshi@enterprisedb.com> wrote: >>>>>>> >>>>>>>> Hi Navnath >>>>>>>> >>>>>>>> Can you please review it? >>>>>>>> >>>>>>>> On Mon, Apr 13, 2020 at 2:40 PM Yogesh Mahajan < >>>>>>>> yogesh.mahajan@enterprisedb.com> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Please find the attached patch for running *features tests* using >>>>>>>>> solenoid(selenium grid + docker). >>>>>>>>> KIndly review. >>>>>>>>> To sun feature tests in parallel, required prerequisites can be >>>>>>>>> checked in '~/web/regression/README' file. >>>>>>>>> Also detailed instructions are added in the same file. >>>>>>>>> After applying the patch, any existing process for execution of >>>>>>>>> API/Features tests remains the same. >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Yogesh Mahajan >>>>>>>>> QA - Team >>>>>>>>> EnterpriseDB Corporation >>>>>>>>> >>>>>>>>> Phone: +91-9741705709 >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> *Thanks & Regards* >>>>>>>> *Akshay Joshi* >>>>>>>> >>>>>>>> *Sr. Software Architect* >>>>>>>> *EnterpriseDB Software India Private Limited* >>>>>>>> *Mobile: +91 976-788-8246* >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Regards, >>>>>>> Navnath Gadakh >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Thanks & Regards, >>>>>> Shubham Agarwal >>>>>> EnterpriseDB Corporation >>>>>> >>>>>> The Postgres Database Company >>>>>> >>>>> >>>> >>>> -- >>>> *Thanks & Regards* >>>> *Akshay Joshi* >>>> >>>> *Sr. Software Architect* >>>> *EnterpriseDB Software India Private Limited* >>>> *Mobile: +91 976-788-8246* >>>> >>> >> >> -- >> *Thanks & Regards* >> *Akshay Joshi* >> >> *Sr. Software Architect* >> *EnterpriseDB Software India Private Limited* >> *Mobile: +91 976-788-8246* >> > --=20 *Thanks & Regards* *Akshay Joshi* *Sr. Software Architect* *EnterpriseDB Software India Private Limited* *Mobile: +91 976-788-8246* --00000000000024cdd205a51ec6de Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi=C2=A0Yogesh

You forgot to attach pat= ch :)

On Fri, May 8, 2020 at 1:35 PM Yogesh Mahajan <yogesh.mahajan@enterprisedb.com>= ; wrote:
Hi,

Please find the updates p= atch with above review comments.
Patch adds below functionality t= o existing framework -=C2=A0
1.Ability to run features in paralle= l using solenoid(selenium + docker).
=C2=A0 =C2=A0 =C2=A0a.Seleno= id=C2=A0setup steps are included in /regression/README
=C2=A0 =C2= =A0 =C2=A0b.'python runtests.py =C2=A0--pkg feature_tests --parallel= 9; will trigger parallel feature tests.
2.Removes dependency for = pyperclip python=C2=A0module.
3.New script in ../tools/update_sel= enoid_browsers.py updates=C2=A0browser images=C2=A0at local selneoid server= setup.


Thanks,
Yogesh Mahajan
QA - Team
EnterpriseDB Corporation

Phone: +91-9741705709


On Tue, May 5, 2020 at 3:58 PM Akshay Joshi <<= a href=3D"mailto:akshay.joshi@enterprisedb.com" target=3D"_blank">akshay.jo= shi@enterprisedb.com> wrote:
Hi=C2=A0Yogesh

Foll= owing are the review comments:
  • pyjq package i= s not required as we used it only in one place. A result is a normal dictio= nary that can be easily looped=C2=A0through.
  • Remove "if (SU= PPORT_SSH_TUNNEL is True and ..." logic from config.py, we have al= ready removed that.
  • Remove yarn.lock file.
  • Remove=C2=A0p= yperclip from the regression/requirements.txt as we are not using it.
  • Please mentioned the value of=C2=A0pgAdmin_default_server sho= uld not be '127.0.0.1' in the README file even though everyt= hing runs on the same machine.
  • Please mentioned that if we set the = value of the browser version is null then selenoid=C2=A0will take th= e latest available browser version.
  • Got the below error if selenoid= _url is not provided:
    • list index out of range
      Unable to find= Selenoid Status
  • &= quot;selenoid_info" should be renamed to "selenoid_config". = Proper alignment is required.
  • "cross_Browsers" should be = renamed to "cross_browsers" or "run_on_browsers" or &qu= ot;run_tests_on_browsers". Provide entries for supported browsers with= version set to null so that it will run on the latest browser version.
  • "selenoid_url": "Selenoid Url" should be changed t= o=C2=A0"selenoid_url": "http://<IP address of Selenoid In= stalled machine>:4444/wd/hub".
If you change the nam= es in test_config.= json.in then please update the same in README as well.

On Mon, May 4, 2020 at 4:27 PM Yogesh Mahajan <yogesh.mahajan@enterpri= sedb.com> wrote:
Hi Akshay,

Please find the upda= ted patch.

Thanks= ,
Yogesh Mahajan
QA - Team
EnterpriseDB Corpor= ation

Phone: +91-9741705709

<= /div>

On Mon, May 4, 2020 at 2:51 PM Akshay Joshi <akshay.joshi@enterprisedb.com<= /a>> wrote:
<= div dir=3D"ltr">Hi=C2=A0Yogesh

The patch is not applied = to the master branch. Can you please rebase and send the patch again.
=

= On Fri, May 1, 2020 at 12:28 PM Yogesh Mahajan <yogesh.mahajan@enterprisedb.co= m> wrote:
Hi,

Please find updated patch modified= according to review comments=C2=A0-=C2=A0
Patch implements below= things=C2=A0-=C2=A0
1.Enable the current framework to provide op= tion to execute Feature tests in parallel =C2=A0on selenium grid set up.
=C2=A0 =C2=A0- Addition of new switch to start parallel features te= sts.
=C2=A0 =C2=A0- New parameters with respect to selenoid in test_config.json.in<= /a>
=C2=A0 =C2=A0- Addition of new script to check solenoid=C2=A0= updates.



Thanks,
Yogesh Mahajan
QA - Te= am
EnterpriseDB Corporation

Phone: +91-9741705709


On Tue, Apr 21, 2020 at 1:18 PM Shubham= Agarwal <shubham.agarwal@enterprisedb.com> wrote:
<= font face=3D"arial, sans-serif">Hi Yogesh,
Below are the review comments-

1.= runtests.py
=C2=A0 =C2= =A0 a. The exception = traceback logic at line number 653 in runtests.py is not correct since it i= s particular to the thread
but= there is much more code in that block which can throw some exception.
b. line number 447 -> The drop_database function w= ill only try to drop the database with the name which is newly created
= at 431 line number, its probabilit= y is 1% instead of this you can write a logic so that it will drop all the = database which starts with name =E2=80=98acceptance_test_db'.
=C2=A0 c. line 584 -=C2= =A0Why we are including resql test case execution in GUI execution logic.
=C2=A0 =C2=A0 d.=C2=A0Change the function name run_tes= t as script name is also runtests.py

= 2. test_utils.py
=C2=A0 = =C2=A0 a.=C2=A0Remove the headless chrom= e code from get_remote_webdriver() in test_utils.py since we are using solenoid and it is not required anymore.
=C2=A0 =C2=A0 b. Create separate functions to instantiate the firefox driver and chrome= driver logic since the same code is used in multiple files.<= /div>
c. launch_url_in_browser() -> you can simplify the definition of th= e function like:
<= span style=3D"color:rgb(0,0,0);font-size:9pt">=C2=A0 =C2=A0=C2=A0ret= ry =3D 60
=EF=BB=BF=C2=A0 =C2=A0 =C2=A0 =C2=A0 while=C2=A0retry > 0:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 try:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 driver.get(url)
= =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 except WebDriverException:
=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0retry -=3D 1
3. Execution logs are not printing as per the logic some time, I ran = the suite for two servers and attached are the execution logs.
4. Readme -
= Please provide the Valid selenoid URL to be p= rovided in the test_config.json, with all the steps mentioned in the readme= it is not clear.
Rev= isit the readme and write the missing steps.
5. copy_selected_query_results_feature_tests.py-
Create the function to a= void duplicate code. The code for pasting the values is repeating 8 times i= n the test code.
6. Prov= ide the valid docstring in newly introduced functions and also valid commen= ts while calling it. for ex.- _update_preference() function is
introduced in pg_utilities_backup= _restrore_test.py but from the function name, it is not clear what preferen= ces are going to update in it.
7. test_inde= x_constraint_add test case is failing due to the latest change, please merg= e and update this test case

On Thu, Apr 16, 2020 at 2:41 PM navnath gadakh &= lt;nav= nath.gadakh@enterprisedb.com> wrote:
Hi,
I thin= k I am not the right person to review this patch now as I already reviewed = this code offline in the last week. I know the=C2=A0approached Yogesh has f= ollowed, also given some review comments on it.
Someone else please review it.=C2=A0

Thanks!
<= br>
= On Mon, Apr 13, 2020 at 2:49 PM Akshay Jos= hi <a= kshay.joshi@enterprisedb.com> wrote:
Hi Navnath

Can you please review it?

On Mon, Apr 13, 2020 at 2:40 PM Yogesh Mahajan <yogesh.mahajan@enterpris= edb.com> wrote:
Hi,

Please find the attached=C2=A0patch for r= unning features tests using solenoid(selenium=C2=A0grid + docker).
KIndly review.
To sun feature tests in parallel, re= quired prerequisites can be checked in '~/web/regression/README' fi= le.
Also detailed=C2=A0in= structions are added in the same=C2=A0file.
After applying the patch, any existing process for execu= tion of API/Features=C2=A0tests remains the same.


Thanks,
Yogesh Mahajan
QA= - Team
EnterpriseDB= Corporation

Phone: +91-9741705709


--
Thanks & Regards=
A= kshay Joshi
Sr. Software Architect
EnterpriseDB Software India Private Limited
Mobile: +91 976-788-8246


--
Regards,
Navnath = Gadakh


--
Thanks & Regards,
Shubham Agarwal
EnterpriseDB Corporation

The Postgres Database Company


--
Thanks & Regards
Akshay Joshi
<= font color=3D"#3333FF">Sr. Software Architect=
<= font color=3D"#000000" face=3D"arial, sans-serif">EnterpriseDB Software = India Private Limited
Mobile: +91 976-788-8246=


--
Thanks & Regards
Akshay Joshi
<= font color=3D"#3333FF">Sr. Software Architect=
<= font color=3D"#000000" face=3D"arial, sans-serif">EnterpriseDB Software = India Private Limited
Mobile: +91 976-788-8246=


--
Thanks & Regards
=
Akshay Joshi
Sr. Software Architect
EnterpriseDB Software India Private Limited
=
Mobile: += 91 976-788-8246
--00000000000024cdd205a51ec6de--