public inbox for [email protected]help / color / mirror / Atom feed
feature test timeouts 9+ messages / 4 participants [nested] [flat]
* feature test timeouts @ 2017-05-26 22:26 George Gelashvili <[email protected]> 0 siblings, 1 reply; 9+ messages in thread From: George Gelashvili @ 2017-05-26 22:26 UTC (permalink / raw) To: pgadmin-hackers Hello Hackers, We are running tests including feature tests in Circle CI, and have been seeing some flakiness due to what we believe are insufficient wait-for timeouts. This shouldn't increase the overall test time, but only allow waiting longer for an element to appear, if needed. We've also split out the app start timeout since that seemed to need more time on CI. Patch attached. Thanks! Joao, Matt, and George diff --git a/web/regression/feature_utils/pgadmin_page.py b/web/regression/feature_utils/pgadmin_page.py index b1dfa182..b9abe7ca 100644 --- a/web/regression/feature_utils/pgadmin_page.py +++ b/web/regression/feature_utils/pgadmin_page.py @@ -26,7 +26,8 @@ class PgadminPage: def __init__(self, driver, app_config): self.driver = driver self.app_config = app_config - self.timeout = 10 + self.timeout = 30 + self.app_start_timeout = 60 def reset_layout(self): self.click_element(self.find_by_partial_link_text("File")) @@ -178,8 +179,10 @@ class PgadminPage: driver.refresh() return False - self._wait_for("app to start", page_shows_app) + self._wait_for("app to start", page_shows_app, self.app_start_timeout) - def _wait_for(self, waiting_for_message, condition_met_function): - return WebDriverWait(self.driver, self.timeout, 0.01).until(condition_met_function, + def _wait_for(self, waiting_for_message, condition_met_function, timeout = None): + if timeout is None: + timeout = self.timeout + return WebDriverWait(self.driver, timeout, 0.01).until(condition_met_function, "Timed out waiting for " + waiting_for_message) -- Sent via pgadmin-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgadmin-hackers Attachments: [text/plain] change-some-feature-test-timeouts.diff (1.3K, 3-change-some-feature-test-timeouts.diff) download | inline diff: diff --git a/web/regression/feature_utils/pgadmin_page.py b/web/regression/feature_utils/pgadmin_page.py index b1dfa182..b9abe7ca 100644 --- a/web/regression/feature_utils/pgadmin_page.py +++ b/web/regression/feature_utils/pgadmin_page.py @@ -26,7 +26,8 @@ class PgadminPage: def __init__(self, driver, app_config): self.driver = driver self.app_config = app_config - self.timeout = 10 + self.timeout = 30 + self.app_start_timeout = 60 def reset_layout(self): self.click_element(self.find_by_partial_link_text("File")) @@ -178,8 +179,10 @@ class PgadminPage: driver.refresh() return False - self._wait_for("app to start", page_shows_app) + self._wait_for("app to start", page_shows_app, self.app_start_timeout) - def _wait_for(self, waiting_for_message, condition_met_function): - return WebDriverWait(self.driver, self.timeout, 0.01).until(condition_met_function, + def _wait_for(self, waiting_for_message, condition_met_function, timeout = None): + if timeout is None: + timeout = self.timeout + return WebDriverWait(self.driver, timeout, 0.01).until(condition_met_function, "Timed out waiting for " + waiting_for_message) ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: feature test timeouts @ 2017-05-27 18:17 Dave Page <[email protected]> parent: George Gelashvili <[email protected]> 0 siblings, 1 reply; 9+ messages in thread From: Dave Page @ 2017-05-27 18:17 UTC (permalink / raw) To: George Gelashvili <[email protected]>; +Cc: pgadmin-hackers Thanks, patch applied. On Fri, May 26, 2017 at 6:26 PM, George Gelashvili <[email protected]> wrote: > Hello Hackers, > > We are running tests including feature tests in Circle CI, and have been > seeing some flakiness due to what we believe are insufficient wait-for > timeouts. > > This shouldn't increase the overall test time, but only allow waiting longer > for an element to appear, if needed. > > We've also split out the app start timeout since that seemed to need more > time on CI. > > Patch attached. > > Thanks! > Joao, Matt, and George > > > -- > Sent via pgadmin-hackers mailing list ([email protected]) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgadmin-hackers > -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgadmin-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgadmin-hackers ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: feature test timeouts @ 2017-05-29 06:23 Murtuza Zabuawala <[email protected]> parent: Dave Page <[email protected]> 0 siblings, 1 reply; 9+ messages in thread From: Murtuza Zabuawala @ 2017-05-29 06:23 UTC (permalink / raw) To: Dave Page <[email protected]>; +Cc: George Gelashvili <[email protected]>; pgadmin-hackers Hi Dave, With latest pull, PG data-type test fails on my system, ====================================================================== ERROR: runTest (pgadmin.feature_tests.pg_datatype_validation_test. PGDataypeFeatureTest) Test checks for PG data-types output ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/ feature_tests/pg_datatype_validation_test.py", line 44, in runTest self._check_datatype() File "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/ feature_tests/pg_datatype_validation_test.py", line 97, in _check_datatype self.page.fill_codemirror_area_with(query) File "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 122, in fill_codemirror_area_with "//pre[contains(@class,'CodeMirror-line')]/../../../*[ contains(@class,'CodeMirror-code')]").click() File "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 87, in find_by_xpath return self.wait_for_element(lambda driver: driver.find_element_by_xpath(xpath)) File "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 152, in wait_for_element return self._wait_for("element to exist", element_if_it_exists) File "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 188, in _wait_for "Timed out waiting for " + waiting_for_message) File "/Users/edb/Documents/projects/venv_pg4/lib/python3. 5/site-packages/selenium/webdriver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions.TimeoutException: Message: Timed out waiting for element to exist ---------------------------------------------------------------------- Ran 6 tests in 170.026s FAILED (errors=1) ====================================================================== Test Result Summary ====================================================================== PG_95: 5 tests passed 1 test failed: PGDataypeFeatureTest (Test checks for PG data-types output) 0 tests skipped ====================================================================== Please check output in file: /Users/edb/Documents/projects/ pgadmin4/web/regression/regression.log -- Regards, Murtuza Zabuawala EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company On Sat, May 27, 2017 at 11:47 PM, Dave Page <[email protected]> wrote: > Thanks, patch applied. > > On Fri, May 26, 2017 at 6:26 PM, George Gelashvili > <[email protected]> wrote: > > Hello Hackers, > > > > We are running tests including feature tests in Circle CI, and have been > > seeing some flakiness due to what we believe are insufficient wait-for > > timeouts. > > > > This shouldn't increase the overall test time, but only allow waiting > longer > > for an element to appear, if needed. > > > > We've also split out the app start timeout since that seemed to need more > > time on CI. > > > > Patch attached. > > > > Thanks! > > Joao, Matt, and George > > > > > > -- > > Sent via pgadmin-hackers mailing list ([email protected]) > > To make changes to your subscription: > > http://www.postgresql.org/mailpref/pgadmin-hackers > > > > > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > > > -- > Sent via pgadmin-hackers mailing list ([email protected]) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgadmin-hackers > ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: feature test timeouts @ 2017-05-30 08:45 Dave Page <[email protected]> parent: Murtuza Zabuawala <[email protected]> 0 siblings, 1 reply; 9+ messages in thread From: Dave Page @ 2017-05-30 08:45 UTC (permalink / raw) To: Murtuza Zabuawala <[email protected]>; +Cc: George Gelashvili <[email protected]>; pgadmin-hackers Hi OK. Do you know why? What's it waiting for? It was working for me (I always run tests multiple times before committing them, to try to ensure they're reliable). On Mon, May 29, 2017 at 7:23 AM, Murtuza Zabuawala <[email protected]> wrote: > Hi Dave, > > With latest pull, PG data-type test fails on my system, > > ====================================================================== > ERROR: runTest > (pgadmin.feature_tests.pg_datatype_validation_test.PGDataypeFeatureTest) > Test checks for PG data-types output > ---------------------------------------------------------------------- > Traceback (most recent call last): > File > "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/feature_tests/pg_datatype_validation_test.py", > line 44, in runTest > self._check_datatype() > File > "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/feature_tests/pg_datatype_validation_test.py", > line 97, in _check_datatype > self.page.fill_codemirror_area_with(query) > File > "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/pgadmin_page.py", > line 122, in fill_codemirror_area_with > > "//pre[contains(@class,'CodeMirror-line')]/../../../*[contains(@class,'CodeMirror-code')]").click() > File > "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/pgadmin_page.py", > line 87, in find_by_xpath > return self.wait_for_element(lambda driver: > driver.find_element_by_xpath(xpath)) > File > "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/pgadmin_page.py", > line 152, in wait_for_element > return self._wait_for("element to exist", element_if_it_exists) > File > "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/pgadmin_page.py", > line 188, in _wait_for > "Timed out waiting for " + waiting_for_message) > File > "/Users/edb/Documents/projects/venv_pg4/lib/python3.5/site-packages/selenium/webdriver/support/wait.py", > line 80, in until > raise TimeoutException(message, screen, stacktrace) > selenium.common.exceptions.TimeoutException: Message: Timed out waiting for > element to exist > > > ---------------------------------------------------------------------- > Ran 6 tests in 170.026s > > FAILED (errors=1) > > ====================================================================== > Test Result Summary > ====================================================================== > > PG_95: > > 5 tests passed > 1 test failed: > PGDataypeFeatureTest (Test checks for PG data-types output) > 0 tests skipped > > ====================================================================== > > Please check output in file: > /Users/edb/Documents/projects/pgadmin4/web/regression/regression.log > > > -- > Regards, > Murtuza Zabuawala > EnterpriseDB: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > > On Sat, May 27, 2017 at 11:47 PM, Dave Page <[email protected]> wrote: >> >> Thanks, patch applied. >> >> On Fri, May 26, 2017 at 6:26 PM, George Gelashvili >> <[email protected]> wrote: >> > Hello Hackers, >> > >> > We are running tests including feature tests in Circle CI, and have been >> > seeing some flakiness due to what we believe are insufficient wait-for >> > timeouts. >> > >> > This shouldn't increase the overall test time, but only allow waiting >> > longer >> > for an element to appear, if needed. >> > >> > We've also split out the app start timeout since that seemed to need >> > more >> > time on CI. >> > >> > Patch attached. >> > >> > Thanks! >> > Joao, Matt, and George >> > >> > >> > -- >> > Sent via pgadmin-hackers mailing list ([email protected]) >> > To make changes to your subscription: >> > http://www.postgresql.org/mailpref/pgadmin-hackers >> > >> >> >> >> -- >> Dave Page >> Blog: http://pgsnake.blogspot.com >> Twitter: @pgsnake >> >> EnterpriseDB UK: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> >> >> -- >> Sent via pgadmin-hackers mailing list ([email protected]) >> To make changes to your subscription: >> http://www.postgresql.org/mailpref/pgadmin-hackers > > -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgadmin-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgadmin-hackers ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: feature test timeouts @ 2017-05-30 10:43 Murtuza Zabuawala <[email protected]> parent: Dave Page <[email protected]> 0 siblings, 1 reply; 9+ messages in thread From: Murtuza Zabuawala @ 2017-05-30 10:43 UTC (permalink / raw) To: Dave Page <[email protected]>; +Cc: George Gelashvili <[email protected]>; pgadmin-hackers Hi Dave, I didn't look into the code for it but every time I run the tests it stays on below screen and fails. @Khushboo, Any suggestion? [image: Inline image 1] -- Regards, Murtuza Zabuawala EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company On Tue, May 30, 2017 at 2:15 PM, Dave Page <[email protected]> wrote: > Hi > > OK. Do you know why? What's it waiting for? It was working for me (I > always run tests multiple times before committing them, to try to > ensure they're reliable). > > On Mon, May 29, 2017 at 7:23 AM, Murtuza Zabuawala > <[email protected]> wrote: > > Hi Dave, > > > > With latest pull, PG data-type test fails on my system, > > > > ====================================================================== > > ERROR: runTest > > (pgadmin.feature_tests.pg_datatype_validation_test.PGDataypeFeatureTest) > > Test checks for PG data-types output > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File > > "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/ > feature_tests/pg_datatype_validation_test.py", > > line 44, in runTest > > self._check_datatype() > > File > > "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/ > feature_tests/pg_datatype_validation_test.py", > > line 97, in _check_datatype > > self.page.fill_codemirror_area_with(query) > > File > > "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/ > pgadmin_page.py", > > line 122, in fill_codemirror_area_with > > > > "//pre[contains(@class,'CodeMirror-line')]/../../../*[ > contains(@class,'CodeMirror-code')]").click() > > File > > "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/ > pgadmin_page.py", > > line 87, in find_by_xpath > > return self.wait_for_element(lambda driver: > > driver.find_element_by_xpath(xpath)) > > File > > "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/ > pgadmin_page.py", > > line 152, in wait_for_element > > return self._wait_for("element to exist", element_if_it_exists) > > File > > "/Users/edb/Documents/projects/pgadmin4/web/regression/feature_utils/ > pgadmin_page.py", > > line 188, in _wait_for > > "Timed out waiting for " + waiting_for_message) > > File > > "/Users/edb/Documents/projects/venv_pg4/lib/python3. > 5/site-packages/selenium/webdriver/support/wait.py", > > line 80, in until > > raise TimeoutException(message, screen, stacktrace) > > selenium.common.exceptions.TimeoutException: Message: Timed out waiting > for > > element to exist > > > > > > ---------------------------------------------------------------------- > > Ran 6 tests in 170.026s > > > > FAILED (errors=1) > > > > ====================================================================== > > Test Result Summary > > ====================================================================== > > > > PG_95: > > > > 5 tests passed > > 1 test failed: > > PGDataypeFeatureTest (Test checks for PG data-types output) > > 0 tests skipped > > > > ====================================================================== > > > > Please check output in file: > > /Users/edb/Documents/projects/pgadmin4/web/regression/regression.log > > > > > > -- > > Regards, > > Murtuza Zabuawala > > EnterpriseDB: http://www.enterprisedb.com > > The Enterprise PostgreSQL Company > > > > On Sat, May 27, 2017 at 11:47 PM, Dave Page <[email protected]> wrote: > >> > >> Thanks, patch applied. > >> > >> On Fri, May 26, 2017 at 6:26 PM, George Gelashvili > >> <[email protected]> wrote: > >> > Hello Hackers, > >> > > >> > We are running tests including feature tests in Circle CI, and have > been > >> > seeing some flakiness due to what we believe are insufficient wait-for > >> > timeouts. > >> > > >> > This shouldn't increase the overall test time, but only allow waiting > >> > longer > >> > for an element to appear, if needed. > >> > > >> > We've also split out the app start timeout since that seemed to need > >> > more > >> > time on CI. > >> > > >> > Patch attached. > >> > > >> > Thanks! > >> > Joao, Matt, and George > >> > > >> > > >> > -- > >> > Sent via pgadmin-hackers mailing list ([email protected] > ) > >> > To make changes to your subscription: > >> > http://www.postgresql.org/mailpref/pgadmin-hackers > >> > > >> > >> > >> > >> -- > >> Dave Page > >> Blog: http://pgsnake.blogspot.com > >> Twitter: @pgsnake > >> > >> EnterpriseDB UK: http://www.enterprisedb.com > >> The Enterprise PostgreSQL Company > >> > >> > >> -- > >> Sent via pgadmin-hackers mailing list ([email protected]) > >> To make changes to your subscription: > >> http://www.postgresql.org/mailpref/pgadmin-hackers > > > > > > > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > Attachments: [image/png] image.png (353.2K, 3-image.png) download | view image ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: feature test timeouts @ 2017-05-30 10:52 Khushboo Vashi <[email protected]> parent: Murtuza Zabuawala <[email protected]> 0 siblings, 1 reply; 9+ messages in thread From: Khushboo Vashi @ 2017-05-30 10:52 UTC (permalink / raw) To: Murtuza Zabuawala <[email protected]>; +Cc: Dave Page <[email protected]>; George Gelashvili <[email protected]>; pgadmin-hackers Hi Murtuza, On Tue, May 30, 2017 at 4:13 PM, Murtuza Zabuawala < [email protected]> wrote: > Hi Dave, > > I didn't look into the code for it but every time I run the tests it stays > on below screen and fails. > > @Khushboo, > Any suggestion? > > Harshal is also getting the same issue with his feature test case. The problem we found that, in the bound time the driver is not able to find the code-mirror. Me and Harshal looked into this yesterday but couldn't find the solution. BTW, I can not reproduce this issue at my end but me and Harshal will look into this issue on his machine. Thanks, Khushboo > [image: Inline image 1] > > -- > Regards, > Murtuza Zabuawala > EnterpriseDB: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > > On Tue, May 30, 2017 at 2:15 PM, Dave Page <[email protected]> wrote: > >> Hi >> >> OK. Do you know why? What's it waiting for? It was working for me (I >> always run tests multiple times before committing them, to try to >> ensure they're reliable). >> >> On Mon, May 29, 2017 at 7:23 AM, Murtuza Zabuawala >> <[email protected]> wrote: >> > Hi Dave, >> > >> > With latest pull, PG data-type test fails on my system, >> > >> > ====================================================================== >> > ERROR: runTest >> > (pgadmin.feature_tests.pg_datatype_validation_test.PGDataype >> FeatureTest) >> > Test checks for PG data-types output >> > ---------------------------------------------------------------------- >> > Traceback (most recent call last): >> > File >> > "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/feature_ >> tests/pg_datatype_validation_test.py", >> > line 44, in runTest >> > self._check_datatype() >> > File >> > "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/feature_ >> tests/pg_datatype_validation_test.py", >> > line 97, in _check_datatype >> > self.page.fill_codemirror_area_with(query) >> > File >> > "/Users/edb/Documents/projects/pgadmin4/web/regression/ >> feature_utils/pgadmin_page.py", >> > line 122, in fill_codemirror_area_with >> > >> > "//pre[contains(@class,'CodeMirror-line')]/../../../*[contai >> ns(@class,'CodeMirror-code')]").click() >> > File >> > "/Users/edb/Documents/projects/pgadmin4/web/regression/ >> feature_utils/pgadmin_page.py", >> > line 87, in find_by_xpath >> > return self.wait_for_element(lambda driver: >> > driver.find_element_by_xpath(xpath)) >> > File >> > "/Users/edb/Documents/projects/pgadmin4/web/regression/ >> feature_utils/pgadmin_page.py", >> > line 152, in wait_for_element >> > return self._wait_for("element to exist", element_if_it_exists) >> > File >> > "/Users/edb/Documents/projects/pgadmin4/web/regression/ >> feature_utils/pgadmin_page.py", >> > line 188, in _wait_for >> > "Timed out waiting for " + waiting_for_message) >> > File >> > "/Users/edb/Documents/projects/venv_pg4/lib/python3.5/site- >> packages/selenium/webdriver/support/wait.py", >> > line 80, in until >> > raise TimeoutException(message, screen, stacktrace) >> > selenium.common.exceptions.TimeoutException: Message: Timed out >> waiting for >> > element to exist >> > >> > >> > ---------------------------------------------------------------------- >> > Ran 6 tests in 170.026s >> > >> > FAILED (errors=1) >> > >> > ====================================================================== >> > Test Result Summary >> > ====================================================================== >> > >> > PG_95: >> > >> > 5 tests passed >> > 1 test failed: >> > PGDataypeFeatureTest (Test checks for PG data-types output) >> > 0 tests skipped >> > >> > ====================================================================== >> > >> > Please check output in file: >> > /Users/edb/Documents/projects/pgadmin4/web/regression/regression.log >> > >> > >> > -- >> > Regards, >> > Murtuza Zabuawala >> > EnterpriseDB: http://www.enterprisedb.com >> > The Enterprise PostgreSQL Company >> > >> > On Sat, May 27, 2017 at 11:47 PM, Dave Page <[email protected]> wrote: >> >> >> >> Thanks, patch applied. >> >> >> >> On Fri, May 26, 2017 at 6:26 PM, George Gelashvili >> >> <[email protected]> wrote: >> >> > Hello Hackers, >> >> > >> >> > We are running tests including feature tests in Circle CI, and have >> been >> >> > seeing some flakiness due to what we believe are insufficient >> wait-for >> >> > timeouts. >> >> > >> >> > This shouldn't increase the overall test time, but only allow waiting >> >> > longer >> >> > for an element to appear, if needed. >> >> > >> >> > We've also split out the app start timeout since that seemed to need >> >> > more >> >> > time on CI. >> >> > >> >> > Patch attached. >> >> > >> >> > Thanks! >> >> > Joao, Matt, and George >> >> > >> >> > >> >> > -- >> >> > Sent via pgadmin-hackers mailing list ([email protected] >> g) >> >> > To make changes to your subscription: >> >> > http://www.postgresql.org/mailpref/pgadmin-hackers >> >> > >> >> >> >> >> >> >> >> -- >> >> Dave Page >> >> Blog: http://pgsnake.blogspot.com >> >> Twitter: @pgsnake >> >> >> >> EnterpriseDB UK: http://www.enterprisedb.com >> >> The Enterprise PostgreSQL Company >> >> >> >> >> >> -- >> >> Sent via pgadmin-hackers mailing list ([email protected]) >> >> To make changes to your subscription: >> >> http://www.postgresql.org/mailpref/pgadmin-hackers >> > >> > >> >> >> >> -- >> Dave Page >> Blog: http://pgsnake.blogspot.com >> Twitter: @pgsnake >> >> EnterpriseDB UK: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> > > Attachments: [image/png] image.png (353.2K, 3-image.png) download | view image ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: feature test timeouts @ 2017-06-05 20:58 George Gelashvili <[email protected]> parent: Khushboo Vashi <[email protected]> 0 siblings, 2 replies; 9+ messages in thread From: George Gelashvili @ 2017-06-05 20:58 UTC (permalink / raw) To: Khushboo Vashi <[email protected]>; +Cc: Murtuza Zabuawala <[email protected]>; Dave Page <[email protected]>; pgadmin-hackers We fixed the codemirror-filling helper function method thing in `pgadmin_page`. This should resolve the PG datatype feature test flakiness and make for a nice buttery crust. It looked like selenium was unable to find an element within the sqleditor iframe, so we made a workaround that re-locates the iframe and codemirror element each iteration of the wait. Thanks Joao & George On Tue, May 30, 2017 at 6:52 AM, Khushboo Vashi < [email protected]> wrote: > Hi Murtuza, > > On Tue, May 30, 2017 at 4:13 PM, Murtuza Zabuawala <murtuza.zabuawala@ > enterprisedb.com> wrote: > >> Hi Dave, >> >> I didn't look into the code for it but every time I run the tests it >> stays on below screen and fails. >> >> @Khushboo, >> Any suggestion? >> >> > Harshal is also getting the same issue with his feature test case. The > problem we found that, in the bound time the driver is not able to find the > code-mirror. Me and Harshal looked into this yesterday but couldn't find > the solution. > > BTW, I can not reproduce this issue at my end but me and Harshal will look > into this issue on his machine. > > > Thanks, > Khushboo > > >> [image: Inline image 1] >> >> -- >> Regards, >> Murtuza Zabuawala >> EnterpriseDB: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> >> On Tue, May 30, 2017 at 2:15 PM, Dave Page <[email protected]> wrote: >> >>> Hi >>> >>> OK. Do you know why? What's it waiting for? It was working for me (I >>> always run tests multiple times before committing them, to try to >>> ensure they're reliable). >>> >>> On Mon, May 29, 2017 at 7:23 AM, Murtuza Zabuawala >>> <[email protected]> wrote: >>> > Hi Dave, >>> > >>> > With latest pull, PG data-type test fails on my system, >>> > >>> > ====================================================================== >>> > ERROR: runTest >>> > (pgadmin.feature_tests.pg_datatype_validation_test.PGDataype >>> FeatureTest) >>> > Test checks for PG data-types output >>> > ---------------------------------------------------------------------- >>> > Traceback (most recent call last): >>> > File >>> > "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/feature_ >>> tests/pg_datatype_validation_test.py", >>> > line 44, in runTest >>> > self._check_datatype() >>> > File >>> > "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/feature_ >>> tests/pg_datatype_validation_test.py", >>> > line 97, in _check_datatype >>> > self.page.fill_codemirror_area_with(query) >>> > File >>> > "/Users/edb/Documents/projects/pgadmin4/web/regression/featu >>> re_utils/pgadmin_page.py", >>> > line 122, in fill_codemirror_area_with >>> > >>> > "//pre[contains(@class,'CodeMirror-line')]/../../../*[contai >>> ns(@class,'CodeMirror-code')]").click() >>> > File >>> > "/Users/edb/Documents/projects/pgadmin4/web/regression/featu >>> re_utils/pgadmin_page.py", >>> > line 87, in find_by_xpath >>> > return self.wait_for_element(lambda driver: >>> > driver.find_element_by_xpath(xpath)) >>> > File >>> > "/Users/edb/Documents/projects/pgadmin4/web/regression/featu >>> re_utils/pgadmin_page.py", >>> > line 152, in wait_for_element >>> > return self._wait_for("element to exist", element_if_it_exists) >>> > File >>> > "/Users/edb/Documents/projects/pgadmin4/web/regression/featu >>> re_utils/pgadmin_page.py", >>> > line 188, in _wait_for >>> > "Timed out waiting for " + waiting_for_message) >>> > File >>> > "/Users/edb/Documents/projects/venv_pg4/lib/python3.5/site-p >>> ackages/selenium/webdriver/support/wait.py", >>> > line 80, in until >>> > raise TimeoutException(message, screen, stacktrace) >>> > selenium.common.exceptions.TimeoutException: Message: Timed out >>> waiting for >>> > element to exist >>> > >>> > >>> > ---------------------------------------------------------------------- >>> > Ran 6 tests in 170.026s >>> > >>> > FAILED (errors=1) >>> > >>> > ====================================================================== >>> > Test Result Summary >>> > ====================================================================== >>> > >>> > PG_95: >>> > >>> > 5 tests passed >>> > 1 test failed: >>> > PGDataypeFeatureTest (Test checks for PG data-types output) >>> > 0 tests skipped >>> > >>> > ====================================================================== >>> > >>> > Please check output in file: >>> > /Users/edb/Documents/projects/pgadmin4/web/regression/regression.log >>> > >>> > >>> > -- >>> > Regards, >>> > Murtuza Zabuawala >>> > EnterpriseDB: http://www.enterprisedb.com >>> > The Enterprise PostgreSQL Company >>> > >>> > On Sat, May 27, 2017 at 11:47 PM, Dave Page <[email protected]> wrote: >>> >> >>> >> Thanks, patch applied. >>> >> >>> >> On Fri, May 26, 2017 at 6:26 PM, George Gelashvili >>> >> <[email protected]> wrote: >>> >> > Hello Hackers, >>> >> > >>> >> > We are running tests including feature tests in Circle CI, and have >>> been >>> >> > seeing some flakiness due to what we believe are insufficient >>> wait-for >>> >> > timeouts. >>> >> > >>> >> > This shouldn't increase the overall test time, but only allow >>> waiting >>> >> > longer >>> >> > for an element to appear, if needed. >>> >> > >>> >> > We've also split out the app start timeout since that seemed to need >>> >> > more >>> >> > time on CI. >>> >> > >>> >> > Patch attached. >>> >> > >>> >> > Thanks! >>> >> > Joao, Matt, and George >>> >> > >>> >> > >>> >> > -- >>> >> > Sent via pgadmin-hackers mailing list ( >>> [email protected]) >>> >> > To make changes to your subscription: >>> >> > http://www.postgresql.org/mailpref/pgadmin-hackers >>> >> > >>> >> >>> >> >>> >> >>> >> -- >>> >> Dave Page >>> >> Blog: http://pgsnake.blogspot.com >>> >> Twitter: @pgsnake >>> >> >>> >> EnterpriseDB UK: http://www.enterprisedb.com >>> >> The Enterprise PostgreSQL Company >>> >> >>> >> >>> >> -- >>> >> Sent via pgadmin-hackers mailing list ([email protected] >>> ) >>> >> To make changes to your subscription: >>> >> http://www.postgresql.org/mailpref/pgadmin-hackers >>> > >>> > >>> >>> >>> >>> -- >>> Dave Page >>> Blog: http://pgsnake.blogspot.com >>> Twitter: @pgsnake >>> >>> EnterpriseDB UK: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >> >> > diff --git a/web/pgadmin/feature_tests/pg_datatype_validation_test.py b/web/pgadmin/feature_tests/pg_datatype_validation_test.py index fdc65a58..69b12f30 100644 --- a/web/pgadmin/feature_tests/pg_datatype_validation_test.py +++ b/web/pgadmin/feature_tests/pg_datatype_validation_test.py @@ -6,7 +6,6 @@ # This software is released under the PostgreSQL Licence # ########################################################################## - from selenium.webdriver import ActionChains from selenium.common.exceptions import TimeoutException from selenium.webdriver.support.ui import WebDriverWait @@ -90,30 +89,28 @@ class PGDataypeFeatureTest(BaseFeatureTest): self.page.driver.find_element_by_link_text("Tools").click() self.page.find_by_partial_link_text("Query Tool").click() + + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() wait = WebDriverWait(self.page.driver, 5) - element = wait.until(EC.frame_to_be_available_and_switch_to_it((By.TAG_NAME, 'iframe'))) - - if element: - self.page.fill_codemirror_area_with(query) - self.page.find_by_id("btn-flash").click() - wait.until(EC.presence_of_element_located( - (By.XPATH, "//*[@id='0']//*[@id='datagrid']/div[5]/div/div[1]/div[2]/span"))) - - # For every sample data-type value, check the expected output. - cnt = 2 - for val in expected_output: - try: - source_code = self.page.find_by_xpath( - "//*[@id='0']//*[@id='datagrid']/div[5]/div/div[1]/div[" + str(cnt) + "]/span" - ).get_attribute('innerHTML') - - PGDataypeFeatureTest.check_result( - source_code, - expected_output[cnt - 2] - ) - cnt += 1 - except TimeoutException: - assert False, "{0} does not match with {1}".format(val, expected_output[cnt]) + wait.until(EC.presence_of_element_located( + (By.XPATH, "//*[@id='0']//*[@id='datagrid']/div[5]/div/div[1]/div[2]/span"))) + + # For every sample data-type value, check the expected output. + cnt = 2 + for val in expected_output: + try: + source_code = self.page.find_by_xpath( + "//*[@id='0']//*[@id='datagrid']/div[5]/div/div[1]/div[" + str(cnt) + "]/span" + ).get_attribute('innerHTML') + + PGDataypeFeatureTest.check_result( + source_code, + expected_output[cnt - 2] + ) + cnt += 1 + except TimeoutException: + assert False, "{0} does not match with {1}".format(val, expected_output[cnt]) @staticmethod def check_result(source_code, string_to_find): diff --git a/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py b/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py index 526d4e76..d59e8ac3 100644 --- a/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py +++ b/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py @@ -149,8 +149,6 @@ class CheckForXssFeatureTest(BaseFeatureTest): def _check_xss_in_query_tool(self): self.page.driver.find_element_by_link_text("Tools").click() self.page.find_by_partial_link_text("Query Tool").click() - time.sleep(3) - self.page.driver.switch_to.frame(self.page.driver.find_element_by_tag_name('iframe')) self.page.fill_codemirror_area_with("select '<img src=\"x\" onerror=\"console.log(1)\">'") time.sleep(1) self.page.find_by_id("btn-flash").click() diff --git a/web/regression/feature_utils/pgadmin_page.py b/web/regression/feature_utils/pgadmin_page.py index b9abe7ca..30f710eb 100644 --- a/web/regression/feature_utils/pgadmin_page.py +++ b/web/regression/feature_utils/pgadmin_page.py @@ -26,7 +26,7 @@ class PgadminPage: def __init__(self, driver, app_config): self.driver = driver self.app_config = app_config - self.timeout = 30 + self.timeout = 20 self.app_start_timeout = 60 def reset_layout(self): @@ -118,14 +118,27 @@ class PgadminPage: # For long text, if we try to execute send_keys and perform back to back, then the actions are # not executed properly as the driver can send only 50 to 60 characters. To avoid this, sleep # on the basis of content length. - self.find_by_xpath( - "//pre[contains(@class,'CodeMirror-line')]/../../../*[contains(@class,'CodeMirror-code')]").click() + def find_codemirror(driver): + try: + driver.switch_to.default_content() + driver.switch_to_frame(driver.find_element_by_tag_name("iframe")) + element = driver.find_element_by_xpath( + "//pre[contains(@class,'CodeMirror-line')]/../../../*[contains(@class,'CodeMirror-code')]") + if element.is_displayed() and element.is_enabled(): + return element + except (NoSuchElementException, WebDriverException): + return False + + WebDriverWait(self.driver, timeout=self.timeout, poll_frequency=0.01).\ + until(find_codemirror, "Timed out waiting for codemirror to appear").\ + click() + time.sleep(1) + action = ActionChains(self.driver) action.send_keys(field_content) + action.perform() sleep_time = math.ceil(len(field_content) / 50) time.sleep(sleep_time) - action.perform() - time.sleep(1) def click_tab(self, tab_name): self.find_by_xpath("//*[contains(@class,'wcTabTop')]//*[contains(@class,'wcPanelTab') " -- Sent via pgadmin-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgadmin-hackers Attachments: [image/png] image.png (353.2K, 3-image.png) download | view image [text/plain] fix_fill_codemirror_test_helper.diff (5.8K, 4-fix_fill_codemirror_test_helper.diff) download | inline diff: diff --git a/web/pgadmin/feature_tests/pg_datatype_validation_test.py b/web/pgadmin/feature_tests/pg_datatype_validation_test.py index fdc65a58..69b12f30 100644 --- a/web/pgadmin/feature_tests/pg_datatype_validation_test.py +++ b/web/pgadmin/feature_tests/pg_datatype_validation_test.py @@ -6,7 +6,6 @@ # This software is released under the PostgreSQL Licence # ########################################################################## - from selenium.webdriver import ActionChains from selenium.common.exceptions import TimeoutException from selenium.webdriver.support.ui import WebDriverWait @@ -90,30 +89,28 @@ class PGDataypeFeatureTest(BaseFeatureTest): self.page.driver.find_element_by_link_text("Tools").click() self.page.find_by_partial_link_text("Query Tool").click() + + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() wait = WebDriverWait(self.page.driver, 5) - element = wait.until(EC.frame_to_be_available_and_switch_to_it((By.TAG_NAME, 'iframe'))) - - if element: - self.page.fill_codemirror_area_with(query) - self.page.find_by_id("btn-flash").click() - wait.until(EC.presence_of_element_located( - (By.XPATH, "//*[@id='0']//*[@id='datagrid']/div[5]/div/div[1]/div[2]/span"))) - - # For every sample data-type value, check the expected output. - cnt = 2 - for val in expected_output: - try: - source_code = self.page.find_by_xpath( - "//*[@id='0']//*[@id='datagrid']/div[5]/div/div[1]/div[" + str(cnt) + "]/span" - ).get_attribute('innerHTML') - - PGDataypeFeatureTest.check_result( - source_code, - expected_output[cnt - 2] - ) - cnt += 1 - except TimeoutException: - assert False, "{0} does not match with {1}".format(val, expected_output[cnt]) + wait.until(EC.presence_of_element_located( + (By.XPATH, "//*[@id='0']//*[@id='datagrid']/div[5]/div/div[1]/div[2]/span"))) + + # For every sample data-type value, check the expected output. + cnt = 2 + for val in expected_output: + try: + source_code = self.page.find_by_xpath( + "//*[@id='0']//*[@id='datagrid']/div[5]/div/div[1]/div[" + str(cnt) + "]/span" + ).get_attribute('innerHTML') + + PGDataypeFeatureTest.check_result( + source_code, + expected_output[cnt - 2] + ) + cnt += 1 + except TimeoutException: + assert False, "{0} does not match with {1}".format(val, expected_output[cnt]) @staticmethod def check_result(source_code, string_to_find): diff --git a/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py b/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py index 526d4e76..d59e8ac3 100644 --- a/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py +++ b/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py @@ -149,8 +149,6 @@ class CheckForXssFeatureTest(BaseFeatureTest): def _check_xss_in_query_tool(self): self.page.driver.find_element_by_link_text("Tools").click() self.page.find_by_partial_link_text("Query Tool").click() - time.sleep(3) - self.page.driver.switch_to.frame(self.page.driver.find_element_by_tag_name('iframe')) self.page.fill_codemirror_area_with("select '<img src=\"x\" onerror=\"console.log(1)\">'") time.sleep(1) self.page.find_by_id("btn-flash").click() diff --git a/web/regression/feature_utils/pgadmin_page.py b/web/regression/feature_utils/pgadmin_page.py index b9abe7ca..30f710eb 100644 --- a/web/regression/feature_utils/pgadmin_page.py +++ b/web/regression/feature_utils/pgadmin_page.py @@ -26,7 +26,7 @@ class PgadminPage: def __init__(self, driver, app_config): self.driver = driver self.app_config = app_config - self.timeout = 30 + self.timeout = 20 self.app_start_timeout = 60 def reset_layout(self): @@ -118,14 +118,27 @@ class PgadminPage: # For long text, if we try to execute send_keys and perform back to back, then the actions are # not executed properly as the driver can send only 50 to 60 characters. To avoid this, sleep # on the basis of content length. - self.find_by_xpath( - "//pre[contains(@class,'CodeMirror-line')]/../../../*[contains(@class,'CodeMirror-code')]").click() + def find_codemirror(driver): + try: + driver.switch_to.default_content() + driver.switch_to_frame(driver.find_element_by_tag_name("iframe")) + element = driver.find_element_by_xpath( + "//pre[contains(@class,'CodeMirror-line')]/../../../*[contains(@class,'CodeMirror-code')]") + if element.is_displayed() and element.is_enabled(): + return element + except (NoSuchElementException, WebDriverException): + return False + + WebDriverWait(self.driver, timeout=self.timeout, poll_frequency=0.01).\ + until(find_codemirror, "Timed out waiting for codemirror to appear").\ + click() + time.sleep(1) + action = ActionChains(self.driver) action.send_keys(field_content) + action.perform() sleep_time = math.ceil(len(field_content) / 50) time.sleep(sleep_time) - action.perform() - time.sleep(1) def click_tab(self, tab_name): self.find_by_xpath("//*[contains(@class,'wcTabTop')]//*[contains(@class,'wcPanelTab') " ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: feature test timeouts @ 2017-06-06 09:30 Murtuza Zabuawala <[email protected]> parent: George Gelashvili <[email protected]> 1 sibling, 0 replies; 9+ messages in thread From: Murtuza Zabuawala @ 2017-06-06 09:30 UTC (permalink / raw) To: George Gelashvili <[email protected]>; +Cc: Khushboo Vashi <[email protected]>; Dave Page <[email protected]>; pgadmin-hackers That fixed the issue for me, PG data-type feature test working now, -- Regards, Murtuza Zabuawala EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company On Tue, Jun 6, 2017 at 2:28 AM, George Gelashvili <[email protected]> wrote: > We fixed the codemirror-filling helper function method thing in > `pgadmin_page`. This should resolve the PG datatype feature test flakiness > and make for a nice buttery crust. > > It looked like selenium was unable to find an element within the sqleditor > iframe, so we made a workaround that re-locates the iframe and codemirror > element each iteration of the wait. > > Thanks > Joao & George > > On Tue, May 30, 2017 at 6:52 AM, Khushboo Vashi < > [email protected]> wrote: > >> Hi Murtuza, >> >> On Tue, May 30, 2017 at 4:13 PM, Murtuza Zabuawala < >> [email protected]> wrote: >> >>> Hi Dave, >>> >>> I didn't look into the code for it but every time I run the tests it >>> stays on below screen and fails. >>> >>> @Khushboo, >>> Any suggestion? >>> >>> >> Harshal is also getting the same issue with his feature test case. The >> problem we found that, in the bound time the driver is not able to find the >> code-mirror. Me and Harshal looked into this yesterday but couldn't find >> the solution. >> >> BTW, I can not reproduce this issue at my end but me and Harshal will >> look into this issue on his machine. >> >> >> Thanks, >> Khushboo >> >> >>> [image: Inline image 1] >>> >>> -- >>> Regards, >>> Murtuza Zabuawala >>> EnterpriseDB: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >>> On Tue, May 30, 2017 at 2:15 PM, Dave Page <[email protected]> wrote: >>> >>>> Hi >>>> >>>> OK. Do you know why? What's it waiting for? It was working for me (I >>>> always run tests multiple times before committing them, to try to >>>> ensure they're reliable). >>>> >>>> On Mon, May 29, 2017 at 7:23 AM, Murtuza Zabuawala >>>> <[email protected]> wrote: >>>> > Hi Dave, >>>> > >>>> > With latest pull, PG data-type test fails on my system, >>>> > >>>> > ============================================================ >>>> ========== >>>> > ERROR: runTest >>>> > (pgadmin.feature_tests.pg_datatype_validation_test.PGDataype >>>> FeatureTest) >>>> > Test checks for PG data-types output >>>> > ------------------------------------------------------------ >>>> ---------- >>>> > Traceback (most recent call last): >>>> > File >>>> > "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/feature_ >>>> tests/pg_datatype_validation_test.py", >>>> > line 44, in runTest >>>> > self._check_datatype() >>>> > File >>>> > "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/feature_ >>>> tests/pg_datatype_validation_test.py", >>>> > line 97, in _check_datatype >>>> > self.page.fill_codemirror_area_with(query) >>>> > File >>>> > "/Users/edb/Documents/projects/pgadmin4/web/regression/featu >>>> re_utils/pgadmin_page.py", >>>> > line 122, in fill_codemirror_area_with >>>> > >>>> > "//pre[contains(@class,'CodeMirror-line')]/../../../*[contai >>>> ns(@class,'CodeMirror-code')]").click() >>>> > File >>>> > "/Users/edb/Documents/projects/pgadmin4/web/regression/featu >>>> re_utils/pgadmin_page.py", >>>> > line 87, in find_by_xpath >>>> > return self.wait_for_element(lambda driver: >>>> > driver.find_element_by_xpath(xpath)) >>>> > File >>>> > "/Users/edb/Documents/projects/pgadmin4/web/regression/featu >>>> re_utils/pgadmin_page.py", >>>> > line 152, in wait_for_element >>>> > return self._wait_for("element to exist", element_if_it_exists) >>>> > File >>>> > "/Users/edb/Documents/projects/pgadmin4/web/regression/featu >>>> re_utils/pgadmin_page.py", >>>> > line 188, in _wait_for >>>> > "Timed out waiting for " + waiting_for_message) >>>> > File >>>> > "/Users/edb/Documents/projects/venv_pg4/lib/python3.5/site-p >>>> ackages/selenium/webdriver/support/wait.py", >>>> > line 80, in until >>>> > raise TimeoutException(message, screen, stacktrace) >>>> > selenium.common.exceptions.TimeoutException: Message: Timed out >>>> waiting for >>>> > element to exist >>>> > >>>> > >>>> > ------------------------------------------------------------ >>>> ---------- >>>> > Ran 6 tests in 170.026s >>>> > >>>> > FAILED (errors=1) >>>> > >>>> > ============================================================ >>>> ========== >>>> > Test Result Summary >>>> > ============================================================ >>>> ========== >>>> > >>>> > PG_95: >>>> > >>>> > 5 tests passed >>>> > 1 test failed: >>>> > PGDataypeFeatureTest (Test checks for PG data-types output) >>>> > 0 tests skipped >>>> > >>>> > ============================================================ >>>> ========== >>>> > >>>> > Please check output in file: >>>> > /Users/edb/Documents/projects/pgadmin4/web/regression/regression.log >>>> > >>>> > >>>> > -- >>>> > Regards, >>>> > Murtuza Zabuawala >>>> > EnterpriseDB: http://www.enterprisedb.com >>>> > The Enterprise PostgreSQL Company >>>> > >>>> > On Sat, May 27, 2017 at 11:47 PM, Dave Page <[email protected]> >>>> wrote: >>>> >> >>>> >> Thanks, patch applied. >>>> >> >>>> >> On Fri, May 26, 2017 at 6:26 PM, George Gelashvili >>>> >> <[email protected]> wrote: >>>> >> > Hello Hackers, >>>> >> > >>>> >> > We are running tests including feature tests in Circle CI, and >>>> have been >>>> >> > seeing some flakiness due to what we believe are insufficient >>>> wait-for >>>> >> > timeouts. >>>> >> > >>>> >> > This shouldn't increase the overall test time, but only allow >>>> waiting >>>> >> > longer >>>> >> > for an element to appear, if needed. >>>> >> > >>>> >> > We've also split out the app start timeout since that seemed to >>>> need >>>> >> > more >>>> >> > time on CI. >>>> >> > >>>> >> > Patch attached. >>>> >> > >>>> >> > Thanks! >>>> >> > Joao, Matt, and George >>>> >> > >>>> >> > >>>> >> > -- >>>> >> > Sent via pgadmin-hackers mailing list ( >>>> [email protected]) >>>> >> > To make changes to your subscription: >>>> >> > http://www.postgresql.org/mailpref/pgadmin-hackers >>>> >> > >>>> >> >>>> >> >>>> >> >>>> >> -- >>>> >> Dave Page >>>> >> Blog: http://pgsnake.blogspot.com >>>> >> Twitter: @pgsnake >>>> >> >>>> >> EnterpriseDB UK: http://www.enterprisedb.com >>>> >> The Enterprise PostgreSQL Company >>>> >> >>>> >> >>>> >> -- >>>> >> Sent via pgadmin-hackers mailing list ([email protected] >>>> g) >>>> >> To make changes to your subscription: >>>> >> http://www.postgresql.org/mailpref/pgadmin-hackers >>>> > >>>> > >>>> >>>> >>>> >>>> -- >>>> Dave Page >>>> Blog: http://pgsnake.blogspot.com >>>> Twitter: @pgsnake >>>> >>>> EnterpriseDB UK: http://www.enterprisedb.com >>>> The Enterprise PostgreSQL Company >>>> >>> >>> >> > Attachments: [image/png] image.png (353.2K, 3-image.png) download | view image ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: feature test timeouts @ 2017-06-07 13:27 Dave Page <[email protected]> parent: George Gelashvili <[email protected]> 1 sibling, 0 replies; 9+ messages in thread From: Dave Page @ 2017-06-07 13:27 UTC (permalink / raw) To: George Gelashvili <[email protected]>; +Cc: Khushboo Vashi <[email protected]>; Murtuza Zabuawala <[email protected]>; pgadmin-hackers Thanks, patch applied. Mmmmmm, pie.... On Mon, Jun 5, 2017 at 9:58 PM, George Gelashvili <[email protected]> wrote: > We fixed the codemirror-filling helper function method thing in > `pgadmin_page`. This should resolve the PG datatype feature test flakiness > and make for a nice buttery crust. > > It looked like selenium was unable to find an element within the sqleditor > iframe, so we made a workaround that re-locates the iframe and codemirror > element each iteration of the wait. > > Thanks > Joao & George > > On Tue, May 30, 2017 at 6:52 AM, Khushboo Vashi < > [email protected]> wrote: > >> Hi Murtuza, >> >> On Tue, May 30, 2017 at 4:13 PM, Murtuza Zabuawala < >> [email protected]> wrote: >> >>> Hi Dave, >>> >>> I didn't look into the code for it but every time I run the tests it >>> stays on below screen and fails. >>> >>> @Khushboo, >>> Any suggestion? >>> >>> >> Harshal is also getting the same issue with his feature test case. The >> problem we found that, in the bound time the driver is not able to find the >> code-mirror. Me and Harshal looked into this yesterday but couldn't find >> the solution. >> >> BTW, I can not reproduce this issue at my end but me and Harshal will >> look into this issue on his machine. >> >> >> Thanks, >> Khushboo >> >> >>> [image: Inline image 1] >>> >>> -- >>> Regards, >>> Murtuza Zabuawala >>> EnterpriseDB: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >>> On Tue, May 30, 2017 at 2:15 PM, Dave Page <[email protected]> wrote: >>> >>>> Hi >>>> >>>> OK. Do you know why? What's it waiting for? It was working for me (I >>>> always run tests multiple times before committing them, to try to >>>> ensure they're reliable). >>>> >>>> On Mon, May 29, 2017 at 7:23 AM, Murtuza Zabuawala >>>> <[email protected]> wrote: >>>> > Hi Dave, >>>> > >>>> > With latest pull, PG data-type test fails on my system, >>>> > >>>> > ============================================================ >>>> ========== >>>> > ERROR: runTest >>>> > (pgadmin.feature_tests.pg_datatype_validation_test.PGDataype >>>> FeatureTest) >>>> > Test checks for PG data-types output >>>> > ------------------------------------------------------------ >>>> ---------- >>>> > Traceback (most recent call last): >>>> > File >>>> > "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/feature_ >>>> tests/pg_datatype_validation_test.py", >>>> > line 44, in runTest >>>> > self._check_datatype() >>>> > File >>>> > "/Users/edb/Documents/projects/pgadmin4/web/pgadmin/feature_ >>>> tests/pg_datatype_validation_test.py", >>>> > line 97, in _check_datatype >>>> > self.page.fill_codemirror_area_with(query) >>>> > File >>>> > "/Users/edb/Documents/projects/pgadmin4/web/regression/featu >>>> re_utils/pgadmin_page.py", >>>> > line 122, in fill_codemirror_area_with >>>> > >>>> > "//pre[contains(@class,'CodeMirror-line')]/../../../*[contai >>>> ns(@class,'CodeMirror-code')]").click() >>>> > File >>>> > "/Users/edb/Documents/projects/pgadmin4/web/regression/featu >>>> re_utils/pgadmin_page.py", >>>> > line 87, in find_by_xpath >>>> > return self.wait_for_element(lambda driver: >>>> > driver.find_element_by_xpath(xpath)) >>>> > File >>>> > "/Users/edb/Documents/projects/pgadmin4/web/regression/featu >>>> re_utils/pgadmin_page.py", >>>> > line 152, in wait_for_element >>>> > return self._wait_for("element to exist", element_if_it_exists) >>>> > File >>>> > "/Users/edb/Documents/projects/pgadmin4/web/regression/featu >>>> re_utils/pgadmin_page.py", >>>> > line 188, in _wait_for >>>> > "Timed out waiting for " + waiting_for_message) >>>> > File >>>> > "/Users/edb/Documents/projects/venv_pg4/lib/python3.5/site-p >>>> ackages/selenium/webdriver/support/wait.py", >>>> > line 80, in until >>>> > raise TimeoutException(message, screen, stacktrace) >>>> > selenium.common.exceptions.TimeoutException: Message: Timed out >>>> waiting for >>>> > element to exist >>>> > >>>> > >>>> > ------------------------------------------------------------ >>>> ---------- >>>> > Ran 6 tests in 170.026s >>>> > >>>> > FAILED (errors=1) >>>> > >>>> > ============================================================ >>>> ========== >>>> > Test Result Summary >>>> > ============================================================ >>>> ========== >>>> > >>>> > PG_95: >>>> > >>>> > 5 tests passed >>>> > 1 test failed: >>>> > PGDataypeFeatureTest (Test checks for PG data-types output) >>>> > 0 tests skipped >>>> > >>>> > ============================================================ >>>> ========== >>>> > >>>> > Please check output in file: >>>> > /Users/edb/Documents/projects/pgadmin4/web/regression/regression.log >>>> > >>>> > >>>> > -- >>>> > Regards, >>>> > Murtuza Zabuawala >>>> > EnterpriseDB: http://www.enterprisedb.com >>>> > The Enterprise PostgreSQL Company >>>> > >>>> > On Sat, May 27, 2017 at 11:47 PM, Dave Page <[email protected]> >>>> wrote: >>>> >> >>>> >> Thanks, patch applied. >>>> >> >>>> >> On Fri, May 26, 2017 at 6:26 PM, George Gelashvili >>>> >> <[email protected]> wrote: >>>> >> > Hello Hackers, >>>> >> > >>>> >> > We are running tests including feature tests in Circle CI, and >>>> have been >>>> >> > seeing some flakiness due to what we believe are insufficient >>>> wait-for >>>> >> > timeouts. >>>> >> > >>>> >> > This shouldn't increase the overall test time, but only allow >>>> waiting >>>> >> > longer >>>> >> > for an element to appear, if needed. >>>> >> > >>>> >> > We've also split out the app start timeout since that seemed to >>>> need >>>> >> > more >>>> >> > time on CI. >>>> >> > >>>> >> > Patch attached. >>>> >> > >>>> >> > Thanks! >>>> >> > Joao, Matt, and George >>>> >> > >>>> >> > >>>> >> > -- >>>> >> > Sent via pgadmin-hackers mailing list ( >>>> [email protected]) >>>> >> > To make changes to your subscription: >>>> >> > http://www.postgresql.org/mailpref/pgadmin-hackers >>>> >> > >>>> >> >>>> >> >>>> >> >>>> >> -- >>>> >> Dave Page >>>> >> Blog: http://pgsnake.blogspot.com >>>> >> Twitter: @pgsnake >>>> >> >>>> >> EnterpriseDB UK: http://www.enterprisedb.com >>>> >> The Enterprise PostgreSQL Company >>>> >> >>>> >> >>>> >> -- >>>> >> Sent via pgadmin-hackers mailing list ([email protected] >>>> g) >>>> >> To make changes to your subscription: >>>> >> http://www.postgresql.org/mailpref/pgadmin-hackers >>>> > >>>> > >>>> >>>> >>>> >>>> -- >>>> Dave Page >>>> Blog: http://pgsnake.blogspot.com >>>> Twitter: @pgsnake >>>> >>>> EnterpriseDB UK: http://www.enterprisedb.com >>>> The Enterprise PostgreSQL Company >>>> >>> >>> >> > -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company Attachments: [image/png] image.png (353.2K, 3-image.png) download | view image ^ permalink raw reply [nested|flat] 9+ messages in thread
end of thread, other threads:[~2017-06-07 13:27 UTC | newest] Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2017-05-26 22:26 feature test timeouts George Gelashvili <[email protected]> 2017-05-27 18:17 ` Dave Page <[email protected]> 2017-05-29 06:23 ` Murtuza Zabuawala <[email protected]> 2017-05-30 08:45 ` Dave Page <[email protected]> 2017-05-30 10:43 ` Murtuza Zabuawala <[email protected]> 2017-05-30 10:52 ` Khushboo Vashi <[email protected]> 2017-06-05 20:58 ` George Gelashvili <[email protected]> 2017-06-06 09:30 ` Murtuza Zabuawala <[email protected]> 2017-06-07 13:27 ` Dave Page <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox