public inbox for [email protected]
help / color / mirror / Atom feedFrom: Dave Page <[email protected]>
To: Harshal Dhumal <[email protected]>
Cc: Harshal Dhumal <[email protected]>
Cc: Ashesh Vashi <[email protected]>
Cc: pgadmin-hackers <[email protected]>
Subject: Re: [pgadmin-hackers] Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4]
Date: Fri, 23 Jun 2017 16:30:56 +0100
Message-ID: <CA+OCxowu1z=wKO=8=dw-xZCfHeegCxnMKS=O_fRae8OhY6k1uQ@mail.gmail.com> (raw)
In-Reply-To: <CA+OCxoxZzWEUj49+OYYdSJiiqMa-4HD9OZN7BFiM4X8_Ws9FtA@mail.gmail.com>
References: <CAFiP3vxaM9ixpmaM04JAkTT1ONwmnMPwUCbK+kzCN+r6F+7-ZA@mail.gmail.com>
<CA+OCxozb7qDTiskHXv08G5TrCi7zmhR7Csz=x9V4ermehyO8cg@mail.gmail.com>
<CAFiP3vzas6ez9u-snfCuXzb1eL6_aZwAfrb+561sq8V19PT4Eg@mail.gmail.com>
<CA+OCxoy-8RyH-QYxXkLn1KSTUJMArev0ukKpfJx+yvATEyJV+Q@mail.gmail.com>
<CAFiP3vz6_Up4OfSAR4J8C35EzgdApbAiZCeZzzCKxnjwgJMdYQ@mail.gmail.com>
<CAGRPzo_KmDF=ThGC9hy6JepsxBn23nuXkamJBQgT6eOFh0p1pg@mail.gmail.com>
<CAFiP3vwYLTLXX5EFPzE_s-0ucQikDERp_iS+4Neb1JN-sHfG9A@mail.gmail.com>
<CAFiP3vxvFMHv=yEOXcbxi9YABtTwOk3Feci92YMA4_9Fcp-4mA@mail.gmail.com>
<CAFiP3vzj2HBjau+N5TN+Asb_7DW6zgmPrzJf0rmd_H0NJ3Sttg@mail.gmail.com>
<CAE+jjamhZ27XJ2ZHGRgpsL0yUUOawDhiMXoxfB-bRCu=M8RBhg@mail.gmail.com>
<CAFiP3vw=rGTY1BFPB2ZLjLBehEaPpXgXRY2SerK+07TrA+nxzA@mail.gmail.com>
<CA+OCxozQj0gy7dsorij5fXSgLuhGDJP7uC4++VviAPR1XFs-Fw@mail.gmail.com>
<CAFiP3vyjbmQN2bRpHJvNXFnK6YvtgPm12OiF_Oa2Z7J73=C3Ag@mail.gmail.com>
<CAFiP3vzomo+37TZ87tVr4AsmUNzzsFQkq8j8fEXNKdpVj9bOxw@mail.gmail.com>
<CAFiP3vz1EGT4cgv3XJNT6wLDz0Ca7dT_W0p6GsDVPua+tWZrpw@mail.gmail.com>
<CA+OCxoxyAD1x_bEhzPS0RGU1SuL8Lfhx39ME7wY=L28ep4PdvQ@mail.gmail.com>
<CAG7mmoxASRd2wTahma6WNLCLfg=4gy5eOQ+e5K0ioOHPUS+7bA@mail.gmail.com>
<CAE29nRn0Z7v4oOWWndLr1Cr8HRwHizBK3+7BK12KnT0uHHHi7Q@mail.gmail.com>
<CAFiP3vxEra_mZGc8ZMJ+zNOpNb+LRGT7LobYktFgTetVMktmsw@mail.gmail.com>
<CA+OCxoz=hJpzbca9kwMesf+K2axYLaa_s7bThmnQh2WQHL0SmQ@mail.gmail.com>
<CAFiP3vzQQ8MxzSGuYcOPR9cDRRdz06LuLuGKo3cnTWXNcgFrew@mail.gmail.com>
<CA+OCxoy=+A0xE2gaFTH0VkAF-f4dpkW4nvF8sLd7Guag3czorA@mail.gmail.com>
<CAFiP3vy3rUMT+t6FBoNZftWf1ATth09t6+_GW_+qiBvsfFr4FQ@mail.gmail.com>
<CA+OCxox=8+nnQOAARWVRaxvBYBAG1sFyV1fWT-whPhdqVoJMmQ@mail.gmail.com>
<CAFiP3vwqO-0CtAvzu4SYa_UCj9-xbgLyauSy7agDc5ss10+LJg@mail.gmail.com>
<CA+OCxoxZzWEUj49+OYYdSJiiqMa-4HD9OZN7BFiM4X8_Ws9FtA@mail.gmail.com>
List-Unsubscribe: <https://lists.postgresql.org/manage/>, <mailto:[email protected]>
Hi Harshal,
When can we expect an updated version of this patch? I think it's
important to get this into the next release.
Thanks!
On Fri, Jun 16, 2017 at 10:55 AM, Dave Page <[email protected]> wrote:
> Hi,
>
> That's better - the failures are far less random now :-). I got the
> following two though, on both PG and EPAS 9.5:
>
> ======================================================================
> ERROR: runTest (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest)
> Query tool feature test
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py",
> line 95, in runTest
> self._query_tool_explain_analyze_buffers()
> File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py",
> line 443, in _query_tool_explain_analyze_buffers
> canvas.find_element_by_xpath("//*[contains(string(), 'Shared Read
> Blocks')]")
> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py",
> line 260, in find_element_by_xpath
> return self.find_element(by=By.XPATH, value=xpath)
> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py",
> line 508, in find_element
> {"using": by, "value": value})['value']
> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py",
> line 491, in _execute
> return self._parent.execute(command, params)
> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py",
> line 238, in execute
> self.error_handler.check_response(response)
> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py",
> line 193, in check_response
> raise exception_class(message, screen, stacktrace)
> NoSuchElementException: Message: no such element: Unable to locate
> element: {"method":"xpath","selector":"//*[contains(string(), 'Shared
> Read Blocks')]"}
> (Session info: chrome=58.0.3029.110)
> (Driver info: chromedriver=2.29.461585
> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3
> x86_64)
>
>
> ======================================================================
> ERROR: runTest (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest)
> Validate Insert, Update operations in View data with given test data
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py",
> line 104, in runTest
> self._add_row()
> File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py",
> line 255, in _add_row
> self._update_cell(cell_xpath, config_data[str(idx)])
> File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py",
> line 164, in _update_cell
> cell_el = self.page.find_by_xpath(xpath)
> File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
> line 122, in find_by_xpath
> return self.wait_for_element(lambda driver:
> driver.find_element_by_xpath(xpath))
> File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
> line 205, in wait_for_element
> return self._wait_for("element to exist", element_if_it_exists)
> File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
> line 255, in _wait_for
> "Timed out waiting for " + waiting_for_message)
> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/support/wait.py",
> line 71, in until
> value = method(self._driver)
> File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py",
> line 200, in element_if_it_exists
> if element.is_displayed() and element.is_enabled():
> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py",
> line 358, in is_displayed
> return self._execute(Command.IS_ELEMENT_DISPLAYED)['value']
> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py",
> line 491, in _execute
> return self._parent.execute(command, params)
> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py",
> line 238, in execute
> self.error_handler.check_response(response)
> File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py",
> line 193, in check_response
> raise exception_class(message, screen, stacktrace)
> StaleElementReferenceException: Message: stale element reference:
> element is not attached to the page document
> (Session info: chrome=58.0.3029.110)
> (Driver info: chromedriver=2.29.461585
> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3
> x86_64)
>
>
> On 9.4 everything passes. On 9.6, I get one failure:
>
> ======================================================================
> FAIL: runTest (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest)
> Validate Insert, Update operations in View data with given test data
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py",
> line 105, in runTest
> self._verify_row_data(True)
> File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py",
> line 282, in _verify_row_data
> self.assertEquals(cells[idx], config_data[str(idx)][1])
> AssertionError: u'[default]' != u'1'
> - [default]
> + 1
>
> Screenshots attached.
>
> On Fri, Jun 16, 2017 at 7:54 AM, Harshal Dhumal
> <[email protected]> wrote:
>> Hi Dave,
>>
>> Please find attached patch where I have added timeout of 2 seconds before
>> selecting object menu in connect to server function.
>>
>> Note: Apply this patch on top of previous patch for on demand loading
>> feature.
>>
>>
>> --
>> Harshal Dhumal
>> Sr. Software Engineer
>>
>> EnterpriseDB India: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>> On Fri, Jun 16, 2017 at 2:25 AM, Dave Page <[email protected]> wrote:
>>>
>>> Sounds good, thanks.
>>>
>>> On Thu, Jun 15, 2017 at 9:54 PM, Harshal Dhumal
>>> <[email protected]> wrote:
>>> > Hi
>>> >
>>> > On Fri, Jun 16, 2017 at 2:07 AM, Dave Page <[email protected]> wrote:
>>> >>
>>> >> Hi
>>> >>
>>> >> On Thu, Jun 15, 2017 at 9:30 PM, Harshal Dhumal
>>> >> <[email protected]> wrote:
>>> >> > Hi Dave,
>>> >> >
>>> >> > Please find attached updated patch.
>>> >> >
>>> >> > On Thu, Jun 15, 2017 at 3:58 PM, Dave Page <[email protected]> wrote:
>>> >> >>
>>> >> >> Hi
>>> >> >>
>>> >> >> On Wed, Jun 14, 2017 at 11:36 PM, Harshal Dhumal
>>> >> >> <[email protected]> wrote:
>>> >> >>>
>>> >> >>> Hi Dave,
>>> >> >>>
>>> >> >>> Please find rebased patch for RM2137.
>>> >> >>
>>> >> >>
>>> >> >> Looking very good. The only issues I see are:
>>> >> >>
>>> >> >> - The row headers should auto-size such that they can display the
>>> >> >> row
>>> >> >> numbers if the last row was displayed. E.g. if there are 12345 rows
>>> >> >> in
>>> >> >> total, then the row header should be sized to display 5 digits.
>>> >> >>
>>> >> > Fixed.
>>> >> >
>>> >> >
>>> >> >>
>>> >> >> - The tests are comprehensive, which is awesome. However, every time
>>> >> >> I
>>> >> >> ran
>>> >> >> them, at least one of the feature tests failed. Unfortunately, it
>>> >> >> was a
>>> >> >> different one each time. In the last two runs, I got:
>>> >> >>
>>> >> >>
>>> >> >> ======================================================================
>>> >> >> ERROR: runTest
>>> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest)
>>> >> >> Query tool feature test
>>> >> >>
>>> >> >> ----------------------------------------------------------------------
>>> >> >> Traceback (most recent call last):
>>> >> >> File
>>> >> >>
>>> >> >>
>>> >> >> "/Users/dpage/git/pgadmin4/web/regression/feature_utils/base_feature_test.py",
>>> >> >> line 40, in setUp
>>> >> >> self.before()
>>> >> >> File
>>> >> >>
>>> >> >>
>>> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py",
>>> >> >> line 40, in before
>>> >> >> self._connects_to_server()
>>> >> >> File
>>> >> >>
>>> >> >>
>>> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py",
>>> >> >> line 144, in _connects_to_server
>>> >> >> self.page.driver.find_element_by_link_text("Create"))\
>>> >> >> File
>>> >> >>
>>> >> >>
>>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py",
>>> >> >> line 319, in find_element_by_link_text
>>> >> >> return self.find_element(by=By.LINK_TEXT, value=link_text)
>>> >> >> File
>>> >> >>
>>> >> >>
>>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py",
>>> >> >> line 756, in find_element
>>> >> >> 'value': value})['value']
>>> >> >> File
>>> >> >>
>>> >> >>
>>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py",
>>> >> >> line 238, in execute
>>> >> >> self.error_handler.check_response(response)
>>> >> >> File
>>> >> >>
>>> >> >>
>>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py",
>>> >> >> line 193, in check_response
>>> >> >> raise exception_class(message, screen, stacktrace)
>>> >> >> NoSuchElementException: Message: no such element: Unable to locate
>>> >> >> element: {"method":"link text","selector":"Create"}
>>> >> >> (Session info: chrome=58.0.3029.110)
>>> >> >> (Driver info: chromedriver=2.29.461585
>>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3
>>> >> >> x86_64)
>>> >> >>
>>> >> > I checked code from _connects_to_server function which is common in
>>> >> > all
>>> >> > features test cases. I didn't find anything wrong with this. If there
>>> >> > is
>>> >> > a
>>> >> > bug in this function then all feature test must fail.
>>> >> > Let me know if you are getting failure consistently in
>>> >> > _connects_to_server
>>> >> > function.
>>> >>
>>> >> I wondered if that one is a race condition. Do we need a short delay
>>> >> before clicking the Object menu? I have seen this occasionally before.
>>> >
>>> >
>>> > OK. In that case let's try putting 1-2 second delay and observer
>>> > behaviour.
>>> > I'll send separate patch for this tomorrow as this is not related to on
>>> > demand query result feature or its test cases.
>>> >
>>> >>
>>> >>
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> ======================================================================
>>> >> >> ERROR: runTest
>>> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest)
>>> >> >> Query tool feature test
>>> >> >>
>>> >> >> ----------------------------------------------------------------------
>>> >> >> Traceback (most recent call last):
>>> >> >> File
>>> >> >>
>>> >> >>
>>> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py",
>>> >> >> line 119, in runTest
>>> >> >> self._query_tool_auto_rollback_enabled()
>>> >> >> File
>>> >> >>
>>> >> >>
>>> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py",
>>> >> >> line 697, in _query_tool_auto_rollback_enabled
>>> >> >> '//div[contains(@class, "sql-editor-message") and
>>> >> >> contains(string(),
>>> >> >> "COMMIT")]'
>>> >> >> File
>>> >> >>
>>> >> >>
>>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py",
>>> >> >> line 295, in find_element_by_xpath
>>> >> >> return self.find_element(by=By.XPATH, value=xpath)
>>> >> >> File
>>> >> >>
>>> >> >>
>>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py",
>>> >> >> line 756, in find_element
>>> >> >> 'value': value})['value']
>>> >> >> File
>>> >> >>
>>> >> >>
>>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py",
>>> >> >> line 238, in execute
>>> >> >> self.error_handler.check_response(response)
>>> >> >> File
>>> >> >>
>>> >> >>
>>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py",
>>> >> >> line 193, in check_response
>>> >> >> raise exception_class(message, screen, stacktrace)
>>> >> >> NoSuchElementException: Message: no such element: Unable to locate
>>> >> >> element: {"method":"xpath","selector":"//div[contains(@class,
>>> >> >> "sql-editor-message") and contains(string(), "COMMIT")]"}
>>> >> >> (Session info: chrome=58.0.3029.110)
>>> >> >> (Driver info: chromedriver=2.29.461585
>>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3
>>> >> >> x86_64)
>>> >> >>
>>> >> > I have updated Auto rollback enabled test in this patch.
>>> >> >
>>> >> >>
>>> >> >> Relevant screenshots attached.
>>> >> >>
>>> >> >> - Can you tidy up the regression output a little please? Instead of:
>>> >> >>
>>> >> >> -------
>>> >> >> runTest
>>> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest)
>>> >> >> Query tool feature test ... On demand result set on scrolling...
>>> >> >> OK.
>>> >> >>
>>> >> >> On demand result set on grid select all...
>>> >> >> OK.
>>> >> >>
>>> >> >> On demand result set on column select all...
>>> >> >> OK.
>>> >> >>
>>> >> >> Explain query...
>>> >> >> OK.
>>> >> >>
>>> >> >> Explain query with verbose...
>>> >> >> OK.
>>> >> >>
>>> >> >> Explain query with costs...
>>> >> >> OK.
>>> >> >>
>>> >> >> Explain analyze query...
>>> >> >> OK.
>>> >> >>
>>> >> >> Explain analyze query with buffers...
>>> >> >> OK.
>>> >> >>
>>> >> >> Explain analyze query with timing...
>>> >> >> OK.
>>> >> >>
>>> >> >> Auto commit disabled...
>>> >> >> OK.
>>> >> >>
>>> >> >> Auto commit enabled...
>>> >> >> OK.
>>> >> >>
>>> >> >> Auto rollback enabled...
>>> >> >> ERROR
>>> >> >> -------
>>> >> >>
>>> >> >> Something like:
>>> >> >>
>>> >> >> -------
>>> >> >> runTest
>>> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest)
>>> >> >> Query tool feature test ...
>>> >> >> On demand result set on scrolling... OK.
>>> >> >> On demand result set on grid select all... OK.
>>> >> >> On demand result set on column select all... OK.
>>> >> >> Explain query... OK.
>>> >> >> Explain query with verbose... OK.
>>> >> >> Explain query with costs... OK.
>>> >> >> Explain analyze query... OK.
>>> >> >> Explain analyze query with buffers... OK.
>>> >> >> Explain analyze query with timing... OK.
>>> >> >> Auto commit disabled... OK.
>>> >> >> Auto commit enabled... OK.
>>> >> >> Auto rollback enabled... ERROR
>>> >> >> --------
>>> >> >>
>>> >> > Fixed.
>>> >> >
>>> >> >
>>> >> >>
>>> >> >> Thanks!
>>> >> >>
>>> >> >> --
>>> >> >> Dave Page
>>> >> >> Blog: http://pgsnake.blogspot.com
>>> >> >> Twitter: @pgsnake
>>> >> >>
>>> >> >> EnterpriseDB UK: http://www.enterprisedb.com
>>> >> >> The Enterprise PostgreSQL Company
>>> >> >
>>> >> >
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Dave Page
>>> >> Blog: http://pgsnake.blogspot.com
>>> >> Twitter: @pgsnake
>>> >>
>>> >> EnterpriseDB UK: http://www.enterprisedb.com
>>> >> The Enterprise PostgreSQL Company
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> Dave Page
>>> Blog: http://pgsnake.blogspot.com
>>> Twitter: @pgsnake
>>>
>>> EnterpriseDB UK: http://www.enterprisedb.com
>>> The Enterprise PostgreSQL Company
>>
>>
>
>
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake
EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
view thread (24+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected]
Subject: Re: [pgadmin-hackers] Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4]
In-Reply-To: <CA+OCxowu1z=wKO=8=dw-xZCfHeegCxnMKS=O_fRae8OhY6k1uQ@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox