Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dLbn3-0001Ea-Nu for pgadmin-hackers@arkaria.postgresql.org; Thu, 15 Jun 2017 20:54:53 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.84_2) (envelope-from ) id 1dLbn3-0002f0-Aj for pgadmin-hackers@arkaria.postgresql.org; Thu, 15 Jun 2017 20:54:53 +0000 Received: from makus.postgresql.org ([2001:4800:1501:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1dLbmo-0002GC-10 for pgadmin-hackers@postgresql.org; Thu, 15 Jun 2017 20:54:38 +0000 Received: from mail-io0-x22c.google.com ([2607:f8b0:4001:c06::22c]) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.84_2) (envelope-from ) id 1dLbmg-0007jz-9K for pgadmin-hackers@postgresql.org; Thu, 15 Jun 2017 20:54:36 +0000 Received: by mail-io0-x22c.google.com with SMTP id t87so19141188ioe.0 for ; Thu, 15 Jun 2017 13:54:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=sscGcfQOJP4NBO9Wr2lO014SoFr9t7mbm8vUxOfi7RY=; b=kZQhRuyoikVuz0hM0YEM5mP5bhgeyO5EJI55YA5ryV++B8u/DebIdB/123twRNfCk1 Bmiq0DXt9ZEQUzqWmhlGS/bStndq2/Z1XmrY+yj38U/Dz2n2uOZKTGubosCOIsuKqhkq 0S7VLckY7swdK5KyBzUCIjap9iEEjxgR4A+/b3MUx3SJsjk5tDm4vjSAP5tpXRrS8zSn wAEb4r8Erzq7hBTurGP4irQR7YFCCu8yt5ANGoVRR72u3xjkVoUnIRYRIZcT/fDpPWmt nx0ZEo6jHvN2J/IW6PJgnce4wB5DN+8+aoHsF24RQk7eUewcGm6//qCmg5Nx9uQ0sok5 nGlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=sscGcfQOJP4NBO9Wr2lO014SoFr9t7mbm8vUxOfi7RY=; b=dhsv8m0VqQXt5ErE/dSDuU8l9uwXEWVMA0gwR2VDd510zy3grdTSrrB46c/a/RT6R9 iA0bcwqOaSqOe76KUQUSZy0W8h8f5yxRn9391UUA6+QtuqSj4c22nrJU9Zn7ww3i2OwO t7NbkcmGXGjUKS4GUIWR7iWXXnW4ALnZRZEUAH3z/wfXAJN7k2djRmWA4vfs3LMVGiXg WoLh/rWK+mj70BCFLMbGkzvd4wqEj82tcO7iCyIl80cpdcojQBaDkOsTbuaj/risQJnf 9JfaQGRrz41xCP0FahqWZHIuwNTFBR5ohGKcK6BzjZpFFDOWOHZEWRPAyj3AY5mOu5PT yBVg== X-Gm-Message-State: AKS2vOwVYLoOqeUg1ZhBpcs/866I/jEuiopVSxy8yXUj6TMEsNSMj11r KkQMkqfnhENkuRYIr0YIHyCpgRoW6piu X-Received: by 10.107.6.143 with SMTP id f15mr7150966ioi.2.1497560069401; Thu, 15 Jun 2017 13:54:29 -0700 (PDT) MIME-Version: 1.0 Received: by 10.79.147.23 with HTTP; Thu, 15 Jun 2017 13:54:28 -0700 (PDT) In-Reply-To: References: From: Harshal Dhumal Date: Fri, 16 Jun 2017 02:24:28 +0530 Message-ID: Subject: Re: Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] To: Dave Page Cc: Harshal Dhumal , Ashesh Vashi , pgadmin-hackers Content-Type: multipart/alternative; boundary="001a113ee4baf7d1fb055205df07" List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgadmin-hackers Precedence: bulk Sender: pgadmin-hackers-owner@postgresql.org --001a113ee4baf7d1fb055205df07 Content-Type: text/plain; charset="UTF-8" Hi On Fri, Jun 16, 2017 at 2:07 AM, Dave Page wrote: > Hi > > On Thu, Jun 15, 2017 at 9:30 PM, Harshal Dhumal > wrote: > > Hi Dave, > > > > Please find attached updated patch. > > > > On Thu, Jun 15, 2017 at 3:58 PM, Dave Page wrote: > >> > >> Hi > >> > >> On Wed, Jun 14, 2017 at 11:36 PM, Harshal Dhumal > >> wrote: > >>> > >>> Hi Dave, > >>> > >>> Please find rebased patch for RM2137. > >> > >> > >> Looking very good. The only issues I see are: > >> > >> - The row headers should auto-size such that they can display the row > >> numbers if the last row was displayed. E.g. if there are 12345 rows in > >> total, then the row header should be sized to display 5 digits. > >> > > Fixed. > > > > > >> > >> - The tests are comprehensive, which is awesome. However, every time I > ran > >> them, at least one of the feature tests failed. Unfortunately, it was a > >> different one each time. In the last two runs, I got: > >> > >> ====================================================================== > >> ERROR: runTest > >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) > >> Query tool feature test > >> ---------------------------------------------------------------------- > >> Traceback (most recent call last): > >> File > >> "/Users/dpage/git/pgadmin4/web/regression/feature_utils/ > base_feature_test.py", > >> line 40, in setUp > >> self.before() > >> File > >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > query_tool_tests.py", > >> line 40, in before > >> self._connects_to_server() > >> File > >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > query_tool_tests.py", > >> line 144, in _connects_to_server > >> self.page.driver.find_element_by_link_text("Create"))\ > >> File > >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> line 319, in find_element_by_link_text > >> return self.find_element(by=By.LINK_TEXT, value=link_text) > >> File > >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> line 756, in find_element > >> 'value': value})['value'] > >> File > >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> line 238, in execute > >> self.error_handler.check_response(response) > >> File > >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/errorhandler.py", > >> line 193, in check_response > >> raise exception_class(message, screen, stacktrace) > >> NoSuchElementException: Message: no such element: Unable to locate > >> element: {"method":"link text","selector":"Create"} > >> (Session info: chrome=58.0.3029.110) > >> (Driver info: chromedriver=2.29.461585 > >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 > x86_64) > >> > > I checked code from _connects_to_server function which is common in all > > features test cases. I didn't find anything wrong with this. If there is > a > > bug in this function then all feature test must fail. > > Let me know if you are getting failure consistently in > _connects_to_server > > function. > > I wondered if that one is a race condition. Do we need a short delay > before clicking the Object menu? I have seen this occasionally before. > OK. In that case let's try putting 1-2 second delay and observer behaviour. I'll send separate patch for this tomorrow as this is not related to on demand query result feature or its test cases. > > >> > >> > >> ====================================================================== > >> ERROR: runTest > >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) > >> Query tool feature test > >> ---------------------------------------------------------------------- > >> Traceback (most recent call last): > >> File > >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > query_tool_tests.py", > >> line 119, in runTest > >> self._query_tool_auto_rollback_enabled() > >> File > >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > query_tool_tests.py", > >> line 697, in _query_tool_auto_rollback_enabled > >> '//div[contains(@class, "sql-editor-message") and contains(string(), > >> "COMMIT")]' > >> File > >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> line 295, in find_element_by_xpath > >> return self.find_element(by=By.XPATH, value=xpath) > >> File > >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> line 756, in find_element > >> 'value': value})['value'] > >> File > >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> line 238, in execute > >> self.error_handler.check_response(response) > >> File > >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/errorhandler.py", > >> line 193, in check_response > >> raise exception_class(message, screen, stacktrace) > >> NoSuchElementException: Message: no such element: Unable to locate > >> element: {"method":"xpath","selector":"//div[contains(@class, > >> "sql-editor-message") and contains(string(), "COMMIT")]"} > >> (Session info: chrome=58.0.3029.110) > >> (Driver info: chromedriver=2.29.461585 > >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 > x86_64) > >> > > I have updated Auto rollback enabled test in this patch. > > > >> > >> Relevant screenshots attached. > >> > >> - Can you tidy up the regression output a little please? Instead of: > >> > >> ------- > >> runTest (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) > >> Query tool feature test ... On demand result set on scrolling... > >> OK. > >> > >> On demand result set on grid select all... > >> OK. > >> > >> On demand result set on column select all... > >> OK. > >> > >> Explain query... > >> OK. > >> > >> Explain query with verbose... > >> OK. > >> > >> Explain query with costs... > >> OK. > >> > >> Explain analyze query... > >> OK. > >> > >> Explain analyze query with buffers... > >> OK. > >> > >> Explain analyze query with timing... > >> OK. > >> > >> Auto commit disabled... > >> OK. > >> > >> Auto commit enabled... > >> OK. > >> > >> Auto rollback enabled... > >> ERROR > >> ------- > >> > >> Something like: > >> > >> ------- > >> runTest (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) > >> Query tool feature test ... > >> On demand result set on scrolling... OK. > >> On demand result set on grid select all... OK. > >> On demand result set on column select all... OK. > >> Explain query... OK. > >> Explain query with verbose... OK. > >> Explain query with costs... OK. > >> Explain analyze query... OK. > >> Explain analyze query with buffers... OK. > >> Explain analyze query with timing... OK. > >> Auto commit disabled... OK. > >> Auto commit enabled... OK. > >> Auto rollback enabled... ERROR > >> -------- > >> > > Fixed. > > > > > >> > >> Thanks! > >> > >> -- > >> Dave Page > >> Blog: http://pgsnake.blogspot.com > >> Twitter: @pgsnake > >> > >> EnterpriseDB UK: http://www.enterprisedb.com > >> The Enterprise PostgreSQL Company > > > > > > > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > --001a113ee4baf7d1fb055205df07 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi

On Fri,= Jun 16, 2017 at 2:07 AM, Dave Page <dpage@pgadmin.org> wrot= e:
Hi

On Thu, Jun 15, 2017 at 9:30 PM, Harshal Dhumal
<harshal.dhumal@enterprisedb.com> wrote:
> Hi Dave,
>
> Please find attached updated patch.
>
> On Thu, Jun 15, 2017 at 3:58 PM, Dave Page <dpage@pgadmin.org> wrote:
>>
>> Hi
>>
>> On Wed, Jun 14, 2017 at 11:36 PM, Harshal Dhumal
>> <harshal.dhu= mal@enterprisedb.com> wrote:
>>>
>>> Hi Dave,
>>>
>>> Please find rebased patch for RM2137.
>>
>>
>> Looking very good. The only issues I see are:
>>
>> - The row headers should auto-size such that they can display the = row
>> numbers if the last row was displayed. E.g. if there are 12345 row= s in
>> total, then the row header should be sized to display 5 digits. >>
> Fixed.
>
>
>>
>> - The tests are comprehensive, which is awesome. However, every ti= me I ran
>> them, at least one of the feature tests failed. Unfortunately, it = was a
>> different one each time. In the last two runs, I got:
>>
>>=C2=A0 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D
>> ERROR: runTest
>> (pgadmin.feature_tests.query_tool_tests.QueryToolFeature= Test)
>> Query tool feature test
>> ------------------------------------------------------------<= wbr>----------
>> Traceback (most recent call last):
>>=C2=A0 =C2=A0File
>> "/Users/dpage/git/pgadmin4/web/regression/feature_utils/= base_feature_test.py",
>> line 40, in setUp
>>=C2=A0 =C2=A0 =C2=A0self.before()
>>=C2=A0 =C2=A0File
>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py",
>> line 40, in before
>>=C2=A0 =C2=A0 =C2=A0self._connects_to_server()
>>=C2=A0 =C2=A0File
>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py",
>> line 144, in _connects_to_server
>>=C2=A0 =C2=A0 =C2=A0self.page.driver.find_element_by_link_text= ("Create"))\
>>=C2=A0 =C2=A0File
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-<= wbr>packages/selenium/webdriver/remote/webdriver.py",
>> line 319, in find_element_by_link_text
>>=C2=A0 =C2=A0 =C2=A0return self.find_element(by=3DBy.LINK_TEXT= , value=3Dlink_text)
>>=C2=A0 =C2=A0File
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-<= wbr>packages/selenium/webdriver/remote/webdriver.py",
>> line 756, in find_element
>>=C2=A0 =C2=A0 =C2=A0'value': value})['value']
>>=C2=A0 =C2=A0File
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-<= wbr>packages/selenium/webdriver/remote/webdriver.py",
>> line 238, in execute
>>=C2=A0 =C2=A0 =C2=A0self.error_handler.check_response(response= )
>>=C2=A0 =C2=A0File
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-<= wbr>packages/selenium/webdriver/remote/errorhandler.py",
>> line 193, in check_response
>>=C2=A0 =C2=A0 =C2=A0raise exception_class(message, screen, stacktra= ce)
>> NoSuchElementException: Message: no such element: Unable to locate=
>> element: {"method":"link text","selector&= quot;:"Create"}
>>=C2=A0 =C2=A0(Session info: chrome=3D58.0.3029.110)
>>=C2=A0 =C2=A0(Driver info: chromedriver=3D2.29.461585
>> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=3DMa= c OS X 10.12.3 x86_64)
>>
> I checked code from _connects_to_server function which is common in al= l
> features test cases. I didn't find anything wrong with this. If th= ere is a
> bug in this function then all feature test must fail.
> Let me know if you are getting failure consistently in _connects_to_se= rver
> function.

I wondered if that one is a race condition. Do we need a short = delay
before clicking the Object menu? I have seen this occasionally before.
<= /blockquote>

OK. In that case let's try putting 1-2 = second delay and observer behaviour.
I'll send separate patch= for this tomorrow as this is not related to on demand query result feature= or its test cases.
=C2=A0

>>
>>
>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D
>> ERROR: runTest
>> (pgadmin.feature_tests.query_tool_tests.QueryToolFeature= Test)
>> Query tool feature test
>> ------------------------------------------------------------<= wbr>----------
>> Traceback (most recent call last):
>>=C2=A0 =C2=A0File
>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py",
>> line 119, in runTest
>>=C2=A0 =C2=A0 =C2=A0self._query_tool_auto_rollback_enabled() >>=C2=A0 =C2=A0File
>> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py",
>> line 697, in _query_tool_auto_rollback_enabled
>>=C2=A0 =C2=A0 =C2=A0'//div[contains(@class, "sql-editor-me= ssage") and contains(string(),
>> "COMMIT")]'
>>=C2=A0 =C2=A0File
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-<= wbr>packages/selenium/webdriver/remote/webdriver.py",
>> line 295, in find_element_by_xpath
>>=C2=A0 =C2=A0 =C2=A0return self.find_element(by=3DBy.XPATH, value= =3Dxpath)
>>=C2=A0 =C2=A0File
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-<= wbr>packages/selenium/webdriver/remote/webdriver.py",
>> line 756, in find_element
>>=C2=A0 =C2=A0 =C2=A0'value': value})['value']
>>=C2=A0 =C2=A0File
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-<= wbr>packages/selenium/webdriver/remote/webdriver.py",
>> line 238, in execute
>>=C2=A0 =C2=A0 =C2=A0self.error_handler.check_response(response= )
>>=C2=A0 =C2=A0File
>> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-<= wbr>packages/selenium/webdriver/remote/errorhandler.py",
>> line 193, in check_response
>>=C2=A0 =C2=A0 =C2=A0raise exception_class(message, screen, stacktra= ce)
>> NoSuchElementException: Message: no such element: Unable to locate=
>> element: {"method":"xpath","selector"= ;:"//div[contains(@class,
>> "sql-editor-message") and contains(string(), "COMMI= T")]"}
>>=C2=A0 =C2=A0(Session info: chrome=3D58.0.3029.110)
>>=C2=A0 =C2=A0(Driver info: chromedriver=3D2.29.461585
>> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=3DMa= c OS X 10.12.3 x86_64)
>>
> I have updated Auto rollback enabled test in this patch.
>
>>
>> Relevant screenshots attached.
>>
>> - Can you tidy up the regression output a little please? Instead o= f:
>>
>> -------
>> runTest (pgadmin.feature_tests.query_tool_tests.QueryToo= lFeatureTest)
>> Query tool feature test ... On demand result set on scrolling... >> OK.
>>
>> On demand result set on grid select all...
>> OK.
>>
>> On demand result set on column select all...
>> OK.
>>
>> Explain query...
>> OK.
>>
>> Explain query with verbose...
>> OK.
>>
>> Explain query with costs...
>> OK.
>>
>> Explain analyze query...
>> OK.
>>
>> Explain analyze query with buffers...
>> OK.
>>
>> Explain analyze query with timing...
>> OK.
>>
>> Auto commit disabled...
>> OK.
>>
>> Auto commit enabled...
>> OK.
>>
>> Auto rollback enabled...
>> ERROR
>> -------
>>
>> Something like:
>>
>> -------
>> runTest (pgadmin.feature_tests.query_tool_tests.QueryToo= lFeatureTest)
>> Query tool feature test ...
>> On demand result set on scrolling... OK.
>> On demand result set on grid select all... OK.
>> On demand result set on column select all... OK.
>> Explain query... OK.
>> Explain query with verbose... OK.
>> Explain query with costs... OK.
>> Explain analyze query... OK.
>> Explain analyze query with buffers... OK.
>> Explain analyze query with timing... OK.
>> Auto commit disabled... OK.
>> Auto commit enabled... OK.
>> Auto rollback enabled... ERROR
>> --------
>>
> Fixed.
>
>
>>
>> Thanks!
>>
>> --
>> Dave Page
>> Blog: http://pgsnake.blogspot.com
>> Twitter: @pgsnake
>>
>> EnterpriseDB UK: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>
>



--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--001a113ee4baf7d1fb055205df07--