public inbox for [email protected]help / color / mirror / Atom feed
Re: Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] 24+ messages / 7 participants [nested] [flat]
* Re: Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] @ 2017-06-14 14:25 Harshal Dhumal <[email protected]> 0 siblings, 1 reply; 24+ messages in thread From: Harshal Dhumal @ 2017-06-14 14:25 UTC (permalink / raw) To: Ashesh Vashi <[email protected]>; +Cc: Dave Page <[email protected]>; Harshal Dhumal <[email protected]>; pgadmin-hackers On Wed, Jun 14, 2017 at 6:21 PM, Ashesh Vashi <[email protected] > wrote: > On Wed, Jun 14, 2017 at 6:19 PM, Dave Page <[email protected]> wrote: > >> Hi, >> >> Sorry - it's drifted out again, I suspect because of the work Ashesh >> has been doing. Can you rebase please? Check with Ashesh first though, >> in case he's about ready to commit another big change. >> > I am not. :-) > > Sure, I'll send updated patch. > -- > > Thanks & Regards, > > Ashesh Vashi > EnterpriseDB INDIA: Enterprise PostgreSQL Company > <http://www.enterprisedb.com/; > > > *http://www.linkedin.com/in/asheshvashi* > <http://www.linkedin.com/in/asheshvashi; > >> >> Thanks. >> >> On Fri, Jun 9, 2017 at 10:08 AM, Harshal Dhumal >> <[email protected]> wrote: >> > Hi, >> > >> > >> > Please find rebased patch >> > >> > -- >> > Harshal Dhumal >> > Sr. Software Engineer >> > >> > EnterpriseDB India: http://www.enterprisedb.com >> > The Enterprise PostgreSQL Company >> > >> > On Thu, Jun 8, 2017 at 6:40 PM, Harshal Dhumal >> > <[email protected]> wrote: >> >> >> >> Ignore this patch. >> >> Rebase and migration of feature tests and jasmine tests required. >> >> >> >> -- >> >> Harshal Dhumal >> >> Sr. Software Engineer >> >> >> >> EnterpriseDB India: http://www.enterprisedb.com >> >> The Enterprise PostgreSQL Company >> >> >> >> On Thu, Jun 8, 2017 at 3:56 PM, Harshal Dhumal >> >> <[email protected]> wrote: >> >>> >> >>> Hi, >> >>> Please find attached updated patch for feature RM2137. >> >>> >> >>> Changes in this patch: >> >>> 1. Patch rebased. >> >>> >> >>> 2. Updated existing feature tests which requires changes due to this >> >>> feature. >> >>> affected feature test cases: >> >>> i. PGDataypeFeatureTest >> >>> ii. CheckForXssFeatureTest >> >>> >> >>> 3. Updated existing jasmine test cases which requires changes due to >> this >> >>> feature. >> >>> affected jasmine test cases: >> >>> i. copy data >> >>> ii. range_boundary_navigator >> >>> iii. row_selector >> >>> iv. set_stages_rows >> >>> >> >>> 4. New feature tests added >> >>> i. on demand result set on scrolling. >> >>> ii. on demand result set on grid select all. >> >>> iii. on demand result set on column select all. >> >>> iv. explain query >> >>> v. explain query with verbose >> >>> vi. explain query with costs >> >>> vii. explain analyze query >> >>> viii. explain analyze query with buffers >> >>> ix. explain analyze query with timing >> >>> x. auto commit disabled. >> >>> xi. auto commit enabled. >> >>> xii. auto rollback enabled. >> >>> xiii. cancel query. >> >>> >> >>> >> >>> >> >>> -- >> >>> Harshal Dhumal >> >>> Sr. Software Engineer >> >>> >> >>> EnterpriseDB India: http://www.enterprisedb.com >> >>> The Enterprise PostgreSQL Company >> >>> >> >>> On Tue, May 16, 2017 at 8:14 PM, Dave Page <[email protected]> wrote: >> >>>> >> >>>> >> >>>> >> >>>> On Mon, May 15, 2017 at 7:40 PM, Harshal Dhumal >> >>>> <[email protected]> wrote: >> >>>>> >> >>>>> Hi, >> >>>>> >> >>>>> On Sat, May 13, 2017 at 12:35 AM, Joao Pedro De Almeida Pereira >> >>>>> <[email protected]> wrote: >> >>>>>> >> >>>>>> We were only able to apply the patch on 1f903ba2 (were seeing patch >> >>>>>> does not apply due to sqleditor.js conflicts) >> >>>>>> The javascript tests passed, but we were unable to copy rows or >> >>>>>> columns or cells when running the application. Could you run >> feature tests? >> >>>>> >> >>>>> There are three modes sqleditor can be launched >> >>>>> 1. Query tool (Tools menus -> Query Tool) >> >>>>> 2. Datagrid. (Right click on any table/view -> View Data -> View >> >>>>> All/First 100/Last 100/Filtered rows) >> >>>>> 3. Scripts (Right click on any table/view -> >> >>>>> INSERT/CREATE/UPDATE/DELETE/SELECT) >> >>>>> >> >>>>> Paste functionality is only enabled in Datagrid and table has >> Primary >> >>>>> key otherwise it's disabled. In your case row might have been >> copied but you >> >>>>> were unable paste because you might be trying to paste the rows in >> Query >> >>>>> tool. Please try again in Datagrid with table having Primary key. >> >>>>> >> >>>>>> >> >>>>>> >> >>>>>> Now that more functionality is being added to sqleditor.js, this >> may >> >>>>>> be a good time to extract the functionality to separate files. >> This will >> >>>>>> increase readability, and encourage separation of concerns. It >> will also >> >>>>>> make changes easier to test in isolation. >> >>>>> >> >>>>> Ok. Let me check if I can separate out ant functionalities. >> >>>>> >> >>>>>> >> >>>>>> It's probably a good idea to test the changes made to the python as >> >>>>>> well as javascript code. In this case, the new behavior of poll() >> in >> >>>>>> sqleditor __init__ should be tested. >> >>>>> >> >>>>> >> >>>>> At this point we don't have any python unit tests that can test >> >>>>> sqleditor backend (python code). >> >>>>> @Dave should I include python unit test cases in this patch? >> >>>> >> >>>> >> >>>> We do have some feature tests that hit the query tool - Murtuza wrote >> >>>> some anti-XSS validation tests for example, and Khushboo has been >> working on >> >>>> some datatype rendering tests. >> >>>> >> >>>> As a general rule, I prefer we focus more on feature tests now than >> the >> >>>> API tests - they cover the whole app end-to-end of course. The >> disadvantages >> >>>> are: >> >>>> >> >>>> - The treeview isn't reliable enough for me to enable those tests on >> the >> >>>> CI server yet. >> >>>> >> >>>> - They can take a long time to run, so we need to test multiple >> things >> >>>> at once wherever possible. That means minimising browser reloads, or >> new >> >>>> instances of tools like the Query Tool - or even the number of >> queries >> >>>> executed through the query tool as part of a test. >> >>>> >> >>>> That said, yes, if there are specific things that are not covered by >> >>>> Murtuza and Khushboo's work, we should test them. For example, >> loading all >> >>>> rows when the user selects all, running/rendering EXPLAIN, >> auto-commit vs. >> >>>> auto-rollback (and combinations thereof). >> >>>> >> >>>> The standard moving forwards should be to include feature tests for >> new >> >>>> functionality and Jasmine tests for algorithmic JS code. >> >>>> >> >>>> I also agree with Joao on the modularisation of JS code. Testable and >> >>>> reusable code should be in "library" files, and we should work to >> minimise >> >>>> the amount of JS templates - for the most part, that means moving to >> the >> >>>> client-side translation mechanism which Tira worked on, and I've >> done some >> >>>> early migration work. >> >>>> >> >>>> Thanks. >> >>>> >> >>>> >> >>>> -- >> >>>> Dave Page >> >>>> Blog: http://pgsnake.blogspot.com >> >>>> Twitter: @pgsnake >> >>>> >> >>>> EnterpriseDB UK: http://www.enterprisedb.com >> >>>> The Enterprise PostgreSQL Company >> >>> >> >>> >> >> >> > >> >> >> >> -- >> Dave Page >> Blog: http://pgsnake.blogspot.com >> Twitter: @pgsnake >> >> EnterpriseDB UK: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> >> >> -- >> Sent via pgadmin-hackers mailing list ([email protected]) >> To make changes to your subscription: >> http://www.postgresql.org/mailpref/pgadmin-hackers >> > > ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] @ 2017-06-14 22:36 Harshal Dhumal <[email protected]> parent: Harshal Dhumal <[email protected]> 0 siblings, 1 reply; 24+ messages in thread From: Harshal Dhumal @ 2017-06-14 22:36 UTC (permalink / raw) To: Harshal Dhumal <[email protected]>; +Cc: Ashesh Vashi <[email protected]>; Dave Page <[email protected]>; pgadmin-hackers Hi Dave, Please find rebased patch for RM2137. -- *Harshal Dhumal* *Sr. Software Engineer* EnterpriseDB India: http://www.enterprisedb.com The Enterprise PostgreSQL Company On Wed, Jun 14, 2017 at 7:55 PM, Harshal Dhumal <[email protected]> wrote: > > > On Wed, Jun 14, 2017 at 6:21 PM, Ashesh Vashi < > [email protected]> wrote: > >> On Wed, Jun 14, 2017 at 6:19 PM, Dave Page <[email protected]> wrote: >> >>> Hi, >>> >>> Sorry - it's drifted out again, I suspect because of the work Ashesh >>> has been doing. Can you rebase please? Check with Ashesh first though, >>> in case he's about ready to commit another big change. >>> >> I am not. :-) >> >> Sure, I'll send updated patch. > > >> -- >> >> Thanks & Regards, >> >> Ashesh Vashi >> EnterpriseDB INDIA: Enterprise PostgreSQL Company >> <http://www.enterprisedb.com/; >> >> >> *http://www.linkedin.com/in/asheshvashi* >> <http://www.linkedin.com/in/asheshvashi; >> >>> >>> Thanks. >>> >>> On Fri, Jun 9, 2017 at 10:08 AM, Harshal Dhumal >>> <[email protected]> wrote: >>> > Hi, >>> > >>> > >>> > Please find rebased patch >>> > >>> > -- >>> > Harshal Dhumal >>> > Sr. Software Engineer >>> > >>> > EnterpriseDB India: http://www.enterprisedb.com >>> > The Enterprise PostgreSQL Company >>> > >>> > On Thu, Jun 8, 2017 at 6:40 PM, Harshal Dhumal >>> > <[email protected]> wrote: >>> >> >>> >> Ignore this patch. >>> >> Rebase and migration of feature tests and jasmine tests required. >>> >> >>> >> -- >>> >> Harshal Dhumal >>> >> Sr. Software Engineer >>> >> >>> >> EnterpriseDB India: http://www.enterprisedb.com >>> >> The Enterprise PostgreSQL Company >>> >> >>> >> On Thu, Jun 8, 2017 at 3:56 PM, Harshal Dhumal >>> >> <[email protected]> wrote: >>> >>> >>> >>> Hi, >>> >>> Please find attached updated patch for feature RM2137. >>> >>> >>> >>> Changes in this patch: >>> >>> 1. Patch rebased. >>> >>> >>> >>> 2. Updated existing feature tests which requires changes due to this >>> >>> feature. >>> >>> affected feature test cases: >>> >>> i. PGDataypeFeatureTest >>> >>> ii. CheckForXssFeatureTest >>> >>> >>> >>> 3. Updated existing jasmine test cases which requires changes due to >>> this >>> >>> feature. >>> >>> affected jasmine test cases: >>> >>> i. copy data >>> >>> ii. range_boundary_navigator >>> >>> iii. row_selector >>> >>> iv. set_stages_rows >>> >>> >>> >>> 4. New feature tests added >>> >>> i. on demand result set on scrolling. >>> >>> ii. on demand result set on grid select all. >>> >>> iii. on demand result set on column select all. >>> >>> iv. explain query >>> >>> v. explain query with verbose >>> >>> vi. explain query with costs >>> >>> vii. explain analyze query >>> >>> viii. explain analyze query with buffers >>> >>> ix. explain analyze query with timing >>> >>> x. auto commit disabled. >>> >>> xi. auto commit enabled. >>> >>> xii. auto rollback enabled. >>> >>> xiii. cancel query. >>> >>> >>> >>> >>> >>> >>> >>> -- >>> >>> Harshal Dhumal >>> >>> Sr. Software Engineer >>> >>> >>> >>> EnterpriseDB India: http://www.enterprisedb.com >>> >>> The Enterprise PostgreSQL Company >>> >>> >>> >>> On Tue, May 16, 2017 at 8:14 PM, Dave Page <[email protected]> >>> wrote: >>> >>>> >>> >>>> >>> >>>> >>> >>>> On Mon, May 15, 2017 at 7:40 PM, Harshal Dhumal >>> >>>> <[email protected]> wrote: >>> >>>>> >>> >>>>> Hi, >>> >>>>> >>> >>>>> On Sat, May 13, 2017 at 12:35 AM, Joao Pedro De Almeida Pereira >>> >>>>> <[email protected]> wrote: >>> >>>>>> >>> >>>>>> We were only able to apply the patch on 1f903ba2 (were seeing >>> patch >>> >>>>>> does not apply due to sqleditor.js conflicts) >>> >>>>>> The javascript tests passed, but we were unable to copy rows or >>> >>>>>> columns or cells when running the application. Could you run >>> feature tests? >>> >>>>> >>> >>>>> There are three modes sqleditor can be launched >>> >>>>> 1. Query tool (Tools menus -> Query Tool) >>> >>>>> 2. Datagrid. (Right click on any table/view -> View Data -> View >>> >>>>> All/First 100/Last 100/Filtered rows) >>> >>>>> 3. Scripts (Right click on any table/view -> >>> >>>>> INSERT/CREATE/UPDATE/DELETE/SELECT) >>> >>>>> >>> >>>>> Paste functionality is only enabled in Datagrid and table has >>> Primary >>> >>>>> key otherwise it's disabled. In your case row might have been >>> copied but you >>> >>>>> were unable paste because you might be trying to paste the rows in >>> Query >>> >>>>> tool. Please try again in Datagrid with table having Primary key. >>> >>>>> >>> >>>>>> >>> >>>>>> >>> >>>>>> Now that more functionality is being added to sqleditor.js, this >>> may >>> >>>>>> be a good time to extract the functionality to separate files. >>> This will >>> >>>>>> increase readability, and encourage separation of concerns. It >>> will also >>> >>>>>> make changes easier to test in isolation. >>> >>>>> >>> >>>>> Ok. Let me check if I can separate out ant functionalities. >>> >>>>> >>> >>>>>> >>> >>>>>> It's probably a good idea to test the changes made to the python >>> as >>> >>>>>> well as javascript code. In this case, the new behavior of poll() >>> in >>> >>>>>> sqleditor __init__ should be tested. >>> >>>>> >>> >>>>> >>> >>>>> At this point we don't have any python unit tests that can test >>> >>>>> sqleditor backend (python code). >>> >>>>> @Dave should I include python unit test cases in this patch? >>> >>>> >>> >>>> >>> >>>> We do have some feature tests that hit the query tool - Murtuza >>> wrote >>> >>>> some anti-XSS validation tests for example, and Khushboo has been >>> working on >>> >>>> some datatype rendering tests. >>> >>>> >>> >>>> As a general rule, I prefer we focus more on feature tests now than >>> the >>> >>>> API tests - they cover the whole app end-to-end of course. The >>> disadvantages >>> >>>> are: >>> >>>> >>> >>>> - The treeview isn't reliable enough for me to enable those tests >>> on the >>> >>>> CI server yet. >>> >>>> >>> >>>> - They can take a long time to run, so we need to test multiple >>> things >>> >>>> at once wherever possible. That means minimising browser reloads, >>> or new >>> >>>> instances of tools like the Query Tool - or even the number of >>> queries >>> >>>> executed through the query tool as part of a test. >>> >>>> >>> >>>> That said, yes, if there are specific things that are not covered by >>> >>>> Murtuza and Khushboo's work, we should test them. For example, >>> loading all >>> >>>> rows when the user selects all, running/rendering EXPLAIN, >>> auto-commit vs. >>> >>>> auto-rollback (and combinations thereof). >>> >>>> >>> >>>> The standard moving forwards should be to include feature tests for >>> new >>> >>>> functionality and Jasmine tests for algorithmic JS code. >>> >>>> >>> >>>> I also agree with Joao on the modularisation of JS code. Testable >>> and >>> >>>> reusable code should be in "library" files, and we should work to >>> minimise >>> >>>> the amount of JS templates - for the most part, that means moving >>> to the >>> >>>> client-side translation mechanism which Tira worked on, and I've >>> done some >>> >>>> early migration work. >>> >>>> >>> >>>> Thanks. >>> >>>> >>> >>>> >>> >>>> -- >>> >>>> Dave Page >>> >>>> Blog: http://pgsnake.blogspot.com >>> >>>> Twitter: @pgsnake >>> >>>> >>> >>>> EnterpriseDB UK: http://www.enterprisedb.com >>> >>>> The Enterprise PostgreSQL Company >>> >>> >>> >>> >>> >> >>> > >>> >>> >>> >>> -- >>> Dave Page >>> Blog: http://pgsnake.blogspot.com >>> Twitter: @pgsnake >>> >>> EnterpriseDB UK: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >>> >>> -- >>> Sent via pgadmin-hackers mailing list ([email protected]) >>> To make changes to your subscription: >>> http://www.postgresql.org/mailpref/pgadmin-hackers >>> >> >> > -- Sent via pgadmin-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgadmin-hackers Attachments: [text/x-patch] RM2137_query_tool_on_demand_result_V6.patch (154.9K, 3-RM2137_query_tool_on_demand_result_V6.patch) download | inline diff: diff --git a/web/config.py b/web/config.py index c3f8285..689581c 100644 --- a/web/config.py +++ b/web/config.py @@ -324,6 +324,12 @@ THREADED_MODE = True SQLALCHEMY_TRACK_MODIFICATIONS = False ########################################################################## +# Number of records to fetch in one batch in query tool when query result +# set is large. +########################################################################## +ON_DEMAND_RECORD_COUNT = 1000 + +########################################################################## # Local config settings ########################################################################## diff --git a/web/pgadmin/feature_tests/pg_datatype_validation_test.py b/web/pgadmin/feature_tests/pg_datatype_validation_test.py index 69b12f3..0ddf4cf 100644 --- a/web/pgadmin/feature_tests/pg_datatype_validation_test.py +++ b/web/pgadmin/feature_tests/pg_datatype_validation_test.py @@ -93,17 +93,18 @@ class PGDataypeFeatureTest(BaseFeatureTest): self.page.fill_codemirror_area_with(query) self.page.find_by_id("btn-flash").click() wait = WebDriverWait(self.page.driver, 5) - wait.until(EC.presence_of_element_located( - (By.XPATH, "//*[@id='0']//*[@id='datagrid']/div[5]/div/div[1]/div[2]/span"))) + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) # For every sample data-type value, check the expected output. cnt = 2 - for val in expected_output: + cells = canvas.find_elements_by_tag_name('span') + # remove first element as it is row number. + cells.pop(0) + for val, cell in zip(expected_output, cells): try: - source_code = self.page.find_by_xpath( - "//*[@id='0']//*[@id='datagrid']/div[5]/div/div[1]/div[" + str(cnt) + "]/span" - ).get_attribute('innerHTML') - + source_code = cell.get_attribute('innerHTML') PGDataypeFeatureTest.check_result( source_code, expected_output[cnt - 2] diff --git a/web/pgadmin/feature_tests/query_tool_tests.py b/web/pgadmin/feature_tests/query_tool_tests.py new file mode 100644 index 0000000..b453f9a --- /dev/null +++ b/web/pgadmin/feature_tests/query_tool_tests.py @@ -0,0 +1,763 @@ +########################################################################## +# +# pgAdmin 4 - PostgreSQL Tools +# +# Copyright (C) 2013 - 2017, The pgAdmin Development Team +# This software is released under the PostgreSQL Licence +# +########################################################################## + +from __future__ import print_function +import time +import sys +import config +from selenium.webdriver import ActionChains +from selenium.webdriver.support.ui import WebDriverWait +from selenium.webdriver.support import expected_conditions as EC +from selenium.webdriver.common.by import By +from regression.python_test_utils import test_utils +from regression.feature_utils.base_feature_test import BaseFeatureTest + + +class QueryToolFeatureTest(BaseFeatureTest): + """ + This feature test will test the different query tool features. + """ + + scenarios = [ + ("Query tool feature test", dict()) + ] + + def before(self): + connection = test_utils.get_db_connection(self.server['db'], + self.server['username'], + self.server['db_password'], + self.server['host'], + self.server['port']) + test_utils.drop_database(connection, "acceptance_test_db") + test_utils.create_database(self.server, "acceptance_test_db") + self.page.wait_for_spinner_to_disappear() + self._connects_to_server() + self._locate_database_tree_node() + self._open_query_tool() + + def runTest(self): + # on demand result set on scrolling. + print("On demand result set on scrolling...", + file=sys.stderr) + self._on_demand_result() + print("OK.\n", + file=sys.stderr) + self._clear_query_tool() + + # on demand result set on grid select all. + print("On demand result set on grid select all...", + file=sys.stderr) + self._on_demand_result_select_all_grid() + print("OK.\n", + file=sys.stderr) + self._clear_query_tool() + + # on demand result set on column select all. + print("On demand result set on column select all...", + file=sys.stderr) + self._on_demand_result_select_all_column() + print("OK.\n", + file=sys.stderr) + self._clear_query_tool() + + # explain query + print("Explain query...", file=sys.stderr) + self._query_tool_explain() + print("OK.\n", file=sys.stderr) + self._clear_query_tool() + + # explain query with verbose + print("Explain query with verbose...", file=sys.stderr) + self._query_tool_explain_verbose() + print("OK.\n", file=sys.stderr) + self._clear_query_tool() + + # explain query with costs + print("Explain query with costs...", file=sys.stderr) + self._query_tool_explain_cost() + print("OK.\n", file=sys.stderr) + self._clear_query_tool() + + # explain analyze query + print("Explain analyze query...", file=sys.stderr) + self._query_tool_explain_analyze() + print("OK.\n", file=sys.stderr) + self._clear_query_tool() + + # explain analyze query with buffers + print("Explain analyze query with buffers...", file=sys.stderr) + self._query_tool_explain_analyze_buffers() + print("OK.\n", file=sys.stderr) + self._clear_query_tool() + + # explain analyze query with timing + print("Explain analyze query with timing...", file=sys.stderr) + self._query_tool_explain_analyze_timing() + print("OK.\n", file=sys.stderr) + self._clear_query_tool() + + # auto commit disabled. + print("Auto commit disabled...", file=sys.stderr) + self._query_tool_auto_commit_disabled() + print("OK.\n", file=sys.stderr) + self._clear_query_tool() + + # auto commit enabled. + print("Auto commit enabled...", file=sys.stderr) + self._query_tool_auto_commit_enabled() + print("OK.\n", file=sys.stderr) + self._clear_query_tool() + + # auto rollback enabled. + print("Auto rollback enabled...", file=sys.stderr) + self._query_tool_auto_rollback_enabled() + print("OK.\n", file=sys.stderr) + self._clear_query_tool() + + # cancel query. + print("Cancel query...", file=sys.stderr) + self._query_tool_cancel_query() + print("OK.\n", file=sys.stderr) + self._clear_query_tool() + + def after(self): + self.page.remove_server(self.server) + connection = test_utils.get_db_connection(self.server['db'], + self.server['username'], + self.server['db_password'], + self.server['host'], + self.server['port']) + test_utils.drop_database(connection, "acceptance_test_db") + + def _connects_to_server(self): + self.page.find_by_xpath( + "//*[@class='aciTreeText' and .='Servers']").click() + self.page.driver.find_element_by_link_text("Object").click() + ActionChains(self.page.driver) \ + .move_to_element( + self.page.driver.find_element_by_link_text("Create"))\ + .perform() + self.page.find_by_partial_link_text("Server...").click() + + server_config = self.server + self.page.fill_input_by_field_name("name", server_config['name']) + self.page.find_by_partial_link_text("Connection").click() + self.page.fill_input_by_field_name("host", server_config['host']) + self.page.fill_input_by_field_name("port", server_config['port']) + self.page.fill_input_by_field_name( + "username", + server_config['username'] + ) + self.page.fill_input_by_field_name( + "password", + server_config['db_password'] + ) + self.page.find_by_xpath("//button[contains(.,'Save')]").click() + + def _locate_database_tree_node(self): + self.page.toggle_open_tree_item(self.server['name']) + self.page.toggle_open_tree_item('Databases') + self.page.toggle_open_tree_item('acceptance_test_db') + + def _open_query_tool(self): + self.page.driver.find_element_by_link_text("Tools").click() + + tools_menu = self.page.driver.find_element_by_id('mnu_tools') + + # Query Tool is first li + query_tool = tools_menu.find_element_by_tag_name('li') + + t = time.time() + # wait until Query Tool menu becomes enabled. + while time.time() - t < 20: # 20 seconds + # if menu is disabled then it will have + # two classes 'menu-item disabled'. + # And if menu is enabled the it will have + # only one class 'menu-item'. + if 'menu-item' == str(query_tool.get_attribute('class')): + break + time.sleep(0.1) + else: + assert False, "'Tools -> Query Tool' menu did not enable." + + self.page.find_by_partial_link_text("Query Tool").click() + self.page.click_tab('Query -') + + # wait until wc docker activates query tool iframe. + # time.sleep(3) + self.page.driver.switch_to_frame( + self.page.driver.find_element_by_tag_name("iframe") + ) + + def _clear_query_tool(self): + # clear codemirror. + self.page.find_by_id("btn-edit").click() + # wait for alertify dialog open animation to complete. + time.sleep(1) + + self.page.click_element(self.page.find_by_xpath("//button[contains(.,'Yes')]")) + # wait for alertify dialog close animation to complete. + time.sleep(1) + + def _on_demand_result(self): + ON_DEMAND_CHUNKS = 2 + query = """-- On demand query result on scroll +SELECT generate_series(1, {}) as id""".format( + config.ON_DEMAND_RECORD_COUNT * ON_DEMAND_CHUNKS) + + wait = WebDriverWait(self.page.driver, 10) + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-flash").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas"))) + + # scroll to bottom to fetch next chunk of result set. + self.driver.execute_script( + "$('.slick-viewport').scrollTop($('.grid-canvas').height());" + ) + + # wait for ajax to complete. + time.sleep(1) + + # again scroll to bottom to bring last row of next chunk in + # viewport. + self.driver.execute_script( + "$('.slick-viewport').scrollTop($('.grid-canvas').height());" + ) + + row_id_to_find = config.ON_DEMAND_RECORD_COUNT * ON_DEMAND_CHUNKS + + canvas.find_element_by_xpath( + '//span[text()="{}"]'.format(row_id_to_find) + ) + + def _on_demand_result_select_all_grid(self): + ON_DEMAND_CHUNKS = 3 + query = """-- On demand query result on grid select all +SELECT generate_series(1, {}) as id""".format( + config.ON_DEMAND_RECORD_COUNT * ON_DEMAND_CHUNKS) + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-flash").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, ".slick-header-column"))).click() + + # wait for until all records are fetched and selected. + time.sleep(1) + # scroll to bottom to bring last row of next chunk in + # viewport. + self.driver.execute_script( + "$('.slick-viewport').scrollTop($('.grid-canvas').height());" + ) + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + + row_id_to_find = config.ON_DEMAND_RECORD_COUNT * ON_DEMAND_CHUNKS + + canvas.find_element_by_xpath( + '//span[text()="{}"]'.format(row_id_to_find) + ) + + def _on_demand_result_select_all_column(self): + ON_DEMAND_CHUNKS = 3 + query = """-- On demand query result on column select all +SELECT generate_series(1, {}) as id1, 'dummy' as id2""".format( + config.ON_DEMAND_RECORD_COUNT * ON_DEMAND_CHUNKS) + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-flash").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + # click on first data column to select all column. + + wait.until(EC.presence_of_element_located( + ( + By.XPATH, + "//span[contains(@class, 'column-name') and contains(., 'id1')]")) + ).click() + + # wait for until all records are fetched and selected. + time.sleep(1) + # scroll to bottom to bring last row of next chunk in + # viewport. + self.driver.execute_script( + "$('.slick-viewport').scrollTop($('.grid-canvas').height());" + ) + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + + row_id_to_find = config.ON_DEMAND_RECORD_COUNT * ON_DEMAND_CHUNKS + + canvas.find_element_by_xpath( + '//span[text()="{}"]'.format(row_id_to_find) + ) + + def _query_tool_explain(self): + query = """-- Explain query +SELECT generate_series(1, 1000) as id order by id desc""" + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-query-dropdown").click() + self.page.find_by_id("btn-explain").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + self.page.click_tab('Data Output') + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + # Search for Plan word in result + canvas.find_element_by_xpath("//*[contains(string(),'Plan')]") + + def _query_tool_explain_verbose(self): + query = """-- Explain query with verbose +SELECT generate_series(1, 1000) as id order by id desc""" + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + query_op = self.page.find_by_id("btn-query-dropdown") + query_op.click() + + ActionChains(self.driver).move_to_element( + query_op.find_element_by_xpath( + "//li[contains(.,'Explain Options')]")).perform() + + self.page.find_by_id("btn-explain-verbose").click() + + self.page.find_by_id("btn-explain").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + self.page.click_tab('Data Output') + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + # Search for 'Output' word in result + canvas.find_element_by_xpath("//*[contains(string(), 'Output')]") + + def _query_tool_explain_cost(self): + query = """-- Explain query with costs +SELECT generate_series(1, 1000) as id order by id desc""" + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + query_op = self.page.find_by_id("btn-query-dropdown") + query_op.click() + + ActionChains(self.driver).move_to_element( + query_op.find_element_by_xpath( + "//li[contains(.,'Explain Options')]")).perform() + + self.page.find_by_id("btn-explain-costs").click() + + self.page.find_by_id("btn-explain").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + self.page.click_tab('Data Output') + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + # Search for 'Total Cost word in result + canvas.find_element_by_xpath("//*[contains(string(),'Total Cost')]") + + def _query_tool_explain_analyze(self): + query = """-- Explain analyze query +SELECT generate_series(1, 1000) as id order by id desc""" + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-query-dropdown").click() + self.page.find_by_id("btn-explain-analyze").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + self.page.click_tab('Data Output') + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + # Search for Actual Rows word in result + canvas.find_element_by_xpath("//*[contains(string(),'Actual Rows')]") + + def _query_tool_explain_analyze_buffers(self): + query = """-- Explain analyze query with buffers +SELECT generate_series(1, 1000) as id order by id desc""" + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + query_op = self.page.find_by_id("btn-query-dropdown") + query_op.click() + + ActionChains(self.driver).move_to_element( + query_op.find_element_by_xpath( + "//li[contains(.,'Explain Options')]")).perform() + + self.page.find_by_id("btn-explain-buffers").click() + + self.page.find_by_id("btn-explain-analyze").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + self.page.click_tab('Data Output') + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + # Search for 'Shared Read Blocks' word in result + canvas.find_element_by_xpath("//*[contains(string(), 'Shared Read Blocks')]") + + def _query_tool_explain_analyze_timing(self): + query = """-- Explain analyze query with timing +SELECT generate_series(1, 1000) as id order by id desc""" + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + query_op = self.page.find_by_id("btn-query-dropdown") + query_op.click() + + ActionChains(self.driver).move_to_element( + query_op.find_element_by_xpath( + "//li[contains(.,'Explain Options')]")).perform() + + self.page.find_by_id("btn-explain-timing").click() + + self.page.find_by_id("btn-explain-analyze").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + self.page.click_tab('Data Output') + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + # Search for 'Actual Total Time' word in result + canvas.find_element_by_xpath("//*[contains(string(), 'Actual Total Time')]") + + def _query_tool_auto_commit_disabled(self): + table_name = 'query_tool_auto_commit_disabled_table' + query = """-- 1. Disable auto commit. +-- 2. Create table in public schema. +-- 3. ROLLBACK transaction. +-- 4. Check if table is *NOT* created. +CREATE TABLE public.{}();""".format(table_name) + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-query-dropdown").click() + + auto_commit_btn = self.page.find_by_id("btn-auto-commit") + + auto_commit_check = auto_commit_btn.find_element_by_tag_name("i") + + # if auto commit is enabled then 'i' element will + # have 'auto-commit fa fa-check' classes + # if auto commit is disabled then 'i' element will + # have 'auto-commit fa fa-check visibility-hidden' classes + + if 'auto-commit fa fa-check' == str(auto_commit_check.get_attribute( + 'class')): + auto_commit_btn.click() + + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "CREATE TABLE")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) Disable auto commit. +-- 2. (Done) Create table in public schema. +-- 3. ROLLBACK transaction. +-- 4. Check if table is *NOT* created. +ROLLBACK;""" + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "ROLLBACK")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) Disable auto commit. +-- 2. (Done) Create table in public schema. +-- 3. (Done) ROLLBACK transaction. +-- 4. Check if table is *NOT* created. +SELECT relname FROM pg_class WHERE relkind IN ('r','s','t') and relnamespace = 2200::oid;""" + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Data Output') + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas"))) + + el = canvas.find_elements_by_xpath("//div[contains(@class, 'slick-cell') and contains(text(), '{}')]".format(table_name)) + + assert len(el) == 0, "Table '{}' created with auto commit disabled and without any explicit commit.".format(table_name) + + def _query_tool_auto_commit_enabled(self): + table_name = 'query_tool_auto_commit_enabled_table' + query = """-- 1. END any open transaction. +-- 2. Enable auto commit. +-- 3. Create table in public schema. +-- 4. ROLLBACK transaction +-- 5. Check if table is created event after ROLLBACK. +END; +CREATE TABLE public.{}();""".format(table_name) + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-query-dropdown").click() + + auto_commit_btn = self.page.find_by_id("btn-auto-commit") + + auto_commit_check = auto_commit_btn.find_element_by_tag_name("i") + + # if auto commit is enabled then 'i' element will + # have 'auto-commit fa fa-check' classes + # if auto commit is disabled then 'i' element will + # have 'auto-commit fa fa-check visibility-hidden' classes + + if 'auto-commit fa fa-check visibility-hidden' == str(auto_commit_check.get_attribute( + 'class')): + auto_commit_btn.click() + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "CREATE TABLE")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) END any open transaction if any. +-- 2. (Done) Enable auto commit. +-- 3. (Done) Create table in public schema. +-- 4. ROLLBACK transaction +-- 5. Check if table is created event after ROLLBACK. +ROLLBACK;""" + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "ROLLBACK")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) END any open transaction if any. +-- 2. (Done) Enable auto commit. +-- 3. (Done) Create table in public schema. +-- 4. (Done) ROLLBACK transaction +-- 5. Check if table is created event after ROLLBACK. +SELECT relname FROM pg_class WHERE relkind IN ('r','s','t') and relnamespace = 2200::oid;""" + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.click_tab('Data Output') + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas"))) + + el = canvas.find_elements_by_xpath("//div[contains(@class, 'slick-cell') and contains(text(), '{}')]".format(table_name)) + + assert len(el) != 0, "Table '{}' is not created with auto commit enabled.".format(table_name) + + def _query_tool_auto_rollback_enabled(self): + table_name = 'query_tool_auto_rollback_enabled_table' + query = """-- 1. END any open transaction. +-- 2. Enable auto rollback and disable auto commit. +-- 3. Create table in public schema. +-- 4. Generate error in transaction. +-- 5. END transaction. +-- 6. Check if table is *not* created after ending transaction. +END;""" + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self._clear_query_tool() + + query = """-- 1. (Done) END any open transaction. +-- 2. Enable auto rollback and disable auto commit. +-- 3. Create table in public schema. +-- 4. Generate error in transaction. +-- 5. END transaction. +-- 6. Check if table is *not* created after ending transaction. +CREATE TABLE public.{}();""".format(table_name) + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-query-dropdown").click() + + auto_rollback_btn = self.page.find_by_id("btn-auto-rollback") + + auto_rollback_check = auto_rollback_btn.find_element_by_tag_name("i") + + # if auto rollback is enabled then 'i' element will + # have 'auto-rollback fa fa-check' classes + # if auto rollback is disabled then 'i' element will + # have 'auto-rollback fa fa-check visibility-hidden' classes + + if 'auto-rollback fa fa-check visibility-hidden' == str(auto_rollback_check.get_attribute( + 'class')): + auto_rollback_btn.click() + + auto_commit_btn = self.page.find_by_id("btn-auto-commit") + + auto_commit_check = auto_commit_btn.find_element_by_tag_name("i") + + # if auto commit is enabled then 'i' element will + # have 'auto-commit fa fa-check' classes + # if auto commit is disabled then 'i' element will + # have 'auto-commit fa fa-check visibility-hidden' classes + + if 'auto-commit fa fa-check' == str(auto_commit_check.get_attribute( + 'class')): + auto_commit_btn.click() + + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "CREATE TABLE")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) END any open transaction. +-- 2. (Done) Enable auto rollback and disable auto commit. +-- 3. (Done) Create table in public schema. +-- 4. Generate error in transaction. +-- 5. END transaction. +-- 6. Check if table is *not* created after ending transaction. +SELECT 1/0;""" + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "division by zero")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) END any open transaction. +-- 2. (Done) Enable auto rollback and disable auto commit. +-- 3. (Done) Create table in public schema. +-- 4. (Done) Generate error in transaction. +-- 5. END transaction. +-- 6. Check if table is *not* created after ending transaction. +END;""" + + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "COMMIT")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) END any open transaction. +-- 2. (Done) Enable auto rollback and disable auto commit. +-- 3. (Done) Create table in public schema. +-- 4. (Done) Generate error in transaction. +-- 5. (Done) END transaction. +-- 6. Check if table is *not* created after ending transaction. +SELECT relname FROM pg_class WHERE relkind IN ('r','s','t') and relnamespace = 2200::oid;""" + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Data Output') + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas"))) + + el = canvas.find_elements_by_xpath("//div[contains(@class, 'slick-cell') and contains(text(), '{}')]".format(table_name)) + + assert len(el) == 0, "Table '{}' created even after ROLLBACK due to sql error.".format(table_name) + + def _query_tool_cancel_query(self): + query = """-- 1. END any open transaction. +-- 2. Enable auto commit and Disable auto rollback. +-- 3. Execute long running query. +-- 4. Cancel long running query execution. +END; +SELECT 1, pg_sleep(10)""" + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-query-dropdown").click() + + auto_rollback_btn = self.page.find_by_id("btn-auto-rollback") + + auto_rollback_check = auto_rollback_btn.find_element_by_tag_name("i") + + # if auto rollback is enabled then 'i' element will + # have 'auto-rollback fa fa-check' classes + # if auto rollback is disabled then 'i' element will + # have 'auto-rollback fa fa-check visibility-hidden' classes + + if 'auto-rollback fa fa-check' == str(auto_rollback_check.get_attribute( + 'class')): + auto_rollback_btn.click() + + auto_commit_btn = self.page.find_by_id("btn-auto-commit") + + auto_commit_check = auto_commit_btn.find_element_by_tag_name("i") + + # if auto commit is enabled then 'i' element will + # have 'auto-commit fa fa-check' classes + # if auto commit is disabled then 'i' element will + # have 'auto-commit fa fa-check visibility-hidden' classes + + if 'auto-commit fa fa-check visibility-hidden' == str(auto_commit_check.get_attribute( + 'class')): + auto_commit_btn.click() + + self.page.find_by_id("btn-flash").click() + self.driver.find_element_by_xpath("//*[@id='fetching_data']") + self.page.find_by_id("btn-cancel-query").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "canceling statement due to user request")]' + ) diff --git a/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py b/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py index 4fd0783..a0df44e 100644 --- a/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py +++ b/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py @@ -10,6 +10,9 @@ from selenium.webdriver import ActionChains from regression.python_test_utils import test_utils from regression.feature_utils.base_feature_test import BaseFeatureTest +from selenium.webdriver.support.ui import WebDriverWait +from selenium.webdriver.support import expected_conditions as EC +from selenium.webdriver.common.by import By import time class CheckForXssFeatureTest(BaseFeatureTest): @@ -152,11 +155,16 @@ class CheckForXssFeatureTest(BaseFeatureTest): self.page.fill_codemirror_area_with("select '<img src=\"x\" onerror=\"console.log(1)\">'") time.sleep(1) self.page.find_by_id("btn-flash").click() - time.sleep(2) + wait = WebDriverWait(self.page.driver, 5) - source_code = self.page.find_by_xpath( - "//*[@id='0']//*[@id='datagrid']/div[5]/div/div[1]/div[2]" - ).get_attribute('innerHTML') + result_row = self.page.find_by_xpath( + "//*[contains(@class, 'ui-widget-content') and contains(@style, 'top:0px')]" + ) + + cells = result_row.find_elements_by_tag_name('div') + + # remove first element as it is row number. + source_code = cells[1].get_attribute('innerHTML') self._check_escaped_characters( source_code, diff --git a/web/pgadmin/static/js/selection/column_selector.js b/web/pgadmin/static/js/selection/column_selector.js index 305b438..f3cb727 100644 --- a/web/pgadmin/static/js/selection/column_selector.js +++ b/web/pgadmin/static/js/selection/column_selector.js @@ -4,13 +4,18 @@ define([ 'slickgrid', ], function ($, RangeSelectionHelper) { var ColumnSelector = function () { - var Slick = window.Slick; - var gridEventBus = new Slick.EventHandler(); + var Slick = window.Slick, + gridEventBus = new Slick.EventHandler(), + onBeforeColumnSelectAll = new Slick.Event(), + onColumnSelectAll = new Slick.Event(); var init = function (grid) { gridEventBus.subscribe(grid.onHeaderClick, handleHeaderClick.bind(null, grid)); grid.getSelectionModel().onSelectedRangesChanged .subscribe(handleSelectedRangesChanged.bind(null, grid)); + onColumnSelectAll.subscribe(function(e, args) { + updateRanges(args.grid, args.column.id); + }); }; var handleHeaderClick = function (grid, event, args) { @@ -21,11 +26,20 @@ define([ if (isColumnSelectable(columnDefinition)) { var $columnHeader = $(event.target); if (hasClickedChildOfColumnHeader(event)) { + if ($(event.target).hasClass('slick-resizable-handle')) { + return; + } $columnHeader = $(event.target).parents('.slick-header-column'); } $columnHeader.toggleClass('selected'); - updateRanges(grid, columnDefinition.id); + if ($columnHeader.hasClass('selected')) { + onBeforeColumnSelectAll.notify(args, event); + } + + if (!(event.isPropagationStopped() || event.isImmediatePropagationStopped())) { + updateRanges(grid, columnDefinition.id); + } } }; @@ -107,6 +121,8 @@ define([ $.extend(this, { 'init': init, 'getColumnDefinitions': getColumnDefinitions, + 'onBeforeColumnSelectAll': onBeforeColumnSelectAll, + 'onColumnSelectAll': onColumnSelectAll, }); }; return ColumnSelector; diff --git a/web/pgadmin/static/js/selection/copy_data.js b/web/pgadmin/static/js/selection/copy_data.js index c226323..3525966 100644 --- a/web/pgadmin/static/js/selection/copy_data.js +++ b/web/pgadmin/static/js/selection/copy_data.js @@ -12,19 +12,19 @@ function ($, _, clipboard, RangeSelectionHelper, rangeBoundaryNavigator) { var grid = self.slickgrid; var columnDefinitions = grid.getColumns(); var selectedRanges = grid.getSelectionModel().getSelectedRanges(); - var data = grid.getData(); + var dataView = grid.getData(); var rows = grid.getSelectedRows(); if (RangeSelectionHelper.areAllRangesCompleteRows(grid, selectedRanges)) { self.copied_rows = rows.map(function (rowIndex) { - return data[rowIndex]; + return grid.getDataItem(rowIndex); }); setPasteRowButtonEnablement(self.can_edit, true); } else { self.copied_rows = []; setPasteRowButtonEnablement(self.can_edit, false); } - var csvText = rangeBoundaryNavigator.rangesToCsv(data, columnDefinitions, selectedRanges); + var csvText = rangeBoundaryNavigator.rangesToCsv(dataView.getItems(), columnDefinitions, selectedRanges); if (csvText) { clipboard.copyTextToClipboard(csvText); } diff --git a/web/pgadmin/static/js/selection/grid_selector.js b/web/pgadmin/static/js/selection/grid_selector.js index 24fd9d6..ddbbe23 100644 --- a/web/pgadmin/static/js/selection/grid_selector.js +++ b/web/pgadmin/static/js/selection/grid_selector.js @@ -6,21 +6,31 @@ define(['jquery', 'sources/url_for', ], function ($, gettext, ColumnSelector, RowSelector, RangeSelectionHelper, url_for) { var GridSelector = function (columnDefinitions) { - var rowSelector = new RowSelector(columnDefinitions); - var columnSelector = new ColumnSelector(columnDefinitions); + var Slick = window.Slick, + rowSelector = new RowSelector(columnDefinitions), + columnSelector = new ColumnSelector(columnDefinitions), + onBeforeGridSelectAll = new Slick.Event(), + onGridSelectAll = new Slick.Event(), + onBeforeGridColumnSelectAll = columnSelector.onBeforeColumnSelectAll, + onGridColumnSelectAll = columnSelector.onColumnSelectAll; var init = function (grid) { this.grid = grid; grid.onHeaderClick.subscribe(function (event, eventArguments) { - if (eventArguments.column.selectAllOnClick) { - toggleSelectAll(grid); + if (eventArguments.column.selectAllOnClick && !$(event.target).hasClass('slick-resizable-handle')) { + toggleSelectAll(grid, event, eventArguments); } }); grid.getSelectionModel().onSelectedRangesChanged - .subscribe(handleSelectedRangesChanged.bind(null, grid)); + .subscribe(handleSelectedRangesChanged.bind(null, grid)); + grid.registerPlugin(rowSelector); grid.registerPlugin(columnSelector); + + onGridSelectAll.subscribe(function(e, args) { + RangeSelectionHelper.selectAll(args.grid); + }); }; var getColumnDefinitions = function (columnDefinitions) { @@ -45,11 +55,14 @@ define(['jquery', } } - function toggleSelectAll(grid) { + function toggleSelectAll(grid, event, eventArguments) { if (RangeSelectionHelper.isEntireGridSelected(grid)) { selectNone(grid); } else { - RangeSelectionHelper.selectAll(grid); + onBeforeGridSelectAll.notify(eventArguments, event); + if (!(event.isPropagationStopped() || event.isImmediatePropagationStopped())) { + RangeSelectionHelper.selectAll(grid); + } } } @@ -61,6 +74,10 @@ define(['jquery', $.extend(this, { 'init': init, 'getColumnDefinitions': getColumnDefinitions, + 'onBeforeGridSelectAll': onBeforeGridSelectAll, + 'onGridSelectAll': onGridSelectAll, + 'onBeforeGridColumnSelectAll': onBeforeGridColumnSelectAll, + 'onGridColumnSelectAll': onGridColumnSelectAll, }); }; diff --git a/web/pgadmin/static/js/selection/range_boundary_navigator.js b/web/pgadmin/static/js/selection/range_boundary_navigator.js index 803f75a..274b621 100644 --- a/web/pgadmin/static/js/selection/range_boundary_navigator.js +++ b/web/pgadmin/static/js/selection/range_boundary_navigator.js @@ -58,6 +58,7 @@ function (RangeSelectionHelper) { }, rangesToCsv: function (data, columnDefinitions, selectedRanges) { + var rowRangeBounds = selectedRanges.map(function (range) { return [range.fromRow, range.toRow]; }); @@ -72,6 +73,7 @@ function (RangeSelectionHelper) { var csvRows = this.mapOver2DArray(rowRangeBounds, colRangeBounds, this.csvCell.bind(this, data, columnDefinitions), function (rowData) { return rowData.join(','); }); + return csvRows.join('\n'); }, @@ -101,7 +103,7 @@ function (RangeSelectionHelper) { }, csvCell: function (data, columnDefinitions, rowId, colId) { - var val = data[rowId][columnDefinitions[colId].pos]; + var val = data[rowId][columnDefinitions[colId].field]; if (val && _.isObject(val)) { val = '\'' + JSON.stringify(val) + '\''; diff --git a/web/pgadmin/static/js/selection/row_selector.js b/web/pgadmin/static/js/selection/row_selector.js index 236ed62..59d2fa2 100644 --- a/web/pgadmin/static/js/selection/row_selector.js +++ b/web/pgadmin/static/js/selection/row_selector.js @@ -82,7 +82,8 @@ define([ formatter: function (rowIndex) { return '<span ' + 'data-row="' + rowIndex + '" ' + - 'data-cell-type="row-header-selector"/>'; + 'data-cell-type="row-header-selector">' + + (rowIndex+1) + '</span>'; }, width: 30, }); diff --git a/web/pgadmin/static/js/selection/set_staged_rows.js b/web/pgadmin/static/js/selection/set_staged_rows.js index 46977e3..e7a529c 100644 --- a/web/pgadmin/static/js/selection/set_staged_rows.js +++ b/web/pgadmin/static/js/selection/set_staged_rows.js @@ -22,53 +22,44 @@ define( $(selector).prop('disabled', false); } - function getRowPrimaryKeyValuesToStage(selectedRows, primaryKeyColumnIndices, gridData) { + function getRowPrimaryKeyValuesToStage(selectedRows, primaryKeys, dataView, client_primary_key) { return _.reduce(selectedRows, function (primaryKeyValuesToStage, dataGridRowIndex) { - var gridRow = gridData[dataGridRowIndex]; - - if (isRowMissingPrimaryKeys(gridRow, primaryKeyColumnIndices)) { + var gridRow = dataView.getItem(dataGridRowIndex); + if (isRowMissingPrimaryKeys(gridRow, primaryKeys)) { return primaryKeyValuesToStage; } - - var tempPK = gridRow.__temp_PK; - primaryKeyValuesToStage[tempPK] = getSingleRowPrimaryKeyValueToStage(primaryKeyColumnIndices, gridRow); - + var tempPK = gridRow[client_primary_key]; + primaryKeyValuesToStage[tempPK] = getSingleRowPrimaryKeyValueToStage(primaryKeys, gridRow); return primaryKeyValuesToStage; }, {}); } - function isRowMissingPrimaryKeys(gridRow, primaryKeyColumnIndices) { + function isRowMissingPrimaryKeys(gridRow, primaryKeys) { if (_.isUndefined(gridRow)) { return true; } return !_.isUndefined( - _.find(primaryKeyColumnIndices, function (pkIndex) { - return _.isUndefined(gridRow[pkIndex]); + _.find(primaryKeys , function (pk) { + return _.isUndefined(gridRow[pk]); }) ); } - function getSingleRowPrimaryKeyValueToStage(primaryKeyColumnIndices, gridRow) { + function getSingleRowPrimaryKeyValueToStage(primaryKeys, gridRow) { var rowToStage = {}; - if (primaryKeyColumnIndices.length) { - _.each(_.keys(gridRow), function (columnPos) { - if (_.contains(primaryKeyColumnIndices, Number(columnPos))) - rowToStage[columnPos] = gridRow[columnPos]; + if (primaryKeys && primaryKeys.length) { + _.each(_.keys(gridRow), function (columnNames) { + if (_.contains(primaryKeys, columnNames)) + rowToStage[columnNames] = gridRow[columnNames]; }); } return rowToStage; } function getPrimaryKeysForSelectedRows(self, selectedRows) { - var primaryKeyColumnIndices = _.map(_.keys(self.keys), function (columnName) { - var columnInfo = _.findWhere(self.columns, {name: columnName}); - return columnInfo['pos']; - }); - - var gridData = self.grid.getData(); - var stagedRows = getRowPrimaryKeyValuesToStage(selectedRows, primaryKeyColumnIndices, gridData); - + var dataView = self.grid.getData(); + var stagedRows = getRowPrimaryKeyValuesToStage(selectedRows, _.keys(self.keys), dataView, self.client_primary_key); return stagedRows; } diff --git a/web/pgadmin/static/js/slickgrid/slick.pgadmin.editors.js b/web/pgadmin/static/js/slickgrid/slick.pgadmin.editors.js index f2dabe0..15b5fe9 100644 --- a/web/pgadmin/static/js/slickgrid/slick.pgadmin.editors.js +++ b/web/pgadmin/static/js/slickgrid/slick.pgadmin.editors.js @@ -76,18 +76,18 @@ last_value = (column_type === 'number') ? (_.isEmpty(last_value) || last_value) : last_value; - item[args.column.pos] = state; + item[args.column.field] = state; if (last_value && _.isNull(state) && (_.isUndefined(grid.copied_rows[row]) || _.isUndefined(grid.copied_rows[row][cell])) ) { - item[args.column.pos] = undefined; + item[args.column.field] = undefined; if (grid.copied_rows[row] == undefined) grid.copied_rows[row] = []; grid.copied_rows[row][cell] = 1; } } else { - item[args.column.pos] = state; + item[args.column.field] = state; } } @@ -189,14 +189,14 @@ this.loadValue = function (item) { var col = args.column; - if (_.isUndefined(item[args.column.pos]) && col.has_default_val) { + if (_.isUndefined(item[args.column.field]) && col.has_default_val) { $input.val(defaultValue = ""); } - else if (item[args.column.pos] === "") { + else if (item[args.column.field] === "") { $input.val(defaultValue = "''"); } else { - $input.val(defaultValue = item[args.column.pos]); + $input.val(defaultValue = item[args.column.field]); $input.select(); } }; @@ -323,7 +323,7 @@ }; this.loadValue = function (item) { - var data = defaultValue = item[args.column.pos]; + var data = defaultValue = item[args.column.field]; if (data && typeof data === "object" && !Array.isArray(data)) { data = JSON.stringify(data); } else if (Array.isArray(data)) { @@ -443,7 +443,7 @@ }; this.loadValue = function (item) { - $input.val(defaultValue = item[args.column.pos]); + $input.val(defaultValue = item[args.column.field]); $input.select(); }; @@ -452,7 +452,7 @@ }; this.applyValue = function (item, state) { - item[args.column.pos] = state; + item[args.column.field] = state; }; this.isValueChanged = function () { @@ -531,13 +531,13 @@ }; this.loadValue = function (item) { - defaultValue = item[args.column.pos]; - if (_.isNull(defaultValue)|| _.isUndefined(defaultValue)) { + defaultValue = item[args.column.field]; + if (_.isNull(defaultValue)||_.isUndefined(defaultValue)) { $select.prop('indeterminate', true); $select.data('checked', 2); } else { - defaultValue = !!item[args.column.pos]; + defaultValue = !!item[args.column.field]; if (defaultValue) { $select.prop('checked', true); $select.data('checked', 0); @@ -556,7 +556,7 @@ }; this.applyValue = function (item, state) { - item[args.column.pos] = state; + item[args.column.field] = state; }; this.isValueChanged = function () { @@ -648,7 +648,7 @@ }; this.loadValue = function (item) { - var data = defaultValue = item[args.column.pos]; + var data = defaultValue = item[args.column.field]; if (typeof data === "object" && !Array.isArray(data)) { data = JSON.stringify(data); } else if (Array.isArray(data)) { @@ -671,7 +671,7 @@ }; this.applyValue = function (item, state) { - item[args.column.pos] = state; + item[args.column.field] = state; }; this.isValueChanged = function () { @@ -725,7 +725,7 @@ }; this.loadValue = function (item) { - var value = item[args.column.pos]; + var value = item[args.column.field]; // Check if value is null or undefined if (value === undefined && typeof value === "undefined") { @@ -858,7 +858,7 @@ }; this.loadValue = function (item) { - defaultValue = item[args.column.pos]; + defaultValue = item[args.column.field]; $input.val(defaultValue); $input[0].defaultValue = defaultValue; $input.select(); diff --git a/web/pgadmin/templates/base.html b/web/pgadmin/templates/base.html index 829c195..7616eac 100755 --- a/web/pgadmin/templates/base.html +++ b/web/pgadmin/templates/base.html @@ -146,6 +146,12 @@ ], "exports": 'Slick.Grid' }, + "slickgrid/slick.dataview": { + "deps": [ + "slickgrid" + ], + "exports": 'Slick.Data.DataView' + }, "flotr2": { deps: ['bean'], exports: function(bean) { diff --git a/web/pgadmin/tools/sqleditor/__init__.py b/web/pgadmin/tools/sqleditor/__init__.py index 828cb99..709332c 100644 --- a/web/pgadmin/tools/sqleditor/__init__.py +++ b/web/pgadmin/tools/sqleditor/__init__.py @@ -27,7 +27,7 @@ from pgadmin.utils.sqlautocomplete.autocomplete import SQLAutoComplete from pgadmin.misc.file_manager import Filemanager -from config import PG_DEFAULT_DRIVER +from config import PG_DEFAULT_DRIVER, ON_DEMAND_RECORD_COUNT MODULE_NAME = 'sqleditor' @@ -229,13 +229,32 @@ def start_view_data(trans_id): # Check the transaction and connection status status, error_msg, conn, trans_obj, session_obj = check_transaction_status(trans_id) + + # get the default connection as current connection which is attached to + # trans id holds the cursor which has query result so we cannot use that + # connection to execute another query otherwise we'll lose query result. + + manager = get_driver(PG_DEFAULT_DRIVER).connection_manager(trans_obj.sid) + default_conn = manager.connection(did=trans_obj.did) + + # Connect to the Server if not connected. + if not default_conn.connected(): + status, msg = default_conn.connect() + if not status: + return make_json_response( + data={'status': status, 'result': u"{}".format(msg)} + ) + if status and conn is not None \ and trans_obj is not None and session_obj is not None: try: + # set fetched row count to 0 as we are executing query again. + trans_obj.update_fetched_row_cnt(0) + session_obj['command_obj'] = pickle.dumps(trans_obj, -1) # Fetch the sql and primary_keys from the object sql = trans_obj.get_sql() - pk_names, primary_keys = trans_obj.get_primary_keys() + pk_names, primary_keys = trans_obj.get_primary_keys(default_conn) # Fetch the applied filter. filter_applied = trans_obj.is_filter_applied() @@ -303,6 +322,8 @@ def start_query_tool(trans_id): # Use pickle.loads function to get the command object session_obj = grid_data[str(trans_id)] trans_obj = pickle.loads(session_obj['command_obj']) + # set fetched row count to 0 as we are executing query again. + trans_obj.update_fetched_row_cnt(0) can_edit = False can_filter = False @@ -429,65 +450,6 @@ def preferences(trans_id): return success_return() [email protected]('/columns/<int:trans_id>', methods=["GET"]) -@login_required -def get_columns(trans_id): - """ - This method will returns list of columns of last async query. - - Args: - trans_id: unique transaction id - """ - columns = dict() - columns_info = None - primary_keys = None - rset = None - status, error_msg, conn, trans_obj, session_obj = check_transaction_status(trans_id) - if status and conn is not None and session_obj is not None: - - ver = conn.manager.version - # Get the template path for the column - template_path = 'column/sql/#{0}#'.format(ver) - command_obj = pickle.loads(session_obj['command_obj']) - if hasattr(command_obj, 'obj_id'): - SQL = render_template("/".join([template_path, - 'nodes.sql']), - tid=command_obj.obj_id) - # rows with attribute not_null - status, rset = conn.execute_2darray(SQL) - if not status: - return internal_server_error(errormsg=rset) - - # Check PK column info is available or not - if 'primary_keys' in session_obj: - primary_keys = session_obj['primary_keys'] - - # Fetch column information - columns_info = conn.get_column_info() - if columns_info is not None: - for key, col in enumerate(columns_info): - col_type = dict() - col_type['type_code'] = col['type_code'] - col_type['type_name'] = None - if rset: - col_type['not_null'] = col['not_null'] = \ - rset['rows'][key]['not_null'] - - col_type['has_default_val'] = col['has_default_val'] = \ - rset['rows'][key]['has_default_val'] - - columns[col['name']] = col_type - - # As we changed the transaction object we need to - # restore it and update the session variable. - session_obj['columns_info'] = columns - update_session_grid_transaction(trans_id, session_obj) - - return make_json_response(data={'status': True, - 'columns': columns_info, - 'primary_keys': primary_keys}) - - @blueprint.route('/poll/<int:trans_id>', methods=["GET"]) @login_required def poll(trans_id): @@ -499,12 +461,21 @@ def poll(trans_id): """ result = None rows_affected = 0 + rows_fetched_from = 0 + rows_fetched_to = 0 + has_more_rows = False additional_result = [] + columns = dict() + columns_info = None + primary_keys = None + types = {} + client_primary_key = None + rset = None # Check the transaction and connection status status, error_msg, conn, trans_obj, session_obj = check_transaction_status(trans_id) if status and conn is not None and session_obj is not None: - status, result = conn.poll(formatted_exception_msg=True) + status, result = conn.poll(formatted_exception_msg=True, no_result=True) if not status: return internal_server_error(result) elif status == ASYNC_OK: @@ -519,6 +490,80 @@ def poll(trans_id): if (trans_status == TX_STATUS_INERROR and trans_obj.auto_rollback): conn.execute_void("ROLLBACK;") + + st, result = conn.async_fetchmany_2darray(ON_DEMAND_RECORD_COUNT) + if st: + if 'primary_keys' in session_obj: + primary_keys = session_obj['primary_keys'] + + # Fetch column information + columns_info = conn.get_column_info() + client_primary_key = generate_client_primary_key_name( + columns_info + ) + session_obj['client_primary_key'] = client_primary_key + + if columns_info is not None: + + command_obj = pickle.loads(session_obj['command_obj']) + if hasattr(command_obj, 'obj_id'): + # Get the template path for the column + template_path = 'column/sql/#{0}#'.format( + conn.manager.version + ) + + SQL = render_template("/".join([template_path, + 'nodes.sql']), + tid=command_obj.obj_id) + # rows with attribute not_null + colst, rset = conn.execute_2darray(SQL) + if not colst: + return internal_server_error(errormsg=rset) + + for key, col in enumerate(columns_info): + col_type = dict() + col_type['type_code'] = col['type_code'] + col_type['type_name'] = None + columns[col['name']] = col_type + + if rset: + col_type['not_null'] = col['not_null'] = \ + rset['rows'][key]['not_null'] + + col_type['has_default_val'] = \ + col['has_default_val'] = \ + rset['rows'][key]['has_default_val'] + + if columns: + st, types = fetch_pg_types(columns, trans_obj) + + if not st: + return internal_server_error(types) + + for col_info in columns.values(): + for col_type in types: + if col_type['oid'] == col_info['type_code']: + col_info['type_name'] = col_type['typname'] + + session_obj['columns_info'] = columns + # status of async_fetchmany_2darray is True and result is none + # means nothing to fetch + if result and rows_affected > -1: + res_len = len(result) + if res_len == ON_DEMAND_RECORD_COUNT: + has_more_rows = True + + if res_len > 0: + rows_fetched_from = trans_obj.get_fetched_row_cnt() + trans_obj.update_fetched_row_cnt(rows_fetched_from + res_len) + rows_fetched_from += 1 + rows_fetched_to = trans_obj.get_fetched_row_cnt() + session_obj['command_obj'] = pickle.dumps(trans_obj, -1) + + # As we changed the transaction object we need to + # restore it and update the session variable. + update_session_grid_transaction(trans_id, session_obj) + elif status == ASYNC_EXECUTION_ABORTED: status = 'Cancel' else: @@ -559,51 +604,123 @@ def poll(trans_id): data={ 'status': status, 'result': result, 'rows_affected': rows_affected, - 'additional_messages': additional_messages + 'rows_fetched_from': rows_fetched_from, + 'rows_fetched_to': rows_fetched_to, + 'additional_messages': additional_messages, + 'has_more_rows': has_more_rows, + 'colinfo': columns_info, + 'primary_keys': primary_keys, + 'types': types, + 'client_primary_key': client_primary_key } ) [email protected]('/fetch/types/<int:trans_id>', methods=["GET"]) [email protected]('/fetch/<int:trans_id>', methods=["GET"]) [email protected]('/fetch/<int:trans_id>/<int:fetch_all>', methods=["GET"]) @login_required -def fetch_pg_types(trans_id): +def fetch(trans_id, fetch_all=None): + result = None + has_more_rows = False + rows_fetched_from = 0 + rows_fetched_to = 0 + fetch_row_cnt = -1 if fetch_all == 1 else ON_DEMAND_RECORD_COUNT + + # Check the transaction and connection status + status, error_msg, conn, trans_obj, session_obj = check_transaction_status(trans_id) + if status and conn is not None and session_obj is not None: + status, result = conn.async_fetchmany_2darray(fetch_row_cnt) + if not status: + status = 'Error' + else: + status = 'Success' + res_len = len(result) + if fetch_row_cnt != -1 and res_len == ON_DEMAND_RECORD_COUNT: + has_more_rows = True + + if res_len: + rows_fetched_from = trans_obj.get_fetched_row_cnt() + trans_obj.update_fetched_row_cnt(rows_fetched_from + res_len) + rows_fetched_from += 1 + rows_fetched_to = trans_obj.get_fetched_row_cnt() + session_obj['command_obj'] = pickle.dumps(trans_obj, -1) + update_session_grid_transaction(trans_id, session_obj) + else: + status = 'NotConnected' + result = error_msg + + return make_json_response( + data={ + 'status': status, 'result': result, + 'has_more_rows': has_more_rows, + 'rows_fetched_from': rows_fetched_from, + 'rows_fetched_to': rows_fetched_to + } + ) + + +def fetch_pg_types(columns_info, trans_obj): """ This method is used to fetch the pg types, which is required to map the data type comes as a result of the query. Args: - trans_id: unique transaction id + columns_info: """ - # Check the transaction and connection status - status, error_msg, conn, trans_obj, session_obj = check_transaction_status(trans_id) - if status and conn is not None \ - and trans_obj is not None and session_obj is not None: - res = {} - if 'columns_info' in session_obj \ - and session_obj['columns_info'] is not None: + # get the default connection as current connection attached to trans id + # holds the cursor which has query result so we cannot use that connection + # to execute another query otherwise we'll lose query result. + + manager = get_driver(PG_DEFAULT_DRIVER).connection_manager(trans_obj.sid) + default_conn = manager.connection(did=trans_obj.did) + + # Connect to the Server if not connected. + res = [] + if not default_conn.connected(): + status, msg = default_conn.connect() + if not status: + return status, msg - oids = [session_obj['columns_info'][col]['type_code'] for col in session_obj['columns_info']] + oids = [columns_info[col]['type_code'] for col in columns_info] - if oids: - status, res = conn.execute_dict( - u"""SELECT oid, format_type(oid,null) as typname FROM pg_type WHERE oid IN %s ORDER BY oid; + if oids: + status, res = default_conn.execute_dict( + u"""SELECT oid, format_type(oid,null) as typname FROM pg_type WHERE oid IN %s ORDER BY oid; """, [tuple(oids)]) - if status: - # iterate through pg_types and update the type name in session object - for record in res['rows']: - for col in session_obj['columns_info']: - type_obj = session_obj['columns_info'][col] - if type_obj['type_code'] == record['oid']: - type_obj['type_name'] = record['typname'] + if not status: + return False, res - update_session_grid_transaction(trans_id, session_obj) + return status, res['rows'] else: - status = False - res = error_msg - - return make_json_response(data={'status': status, 'result': res}) + return True, [] + + +def generate_client_primary_key_name(columns_info): + temp_key = '__temp_PK' + if not columns_info: + return temp_key + + initial_temp_key_len = len(temp_key) + duplicate = False + suffix = 1 + while 1: + for col in columns_info: + if col['name'] == temp_key: + duplicate = True + break + if duplicate: + if initial_temp_key_len == len(temp_key): + temp_key += str(suffix) + suffix += 1 + else: + temp_key = temp_key[:-1] + str(suffix) + suffix += 1 + duplicate = False + else: + break + return temp_key @blueprint.route('/save/<int:trans_id>', methods=["PUT", "POST"]) @@ -615,7 +732,6 @@ def save(trans_id): Args: trans_id: unique transaction id """ - if request.data: changed_data = json.loads(request.data, encoding='utf-8') else: @@ -625,7 +741,6 @@ def save(trans_id): status, error_msg, conn, trans_obj, session_obj = check_transaction_status(trans_id) if status and conn is not None \ and trans_obj is not None and session_obj is not None: - setattr(trans_obj, 'columns_info', session_obj['columns_info']) # If there is no primary key found then return from the function. if len(session_obj['primary_keys']) <= 0 or len(changed_data) <= 0: @@ -636,7 +751,22 @@ def save(trans_id): } ) - status, res, query_res, _rowid = trans_obj.save(changed_data) + manager = get_driver(PG_DEFAULT_DRIVER).connection_manager(trans_obj.sid) + default_conn = manager.connection(did=trans_obj.did) + + # Connect to the Server if not connected. + if not default_conn.connected(): + status, msg = default_conn.connect() + if not status: + return make_json_response( + data={'status': status, 'result': u"{}".format(msg)} + ) + + status, res, query_res, _rowid = trans_obj.save( + changed_data, + session_obj['columns_info'], + session_obj['client_primary_key'], + default_conn) else: status = False res = error_msg diff --git a/web/pgadmin/tools/sqleditor/command.py b/web/pgadmin/tools/sqleditor/command.py index b7d8a78..51b1f56 100644 --- a/web/pgadmin/tools/sqleditor/command.py +++ b/web/pgadmin/tools/sqleditor/command.py @@ -258,7 +258,21 @@ class SQLFilter(object): return status, result -class GridCommand(BaseCommand, SQLFilter): +class FetchedRowTracker(object): + """ + Keeps track of fetched row count. + """ + def __init__(self, **kwargs): + self.fetched_rows = 0 + + def get_fetched_row_cnt(self): + return self.fetched_rows + + def update_fetched_row_cnt(self, rows_cnt): + self.fetched_rows = rows_cnt + + +class GridCommand(BaseCommand, SQLFilter, FetchedRowTracker): """ class GridCommand(object) @@ -290,6 +304,7 @@ class GridCommand(BaseCommand, SQLFilter): """ BaseCommand.__init__(self, **kwargs) SQLFilter.__init__(self, **kwargs) + FetchedRowTracker.__init__(self, **kwargs) # Save the connection id, command type self.conn_id = kwargs['conn_id'] if 'conn_id' in kwargs else None @@ -299,10 +314,10 @@ class GridCommand(BaseCommand, SQLFilter): if self.cmd_type == VIEW_FIRST_100_ROWS or self.cmd_type == VIEW_LAST_100_ROWS: self.limit = 100 - def get_primary_keys(self): + def get_primary_keys(self, *args, **kwargs): return None, None - def save(self, changed_data): + def save(self, changed_data, default_conn=None): return forbidden(errmsg=gettext("Data cannot be saved for the current object.")) def get_limit(self): @@ -340,14 +355,14 @@ class TableCommand(GridCommand): # call base class init to fetch the table name super(TableCommand, self).__init__(**kwargs) - def get_sql(self): + def get_sql(self, default_conn=None): """ This method is used to create a proper SQL query to fetch the data for the specified table """ # Fetch the primary keys for the table - pk_names, primary_keys = self.get_primary_keys() + pk_names, primary_keys = self.get_primary_keys(default_conn) sql_filter = self.get_filter() @@ -362,13 +377,16 @@ class TableCommand(GridCommand): return sql - def get_primary_keys(self): + def get_primary_keys(self, default_conn=None): """ This function is used to fetch the primary key columns. """ driver = get_driver(PG_DEFAULT_DRIVER) - manager = driver.connection_manager(self.sid) - conn = manager.connection(did=self.did, conn_id=self.conn_id) + if default_conn is None: + manager = driver.connection_manager(self.sid) + conn = manager.connection(did=self.did, conn_id=self.conn_id) + else: + conn = default_conn pk_names = '' primary_keys = OrderedDict() @@ -400,7 +418,11 @@ class TableCommand(GridCommand): def can_filter(self): return True - def save(self, changed_data): + def save(self, + changed_data, + columns_info, + client_primary_key='__temp_PK', + default_conn=None): """ This function is used to save the data into the database. Depending on condition it will either update or insert the @@ -408,10 +430,16 @@ class TableCommand(GridCommand): Args: changed_data: Contains data to be saved + columns_info: + default_conn: + client_primary_key: """ - - manager = get_driver(PG_DEFAULT_DRIVER).connection_manager(self.sid) - conn = manager.connection(did=self.did, conn_id=self.conn_id) + driver = get_driver(PG_DEFAULT_DRIVER) + if default_conn is None: + manager = driver.connection_manager(self.sid) + conn = manager.connection(did=self.did, conn_id=self.conn_id) + else: + conn = default_conn status = False res = None @@ -421,14 +449,6 @@ class TableCommand(GridCommand): list_of_sql = [] _rowid = None - # Replace column positions with names - def set_column_names(data): - new_data = {} - for key in data: - new_data[changed_data['columns'][int(key)]['name']] = data[key] - - return new_data - if conn.connected(): # Start the transaction @@ -454,28 +474,26 @@ class TableCommand(GridCommand): column_type = {} pk_names, primary_keys = self.get_primary_keys() - for each_col in self.columns_info: + for each_col in columns_info: if ( - self.columns_info[each_col]['not_null'] and - not self.columns_info[each_col][ + columns_info[each_col]['not_null'] and + not columns_info[each_col][ 'has_default_val'] ): column_data[each_col] = None column_type[each_col] =\ - self.columns_info[each_col]['type_name'] + columns_info[each_col]['type_name'] else: column_type[each_col] = \ - self.columns_info[each_col]['type_name'] - + columns_info[each_col]['type_name'] for each_row in changed_data[of_type]: data = changed_data[of_type][each_row]['data'] # Remove our unique tracking key - data.pop('__temp_PK', None) + data.pop(client_primary_key, None) data.pop('is_row_copied', None) - data = set_column_names(data) - data_type = set_column_names(changed_data[of_type][each_row]['data_type']) - list_of_rowid.append(data.get('__temp_PK')) + data_type = changed_data[of_type][each_row]['data_type'] + list_of_rowid.append(data.get(client_primary_key)) # Update columns value and data type # with columns having not_null=False and has @@ -497,9 +515,9 @@ class TableCommand(GridCommand): # For updated rows elif of_type == 'updated': for each_row in changed_data[of_type]: - data = set_column_names(changed_data[of_type][each_row]['data']) - pk = set_column_names(changed_data[of_type][each_row]['primary_keys']) - data_type = set_column_names(changed_data[of_type][each_row]['data_type']) + data = changed_data[of_type][each_row]['data'] + pk = changed_data[of_type][each_row]['primary_keys'] + data_type = changed_data[of_type][each_row]['data_type'] sql = render_template("/".join([self.sql_path, 'update.sql']), data_to_be_saved=data, primary_keys=pk, @@ -519,18 +537,19 @@ class TableCommand(GridCommand): rows_to_delete.append(changed_data[of_type][each_row]) # Fetch the keys for SQL generation if is_first: - # We need to covert dict_keys to normal list in Python3 - # In Python2, it's already a list & We will also fetch column names using index - keys = [ - changed_data['columns'][int(k)]['name'] - for k in list(changed_data[of_type][each_row].keys()) - ] + # We need to covert dict_keys to normal list in + # Python3 + # In Python2, it's already a list & We will also + # fetch column names using index + keys = list(changed_data[of_type][each_row].keys()) + no_of_keys = len(keys) is_first = False # Map index with column name for each row for row in rows_to_delete: for k, v in row.items(): - # Set primary key with label & delete index based mapped key + # Set primary key with label & delete index based + # mapped key try: row[changed_data['columns'][int(k)]['name']] = v except ValueError: @@ -597,7 +616,7 @@ class ViewCommand(GridCommand): # call base class init to fetch the table name super(ViewCommand, self).__init__(**kwargs) - def get_sql(self): + def get_sql(self, default_conn=None): """ This method is used to create a proper SQL query to fetch the data for the specified view @@ -652,7 +671,7 @@ class ForeignTableCommand(GridCommand): # call base class init to fetch the table name super(ForeignTableCommand, self).__init__(**kwargs) - def get_sql(self): + def get_sql(self, default_conn=None): """ This method is used to create a proper SQL query to fetch the data for the specified foreign table @@ -697,7 +716,7 @@ class CatalogCommand(GridCommand): # call base class init to fetch the table name super(CatalogCommand, self).__init__(**kwargs) - def get_sql(self): + def get_sql(self, default_conn=None): """ This method is used to create a proper SQL query to fetch the data for the specified catalog object @@ -722,7 +741,7 @@ class CatalogCommand(GridCommand): return True -class QueryToolCommand(BaseCommand): +class QueryToolCommand(BaseCommand, FetchedRowTracker): """ class QueryToolCommand(BaseCommand) @@ -732,13 +751,15 @@ class QueryToolCommand(BaseCommand): def __init__(self, **kwargs): # call base class init to fetch the table name - super(QueryToolCommand, self).__init__(**kwargs) + + BaseCommand.__init__(self, **kwargs) + FetchedRowTracker.__init__(self, **kwargs) self.conn_id = None self.auto_rollback = False self.auto_commit = True - def get_sql(self): + def get_sql(self, default_conn=None): return None def can_edit(self): diff --git a/web/pgadmin/tools/sqleditor/static/css/sqleditor.css b/web/pgadmin/tools/sqleditor/static/css/sqleditor.css index 7266b34..8ed5033 100644 --- a/web/pgadmin/tools/sqleditor/static/css/sqleditor.css +++ b/web/pgadmin/tools/sqleditor/static/css/sqleditor.css @@ -460,6 +460,11 @@ input.editor-checkbox:focus { background-color: #2C76B4; } +.slick-cell span[data-cell-type="row-header-selector"] { + display: block; + text-align: right; +} + #datagrid div.slick-header.ui-state-default { background: #ffffff; border-bottom: none; diff --git a/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js b/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js index b9f017e..ad8a20b 100644 --- a/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js +++ b/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js @@ -33,7 +33,8 @@ define([ 'slickgrid/plugins/slick.cellselectionmodel', 'slickgrid/plugins/slick.cellcopymanager', 'slickgrid/plugins/slick.rowselectionmodel', - 'slickgrid/slick.grid' + 'slickgrid/slick.grid', + 'slickgrid/slick.dataview' ], function( $, _, S, alertify, pgAdmin, Backbone, Backgrid, CodeMirror, pgExplain, GridSelector, ActiveCellCapture, clipboard, copyData, RangeSelectionHelper, handleQueryOutputKeyboardEvent, @@ -470,7 +471,7 @@ define([ - staged_rows: This will hold all the data which user copies/pastes/deletes in grid - deleted: - This will hold all the data which user delets in grid + This will hold all the data which user deletes in grid Events handling: ---------------- @@ -486,34 +487,10 @@ define([ - We are using this event for Copy operation on grid */ - // Get the item column value using a custom 'fieldIdx' column param - get_item_column_value: function (item, column) { - if (column.pos !== undefined) { - return item[column.pos]; - } else { - return null; - } - }, - // This function is responsible to create and render the SlickGrid. - render_grid: function(collection, columns, is_editable) { + render_grid: function(collection, columns, is_editable, client_primary_key) { var self = this; - // returns primary keys - self.handler.get_row_primary_key = function() { - var self = this, - tmp_keys = []; - _.each(self.primary_keys, function(p, idx) { - // For each columns search primary key position - _.each(self.columns, function(c) { - if(c.name == idx) { - tmp_keys.push(c.pos); - } - }); - }); - return tmp_keys; - }; - // This will work as data store and holds all the // inserted/updated/deleted data from grid self.handler.data_store = { @@ -528,8 +505,9 @@ define([ // To store primary keys before they gets changed self.handler.primary_keys_data = {}; - // Add getItemMetadata into handler for later use - self.handler.data_view = collection; + self.client_primary_key = client_primary_key; + + self.client_primary_key_counter = 0; // Remove any existing grid first if (self.handler.slickgrid) { @@ -607,7 +585,7 @@ define([ }); var gridSelector = new GridSelector(); - grid_columns = gridSelector.getColumnDefinitions(grid_columns); + grid_columns = self.grid_columns = gridSelector.getColumnDefinitions(grid_columns); var grid_options = { editable: true, @@ -615,8 +593,7 @@ define([ enableCellNavigation: true, enableColumnReorder: false, asyncEditorLoading: false, - autoEdit: false, - dataItemColumnValueExtractor: this.get_item_column_value + autoEdit: false }; var $data_grid = self.$el.find('#datagrid'); @@ -624,17 +601,15 @@ define([ var grid_height = $($('#editor-panel').find('.wcFrame')[1]).height() - 35; $data_grid.height(grid_height); - // Add our own custom primary key to keep track of changes - _.each(collection, function(row){ - row['__temp_PK'] = SqlEditorUtils.epicRandomString(15); - }); + var dataView = self.dataView = new Slick.Data.DataView(), + grid = self.grid = new Slick.Grid($data_grid, dataView, grid_columns, grid_options); // Add-on function which allow us to identify the faulty row after insert/update // and apply css accordingly - collection.getItemMetadata = function(i) { - var res = {}, - cssClass = '', - data_store = self.handler.data_store; + + dataView.getItemMetadata = function(i) { + var res = {}, cssClass = '', + data_store = self.handler.data_store; if (_.has(self.handler, 'data_store')) { if (i in data_store.added_index && @@ -657,9 +632,8 @@ define([ cssClass += ' disabled_row'; } return {'cssClasses': cssClass}; - } + }; - var grid = new Slick.Grid($data_grid, collection, grid_columns, grid_options); grid.registerPlugin( new Slick.AutoTooltips({ enableForHeaderCells: false }) ); grid.registerPlugin(new ActiveCellCapture()); grid.setSelectionModel(new XCellSelectionModel()); @@ -671,7 +645,8 @@ define([ columns: columns, grid: grid, selection: grid.getSelectionModel(), - editor: self + editor: self, + client_primary_key: self.client_primary_key }; self.handler.slickgrid = grid; @@ -690,6 +665,42 @@ define([ }); }); + gridSelector.onBeforeGridSelectAll.subscribe(function(e, args) { + if (self.handler.has_more_rows) { + // this will prevent selection un-till we load all data + e.stopImmediatePropagation(); + self.fetch_next_all(function() { + // since we've stopped event propagation we need to + // trigger onGridSelectAll manually with new event data. + gridSelector.onGridSelectAll.notify(args, new Slick.EventData()); + }); + } + }); + + gridSelector.onBeforeGridColumnSelectAll.subscribe(function(e, args) { + if (self.handler.has_more_rows) { + // this will prevent selection un-till we load all data + e.stopImmediatePropagation(); + self.fetch_next_all(function() { + // since we've stopped event propagation we need to + // trigger onGridColumnSelectAll manually with new event data. + gridSelector.onGridColumnSelectAll.notify(args, new Slick.EventData()); + }); + } + }); + + // listen for row count change. + dataView.onRowCountChanged.subscribe(function (e, args) { + grid.updateRowCount(); + grid.render(); + }); + + // listen for rows change. + dataView.onRowsChanged.subscribe(function (e, args) { + grid.invalidateRows(args.rows); + grid.render(); + }); + // Listener function which will be called before user updates existing cell // This will be used to collect primary key for that row grid.onBeforeEditCell.subscribe(function (e, args) { @@ -702,8 +713,8 @@ define([ return false; } - if(self.handler.can_edit && before_data && '__temp_PK' in before_data) { - var _pk = before_data.__temp_PK, + if(self.handler.can_edit && before_data && self.client_primary_key in before_data) { + var _pk = before_data[self.client_primary_key], _keys = self.handler.primary_keys, current_pk = {}, each_pk_key = {}; @@ -715,22 +726,34 @@ define([ // Fetch primary keys for the row before they gets modified var _columns = self.handler.columns; _.each(_keys, function(value, key) { - pos = _.where(_columns, {name: key})[0]['pos'] - current_pk[pos] = before_data[pos]; + current_pk[key] = before_data[key]; }); // Place it in main variable for later use self.handler.primary_keys_data[_pk] = current_pk } }); - grid.onKeyDown.subscribe(handleQueryOutputKeyboardEvent); + grid.onKeyDown.subscribe(function(event, args) { + var KEY_A = 65; + var modifiedKey = event.keyCode; + var isModifierDown = event.ctrlKey || event.metaKey; + // Intercept Ctrl/Cmd + A key board event. + // As we might want to load all rows before selecting all. + if (isModifierDown && modifiedKey == KEY_A && self.handler.has_more_rows) { + self.fetch_next_all(function() { + handleQueryOutputKeyboardEvent(event, args); + }); + } else { + handleQueryOutputKeyboardEvent(event, args); + } + }); // Listener function which will be called when user updates existing rows grid.onCellChange.subscribe(function (e, args) { // self.handler.data_store.updated will holds all the updated data - var changed_column = args.grid.getColumns()[args.cell].pos, // Current field pos + var changed_column = args.grid.getColumns()[args.cell].field, updated_data = args.item[changed_column], // New value for current field - _pk = args.item.__temp_PK || null, // Unique key to identify row + _pk = args.item[self.client_primary_key] || null, // Unique key to identify row column_data = {}, _type; @@ -740,11 +763,16 @@ define([ // so that cell edit is enabled for that row. var grid = args.grid, row_data = grid.getDataItem(args.row), - p_keys_list = _.pick( - row_data, self.handler.get_row_primary_key() - ), - is_primary_key = Object.keys(p_keys_list).length ? - p_keys_list[0] : undefined; + is_primary_key = _.all( + _.values( + _.pick( + row_data, self.primary_keys + ) + ), + function(val) { + return val != undefined + } + ); // temp_new_rows is available only for view data. if (is_primary_key && self.handler.temp_new_rows) { @@ -764,7 +792,7 @@ define([ column_data); //Find type for current column self.handler.data_store.added[_pk]['err'] = false - self.handler.data_store.added[_pk]['data_type'][changed_column] = _.where(this.columns, {pos: changed_column})[0]['type']; + self.handler.data_store.added[_pk]['data_type'][changed_column] = _.where(this.columns, {name: changed_column})[0]['type']; // Check if it is updated data from existing rows? } else if(_pk in self.handler.data_store.updated) { _.extend( @@ -774,7 +802,7 @@ define([ self.handler.data_store.updated[_pk]['err'] = false //Find type for current column - self.handler.data_store.updated[_pk]['data_type'][changed_column] = _.where(this.columns, {pos: changed_column})[0]['type']; + self.handler.data_store.updated[_pk]['data_type'][changed_column] = _.where(this.columns, {name: changed_column})[0]['type']; } else { // First updated data for this primary key self.handler.data_store.updated[_pk] = { @@ -784,7 +812,7 @@ define([ self.handler.data_store.updated_index[args.row] = _pk; // Find & add column data type for current changed column var temp = {}; - temp[changed_column] = _.where(this.columns, {pos: changed_column})[0]['type']; + temp[changed_column] = _.where(this.columns, {name: changed_column})[0]['type']; self.handler.data_store.updated[_pk]['data_type'] = temp; } } @@ -803,11 +831,10 @@ define([ // Listener function which will be called when user adds new rows grid.onAddNewRow.subscribe(function (e, args) { // self.handler.data_store.added will holds all the newly added rows/data - var _key = SqlEditorUtils.epicRandomString(10), - column = args.column, - item = args.item, - data_length = this.grid.getDataLength(), - new_collection = args.grid.getData(); + var column = args.column, + item = args.item, data_length = this.grid.getDataLength(), + _key = (self.client_primary_key_counter++).toString(), + dataView = this.grid.getData(); // Add new row in list to keep track of it if (_.isUndefined(item[0])) { @@ -816,17 +843,16 @@ define([ // If copied item has already primary key, use it. if(item) { - item.__temp_PK = _key; + item[self.client_primary_key] = _key; } - new_collection.push(item); + dataView.addItem(item); self.handler.data_store.added[_key] = {'err': false, 'data': item}; self.handler.data_store.added_index[data_length] = _key; // Fetch data type & add it for the column var temp = {}; - temp[column.pos] = _.where(this.columns, {pos: column.pos})[0]['type']; + temp[column.name] = _.where(this.columns, {pos: column.pos})[0]['type']; self.handler.data_store.added[_key]['data_type'] = temp; - grid.invalidateRows([new_collection.length - 1]); grid.updateRowCount(); grid.render(); @@ -839,6 +865,16 @@ define([ $("#btn-save").prop('disabled', false); }.bind(editor_data)); + // Listen grid viewportChanged event to load next chunk of data. + grid.onViewportChanged.subscribe(function(e, args) { + var rendered_range = args.grid.getRenderedRange(), + data_len = args.grid.getDataLength(); + // start fetching next batch of records before reaching to bottom. + if (self.handler.has_more_rows && !self.handler.fetching_rows && rendered_range.bottom > data_len - 100) { + // fetch asynchronous + setTimeout(self.fetch_next.bind(self)); + } + }) // Resize SlickGrid when window resize $( window ).resize( function() { // Resize grid only when 'Data Output' panel is visible. @@ -861,6 +897,84 @@ define([ if(self.data_output_panel.isVisible()) self.grid_resize(grid); }); + + for (var i = 0; i < collection.length; i++) { + // Convert to dict from 2darray + var item = {}; + for (var j = 1; j < grid_columns.length; j++) { + item[grid_columns[j]['field']] = collection[i][grid_columns[j]['pos']] + } + + item[self.client_primary_key] = (self.client_primary_key_counter++).toString(); + collection[i] = item; + } + dataView.setItems(collection, self.client_primary_key); + }, + fetch_next_all(cb) { + this.fetch_next(true, cb); + }, + fetch_next: function(fetch_all, cb) { + var self = this; + + // This will prevent fetch operation if previous fetch operation is + // already in progress. + self.fetching_rows = true; + + $("#btn-flash").prop('disabled', true); + + if (fetch_all) { + self.handler.trigger( + 'pgadmin-sqleditor:loading-icon:show', + gettext('Fetching all records...') + ); + } + + $.ajax({ + url: "{{ url_for('sqleditor.index') }}" + "fetch/" + self.transId + (fetch_all ? "/1": ""), + method: 'GET', + success: function(res) { + self.handler.has_more_rows = res.data.has_more_rows; + $("#btn-flash").prop('disabled', false); + self.handler.trigger('pgadmin-sqleditor:loading-icon:hide'); + self.update_grid_data(res.data.result); + if (typeof cb == "function") { + cb(); + } + }, + error: function(e) { + $("#btn-flash").prop('disabled', false); + self.handler.trigger('pgadmin-sqleditor:loading-icon:hide'); + self.handler.has_more_rows = false; + self.handler.fetching_rows = false; + if (typeof cb == "function") { + cb(); + } + if (e.readyState == 0) { + self.update_msg_history(false, + gettext('Not connected to the server or the connection to the server has been closed.') + ); + return; + } + } + }); + }, + + update_grid_data: function(data) { + this.dataView.beginUpdate(); + + for (var i = 0; i < data.length; i++) { + // Convert 2darray to dict. + var item = {}; + for (var j = 1; j < this.grid_columns.length; j++) { + item[this.grid_columns[j]['field']] = data[i][this.grid_columns[j]['pos']] + } + + item[this.client_primary_key] = (this.client_primary_key_counter++).toString(); + this.dataView.addItem(item); + } + + this.dataView.endUpdate(); + this.handler.fetching_rows = false; }, /* This function is responsible to render output grid */ @@ -1402,6 +1516,8 @@ define([ self.explain_buffers = false; self.explain_timing = false; self.is_new_browser_tab = is_new_browser_tab; + self.has_more_rows = false; + self.fetching_rows = false; // We do not allow to call the start multiple times. if (self.gridView) @@ -1509,6 +1625,8 @@ define([ self.rows_to_disable = new Array(); // Temporarily hold new rows added self.temp_new_rows = new Array(); + self.has_more_rows = false; + self.fetching_rows = false; self.trigger( 'pgadmin-sqleditor:loading-icon:show', @@ -1587,45 +1705,14 @@ define([ }); }, - // This function makes the ajax call to fetch columns for last async query, - get_columns: function(poll_result) { - var self = this; - // Check the flag and decide if we need to fetch columns from server - // or use the columns data stored locally from previous call? - if (self.FETCH_COLUMNS_FROM_SERVER) { - $.ajax({ - url: "{{ url_for('sqleditor.index') }}" + "columns/" + self.transId, - method: 'GET', - success: function(res) { - poll_result.colinfo = res.data.columns; - poll_result.primary_keys = res.data.primary_keys; - self.call_render_after_poll(poll_result); - // Set a flag to get columns to false & set the value for future use - self.FETCH_COLUMNS_FROM_SERVER = false; - self.COLUMNS_DATA = res; - }, - error: function(e) { - var msg = e.responseText; - if (e.responseJSON != undefined && e.responseJSON.errormsg != undefined) - msg = e.responseJSON.errormsg; - alertify.error(msg, 5); - } - }); - } else { - // Use the previously saved columns data - poll_result.colinfo = self.COLUMNS_DATA.data.columns; - poll_result.primary_keys = self.COLUMNS_DATA.data.primary_keys; - self.call_render_after_poll(poll_result); - } - }, - // This is a wrapper to call _render function // We need this because we have separated columns route & result route // We need to combine both result here in wrapper before rendering grid call_render_after_poll: function(res) { var self = this; self.query_end_time = new Date(); - self.rows_affected = res.rows_affected; + self.rows_affected = res.rows_affected, + self.has_more_rows = res.has_more_rows; /* If no column information is available it means query runs successfully with no result to display. In this @@ -1674,7 +1761,8 @@ define([ 'pgadmin-sqleditor:loading-icon:message', gettext("Loading data from the database server and rendering...") ); - self.get_columns(res.data); + + self.call_render_after_poll(res.data); } else if (res.data.status === 'Busy') { // If status is Busy then poll the result by recursive call to the poll function @@ -1734,6 +1822,7 @@ define([ var self = this; self.colinfo = data.col_info; self.primary_keys = data.primary_keys; + self.client_primary_key = data.client_primary_key; self.cell_selected = false; self.selected_model = null; self.changedModels = []; @@ -1786,7 +1875,7 @@ define([ self.total_time = self.get_query_run_time(self.query_start_time, self.query_end_time); self.update_msg_history(true, "", false); var msg1 = S(gettext("Total query runtime: %s.")).sprintf(self.total_time).value(); - var msg2 = S(gettext("%s rows retrieved.")).sprintf(self.rows_affected).value(); + var msg2 = S(gettext("%s rows affected.")).sprintf(self.rows_affected).value(); // Display the notifier if the timeout is set to >= 0 if (self.info_notifier_timeout >= 0) { @@ -1822,7 +1911,8 @@ define([ setTimeout( function() { self.gridView.render_grid( - explain_data_array, self.columns, self.can_edit + explain_data_array, self.columns, self.can_edit, + self.client_primary_key ); // Make sure - the 'Explain' panel is visible, before - we // start rendering the grid. @@ -1838,7 +1928,8 @@ define([ self.gridView.data_output_panel.focus(); setTimeout( function() { - self.gridView.render_grid(data.result, self.columns, self.can_edit); + self.gridView.render_grid(data.result, self.columns, + self.can_edit, self.client_primary_key); }, 10 ); } @@ -1846,136 +1937,114 @@ define([ // Hide the loading icon self.trigger('pgadmin-sqleditor:loading-icon:hide'); $("#btn-flash").prop('disabled', false); - }.bind(self), - function() { - this.trigger('pgadmin-sqleditor:loading-icon:hide'); - $("#btn-flash").prop('disabled', false); }.bind(self) ); }, // This function creates the columns as required by the backgrid - _fetch_column_metadata: function(data, cb, _fail) { + _fetch_column_metadata: function(data, cb) { var colinfo = data.colinfo, primary_keys = data.primary_keys, result = data.result, columns = [], self = this; + // Store pg_types in an array + var pg_types = new Array(); + _.each(data.types, function(r) { + pg_types[r.oid] = [r.typname]; + }); - self.trigger( - 'pgadmin-sqleditor:loading-icon:message', - gettext("Retrieving information about the columns returned...") - ); - - // Make ajax call to fetch the pg types to map numeric data type - $.ajax({ - url: "{{ url_for('sqleditor.index') }}" + "fetch/types/" + self.transId, - method: 'GET', - success: function(res) { - if (res.data.status) { - // Store pg_types in an array - var pg_types = new Array(); - _.each(res.data.result.rows, function(r) { - pg_types[r.oid] = [r.typname]; - }); - - // Create columns required by backgrid to render - _.each(colinfo, function(c) { - var is_primary_key = false; + // Create columns required by slick grid to render + _.each(colinfo, function(c) { + var is_primary_key = false; - // Check whether table have primary key - if (_.size(primary_keys) > 0) { - _.each(primary_keys, function (value, key) { - if (key === c.name) - is_primary_key = true; - }); - } + // Check whether table have primary key + if (_.size(primary_keys) > 0) { + _.each(primary_keys, function (value, key) { + if (key === c.name) + is_primary_key = true; + }); + } - // To show column label and data type in multiline, - // The elements should be put inside the div. - // Create column label and type. - var col_type = column_label = ''; - var type = pg_types[c.type_code] ? - pg_types[c.type_code][0] : - // This is the case where user might - // have use casting so we will use type - // returned by cast function - pg_types[pg_types.length - 1][0] ? - pg_types[pg_types.length - 1][0] : 'unknown'; - - if (!is_primary_key) - col_type += ' ' + type; - else - col_type += ' [PK] ' + type; + // To show column label and data type in multiline, + // The elements should be put inside the div. + // Create column label and type. + var col_type = column_label = ''; + var type = pg_types[c.type_code] ? + pg_types[c.type_code][0] : + // This is the case where user might + // have use casting so we will use type + // returned by cast function + pg_types[pg_types.length - 1][0] ? + pg_types[pg_types.length - 1][0] : 'unknown'; + + if (!is_primary_key) + col_type += ' ' + type; + else + col_type += ' [PK] ' + type; - if (c.precision && c.precision >= 0 && c.precision != 65535) { - col_type += ' (' + c.precision; - col_type += c.scale && c.scale != 65535 ? - ',' + c.scale + ')': - ')'; - } + if (c.precision && c.precision >= 0 && c.precision != 65535) { + col_type += ' (' + c.precision; + col_type += c.scale && c.scale != 65535 ? + ',' + c.scale + ')': + ')'; + } - // Identify cell type of column. - switch(type) { - case "json": - case "json[]": - case "jsonb": - case "jsonb[]": - col_cell = 'Json'; - break; - case "smallint": - case "integer": - case "bigint": - case "decimal": - case "numeric": - case "real": - case "double precision": - col_cell = 'number'; - break; - case "boolean": - col_cell = 'boolean'; - break; - case "character": - case "character[]": - case "character varying": - case "character varying[]": - if (c.internal_size && c.internal_size >= 0 && c.internal_size != 65535) { - // Update column type to display length on column header - col_type += ' (' + c.internal_size + ')'; - } - col_cell = 'string'; - break; - default: - col_cell = 'string'; - } + // Identify cell type of column. + switch(type) { + case "json": + case "json[]": + case "jsonb": + case "jsonb[]": + col_cell = 'Json'; + break; + case "smallint": + case "integer": + case "bigint": + case "decimal": + case "numeric": + case "real": + case "double precision": + col_cell = 'number'; + break; + case "boolean": + col_cell = 'boolean'; + break; + case "character": + case "character[]": + case "character varying": + case "character varying[]": + if (c.internal_size && c.internal_size >= 0 && c.internal_size != 65535) { + // Update column type to display length on column header + col_type += ' (' + c.internal_size + ')'; + } + col_cell = 'string'; + break; + default: + col_cell = 'string'; + } - column_label = c.display_name + '<br>' + col_type; - - var col = { - 'name': c.name, - 'pos': c.pos, - 'label': column_label, - 'cell': col_cell, - 'can_edit': self.can_edit, - 'type': type, - 'display_name': c.display_name, - 'column_type': col_type, - 'not_null': c.not_null, - 'has_default_val': c.has_default_val - }; - columns.push(col); - }); - } - else { - alertify.alert('Fetching Type Error', res.data.result); - } - self.columns = columns; - if (cb && typeof(cb) == 'function') { - cb(); - } - }, - fail: _fail + column_label = c.display_name + '<br>' + col_type; + + var col = { + 'name': c.name, + 'display_name': c.display_name, + 'column_type': col_type, + 'pos': c.pos, + 'label': column_label, + 'cell': col_cell, + 'can_edit': self.can_edit, + 'type': type, + 'not_null': c.not_null, + 'has_default_val': c.has_default_val + }; + columns.push(col); }); + + self.columns = columns; + if (cb && typeof(cb) == 'function') { + cb(); + } }, // This function is used to raise appropriate message. @@ -1986,22 +2055,21 @@ define([ self.gridView.messages_panel.focus(); - if (self.is_query_tool) { - if (clear_grid) { - // Delete grid - if (self.gridView.handler.slickgrid) { - self.gridView.handler.slickgrid.destroy(); + if (clear_grid) { + // Delete grid + if (self.gridView.handler.slickgrid) { + self.gridView.handler.slickgrid.destroy(); - } - // Misc cleaning - self.columns = undefined; - self.collection = undefined; - - $('.sql-editor-message').text(msg); - } else { - $('.sql-editor-message').append(msg); } + // Misc cleaning + self.columns = undefined; + self.collection = undefined; + + $('.sql-editor-message').text(msg); + } else { + $('.sql-editor-message').append(msg); } + // Scroll automatically when msgs appends to element setTimeout(function(){ $(".sql-editor-message").scrollTop($(".sql-editor-message")[0].scrollHeight);; @@ -2055,7 +2123,7 @@ define([ rows_to_delete: function(data) { var self = this, - tmp_keys = self.get_row_primary_key.call(self); + tmp_keys = self.primary_keys; // re-calculate rows with no primary keys self.temp_new_rows = []; @@ -2068,7 +2136,6 @@ define([ self.temp_new_rows.push(idx); } }); - data.getItemMetadata = self.data_view.getItemMetadata; self.rows_to_disable = _.clone(self.temp_new_rows); }, @@ -2079,69 +2146,73 @@ define([ is_added = _.size(self.data_store.added), is_updated = _.size(self.data_store.updated); - // Remove newly added rows from staged rows as we don't want to send them on server - if(is_added) { - _.each(self.data_store.added, function(val, key) { - if(key in self.data_store.staged_rows) { - // Remove the row from data store so that we do not send it on server - deleted_keys.push(key); - delete self.data_store.staged_rows[key]; - delete self.data_store.added[key] - } - }); + // Remove newly added rows from staged rows as we don't want to send them on server + if(is_added) { + _.each(self.data_store.added, function(val, key) { + if(key in self.data_store.staged_rows) { + // Remove the row from data store so that we do not send it on server + deleted_keys.push(key); + delete self.data_store.staged_rows[key]; + delete self.data_store.added[key]; + delete self.data_store.added_index[key]; } - - // If only newly rows to delete and no data is there to send on server - // then just re-render the grid - if(_.size(self.data_store.staged_rows) == 0) { - var grid = self.slickgrid, data = grid.getData(), idx = 0; - if(deleted_keys.length){ - // Remove new rows from grid data using deleted keys - data = _.reject(data, function(d){ - return (d && _.indexOf(deleted_keys, d.__temp_PK) > -1) - }); - } - self.rows_to_delete.apply(self, [data]); - grid.resetActiveCell(); - grid.setData(data, true); - grid.setSelectedRows([]); - grid.invalidate(); - // Nothing to copy or delete here - $("#btn-delete-row").prop('disabled', true); - $("#btn-copy-row").prop('disabled', true); - if(_.size(self.data_store.added) || is_updated) { - // Do not disable save button if there are - // any other changes present in grid data - $("#btn-save").prop('disabled', false); - } else { - $("#btn-save").prop('disabled', true); - } - alertify.success(gettext("Row(s) deleted")); + }); + } + // If only newly rows to delete and no data is there to send on server + // then just re-render the grid + if(_.size(self.data_store.staged_rows) == 0) { + var grid = self.slickgrid, + dataView = grid.getData(), + data = dataView.getItems(), + idx = 0; + + grid.resetActiveCell(); + + dataView.beginUpdate(); + for (var i = 0; i < deleted_keys.length; i++) { + dataView.deleteItem(deleted_keys[i]); + } + dataView.endUpdate(); + self.rows_to_delete.apply(self, [dataView.getItems()]); + grid.resetActiveCell(); + grid.setSelectedRows([]); + grid.invalidate(); + + // Nothing to copy or delete here + $("#btn-delete-row").prop('disabled', true); + $("#btn-copy-row").prop('disabled', true); + if(_.size(self.data_store.added) || is_updated) { + // Do not disable save button if there are + // any other changes present in grid data + $("#btn-save").prop('disabled', false); } else { - // There are other data to needs to be updated on server - if(is_updated) { - alertify.alert(gettext("Operation failed"), + $("#btn-save").prop('disabled', true); + } + alertify.success(gettext("Row(s) deleted")); + } else { + // There are other data to needs to be updated on server + if(is_updated) { + alertify.alert(gettext("Operation failed"), gettext("There are unsaved changes in grid, Please save them first to avoid inconsistency in data") ); - return; - } - alertify.confirm(gettext("Delete Row(s)"), + return; + } + alertify.confirm(gettext("Delete Row(s)"), gettext("Are you sure you wish to delete selected row(s)?"), - function() { - $("#btn-delete-row").prop('disabled', true); - $("#btn-copy-row").prop('disabled', true); - // Change the state - self.data_store.deleted = self.data_store.staged_rows; - self.data_store.staged_rows = {}; - // Save the changes on server - self._save(); - }, - function() { - // Do nothing as user canceled the operation. - } - ).set('labels', {ok:'Yes', cancel:'No'}); + function() { + $("#btn-delete-row").prop('disabled', true); + $("#btn-copy-row").prop('disabled', true); + // Change the state + self.data_store.deleted = self.data_store.staged_rows; + self.data_store.staged_rows = {}; + // Save the changes on server + self._save(); + }, + function() { + // Do nothing as user canceled the operation. } - + ).set('labels', {ok: gettext("Yes"), cancel:gettext("No")}); + } }, /* This function will fetch the list of changed models and make @@ -2203,7 +2274,9 @@ define([ data: JSON.stringify(req_data), success: function(res) { var grid = self.slickgrid, - data = grid.getData(); + dataView = grid.getData(), + data_length = dataView.getLength(), + data = []; if (res.data.status) { // Remove flag is_row_copied from copied rows _.each(data, function(row, idx) { @@ -2220,22 +2293,20 @@ define([ // Remove deleted rows from client as well if(is_deleted) { var rows = grid.getSelectedRows(); - /* In JavaScript sorting by default is lexical, - * To make sorting numerical we need to pass function - * After that we will Reverse the order of sorted array - * so that when we remove it does not affect array index - */ - if(data.length == rows.length) { + if(data_length == rows.length) { // This means all the rows are selected, clear all data data = []; + dataView.setItems(data, self.client_primary_key); } else { - rows = rows.sort(function(a,b){return a - b}).reverse(); - rows.forEach(function(idx) { - data.splice(idx, 1); - }); + dataView.beginUpdate(); + for (var i = 0; i < rows.length; i++) { + item = grid.getDataItem(rows[i]); + data.push(item); + dataView.deleteItem(item[self.client_primary_key]); + } + dataView.endUpdate(); } self.rows_to_delete.apply(self, [data]); - grid.setData(data, true); grid.setSelectedRows([]); } @@ -2248,6 +2319,7 @@ define([ self.rows_to_disable = _.clone(self.temp_new_rows); } + grid.setSelectedRows([]); // Reset data store self.data_store = { 'added': {}, @@ -2268,7 +2340,7 @@ define([ $('.sql-editor-message').text(res.data.result); var err_msg = S(gettext("%s.")).sprintf(res.data.result).value(); alertify.notify(err_msg, 'error', 20); - + grid.setSelectedRows([]); // To highlight the row at fault if(_.has(res.data, '_rowid') && (!_.isUndefined(res.data._rowid)|| !_.isNull(res.data._rowid))) { @@ -2324,14 +2396,21 @@ define([ // Find index of row at fault from grid data _find_rowindex: function(rowid) { - var self = this; - var grid = self.slickgrid, - data = grid.getData(), _rowid, count = 0, _idx = -1; + var self = this, + grid = self.slickgrid, + dataView = grid.getData(), + data = dataView.getItems(), + _rowid, + count = 0, + _idx = -1; + // If _rowid is object then it's update/delete operation if(_.isObject(rowid)) { _rowid = rowid; - } else if (_.isString(rowid)) { // Insert opration - _rowid = { '__temp_PK': rowid }; + } else if (_.isString(rowid)) { // Insert operation + var rowid = {}; + rowid[self.client_primary_key]= rowid; + _rowid = rowid; } else { // Something is wrong with unique id return _idx; @@ -2553,11 +2632,6 @@ define([ // This function will set the required flag for polling response data _init_polling_flags: function() { var self = this; - // Set a flag to get columns - self.FETCH_COLUMNS_FROM_SERVER = true; - // We will set columns data in this variable for future use once we fetch it - // from server - self.COLUMNS_DATA = {}; // To get a timeout for polling fallback timer in seconds in // regards to elapsed time @@ -2653,7 +2727,7 @@ define([ return; // Add column position and it's value to data - data[column_info.field] = _values[column_info.pos] || ''; + data[column_info.field] = _values[column_info.field] || ''; self.trigger( 'pgadmin-sqleditor:loading-icon:show', @@ -2723,7 +2797,7 @@ define([ return; // Add column position and it's value to data - data[column_info.field] = _values[column_info.pos] || ''; + data[column_info.field] = _values[column_info.field] || ''; self.trigger( 'pgadmin-sqleditor:loading-icon:show', @@ -2886,16 +2960,18 @@ define([ _paste_row: function() { var self = this, col_info = {}, grid = self.slickgrid, - data = grid.getData(), - count = Object.keys(data).length-1; - - var rows = grid.getSelectedRows().sort( + dataView = grid.getData(), + data = dataView.getItems(), + count = dataView.getLength(), + rows = grid.getSelectedRows().sort( function (a, b) { return a - b; } ), - rows = rows.length == 0 ? self.last_copied_rows : rows, copied_rows = rows.map(function (rowIndex) { return data[rowIndex]; }); + + rows = rows.length == 0 ? self.last_copied_rows : rows + self.last_copied_rows = rows; // If there are rows to paste? @@ -3015,6 +3091,9 @@ define([ sql = '', history_msg = ''; + self.has_more_rows = false; + self.fetching_rows = false; + /* If code is selected in the code mirror then execute * the selected part else execute the complete code. */ diff --git a/web/pgadmin/utils/driver/abstract.py b/web/pgadmin/utils/driver/abstract.py index 9b2363c..7db3e37 100644 --- a/web/pgadmin/utils/driver/abstract.py +++ b/web/pgadmin/utils/driver/abstract.py @@ -101,6 +101,12 @@ class BaseConnection(object): - Implement this method to execute the given query and returns the result as an array of dict (column name -> value) format. + * def async_fetchmany_2darray(records=-1, formatted_exception_msg=False): + - Implement this method to retrieve result of asynchronous connection and + polling with no_result flag set to True. + This returns the result as a 2 dimensional array. + If records is -1 then fetchmany will behave as fetchall. + * connected() - Implement this method to get the status of the connection. It should return True for connected, otherwise False @@ -133,7 +139,7 @@ class BaseConnection(object): - Implement this method to wait for asynchronous connection with timeout. This must be a non blocking call. - * poll(formatted_exception_msg) + * poll(formatted_exception_msg, no_result) - Implement this method to poll the data of query running on asynchronous connection. @@ -180,6 +186,10 @@ class BaseConnection(object): pass @abstractmethod + def async_fetchmany_2darray(self, records=-1, formatted_exception_msg=False): + pass + + @abstractmethod def connected(self): pass @@ -208,7 +218,7 @@ class BaseConnection(object): pass @abstractmethod - def poll(self, formatted_exception_msg=True): + def poll(self, formatted_exception_msg=True, no_result=False): pass @abstractmethod diff --git a/web/pgadmin/utils/driver/psycopg2/__init__.py b/web/pgadmin/utils/driver/psycopg2/__init__.py index 92562bc..d3dc5c5 100644 --- a/web/pgadmin/utils/driver/psycopg2/__init__.py +++ b/web/pgadmin/utils/driver/psycopg2/__init__.py @@ -1078,6 +1078,55 @@ Failed to execute query (execute_void) for the server #{server_id} - {conn_id} return True, {'columns': columns, 'rows': rows} + def async_fetchmany_2darray(self, records=2000, formatted_exception_msg=False): + """ + User should poll and check if status is ASYNC_OK before calling this + function + Args: + records: no of records to fetch. use -1 to fetchall. + formatted_exception_msg: + + Returns: + + """ + cur = self.__async_cursor + if not cur: + return False, gettext( + "Cursor could not be found for the async connection." + ) + + if self.conn.isexecuting(): + return False, gettext( + "Asynchronous query execution/operation underway." + ) + + if self.row_count > 0: + result = [] + # For DDL operation, we may not have result. + # + # Because - there is not direct way to differentiate DML and + # DDL operations, we need to rely on exception to figure + # that out at the moment. + try: + if records == -1: + res = cur.fetchall() + else: + res = cur.fetchmany(records) + for row in res: + new_row = [] + for col in self.column_info: + new_row.append(row[col['name']]) + result.append(new_row) + except psycopg2.ProgrammingError as e: + result = None + else: + # User performed operation which dose not produce record/s as + # result. + # for eg. DDL operations. + return True, None + + return True, result + def connected(self): if self.conn: if not self.conn.closed: @@ -1224,7 +1273,7 @@ Failed to reset the connection to the server due to following error: "poll() returned %s from _wait_timeout function" % state ) - def poll(self, formatted_exception_msg=False): + def poll(self, formatted_exception_msg=False, no_result=False): """ This function is a wrapper around connection's poll function. It internally uses the _wait_timeout method to poll the @@ -1234,6 +1283,7 @@ Failed to reset the connection to the server due to following error: Args: formatted_exception_msg: if True then function return the formatted exception message, otherwise error string. + no_result: If True then only poll status will be returned. """ cur = self.__async_cursor @@ -1289,23 +1339,23 @@ Failed to reset the connection to the server due to following error: pos += 1 self.row_count = cur.rowcount - - if cur.rowcount > 0: - result = [] - # For DDL operation, we may not have result. - # - # Because - there is not direct way to differentiate DML and - # DDL operations, we need to rely on exception to figure that - # out at the moment. - try: - for row in cur: - new_row = [] - for col in self.column_info: - new_row.append(row[col['name']]) - result.append(new_row) - - except psycopg2.ProgrammingError: - result = None + if not no_result: + if cur.rowcount > 0: + result = [] + # For DDL operation, we may not have result. + # + # Because - there is not direct way to differentiate DML and + # DDL operations, we need to rely on exception to figure + # that out at the moment. + try: + for row in cur: + new_row = [] + for col in self.column_info: + new_row.append(row[col['name']]) + result.append(new_row) + + except psycopg2.ProgrammingError: + result = None return status, result diff --git a/web/regression/feature_utils/pgadmin_page.py b/web/regression/feature_utils/pgadmin_page.py index bb111a1..1f12a80 100644 --- a/web/regression/feature_utils/pgadmin_page.py +++ b/web/regression/feature_utils/pgadmin_page.py @@ -224,6 +224,20 @@ class PgadminPage: self._wait_for("spinner to disappear", spinner_has_disappeared) + def wait_for_query_tool_loading_indicator_to_disappear(self): + def spinner_has_disappeared(driver): + try: + driver.find_element_by_xpath( + "//*[@id='fetching_data' and @class='hide']" + ) + return False + except NoSuchElementException: + # wait for loading indicator disappear animation to complete. + time.sleep(0.5) + return True + + self._wait_for("spinner to disappear", spinner_has_disappeared) + def wait_for_app(self): def page_shows_app(driver): if driver.title == self.app_config.APP_NAME: diff --git a/web/regression/javascript/selection/copy_data_spec.js b/web/regression/javascript/selection/copy_data_spec.js index 9e05e84..2b623b3 100644 --- a/web/regression/javascript/selection/copy_data_spec.js +++ b/web/regression/javascript/selection/copy_data_spec.js @@ -16,16 +16,16 @@ import clipboard from '../../../pgadmin/static/js/selection/clipboard'; import copyData from '../../../pgadmin/static/js/selection/copy_data'; import RangeSelectionHelper from 'sources/selection/range_selection_helper'; import XCellSelectionModel from 'sources/selection/xcell_selection_model'; - describe('copyData', function () { var grid, sqlEditor, gridContainer, buttonPasteRow; var SlickGrid; beforeEach(function () { SlickGrid = Slick.Grid; - var data = [[1, 'leopord', '12'], - [2, 'lion', '13'], - [3, 'puma', '9']]; + var data = [{'id': 1, 'brand':'leopord', 'size':'12', '__temp_PK': '123'}, + {'id': 2, 'brand':'lion', 'size':'13', '__temp_PK': '456'}, + {'id': 3, 'brand':'puma', 'size':'9', '__temp_PK': '789'}], + dataView = new Slick.Data.DataView(); var columns = [ { @@ -37,6 +37,7 @@ describe('copyData', function () { }, { name: 'id', + field: 'id', pos: 0, label: 'id<br> numeric', cell: 'number', @@ -44,6 +45,7 @@ describe('copyData', function () { type: 'numeric', }, { name: 'brand', + field: 'brand', pos: 1, label: 'flavor<br> character varying', cell: 'string', @@ -51,24 +53,26 @@ describe('copyData', function () { type: 'character varying', }, { name: 'size', + field: 'size', pos: 2, label: 'size<br> numeric', cell: 'number', can_edit: false, type: 'numeric', }, - ] - ; - gridContainer = $('<div id=\'grid\'></div>'); + ]; + gridContainer = $('<div id="grid"></div>'); $('body').append(gridContainer); - buttonPasteRow = $('<button id=\'btn-paste-row\' disabled></button>'); + buttonPasteRow = $('<button id="btn-paste-row" disabled></button>'); $('body').append(buttonPasteRow); - grid = new SlickGrid('#grid', data, columns, {}); + grid = new SlickGrid('#grid', dataView, columns, {}); + dataView.setItems(data, '__temp_PK'); grid.setSelectionModel(new XCellSelectionModel()); sqlEditor = {slickgrid: grid}; }); - afterEach(function () { + afterEach(function() { + grid.destroy(); gridContainer.remove(); buttonPasteRow.remove(); }); diff --git a/web/regression/javascript/selection/range_boundary_navigator_spec.js b/web/regression/javascript/selection/range_boundary_navigator_spec.js index 1de6d81..2d56b05 100644 --- a/web/regression/javascript/selection/range_boundary_navigator_spec.js +++ b/web/regression/javascript/selection/range_boundary_navigator_spec.js @@ -134,17 +134,19 @@ describe('RangeBoundaryNavigator', function () { describe('#rangesToCsv', function () { var data, columnDefinitions, ranges; beforeEach(function () { - data = [[1, 'leopard', '12'], - [2, 'lion', '13'], - [3, 'cougar', '9'], - [4, 'tiger', '10']]; - columnDefinitions = [{name: 'id', pos: 0}, {name: 'animal', pos: 1}, {name: 'size', pos: 2}]; + data = [{'id':1, 'animal':'leopard', 'size':'12'}, + {'id':2, 'animal':'lion', 'size':'13'}, + {'id':3, 'animal':'cougar', 'size':'9'}, + {'id':4, 'animal':'tiger', 'size':'10'}]; + + columnDefinitions = [{name: 'id', field: 'id', pos: 0}, + {name: 'animal', field: 'animal', pos: 1}, + {name: 'size', field: 'size', pos: 2}]; ranges = [new Slick.Range(0, 0, 0, 2), new Slick.Range(3, 0, 3, 2)]; }); it('returns csv for the provided ranges', function () { var csvResult = rangeBoundaryNavigator.rangesToCsv(data, columnDefinitions, ranges); - expect(csvResult).toEqual('1,\'leopard\',\'12\'\n4,\'tiger\',\'10\''); }); @@ -158,10 +160,10 @@ describe('RangeBoundaryNavigator', function () { describe('when there is an extra column with checkboxes', function () { beforeEach(function () { - columnDefinitions = [{name: 'not-a-data-column'}, {name: 'id', pos: 0}, {name: 'animal', pos: 1}, { - name: 'size', - pos: 2, - }]; + columnDefinitions = [{name: 'not-a-data-column'}, + {name: 'id', field: 'id', pos: 0}, + {name: 'animal', field: 'animal', pos: 1}, + {name: 'size', field: 'size',pos: 2}]; ranges = [new Slick.Range(0, 0, 0, 3), new Slick.Range(3, 0, 3, 3)]; }); diff --git a/web/regression/javascript/selection/row_selector_spec.js b/web/regression/javascript/selection/row_selector_spec.js index db3c0b4..d4152f8 100644 --- a/web/regression/javascript/selection/row_selector_spec.js +++ b/web/regression/javascript/selection/row_selector_spec.js @@ -6,11 +6,11 @@ // This software is released under the PostgreSQL Licence // ////////////////////////////////////////////////////////////////////////// - import $ from 'jquery'; import Slick from 'slickgrid'; import 'slickgrid.grid'; +import 'slick.dataview'; import RowSelector from 'sources/selection/row_selector'; import ActiveCellCapture from 'sources/selection/active_cell_capture'; @@ -23,7 +23,7 @@ describe('RowSelector', function () { UP: 38, DOWN: 40, }; - var container, data, columnDefinitions, grid, cellSelectionModel; + var container, dataView, columnDefinitions, grid, cellSelectionModel; var SlickGrid = Slick.Grid; beforeEach(function () { @@ -43,14 +43,15 @@ describe('RowSelector', function () { pos: 1, }]; + dataView = new Slick.Data.DataView(); var rowSelector = new RowSelector(); - data = []; + var data = []; for (var i = 0; i < 10; i++) { - data.push(['some-value-' + i, 'second value ' + i]); + data.push({'some-column-name':'some-value-' + i, 'second column':'second value ' + i}); } columnDefinitions = rowSelector.getColumnDefinitions(columnDefinitions); - grid = new SlickGrid(container, data, columnDefinitions); - + dataView.setItems(data, 'some-column-name'); + grid = new SlickGrid(container, dataView, columnDefinitions); grid.registerPlugin(new ActiveCellCapture()); cellSelectionModel = new XCellSelectionModel(); grid.setSelectionModel(cellSelectionModel); diff --git a/web/regression/javascript/selection/set_staged_rows_spec.js b/web/regression/javascript/selection/set_staged_rows_spec.js index 6735314..cf201c4 100644 --- a/web/regression/javascript/selection/set_staged_rows_spec.js +++ b/web/regression/javascript/selection/set_staged_rows_spec.js @@ -7,236 +7,248 @@ // ////////////////////////////////////////////////////////////// -define([ - 'jquery', - 'underscore', - 'sources/selection/set_staged_rows', -], function ($, _, SetStagedRows) { - describe('set_staged_rows', function () { - var sqlEditorObj, gridSpy, deleteButton, copyButton, selectionSpy; - beforeEach(function () { - gridSpy = jasmine.createSpyObj('gridSpy', ['getData', 'getCellNode', 'getColumns']); - gridSpy.getData.and.returnValue([ - {0: 'one', 1: 'two', __temp_PK: '123'}, - {0: 'three', 1: 'four', __temp_PK: '456'}, - {0: 'five', 1: 'six', __temp_PK: '789'}, - {0: 'seven', 1: 'eight', __temp_PK: '432'}, - ]); - gridSpy.getColumns.and.returnValue([ +import $ from 'jquery'; +import 'slickgrid.grid'; +import Slick from 'slickgrid'; +import 'slick.dataview'; +import SetStagedRows from 'sources/selection/set_staged_rows'; + +describe('set_staged_rows', function () { + var sqlEditorObj, gridSpy, deleteButton, copyButton, selectionSpy; + beforeEach(function () { + var data = [{'a pk column': 'one', 'some column': 'two', '__temp_PK': '123'}, + {'a pk column': 'three', 'some column': 'four', '__temp_PK': '456'}, + {'a pk column': 'five', 'some column': 'six', '__temp_PK': '789'}, + {'a pk column': 'seven', 'some column': 'eight', '__temp_PK': '432'}], + dataView = new Slick.Data.DataView(); + + dataView.setItems(data, '__temp_PK'); + + gridSpy = jasmine.createSpyObj('gridSpy', ['getData', 'getCellNode', 'getColumns']); + gridSpy.getData.and.returnValue(dataView); + gridSpy.getColumns.and.returnValue([ + { + name: 'a pk column', + field: 'a pk column', + pos: 0, + selectable: true, + }, { + name: 'some column', + field: 'some column', + pos: 1, + selectable: true, + }, + ]); + selectionSpy = jasmine.createSpyObj('selectionSpy', ['setSelectedRows', 'getSelectedRanges']); + deleteButton = $('<button id="btn-delete-row"></button>'); + copyButton = $('<button id="btn-copy-row"></button>'); + + sqlEditorObj = { + grid: gridSpy, + editor: { + handler: { + data_store: { + staged_rows: {'456': {}}, + }, + can_edit: false, + }, + }, + keys: null, + selection: selectionSpy, + columns: [ { + name: 'a pk column', + field: 'a pk column', pos: 0, - selectable: true, - }, { + }, + { + name: 'some column', + field: 'some column', pos: 1, - selectable: true, }, - ]); + ], + client_primary_key: '__temp_PK', + }; - selectionSpy = jasmine.createSpyObj('selectionSpy', ['setSelectedRows', 'getSelectedRanges']); + $('body').append(deleteButton); + $('body').append(copyButton); - deleteButton = $('<button id="btn-delete-row"></button>'); - copyButton = $('<button id="btn-copy-row"></button>'); + deleteButton.prop('disabled', true); + copyButton.prop('disabled', true); - sqlEditorObj = { - grid: gridSpy, - editor: { - handler: { - data_store: { - staged_rows: {'456': {}}, - }, - can_edit: false, - }, - }, - keys: null, - selection: selectionSpy, - columns: [ - { - name: 'a pk column', - pos: 0, - }, - { - name: 'some column', - pos: 1, - }, - ], - }; + selectionSpy = jasmine.createSpyObj('selectionSpy', [ + 'setSelectedRows', + 'getSelectedRanges', + ]); + }); - $('body').append(deleteButton); - $('body').append(copyButton); + afterEach(function () { + copyButton.remove(); + deleteButton.remove(); + }); + describe('when no full rows are selected', function () { + describe('when nothing is selected', function () { + beforeEach(function () { + selectionSpy.getSelectedRanges.and.returnValue([]); + sqlEditorObj.selection = selectionSpy; + SetStagedRows.call(sqlEditorObj, {}, {}); + }); - deleteButton.prop('disabled', true); - copyButton.prop('disabled', true); + it('should disable the delete row button', function () { + expect($('#btn-delete-row').prop('disabled')).toBeTruthy(); + }); - selectionSpy = jasmine.createSpyObj('selectionSpy', [ - 'setSelectedRows', - 'getSelectedRanges', - ]); - }); + it('should disable the copy row button', function () { + expect($('#btn-copy-row').prop('disabled')).toBeTruthy(); + }); - afterEach(function () { - copyButton.remove(); - deleteButton.remove(); + it('should clear staged rows', function () { + expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({}); + }); }); - describe('when no full rows are selected', function () { - describe('when nothing is selected', function () { - beforeEach(function () { - selectionSpy.getSelectedRanges.and.returnValue([]); - sqlEditorObj.selection = selectionSpy; - SetStagedRows.call(sqlEditorObj, {}, {}); - }); - it('should disable the delete row button', function () { - expect($('#btn-delete-row').prop('disabled')).toBeTruthy(); - }); + describe('when there is a selection', function () { + beforeEach(function () { + var range = { + fromCell: 0, + toCell: 0, + fromRow: 1, + toRow: 1, + }; - it('should disable the copy row button', function () { - expect($('#btn-copy-row').prop('disabled')).toBeTruthy(); - }); + selectionSpy.getSelectedRanges.and.returnValue([range]); + sqlEditorObj.selection = selectionSpy; + SetStagedRows.call(sqlEditorObj, {}, {}); + }); - it('should clear staged rows', function () { - expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({}); - }); + it('should disable the delete row button', function () { + expect($('#btn-delete-row').prop('disabled')).toBeTruthy(); }); - describe('when there is a selection', function () { - beforeEach(function () { - var range = { - fromCell: 0, - toCell: 0, - fromRow: 1, - toRow: 1, - }; - - selectionSpy.getSelectedRanges.and.returnValue([range]); - sqlEditorObj.selection = selectionSpy; - SetStagedRows.call(sqlEditorObj, {}, {}); - }); + it('should disable the copy row button', function () { + expect($('#btn-copy-row').prop('disabled')).toBeFalsy(); + }); - it('should disable the delete row button', function () { - expect($('#btn-delete-row').prop('disabled')).toBeTruthy(); - }); + it('should clear staged rows', function () { + expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({}); + }); + }); + }); - it('should disable the copy row button', function () { - expect($('#btn-copy-row').prop('disabled')).toBeFalsy(); - }); + describe('when 2 full rows are selected', function () { + beforeEach(function () { + var range1 = { + fromCell: 0, + toCell: 1, + fromRow: 1, + toRow: 1, + }; + var range2 = { + fromCell: 0, + toCell: 1, + fromRow: 2, + toRow: 2, + }; - it('should clear staged rows', function () { - expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({}); - }); - }); + selectionSpy.getSelectedRanges.and.returnValue([range1, range2]); + sqlEditorObj.selection = selectionSpy; }); - describe('when 2 full rows are selected', function () { - beforeEach(function () { - var range1 = { - fromCell: 0, - toCell: 1, - fromRow: 1, - toRow: 1, - }; - var range2 = { - fromCell: 0, - toCell: 1, - fromRow: 2, - toRow: 2, - }; + describe('when table does not have primary keys', function () { + it('should enable the copy row button', function () { + SetStagedRows.call(sqlEditorObj, {}, {}); + expect($('#btn-copy-row').prop('disabled')).toBeFalsy(); + }); - selectionSpy.getSelectedRanges.and.returnValue([range1, range2]); - sqlEditorObj.selection = selectionSpy; + it('should not enable the delete row button', function () { + SetStagedRows.call(sqlEditorObj, {}, {}); + expect($('#btn-delete-row').prop('disabled')).toBeTruthy(); }); - describe('when table does not have primary keys', function () { - it('should enable the copy row button', function () { - SetStagedRows.call(sqlEditorObj, {}, {}); - expect($('#btn-copy-row').prop('disabled')).toBeFalsy(); - }); + it('should update staged rows with the __temp_PK value of the new Selected Rows', function () { + SetStagedRows.call(sqlEditorObj, {}, {}); + expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({'456': {}, '789': {}}); + }); - it('should not enable the delete row button', function () { + describe('the user can edit', function () { + it('should enable the delete row button', function () { + sqlEditorObj.editor.handler.can_edit = true; SetStagedRows.call(sqlEditorObj, {}, {}); - expect($('#btn-delete-row').prop('disabled')).toBeTruthy(); + expect($('#btn-delete-row').prop('disabled')).toBeFalsy(); }); + }); + }); - it('should update staged rows with the __temp_PK value of the new Selected Rows', function () { + describe('when table has primary keys', function () { + beforeEach(function () { + sqlEditorObj.keys = {'a pk column': 'varchar'}; + sqlEditorObj.editor.handler.data_store.staged_rows = {'456': {'a pk column': 'three'}}; + }); + + describe('selected rows have primary key', function () { + it('should set the staged rows correctly', function () { SetStagedRows.call(sqlEditorObj, {}, {}); - expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({'456': {}, '789': {}}); + expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual( + {'456': {'a pk column': 'three'}, '789': {'a pk column': 'five'}}); }); - describe('the user can edit', function () { - it('should enable the delete row button', function () { - sqlEditorObj.editor.handler.can_edit = true; - SetStagedRows.call(sqlEditorObj, {}, {}); - expect($('#btn-delete-row').prop('disabled')).toBeFalsy(); - }); + it('should not clear selected rows in Cell Selection Model', function () { + SetStagedRows.call(sqlEditorObj, {}, {}); + expect(sqlEditorObj.selection.setSelectedRows).not.toHaveBeenCalledWith(); }); }); - describe('when table has primary keys', function () { + describe('selected rows missing primary key', function () { beforeEach(function () { - sqlEditorObj.keys = {'a pk column': 'varchar'}; - sqlEditorObj.editor.handler.data_store.staged_rows = {'456': {0: 'three'}}; - }); + var data = [{'a pk column': 'one', 'some column': 'two', '__temp_PK': '123'}, + {'some column': 'four', '__temp_PK': '456'}, + {'some column': 'six', '__temp_PK': '789'}, + {'a pk column': 'seven', 'some column': 'eight', '__temp_PK': '432'}], + dataView = new Slick.Data.DataView(); - describe('selected rows have primary key', function () { - it('should set the staged rows correctly', function () { - SetStagedRows.call(sqlEditorObj, {}, {}); - expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual( - {'456': {0: 'three'}, '789': {0: 'five'}}); - }); + dataView.setItems(data, '__temp_PK'); - it('should not clear selected rows in Cell Selection Model', function () { - SetStagedRows.call(sqlEditorObj, {}, {}); - expect(sqlEditorObj.selection.setSelectedRows).not.toHaveBeenCalledWith(); - }); + gridSpy.getData.and.returnValue(dataView); }); - describe('selected rows missing primary key', function () { - beforeEach(function () { - gridSpy.getData.and.returnValue([ - {0: 'one', 1: 'two', __temp_PK: '123'}, - {1: 'four', __temp_PK: '456'}, - {1: 'six', __temp_PK: '789'}, - {0: 'seven', 1: 'eight', __temp_PK: '432'}, - ]); - }); - - it('should clear the staged rows', function () { - SetStagedRows.call(sqlEditorObj, {}, {}); - expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({}); - }); + it('should clear the staged rows', function () { + SetStagedRows.call(sqlEditorObj, {}, {}); + expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({}); + }); - it('should clear selected rows in Cell Selection Model', function () { - SetStagedRows.call(sqlEditorObj, {}, {}); - expect(sqlEditorObj.selection.setSelectedRows).toHaveBeenCalledWith([]); - }); + it('should clear selected rows in Cell Selection Model', function () { + SetStagedRows.call(sqlEditorObj, {}, {}); + expect(sqlEditorObj.selection.setSelectedRows).toHaveBeenCalledWith([]); }); + }); - describe('when the selected row is a new row', function () { - var parentDiv; - beforeEach(function () { - var childDiv = $('<div></div>'); - parentDiv = $('<div class="new_row"></div>'); - parentDiv.append(childDiv); - $('body').append(parentDiv); - gridSpy.getCellNode.and.returnValue(childDiv); - SetStagedRows.call(sqlEditorObj, {}, {}); - }); + describe('when the selected row is a new row', function () { + var parentDiv; + beforeEach(function () { + var childDiv = $('<div></div>'); + parentDiv = $('<div class="new_row"></div>'); + parentDiv.append(childDiv); + $('body').append(parentDiv); + gridSpy.getCellNode.and.returnValue(childDiv); + SetStagedRows.call(sqlEditorObj, {}, {}); + }); - afterEach(function () { - parentDiv.remove(); - }); + afterEach(function () { + parentDiv.remove(); + }); - it('should not clear the staged rows', function () { - expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({ - '456': {0: 'three'}, - '789': {0: 'five'}, - }); + it('should not clear the staged rows', function () { + expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({ + '456': {'a pk column': 'three'}, + '789': {'a pk column': 'five'}, }); + }); - it('should not clear selected rows in Cell Selection Model', function () { - expect(sqlEditorObj.selection.setSelectedRows).not.toHaveBeenCalled(); - }); + it('should not clear selected rows in Cell Selection Model', function () { + expect(sqlEditorObj.selection.setSelectedRows).not.toHaveBeenCalled(); }); }); }); }); }); + diff --git a/web/regression/javascript/selection/xcell_selection_model_spec.js b/web/regression/javascript/selection/xcell_selection_model_spec.js index db3f6d6..2882e49 100644 --- a/web/regression/javascript/selection/xcell_selection_model_spec.js +++ b/web/regression/javascript/selection/xcell_selection_model_spec.js @@ -32,14 +32,17 @@ describe('XCellSelectionModel', function () { }, { id: '1', name: 'some-column-name', + field: 'some-column-name', pos: 0, }, { id: 'second-column-id', name: 'second column', + field: 'second column', pos: 1, }, { id: 'third-column-id', name: 'third column', + field: 'third column', pos: 2, }, ]; @@ -51,13 +54,15 @@ describe('XCellSelectionModel', function () { 'second column': 'second value ' + i, 'third column': 'third value ' + i, 'fourth column': 'fourth value ' + i, + '__temp_PK': '123' + i, }); } container = $('<div></div>'); + var dataView = new Slick.Data.DataView(); container.height(9999); container.width(9999); - - grid = new SlickGrid(container, data, columns); + dataView.setItems(data, '__temp_PK'); + grid = new SlickGrid(container, dataView, columns); grid.setSelectionModel(new XCellSelectionModel()); $('body').append(container); }); diff --git a/web/regression/javascript/slickgrid/event_handlers/handle_query_output_keyboard_event_spec.js b/web/regression/javascript/slickgrid/event_handlers/handle_query_output_keyboard_event_spec.js index eb4411d..e790a8f 100644 --- a/web/regression/javascript/slickgrid/event_handlers/handle_query_output_keyboard_event_spec.js +++ b/web/regression/javascript/slickgrid/event_handlers/handle_query_output_keyboard_event_spec.js @@ -27,21 +27,22 @@ describe('#handleQueryOutputKeyboardEvent', function () { metaKey: false, which: -1, keyCode: -1, - preventDefault: jasmine.createSpy('preventDefault'), + preventDefault: jasmine.createSpy('preventDefault') }; - var data = [['', '0,0-cell-content', '0,1-cell-content'], - ['', '1,0-cell-content', '1,1-cell-content'], - ['', '2,0-cell-content', '2,1-cell-content']]; - var columnDefinitions = [{name: 'checkboxColumn'}, {pos: 1, name: 'firstColumn'}, { - pos: 2, - name: 'secondColumn', - }]; - grid = new SlickGrid($('<div></div>'), data, columnDefinitions); - grid.setSelectionModel(new XCellSelectionModel()); + var data = [{'checkboxColumn': '', 'firstColumn': '0,0-cell-content', 'secondColumn': '0,1-cell-content', '__temp_PK': '123'}, + {'checkboxColumn': '', 'firstColumn': '1,0-cell-content', 'secondColumn': '1,1-cell-content', '__temp_PK': '456'}, + {'checkboxColumn': '', 'firstColumn': '2,0-cell-content', 'secondColumn': '2,1-cell-content', '__temp_PK': '789'}], + columnDefinitions = [{name: 'checkboxColumn'}, + {pos: 1, name: 'firstColumn', field: 'firstColumn'}, + { pos: 2, name: 'secondColumn', field: 'secondColumn'}], + dataView = new Slick.Data.DataView(); + grid = new Slick.Grid($('<div></div>'), dataView, columnDefinitions); + grid.setSelectionModel(new XCellSelectionModel()); + dataView.setItems(data, '__temp_PK'); slickEvent = { - grid: grid, + grid: grid }; spyOn(clipboard, 'copyTextToClipboard'); diff --git a/web/webpack.test.config.js b/web/webpack.test.config.js index 6754f04..8d3e9e9 100644 --- a/web/webpack.test.config.js +++ b/web/webpack.test.config.js @@ -63,6 +63,7 @@ module.exports = { 'pgadmin.browser.endpoints': regressionDir + '/javascript/fake_endpoints', 'slickgrid': sourcesDir + '/vendor/slickgrid/slick.core', 'slickgrid.grid': sourcesDir + '/vendor/slickgrid/slick.grid', + 'slick.dataview': sourcesDir + '/vendor/slickgrid/slick.dataview', 'slickgrid.rowselectionmodel': sourcesDir + '/vendor/slickgrid/plugins/slick.rowselectionmodel', 'browser': path.resolve(__dirname, 'pgadmin/browser/static/js'), 'pgadmin': sourcesDir + '/js/pgadmin', ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] @ 2017-06-15 10:28 Dave Page <[email protected]> parent: Harshal Dhumal <[email protected]> 0 siblings, 1 reply; 24+ messages in thread From: Dave Page @ 2017-06-15 10:28 UTC (permalink / raw) To: Harshal Dhumal <[email protected]>; +Cc: Harshal Dhumal <[email protected]>; Ashesh Vashi <[email protected]>; pgadmin-hackers Hi On Wed, Jun 14, 2017 at 11:36 PM, Harshal Dhumal < [email protected]> wrote: > Hi Dave, > > Please find rebased patch for RM2137. > Looking very good. The only issues I see are: - The row headers should auto-size such that they can display the row numbers if the last row was displayed. E.g. if there are 12345 rows in total, then the row header should be sized to display 5 digits. - The tests are comprehensive, which is awesome. However, every time I ran them, at least one of the feature tests failed. Unfortunately, it was a different one each time. In the last two runs, I got: ====================================================================== ERROR: runTest (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) Query tool feature test ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/base_feature_test.py", line 40, in setUp self.before() File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", line 40, in before self._connects_to_server() File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", line 144, in _connects_to_server self.page.driver.find_element_by_link_text("Create"))\ File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 319, in find_element_by_link_text return self.find_element(by=By.LINK_TEXT, value=link_text) File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 756, in find_element 'value': value})['value'] File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 238, in execute self.error_handler.check_response(response) File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 193, in check_response raise exception_class(message, screen, stacktrace) NoSuchElementException: Message: no such element: Unable to locate element: {"method":"link text","selector":"Create"} (Session info: chrome=58.0.3029.110) (Driver info: chromedriver=2.29.461585 (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 x86_64) and ====================================================================== ERROR: runTest (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) Query tool feature test ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", line 119, in runTest self._query_tool_auto_rollback_enabled() File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", line 697, in _query_tool_auto_rollback_enabled '//div[contains(@class, "sql-editor-message") and contains(string(), "COMMIT")]' File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 295, in find_element_by_xpath return self.find_element(by=By.XPATH, value=xpath) File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 756, in find_element 'value': value})['value'] File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 238, in execute self.error_handler.check_response(response) File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 193, in check_response raise exception_class(message, screen, stacktrace) NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//div[contains(@class, "sql-editor-message") and contains(string(), "COMMIT")]"} (Session info: chrome=58.0.3029.110) (Driver info: chromedriver=2.29.461585 (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 x86_64) Relevant screenshots attached. - Can you tidy up the regression output a little please? Instead of: ------- runTest (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) Query tool feature test ... On demand result set on scrolling... OK. On demand result set on grid select all... OK. On demand result set on column select all... OK. Explain query... OK. Explain query with verbose... OK. Explain query with costs... OK. Explain analyze query... OK. Explain analyze query with buffers... OK. Explain analyze query with timing... OK. Auto commit disabled... OK. Auto commit enabled... OK. Auto rollback enabled... ERROR ------- Something like: ------- runTest (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) Query tool feature test ... On demand result set on scrolling... OK. On demand result set on grid select all... OK. On demand result set on column select all... OK. Explain query... OK. Explain query with verbose... OK. Explain query with costs... OK. Explain analyze query... OK. Explain analyze query with buffers... OK. Explain analyze query with timing... OK. Auto commit disabled... OK. Auto commit enabled... OK. Auto rollback enabled... ERROR -------- Thanks! -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgadmin-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgadmin-hackers Attachments: [image/png] QueryToolFeatureTest-2017.06.15_11.15.56-Python-2.7.10.png (139.5K, 3-QueryToolFeatureTest-2017.06.15_11.15.56-Python-2.7.10.png) download | view image [image/png] QueryToolFeatureTest-2017.06.15_11.10.19-Python-2.7.10.png (153.5K, 4-QueryToolFeatureTest-2017.06.15_11.10.19-Python-2.7.10.png) download | view image ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] @ 2017-06-15 20:30 Harshal Dhumal <[email protected]> parent: Dave Page <[email protected]> 0 siblings, 1 reply; 24+ messages in thread From: Harshal Dhumal @ 2017-06-15 20:30 UTC (permalink / raw) To: Dave Page <[email protected]>; +Cc: Harshal Dhumal <[email protected]>; Ashesh Vashi <[email protected]>; pgadmin-hackers Hi Dave, Please find attached updated patch. On Thu, Jun 15, 2017 at 3:58 PM, Dave Page <[email protected]> wrote: > Hi > > On Wed, Jun 14, 2017 at 11:36 PM, Harshal Dhumal < > [email protected]> wrote: > >> Hi Dave, >> >> Please find rebased patch for RM2137. >> > > Looking very good. The only issues I see are: > > - The row headers should auto-size such that they can display the row > numbers if the last row was displayed. E.g. if there are 12345 rows in > total, then the row header should be sized to display 5 digits. > > Fixed. > - The tests are comprehensive, which is awesome. However, every time I ran > them, at least one of the feature tests failed. Unfortunately, it was a > different one each time. In the last two runs, I got: > > ====================================================================== > ERROR: runTest (pgadmin.feature_tests.query_tool_tests. > QueryToolFeatureTest) > Query tool feature test > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/base_feature_test.py", > line 40, in setUp > self.before() > File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", > line 40, in before > self._connects_to_server() > File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", > line 144, in _connects_to_server > self.page.driver.find_element_by_link_text("Create"))\ > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", line 319, in > find_element_by_link_text > return self.find_element(by=By.LINK_TEXT, value=link_text) > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", line 756, in > find_element > 'value': value})['value'] > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", line 238, in execute > self.error_handler.check_response(response) > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/errorhandler.py", line 193, in > check_response > raise exception_class(message, screen, stacktrace) > NoSuchElementException: Message: no such element: Unable to locate > element: {"method":"link text","selector":"Create"} > (Session info: chrome=58.0.3029.110) > (Driver info: chromedriver=2.29.461585 (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac > OS X 10.12.3 x86_64) > > I checked code from *_connects_to_server* function which is common in all features test cases. I didn't find anything wrong with this. If there is a bug in this function then all feature test must fail. Let me know if you are getting failure consistently in *_connects_to_server* function. > and > > > ====================================================================== > ERROR: runTest (pgadmin.feature_tests.query_tool_tests. > QueryToolFeatureTest) > Query tool feature test > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", > line 119, in runTest > self._query_tool_auto_rollback_enabled() > File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", > line 697, in _query_tool_auto_rollback_enabled > '//div[contains(@class, "sql-editor-message") and contains(string(), > "COMMIT")]' > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", line 295, in > find_element_by_xpath > return self.find_element(by=By.XPATH, value=xpath) > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", line 756, in > find_element > 'value': value})['value'] > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", line 238, in execute > self.error_handler.check_response(response) > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/errorhandler.py", line 193, in > check_response > raise exception_class(message, screen, stacktrace) > NoSuchElementException: Message: no such element: Unable to locate > element: {"method":"xpath","selector":"//div[contains(@class, > "sql-editor-message") and contains(string(), "COMMIT")]"} > (Session info: chrome=58.0.3029.110) > (Driver info: chromedriver=2.29.461585 (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac > OS X 10.12.3 x86_64) > > I have updated Auto rollback enabled test in this patch. > Relevant screenshots attached. > > - Can you tidy up the regression output a little please? Instead of: > > ------- > runTest (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) > Query tool feature test ... On demand result set on scrolling... > OK. > > On demand result set on grid select all... > OK. > > On demand result set on column select all... > OK. > > Explain query... > OK. > > Explain query with verbose... > OK. > > Explain query with costs... > OK. > > Explain analyze query... > OK. > > Explain analyze query with buffers... > OK. > > Explain analyze query with timing... > OK. > > Auto commit disabled... > OK. > > Auto commit enabled... > OK. > > Auto rollback enabled... > ERROR > ------- > > Something like: > > ------- > runTest (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) > Query tool feature test ... > On demand result set on scrolling... OK. > On demand result set on grid select all... OK. > On demand result set on column select all... OK. > Explain query... OK. > Explain query with verbose... OK. > Explain query with costs... OK. > Explain analyze query... OK. > Explain analyze query with buffers... OK. > Explain analyze query with timing... OK. > Auto commit disabled... OK. > Auto commit enabled... OK. > Auto rollback enabled... ERROR > -------- > > Fixed. > Thanks! > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > -- Sent via pgadmin-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgadmin-hackers Attachments: [text/x-patch] RM2137_query_tool_on_demand_result_V7.patch (155.9K, 3-RM2137_query_tool_on_demand_result_V7.patch) download | inline diff: diff --git a/web/config.py b/web/config.py index d45c43c..c177cab 100644 --- a/web/config.py +++ b/web/config.py @@ -324,6 +324,12 @@ THREADED_MODE = True SQLALCHEMY_TRACK_MODIFICATIONS = False ########################################################################## +# Number of records to fetch in one batch in query tool when query result +# set is large. +########################################################################## +ON_DEMAND_RECORD_COUNT = 1000 + +########################################################################## # Local config settings ########################################################################## diff --git a/web/pgadmin/feature_tests/pg_datatype_validation_test.py b/web/pgadmin/feature_tests/pg_datatype_validation_test.py index b883aac..b72f727 100644 --- a/web/pgadmin/feature_tests/pg_datatype_validation_test.py +++ b/web/pgadmin/feature_tests/pg_datatype_validation_test.py @@ -106,39 +106,19 @@ class PGDataypeFeatureTest(BaseFeatureTest): self.page.fill_codemirror_area_with(query) self.page.find_by_id("btn-flash").click() wait = WebDriverWait(self.page.driver, 5) - wait.until(EC.presence_of_element_located( - (By.XPATH, "//*[@id='0']//*[@id='datagrid']/div[5]/div/div[1]/" - "div[2]/span"))) + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) # For every sample data-type value, check the expected output. cnt = 2 - for val in expected_output[:10]: + cells = canvas.find_elements_by_tag_name('span') + # remove first element as it is row number. + cells.pop(0) + for val, cell in zip(expected_output, cells): try: - source_code = self.page.find_by_xpath( - "//*[@id='0']//*[@id='datagrid']/div[5]/div/div[1]/div[" - + str(cnt) - + "]/span" - ).get_attribute('innerHTML') - - PGDataypeFeatureTest.check_result( - source_code, - expected_output[cnt - 2] - ) - cnt += 1 - except TimeoutException: - assert False, "{0} does not match with {1}".format( - val, expected_output[cnt] - ) - - cnt = 12 - for val in expected_output[10:]: - try: - source_code = self.page.find_by_xpath( - "//*[@id='0']//*[@id='datagrid']/div[5]/div/div/div[" - + str(cnt) - + "]" - ).get_attribute('innerHTML') - + source_code = cell.get_attribute('innerHTML') PGDataypeFeatureTest.check_result( source_code, expected_output[cnt - 2] diff --git a/web/pgadmin/feature_tests/query_tool_tests.py b/web/pgadmin/feature_tests/query_tool_tests.py new file mode 100644 index 0000000..1861599 --- /dev/null +++ b/web/pgadmin/feature_tests/query_tool_tests.py @@ -0,0 +1,761 @@ +########################################################################## +# +# pgAdmin 4 - PostgreSQL Tools +# +# Copyright (C) 2013 - 2017, The pgAdmin Development Team +# This software is released under the PostgreSQL Licence +# +########################################################################## + +from __future__ import print_function +import time +import sys +import config +from selenium.webdriver import ActionChains +from selenium.webdriver.support.ui import WebDriverWait +from selenium.webdriver.support import expected_conditions as EC +from selenium.webdriver.common.by import By +from regression.python_test_utils import test_utils +from regression.feature_utils.base_feature_test import BaseFeatureTest + + +class QueryToolFeatureTest(BaseFeatureTest): + """ + This feature test will test the different query tool features. + """ + + scenarios = [ + ("Query tool feature test", dict()) + ] + + def before(self): + connection = test_utils.get_db_connection(self.server['db'], + self.server['username'], + self.server['db_password'], + self.server['host'], + self.server['port']) + test_utils.drop_database(connection, "acceptance_test_db") + test_utils.create_database(self.server, "acceptance_test_db") + self.page.wait_for_spinner_to_disappear() + self._connects_to_server() + self._locate_database_tree_node() + self._open_query_tool() + + def runTest(self): + # on demand result set on scrolling. + print("\nOn demand result set on scrolling... ", + file=sys.stderr, end="") + self._on_demand_result() + print("OK.", + file=sys.stderr) + self._clear_query_tool() + + # on demand result set on grid select all. + print("On demand result set on grid select all... ", + file=sys.stderr, end="") + self._on_demand_result_select_all_grid() + print("OK.", + file=sys.stderr) + self._clear_query_tool() + + # on demand result set on column select all. + print("On demand result set on column select all... ", + file=sys.stderr, end="") + self._on_demand_result_select_all_column() + print("OK.", + file=sys.stderr) + self._clear_query_tool() + + # explain query + print("Explain query... ", file=sys.stderr, end="") + self._query_tool_explain() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + # explain query with verbose + print("Explain query with verbose... ", file=sys.stderr, end="") + self._query_tool_explain_verbose() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + # explain query with costs + print("Explain query with costs... ", file=sys.stderr, end="") + self._query_tool_explain_cost() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + # explain analyze query + print("Explain analyze query... ", file=sys.stderr, end="") + self._query_tool_explain_analyze() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + # explain analyze query with buffers + print("Explain analyze query with buffers... ", file=sys.stderr, end="") + self._query_tool_explain_analyze_buffers() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + # explain analyze query with timing + print("Explain analyze query with timing... ", file=sys.stderr, end="") + self._query_tool_explain_analyze_timing() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + # auto commit disabled. + print("Auto commit disabled... ", file=sys.stderr, end="") + self._query_tool_auto_commit_disabled() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + # auto commit enabled. + print("Auto commit enabled... ", file=sys.stderr, end="") + self._query_tool_auto_commit_enabled() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + # auto rollback enabled. + print("Auto rollback enabled...", file=sys.stderr, end="") + self._query_tool_auto_rollback_enabled() + print(" OK.", file=sys.stderr) + self._clear_query_tool() + + # cancel query. + print("Cancel query... ", file=sys.stderr, end="") + self._query_tool_cancel_query() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + def after(self): + self.page.remove_server(self.server) + connection = test_utils.get_db_connection(self.server['db'], + self.server['username'], + self.server['db_password'], + self.server['host'], + self.server['port']) + test_utils.drop_database(connection, "acceptance_test_db") + + def _connects_to_server(self): + self.page.find_by_xpath( + "//*[@class='aciTreeText' and .='Servers']").click() + self.page.driver.find_element_by_link_text("Object").click() + ActionChains(self.page.driver) \ + .move_to_element( + self.page.driver.find_element_by_link_text("Create"))\ + .perform() + self.page.find_by_partial_link_text("Server...").click() + + server_config = self.server + self.page.fill_input_by_field_name("name", server_config['name']) + self.page.find_by_partial_link_text("Connection").click() + self.page.fill_input_by_field_name("host", server_config['host']) + self.page.fill_input_by_field_name("port", server_config['port']) + self.page.fill_input_by_field_name( + "username", + server_config['username'] + ) + self.page.fill_input_by_field_name( + "password", + server_config['db_password'] + ) + self.page.find_by_xpath("//button[contains(.,'Save')]").click() + + def _locate_database_tree_node(self): + self.page.toggle_open_tree_item(self.server['name']) + self.page.toggle_open_tree_item('Databases') + self.page.toggle_open_tree_item('acceptance_test_db') + + def _open_query_tool(self): + self.page.driver.find_element_by_link_text("Tools").click() + + tools_menu = self.page.driver.find_element_by_id('mnu_tools') + + # Query Tool is first li + query_tool = tools_menu.find_element_by_tag_name('li') + + t = time.time() + # wait until Query Tool menu becomes enabled. + while time.time() - t < 20: # 20 seconds + # if menu is disabled then it will have + # two classes 'menu-item disabled'. + # And if menu is enabled the it will have + # only one class 'menu-item'. + if 'menu-item' == str(query_tool.get_attribute('class')): + break + time.sleep(0.1) + else: + assert False, "'Tools -> Query Tool' menu did not enable." + + self.page.find_by_partial_link_text("Query Tool").click() + self.page.click_tab('Query -') + + self.page.driver.switch_to_frame( + self.page.driver.find_element_by_tag_name("iframe") + ) + + def _clear_query_tool(self): + # clear codemirror. + self.page.find_by_id("btn-edit").click() + # wait for alertify dialog open animation to complete. + time.sleep(1) + + self.page.click_element(self.page.find_by_xpath("//button[contains(.,'Yes')]")) + # wait for alertify dialog close animation to complete. + time.sleep(1) + + def _on_demand_result(self): + ON_DEMAND_CHUNKS = 2 + query = """-- On demand query result on scroll +SELECT generate_series(1, {}) as id""".format( + config.ON_DEMAND_RECORD_COUNT * ON_DEMAND_CHUNKS) + + wait = WebDriverWait(self.page.driver, 10) + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-flash").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas"))) + + # scroll to bottom to fetch next chunk of result set. + self.driver.execute_script( + "$('.slick-viewport').scrollTop($('.grid-canvas').height());" + ) + + # wait for ajax to complete. + time.sleep(1) + + # again scroll to bottom to bring last row of next chunk in + # viewport. + self.driver.execute_script( + "$('.slick-viewport').scrollTop($('.grid-canvas').height());" + ) + + row_id_to_find = config.ON_DEMAND_RECORD_COUNT * ON_DEMAND_CHUNKS + + canvas.find_element_by_xpath( + '//span[text()="{}"]'.format(row_id_to_find) + ) + + def _on_demand_result_select_all_grid(self): + ON_DEMAND_CHUNKS = 3 + query = """-- On demand query result on grid select all +SELECT generate_series(1, {}) as id""".format( + config.ON_DEMAND_RECORD_COUNT * ON_DEMAND_CHUNKS) + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-flash").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, ".slick-header-column"))).click() + + # wait for until all records are fetched and selected. + time.sleep(1) + # scroll to bottom to bring last row of next chunk in + # viewport. + self.driver.execute_script( + "$('.slick-viewport').scrollTop($('.grid-canvas').height());" + ) + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + + row_id_to_find = config.ON_DEMAND_RECORD_COUNT * ON_DEMAND_CHUNKS + + canvas.find_element_by_xpath( + '//span[text()="{}"]'.format(row_id_to_find) + ) + + def _on_demand_result_select_all_column(self): + ON_DEMAND_CHUNKS = 4 + query = """-- On demand query result on column select all +SELECT generate_series(1, {}) as id1, 'dummy' as id2""".format( + config.ON_DEMAND_RECORD_COUNT * ON_DEMAND_CHUNKS) + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-flash").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + # click on first data column to select all column. + + wait.until(EC.presence_of_element_located( + ( + By.XPATH, + "//span[contains(@class, 'column-name') and contains(., 'id1')]")) + ).click() + + # wait for until all records are fetched and selected. + time.sleep(1) + # scroll to bottom to bring last row of next chunk in + # viewport. + self.driver.execute_script( + "$('.slick-viewport').scrollTop($('.grid-canvas').height());" + ) + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + + row_id_to_find = config.ON_DEMAND_RECORD_COUNT * ON_DEMAND_CHUNKS + + canvas.find_element_by_xpath( + '//span[text()="{}"]'.format(row_id_to_find) + ) + + def _query_tool_explain(self): + query = """-- Explain query +SELECT generate_series(1, 1000) as id order by id desc""" + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-query-dropdown").click() + self.page.find_by_id("btn-explain").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + self.page.click_tab('Data Output') + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + # Search for Plan word in result + canvas.find_element_by_xpath("//*[contains(string(),'Plan')]") + + def _query_tool_explain_verbose(self): + query = """-- Explain query with verbose +SELECT generate_series(1, 1000) as id order by id desc""" + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + query_op = self.page.find_by_id("btn-query-dropdown") + query_op.click() + + ActionChains(self.driver).move_to_element( + query_op.find_element_by_xpath( + "//li[contains(.,'Explain Options')]")).perform() + + self.page.find_by_id("btn-explain-verbose").click() + + self.page.find_by_id("btn-explain").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + self.page.click_tab('Data Output') + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + # Search for 'Output' word in result + canvas.find_element_by_xpath("//*[contains(string(), 'Output')]") + + def _query_tool_explain_cost(self): + query = """-- Explain query with costs +SELECT generate_series(1, 1000) as id order by id desc""" + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + query_op = self.page.find_by_id("btn-query-dropdown") + query_op.click() + + ActionChains(self.driver).move_to_element( + query_op.find_element_by_xpath( + "//li[contains(.,'Explain Options')]")).perform() + + self.page.find_by_id("btn-explain-costs").click() + + self.page.find_by_id("btn-explain").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + self.page.click_tab('Data Output') + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + # Search for 'Total Cost word in result + canvas.find_element_by_xpath("//*[contains(string(),'Total Cost')]") + + def _query_tool_explain_analyze(self): + query = """-- Explain analyze query +SELECT generate_series(1, 1000) as id order by id desc""" + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-query-dropdown").click() + self.page.find_by_id("btn-explain-analyze").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + self.page.click_tab('Data Output') + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + # Search for Actual Rows word in result + canvas.find_element_by_xpath("//*[contains(string(),'Actual Rows')]") + + def _query_tool_explain_analyze_buffers(self): + query = """-- Explain analyze query with buffers +SELECT generate_series(1, 1000) as id order by id desc""" + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + query_op = self.page.find_by_id("btn-query-dropdown") + query_op.click() + + ActionChains(self.driver).move_to_element( + query_op.find_element_by_xpath( + "//li[contains(.,'Explain Options')]")).perform() + + self.page.find_by_id("btn-explain-buffers").click() + + self.page.find_by_id("btn-explain-analyze").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + self.page.click_tab('Data Output') + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + # Search for 'Shared Read Blocks' word in result + canvas.find_element_by_xpath("//*[contains(string(), 'Shared Read Blocks')]") + + def _query_tool_explain_analyze_timing(self): + query = """-- Explain analyze query with timing +SELECT generate_series(1, 1000) as id order by id desc""" + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + query_op = self.page.find_by_id("btn-query-dropdown") + query_op.click() + + ActionChains(self.driver).move_to_element( + query_op.find_element_by_xpath( + "//li[contains(.,'Explain Options')]")).perform() + + self.page.find_by_id("btn-explain-timing").click() + + self.page.find_by_id("btn-explain-analyze").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + self.page.click_tab('Data Output') + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + # Search for 'Actual Total Time' word in result + canvas.find_element_by_xpath("//*[contains(string(), 'Actual Total Time')]") + + def _query_tool_auto_commit_disabled(self): + table_name = 'query_tool_auto_commit_disabled_table' + query = """-- 1. Disable auto commit. +-- 2. Create table in public schema. +-- 3. ROLLBACK transaction. +-- 4. Check if table is *NOT* created. +CREATE TABLE public.{}();""".format(table_name) + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-query-dropdown").click() + + auto_commit_btn = self.page.find_by_id("btn-auto-commit") + + auto_commit_check = auto_commit_btn.find_element_by_tag_name("i") + + # if auto commit is enabled then 'i' element will + # have 'auto-commit fa fa-check' classes + # if auto commit is disabled then 'i' element will + # have 'auto-commit fa fa-check visibility-hidden' classes + + if 'auto-commit fa fa-check' == str(auto_commit_check.get_attribute( + 'class')): + auto_commit_btn.click() + + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "CREATE TABLE")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) Disable auto commit. +-- 2. (Done) Create table in public schema. +-- 3. ROLLBACK transaction. +-- 4. Check if table is *NOT* created. +ROLLBACK;""" + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "ROLLBACK")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) Disable auto commit. +-- 2. (Done) Create table in public schema. +-- 3. (Done) ROLLBACK transaction. +-- 4. Check if table is *NOT* created. +SELECT relname FROM pg_class WHERE relkind IN ('r','s','t') and relnamespace = 2200::oid;""" + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Data Output') + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas"))) + + el = canvas.find_elements_by_xpath("//div[contains(@class, 'slick-cell') and contains(text(), '{}')]".format(table_name)) + + assert len(el) == 0, "Table '{}' created with auto commit disabled and without any explicit commit.".format(table_name) + + def _query_tool_auto_commit_enabled(self): + table_name = 'query_tool_auto_commit_enabled_table' + query = """-- 1. END any open transaction. +-- 2. Enable auto commit. +-- 3. Create table in public schema. +-- 4. ROLLBACK transaction +-- 5. Check if table is created event after ROLLBACK. +END; +CREATE TABLE public.{}();""".format(table_name) + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-query-dropdown").click() + + auto_commit_btn = self.page.find_by_id("btn-auto-commit") + + auto_commit_check = auto_commit_btn.find_element_by_tag_name("i") + + # if auto commit is enabled then 'i' element will + # have 'auto-commit fa fa-check' classes + # if auto commit is disabled then 'i' element will + # have 'auto-commit fa fa-check visibility-hidden' classes + + if 'auto-commit fa fa-check visibility-hidden' == str(auto_commit_check.get_attribute( + 'class')): + auto_commit_btn.click() + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "CREATE TABLE")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) END any open transaction if any. +-- 2. (Done) Enable auto commit. +-- 3. (Done) Create table in public schema. +-- 4. ROLLBACK transaction +-- 5. Check if table is created event after ROLLBACK. +ROLLBACK;""" + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "ROLLBACK")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) END any open transaction if any. +-- 2. (Done) Enable auto commit. +-- 3. (Done) Create table in public schema. +-- 4. (Done) ROLLBACK transaction +-- 5. Check if table is created event after ROLLBACK. +SELECT relname FROM pg_class WHERE relkind IN ('r','s','t') and relnamespace = 2200::oid;""" + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.click_tab('Data Output') + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas"))) + + el = canvas.find_elements_by_xpath("//div[contains(@class, 'slick-cell') and contains(text(), '{}')]".format(table_name)) + + assert len(el) != 0, "Table '{}' is not created with auto commit enabled.".format(table_name) + + def _query_tool_auto_rollback_enabled(self): + table_name = 'query_tool_auto_rollback_enabled_table' + query = """-- 1. END any open transaction. +-- 2. Enable auto rollback and disable auto commit. +-- 3. Create table in public schema. +-- 4. Generate error in transaction. +-- 5. END transaction. +-- 6. Check if table is *NOT* created after ending transaction. +END;""" + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self._clear_query_tool() + + query = """-- 1. (Done) END any open transaction. +-- 2. Enable auto rollback and disable auto commit. +-- 3. Create table in public schema. +-- 4. Generate error in transaction. +-- 5. END transaction. +-- 6. Check if table is *NOT* created after ending transaction. +CREATE TABLE public.{}();""".format(table_name) + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-query-dropdown").click() + + auto_rollback_btn = self.page.find_by_id("btn-auto-rollback") + + auto_rollback_check = auto_rollback_btn.find_element_by_tag_name("i") + + # if auto rollback is enabled then 'i' element will + # have 'auto-rollback fa fa-check' classes + # if auto rollback is disabled then 'i' element will + # have 'auto-rollback fa fa-check visibility-hidden' classes + + if 'auto-rollback fa fa-check visibility-hidden' == str(auto_rollback_check.get_attribute( + 'class')): + auto_rollback_btn.click() + + auto_commit_btn = self.page.find_by_id("btn-auto-commit") + + auto_commit_check = auto_commit_btn.find_element_by_tag_name("i") + + # if auto commit is enabled then 'i' element will + # have 'auto-commit fa fa-check' classes + # if auto commit is disabled then 'i' element will + # have 'auto-commit fa fa-check visibility-hidden' classes + + if 'auto-commit fa fa-check' == str(auto_commit_check.get_attribute( + 'class')): + auto_commit_btn.click() + + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "CREATE TABLE")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) END any open transaction. +-- 2. (Done) Enable auto rollback and disable auto commit. +-- 3. (Done) Create table in public schema. +-- 4. Generate error in transaction. +-- 5. END transaction. +-- 6. Check if table is *NOT* created after ending transaction. +SELECT 1/0;""" + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "division by zero")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) END any open transaction. +-- 2. (Done) Enable auto rollback and disable auto commit. +-- 3. (Done) Create table in public schema. +-- 4. (Done) Generate error in transaction. +-- 5. END transaction. +-- 6. Check if table is *NOT* created after ending transaction. +END;""" + + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "Query returned successfully")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) END any open transaction. +-- 2. (Done) Enable auto rollback and disable auto commit. +-- 3. (Done) Create table in public schema. +-- 4. (Done) Generate error in transaction. +-- 5. (Done) END transaction. +-- 6. Check if table is *NOT* created after ending transaction. +SELECT relname FROM pg_class WHERE relkind IN ('r','s','t') and relnamespace = 2200::oid;""" + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Data Output') + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas"))) + + el = canvas.find_elements_by_xpath("//div[contains(@class, 'slick-cell') and contains(text(), '{}')]".format(table_name)) + + assert len(el) == 0, "Table '{}' created even after ROLLBACK due to sql error.".format(table_name) + + def _query_tool_cancel_query(self): + query = """-- 1. END any open transaction. +-- 2. Enable auto commit and Disable auto rollback. +-- 3. Execute long running query. +-- 4. Cancel long running query execution. +END; +SELECT 1, pg_sleep(10)""" + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-query-dropdown").click() + + auto_rollback_btn = self.page.find_by_id("btn-auto-rollback") + + auto_rollback_check = auto_rollback_btn.find_element_by_tag_name("i") + + # if auto rollback is enabled then 'i' element will + # have 'auto-rollback fa fa-check' classes + # if auto rollback is disabled then 'i' element will + # have 'auto-rollback fa fa-check visibility-hidden' classes + + if 'auto-rollback fa fa-check' == str(auto_rollback_check.get_attribute( + 'class')): + auto_rollback_btn.click() + + auto_commit_btn = self.page.find_by_id("btn-auto-commit") + + auto_commit_check = auto_commit_btn.find_element_by_tag_name("i") + + # if auto commit is enabled then 'i' element will + # have 'auto-commit fa fa-check' classes + # if auto commit is disabled then 'i' element will + # have 'auto-commit fa fa-check visibility-hidden' classes + + if 'auto-commit fa fa-check visibility-hidden' == str(auto_commit_check.get_attribute( + 'class')): + auto_commit_btn.click() + + self.page.find_by_id("btn-flash").click() + self.driver.find_element_by_xpath("//*[@id='fetching_data']") + self.page.find_by_id("btn-cancel-query").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "canceling statement due to user request")]' + ) diff --git a/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py b/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py index 4fd0783..a0df44e 100644 --- a/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py +++ b/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py @@ -10,6 +10,9 @@ from selenium.webdriver import ActionChains from regression.python_test_utils import test_utils from regression.feature_utils.base_feature_test import BaseFeatureTest +from selenium.webdriver.support.ui import WebDriverWait +from selenium.webdriver.support import expected_conditions as EC +from selenium.webdriver.common.by import By import time class CheckForXssFeatureTest(BaseFeatureTest): @@ -152,11 +155,16 @@ class CheckForXssFeatureTest(BaseFeatureTest): self.page.fill_codemirror_area_with("select '<img src=\"x\" onerror=\"console.log(1)\">'") time.sleep(1) self.page.find_by_id("btn-flash").click() - time.sleep(2) + wait = WebDriverWait(self.page.driver, 5) - source_code = self.page.find_by_xpath( - "//*[@id='0']//*[@id='datagrid']/div[5]/div/div[1]/div[2]" - ).get_attribute('innerHTML') + result_row = self.page.find_by_xpath( + "//*[contains(@class, 'ui-widget-content') and contains(@style, 'top:0px')]" + ) + + cells = result_row.find_elements_by_tag_name('div') + + # remove first element as it is row number. + source_code = cells[1].get_attribute('innerHTML') self._check_escaped_characters( source_code, diff --git a/web/pgadmin/static/js/selection/column_selector.js b/web/pgadmin/static/js/selection/column_selector.js index 305b438..f3cb727 100644 --- a/web/pgadmin/static/js/selection/column_selector.js +++ b/web/pgadmin/static/js/selection/column_selector.js @@ -4,13 +4,18 @@ define([ 'slickgrid', ], function ($, RangeSelectionHelper) { var ColumnSelector = function () { - var Slick = window.Slick; - var gridEventBus = new Slick.EventHandler(); + var Slick = window.Slick, + gridEventBus = new Slick.EventHandler(), + onBeforeColumnSelectAll = new Slick.Event(), + onColumnSelectAll = new Slick.Event(); var init = function (grid) { gridEventBus.subscribe(grid.onHeaderClick, handleHeaderClick.bind(null, grid)); grid.getSelectionModel().onSelectedRangesChanged .subscribe(handleSelectedRangesChanged.bind(null, grid)); + onColumnSelectAll.subscribe(function(e, args) { + updateRanges(args.grid, args.column.id); + }); }; var handleHeaderClick = function (grid, event, args) { @@ -21,11 +26,20 @@ define([ if (isColumnSelectable(columnDefinition)) { var $columnHeader = $(event.target); if (hasClickedChildOfColumnHeader(event)) { + if ($(event.target).hasClass('slick-resizable-handle')) { + return; + } $columnHeader = $(event.target).parents('.slick-header-column'); } $columnHeader.toggleClass('selected'); - updateRanges(grid, columnDefinition.id); + if ($columnHeader.hasClass('selected')) { + onBeforeColumnSelectAll.notify(args, event); + } + + if (!(event.isPropagationStopped() || event.isImmediatePropagationStopped())) { + updateRanges(grid, columnDefinition.id); + } } }; @@ -107,6 +121,8 @@ define([ $.extend(this, { 'init': init, 'getColumnDefinitions': getColumnDefinitions, + 'onBeforeColumnSelectAll': onBeforeColumnSelectAll, + 'onColumnSelectAll': onColumnSelectAll, }); }; return ColumnSelector; diff --git a/web/pgadmin/static/js/selection/copy_data.js b/web/pgadmin/static/js/selection/copy_data.js index c226323..3525966 100644 --- a/web/pgadmin/static/js/selection/copy_data.js +++ b/web/pgadmin/static/js/selection/copy_data.js @@ -12,19 +12,19 @@ function ($, _, clipboard, RangeSelectionHelper, rangeBoundaryNavigator) { var grid = self.slickgrid; var columnDefinitions = grid.getColumns(); var selectedRanges = grid.getSelectionModel().getSelectedRanges(); - var data = grid.getData(); + var dataView = grid.getData(); var rows = grid.getSelectedRows(); if (RangeSelectionHelper.areAllRangesCompleteRows(grid, selectedRanges)) { self.copied_rows = rows.map(function (rowIndex) { - return data[rowIndex]; + return grid.getDataItem(rowIndex); }); setPasteRowButtonEnablement(self.can_edit, true); } else { self.copied_rows = []; setPasteRowButtonEnablement(self.can_edit, false); } - var csvText = rangeBoundaryNavigator.rangesToCsv(data, columnDefinitions, selectedRanges); + var csvText = rangeBoundaryNavigator.rangesToCsv(dataView.getItems(), columnDefinitions, selectedRanges); if (csvText) { clipboard.copyTextToClipboard(csvText); } diff --git a/web/pgadmin/static/js/selection/grid_selector.js b/web/pgadmin/static/js/selection/grid_selector.js index 24fd9d6..ddbbe23 100644 --- a/web/pgadmin/static/js/selection/grid_selector.js +++ b/web/pgadmin/static/js/selection/grid_selector.js @@ -6,21 +6,31 @@ define(['jquery', 'sources/url_for', ], function ($, gettext, ColumnSelector, RowSelector, RangeSelectionHelper, url_for) { var GridSelector = function (columnDefinitions) { - var rowSelector = new RowSelector(columnDefinitions); - var columnSelector = new ColumnSelector(columnDefinitions); + var Slick = window.Slick, + rowSelector = new RowSelector(columnDefinitions), + columnSelector = new ColumnSelector(columnDefinitions), + onBeforeGridSelectAll = new Slick.Event(), + onGridSelectAll = new Slick.Event(), + onBeforeGridColumnSelectAll = columnSelector.onBeforeColumnSelectAll, + onGridColumnSelectAll = columnSelector.onColumnSelectAll; var init = function (grid) { this.grid = grid; grid.onHeaderClick.subscribe(function (event, eventArguments) { - if (eventArguments.column.selectAllOnClick) { - toggleSelectAll(grid); + if (eventArguments.column.selectAllOnClick && !$(event.target).hasClass('slick-resizable-handle')) { + toggleSelectAll(grid, event, eventArguments); } }); grid.getSelectionModel().onSelectedRangesChanged - .subscribe(handleSelectedRangesChanged.bind(null, grid)); + .subscribe(handleSelectedRangesChanged.bind(null, grid)); + grid.registerPlugin(rowSelector); grid.registerPlugin(columnSelector); + + onGridSelectAll.subscribe(function(e, args) { + RangeSelectionHelper.selectAll(args.grid); + }); }; var getColumnDefinitions = function (columnDefinitions) { @@ -45,11 +55,14 @@ define(['jquery', } } - function toggleSelectAll(grid) { + function toggleSelectAll(grid, event, eventArguments) { if (RangeSelectionHelper.isEntireGridSelected(grid)) { selectNone(grid); } else { - RangeSelectionHelper.selectAll(grid); + onBeforeGridSelectAll.notify(eventArguments, event); + if (!(event.isPropagationStopped() || event.isImmediatePropagationStopped())) { + RangeSelectionHelper.selectAll(grid); + } } } @@ -61,6 +74,10 @@ define(['jquery', $.extend(this, { 'init': init, 'getColumnDefinitions': getColumnDefinitions, + 'onBeforeGridSelectAll': onBeforeGridSelectAll, + 'onGridSelectAll': onGridSelectAll, + 'onBeforeGridColumnSelectAll': onBeforeGridColumnSelectAll, + 'onGridColumnSelectAll': onGridColumnSelectAll, }); }; diff --git a/web/pgadmin/static/js/selection/range_boundary_navigator.js b/web/pgadmin/static/js/selection/range_boundary_navigator.js index 803f75a..274b621 100644 --- a/web/pgadmin/static/js/selection/range_boundary_navigator.js +++ b/web/pgadmin/static/js/selection/range_boundary_navigator.js @@ -58,6 +58,7 @@ function (RangeSelectionHelper) { }, rangesToCsv: function (data, columnDefinitions, selectedRanges) { + var rowRangeBounds = selectedRanges.map(function (range) { return [range.fromRow, range.toRow]; }); @@ -72,6 +73,7 @@ function (RangeSelectionHelper) { var csvRows = this.mapOver2DArray(rowRangeBounds, colRangeBounds, this.csvCell.bind(this, data, columnDefinitions), function (rowData) { return rowData.join(','); }); + return csvRows.join('\n'); }, @@ -101,7 +103,7 @@ function (RangeSelectionHelper) { }, csvCell: function (data, columnDefinitions, rowId, colId) { - var val = data[rowId][columnDefinitions[colId].pos]; + var val = data[rowId][columnDefinitions[colId].field]; if (val && _.isObject(val)) { val = '\'' + JSON.stringify(val) + '\''; diff --git a/web/pgadmin/static/js/selection/row_selector.js b/web/pgadmin/static/js/selection/row_selector.js index 236ed62..59d2fa2 100644 --- a/web/pgadmin/static/js/selection/row_selector.js +++ b/web/pgadmin/static/js/selection/row_selector.js @@ -82,7 +82,8 @@ define([ formatter: function (rowIndex) { return '<span ' + 'data-row="' + rowIndex + '" ' + - 'data-cell-type="row-header-selector"/>'; + 'data-cell-type="row-header-selector">' + + (rowIndex+1) + '</span>'; }, width: 30, }); diff --git a/web/pgadmin/static/js/selection/set_staged_rows.js b/web/pgadmin/static/js/selection/set_staged_rows.js index 46977e3..e7a529c 100644 --- a/web/pgadmin/static/js/selection/set_staged_rows.js +++ b/web/pgadmin/static/js/selection/set_staged_rows.js @@ -22,53 +22,44 @@ define( $(selector).prop('disabled', false); } - function getRowPrimaryKeyValuesToStage(selectedRows, primaryKeyColumnIndices, gridData) { + function getRowPrimaryKeyValuesToStage(selectedRows, primaryKeys, dataView, client_primary_key) { return _.reduce(selectedRows, function (primaryKeyValuesToStage, dataGridRowIndex) { - var gridRow = gridData[dataGridRowIndex]; - - if (isRowMissingPrimaryKeys(gridRow, primaryKeyColumnIndices)) { + var gridRow = dataView.getItem(dataGridRowIndex); + if (isRowMissingPrimaryKeys(gridRow, primaryKeys)) { return primaryKeyValuesToStage; } - - var tempPK = gridRow.__temp_PK; - primaryKeyValuesToStage[tempPK] = getSingleRowPrimaryKeyValueToStage(primaryKeyColumnIndices, gridRow); - + var tempPK = gridRow[client_primary_key]; + primaryKeyValuesToStage[tempPK] = getSingleRowPrimaryKeyValueToStage(primaryKeys, gridRow); return primaryKeyValuesToStage; }, {}); } - function isRowMissingPrimaryKeys(gridRow, primaryKeyColumnIndices) { + function isRowMissingPrimaryKeys(gridRow, primaryKeys) { if (_.isUndefined(gridRow)) { return true; } return !_.isUndefined( - _.find(primaryKeyColumnIndices, function (pkIndex) { - return _.isUndefined(gridRow[pkIndex]); + _.find(primaryKeys , function (pk) { + return _.isUndefined(gridRow[pk]); }) ); } - function getSingleRowPrimaryKeyValueToStage(primaryKeyColumnIndices, gridRow) { + function getSingleRowPrimaryKeyValueToStage(primaryKeys, gridRow) { var rowToStage = {}; - if (primaryKeyColumnIndices.length) { - _.each(_.keys(gridRow), function (columnPos) { - if (_.contains(primaryKeyColumnIndices, Number(columnPos))) - rowToStage[columnPos] = gridRow[columnPos]; + if (primaryKeys && primaryKeys.length) { + _.each(_.keys(gridRow), function (columnNames) { + if (_.contains(primaryKeys, columnNames)) + rowToStage[columnNames] = gridRow[columnNames]; }); } return rowToStage; } function getPrimaryKeysForSelectedRows(self, selectedRows) { - var primaryKeyColumnIndices = _.map(_.keys(self.keys), function (columnName) { - var columnInfo = _.findWhere(self.columns, {name: columnName}); - return columnInfo['pos']; - }); - - var gridData = self.grid.getData(); - var stagedRows = getRowPrimaryKeyValuesToStage(selectedRows, primaryKeyColumnIndices, gridData); - + var dataView = self.grid.getData(); + var stagedRows = getRowPrimaryKeyValuesToStage(selectedRows, _.keys(self.keys), dataView, self.client_primary_key); return stagedRows; } diff --git a/web/pgadmin/static/js/slickgrid/slick.pgadmin.editors.js b/web/pgadmin/static/js/slickgrid/slick.pgadmin.editors.js index f2dabe0..15b5fe9 100644 --- a/web/pgadmin/static/js/slickgrid/slick.pgadmin.editors.js +++ b/web/pgadmin/static/js/slickgrid/slick.pgadmin.editors.js @@ -76,18 +76,18 @@ last_value = (column_type === 'number') ? (_.isEmpty(last_value) || last_value) : last_value; - item[args.column.pos] = state; + item[args.column.field] = state; if (last_value && _.isNull(state) && (_.isUndefined(grid.copied_rows[row]) || _.isUndefined(grid.copied_rows[row][cell])) ) { - item[args.column.pos] = undefined; + item[args.column.field] = undefined; if (grid.copied_rows[row] == undefined) grid.copied_rows[row] = []; grid.copied_rows[row][cell] = 1; } } else { - item[args.column.pos] = state; + item[args.column.field] = state; } } @@ -189,14 +189,14 @@ this.loadValue = function (item) { var col = args.column; - if (_.isUndefined(item[args.column.pos]) && col.has_default_val) { + if (_.isUndefined(item[args.column.field]) && col.has_default_val) { $input.val(defaultValue = ""); } - else if (item[args.column.pos] === "") { + else if (item[args.column.field] === "") { $input.val(defaultValue = "''"); } else { - $input.val(defaultValue = item[args.column.pos]); + $input.val(defaultValue = item[args.column.field]); $input.select(); } }; @@ -323,7 +323,7 @@ }; this.loadValue = function (item) { - var data = defaultValue = item[args.column.pos]; + var data = defaultValue = item[args.column.field]; if (data && typeof data === "object" && !Array.isArray(data)) { data = JSON.stringify(data); } else if (Array.isArray(data)) { @@ -443,7 +443,7 @@ }; this.loadValue = function (item) { - $input.val(defaultValue = item[args.column.pos]); + $input.val(defaultValue = item[args.column.field]); $input.select(); }; @@ -452,7 +452,7 @@ }; this.applyValue = function (item, state) { - item[args.column.pos] = state; + item[args.column.field] = state; }; this.isValueChanged = function () { @@ -531,13 +531,13 @@ }; this.loadValue = function (item) { - defaultValue = item[args.column.pos]; - if (_.isNull(defaultValue)|| _.isUndefined(defaultValue)) { + defaultValue = item[args.column.field]; + if (_.isNull(defaultValue)||_.isUndefined(defaultValue)) { $select.prop('indeterminate', true); $select.data('checked', 2); } else { - defaultValue = !!item[args.column.pos]; + defaultValue = !!item[args.column.field]; if (defaultValue) { $select.prop('checked', true); $select.data('checked', 0); @@ -556,7 +556,7 @@ }; this.applyValue = function (item, state) { - item[args.column.pos] = state; + item[args.column.field] = state; }; this.isValueChanged = function () { @@ -648,7 +648,7 @@ }; this.loadValue = function (item) { - var data = defaultValue = item[args.column.pos]; + var data = defaultValue = item[args.column.field]; if (typeof data === "object" && !Array.isArray(data)) { data = JSON.stringify(data); } else if (Array.isArray(data)) { @@ -671,7 +671,7 @@ }; this.applyValue = function (item, state) { - item[args.column.pos] = state; + item[args.column.field] = state; }; this.isValueChanged = function () { @@ -725,7 +725,7 @@ }; this.loadValue = function (item) { - var value = item[args.column.pos]; + var value = item[args.column.field]; // Check if value is null or undefined if (value === undefined && typeof value === "undefined") { @@ -858,7 +858,7 @@ }; this.loadValue = function (item) { - defaultValue = item[args.column.pos]; + defaultValue = item[args.column.field]; $input.val(defaultValue); $input[0].defaultValue = defaultValue; $input.select(); diff --git a/web/pgadmin/templates/base.html b/web/pgadmin/templates/base.html index 829c195..7616eac 100755 --- a/web/pgadmin/templates/base.html +++ b/web/pgadmin/templates/base.html @@ -146,6 +146,12 @@ ], "exports": 'Slick.Grid' }, + "slickgrid/slick.dataview": { + "deps": [ + "slickgrid" + ], + "exports": 'Slick.Data.DataView' + }, "flotr2": { deps: ['bean'], exports: function(bean) { diff --git a/web/pgadmin/tools/sqleditor/__init__.py b/web/pgadmin/tools/sqleditor/__init__.py index 828cb99..709332c 100644 --- a/web/pgadmin/tools/sqleditor/__init__.py +++ b/web/pgadmin/tools/sqleditor/__init__.py @@ -27,7 +27,7 @@ from pgadmin.utils.sqlautocomplete.autocomplete import SQLAutoComplete from pgadmin.misc.file_manager import Filemanager -from config import PG_DEFAULT_DRIVER +from config import PG_DEFAULT_DRIVER, ON_DEMAND_RECORD_COUNT MODULE_NAME = 'sqleditor' @@ -229,13 +229,32 @@ def start_view_data(trans_id): # Check the transaction and connection status status, error_msg, conn, trans_obj, session_obj = check_transaction_status(trans_id) + + # get the default connection as current connection which is attached to + # trans id holds the cursor which has query result so we cannot use that + # connection to execute another query otherwise we'll lose query result. + + manager = get_driver(PG_DEFAULT_DRIVER).connection_manager(trans_obj.sid) + default_conn = manager.connection(did=trans_obj.did) + + # Connect to the Server if not connected. + if not default_conn.connected(): + status, msg = default_conn.connect() + if not status: + return make_json_response( + data={'status': status, 'result': u"{}".format(msg)} + ) + if status and conn is not None \ and trans_obj is not None and session_obj is not None: try: + # set fetched row count to 0 as we are executing query again. + trans_obj.update_fetched_row_cnt(0) + session_obj['command_obj'] = pickle.dumps(trans_obj, -1) # Fetch the sql and primary_keys from the object sql = trans_obj.get_sql() - pk_names, primary_keys = trans_obj.get_primary_keys() + pk_names, primary_keys = trans_obj.get_primary_keys(default_conn) # Fetch the applied filter. filter_applied = trans_obj.is_filter_applied() @@ -303,6 +322,8 @@ def start_query_tool(trans_id): # Use pickle.loads function to get the command object session_obj = grid_data[str(trans_id)] trans_obj = pickle.loads(session_obj['command_obj']) + # set fetched row count to 0 as we are executing query again. + trans_obj.update_fetched_row_cnt(0) can_edit = False can_filter = False @@ -429,65 +450,6 @@ def preferences(trans_id): return success_return() [email protected]('/columns/<int:trans_id>', methods=["GET"]) -@login_required -def get_columns(trans_id): - """ - This method will returns list of columns of last async query. - - Args: - trans_id: unique transaction id - """ - columns = dict() - columns_info = None - primary_keys = None - rset = None - status, error_msg, conn, trans_obj, session_obj = check_transaction_status(trans_id) - if status and conn is not None and session_obj is not None: - - ver = conn.manager.version - # Get the template path for the column - template_path = 'column/sql/#{0}#'.format(ver) - command_obj = pickle.loads(session_obj['command_obj']) - if hasattr(command_obj, 'obj_id'): - SQL = render_template("/".join([template_path, - 'nodes.sql']), - tid=command_obj.obj_id) - # rows with attribute not_null - status, rset = conn.execute_2darray(SQL) - if not status: - return internal_server_error(errormsg=rset) - - # Check PK column info is available or not - if 'primary_keys' in session_obj: - primary_keys = session_obj['primary_keys'] - - # Fetch column information - columns_info = conn.get_column_info() - if columns_info is not None: - for key, col in enumerate(columns_info): - col_type = dict() - col_type['type_code'] = col['type_code'] - col_type['type_name'] = None - if rset: - col_type['not_null'] = col['not_null'] = \ - rset['rows'][key]['not_null'] - - col_type['has_default_val'] = col['has_default_val'] = \ - rset['rows'][key]['has_default_val'] - - columns[col['name']] = col_type - - # As we changed the transaction object we need to - # restore it and update the session variable. - session_obj['columns_info'] = columns - update_session_grid_transaction(trans_id, session_obj) - - return make_json_response(data={'status': True, - 'columns': columns_info, - 'primary_keys': primary_keys}) - - @blueprint.route('/poll/<int:trans_id>', methods=["GET"]) @login_required def poll(trans_id): @@ -499,12 +461,21 @@ def poll(trans_id): """ result = None rows_affected = 0 + rows_fetched_from = 0 + rows_fetched_to = 0 + has_more_rows = False additional_result = [] + columns = dict() + columns_info = None + primary_keys = None + types = {} + client_primary_key = None + rset = None # Check the transaction and connection status status, error_msg, conn, trans_obj, session_obj = check_transaction_status(trans_id) if status and conn is not None and session_obj is not None: - status, result = conn.poll(formatted_exception_msg=True) + status, result = conn.poll(formatted_exception_msg=True, no_result=True) if not status: return internal_server_error(result) elif status == ASYNC_OK: @@ -519,6 +490,80 @@ def poll(trans_id): if (trans_status == TX_STATUS_INERROR and trans_obj.auto_rollback): conn.execute_void("ROLLBACK;") + + st, result = conn.async_fetchmany_2darray(ON_DEMAND_RECORD_COUNT) + if st: + if 'primary_keys' in session_obj: + primary_keys = session_obj['primary_keys'] + + # Fetch column information + columns_info = conn.get_column_info() + client_primary_key = generate_client_primary_key_name( + columns_info + ) + session_obj['client_primary_key'] = client_primary_key + + if columns_info is not None: + + command_obj = pickle.loads(session_obj['command_obj']) + if hasattr(command_obj, 'obj_id'): + # Get the template path for the column + template_path = 'column/sql/#{0}#'.format( + conn.manager.version + ) + + SQL = render_template("/".join([template_path, + 'nodes.sql']), + tid=command_obj.obj_id) + # rows with attribute not_null + colst, rset = conn.execute_2darray(SQL) + if not colst: + return internal_server_error(errormsg=rset) + + for key, col in enumerate(columns_info): + col_type = dict() + col_type['type_code'] = col['type_code'] + col_type['type_name'] = None + columns[col['name']] = col_type + + if rset: + col_type['not_null'] = col['not_null'] = \ + rset['rows'][key]['not_null'] + + col_type['has_default_val'] = \ + col['has_default_val'] = \ + rset['rows'][key]['has_default_val'] + + if columns: + st, types = fetch_pg_types(columns, trans_obj) + + if not st: + return internal_server_error(types) + + for col_info in columns.values(): + for col_type in types: + if col_type['oid'] == col_info['type_code']: + col_info['type_name'] = col_type['typname'] + + session_obj['columns_info'] = columns + # status of async_fetchmany_2darray is True and result is none + # means nothing to fetch + if result and rows_affected > -1: + res_len = len(result) + if res_len == ON_DEMAND_RECORD_COUNT: + has_more_rows = True + + if res_len > 0: + rows_fetched_from = trans_obj.get_fetched_row_cnt() + trans_obj.update_fetched_row_cnt(rows_fetched_from + res_len) + rows_fetched_from += 1 + rows_fetched_to = trans_obj.get_fetched_row_cnt() + session_obj['command_obj'] = pickle.dumps(trans_obj, -1) + + # As we changed the transaction object we need to + # restore it and update the session variable. + update_session_grid_transaction(trans_id, session_obj) + elif status == ASYNC_EXECUTION_ABORTED: status = 'Cancel' else: @@ -559,51 +604,123 @@ def poll(trans_id): data={ 'status': status, 'result': result, 'rows_affected': rows_affected, - 'additional_messages': additional_messages + 'rows_fetched_from': rows_fetched_from, + 'rows_fetched_to': rows_fetched_to, + 'additional_messages': additional_messages, + 'has_more_rows': has_more_rows, + 'colinfo': columns_info, + 'primary_keys': primary_keys, + 'types': types, + 'client_primary_key': client_primary_key } ) [email protected]('/fetch/types/<int:trans_id>', methods=["GET"]) [email protected]('/fetch/<int:trans_id>', methods=["GET"]) [email protected]('/fetch/<int:trans_id>/<int:fetch_all>', methods=["GET"]) @login_required -def fetch_pg_types(trans_id): +def fetch(trans_id, fetch_all=None): + result = None + has_more_rows = False + rows_fetched_from = 0 + rows_fetched_to = 0 + fetch_row_cnt = -1 if fetch_all == 1 else ON_DEMAND_RECORD_COUNT + + # Check the transaction and connection status + status, error_msg, conn, trans_obj, session_obj = check_transaction_status(trans_id) + if status and conn is not None and session_obj is not None: + status, result = conn.async_fetchmany_2darray(fetch_row_cnt) + if not status: + status = 'Error' + else: + status = 'Success' + res_len = len(result) + if fetch_row_cnt != -1 and res_len == ON_DEMAND_RECORD_COUNT: + has_more_rows = True + + if res_len: + rows_fetched_from = trans_obj.get_fetched_row_cnt() + trans_obj.update_fetched_row_cnt(rows_fetched_from + res_len) + rows_fetched_from += 1 + rows_fetched_to = trans_obj.get_fetched_row_cnt() + session_obj['command_obj'] = pickle.dumps(trans_obj, -1) + update_session_grid_transaction(trans_id, session_obj) + else: + status = 'NotConnected' + result = error_msg + + return make_json_response( + data={ + 'status': status, 'result': result, + 'has_more_rows': has_more_rows, + 'rows_fetched_from': rows_fetched_from, + 'rows_fetched_to': rows_fetched_to + } + ) + + +def fetch_pg_types(columns_info, trans_obj): """ This method is used to fetch the pg types, which is required to map the data type comes as a result of the query. Args: - trans_id: unique transaction id + columns_info: """ - # Check the transaction and connection status - status, error_msg, conn, trans_obj, session_obj = check_transaction_status(trans_id) - if status and conn is not None \ - and trans_obj is not None and session_obj is not None: - res = {} - if 'columns_info' in session_obj \ - and session_obj['columns_info'] is not None: + # get the default connection as current connection attached to trans id + # holds the cursor which has query result so we cannot use that connection + # to execute another query otherwise we'll lose query result. + + manager = get_driver(PG_DEFAULT_DRIVER).connection_manager(trans_obj.sid) + default_conn = manager.connection(did=trans_obj.did) + + # Connect to the Server if not connected. + res = [] + if not default_conn.connected(): + status, msg = default_conn.connect() + if not status: + return status, msg - oids = [session_obj['columns_info'][col]['type_code'] for col in session_obj['columns_info']] + oids = [columns_info[col]['type_code'] for col in columns_info] - if oids: - status, res = conn.execute_dict( - u"""SELECT oid, format_type(oid,null) as typname FROM pg_type WHERE oid IN %s ORDER BY oid; + if oids: + status, res = default_conn.execute_dict( + u"""SELECT oid, format_type(oid,null) as typname FROM pg_type WHERE oid IN %s ORDER BY oid; """, [tuple(oids)]) - if status: - # iterate through pg_types and update the type name in session object - for record in res['rows']: - for col in session_obj['columns_info']: - type_obj = session_obj['columns_info'][col] - if type_obj['type_code'] == record['oid']: - type_obj['type_name'] = record['typname'] + if not status: + return False, res - update_session_grid_transaction(trans_id, session_obj) + return status, res['rows'] else: - status = False - res = error_msg - - return make_json_response(data={'status': status, 'result': res}) + return True, [] + + +def generate_client_primary_key_name(columns_info): + temp_key = '__temp_PK' + if not columns_info: + return temp_key + + initial_temp_key_len = len(temp_key) + duplicate = False + suffix = 1 + while 1: + for col in columns_info: + if col['name'] == temp_key: + duplicate = True + break + if duplicate: + if initial_temp_key_len == len(temp_key): + temp_key += str(suffix) + suffix += 1 + else: + temp_key = temp_key[:-1] + str(suffix) + suffix += 1 + duplicate = False + else: + break + return temp_key @blueprint.route('/save/<int:trans_id>', methods=["PUT", "POST"]) @@ -615,7 +732,6 @@ def save(trans_id): Args: trans_id: unique transaction id """ - if request.data: changed_data = json.loads(request.data, encoding='utf-8') else: @@ -625,7 +741,6 @@ def save(trans_id): status, error_msg, conn, trans_obj, session_obj = check_transaction_status(trans_id) if status and conn is not None \ and trans_obj is not None and session_obj is not None: - setattr(trans_obj, 'columns_info', session_obj['columns_info']) # If there is no primary key found then return from the function. if len(session_obj['primary_keys']) <= 0 or len(changed_data) <= 0: @@ -636,7 +751,22 @@ def save(trans_id): } ) - status, res, query_res, _rowid = trans_obj.save(changed_data) + manager = get_driver(PG_DEFAULT_DRIVER).connection_manager(trans_obj.sid) + default_conn = manager.connection(did=trans_obj.did) + + # Connect to the Server if not connected. + if not default_conn.connected(): + status, msg = default_conn.connect() + if not status: + return make_json_response( + data={'status': status, 'result': u"{}".format(msg)} + ) + + status, res, query_res, _rowid = trans_obj.save( + changed_data, + session_obj['columns_info'], + session_obj['client_primary_key'], + default_conn) else: status = False res = error_msg diff --git a/web/pgadmin/tools/sqleditor/command.py b/web/pgadmin/tools/sqleditor/command.py index b7d8a78..51b1f56 100644 --- a/web/pgadmin/tools/sqleditor/command.py +++ b/web/pgadmin/tools/sqleditor/command.py @@ -258,7 +258,21 @@ class SQLFilter(object): return status, result -class GridCommand(BaseCommand, SQLFilter): +class FetchedRowTracker(object): + """ + Keeps track of fetched row count. + """ + def __init__(self, **kwargs): + self.fetched_rows = 0 + + def get_fetched_row_cnt(self): + return self.fetched_rows + + def update_fetched_row_cnt(self, rows_cnt): + self.fetched_rows = rows_cnt + + +class GridCommand(BaseCommand, SQLFilter, FetchedRowTracker): """ class GridCommand(object) @@ -290,6 +304,7 @@ class GridCommand(BaseCommand, SQLFilter): """ BaseCommand.__init__(self, **kwargs) SQLFilter.__init__(self, **kwargs) + FetchedRowTracker.__init__(self, **kwargs) # Save the connection id, command type self.conn_id = kwargs['conn_id'] if 'conn_id' in kwargs else None @@ -299,10 +314,10 @@ class GridCommand(BaseCommand, SQLFilter): if self.cmd_type == VIEW_FIRST_100_ROWS or self.cmd_type == VIEW_LAST_100_ROWS: self.limit = 100 - def get_primary_keys(self): + def get_primary_keys(self, *args, **kwargs): return None, None - def save(self, changed_data): + def save(self, changed_data, default_conn=None): return forbidden(errmsg=gettext("Data cannot be saved for the current object.")) def get_limit(self): @@ -340,14 +355,14 @@ class TableCommand(GridCommand): # call base class init to fetch the table name super(TableCommand, self).__init__(**kwargs) - def get_sql(self): + def get_sql(self, default_conn=None): """ This method is used to create a proper SQL query to fetch the data for the specified table """ # Fetch the primary keys for the table - pk_names, primary_keys = self.get_primary_keys() + pk_names, primary_keys = self.get_primary_keys(default_conn) sql_filter = self.get_filter() @@ -362,13 +377,16 @@ class TableCommand(GridCommand): return sql - def get_primary_keys(self): + def get_primary_keys(self, default_conn=None): """ This function is used to fetch the primary key columns. """ driver = get_driver(PG_DEFAULT_DRIVER) - manager = driver.connection_manager(self.sid) - conn = manager.connection(did=self.did, conn_id=self.conn_id) + if default_conn is None: + manager = driver.connection_manager(self.sid) + conn = manager.connection(did=self.did, conn_id=self.conn_id) + else: + conn = default_conn pk_names = '' primary_keys = OrderedDict() @@ -400,7 +418,11 @@ class TableCommand(GridCommand): def can_filter(self): return True - def save(self, changed_data): + def save(self, + changed_data, + columns_info, + client_primary_key='__temp_PK', + default_conn=None): """ This function is used to save the data into the database. Depending on condition it will either update or insert the @@ -408,10 +430,16 @@ class TableCommand(GridCommand): Args: changed_data: Contains data to be saved + columns_info: + default_conn: + client_primary_key: """ - - manager = get_driver(PG_DEFAULT_DRIVER).connection_manager(self.sid) - conn = manager.connection(did=self.did, conn_id=self.conn_id) + driver = get_driver(PG_DEFAULT_DRIVER) + if default_conn is None: + manager = driver.connection_manager(self.sid) + conn = manager.connection(did=self.did, conn_id=self.conn_id) + else: + conn = default_conn status = False res = None @@ -421,14 +449,6 @@ class TableCommand(GridCommand): list_of_sql = [] _rowid = None - # Replace column positions with names - def set_column_names(data): - new_data = {} - for key in data: - new_data[changed_data['columns'][int(key)]['name']] = data[key] - - return new_data - if conn.connected(): # Start the transaction @@ -454,28 +474,26 @@ class TableCommand(GridCommand): column_type = {} pk_names, primary_keys = self.get_primary_keys() - for each_col in self.columns_info: + for each_col in columns_info: if ( - self.columns_info[each_col]['not_null'] and - not self.columns_info[each_col][ + columns_info[each_col]['not_null'] and + not columns_info[each_col][ 'has_default_val'] ): column_data[each_col] = None column_type[each_col] =\ - self.columns_info[each_col]['type_name'] + columns_info[each_col]['type_name'] else: column_type[each_col] = \ - self.columns_info[each_col]['type_name'] - + columns_info[each_col]['type_name'] for each_row in changed_data[of_type]: data = changed_data[of_type][each_row]['data'] # Remove our unique tracking key - data.pop('__temp_PK', None) + data.pop(client_primary_key, None) data.pop('is_row_copied', None) - data = set_column_names(data) - data_type = set_column_names(changed_data[of_type][each_row]['data_type']) - list_of_rowid.append(data.get('__temp_PK')) + data_type = changed_data[of_type][each_row]['data_type'] + list_of_rowid.append(data.get(client_primary_key)) # Update columns value and data type # with columns having not_null=False and has @@ -497,9 +515,9 @@ class TableCommand(GridCommand): # For updated rows elif of_type == 'updated': for each_row in changed_data[of_type]: - data = set_column_names(changed_data[of_type][each_row]['data']) - pk = set_column_names(changed_data[of_type][each_row]['primary_keys']) - data_type = set_column_names(changed_data[of_type][each_row]['data_type']) + data = changed_data[of_type][each_row]['data'] + pk = changed_data[of_type][each_row]['primary_keys'] + data_type = changed_data[of_type][each_row]['data_type'] sql = render_template("/".join([self.sql_path, 'update.sql']), data_to_be_saved=data, primary_keys=pk, @@ -519,18 +537,19 @@ class TableCommand(GridCommand): rows_to_delete.append(changed_data[of_type][each_row]) # Fetch the keys for SQL generation if is_first: - # We need to covert dict_keys to normal list in Python3 - # In Python2, it's already a list & We will also fetch column names using index - keys = [ - changed_data['columns'][int(k)]['name'] - for k in list(changed_data[of_type][each_row].keys()) - ] + # We need to covert dict_keys to normal list in + # Python3 + # In Python2, it's already a list & We will also + # fetch column names using index + keys = list(changed_data[of_type][each_row].keys()) + no_of_keys = len(keys) is_first = False # Map index with column name for each row for row in rows_to_delete: for k, v in row.items(): - # Set primary key with label & delete index based mapped key + # Set primary key with label & delete index based + # mapped key try: row[changed_data['columns'][int(k)]['name']] = v except ValueError: @@ -597,7 +616,7 @@ class ViewCommand(GridCommand): # call base class init to fetch the table name super(ViewCommand, self).__init__(**kwargs) - def get_sql(self): + def get_sql(self, default_conn=None): """ This method is used to create a proper SQL query to fetch the data for the specified view @@ -652,7 +671,7 @@ class ForeignTableCommand(GridCommand): # call base class init to fetch the table name super(ForeignTableCommand, self).__init__(**kwargs) - def get_sql(self): + def get_sql(self, default_conn=None): """ This method is used to create a proper SQL query to fetch the data for the specified foreign table @@ -697,7 +716,7 @@ class CatalogCommand(GridCommand): # call base class init to fetch the table name super(CatalogCommand, self).__init__(**kwargs) - def get_sql(self): + def get_sql(self, default_conn=None): """ This method is used to create a proper SQL query to fetch the data for the specified catalog object @@ -722,7 +741,7 @@ class CatalogCommand(GridCommand): return True -class QueryToolCommand(BaseCommand): +class QueryToolCommand(BaseCommand, FetchedRowTracker): """ class QueryToolCommand(BaseCommand) @@ -732,13 +751,15 @@ class QueryToolCommand(BaseCommand): def __init__(self, **kwargs): # call base class init to fetch the table name - super(QueryToolCommand, self).__init__(**kwargs) + + BaseCommand.__init__(self, **kwargs) + FetchedRowTracker.__init__(self, **kwargs) self.conn_id = None self.auto_rollback = False self.auto_commit = True - def get_sql(self): + def get_sql(self, default_conn=None): return None def can_edit(self): diff --git a/web/pgadmin/tools/sqleditor/static/css/sqleditor.css b/web/pgadmin/tools/sqleditor/static/css/sqleditor.css index 7266b34..8ed5033 100644 --- a/web/pgadmin/tools/sqleditor/static/css/sqleditor.css +++ b/web/pgadmin/tools/sqleditor/static/css/sqleditor.css @@ -460,6 +460,11 @@ input.editor-checkbox:focus { background-color: #2C76B4; } +.slick-cell span[data-cell-type="row-header-selector"] { + display: block; + text-align: right; +} + #datagrid div.slick-header.ui-state-default { background: #ffffff; border-bottom: none; diff --git a/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js b/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js index b9f017e..5b6bbcc 100644 --- a/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js +++ b/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js @@ -33,7 +33,8 @@ define([ 'slickgrid/plugins/slick.cellselectionmodel', 'slickgrid/plugins/slick.cellcopymanager', 'slickgrid/plugins/slick.rowselectionmodel', - 'slickgrid/slick.grid' + 'slickgrid/slick.grid', + 'slickgrid/slick.dataview' ], function( $, _, S, alertify, pgAdmin, Backbone, Backgrid, CodeMirror, pgExplain, GridSelector, ActiveCellCapture, clipboard, copyData, RangeSelectionHelper, handleQueryOutputKeyboardEvent, @@ -470,7 +471,7 @@ define([ - staged_rows: This will hold all the data which user copies/pastes/deletes in grid - deleted: - This will hold all the data which user delets in grid + This will hold all the data which user deletes in grid Events handling: ---------------- @@ -486,34 +487,10 @@ define([ - We are using this event for Copy operation on grid */ - // Get the item column value using a custom 'fieldIdx' column param - get_item_column_value: function (item, column) { - if (column.pos !== undefined) { - return item[column.pos]; - } else { - return null; - } - }, - // This function is responsible to create and render the SlickGrid. - render_grid: function(collection, columns, is_editable) { + render_grid: function(collection, columns, is_editable, client_primary_key, rows_affected) { var self = this; - // returns primary keys - self.handler.get_row_primary_key = function() { - var self = this, - tmp_keys = []; - _.each(self.primary_keys, function(p, idx) { - // For each columns search primary key position - _.each(self.columns, function(c) { - if(c.name == idx) { - tmp_keys.push(c.pos); - } - }); - }); - return tmp_keys; - }; - // This will work as data store and holds all the // inserted/updated/deleted data from grid self.handler.data_store = { @@ -528,8 +505,9 @@ define([ // To store primary keys before they gets changed self.handler.primary_keys_data = {}; - // Add getItemMetadata into handler for later use - self.handler.data_view = collection; + self.client_primary_key = client_primary_key; + + self.client_primary_key_counter = 0; // Remove any existing grid first if (self.handler.slickgrid) { @@ -607,7 +585,12 @@ define([ }); var gridSelector = new GridSelector(); - grid_columns = gridSelector.getColumnDefinitions(grid_columns); + grid_columns = self.grid_columns = gridSelector.getColumnDefinitions(grid_columns); + + if (rows_affected) { + // calculate with for header row column. + grid_columns[0]['width'] = SqlEditorUtils.calculateColumnWidth(rows_affected); + } var grid_options = { editable: true, @@ -615,8 +598,7 @@ define([ enableCellNavigation: true, enableColumnReorder: false, asyncEditorLoading: false, - autoEdit: false, - dataItemColumnValueExtractor: this.get_item_column_value + autoEdit: false }; var $data_grid = self.$el.find('#datagrid'); @@ -624,17 +606,15 @@ define([ var grid_height = $($('#editor-panel').find('.wcFrame')[1]).height() - 35; $data_grid.height(grid_height); - // Add our own custom primary key to keep track of changes - _.each(collection, function(row){ - row['__temp_PK'] = SqlEditorUtils.epicRandomString(15); - }); + var dataView = self.dataView = new Slick.Data.DataView(), + grid = self.grid = new Slick.Grid($data_grid, dataView, grid_columns, grid_options); // Add-on function which allow us to identify the faulty row after insert/update // and apply css accordingly - collection.getItemMetadata = function(i) { - var res = {}, - cssClass = '', - data_store = self.handler.data_store; + + dataView.getItemMetadata = function(i) { + var res = {}, cssClass = '', + data_store = self.handler.data_store; if (_.has(self.handler, 'data_store')) { if (i in data_store.added_index && @@ -657,9 +637,8 @@ define([ cssClass += ' disabled_row'; } return {'cssClasses': cssClass}; - } + }; - var grid = new Slick.Grid($data_grid, collection, grid_columns, grid_options); grid.registerPlugin( new Slick.AutoTooltips({ enableForHeaderCells: false }) ); grid.registerPlugin(new ActiveCellCapture()); grid.setSelectionModel(new XCellSelectionModel()); @@ -671,7 +650,8 @@ define([ columns: columns, grid: grid, selection: grid.getSelectionModel(), - editor: self + editor: self, + client_primary_key: self.client_primary_key }; self.handler.slickgrid = grid; @@ -690,6 +670,42 @@ define([ }); }); + gridSelector.onBeforeGridSelectAll.subscribe(function(e, args) { + if (self.handler.has_more_rows) { + // this will prevent selection un-till we load all data + e.stopImmediatePropagation(); + self.fetch_next_all(function() { + // since we've stopped event propagation we need to + // trigger onGridSelectAll manually with new event data. + gridSelector.onGridSelectAll.notify(args, new Slick.EventData()); + }); + } + }); + + gridSelector.onBeforeGridColumnSelectAll.subscribe(function(e, args) { + if (self.handler.has_more_rows) { + // this will prevent selection un-till we load all data + e.stopImmediatePropagation(); + self.fetch_next_all(function() { + // since we've stopped event propagation we need to + // trigger onGridColumnSelectAll manually with new event data. + gridSelector.onGridColumnSelectAll.notify(args, new Slick.EventData()); + }); + } + }); + + // listen for row count change. + dataView.onRowCountChanged.subscribe(function (e, args) { + grid.updateRowCount(); + grid.render(); + }); + + // listen for rows change. + dataView.onRowsChanged.subscribe(function (e, args) { + grid.invalidateRows(args.rows); + grid.render(); + }); + // Listener function which will be called before user updates existing cell // This will be used to collect primary key for that row grid.onBeforeEditCell.subscribe(function (e, args) { @@ -702,8 +718,8 @@ define([ return false; } - if(self.handler.can_edit && before_data && '__temp_PK' in before_data) { - var _pk = before_data.__temp_PK, + if(self.handler.can_edit && before_data && self.client_primary_key in before_data) { + var _pk = before_data[self.client_primary_key], _keys = self.handler.primary_keys, current_pk = {}, each_pk_key = {}; @@ -715,22 +731,34 @@ define([ // Fetch primary keys for the row before they gets modified var _columns = self.handler.columns; _.each(_keys, function(value, key) { - pos = _.where(_columns, {name: key})[0]['pos'] - current_pk[pos] = before_data[pos]; + current_pk[key] = before_data[key]; }); // Place it in main variable for later use self.handler.primary_keys_data[_pk] = current_pk } }); - grid.onKeyDown.subscribe(handleQueryOutputKeyboardEvent); + grid.onKeyDown.subscribe(function(event, args) { + var KEY_A = 65; + var modifiedKey = event.keyCode; + var isModifierDown = event.ctrlKey || event.metaKey; + // Intercept Ctrl/Cmd + A key board event. + // As we might want to load all rows before selecting all. + if (isModifierDown && modifiedKey == KEY_A && self.handler.has_more_rows) { + self.fetch_next_all(function() { + handleQueryOutputKeyboardEvent(event, args); + }); + } else { + handleQueryOutputKeyboardEvent(event, args); + } + }); // Listener function which will be called when user updates existing rows grid.onCellChange.subscribe(function (e, args) { // self.handler.data_store.updated will holds all the updated data - var changed_column = args.grid.getColumns()[args.cell].pos, // Current field pos + var changed_column = args.grid.getColumns()[args.cell].field, updated_data = args.item[changed_column], // New value for current field - _pk = args.item.__temp_PK || null, // Unique key to identify row + _pk = args.item[self.client_primary_key] || null, // Unique key to identify row column_data = {}, _type; @@ -740,11 +768,16 @@ define([ // so that cell edit is enabled for that row. var grid = args.grid, row_data = grid.getDataItem(args.row), - p_keys_list = _.pick( - row_data, self.handler.get_row_primary_key() - ), - is_primary_key = Object.keys(p_keys_list).length ? - p_keys_list[0] : undefined; + is_primary_key = _.all( + _.values( + _.pick( + row_data, self.primary_keys + ) + ), + function(val) { + return val != undefined + } + ); // temp_new_rows is available only for view data. if (is_primary_key && self.handler.temp_new_rows) { @@ -764,7 +797,7 @@ define([ column_data); //Find type for current column self.handler.data_store.added[_pk]['err'] = false - self.handler.data_store.added[_pk]['data_type'][changed_column] = _.where(this.columns, {pos: changed_column})[0]['type']; + self.handler.data_store.added[_pk]['data_type'][changed_column] = _.where(this.columns, {name: changed_column})[0]['type']; // Check if it is updated data from existing rows? } else if(_pk in self.handler.data_store.updated) { _.extend( @@ -774,7 +807,7 @@ define([ self.handler.data_store.updated[_pk]['err'] = false //Find type for current column - self.handler.data_store.updated[_pk]['data_type'][changed_column] = _.where(this.columns, {pos: changed_column})[0]['type']; + self.handler.data_store.updated[_pk]['data_type'][changed_column] = _.where(this.columns, {name: changed_column})[0]['type']; } else { // First updated data for this primary key self.handler.data_store.updated[_pk] = { @@ -784,7 +817,7 @@ define([ self.handler.data_store.updated_index[args.row] = _pk; // Find & add column data type for current changed column var temp = {}; - temp[changed_column] = _.where(this.columns, {pos: changed_column})[0]['type']; + temp[changed_column] = _.where(this.columns, {name: changed_column})[0]['type']; self.handler.data_store.updated[_pk]['data_type'] = temp; } } @@ -803,11 +836,10 @@ define([ // Listener function which will be called when user adds new rows grid.onAddNewRow.subscribe(function (e, args) { // self.handler.data_store.added will holds all the newly added rows/data - var _key = SqlEditorUtils.epicRandomString(10), - column = args.column, - item = args.item, - data_length = this.grid.getDataLength(), - new_collection = args.grid.getData(); + var column = args.column, + item = args.item, data_length = this.grid.getDataLength(), + _key = (self.client_primary_key_counter++).toString(), + dataView = this.grid.getData(); // Add new row in list to keep track of it if (_.isUndefined(item[0])) { @@ -816,17 +848,16 @@ define([ // If copied item has already primary key, use it. if(item) { - item.__temp_PK = _key; + item[self.client_primary_key] = _key; } - new_collection.push(item); + dataView.addItem(item); self.handler.data_store.added[_key] = {'err': false, 'data': item}; self.handler.data_store.added_index[data_length] = _key; // Fetch data type & add it for the column var temp = {}; - temp[column.pos] = _.where(this.columns, {pos: column.pos})[0]['type']; + temp[column.name] = _.where(this.columns, {pos: column.pos})[0]['type']; self.handler.data_store.added[_key]['data_type'] = temp; - grid.invalidateRows([new_collection.length - 1]); grid.updateRowCount(); grid.render(); @@ -839,6 +870,16 @@ define([ $("#btn-save").prop('disabled', false); }.bind(editor_data)); + // Listen grid viewportChanged event to load next chunk of data. + grid.onViewportChanged.subscribe(function(e, args) { + var rendered_range = args.grid.getRenderedRange(), + data_len = args.grid.getDataLength(); + // start fetching next batch of records before reaching to bottom. + if (self.handler.has_more_rows && !self.handler.fetching_rows && rendered_range.bottom > data_len - 100) { + // fetch asynchronous + setTimeout(self.fetch_next.bind(self)); + } + }) // Resize SlickGrid when window resize $( window ).resize( function() { // Resize grid only when 'Data Output' panel is visible. @@ -861,6 +902,84 @@ define([ if(self.data_output_panel.isVisible()) self.grid_resize(grid); }); + + for (var i = 0; i < collection.length; i++) { + // Convert to dict from 2darray + var item = {}; + for (var j = 1; j < grid_columns.length; j++) { + item[grid_columns[j]['field']] = collection[i][grid_columns[j]['pos']] + } + + item[self.client_primary_key] = (self.client_primary_key_counter++).toString(); + collection[i] = item; + } + dataView.setItems(collection, self.client_primary_key); + }, + fetch_next_all(cb) { + this.fetch_next(true, cb); + }, + fetch_next: function(fetch_all, cb) { + var self = this; + + // This will prevent fetch operation if previous fetch operation is + // already in progress. + self.fetching_rows = true; + + $("#btn-flash").prop('disabled', true); + + if (fetch_all) { + self.handler.trigger( + 'pgadmin-sqleditor:loading-icon:show', + gettext('Fetching all records...') + ); + } + + $.ajax({ + url: "{{ url_for('sqleditor.index') }}" + "fetch/" + self.transId + (fetch_all ? "/1": ""), + method: 'GET', + success: function(res) { + self.handler.has_more_rows = res.data.has_more_rows; + $("#btn-flash").prop('disabled', false); + self.handler.trigger('pgadmin-sqleditor:loading-icon:hide'); + self.update_grid_data(res.data.result); + if (typeof cb == "function") { + cb(); + } + }, + error: function(e) { + $("#btn-flash").prop('disabled', false); + self.handler.trigger('pgadmin-sqleditor:loading-icon:hide'); + self.handler.has_more_rows = false; + self.handler.fetching_rows = false; + if (typeof cb == "function") { + cb(); + } + if (e.readyState == 0) { + self.update_msg_history(false, + gettext('Not connected to the server or the connection to the server has been closed.') + ); + return; + } + } + }); + }, + + update_grid_data: function(data) { + this.dataView.beginUpdate(); + + for (var i = 0; i < data.length; i++) { + // Convert 2darray to dict. + var item = {}; + for (var j = 1; j < this.grid_columns.length; j++) { + item[this.grid_columns[j]['field']] = data[i][this.grid_columns[j]['pos']] + } + + item[this.client_primary_key] = (this.client_primary_key_counter++).toString(); + this.dataView.addItem(item); + } + + this.dataView.endUpdate(); + this.handler.fetching_rows = false; }, /* This function is responsible to render output grid */ @@ -1402,6 +1521,8 @@ define([ self.explain_buffers = false; self.explain_timing = false; self.is_new_browser_tab = is_new_browser_tab; + self.has_more_rows = false; + self.fetching_rows = false; // We do not allow to call the start multiple times. if (self.gridView) @@ -1509,6 +1630,8 @@ define([ self.rows_to_disable = new Array(); // Temporarily hold new rows added self.temp_new_rows = new Array(); + self.has_more_rows = false; + self.fetching_rows = false; self.trigger( 'pgadmin-sqleditor:loading-icon:show', @@ -1587,45 +1710,14 @@ define([ }); }, - // This function makes the ajax call to fetch columns for last async query, - get_columns: function(poll_result) { - var self = this; - // Check the flag and decide if we need to fetch columns from server - // or use the columns data stored locally from previous call? - if (self.FETCH_COLUMNS_FROM_SERVER) { - $.ajax({ - url: "{{ url_for('sqleditor.index') }}" + "columns/" + self.transId, - method: 'GET', - success: function(res) { - poll_result.colinfo = res.data.columns; - poll_result.primary_keys = res.data.primary_keys; - self.call_render_after_poll(poll_result); - // Set a flag to get columns to false & set the value for future use - self.FETCH_COLUMNS_FROM_SERVER = false; - self.COLUMNS_DATA = res; - }, - error: function(e) { - var msg = e.responseText; - if (e.responseJSON != undefined && e.responseJSON.errormsg != undefined) - msg = e.responseJSON.errormsg; - alertify.error(msg, 5); - } - }); - } else { - // Use the previously saved columns data - poll_result.colinfo = self.COLUMNS_DATA.data.columns; - poll_result.primary_keys = self.COLUMNS_DATA.data.primary_keys; - self.call_render_after_poll(poll_result); - } - }, - // This is a wrapper to call _render function // We need this because we have separated columns route & result route // We need to combine both result here in wrapper before rendering grid call_render_after_poll: function(res) { var self = this; self.query_end_time = new Date(); - self.rows_affected = res.rows_affected; + self.rows_affected = res.rows_affected, + self.has_more_rows = res.has_more_rows; /* If no column information is available it means query runs successfully with no result to display. In this @@ -1674,7 +1766,8 @@ define([ 'pgadmin-sqleditor:loading-icon:message', gettext("Loading data from the database server and rendering...") ); - self.get_columns(res.data); + + self.call_render_after_poll(res.data); } else if (res.data.status === 'Busy') { // If status is Busy then poll the result by recursive call to the poll function @@ -1734,6 +1827,7 @@ define([ var self = this; self.colinfo = data.col_info; self.primary_keys = data.primary_keys; + self.client_primary_key = data.client_primary_key; self.cell_selected = false; self.selected_model = null; self.changedModels = []; @@ -1786,7 +1880,7 @@ define([ self.total_time = self.get_query_run_time(self.query_start_time, self.query_end_time); self.update_msg_history(true, "", false); var msg1 = S(gettext("Total query runtime: %s.")).sprintf(self.total_time).value(); - var msg2 = S(gettext("%s rows retrieved.")).sprintf(self.rows_affected).value(); + var msg2 = S(gettext("%s rows affected.")).sprintf(self.rows_affected).value(); // Display the notifier if the timeout is set to >= 0 if (self.info_notifier_timeout >= 0) { @@ -1822,7 +1916,8 @@ define([ setTimeout( function() { self.gridView.render_grid( - explain_data_array, self.columns, self.can_edit + explain_data_array, self.columns, self.can_edit, + self.client_primary_key ); // Make sure - the 'Explain' panel is visible, before - we // start rendering the grid. @@ -1838,7 +1933,8 @@ define([ self.gridView.data_output_panel.focus(); setTimeout( function() { - self.gridView.render_grid(data.result, self.columns, self.can_edit); + self.gridView.render_grid(data.result, self.columns, + self.can_edit, self.client_primary_key, data.rows_affected); }, 10 ); } @@ -1846,136 +1942,114 @@ define([ // Hide the loading icon self.trigger('pgadmin-sqleditor:loading-icon:hide'); $("#btn-flash").prop('disabled', false); - }.bind(self), - function() { - this.trigger('pgadmin-sqleditor:loading-icon:hide'); - $("#btn-flash").prop('disabled', false); }.bind(self) ); }, // This function creates the columns as required by the backgrid - _fetch_column_metadata: function(data, cb, _fail) { + _fetch_column_metadata: function(data, cb) { var colinfo = data.colinfo, primary_keys = data.primary_keys, result = data.result, columns = [], self = this; + // Store pg_types in an array + var pg_types = new Array(); + _.each(data.types, function(r) { + pg_types[r.oid] = [r.typname]; + }); - self.trigger( - 'pgadmin-sqleditor:loading-icon:message', - gettext("Retrieving information about the columns returned...") - ); - - // Make ajax call to fetch the pg types to map numeric data type - $.ajax({ - url: "{{ url_for('sqleditor.index') }}" + "fetch/types/" + self.transId, - method: 'GET', - success: function(res) { - if (res.data.status) { - // Store pg_types in an array - var pg_types = new Array(); - _.each(res.data.result.rows, function(r) { - pg_types[r.oid] = [r.typname]; - }); - - // Create columns required by backgrid to render - _.each(colinfo, function(c) { - var is_primary_key = false; + // Create columns required by slick grid to render + _.each(colinfo, function(c) { + var is_primary_key = false; - // Check whether table have primary key - if (_.size(primary_keys) > 0) { - _.each(primary_keys, function (value, key) { - if (key === c.name) - is_primary_key = true; - }); - } + // Check whether table have primary key + if (_.size(primary_keys) > 0) { + _.each(primary_keys, function (value, key) { + if (key === c.name) + is_primary_key = true; + }); + } - // To show column label and data type in multiline, - // The elements should be put inside the div. - // Create column label and type. - var col_type = column_label = ''; - var type = pg_types[c.type_code] ? - pg_types[c.type_code][0] : - // This is the case where user might - // have use casting so we will use type - // returned by cast function - pg_types[pg_types.length - 1][0] ? - pg_types[pg_types.length - 1][0] : 'unknown'; - - if (!is_primary_key) - col_type += ' ' + type; - else - col_type += ' [PK] ' + type; + // To show column label and data type in multiline, + // The elements should be put inside the div. + // Create column label and type. + var col_type = column_label = ''; + var type = pg_types[c.type_code] ? + pg_types[c.type_code][0] : + // This is the case where user might + // have use casting so we will use type + // returned by cast function + pg_types[pg_types.length - 1][0] ? + pg_types[pg_types.length - 1][0] : 'unknown'; + + if (!is_primary_key) + col_type += ' ' + type; + else + col_type += ' [PK] ' + type; - if (c.precision && c.precision >= 0 && c.precision != 65535) { - col_type += ' (' + c.precision; - col_type += c.scale && c.scale != 65535 ? - ',' + c.scale + ')': - ')'; - } + if (c.precision && c.precision >= 0 && c.precision != 65535) { + col_type += ' (' + c.precision; + col_type += c.scale && c.scale != 65535 ? + ',' + c.scale + ')': + ')'; + } - // Identify cell type of column. - switch(type) { - case "json": - case "json[]": - case "jsonb": - case "jsonb[]": - col_cell = 'Json'; - break; - case "smallint": - case "integer": - case "bigint": - case "decimal": - case "numeric": - case "real": - case "double precision": - col_cell = 'number'; - break; - case "boolean": - col_cell = 'boolean'; - break; - case "character": - case "character[]": - case "character varying": - case "character varying[]": - if (c.internal_size && c.internal_size >= 0 && c.internal_size != 65535) { - // Update column type to display length on column header - col_type += ' (' + c.internal_size + ')'; - } - col_cell = 'string'; - break; - default: - col_cell = 'string'; - } + // Identify cell type of column. + switch(type) { + case "json": + case "json[]": + case "jsonb": + case "jsonb[]": + col_cell = 'Json'; + break; + case "smallint": + case "integer": + case "bigint": + case "decimal": + case "numeric": + case "real": + case "double precision": + col_cell = 'number'; + break; + case "boolean": + col_cell = 'boolean'; + break; + case "character": + case "character[]": + case "character varying": + case "character varying[]": + if (c.internal_size && c.internal_size >= 0 && c.internal_size != 65535) { + // Update column type to display length on column header + col_type += ' (' + c.internal_size + ')'; + } + col_cell = 'string'; + break; + default: + col_cell = 'string'; + } - column_label = c.display_name + '<br>' + col_type; - - var col = { - 'name': c.name, - 'pos': c.pos, - 'label': column_label, - 'cell': col_cell, - 'can_edit': self.can_edit, - 'type': type, - 'display_name': c.display_name, - 'column_type': col_type, - 'not_null': c.not_null, - 'has_default_val': c.has_default_val - }; - columns.push(col); - }); - } - else { - alertify.alert('Fetching Type Error', res.data.result); - } - self.columns = columns; - if (cb && typeof(cb) == 'function') { - cb(); - } - }, - fail: _fail + column_label = c.display_name + '<br>' + col_type; + + var col = { + 'name': c.name, + 'display_name': c.display_name, + 'column_type': col_type, + 'pos': c.pos, + 'label': column_label, + 'cell': col_cell, + 'can_edit': self.can_edit, + 'type': type, + 'not_null': c.not_null, + 'has_default_val': c.has_default_val + }; + columns.push(col); }); + + self.columns = columns; + if (cb && typeof(cb) == 'function') { + cb(); + } }, // This function is used to raise appropriate message. @@ -1986,22 +2060,21 @@ define([ self.gridView.messages_panel.focus(); - if (self.is_query_tool) { - if (clear_grid) { - // Delete grid - if (self.gridView.handler.slickgrid) { - self.gridView.handler.slickgrid.destroy(); + if (clear_grid) { + // Delete grid + if (self.gridView.handler.slickgrid) { + self.gridView.handler.slickgrid.destroy(); - } - // Misc cleaning - self.columns = undefined; - self.collection = undefined; - - $('.sql-editor-message').text(msg); - } else { - $('.sql-editor-message').append(msg); } + // Misc cleaning + self.columns = undefined; + self.collection = undefined; + + $('.sql-editor-message').text(msg); + } else { + $('.sql-editor-message').append(msg); } + // Scroll automatically when msgs appends to element setTimeout(function(){ $(".sql-editor-message").scrollTop($(".sql-editor-message")[0].scrollHeight);; @@ -2055,7 +2128,7 @@ define([ rows_to_delete: function(data) { var self = this, - tmp_keys = self.get_row_primary_key.call(self); + tmp_keys = self.primary_keys; // re-calculate rows with no primary keys self.temp_new_rows = []; @@ -2068,7 +2141,6 @@ define([ self.temp_new_rows.push(idx); } }); - data.getItemMetadata = self.data_view.getItemMetadata; self.rows_to_disable = _.clone(self.temp_new_rows); }, @@ -2079,69 +2151,73 @@ define([ is_added = _.size(self.data_store.added), is_updated = _.size(self.data_store.updated); - // Remove newly added rows from staged rows as we don't want to send them on server - if(is_added) { - _.each(self.data_store.added, function(val, key) { - if(key in self.data_store.staged_rows) { - // Remove the row from data store so that we do not send it on server - deleted_keys.push(key); - delete self.data_store.staged_rows[key]; - delete self.data_store.added[key] - } - }); + // Remove newly added rows from staged rows as we don't want to send them on server + if(is_added) { + _.each(self.data_store.added, function(val, key) { + if(key in self.data_store.staged_rows) { + // Remove the row from data store so that we do not send it on server + deleted_keys.push(key); + delete self.data_store.staged_rows[key]; + delete self.data_store.added[key]; + delete self.data_store.added_index[key]; } - - // If only newly rows to delete and no data is there to send on server - // then just re-render the grid - if(_.size(self.data_store.staged_rows) == 0) { - var grid = self.slickgrid, data = grid.getData(), idx = 0; - if(deleted_keys.length){ - // Remove new rows from grid data using deleted keys - data = _.reject(data, function(d){ - return (d && _.indexOf(deleted_keys, d.__temp_PK) > -1) - }); - } - self.rows_to_delete.apply(self, [data]); - grid.resetActiveCell(); - grid.setData(data, true); - grid.setSelectedRows([]); - grid.invalidate(); - // Nothing to copy or delete here - $("#btn-delete-row").prop('disabled', true); - $("#btn-copy-row").prop('disabled', true); - if(_.size(self.data_store.added) || is_updated) { - // Do not disable save button if there are - // any other changes present in grid data - $("#btn-save").prop('disabled', false); - } else { - $("#btn-save").prop('disabled', true); - } - alertify.success(gettext("Row(s) deleted")); + }); + } + // If only newly rows to delete and no data is there to send on server + // then just re-render the grid + if(_.size(self.data_store.staged_rows) == 0) { + var grid = self.slickgrid, + dataView = grid.getData(), + data = dataView.getItems(), + idx = 0; + + grid.resetActiveCell(); + + dataView.beginUpdate(); + for (var i = 0; i < deleted_keys.length; i++) { + dataView.deleteItem(deleted_keys[i]); + } + dataView.endUpdate(); + self.rows_to_delete.apply(self, [dataView.getItems()]); + grid.resetActiveCell(); + grid.setSelectedRows([]); + grid.invalidate(); + + // Nothing to copy or delete here + $("#btn-delete-row").prop('disabled', true); + $("#btn-copy-row").prop('disabled', true); + if(_.size(self.data_store.added) || is_updated) { + // Do not disable save button if there are + // any other changes present in grid data + $("#btn-save").prop('disabled', false); } else { - // There are other data to needs to be updated on server - if(is_updated) { - alertify.alert(gettext("Operation failed"), + $("#btn-save").prop('disabled', true); + } + alertify.success(gettext("Row(s) deleted")); + } else { + // There are other data to needs to be updated on server + if(is_updated) { + alertify.alert(gettext("Operation failed"), gettext("There are unsaved changes in grid, Please save them first to avoid inconsistency in data") ); - return; - } - alertify.confirm(gettext("Delete Row(s)"), + return; + } + alertify.confirm(gettext("Delete Row(s)"), gettext("Are you sure you wish to delete selected row(s)?"), - function() { - $("#btn-delete-row").prop('disabled', true); - $("#btn-copy-row").prop('disabled', true); - // Change the state - self.data_store.deleted = self.data_store.staged_rows; - self.data_store.staged_rows = {}; - // Save the changes on server - self._save(); - }, - function() { - // Do nothing as user canceled the operation. - } - ).set('labels', {ok:'Yes', cancel:'No'}); + function() { + $("#btn-delete-row").prop('disabled', true); + $("#btn-copy-row").prop('disabled', true); + // Change the state + self.data_store.deleted = self.data_store.staged_rows; + self.data_store.staged_rows = {}; + // Save the changes on server + self._save(); + }, + function() { + // Do nothing as user canceled the operation. } - + ).set('labels', {ok: gettext("Yes"), cancel:gettext("No")}); + } }, /* This function will fetch the list of changed models and make @@ -2203,7 +2279,9 @@ define([ data: JSON.stringify(req_data), success: function(res) { var grid = self.slickgrid, - data = grid.getData(); + dataView = grid.getData(), + data_length = dataView.getLength(), + data = []; if (res.data.status) { // Remove flag is_row_copied from copied rows _.each(data, function(row, idx) { @@ -2220,22 +2298,20 @@ define([ // Remove deleted rows from client as well if(is_deleted) { var rows = grid.getSelectedRows(); - /* In JavaScript sorting by default is lexical, - * To make sorting numerical we need to pass function - * After that we will Reverse the order of sorted array - * so that when we remove it does not affect array index - */ - if(data.length == rows.length) { + if(data_length == rows.length) { // This means all the rows are selected, clear all data data = []; + dataView.setItems(data, self.client_primary_key); } else { - rows = rows.sort(function(a,b){return a - b}).reverse(); - rows.forEach(function(idx) { - data.splice(idx, 1); - }); + dataView.beginUpdate(); + for (var i = 0; i < rows.length; i++) { + item = grid.getDataItem(rows[i]); + data.push(item); + dataView.deleteItem(item[self.client_primary_key]); + } + dataView.endUpdate(); } self.rows_to_delete.apply(self, [data]); - grid.setData(data, true); grid.setSelectedRows([]); } @@ -2248,6 +2324,7 @@ define([ self.rows_to_disable = _.clone(self.temp_new_rows); } + grid.setSelectedRows([]); // Reset data store self.data_store = { 'added': {}, @@ -2268,7 +2345,7 @@ define([ $('.sql-editor-message').text(res.data.result); var err_msg = S(gettext("%s.")).sprintf(res.data.result).value(); alertify.notify(err_msg, 'error', 20); - + grid.setSelectedRows([]); // To highlight the row at fault if(_.has(res.data, '_rowid') && (!_.isUndefined(res.data._rowid)|| !_.isNull(res.data._rowid))) { @@ -2324,14 +2401,21 @@ define([ // Find index of row at fault from grid data _find_rowindex: function(rowid) { - var self = this; - var grid = self.slickgrid, - data = grid.getData(), _rowid, count = 0, _idx = -1; + var self = this, + grid = self.slickgrid, + dataView = grid.getData(), + data = dataView.getItems(), + _rowid, + count = 0, + _idx = -1; + // If _rowid is object then it's update/delete operation if(_.isObject(rowid)) { _rowid = rowid; - } else if (_.isString(rowid)) { // Insert opration - _rowid = { '__temp_PK': rowid }; + } else if (_.isString(rowid)) { // Insert operation + var rowid = {}; + rowid[self.client_primary_key]= rowid; + _rowid = rowid; } else { // Something is wrong with unique id return _idx; @@ -2553,11 +2637,6 @@ define([ // This function will set the required flag for polling response data _init_polling_flags: function() { var self = this; - // Set a flag to get columns - self.FETCH_COLUMNS_FROM_SERVER = true; - // We will set columns data in this variable for future use once we fetch it - // from server - self.COLUMNS_DATA = {}; // To get a timeout for polling fallback timer in seconds in // regards to elapsed time @@ -2653,7 +2732,7 @@ define([ return; // Add column position and it's value to data - data[column_info.field] = _values[column_info.pos] || ''; + data[column_info.field] = _values[column_info.field] || ''; self.trigger( 'pgadmin-sqleditor:loading-icon:show', @@ -2723,7 +2802,7 @@ define([ return; // Add column position and it's value to data - data[column_info.field] = _values[column_info.pos] || ''; + data[column_info.field] = _values[column_info.field] || ''; self.trigger( 'pgadmin-sqleditor:loading-icon:show', @@ -2886,16 +2965,18 @@ define([ _paste_row: function() { var self = this, col_info = {}, grid = self.slickgrid, - data = grid.getData(), - count = Object.keys(data).length-1; - - var rows = grid.getSelectedRows().sort( + dataView = grid.getData(), + data = dataView.getItems(), + count = dataView.getLength(), + rows = grid.getSelectedRows().sort( function (a, b) { return a - b; } ), - rows = rows.length == 0 ? self.last_copied_rows : rows, copied_rows = rows.map(function (rowIndex) { return data[rowIndex]; }); + + rows = rows.length == 0 ? self.last_copied_rows : rows + self.last_copied_rows = rows; // If there are rows to paste? @@ -3015,6 +3096,9 @@ define([ sql = '', history_msg = ''; + self.has_more_rows = false; + self.fetching_rows = false; + /* If code is selected in the code mirror then execute * the selected part else execute the complete code. */ diff --git a/web/pgadmin/utils/driver/abstract.py b/web/pgadmin/utils/driver/abstract.py index 9b2363c..7db3e37 100644 --- a/web/pgadmin/utils/driver/abstract.py +++ b/web/pgadmin/utils/driver/abstract.py @@ -101,6 +101,12 @@ class BaseConnection(object): - Implement this method to execute the given query and returns the result as an array of dict (column name -> value) format. + * def async_fetchmany_2darray(records=-1, formatted_exception_msg=False): + - Implement this method to retrieve result of asynchronous connection and + polling with no_result flag set to True. + This returns the result as a 2 dimensional array. + If records is -1 then fetchmany will behave as fetchall. + * connected() - Implement this method to get the status of the connection. It should return True for connected, otherwise False @@ -133,7 +139,7 @@ class BaseConnection(object): - Implement this method to wait for asynchronous connection with timeout. This must be a non blocking call. - * poll(formatted_exception_msg) + * poll(formatted_exception_msg, no_result) - Implement this method to poll the data of query running on asynchronous connection. @@ -180,6 +186,10 @@ class BaseConnection(object): pass @abstractmethod + def async_fetchmany_2darray(self, records=-1, formatted_exception_msg=False): + pass + + @abstractmethod def connected(self): pass @@ -208,7 +218,7 @@ class BaseConnection(object): pass @abstractmethod - def poll(self, formatted_exception_msg=True): + def poll(self, formatted_exception_msg=True, no_result=False): pass @abstractmethod diff --git a/web/pgadmin/utils/driver/psycopg2/__init__.py b/web/pgadmin/utils/driver/psycopg2/__init__.py index 92562bc..d3dc5c5 100644 --- a/web/pgadmin/utils/driver/psycopg2/__init__.py +++ b/web/pgadmin/utils/driver/psycopg2/__init__.py @@ -1078,6 +1078,55 @@ Failed to execute query (execute_void) for the server #{server_id} - {conn_id} return True, {'columns': columns, 'rows': rows} + def async_fetchmany_2darray(self, records=2000, formatted_exception_msg=False): + """ + User should poll and check if status is ASYNC_OK before calling this + function + Args: + records: no of records to fetch. use -1 to fetchall. + formatted_exception_msg: + + Returns: + + """ + cur = self.__async_cursor + if not cur: + return False, gettext( + "Cursor could not be found for the async connection." + ) + + if self.conn.isexecuting(): + return False, gettext( + "Asynchronous query execution/operation underway." + ) + + if self.row_count > 0: + result = [] + # For DDL operation, we may not have result. + # + # Because - there is not direct way to differentiate DML and + # DDL operations, we need to rely on exception to figure + # that out at the moment. + try: + if records == -1: + res = cur.fetchall() + else: + res = cur.fetchmany(records) + for row in res: + new_row = [] + for col in self.column_info: + new_row.append(row[col['name']]) + result.append(new_row) + except psycopg2.ProgrammingError as e: + result = None + else: + # User performed operation which dose not produce record/s as + # result. + # for eg. DDL operations. + return True, None + + return True, result + def connected(self): if self.conn: if not self.conn.closed: @@ -1224,7 +1273,7 @@ Failed to reset the connection to the server due to following error: "poll() returned %s from _wait_timeout function" % state ) - def poll(self, formatted_exception_msg=False): + def poll(self, formatted_exception_msg=False, no_result=False): """ This function is a wrapper around connection's poll function. It internally uses the _wait_timeout method to poll the @@ -1234,6 +1283,7 @@ Failed to reset the connection to the server due to following error: Args: formatted_exception_msg: if True then function return the formatted exception message, otherwise error string. + no_result: If True then only poll status will be returned. """ cur = self.__async_cursor @@ -1289,23 +1339,23 @@ Failed to reset the connection to the server due to following error: pos += 1 self.row_count = cur.rowcount - - if cur.rowcount > 0: - result = [] - # For DDL operation, we may not have result. - # - # Because - there is not direct way to differentiate DML and - # DDL operations, we need to rely on exception to figure that - # out at the moment. - try: - for row in cur: - new_row = [] - for col in self.column_info: - new_row.append(row[col['name']]) - result.append(new_row) - - except psycopg2.ProgrammingError: - result = None + if not no_result: + if cur.rowcount > 0: + result = [] + # For DDL operation, we may not have result. + # + # Because - there is not direct way to differentiate DML and + # DDL operations, we need to rely on exception to figure + # that out at the moment. + try: + for row in cur: + new_row = [] + for col in self.column_info: + new_row.append(row[col['name']]) + result.append(new_row) + + except psycopg2.ProgrammingError: + result = None return status, result diff --git a/web/regression/feature_utils/pgadmin_page.py b/web/regression/feature_utils/pgadmin_page.py index bb111a1..1f12a80 100644 --- a/web/regression/feature_utils/pgadmin_page.py +++ b/web/regression/feature_utils/pgadmin_page.py @@ -224,6 +224,20 @@ class PgadminPage: self._wait_for("spinner to disappear", spinner_has_disappeared) + def wait_for_query_tool_loading_indicator_to_disappear(self): + def spinner_has_disappeared(driver): + try: + driver.find_element_by_xpath( + "//*[@id='fetching_data' and @class='hide']" + ) + return False + except NoSuchElementException: + # wait for loading indicator disappear animation to complete. + time.sleep(0.5) + return True + + self._wait_for("spinner to disappear", spinner_has_disappeared) + def wait_for_app(self): def page_shows_app(driver): if driver.title == self.app_config.APP_NAME: diff --git a/web/regression/javascript/selection/copy_data_spec.js b/web/regression/javascript/selection/copy_data_spec.js index 9e05e84..2b623b3 100644 --- a/web/regression/javascript/selection/copy_data_spec.js +++ b/web/regression/javascript/selection/copy_data_spec.js @@ -16,16 +16,16 @@ import clipboard from '../../../pgadmin/static/js/selection/clipboard'; import copyData from '../../../pgadmin/static/js/selection/copy_data'; import RangeSelectionHelper from 'sources/selection/range_selection_helper'; import XCellSelectionModel from 'sources/selection/xcell_selection_model'; - describe('copyData', function () { var grid, sqlEditor, gridContainer, buttonPasteRow; var SlickGrid; beforeEach(function () { SlickGrid = Slick.Grid; - var data = [[1, 'leopord', '12'], - [2, 'lion', '13'], - [3, 'puma', '9']]; + var data = [{'id': 1, 'brand':'leopord', 'size':'12', '__temp_PK': '123'}, + {'id': 2, 'brand':'lion', 'size':'13', '__temp_PK': '456'}, + {'id': 3, 'brand':'puma', 'size':'9', '__temp_PK': '789'}], + dataView = new Slick.Data.DataView(); var columns = [ { @@ -37,6 +37,7 @@ describe('copyData', function () { }, { name: 'id', + field: 'id', pos: 0, label: 'id<br> numeric', cell: 'number', @@ -44,6 +45,7 @@ describe('copyData', function () { type: 'numeric', }, { name: 'brand', + field: 'brand', pos: 1, label: 'flavor<br> character varying', cell: 'string', @@ -51,24 +53,26 @@ describe('copyData', function () { type: 'character varying', }, { name: 'size', + field: 'size', pos: 2, label: 'size<br> numeric', cell: 'number', can_edit: false, type: 'numeric', }, - ] - ; - gridContainer = $('<div id=\'grid\'></div>'); + ]; + gridContainer = $('<div id="grid"></div>'); $('body').append(gridContainer); - buttonPasteRow = $('<button id=\'btn-paste-row\' disabled></button>'); + buttonPasteRow = $('<button id="btn-paste-row" disabled></button>'); $('body').append(buttonPasteRow); - grid = new SlickGrid('#grid', data, columns, {}); + grid = new SlickGrid('#grid', dataView, columns, {}); + dataView.setItems(data, '__temp_PK'); grid.setSelectionModel(new XCellSelectionModel()); sqlEditor = {slickgrid: grid}; }); - afterEach(function () { + afterEach(function() { + grid.destroy(); gridContainer.remove(); buttonPasteRow.remove(); }); diff --git a/web/regression/javascript/selection/range_boundary_navigator_spec.js b/web/regression/javascript/selection/range_boundary_navigator_spec.js index 1de6d81..2d56b05 100644 --- a/web/regression/javascript/selection/range_boundary_navigator_spec.js +++ b/web/regression/javascript/selection/range_boundary_navigator_spec.js @@ -134,17 +134,19 @@ describe('RangeBoundaryNavigator', function () { describe('#rangesToCsv', function () { var data, columnDefinitions, ranges; beforeEach(function () { - data = [[1, 'leopard', '12'], - [2, 'lion', '13'], - [3, 'cougar', '9'], - [4, 'tiger', '10']]; - columnDefinitions = [{name: 'id', pos: 0}, {name: 'animal', pos: 1}, {name: 'size', pos: 2}]; + data = [{'id':1, 'animal':'leopard', 'size':'12'}, + {'id':2, 'animal':'lion', 'size':'13'}, + {'id':3, 'animal':'cougar', 'size':'9'}, + {'id':4, 'animal':'tiger', 'size':'10'}]; + + columnDefinitions = [{name: 'id', field: 'id', pos: 0}, + {name: 'animal', field: 'animal', pos: 1}, + {name: 'size', field: 'size', pos: 2}]; ranges = [new Slick.Range(0, 0, 0, 2), new Slick.Range(3, 0, 3, 2)]; }); it('returns csv for the provided ranges', function () { var csvResult = rangeBoundaryNavigator.rangesToCsv(data, columnDefinitions, ranges); - expect(csvResult).toEqual('1,\'leopard\',\'12\'\n4,\'tiger\',\'10\''); }); @@ -158,10 +160,10 @@ describe('RangeBoundaryNavigator', function () { describe('when there is an extra column with checkboxes', function () { beforeEach(function () { - columnDefinitions = [{name: 'not-a-data-column'}, {name: 'id', pos: 0}, {name: 'animal', pos: 1}, { - name: 'size', - pos: 2, - }]; + columnDefinitions = [{name: 'not-a-data-column'}, + {name: 'id', field: 'id', pos: 0}, + {name: 'animal', field: 'animal', pos: 1}, + {name: 'size', field: 'size',pos: 2}]; ranges = [new Slick.Range(0, 0, 0, 3), new Slick.Range(3, 0, 3, 3)]; }); diff --git a/web/regression/javascript/selection/row_selector_spec.js b/web/regression/javascript/selection/row_selector_spec.js index db3c0b4..d4152f8 100644 --- a/web/regression/javascript/selection/row_selector_spec.js +++ b/web/regression/javascript/selection/row_selector_spec.js @@ -6,11 +6,11 @@ // This software is released under the PostgreSQL Licence // ////////////////////////////////////////////////////////////////////////// - import $ from 'jquery'; import Slick from 'slickgrid'; import 'slickgrid.grid'; +import 'slick.dataview'; import RowSelector from 'sources/selection/row_selector'; import ActiveCellCapture from 'sources/selection/active_cell_capture'; @@ -23,7 +23,7 @@ describe('RowSelector', function () { UP: 38, DOWN: 40, }; - var container, data, columnDefinitions, grid, cellSelectionModel; + var container, dataView, columnDefinitions, grid, cellSelectionModel; var SlickGrid = Slick.Grid; beforeEach(function () { @@ -43,14 +43,15 @@ describe('RowSelector', function () { pos: 1, }]; + dataView = new Slick.Data.DataView(); var rowSelector = new RowSelector(); - data = []; + var data = []; for (var i = 0; i < 10; i++) { - data.push(['some-value-' + i, 'second value ' + i]); + data.push({'some-column-name':'some-value-' + i, 'second column':'second value ' + i}); } columnDefinitions = rowSelector.getColumnDefinitions(columnDefinitions); - grid = new SlickGrid(container, data, columnDefinitions); - + dataView.setItems(data, 'some-column-name'); + grid = new SlickGrid(container, dataView, columnDefinitions); grid.registerPlugin(new ActiveCellCapture()); cellSelectionModel = new XCellSelectionModel(); grid.setSelectionModel(cellSelectionModel); diff --git a/web/regression/javascript/selection/set_staged_rows_spec.js b/web/regression/javascript/selection/set_staged_rows_spec.js index 6735314..cf201c4 100644 --- a/web/regression/javascript/selection/set_staged_rows_spec.js +++ b/web/regression/javascript/selection/set_staged_rows_spec.js @@ -7,236 +7,248 @@ // ////////////////////////////////////////////////////////////// -define([ - 'jquery', - 'underscore', - 'sources/selection/set_staged_rows', -], function ($, _, SetStagedRows) { - describe('set_staged_rows', function () { - var sqlEditorObj, gridSpy, deleteButton, copyButton, selectionSpy; - beforeEach(function () { - gridSpy = jasmine.createSpyObj('gridSpy', ['getData', 'getCellNode', 'getColumns']); - gridSpy.getData.and.returnValue([ - {0: 'one', 1: 'two', __temp_PK: '123'}, - {0: 'three', 1: 'four', __temp_PK: '456'}, - {0: 'five', 1: 'six', __temp_PK: '789'}, - {0: 'seven', 1: 'eight', __temp_PK: '432'}, - ]); - gridSpy.getColumns.and.returnValue([ +import $ from 'jquery'; +import 'slickgrid.grid'; +import Slick from 'slickgrid'; +import 'slick.dataview'; +import SetStagedRows from 'sources/selection/set_staged_rows'; + +describe('set_staged_rows', function () { + var sqlEditorObj, gridSpy, deleteButton, copyButton, selectionSpy; + beforeEach(function () { + var data = [{'a pk column': 'one', 'some column': 'two', '__temp_PK': '123'}, + {'a pk column': 'three', 'some column': 'four', '__temp_PK': '456'}, + {'a pk column': 'five', 'some column': 'six', '__temp_PK': '789'}, + {'a pk column': 'seven', 'some column': 'eight', '__temp_PK': '432'}], + dataView = new Slick.Data.DataView(); + + dataView.setItems(data, '__temp_PK'); + + gridSpy = jasmine.createSpyObj('gridSpy', ['getData', 'getCellNode', 'getColumns']); + gridSpy.getData.and.returnValue(dataView); + gridSpy.getColumns.and.returnValue([ + { + name: 'a pk column', + field: 'a pk column', + pos: 0, + selectable: true, + }, { + name: 'some column', + field: 'some column', + pos: 1, + selectable: true, + }, + ]); + selectionSpy = jasmine.createSpyObj('selectionSpy', ['setSelectedRows', 'getSelectedRanges']); + deleteButton = $('<button id="btn-delete-row"></button>'); + copyButton = $('<button id="btn-copy-row"></button>'); + + sqlEditorObj = { + grid: gridSpy, + editor: { + handler: { + data_store: { + staged_rows: {'456': {}}, + }, + can_edit: false, + }, + }, + keys: null, + selection: selectionSpy, + columns: [ { + name: 'a pk column', + field: 'a pk column', pos: 0, - selectable: true, - }, { + }, + { + name: 'some column', + field: 'some column', pos: 1, - selectable: true, }, - ]); + ], + client_primary_key: '__temp_PK', + }; - selectionSpy = jasmine.createSpyObj('selectionSpy', ['setSelectedRows', 'getSelectedRanges']); + $('body').append(deleteButton); + $('body').append(copyButton); - deleteButton = $('<button id="btn-delete-row"></button>'); - copyButton = $('<button id="btn-copy-row"></button>'); + deleteButton.prop('disabled', true); + copyButton.prop('disabled', true); - sqlEditorObj = { - grid: gridSpy, - editor: { - handler: { - data_store: { - staged_rows: {'456': {}}, - }, - can_edit: false, - }, - }, - keys: null, - selection: selectionSpy, - columns: [ - { - name: 'a pk column', - pos: 0, - }, - { - name: 'some column', - pos: 1, - }, - ], - }; + selectionSpy = jasmine.createSpyObj('selectionSpy', [ + 'setSelectedRows', + 'getSelectedRanges', + ]); + }); - $('body').append(deleteButton); - $('body').append(copyButton); + afterEach(function () { + copyButton.remove(); + deleteButton.remove(); + }); + describe('when no full rows are selected', function () { + describe('when nothing is selected', function () { + beforeEach(function () { + selectionSpy.getSelectedRanges.and.returnValue([]); + sqlEditorObj.selection = selectionSpy; + SetStagedRows.call(sqlEditorObj, {}, {}); + }); - deleteButton.prop('disabled', true); - copyButton.prop('disabled', true); + it('should disable the delete row button', function () { + expect($('#btn-delete-row').prop('disabled')).toBeTruthy(); + }); - selectionSpy = jasmine.createSpyObj('selectionSpy', [ - 'setSelectedRows', - 'getSelectedRanges', - ]); - }); + it('should disable the copy row button', function () { + expect($('#btn-copy-row').prop('disabled')).toBeTruthy(); + }); - afterEach(function () { - copyButton.remove(); - deleteButton.remove(); + it('should clear staged rows', function () { + expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({}); + }); }); - describe('when no full rows are selected', function () { - describe('when nothing is selected', function () { - beforeEach(function () { - selectionSpy.getSelectedRanges.and.returnValue([]); - sqlEditorObj.selection = selectionSpy; - SetStagedRows.call(sqlEditorObj, {}, {}); - }); - it('should disable the delete row button', function () { - expect($('#btn-delete-row').prop('disabled')).toBeTruthy(); - }); + describe('when there is a selection', function () { + beforeEach(function () { + var range = { + fromCell: 0, + toCell: 0, + fromRow: 1, + toRow: 1, + }; - it('should disable the copy row button', function () { - expect($('#btn-copy-row').prop('disabled')).toBeTruthy(); - }); + selectionSpy.getSelectedRanges.and.returnValue([range]); + sqlEditorObj.selection = selectionSpy; + SetStagedRows.call(sqlEditorObj, {}, {}); + }); - it('should clear staged rows', function () { - expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({}); - }); + it('should disable the delete row button', function () { + expect($('#btn-delete-row').prop('disabled')).toBeTruthy(); }); - describe('when there is a selection', function () { - beforeEach(function () { - var range = { - fromCell: 0, - toCell: 0, - fromRow: 1, - toRow: 1, - }; - - selectionSpy.getSelectedRanges.and.returnValue([range]); - sqlEditorObj.selection = selectionSpy; - SetStagedRows.call(sqlEditorObj, {}, {}); - }); + it('should disable the copy row button', function () { + expect($('#btn-copy-row').prop('disabled')).toBeFalsy(); + }); - it('should disable the delete row button', function () { - expect($('#btn-delete-row').prop('disabled')).toBeTruthy(); - }); + it('should clear staged rows', function () { + expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({}); + }); + }); + }); - it('should disable the copy row button', function () { - expect($('#btn-copy-row').prop('disabled')).toBeFalsy(); - }); + describe('when 2 full rows are selected', function () { + beforeEach(function () { + var range1 = { + fromCell: 0, + toCell: 1, + fromRow: 1, + toRow: 1, + }; + var range2 = { + fromCell: 0, + toCell: 1, + fromRow: 2, + toRow: 2, + }; - it('should clear staged rows', function () { - expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({}); - }); - }); + selectionSpy.getSelectedRanges.and.returnValue([range1, range2]); + sqlEditorObj.selection = selectionSpy; }); - describe('when 2 full rows are selected', function () { - beforeEach(function () { - var range1 = { - fromCell: 0, - toCell: 1, - fromRow: 1, - toRow: 1, - }; - var range2 = { - fromCell: 0, - toCell: 1, - fromRow: 2, - toRow: 2, - }; + describe('when table does not have primary keys', function () { + it('should enable the copy row button', function () { + SetStagedRows.call(sqlEditorObj, {}, {}); + expect($('#btn-copy-row').prop('disabled')).toBeFalsy(); + }); - selectionSpy.getSelectedRanges.and.returnValue([range1, range2]); - sqlEditorObj.selection = selectionSpy; + it('should not enable the delete row button', function () { + SetStagedRows.call(sqlEditorObj, {}, {}); + expect($('#btn-delete-row').prop('disabled')).toBeTruthy(); }); - describe('when table does not have primary keys', function () { - it('should enable the copy row button', function () { - SetStagedRows.call(sqlEditorObj, {}, {}); - expect($('#btn-copy-row').prop('disabled')).toBeFalsy(); - }); + it('should update staged rows with the __temp_PK value of the new Selected Rows', function () { + SetStagedRows.call(sqlEditorObj, {}, {}); + expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({'456': {}, '789': {}}); + }); - it('should not enable the delete row button', function () { + describe('the user can edit', function () { + it('should enable the delete row button', function () { + sqlEditorObj.editor.handler.can_edit = true; SetStagedRows.call(sqlEditorObj, {}, {}); - expect($('#btn-delete-row').prop('disabled')).toBeTruthy(); + expect($('#btn-delete-row').prop('disabled')).toBeFalsy(); }); + }); + }); - it('should update staged rows with the __temp_PK value of the new Selected Rows', function () { + describe('when table has primary keys', function () { + beforeEach(function () { + sqlEditorObj.keys = {'a pk column': 'varchar'}; + sqlEditorObj.editor.handler.data_store.staged_rows = {'456': {'a pk column': 'three'}}; + }); + + describe('selected rows have primary key', function () { + it('should set the staged rows correctly', function () { SetStagedRows.call(sqlEditorObj, {}, {}); - expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({'456': {}, '789': {}}); + expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual( + {'456': {'a pk column': 'three'}, '789': {'a pk column': 'five'}}); }); - describe('the user can edit', function () { - it('should enable the delete row button', function () { - sqlEditorObj.editor.handler.can_edit = true; - SetStagedRows.call(sqlEditorObj, {}, {}); - expect($('#btn-delete-row').prop('disabled')).toBeFalsy(); - }); + it('should not clear selected rows in Cell Selection Model', function () { + SetStagedRows.call(sqlEditorObj, {}, {}); + expect(sqlEditorObj.selection.setSelectedRows).not.toHaveBeenCalledWith(); }); }); - describe('when table has primary keys', function () { + describe('selected rows missing primary key', function () { beforeEach(function () { - sqlEditorObj.keys = {'a pk column': 'varchar'}; - sqlEditorObj.editor.handler.data_store.staged_rows = {'456': {0: 'three'}}; - }); + var data = [{'a pk column': 'one', 'some column': 'two', '__temp_PK': '123'}, + {'some column': 'four', '__temp_PK': '456'}, + {'some column': 'six', '__temp_PK': '789'}, + {'a pk column': 'seven', 'some column': 'eight', '__temp_PK': '432'}], + dataView = new Slick.Data.DataView(); - describe('selected rows have primary key', function () { - it('should set the staged rows correctly', function () { - SetStagedRows.call(sqlEditorObj, {}, {}); - expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual( - {'456': {0: 'three'}, '789': {0: 'five'}}); - }); + dataView.setItems(data, '__temp_PK'); - it('should not clear selected rows in Cell Selection Model', function () { - SetStagedRows.call(sqlEditorObj, {}, {}); - expect(sqlEditorObj.selection.setSelectedRows).not.toHaveBeenCalledWith(); - }); + gridSpy.getData.and.returnValue(dataView); }); - describe('selected rows missing primary key', function () { - beforeEach(function () { - gridSpy.getData.and.returnValue([ - {0: 'one', 1: 'two', __temp_PK: '123'}, - {1: 'four', __temp_PK: '456'}, - {1: 'six', __temp_PK: '789'}, - {0: 'seven', 1: 'eight', __temp_PK: '432'}, - ]); - }); - - it('should clear the staged rows', function () { - SetStagedRows.call(sqlEditorObj, {}, {}); - expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({}); - }); + it('should clear the staged rows', function () { + SetStagedRows.call(sqlEditorObj, {}, {}); + expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({}); + }); - it('should clear selected rows in Cell Selection Model', function () { - SetStagedRows.call(sqlEditorObj, {}, {}); - expect(sqlEditorObj.selection.setSelectedRows).toHaveBeenCalledWith([]); - }); + it('should clear selected rows in Cell Selection Model', function () { + SetStagedRows.call(sqlEditorObj, {}, {}); + expect(sqlEditorObj.selection.setSelectedRows).toHaveBeenCalledWith([]); }); + }); - describe('when the selected row is a new row', function () { - var parentDiv; - beforeEach(function () { - var childDiv = $('<div></div>'); - parentDiv = $('<div class="new_row"></div>'); - parentDiv.append(childDiv); - $('body').append(parentDiv); - gridSpy.getCellNode.and.returnValue(childDiv); - SetStagedRows.call(sqlEditorObj, {}, {}); - }); + describe('when the selected row is a new row', function () { + var parentDiv; + beforeEach(function () { + var childDiv = $('<div></div>'); + parentDiv = $('<div class="new_row"></div>'); + parentDiv.append(childDiv); + $('body').append(parentDiv); + gridSpy.getCellNode.and.returnValue(childDiv); + SetStagedRows.call(sqlEditorObj, {}, {}); + }); - afterEach(function () { - parentDiv.remove(); - }); + afterEach(function () { + parentDiv.remove(); + }); - it('should not clear the staged rows', function () { - expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({ - '456': {0: 'three'}, - '789': {0: 'five'}, - }); + it('should not clear the staged rows', function () { + expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({ + '456': {'a pk column': 'three'}, + '789': {'a pk column': 'five'}, }); + }); - it('should not clear selected rows in Cell Selection Model', function () { - expect(sqlEditorObj.selection.setSelectedRows).not.toHaveBeenCalled(); - }); + it('should not clear selected rows in Cell Selection Model', function () { + expect(sqlEditorObj.selection.setSelectedRows).not.toHaveBeenCalled(); }); }); }); }); }); + diff --git a/web/regression/javascript/selection/xcell_selection_model_spec.js b/web/regression/javascript/selection/xcell_selection_model_spec.js index db3f6d6..2882e49 100644 --- a/web/regression/javascript/selection/xcell_selection_model_spec.js +++ b/web/regression/javascript/selection/xcell_selection_model_spec.js @@ -32,14 +32,17 @@ describe('XCellSelectionModel', function () { }, { id: '1', name: 'some-column-name', + field: 'some-column-name', pos: 0, }, { id: 'second-column-id', name: 'second column', + field: 'second column', pos: 1, }, { id: 'third-column-id', name: 'third column', + field: 'third column', pos: 2, }, ]; @@ -51,13 +54,15 @@ describe('XCellSelectionModel', function () { 'second column': 'second value ' + i, 'third column': 'third value ' + i, 'fourth column': 'fourth value ' + i, + '__temp_PK': '123' + i, }); } container = $('<div></div>'); + var dataView = new Slick.Data.DataView(); container.height(9999); container.width(9999); - - grid = new SlickGrid(container, data, columns); + dataView.setItems(data, '__temp_PK'); + grid = new SlickGrid(container, dataView, columns); grid.setSelectionModel(new XCellSelectionModel()); $('body').append(container); }); diff --git a/web/regression/javascript/slickgrid/event_handlers/handle_query_output_keyboard_event_spec.js b/web/regression/javascript/slickgrid/event_handlers/handle_query_output_keyboard_event_spec.js index eb4411d..e790a8f 100644 --- a/web/regression/javascript/slickgrid/event_handlers/handle_query_output_keyboard_event_spec.js +++ b/web/regression/javascript/slickgrid/event_handlers/handle_query_output_keyboard_event_spec.js @@ -27,21 +27,22 @@ describe('#handleQueryOutputKeyboardEvent', function () { metaKey: false, which: -1, keyCode: -1, - preventDefault: jasmine.createSpy('preventDefault'), + preventDefault: jasmine.createSpy('preventDefault') }; - var data = [['', '0,0-cell-content', '0,1-cell-content'], - ['', '1,0-cell-content', '1,1-cell-content'], - ['', '2,0-cell-content', '2,1-cell-content']]; - var columnDefinitions = [{name: 'checkboxColumn'}, {pos: 1, name: 'firstColumn'}, { - pos: 2, - name: 'secondColumn', - }]; - grid = new SlickGrid($('<div></div>'), data, columnDefinitions); - grid.setSelectionModel(new XCellSelectionModel()); + var data = [{'checkboxColumn': '', 'firstColumn': '0,0-cell-content', 'secondColumn': '0,1-cell-content', '__temp_PK': '123'}, + {'checkboxColumn': '', 'firstColumn': '1,0-cell-content', 'secondColumn': '1,1-cell-content', '__temp_PK': '456'}, + {'checkboxColumn': '', 'firstColumn': '2,0-cell-content', 'secondColumn': '2,1-cell-content', '__temp_PK': '789'}], + columnDefinitions = [{name: 'checkboxColumn'}, + {pos: 1, name: 'firstColumn', field: 'firstColumn'}, + { pos: 2, name: 'secondColumn', field: 'secondColumn'}], + dataView = new Slick.Data.DataView(); + grid = new Slick.Grid($('<div></div>'), dataView, columnDefinitions); + grid.setSelectionModel(new XCellSelectionModel()); + dataView.setItems(data, '__temp_PK'); slickEvent = { - grid: grid, + grid: grid }; spyOn(clipboard, 'copyTextToClipboard'); diff --git a/web/webpack.test.config.js b/web/webpack.test.config.js index 6754f04..8d3e9e9 100644 --- a/web/webpack.test.config.js +++ b/web/webpack.test.config.js @@ -63,6 +63,7 @@ module.exports = { 'pgadmin.browser.endpoints': regressionDir + '/javascript/fake_endpoints', 'slickgrid': sourcesDir + '/vendor/slickgrid/slick.core', 'slickgrid.grid': sourcesDir + '/vendor/slickgrid/slick.grid', + 'slick.dataview': sourcesDir + '/vendor/slickgrid/slick.dataview', 'slickgrid.rowselectionmodel': sourcesDir + '/vendor/slickgrid/plugins/slick.rowselectionmodel', 'browser': path.resolve(__dirname, 'pgadmin/browser/static/js'), 'pgadmin': sourcesDir + '/js/pgadmin', ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] @ 2017-06-15 20:37 Dave Page <[email protected]> parent: Harshal Dhumal <[email protected]> 0 siblings, 1 reply; 24+ messages in thread From: Dave Page @ 2017-06-15 20:37 UTC (permalink / raw) To: Harshal Dhumal <[email protected]>; +Cc: Harshal Dhumal <[email protected]>; Ashesh Vashi <[email protected]>; pgadmin-hackers Hi On Thu, Jun 15, 2017 at 9:30 PM, Harshal Dhumal <[email protected]> wrote: > Hi Dave, > > Please find attached updated patch. > > On Thu, Jun 15, 2017 at 3:58 PM, Dave Page <[email protected]> wrote: >> >> Hi >> >> On Wed, Jun 14, 2017 at 11:36 PM, Harshal Dhumal >> <[email protected]> wrote: >>> >>> Hi Dave, >>> >>> Please find rebased patch for RM2137. >> >> >> Looking very good. The only issues I see are: >> >> - The row headers should auto-size such that they can display the row >> numbers if the last row was displayed. E.g. if there are 12345 rows in >> total, then the row header should be sized to display 5 digits. >> > Fixed. > > >> >> - The tests are comprehensive, which is awesome. However, every time I ran >> them, at least one of the feature tests failed. Unfortunately, it was a >> different one each time. In the last two runs, I got: >> >> ====================================================================== >> ERROR: runTest >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >> Query tool feature test >> ---------------------------------------------------------------------- >> Traceback (most recent call last): >> File >> "/Users/dpage/git/pgadmin4/web/regression/feature_utils/base_feature_test.py", >> line 40, in setUp >> self.before() >> File >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", >> line 40, in before >> self._connects_to_server() >> File >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", >> line 144, in _connects_to_server >> self.page.driver.find_element_by_link_text("Create"))\ >> File >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> line 319, in find_element_by_link_text >> return self.find_element(by=By.LINK_TEXT, value=link_text) >> File >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> line 756, in find_element >> 'value': value})['value'] >> File >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> line 238, in execute >> self.error_handler.check_response(response) >> File >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", >> line 193, in check_response >> raise exception_class(message, screen, stacktrace) >> NoSuchElementException: Message: no such element: Unable to locate >> element: {"method":"link text","selector":"Create"} >> (Session info: chrome=58.0.3029.110) >> (Driver info: chromedriver=2.29.461585 >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 x86_64) >> > I checked code from _connects_to_server function which is common in all > features test cases. I didn't find anything wrong with this. If there is a > bug in this function then all feature test must fail. > Let me know if you are getting failure consistently in _connects_to_server > function. I wondered if that one is a race condition. Do we need a short delay before clicking the Object menu? I have seen this occasionally before. >> >> >> ====================================================================== >> ERROR: runTest >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >> Query tool feature test >> ---------------------------------------------------------------------- >> Traceback (most recent call last): >> File >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", >> line 119, in runTest >> self._query_tool_auto_rollback_enabled() >> File >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", >> line 697, in _query_tool_auto_rollback_enabled >> '//div[contains(@class, "sql-editor-message") and contains(string(), >> "COMMIT")]' >> File >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> line 295, in find_element_by_xpath >> return self.find_element(by=By.XPATH, value=xpath) >> File >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> line 756, in find_element >> 'value': value})['value'] >> File >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> line 238, in execute >> self.error_handler.check_response(response) >> File >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", >> line 193, in check_response >> raise exception_class(message, screen, stacktrace) >> NoSuchElementException: Message: no such element: Unable to locate >> element: {"method":"xpath","selector":"//div[contains(@class, >> "sql-editor-message") and contains(string(), "COMMIT")]"} >> (Session info: chrome=58.0.3029.110) >> (Driver info: chromedriver=2.29.461585 >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 x86_64) >> > I have updated Auto rollback enabled test in this patch. > >> >> Relevant screenshots attached. >> >> - Can you tidy up the regression output a little please? Instead of: >> >> ------- >> runTest (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >> Query tool feature test ... On demand result set on scrolling... >> OK. >> >> On demand result set on grid select all... >> OK. >> >> On demand result set on column select all... >> OK. >> >> Explain query... >> OK. >> >> Explain query with verbose... >> OK. >> >> Explain query with costs... >> OK. >> >> Explain analyze query... >> OK. >> >> Explain analyze query with buffers... >> OK. >> >> Explain analyze query with timing... >> OK. >> >> Auto commit disabled... >> OK. >> >> Auto commit enabled... >> OK. >> >> Auto rollback enabled... >> ERROR >> ------- >> >> Something like: >> >> ------- >> runTest (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >> Query tool feature test ... >> On demand result set on scrolling... OK. >> On demand result set on grid select all... OK. >> On demand result set on column select all... OK. >> Explain query... OK. >> Explain query with verbose... OK. >> Explain query with costs... OK. >> Explain analyze query... OK. >> Explain analyze query with buffers... OK. >> Explain analyze query with timing... OK. >> Auto commit disabled... OK. >> Auto commit enabled... OK. >> Auto rollback enabled... ERROR >> -------- >> > Fixed. > > >> >> Thanks! >> >> -- >> Dave Page >> Blog: http://pgsnake.blogspot.com >> Twitter: @pgsnake >> >> EnterpriseDB UK: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company > > -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgadmin-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgadmin-hackers ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] @ 2017-06-15 20:54 Harshal Dhumal <[email protected]> parent: Dave Page <[email protected]> 0 siblings, 1 reply; 24+ messages in thread From: Harshal Dhumal @ 2017-06-15 20:54 UTC (permalink / raw) To: Dave Page <[email protected]>; +Cc: Harshal Dhumal <[email protected]>; Ashesh Vashi <[email protected]>; pgadmin-hackers Hi On Fri, Jun 16, 2017 at 2:07 AM, Dave Page <[email protected]> wrote: > Hi > > On Thu, Jun 15, 2017 at 9:30 PM, Harshal Dhumal > <[email protected]> wrote: > > Hi Dave, > > > > Please find attached updated patch. > > > > On Thu, Jun 15, 2017 at 3:58 PM, Dave Page <[email protected]> wrote: > >> > >> Hi > >> > >> On Wed, Jun 14, 2017 at 11:36 PM, Harshal Dhumal > >> <[email protected]> wrote: > >>> > >>> Hi Dave, > >>> > >>> Please find rebased patch for RM2137. > >> > >> > >> Looking very good. The only issues I see are: > >> > >> - The row headers should auto-size such that they can display the row > >> numbers if the last row was displayed. E.g. if there are 12345 rows in > >> total, then the row header should be sized to display 5 digits. > >> > > Fixed. > > > > > >> > >> - The tests are comprehensive, which is awesome. However, every time I > ran > >> them, at least one of the feature tests failed. Unfortunately, it was a > >> different one each time. In the last two runs, I got: > >> > >> ====================================================================== > >> ERROR: runTest > >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) > >> Query tool feature test > >> ---------------------------------------------------------------------- > >> Traceback (most recent call last): > >> File > >> "/Users/dpage/git/pgadmin4/web/regression/feature_utils/ > base_feature_test.py", > >> line 40, in setUp > >> self.before() > >> File > >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > query_tool_tests.py", > >> line 40, in before > >> self._connects_to_server() > >> File > >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > query_tool_tests.py", > >> line 144, in _connects_to_server > >> self.page.driver.find_element_by_link_text("Create"))\ > >> File > >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> line 319, in find_element_by_link_text > >> return self.find_element(by=By.LINK_TEXT, value=link_text) > >> File > >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> line 756, in find_element > >> 'value': value})['value'] > >> File > >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> line 238, in execute > >> self.error_handler.check_response(response) > >> File > >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/errorhandler.py", > >> line 193, in check_response > >> raise exception_class(message, screen, stacktrace) > >> NoSuchElementException: Message: no such element: Unable to locate > >> element: {"method":"link text","selector":"Create"} > >> (Session info: chrome=58.0.3029.110) > >> (Driver info: chromedriver=2.29.461585 > >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 > x86_64) > >> > > I checked code from _connects_to_server function which is common in all > > features test cases. I didn't find anything wrong with this. If there is > a > > bug in this function then all feature test must fail. > > Let me know if you are getting failure consistently in > _connects_to_server > > function. > > I wondered if that one is a race condition. Do we need a short delay > before clicking the Object menu? I have seen this occasionally before. > OK. In that case let's try putting 1-2 second delay and observer behaviour. I'll send separate patch for this tomorrow as this is not related to on demand query result feature or its test cases. > > >> > >> > >> ====================================================================== > >> ERROR: runTest > >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) > >> Query tool feature test > >> ---------------------------------------------------------------------- > >> Traceback (most recent call last): > >> File > >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > query_tool_tests.py", > >> line 119, in runTest > >> self._query_tool_auto_rollback_enabled() > >> File > >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > query_tool_tests.py", > >> line 697, in _query_tool_auto_rollback_enabled > >> '//div[contains(@class, "sql-editor-message") and contains(string(), > >> "COMMIT")]' > >> File > >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> line 295, in find_element_by_xpath > >> return self.find_element(by=By.XPATH, value=xpath) > >> File > >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> line 756, in find_element > >> 'value': value})['value'] > >> File > >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> line 238, in execute > >> self.error_handler.check_response(response) > >> File > >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/errorhandler.py", > >> line 193, in check_response > >> raise exception_class(message, screen, stacktrace) > >> NoSuchElementException: Message: no such element: Unable to locate > >> element: {"method":"xpath","selector":"//div[contains(@class, > >> "sql-editor-message") and contains(string(), "COMMIT")]"} > >> (Session info: chrome=58.0.3029.110) > >> (Driver info: chromedriver=2.29.461585 > >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 > x86_64) > >> > > I have updated Auto rollback enabled test in this patch. > > > >> > >> Relevant screenshots attached. > >> > >> - Can you tidy up the regression output a little please? Instead of: > >> > >> ------- > >> runTest (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) > >> Query tool feature test ... On demand result set on scrolling... > >> OK. > >> > >> On demand result set on grid select all... > >> OK. > >> > >> On demand result set on column select all... > >> OK. > >> > >> Explain query... > >> OK. > >> > >> Explain query with verbose... > >> OK. > >> > >> Explain query with costs... > >> OK. > >> > >> Explain analyze query... > >> OK. > >> > >> Explain analyze query with buffers... > >> OK. > >> > >> Explain analyze query with timing... > >> OK. > >> > >> Auto commit disabled... > >> OK. > >> > >> Auto commit enabled... > >> OK. > >> > >> Auto rollback enabled... > >> ERROR > >> ------- > >> > >> Something like: > >> > >> ------- > >> runTest (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) > >> Query tool feature test ... > >> On demand result set on scrolling... OK. > >> On demand result set on grid select all... OK. > >> On demand result set on column select all... OK. > >> Explain query... OK. > >> Explain query with verbose... OK. > >> Explain query with costs... OK. > >> Explain analyze query... OK. > >> Explain analyze query with buffers... OK. > >> Explain analyze query with timing... OK. > >> Auto commit disabled... OK. > >> Auto commit enabled... OK. > >> Auto rollback enabled... ERROR > >> -------- > >> > > Fixed. > > > > > >> > >> Thanks! > >> > >> -- > >> Dave Page > >> Blog: http://pgsnake.blogspot.com > >> Twitter: @pgsnake > >> > >> EnterpriseDB UK: http://www.enterprisedb.com > >> The Enterprise PostgreSQL Company > > > > > > > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] @ 2017-06-15 20:55 Dave Page <[email protected]> parent: Harshal Dhumal <[email protected]> 0 siblings, 1 reply; 24+ messages in thread From: Dave Page @ 2017-06-15 20:55 UTC (permalink / raw) To: Harshal Dhumal <[email protected]>; +Cc: Harshal Dhumal <[email protected]>; Ashesh Vashi <[email protected]>; pgadmin-hackers Sounds good, thanks. On Thu, Jun 15, 2017 at 9:54 PM, Harshal Dhumal <[email protected]> wrote: > Hi > > On Fri, Jun 16, 2017 at 2:07 AM, Dave Page <[email protected]> wrote: >> >> Hi >> >> On Thu, Jun 15, 2017 at 9:30 PM, Harshal Dhumal >> <[email protected]> wrote: >> > Hi Dave, >> > >> > Please find attached updated patch. >> > >> > On Thu, Jun 15, 2017 at 3:58 PM, Dave Page <[email protected]> wrote: >> >> >> >> Hi >> >> >> >> On Wed, Jun 14, 2017 at 11:36 PM, Harshal Dhumal >> >> <[email protected]> wrote: >> >>> >> >>> Hi Dave, >> >>> >> >>> Please find rebased patch for RM2137. >> >> >> >> >> >> Looking very good. The only issues I see are: >> >> >> >> - The row headers should auto-size such that they can display the row >> >> numbers if the last row was displayed. E.g. if there are 12345 rows in >> >> total, then the row header should be sized to display 5 digits. >> >> >> > Fixed. >> > >> > >> >> >> >> - The tests are comprehensive, which is awesome. However, every time I >> >> ran >> >> them, at least one of the feature tests failed. Unfortunately, it was a >> >> different one each time. In the last two runs, I got: >> >> >> >> ====================================================================== >> >> ERROR: runTest >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >> >> Query tool feature test >> >> ---------------------------------------------------------------------- >> >> Traceback (most recent call last): >> >> File >> >> >> >> "/Users/dpage/git/pgadmin4/web/regression/feature_utils/base_feature_test.py", >> >> line 40, in setUp >> >> self.before() >> >> File >> >> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", >> >> line 40, in before >> >> self._connects_to_server() >> >> File >> >> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", >> >> line 144, in _connects_to_server >> >> self.page.driver.find_element_by_link_text("Create"))\ >> >> File >> >> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> >> line 319, in find_element_by_link_text >> >> return self.find_element(by=By.LINK_TEXT, value=link_text) >> >> File >> >> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> >> line 756, in find_element >> >> 'value': value})['value'] >> >> File >> >> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> >> line 238, in execute >> >> self.error_handler.check_response(response) >> >> File >> >> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", >> >> line 193, in check_response >> >> raise exception_class(message, screen, stacktrace) >> >> NoSuchElementException: Message: no such element: Unable to locate >> >> element: {"method":"link text","selector":"Create"} >> >> (Session info: chrome=58.0.3029.110) >> >> (Driver info: chromedriver=2.29.461585 >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 >> >> x86_64) >> >> >> > I checked code from _connects_to_server function which is common in all >> > features test cases. I didn't find anything wrong with this. If there is >> > a >> > bug in this function then all feature test must fail. >> > Let me know if you are getting failure consistently in >> > _connects_to_server >> > function. >> >> I wondered if that one is a race condition. Do we need a short delay >> before clicking the Object menu? I have seen this occasionally before. > > > OK. In that case let's try putting 1-2 second delay and observer behaviour. > I'll send separate patch for this tomorrow as this is not related to on > demand query result feature or its test cases. > >> >> >> >> >> >> >> >> ====================================================================== >> >> ERROR: runTest >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >> >> Query tool feature test >> >> ---------------------------------------------------------------------- >> >> Traceback (most recent call last): >> >> File >> >> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", >> >> line 119, in runTest >> >> self._query_tool_auto_rollback_enabled() >> >> File >> >> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", >> >> line 697, in _query_tool_auto_rollback_enabled >> >> '//div[contains(@class, "sql-editor-message") and >> >> contains(string(), >> >> "COMMIT")]' >> >> File >> >> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> >> line 295, in find_element_by_xpath >> >> return self.find_element(by=By.XPATH, value=xpath) >> >> File >> >> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> >> line 756, in find_element >> >> 'value': value})['value'] >> >> File >> >> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> >> line 238, in execute >> >> self.error_handler.check_response(response) >> >> File >> >> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", >> >> line 193, in check_response >> >> raise exception_class(message, screen, stacktrace) >> >> NoSuchElementException: Message: no such element: Unable to locate >> >> element: {"method":"xpath","selector":"//div[contains(@class, >> >> "sql-editor-message") and contains(string(), "COMMIT")]"} >> >> (Session info: chrome=58.0.3029.110) >> >> (Driver info: chromedriver=2.29.461585 >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 >> >> x86_64) >> >> >> > I have updated Auto rollback enabled test in this patch. >> > >> >> >> >> Relevant screenshots attached. >> >> >> >> - Can you tidy up the regression output a little please? Instead of: >> >> >> >> ------- >> >> runTest (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >> >> Query tool feature test ... On demand result set on scrolling... >> >> OK. >> >> >> >> On demand result set on grid select all... >> >> OK. >> >> >> >> On demand result set on column select all... >> >> OK. >> >> >> >> Explain query... >> >> OK. >> >> >> >> Explain query with verbose... >> >> OK. >> >> >> >> Explain query with costs... >> >> OK. >> >> >> >> Explain analyze query... >> >> OK. >> >> >> >> Explain analyze query with buffers... >> >> OK. >> >> >> >> Explain analyze query with timing... >> >> OK. >> >> >> >> Auto commit disabled... >> >> OK. >> >> >> >> Auto commit enabled... >> >> OK. >> >> >> >> Auto rollback enabled... >> >> ERROR >> >> ------- >> >> >> >> Something like: >> >> >> >> ------- >> >> runTest (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >> >> Query tool feature test ... >> >> On demand result set on scrolling... OK. >> >> On demand result set on grid select all... OK. >> >> On demand result set on column select all... OK. >> >> Explain query... OK. >> >> Explain query with verbose... OK. >> >> Explain query with costs... OK. >> >> Explain analyze query... OK. >> >> Explain analyze query with buffers... OK. >> >> Explain analyze query with timing... OK. >> >> Auto commit disabled... OK. >> >> Auto commit enabled... OK. >> >> Auto rollback enabled... ERROR >> >> -------- >> >> >> > Fixed. >> > >> > >> >> >> >> Thanks! >> >> >> >> -- >> >> Dave Page >> >> Blog: http://pgsnake.blogspot.com >> >> Twitter: @pgsnake >> >> >> >> EnterpriseDB UK: http://www.enterprisedb.com >> >> The Enterprise PostgreSQL Company >> > >> > >> >> >> >> -- >> Dave Page >> Blog: http://pgsnake.blogspot.com >> Twitter: @pgsnake >> >> EnterpriseDB UK: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company > > -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgadmin-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgadmin-hackers ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] @ 2017-06-16 06:54 Harshal Dhumal <[email protected]> parent: Dave Page <[email protected]> 0 siblings, 1 reply; 24+ messages in thread From: Harshal Dhumal @ 2017-06-16 06:54 UTC (permalink / raw) To: Dave Page <[email protected]>; +Cc: Harshal Dhumal <[email protected]>; Ashesh Vashi <[email protected]>; pgadmin-hackers Hi Dave, Please find attached patch where I have added timeout of 2 seconds before selecting object menu in connect to server function. Note: Apply this patch on top of previous patch for on demand loading feature. -- *Harshal Dhumal* *Sr. Software Engineer* EnterpriseDB India: http://www.enterprisedb.com The Enterprise PostgreSQL Company On Fri, Jun 16, 2017 at 2:25 AM, Dave Page <[email protected]> wrote: > Sounds good, thanks. > > On Thu, Jun 15, 2017 at 9:54 PM, Harshal Dhumal > <[email protected]> wrote: > > Hi > > > > On Fri, Jun 16, 2017 at 2:07 AM, Dave Page <[email protected]> wrote: > >> > >> Hi > >> > >> On Thu, Jun 15, 2017 at 9:30 PM, Harshal Dhumal > >> <[email protected]> wrote: > >> > Hi Dave, > >> > > >> > Please find attached updated patch. > >> > > >> > On Thu, Jun 15, 2017 at 3:58 PM, Dave Page <[email protected]> wrote: > >> >> > >> >> Hi > >> >> > >> >> On Wed, Jun 14, 2017 at 11:36 PM, Harshal Dhumal > >> >> <[email protected]> wrote: > >> >>> > >> >>> Hi Dave, > >> >>> > >> >>> Please find rebased patch for RM2137. > >> >> > >> >> > >> >> Looking very good. The only issues I see are: > >> >> > >> >> - The row headers should auto-size such that they can display the row > >> >> numbers if the last row was displayed. E.g. if there are 12345 rows > in > >> >> total, then the row header should be sized to display 5 digits. > >> >> > >> > Fixed. > >> > > >> > > >> >> > >> >> - The tests are comprehensive, which is awesome. However, every time > I > >> >> ran > >> >> them, at least one of the feature tests failed. Unfortunately, it > was a > >> >> different one each time. In the last two runs, I got: > >> >> > >> >> ============================================================ > ========== > >> >> ERROR: runTest > >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) > >> >> Query tool feature test > >> >> ------------------------------------------------------------ > ---------- > >> >> Traceback (most recent call last): > >> >> File > >> >> > >> >> "/Users/dpage/git/pgadmin4/web/regression/feature_utils/ > base_feature_test.py", > >> >> line 40, in setUp > >> >> self.before() > >> >> File > >> >> > >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > query_tool_tests.py", > >> >> line 40, in before > >> >> self._connects_to_server() > >> >> File > >> >> > >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > query_tool_tests.py", > >> >> line 144, in _connects_to_server > >> >> self.page.driver.find_element_by_link_text("Create"))\ > >> >> File > >> >> > >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> >> line 319, in find_element_by_link_text > >> >> return self.find_element(by=By.LINK_TEXT, value=link_text) > >> >> File > >> >> > >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> >> line 756, in find_element > >> >> 'value': value})['value'] > >> >> File > >> >> > >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> >> line 238, in execute > >> >> self.error_handler.check_response(response) > >> >> File > >> >> > >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/errorhandler.py", > >> >> line 193, in check_response > >> >> raise exception_class(message, screen, stacktrace) > >> >> NoSuchElementException: Message: no such element: Unable to locate > >> >> element: {"method":"link text","selector":"Create"} > >> >> (Session info: chrome=58.0.3029.110) > >> >> (Driver info: chromedriver=2.29.461585 > >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 > >> >> x86_64) > >> >> > >> > I checked code from _connects_to_server function which is common in > all > >> > features test cases. I didn't find anything wrong with this. If there > is > >> > a > >> > bug in this function then all feature test must fail. > >> > Let me know if you are getting failure consistently in > >> > _connects_to_server > >> > function. > >> > >> I wondered if that one is a race condition. Do we need a short delay > >> before clicking the Object menu? I have seen this occasionally before. > > > > > > OK. In that case let's try putting 1-2 second delay and observer > behaviour. > > I'll send separate patch for this tomorrow as this is not related to on > > demand query result feature or its test cases. > > > >> > >> > >> >> > >> >> > >> >> ============================================================ > ========== > >> >> ERROR: runTest > >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) > >> >> Query tool feature test > >> >> ------------------------------------------------------------ > ---------- > >> >> Traceback (most recent call last): > >> >> File > >> >> > >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > query_tool_tests.py", > >> >> line 119, in runTest > >> >> self._query_tool_auto_rollback_enabled() > >> >> File > >> >> > >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > query_tool_tests.py", > >> >> line 697, in _query_tool_auto_rollback_enabled > >> >> '//div[contains(@class, "sql-editor-message") and > >> >> contains(string(), > >> >> "COMMIT")]' > >> >> File > >> >> > >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> >> line 295, in find_element_by_xpath > >> >> return self.find_element(by=By.XPATH, value=xpath) > >> >> File > >> >> > >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> >> line 756, in find_element > >> >> 'value': value})['value'] > >> >> File > >> >> > >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> >> line 238, in execute > >> >> self.error_handler.check_response(response) > >> >> File > >> >> > >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/errorhandler.py", > >> >> line 193, in check_response > >> >> raise exception_class(message, screen, stacktrace) > >> >> NoSuchElementException: Message: no such element: Unable to locate > >> >> element: {"method":"xpath","selector":"//div[contains(@class, > >> >> "sql-editor-message") and contains(string(), "COMMIT")]"} > >> >> (Session info: chrome=58.0.3029.110) > >> >> (Driver info: chromedriver=2.29.461585 > >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 > >> >> x86_64) > >> >> > >> > I have updated Auto rollback enabled test in this patch. > >> > > >> >> > >> >> Relevant screenshots attached. > >> >> > >> >> - Can you tidy up the regression output a little please? Instead of: > >> >> > >> >> ------- > >> >> runTest (pgadmin.feature_tests.query_tool_tests. > QueryToolFeatureTest) > >> >> Query tool feature test ... On demand result set on scrolling... > >> >> OK. > >> >> > >> >> On demand result set on grid select all... > >> >> OK. > >> >> > >> >> On demand result set on column select all... > >> >> OK. > >> >> > >> >> Explain query... > >> >> OK. > >> >> > >> >> Explain query with verbose... > >> >> OK. > >> >> > >> >> Explain query with costs... > >> >> OK. > >> >> > >> >> Explain analyze query... > >> >> OK. > >> >> > >> >> Explain analyze query with buffers... > >> >> OK. > >> >> > >> >> Explain analyze query with timing... > >> >> OK. > >> >> > >> >> Auto commit disabled... > >> >> OK. > >> >> > >> >> Auto commit enabled... > >> >> OK. > >> >> > >> >> Auto rollback enabled... > >> >> ERROR > >> >> ------- > >> >> > >> >> Something like: > >> >> > >> >> ------- > >> >> runTest (pgadmin.feature_tests.query_tool_tests. > QueryToolFeatureTest) > >> >> Query tool feature test ... > >> >> On demand result set on scrolling... OK. > >> >> On demand result set on grid select all... OK. > >> >> On demand result set on column select all... OK. > >> >> Explain query... OK. > >> >> Explain query with verbose... OK. > >> >> Explain query with costs... OK. > >> >> Explain analyze query... OK. > >> >> Explain analyze query with buffers... OK. > >> >> Explain analyze query with timing... OK. > >> >> Auto commit disabled... OK. > >> >> Auto commit enabled... OK. > >> >> Auto rollback enabled... ERROR > >> >> -------- > >> >> > >> > Fixed. > >> > > >> > > >> >> > >> >> Thanks! > >> >> > >> >> -- > >> >> Dave Page > >> >> Blog: http://pgsnake.blogspot.com > >> >> Twitter: @pgsnake > >> >> > >> >> EnterpriseDB UK: http://www.enterprisedb.com > >> >> The Enterprise PostgreSQL Company > >> > > >> > > >> > >> > >> > >> -- > >> Dave Page > >> Blog: http://pgsnake.blogspot.com > >> Twitter: @pgsnake > >> > >> EnterpriseDB UK: http://www.enterprisedb.com > >> The Enterprise PostgreSQL Company > > > > > > > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > -- Sent via pgadmin-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgadmin-hackers Attachments: [text/x-patch] timeout_before_object_menu_clcik_in_connect_to_server.patch (3.8K, 3-timeout_before_object_menu_clcik_in_connect_to_server.patch) download | inline diff: diff --git a/web/pgadmin/feature_tests/connect_to_server_feature_test.py b/web/pgadmin/feature_tests/connect_to_server_feature_test.py index e8de161..6bec4ac 100644 --- a/web/pgadmin/feature_tests/connect_to_server_feature_test.py +++ b/web/pgadmin/feature_tests/connect_to_server_feature_test.py @@ -7,6 +7,7 @@ # ########################################################################## +import time from selenium.webdriver import ActionChains import config as app_config @@ -53,6 +54,7 @@ class ConnectsToServerFeatureTest(BaseFeatureTest): def _connects_to_server(self): self.page.find_by_xpath("//*[@class='aciTreeText' and .='Servers']").click() + time.sleep(2) self.page.driver.find_element_by_link_text("Object").click() ActionChains(self.page.driver) \ .move_to_element(self.page.driver.find_element_by_link_text("Create")) \ diff --git a/web/pgadmin/feature_tests/pg_datatype_validation_test.py b/web/pgadmin/feature_tests/pg_datatype_validation_test.py index b72f727..6b85848 100644 --- a/web/pgadmin/feature_tests/pg_datatype_validation_test.py +++ b/web/pgadmin/feature_tests/pg_datatype_validation_test.py @@ -6,6 +6,7 @@ # This software is released under the PostgreSQL Licence # ########################################################################## +import time from selenium.webdriver import ActionChains from selenium.common.exceptions import TimeoutException from selenium.webdriver.support.ui import WebDriverWait @@ -56,6 +57,7 @@ class PGDataypeFeatureTest(BaseFeatureTest): self.page.find_by_xpath( "//*[@class='aciTreeText' and .='Servers']" ).click() + time.sleep(2) self.page.driver.find_element_by_link_text("Object").click() ActionChains(self.page.driver) \ .move_to_element( diff --git a/web/pgadmin/feature_tests/query_tool_tests.py b/web/pgadmin/feature_tests/query_tool_tests.py index 1861599..c5817e3 100644 --- a/web/pgadmin/feature_tests/query_tool_tests.py +++ b/web/pgadmin/feature_tests/query_tool_tests.py @@ -138,6 +138,7 @@ class QueryToolFeatureTest(BaseFeatureTest): def _connects_to_server(self): self.page.find_by_xpath( "//*[@class='aciTreeText' and .='Servers']").click() + time.sleep(2) self.page.driver.find_element_by_link_text("Object").click() ActionChains(self.page.driver) \ .move_to_element( diff --git a/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py b/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py index a0df44e..75b4222 100644 --- a/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py +++ b/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py @@ -75,6 +75,7 @@ class CheckForXssFeatureTest(BaseFeatureTest): def _connects_to_server(self): self.page.find_by_xpath("//*[@class='aciTreeText' and .='Servers']").click() + time.sleep(2) self.page.driver.find_element_by_link_text("Object").click() ActionChains(self.page.driver) \ .move_to_element(self.page.driver.find_element_by_link_text("Create")) \ diff --git a/web/pgadmin/feature_tests/xss_checks_pgadmin_debugger_test.py b/web/pgadmin/feature_tests/xss_checks_pgadmin_debugger_test.py index 094dfed..e847040 100644 --- a/web/pgadmin/feature_tests/xss_checks_pgadmin_debugger_test.py +++ b/web/pgadmin/feature_tests/xss_checks_pgadmin_debugger_test.py @@ -42,6 +42,7 @@ class CheckDebuggerForXssFeatureTest(BaseFeatureTest): def _connects_to_server(self): self.page.find_by_xpath("//*[@class='aciTreeText' and .='Servers']").click() + time.sleep(2) self.page.driver.find_element_by_link_text("Object").click() ActionChains(self.page.driver) \ .move_to_element(self.page.driver.find_element_by_link_text("Create")) \ ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] @ 2017-06-16 09:55 Dave Page <[email protected]> parent: Harshal Dhumal <[email protected]> 0 siblings, 1 reply; 24+ messages in thread From: Dave Page @ 2017-06-16 09:55 UTC (permalink / raw) To: Harshal Dhumal <[email protected]>; +Cc: Harshal Dhumal <[email protected]>; Ashesh Vashi <[email protected]>; pgadmin-hackers Hi, That's better - the failures are far less random now :-). I got the following two though, on both PG and EPAS 9.5: ====================================================================== ERROR: runTest (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) Query tool feature test ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", line 95, in runTest self._query_tool_explain_analyze_buffers() File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", line 443, in _query_tool_explain_analyze_buffers canvas.find_element_by_xpath("//*[contains(string(), 'Shared Read Blocks')]") File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", line 260, in find_element_by_xpath return self.find_element(by=By.XPATH, value=xpath) File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", line 508, in find_element {"using": by, "value": value})['value'] File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", line 491, in _execute return self._parent.execute(command, params) File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 238, in execute self.error_handler.check_response(response) File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 193, in check_response raise exception_class(message, screen, stacktrace) NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[contains(string(), 'Shared Read Blocks')]"} (Session info: chrome=58.0.3029.110) (Driver info: chromedriver=2.29.461585 (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 x86_64) ====================================================================== ERROR: runTest (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest) Validate Insert, Update operations in View data with given test data ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py", line 104, in runTest self._add_row() File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py", line 255, in _add_row self._update_cell(cell_xpath, config_data[str(idx)]) File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py", line 164, in _update_cell cell_el = self.page.find_by_xpath(xpath) File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 122, in find_by_xpath return self.wait_for_element(lambda driver: driver.find_element_by_xpath(xpath)) File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 205, in wait_for_element return self._wait_for("element to exist", element_if_it_exists) File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 255, in _wait_for "Timed out waiting for " + waiting_for_message) File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/support/wait.py", line 71, in until value = method(self._driver) File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py", line 200, in element_if_it_exists if element.is_displayed() and element.is_enabled(): File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", line 358, in is_displayed return self._execute(Command.IS_ELEMENT_DISPLAYED)['value'] File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", line 491, in _execute return self._parent.execute(command, params) File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 238, in execute self.error_handler.check_response(response) File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 193, in check_response raise exception_class(message, screen, stacktrace) StaleElementReferenceException: Message: stale element reference: element is not attached to the page document (Session info: chrome=58.0.3029.110) (Driver info: chromedriver=2.29.461585 (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 x86_64) On 9.4 everything passes. On 9.6, I get one failure: ====================================================================== FAIL: runTest (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest) Validate Insert, Update operations in View data with given test data ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py", line 105, in runTest self._verify_row_data(True) File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py", line 282, in _verify_row_data self.assertEquals(cells[idx], config_data[str(idx)][1]) AssertionError: u'[default]' != u'1' - [default] + 1 Screenshots attached. On Fri, Jun 16, 2017 at 7:54 AM, Harshal Dhumal <[email protected]> wrote: > Hi Dave, > > Please find attached patch where I have added timeout of 2 seconds before > selecting object menu in connect to server function. > > Note: Apply this patch on top of previous patch for on demand loading > feature. > > > -- > Harshal Dhumal > Sr. Software Engineer > > EnterpriseDB India: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > > On Fri, Jun 16, 2017 at 2:25 AM, Dave Page <[email protected]> wrote: >> >> Sounds good, thanks. >> >> On Thu, Jun 15, 2017 at 9:54 PM, Harshal Dhumal >> <[email protected]> wrote: >> > Hi >> > >> > On Fri, Jun 16, 2017 at 2:07 AM, Dave Page <[email protected]> wrote: >> >> >> >> Hi >> >> >> >> On Thu, Jun 15, 2017 at 9:30 PM, Harshal Dhumal >> >> <[email protected]> wrote: >> >> > Hi Dave, >> >> > >> >> > Please find attached updated patch. >> >> > >> >> > On Thu, Jun 15, 2017 at 3:58 PM, Dave Page <[email protected]> wrote: >> >> >> >> >> >> Hi >> >> >> >> >> >> On Wed, Jun 14, 2017 at 11:36 PM, Harshal Dhumal >> >> >> <[email protected]> wrote: >> >> >>> >> >> >>> Hi Dave, >> >> >>> >> >> >>> Please find rebased patch for RM2137. >> >> >> >> >> >> >> >> >> Looking very good. The only issues I see are: >> >> >> >> >> >> - The row headers should auto-size such that they can display the >> >> >> row >> >> >> numbers if the last row was displayed. E.g. if there are 12345 rows >> >> >> in >> >> >> total, then the row header should be sized to display 5 digits. >> >> >> >> >> > Fixed. >> >> > >> >> > >> >> >> >> >> >> - The tests are comprehensive, which is awesome. However, every time >> >> >> I >> >> >> ran >> >> >> them, at least one of the feature tests failed. Unfortunately, it >> >> >> was a >> >> >> different one each time. In the last two runs, I got: >> >> >> >> >> >> >> >> >> ====================================================================== >> >> >> ERROR: runTest >> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >> >> >> Query tool feature test >> >> >> >> >> >> ---------------------------------------------------------------------- >> >> >> Traceback (most recent call last): >> >> >> File >> >> >> >> >> >> >> >> >> "/Users/dpage/git/pgadmin4/web/regression/feature_utils/base_feature_test.py", >> >> >> line 40, in setUp >> >> >> self.before() >> >> >> File >> >> >> >> >> >> >> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", >> >> >> line 40, in before >> >> >> self._connects_to_server() >> >> >> File >> >> >> >> >> >> >> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", >> >> >> line 144, in _connects_to_server >> >> >> self.page.driver.find_element_by_link_text("Create"))\ >> >> >> File >> >> >> >> >> >> >> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> >> >> line 319, in find_element_by_link_text >> >> >> return self.find_element(by=By.LINK_TEXT, value=link_text) >> >> >> File >> >> >> >> >> >> >> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> >> >> line 756, in find_element >> >> >> 'value': value})['value'] >> >> >> File >> >> >> >> >> >> >> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> >> >> line 238, in execute >> >> >> self.error_handler.check_response(response) >> >> >> File >> >> >> >> >> >> >> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", >> >> >> line 193, in check_response >> >> >> raise exception_class(message, screen, stacktrace) >> >> >> NoSuchElementException: Message: no such element: Unable to locate >> >> >> element: {"method":"link text","selector":"Create"} >> >> >> (Session info: chrome=58.0.3029.110) >> >> >> (Driver info: chromedriver=2.29.461585 >> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 >> >> >> x86_64) >> >> >> >> >> > I checked code from _connects_to_server function which is common in >> >> > all >> >> > features test cases. I didn't find anything wrong with this. If there >> >> > is >> >> > a >> >> > bug in this function then all feature test must fail. >> >> > Let me know if you are getting failure consistently in >> >> > _connects_to_server >> >> > function. >> >> >> >> I wondered if that one is a race condition. Do we need a short delay >> >> before clicking the Object menu? I have seen this occasionally before. >> > >> > >> > OK. In that case let's try putting 1-2 second delay and observer >> > behaviour. >> > I'll send separate patch for this tomorrow as this is not related to on >> > demand query result feature or its test cases. >> > >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> ====================================================================== >> >> >> ERROR: runTest >> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >> >> >> Query tool feature test >> >> >> >> >> >> ---------------------------------------------------------------------- >> >> >> Traceback (most recent call last): >> >> >> File >> >> >> >> >> >> >> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", >> >> >> line 119, in runTest >> >> >> self._query_tool_auto_rollback_enabled() >> >> >> File >> >> >> >> >> >> >> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", >> >> >> line 697, in _query_tool_auto_rollback_enabled >> >> >> '//div[contains(@class, "sql-editor-message") and >> >> >> contains(string(), >> >> >> "COMMIT")]' >> >> >> File >> >> >> >> >> >> >> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> >> >> line 295, in find_element_by_xpath >> >> >> return self.find_element(by=By.XPATH, value=xpath) >> >> >> File >> >> >> >> >> >> >> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> >> >> line 756, in find_element >> >> >> 'value': value})['value'] >> >> >> File >> >> >> >> >> >> >> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> >> >> line 238, in execute >> >> >> self.error_handler.check_response(response) >> >> >> File >> >> >> >> >> >> >> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", >> >> >> line 193, in check_response >> >> >> raise exception_class(message, screen, stacktrace) >> >> >> NoSuchElementException: Message: no such element: Unable to locate >> >> >> element: {"method":"xpath","selector":"//div[contains(@class, >> >> >> "sql-editor-message") and contains(string(), "COMMIT")]"} >> >> >> (Session info: chrome=58.0.3029.110) >> >> >> (Driver info: chromedriver=2.29.461585 >> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 >> >> >> x86_64) >> >> >> >> >> > I have updated Auto rollback enabled test in this patch. >> >> > >> >> >> >> >> >> Relevant screenshots attached. >> >> >> >> >> >> - Can you tidy up the regression output a little please? Instead of: >> >> >> >> >> >> ------- >> >> >> runTest >> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >> >> >> Query tool feature test ... On demand result set on scrolling... >> >> >> OK. >> >> >> >> >> >> On demand result set on grid select all... >> >> >> OK. >> >> >> >> >> >> On demand result set on column select all... >> >> >> OK. >> >> >> >> >> >> Explain query... >> >> >> OK. >> >> >> >> >> >> Explain query with verbose... >> >> >> OK. >> >> >> >> >> >> Explain query with costs... >> >> >> OK. >> >> >> >> >> >> Explain analyze query... >> >> >> OK. >> >> >> >> >> >> Explain analyze query with buffers... >> >> >> OK. >> >> >> >> >> >> Explain analyze query with timing... >> >> >> OK. >> >> >> >> >> >> Auto commit disabled... >> >> >> OK. >> >> >> >> >> >> Auto commit enabled... >> >> >> OK. >> >> >> >> >> >> Auto rollback enabled... >> >> >> ERROR >> >> >> ------- >> >> >> >> >> >> Something like: >> >> >> >> >> >> ------- >> >> >> runTest >> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >> >> >> Query tool feature test ... >> >> >> On demand result set on scrolling... OK. >> >> >> On demand result set on grid select all... OK. >> >> >> On demand result set on column select all... OK. >> >> >> Explain query... OK. >> >> >> Explain query with verbose... OK. >> >> >> Explain query with costs... OK. >> >> >> Explain analyze query... OK. >> >> >> Explain analyze query with buffers... OK. >> >> >> Explain analyze query with timing... OK. >> >> >> Auto commit disabled... OK. >> >> >> Auto commit enabled... OK. >> >> >> Auto rollback enabled... ERROR >> >> >> -------- >> >> >> >> >> > Fixed. >> >> > >> >> > >> >> >> >> >> >> Thanks! >> >> >> >> >> >> -- >> >> >> Dave Page >> >> >> Blog: http://pgsnake.blogspot.com >> >> >> Twitter: @pgsnake >> >> >> >> >> >> EnterpriseDB UK: http://www.enterprisedb.com >> >> >> The Enterprise PostgreSQL Company >> >> > >> >> > >> >> >> >> >> >> >> >> -- >> >> Dave Page >> >> Blog: http://pgsnake.blogspot.com >> >> Twitter: @pgsnake >> >> >> >> EnterpriseDB UK: http://www.enterprisedb.com >> >> The Enterprise PostgreSQL Company >> > >> > >> >> >> >> -- >> Dave Page >> Blog: http://pgsnake.blogspot.com >> Twitter: @pgsnake >> >> EnterpriseDB UK: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company > > -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgadmin-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgadmin-hackers Attachments: [application/zip] screenshots.zip (717.8K, 2-screenshots.zip) download ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: [pgadmin-hackers] Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] @ 2017-06-23 15:30 Dave Page <[email protected]> parent: Dave Page <[email protected]> 0 siblings, 1 reply; 24+ messages in thread From: Dave Page @ 2017-06-23 15:30 UTC (permalink / raw) To: Harshal Dhumal <[email protected]>; +Cc: Harshal Dhumal <[email protected]>; Ashesh Vashi <[email protected]>; pgadmin-hackers Hi Harshal, When can we expect an updated version of this patch? I think it's important to get this into the next release. Thanks! On Fri, Jun 16, 2017 at 10:55 AM, Dave Page <[email protected]> wrote: > Hi, > > That's better - the failures are far less random now :-). I got the > following two though, on both PG and EPAS 9.5: > > ====================================================================== > ERROR: runTest (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) > Query tool feature test > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", > line 95, in runTest > self._query_tool_explain_analyze_buffers() > File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", > line 443, in _query_tool_explain_analyze_buffers > canvas.find_element_by_xpath("//*[contains(string(), 'Shared Read > Blocks')]") > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", > line 260, in find_element_by_xpath > return self.find_element(by=By.XPATH, value=xpath) > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", > line 508, in find_element > {"using": by, "value": value})['value'] > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", > line 491, in _execute > return self._parent.execute(command, params) > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", > line 238, in execute > self.error_handler.check_response(response) > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", > line 193, in check_response > raise exception_class(message, screen, stacktrace) > NoSuchElementException: Message: no such element: Unable to locate > element: {"method":"xpath","selector":"//*[contains(string(), 'Shared > Read Blocks')]"} > (Session info: chrome=58.0.3029.110) > (Driver info: chromedriver=2.29.461585 > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 > x86_64) > > > ====================================================================== > ERROR: runTest (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest) > Validate Insert, Update operations in View data with given test data > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py", > line 104, in runTest > self._add_row() > File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py", > line 255, in _add_row > self._update_cell(cell_xpath, config_data[str(idx)]) > File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py", > line 164, in _update_cell > cell_el = self.page.find_by_xpath(xpath) > File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py", > line 122, in find_by_xpath > return self.wait_for_element(lambda driver: > driver.find_element_by_xpath(xpath)) > File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py", > line 205, in wait_for_element > return self._wait_for("element to exist", element_if_it_exists) > File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py", > line 255, in _wait_for > "Timed out waiting for " + waiting_for_message) > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/support/wait.py", > line 71, in until > value = method(self._driver) > File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py", > line 200, in element_if_it_exists > if element.is_displayed() and element.is_enabled(): > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", > line 358, in is_displayed > return self._execute(Command.IS_ELEMENT_DISPLAYED)['value'] > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", > line 491, in _execute > return self._parent.execute(command, params) > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", > line 238, in execute > self.error_handler.check_response(response) > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", > line 193, in check_response > raise exception_class(message, screen, stacktrace) > StaleElementReferenceException: Message: stale element reference: > element is not attached to the page document > (Session info: chrome=58.0.3029.110) > (Driver info: chromedriver=2.29.461585 > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 > x86_64) > > > On 9.4 everything passes. On 9.6, I get one failure: > > ====================================================================== > FAIL: runTest (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest) > Validate Insert, Update operations in View data with given test data > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py", > line 105, in runTest > self._verify_row_data(True) > File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py", > line 282, in _verify_row_data > self.assertEquals(cells[idx], config_data[str(idx)][1]) > AssertionError: u'[default]' != u'1' > - [default] > + 1 > > Screenshots attached. > > On Fri, Jun 16, 2017 at 7:54 AM, Harshal Dhumal > <[email protected]> wrote: >> Hi Dave, >> >> Please find attached patch where I have added timeout of 2 seconds before >> selecting object menu in connect to server function. >> >> Note: Apply this patch on top of previous patch for on demand loading >> feature. >> >> >> -- >> Harshal Dhumal >> Sr. Software Engineer >> >> EnterpriseDB India: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> >> On Fri, Jun 16, 2017 at 2:25 AM, Dave Page <[email protected]> wrote: >>> >>> Sounds good, thanks. >>> >>> On Thu, Jun 15, 2017 at 9:54 PM, Harshal Dhumal >>> <[email protected]> wrote: >>> > Hi >>> > >>> > On Fri, Jun 16, 2017 at 2:07 AM, Dave Page <[email protected]> wrote: >>> >> >>> >> Hi >>> >> >>> >> On Thu, Jun 15, 2017 at 9:30 PM, Harshal Dhumal >>> >> <[email protected]> wrote: >>> >> > Hi Dave, >>> >> > >>> >> > Please find attached updated patch. >>> >> > >>> >> > On Thu, Jun 15, 2017 at 3:58 PM, Dave Page <[email protected]> wrote: >>> >> >> >>> >> >> Hi >>> >> >> >>> >> >> On Wed, Jun 14, 2017 at 11:36 PM, Harshal Dhumal >>> >> >> <[email protected]> wrote: >>> >> >>> >>> >> >>> Hi Dave, >>> >> >>> >>> >> >>> Please find rebased patch for RM2137. >>> >> >> >>> >> >> >>> >> >> Looking very good. The only issues I see are: >>> >> >> >>> >> >> - The row headers should auto-size such that they can display the >>> >> >> row >>> >> >> numbers if the last row was displayed. E.g. if there are 12345 rows >>> >> >> in >>> >> >> total, then the row header should be sized to display 5 digits. >>> >> >> >>> >> > Fixed. >>> >> > >>> >> > >>> >> >> >>> >> >> - The tests are comprehensive, which is awesome. However, every time >>> >> >> I >>> >> >> ran >>> >> >> them, at least one of the feature tests failed. Unfortunately, it >>> >> >> was a >>> >> >> different one each time. In the last two runs, I got: >>> >> >> >>> >> >> >>> >> >> ====================================================================== >>> >> >> ERROR: runTest >>> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >>> >> >> Query tool feature test >>> >> >> >>> >> >> ---------------------------------------------------------------------- >>> >> >> Traceback (most recent call last): >>> >> >> File >>> >> >> >>> >> >> >>> >> >> "/Users/dpage/git/pgadmin4/web/regression/feature_utils/base_feature_test.py", >>> >> >> line 40, in setUp >>> >> >> self.before() >>> >> >> File >>> >> >> >>> >> >> >>> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", >>> >> >> line 40, in before >>> >> >> self._connects_to_server() >>> >> >> File >>> >> >> >>> >> >> >>> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", >>> >> >> line 144, in _connects_to_server >>> >> >> self.page.driver.find_element_by_link_text("Create"))\ >>> >> >> File >>> >> >> >>> >> >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >>> >> >> line 319, in find_element_by_link_text >>> >> >> return self.find_element(by=By.LINK_TEXT, value=link_text) >>> >> >> File >>> >> >> >>> >> >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >>> >> >> line 756, in find_element >>> >> >> 'value': value})['value'] >>> >> >> File >>> >> >> >>> >> >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >>> >> >> line 238, in execute >>> >> >> self.error_handler.check_response(response) >>> >> >> File >>> >> >> >>> >> >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", >>> >> >> line 193, in check_response >>> >> >> raise exception_class(message, screen, stacktrace) >>> >> >> NoSuchElementException: Message: no such element: Unable to locate >>> >> >> element: {"method":"link text","selector":"Create"} >>> >> >> (Session info: chrome=58.0.3029.110) >>> >> >> (Driver info: chromedriver=2.29.461585 >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 >>> >> >> x86_64) >>> >> >> >>> >> > I checked code from _connects_to_server function which is common in >>> >> > all >>> >> > features test cases. I didn't find anything wrong with this. If there >>> >> > is >>> >> > a >>> >> > bug in this function then all feature test must fail. >>> >> > Let me know if you are getting failure consistently in >>> >> > _connects_to_server >>> >> > function. >>> >> >>> >> I wondered if that one is a race condition. Do we need a short delay >>> >> before clicking the Object menu? I have seen this occasionally before. >>> > >>> > >>> > OK. In that case let's try putting 1-2 second delay and observer >>> > behaviour. >>> > I'll send separate patch for this tomorrow as this is not related to on >>> > demand query result feature or its test cases. >>> > >>> >> >>> >> >>> >> >> >>> >> >> >>> >> >> >>> >> >> ====================================================================== >>> >> >> ERROR: runTest >>> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >>> >> >> Query tool feature test >>> >> >> >>> >> >> ---------------------------------------------------------------------- >>> >> >> Traceback (most recent call last): >>> >> >> File >>> >> >> >>> >> >> >>> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", >>> >> >> line 119, in runTest >>> >> >> self._query_tool_auto_rollback_enabled() >>> >> >> File >>> >> >> >>> >> >> >>> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", >>> >> >> line 697, in _query_tool_auto_rollback_enabled >>> >> >> '//div[contains(@class, "sql-editor-message") and >>> >> >> contains(string(), >>> >> >> "COMMIT")]' >>> >> >> File >>> >> >> >>> >> >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >>> >> >> line 295, in find_element_by_xpath >>> >> >> return self.find_element(by=By.XPATH, value=xpath) >>> >> >> File >>> >> >> >>> >> >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >>> >> >> line 756, in find_element >>> >> >> 'value': value})['value'] >>> >> >> File >>> >> >> >>> >> >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >>> >> >> line 238, in execute >>> >> >> self.error_handler.check_response(response) >>> >> >> File >>> >> >> >>> >> >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", >>> >> >> line 193, in check_response >>> >> >> raise exception_class(message, screen, stacktrace) >>> >> >> NoSuchElementException: Message: no such element: Unable to locate >>> >> >> element: {"method":"xpath","selector":"//div[contains(@class, >>> >> >> "sql-editor-message") and contains(string(), "COMMIT")]"} >>> >> >> (Session info: chrome=58.0.3029.110) >>> >> >> (Driver info: chromedriver=2.29.461585 >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 >>> >> >> x86_64) >>> >> >> >>> >> > I have updated Auto rollback enabled test in this patch. >>> >> > >>> >> >> >>> >> >> Relevant screenshots attached. >>> >> >> >>> >> >> - Can you tidy up the regression output a little please? Instead of: >>> >> >> >>> >> >> ------- >>> >> >> runTest >>> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >>> >> >> Query tool feature test ... On demand result set on scrolling... >>> >> >> OK. >>> >> >> >>> >> >> On demand result set on grid select all... >>> >> >> OK. >>> >> >> >>> >> >> On demand result set on column select all... >>> >> >> OK. >>> >> >> >>> >> >> Explain query... >>> >> >> OK. >>> >> >> >>> >> >> Explain query with verbose... >>> >> >> OK. >>> >> >> >>> >> >> Explain query with costs... >>> >> >> OK. >>> >> >> >>> >> >> Explain analyze query... >>> >> >> OK. >>> >> >> >>> >> >> Explain analyze query with buffers... >>> >> >> OK. >>> >> >> >>> >> >> Explain analyze query with timing... >>> >> >> OK. >>> >> >> >>> >> >> Auto commit disabled... >>> >> >> OK. >>> >> >> >>> >> >> Auto commit enabled... >>> >> >> OK. >>> >> >> >>> >> >> Auto rollback enabled... >>> >> >> ERROR >>> >> >> ------- >>> >> >> >>> >> >> Something like: >>> >> >> >>> >> >> ------- >>> >> >> runTest >>> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >>> >> >> Query tool feature test ... >>> >> >> On demand result set on scrolling... OK. >>> >> >> On demand result set on grid select all... OK. >>> >> >> On demand result set on column select all... OK. >>> >> >> Explain query... OK. >>> >> >> Explain query with verbose... OK. >>> >> >> Explain query with costs... OK. >>> >> >> Explain analyze query... OK. >>> >> >> Explain analyze query with buffers... OK. >>> >> >> Explain analyze query with timing... OK. >>> >> >> Auto commit disabled... OK. >>> >> >> Auto commit enabled... OK. >>> >> >> Auto rollback enabled... ERROR >>> >> >> -------- >>> >> >> >>> >> > Fixed. >>> >> > >>> >> > >>> >> >> >>> >> >> Thanks! >>> >> >> >>> >> >> -- >>> >> >> Dave Page >>> >> >> Blog: http://pgsnake.blogspot.com >>> >> >> Twitter: @pgsnake >>> >> >> >>> >> >> EnterpriseDB UK: http://www.enterprisedb.com >>> >> >> The Enterprise PostgreSQL Company >>> >> > >>> >> > >>> >> >>> >> >>> >> >>> >> -- >>> >> Dave Page >>> >> Blog: http://pgsnake.blogspot.com >>> >> Twitter: @pgsnake >>> >> >>> >> EnterpriseDB UK: http://www.enterprisedb.com >>> >> The Enterprise PostgreSQL Company >>> > >>> > >>> >>> >>> >>> -- >>> Dave Page >>> Blog: http://pgsnake.blogspot.com >>> Twitter: @pgsnake >>> >>> EnterpriseDB UK: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >> >> > > > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: [pgadmin-hackers] Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] @ 2017-06-26 09:16 Harshal Dhumal <[email protected]> parent: Dave Page <[email protected]> 0 siblings, 1 reply; 24+ messages in thread From: Harshal Dhumal @ 2017-06-26 09:16 UTC (permalink / raw) To: Dave Page <[email protected]>; +Cc: Harshal Dhumal <[email protected]>; Ashesh Vashi <[email protected]>; pgadmin-hackers Hi Dave, Please find updated rebased patch for RM2137 On Fri, Jun 23, 2017 at 9:00 PM, Dave Page <[email protected]> wrote: > Hi Harshal, > > When can we expect an updated version of this patch? I think it's > important to get this into the next release. > > Thanks! > > On Fri, Jun 16, 2017 at 10:55 AM, Dave Page <[email protected]> wrote: > > Hi, > > > > That's better - the failures are far less random now :-). I got the > > following two though, on both PG and EPAS 9.5: > > > > ====================================================================== > > ERROR: runTest (pgadmin.feature_tests.query_tool_tests. > QueryToolFeatureTest) > > Query tool feature test > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > query_tool_tests.py", > > line 95, in runTest > > self._query_tool_explain_analyze_buffers() > > File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > query_tool_tests.py", > > line 443, in _query_tool_explain_analyze_buffers > > canvas.find_element_by_xpath("//*[contains(string(), 'Shared Read > > Blocks')]") > > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webelement.py", > > line 260, in find_element_by_xpath > > return self.find_element(by=By.XPATH, value=xpath) > > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webelement.py", > > line 508, in find_element > > {"using": by, "value": value})['value'] > > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webelement.py", > > line 491, in _execute > > return self._parent.execute(command, params) > > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > > line 238, in execute > > self.error_handler.check_response(response) > > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/errorhandler.py", > > line 193, in check_response > > raise exception_class(message, screen, stacktrace) > > NoSuchElementException: Message: no such element: Unable to locate > > element: {"method":"xpath","selector":"//*[contains(string(), 'Shared > > Read Blocks')]"} > > (Session info: chrome=58.0.3029.110) > > (Driver info: chromedriver=2.29.461585 > > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 > > x86_64) > > > > > > ====================================================================== > > ERROR: runTest (pgadmin.feature_tests.view_data_dml_queries. > CheckForViewDataTest) > > Validate Insert, Update operations in View data with given test data > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > view_data_dml_queries.py", > > line 104, in runTest > > self._add_row() > > File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > view_data_dml_queries.py", > > line 255, in _add_row > > self._update_cell(cell_xpath, config_data[str(idx)]) > > File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > view_data_dml_queries.py", > > line 164, in _update_cell > > cell_el = self.page.find_by_xpath(xpath) > > File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/ > pgadmin_page.py", > > line 122, in find_by_xpath > > return self.wait_for_element(lambda driver: > > driver.find_element_by_xpath(xpath)) > > File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/ > pgadmin_page.py", > > line 205, in wait_for_element > > return self._wait_for("element to exist", element_if_it_exists) > > File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/ > pgadmin_page.py", > > line 255, in _wait_for > > "Timed out waiting for " + waiting_for_message) > > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/support/wait.py", > > line 71, in until > > value = method(self._driver) > > File "/Users/dpage/git/pgadmin4/web/regression/feature_utils/ > pgadmin_page.py", > > line 200, in element_if_it_exists > > if element.is_displayed() and element.is_enabled(): > > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webelement.py", > > line 358, in is_displayed > > return self._execute(Command.IS_ELEMENT_DISPLAYED)['value'] > > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webelement.py", > > line 491, in _execute > > return self._parent.execute(command, params) > > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > > line 238, in execute > > self.error_handler.check_response(response) > > File "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/errorhandler.py", > > line 193, in check_response > > raise exception_class(message, screen, stacktrace) > > StaleElementReferenceException: Message: stale element reference: > > element is not attached to the page document > > (Session info: chrome=58.0.3029.110) > > (Driver info: chromedriver=2.29.461585 > > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 > > x86_64) > > > I checked my feature test cases for any database server version/type (PG, EPAS) specific failures and I found that test cases are working fine on all of them. However I have slightly modified test case to overcome above mentioned failures. > > > > On 9.4 everything passes. On 9.6, I get one failure: > > > > ====================================================================== > > FAIL: runTest (pgadmin.feature_tests.view_data_dml_queries. > CheckForViewDataTest) > > Validate Insert, Update operations in View data with given test data > > ---------------------------------------------------------------------- > > Traceback (most recent call last): > > File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > view_data_dml_queries.py", > > line 105, in runTest > > self._verify_row_data(True) > > File "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > view_data_dml_queries.py", > > line 282, in _verify_row_data > > self.assertEquals(cells[idx], config_data[str(idx)][1]) > > AssertionError: u'[default]' != u'1' > > - [default] > > + 1 > > > I guess Khushboo has fixed this recently. > > Screenshots attached. > > > > On Fri, Jun 16, 2017 at 7:54 AM, Harshal Dhumal > > <[email protected]> wrote: > >> Hi Dave, > >> > >> Please find attached patch where I have added timeout of 2 seconds > before > >> selecting object menu in connect to server function. > >> > >> Note: Apply this patch on top of previous patch for on demand loading > >> feature. > >> > >> > >> -- > >> Harshal Dhumal > >> Sr. Software Engineer > >> > >> EnterpriseDB India: http://www.enterprisedb.com > >> The Enterprise PostgreSQL Company > >> > >> On Fri, Jun 16, 2017 at 2:25 AM, Dave Page <[email protected]> wrote: > >>> > >>> Sounds good, thanks. > >>> > >>> On Thu, Jun 15, 2017 at 9:54 PM, Harshal Dhumal > >>> <[email protected]> wrote: > >>> > Hi > >>> > > >>> > On Fri, Jun 16, 2017 at 2:07 AM, Dave Page <[email protected]> > wrote: > >>> >> > >>> >> Hi > >>> >> > >>> >> On Thu, Jun 15, 2017 at 9:30 PM, Harshal Dhumal > >>> >> <[email protected]> wrote: > >>> >> > Hi Dave, > >>> >> > > >>> >> > Please find attached updated patch. > >>> >> > > >>> >> > On Thu, Jun 15, 2017 at 3:58 PM, Dave Page <[email protected]> > wrote: > >>> >> >> > >>> >> >> Hi > >>> >> >> > >>> >> >> On Wed, Jun 14, 2017 at 11:36 PM, Harshal Dhumal > >>> >> >> <[email protected]> wrote: > >>> >> >>> > >>> >> >>> Hi Dave, > >>> >> >>> > >>> >> >>> Please find rebased patch for RM2137. > >>> >> >> > >>> >> >> > >>> >> >> Looking very good. The only issues I see are: > >>> >> >> > >>> >> >> - The row headers should auto-size such that they can display the > >>> >> >> row > >>> >> >> numbers if the last row was displayed. E.g. if there are 12345 > rows > >>> >> >> in > >>> >> >> total, then the row header should be sized to display 5 digits. > >>> >> >> > >>> >> > Fixed. > >>> >> > > >>> >> > > >>> >> >> > >>> >> >> - The tests are comprehensive, which is awesome. However, every > time > >>> >> >> I > >>> >> >> ran > >>> >> >> them, at least one of the feature tests failed. Unfortunately, it > >>> >> >> was a > >>> >> >> different one each time. In the last two runs, I got: > >>> >> >> > >>> >> >> > >>> >> >> ============================================================ > ========== > >>> >> >> ERROR: runTest > >>> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) > >>> >> >> Query tool feature test > >>> >> >> > >>> >> >> ------------------------------------------------------------ > ---------- > >>> >> >> Traceback (most recent call last): > >>> >> >> File > >>> >> >> > >>> >> >> > >>> >> >> "/Users/dpage/git/pgadmin4/web/regression/feature_utils/ > base_feature_test.py", > >>> >> >> line 40, in setUp > >>> >> >> self.before() > >>> >> >> File > >>> >> >> > >>> >> >> > >>> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > query_tool_tests.py", > >>> >> >> line 40, in before > >>> >> >> self._connects_to_server() > >>> >> >> File > >>> >> >> > >>> >> >> > >>> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > query_tool_tests.py", > >>> >> >> line 144, in _connects_to_server > >>> >> >> self.page.driver.find_element_by_link_text("Create"))\ > >>> >> >> File > >>> >> >> > >>> >> >> > >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >>> >> >> line 319, in find_element_by_link_text > >>> >> >> return self.find_element(by=By.LINK_TEXT, value=link_text) > >>> >> >> File > >>> >> >> > >>> >> >> > >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >>> >> >> line 756, in find_element > >>> >> >> 'value': value})['value'] > >>> >> >> File > >>> >> >> > >>> >> >> > >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >>> >> >> line 238, in execute > >>> >> >> self.error_handler.check_response(response) > >>> >> >> File > >>> >> >> > >>> >> >> > >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/errorhandler.py", > >>> >> >> line 193, in check_response > >>> >> >> raise exception_class(message, screen, stacktrace) > >>> >> >> NoSuchElementException: Message: no such element: Unable to > locate > >>> >> >> element: {"method":"link text","selector":"Create"} > >>> >> >> (Session info: chrome=58.0.3029.110) > >>> >> >> (Driver info: chromedriver=2.29.461585 > >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X > 10.12.3 > >>> >> >> x86_64) > >>> >> >> > >>> >> > I checked code from _connects_to_server function which is common > in > >>> >> > all > >>> >> > features test cases. I didn't find anything wrong with this. If > there > >>> >> > is > >>> >> > a > >>> >> > bug in this function then all feature test must fail. > >>> >> > Let me know if you are getting failure consistently in > >>> >> > _connects_to_server > >>> >> > function. > >>> >> > >>> >> I wondered if that one is a race condition. Do we need a short delay > >>> >> before clicking the Object menu? I have seen this occasionally > before. > >>> > > >>> > > >>> > OK. In that case let's try putting 1-2 second delay and observer > >>> > behaviour. > >>> > I'll send separate patch for this tomorrow as this is not related to > on > >>> > demand query result feature or its test cases. > >>> > > >>> >> > >>> >> > >>> >> >> > >>> >> >> > >>> >> >> > >>> >> >> ============================================================ > ========== > >>> >> >> ERROR: runTest > >>> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) > >>> >> >> Query tool feature test > >>> >> >> > >>> >> >> ------------------------------------------------------------ > ---------- > >>> >> >> Traceback (most recent call last): > >>> >> >> File > >>> >> >> > >>> >> >> > >>> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > query_tool_tests.py", > >>> >> >> line 119, in runTest > >>> >> >> self._query_tool_auto_rollback_enabled() > >>> >> >> File > >>> >> >> > >>> >> >> > >>> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > query_tool_tests.py", > >>> >> >> line 697, in _query_tool_auto_rollback_enabled > >>> >> >> '//div[contains(@class, "sql-editor-message") and > >>> >> >> contains(string(), > >>> >> >> "COMMIT")]' > >>> >> >> File > >>> >> >> > >>> >> >> > >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >>> >> >> line 295, in find_element_by_xpath > >>> >> >> return self.find_element(by=By.XPATH, value=xpath) > >>> >> >> File > >>> >> >> > >>> >> >> > >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >>> >> >> line 756, in find_element > >>> >> >> 'value': value})['value'] > >>> >> >> File > >>> >> >> > >>> >> >> > >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >>> >> >> line 238, in execute > >>> >> >> self.error_handler.check_response(response) > >>> >> >> File > >>> >> >> > >>> >> >> > >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/errorhandler.py", > >>> >> >> line 193, in check_response > >>> >> >> raise exception_class(message, screen, stacktrace) > >>> >> >> NoSuchElementException: Message: no such element: Unable to > locate > >>> >> >> element: {"method":"xpath","selector":"//div[contains(@class, > >>> >> >> "sql-editor-message") and contains(string(), "COMMIT")]"} > >>> >> >> (Session info: chrome=58.0.3029.110) > >>> >> >> (Driver info: chromedriver=2.29.461585 > >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X > 10.12.3 > >>> >> >> x86_64) > >>> >> >> > >>> >> > I have updated Auto rollback enabled test in this patch. > >>> >> > > >>> >> >> > >>> >> >> Relevant screenshots attached. > >>> >> >> > >>> >> >> - Can you tidy up the regression output a little please? Instead > of: > >>> >> >> > >>> >> >> ------- > >>> >> >> runTest > >>> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) > >>> >> >> Query tool feature test ... On demand result set on scrolling... > >>> >> >> OK. > >>> >> >> > >>> >> >> On demand result set on grid select all... > >>> >> >> OK. > >>> >> >> > >>> >> >> On demand result set on column select all... > >>> >> >> OK. > >>> >> >> > >>> >> >> Explain query... > >>> >> >> OK. > >>> >> >> > >>> >> >> Explain query with verbose... > >>> >> >> OK. > >>> >> >> > >>> >> >> Explain query with costs... > >>> >> >> OK. > >>> >> >> > >>> >> >> Explain analyze query... > >>> >> >> OK. > >>> >> >> > >>> >> >> Explain analyze query with buffers... > >>> >> >> OK. > >>> >> >> > >>> >> >> Explain analyze query with timing... > >>> >> >> OK. > >>> >> >> > >>> >> >> Auto commit disabled... > >>> >> >> OK. > >>> >> >> > >>> >> >> Auto commit enabled... > >>> >> >> OK. > >>> >> >> > >>> >> >> Auto rollback enabled... > >>> >> >> ERROR > >>> >> >> ------- > >>> >> >> > >>> >> >> Something like: > >>> >> >> > >>> >> >> ------- > >>> >> >> runTest > >>> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) > >>> >> >> Query tool feature test ... > >>> >> >> On demand result set on scrolling... OK. > >>> >> >> On demand result set on grid select all... OK. > >>> >> >> On demand result set on column select all... OK. > >>> >> >> Explain query... OK. > >>> >> >> Explain query with verbose... OK. > >>> >> >> Explain query with costs... OK. > >>> >> >> Explain analyze query... OK. > >>> >> >> Explain analyze query with buffers... OK. > >>> >> >> Explain analyze query with timing... OK. > >>> >> >> Auto commit disabled... OK. > >>> >> >> Auto commit enabled... OK. > >>> >> >> Auto rollback enabled... ERROR > >>> >> >> -------- > >>> >> >> > >>> >> > Fixed. > >>> >> > > >>> >> > > >>> >> >> > >>> >> >> Thanks! > >>> >> >> > >>> >> >> -- > >>> >> >> Dave Page > >>> >> >> Blog: http://pgsnake.blogspot.com > >>> >> >> Twitter: @pgsnake > >>> >> >> > >>> >> >> EnterpriseDB UK: http://www.enterprisedb.com > >>> >> >> The Enterprise PostgreSQL Company > >>> >> > > >>> >> > > >>> >> > >>> >> > >>> >> > >>> >> -- > >>> >> Dave Page > >>> >> Blog: http://pgsnake.blogspot.com > >>> >> Twitter: @pgsnake > >>> >> > >>> >> EnterpriseDB UK: http://www.enterprisedb.com > >>> >> The Enterprise PostgreSQL Company > >>> > > >>> > > >>> > >>> > >>> > >>> -- > >>> Dave Page > >>> Blog: http://pgsnake.blogspot.com > >>> Twitter: @pgsnake > >>> > >>> EnterpriseDB UK: http://www.enterprisedb.com > >>> The Enterprise PostgreSQL Company > >> > >> > > > > > > > > -- > > Dave Page > > Blog: http://pgsnake.blogspot.com > > Twitter: @pgsnake > > > > EnterpriseDB UK: http://www.enterprisedb.com > > The Enterprise PostgreSQL Company > > > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > Attachments: [text/x-patch] RM2137_query_tool_on_demand_result_V8.patch (164.9K, 3-RM2137_query_tool_on_demand_result_V8.patch) download | inline diff: diff --git a/web/config.py b/web/config.py index 6a3128e..e40201d 100644 --- a/web/config.py +++ b/web/config.py @@ -322,6 +322,12 @@ THREADED_MODE = True SQLALCHEMY_TRACK_MODIFICATIONS = False ########################################################################## +# Number of records to fetch in one batch in query tool when query result +# set is large. +########################################################################## +ON_DEMAND_RECORD_COUNT = 1000 + +########################################################################## # Local config settings ########################################################################## diff --git a/web/pgadmin/feature_tests/connect_to_server_feature_test.py b/web/pgadmin/feature_tests/connect_to_server_feature_test.py index e8de161..6bec4ac 100644 --- a/web/pgadmin/feature_tests/connect_to_server_feature_test.py +++ b/web/pgadmin/feature_tests/connect_to_server_feature_test.py @@ -7,6 +7,7 @@ # ########################################################################## +import time from selenium.webdriver import ActionChains import config as app_config @@ -53,6 +54,7 @@ class ConnectsToServerFeatureTest(BaseFeatureTest): def _connects_to_server(self): self.page.find_by_xpath("//*[@class='aciTreeText' and .='Servers']").click() + time.sleep(2) self.page.driver.find_element_by_link_text("Object").click() ActionChains(self.page.driver) \ .move_to_element(self.page.driver.find_element_by_link_text("Create")) \ diff --git a/web/pgadmin/feature_tests/pg_datatype_validation_test.py b/web/pgadmin/feature_tests/pg_datatype_validation_test.py index 8483876..26c787b 100644 --- a/web/pgadmin/feature_tests/pg_datatype_validation_test.py +++ b/web/pgadmin/feature_tests/pg_datatype_validation_test.py @@ -6,6 +6,7 @@ # This software is released under the PostgreSQL Licence # ########################################################################## +import time from selenium.webdriver import ActionChains from selenium.common.exceptions import TimeoutException from selenium.webdriver.support.ui import WebDriverWait @@ -56,6 +57,7 @@ class PGDataypeFeatureTest(BaseFeatureTest): self.page.find_by_xpath( "//*[@class='aciTreeText' and .='Servers']" ).click() + time.sleep(2) self.page.driver.find_element_by_link_text("Object").click() ActionChains(self.page.driver) \ .move_to_element( @@ -106,45 +108,19 @@ class PGDataypeFeatureTest(BaseFeatureTest): self.page.fill_codemirror_area_with(query) self.page.find_by_id("btn-flash").click() wait = WebDriverWait(self.page.driver, 5) - wait.until(EC.presence_of_element_located( - (By.XPATH, "//*[@id='0']//*[@id='datagrid']/div[5]/div/div[1]/" - "div[2]/span"))) + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) # For every sample data-type value, check the expected output. cnt = 2 - for val in expected_output[:10]: - try: - source_code = self.page.find_by_xpath( - "//*[@id='0']//*[@id='datagrid']/div[5]/div/div[1]/div[" - + str(cnt) - + "]/span" - ).get_attribute('innerHTML') - - PGDataypeFeatureTest.check_result( - source_code, - expected_output[cnt - 2] - ) - cnt += 1 - except TimeoutException: - assert False, "{0} does not match with {1}".format( - val, expected_output[cnt] - ) - - cnt = 12 - for val in expected_output[10:]: + cells = canvas.find_elements_by_css_selector('.slick-cell') + # remove first element as it is row number. + cells.pop(0) + for val, cell in zip(expected_output, cells): try: - if cnt == 14: - xpath = "//*[@id='0']//*[@id='datagrid']/div[5]/div/div[1]/div[" \ - + str(cnt) \ - + "]/span" - else: - xpath = "//*[@id='0']//*[@id='datagrid']/div[5]/div/div/div[" \ - + str(cnt) \ - + "]" - - source_code = self.page.find_by_xpath( - xpath - ).get_attribute('innerHTML') + source_code = cell.get_attribute('innerHTML') PGDataypeFeatureTest.check_result( source_code, diff --git a/web/pgadmin/feature_tests/query_tool_tests.py b/web/pgadmin/feature_tests/query_tool_tests.py new file mode 100644 index 0000000..520f41e --- /dev/null +++ b/web/pgadmin/feature_tests/query_tool_tests.py @@ -0,0 +1,761 @@ +########################################################################## +# +# pgAdmin 4 - PostgreSQL Tools +# +# Copyright (C) 2013 - 2017, The pgAdmin Development Team +# This software is released under the PostgreSQL Licence +# +########################################################################## + +from __future__ import print_function +import time +import sys +import config +from selenium.webdriver import ActionChains +from selenium.webdriver.support.ui import WebDriverWait +from selenium.webdriver.support import expected_conditions as EC +from selenium.webdriver.common.by import By +from regression.python_test_utils import test_utils +from regression.feature_utils.base_feature_test import BaseFeatureTest + + +class QueryToolFeatureTest(BaseFeatureTest): + """ + This feature test will test the different query tool features. + """ + + scenarios = [ + ("Query tool feature test", dict()) + ] + + def before(self): + connection = test_utils.get_db_connection(self.server['db'], + self.server['username'], + self.server['db_password'], + self.server['host'], + self.server['port']) + test_utils.drop_database(connection, "acceptance_test_db") + test_utils.create_database(self.server, "acceptance_test_db") + self.page.wait_for_spinner_to_disappear() + self._connects_to_server() + self._locate_database_tree_node() + self.page.open_query_tool() + self.page.driver.switch_to_frame( + self.page.driver.find_element_by_tag_name("iframe") + ) + + def runTest(self): + # on demand result set on scrolling. + print("\nOn demand result set on scrolling... ", + file=sys.stderr, end="") + self._on_demand_result() + print("OK.", + file=sys.stderr) + self._clear_query_tool() + + # on demand result set on grid select all. + print("On demand result set on grid select all... ", + file=sys.stderr, end="") + self._on_demand_result_select_all_grid() + print("OK.", + file=sys.stderr) + self._clear_query_tool() + + # on demand result set on column select all. + print("On demand result set on column select all... ", + file=sys.stderr, end="") + self._on_demand_result_select_all_column() + print("OK.", + file=sys.stderr) + self._clear_query_tool() + + # explain query + print("Explain query... ", file=sys.stderr, end="") + self._query_tool_explain() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + # explain query with verbose + print("Explain query with verbose... ", file=sys.stderr, end="") + self._query_tool_explain_verbose() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + # explain query with costs + print("Explain query with costs... ", file=sys.stderr, end="") + self._query_tool_explain_cost() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + # explain analyze query + print("Explain analyze query... ", file=sys.stderr, end="") + self._query_tool_explain_analyze() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + # explain analyze query with buffers + print("Explain analyze query with buffers... ", file=sys.stderr, end="") + self._query_tool_explain_analyze_buffers() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + # explain analyze query with timing + print("Explain analyze query with timing... ", file=sys.stderr, end="") + self._query_tool_explain_analyze_timing() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + # auto commit disabled. + print("Auto commit disabled... ", file=sys.stderr, end="") + self._query_tool_auto_commit_disabled() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + # auto commit enabled. + print("Auto commit enabled... ", file=sys.stderr, end="") + self._query_tool_auto_commit_enabled() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + # auto rollback enabled. + print("Auto rollback enabled...", file=sys.stderr, end="") + self._query_tool_auto_rollback_enabled() + print(" OK.", file=sys.stderr) + self._clear_query_tool() + + # cancel query. + print("Cancel query... ", file=sys.stderr, end="") + self._query_tool_cancel_query() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + def after(self): + self.page.remove_server(self.server) + connection = test_utils.get_db_connection(self.server['db'], + self.server['username'], + self.server['db_password'], + self.server['host'], + self.server['port']) + test_utils.drop_database(connection, "acceptance_test_db") + + def _connects_to_server(self): + self.page.find_by_xpath( + "//*[@class='aciTreeText' and .='Servers']").click() + time.sleep(2) + self.page.driver.find_element_by_link_text("Object").click() + ActionChains(self.page.driver) \ + .move_to_element( + self.page.driver.find_element_by_link_text("Create"))\ + .perform() + self.page.find_by_partial_link_text("Server...").click() + + server_config = self.server + self.page.fill_input_by_field_name("name", server_config['name']) + self.page.find_by_partial_link_text("Connection").click() + self.page.fill_input_by_field_name("host", server_config['host']) + self.page.fill_input_by_field_name("port", server_config['port']) + self.page.fill_input_by_field_name( + "username", + server_config['username'] + ) + self.page.fill_input_by_field_name( + "password", + server_config['db_password'] + ) + self.page.find_by_xpath("//button[contains(.,'Save')]").click() + + def _locate_database_tree_node(self): + self.page.toggle_open_tree_item(self.server['name']) + self.page.toggle_open_tree_item('Databases') + self.page.toggle_open_tree_item('acceptance_test_db') + + def _open_query_tool(self): + self.page.driver.find_element_by_link_text("Tools").click() + + tools_menu = self.page.driver.find_element_by_id('mnu_tools') + + # Query Tool is first li + query_tool = tools_menu.find_element_by_tag_name('li') + + t = time.time() + # wait until Query Tool menu becomes enabled. + while time.time() - t < 20: # 20 seconds + # if menu is disabled then it will have + # two classes 'menu-item disabled'. + # And if menu is enabled the it will have + # only one class 'menu-item'. + if 'menu-item' == str(query_tool.get_attribute('class')): + break + time.sleep(0.1) + else: + assert False, "'Tools -> Query Tool' menu did not enable." + + self.page.find_by_partial_link_text("Query Tool").click() + self.page.click_tab('Query -') + + def _clear_query_tool(self): + # clear codemirror. + self.page.find_by_id("btn-edit").click() + # wait for alertify dialog open animation to complete. + time.sleep(1) + + self.page.click_element(self.page.find_by_xpath("//button[contains(.,'Yes')]")) + # wait for alertify dialog close animation to complete. + time.sleep(1) + + def _on_demand_result(self): + ON_DEMAND_CHUNKS = 2 + query = """-- On demand query result on scroll +SELECT generate_series(1, {}) as id""".format( + config.ON_DEMAND_RECORD_COUNT * ON_DEMAND_CHUNKS) + + wait = WebDriverWait(self.page.driver, 10) + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-flash").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas"))) + + # scroll to bottom to fetch next chunk of result set. + self.driver.execute_script( + "$('.slick-viewport').scrollTop($('.grid-canvas').height());" + ) + + # wait for ajax to complete. + time.sleep(1) + + # again scroll to bottom to bring last row of next chunk in + # viewport. + self.driver.execute_script( + "$('.slick-viewport').scrollTop($('.grid-canvas').height());" + ) + + row_id_to_find = config.ON_DEMAND_RECORD_COUNT * ON_DEMAND_CHUNKS + + canvas.find_element_by_xpath( + '//span[text()="{}"]'.format(row_id_to_find) + ) + + def _on_demand_result_select_all_grid(self): + ON_DEMAND_CHUNKS = 3 + query = """-- On demand query result on grid select all +SELECT generate_series(1, {}) as id""".format( + config.ON_DEMAND_RECORD_COUNT * ON_DEMAND_CHUNKS) + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-flash").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, ".slick-header-column"))).click() + + # wait for until all records are fetched and selected. + time.sleep(1) + # scroll to bottom to bring last row of next chunk in + # viewport. + self.driver.execute_script( + "$('.slick-viewport').scrollTop($('.grid-canvas').height());" + ) + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + + row_id_to_find = config.ON_DEMAND_RECORD_COUNT * ON_DEMAND_CHUNKS + + canvas.find_element_by_xpath( + '//span[text()="{}"]'.format(row_id_to_find) + ) + + def _on_demand_result_select_all_column(self): + ON_DEMAND_CHUNKS = 4 + query = """-- On demand query result on column select all +SELECT generate_series(1, {}) as id1, 'dummy' as id2""".format( + config.ON_DEMAND_RECORD_COUNT * ON_DEMAND_CHUNKS) + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-flash").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + # click on first data column to select all column. + + wait.until(EC.presence_of_element_located( + ( + By.XPATH, + "//span[contains(@class, 'column-name') and contains(., 'id1')]")) + ).click() + + # wait for until all records are fetched and selected. + time.sleep(1) + # scroll to bottom to bring last row of next chunk in + # viewport. + self.driver.execute_script( + "$('.slick-viewport').scrollTop($('.grid-canvas').height());" + ) + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + + row_id_to_find = config.ON_DEMAND_RECORD_COUNT * ON_DEMAND_CHUNKS + + canvas.find_element_by_xpath( + '//span[text()="{}"]'.format(row_id_to_find) + ) + + def _query_tool_explain(self): + query = """-- Explain query +SELECT generate_series(1, 1000) as id order by id desc""" + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-query-dropdown").click() + self.page.find_by_id("btn-explain").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + self.page.click_tab('Data Output') + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + # Search for Plan word in result + canvas.find_element_by_xpath("//*[contains(string(),'Plan')]") + + def _query_tool_explain_verbose(self): + query = """-- Explain query with verbose +SELECT generate_series(1, 1000) as id order by id desc""" + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + query_op = self.page.find_by_id("btn-query-dropdown") + query_op.click() + + ActionChains(self.driver).move_to_element( + query_op.find_element_by_xpath( + "//li[contains(.,'Explain Options')]")).perform() + + self.page.find_by_id("btn-explain-verbose").click() + + self.page.find_by_id("btn-explain").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + self.page.click_tab('Data Output') + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + # Search for 'Output' word in result + canvas.find_element_by_xpath("//*[contains(string(), 'Output')]") + + def _query_tool_explain_cost(self): + query = """-- Explain query with costs +SELECT generate_series(1, 1000) as id order by id desc""" + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + query_op = self.page.find_by_id("btn-query-dropdown") + query_op.click() + + ActionChains(self.driver).move_to_element( + query_op.find_element_by_xpath( + "//li[contains(.,'Explain Options')]")).perform() + + self.page.find_by_id("btn-explain-costs").click() + + self.page.find_by_id("btn-explain").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + self.page.click_tab('Data Output') + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + # Search for 'Total Cost word in result + canvas.find_element_by_xpath("//*[contains(string(),'Total Cost')]") + + def _query_tool_explain_analyze(self): + query = """-- Explain analyze query +SELECT generate_series(1, 1000) as id order by id desc""" + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-query-dropdown").click() + self.page.find_by_id("btn-explain-analyze").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + self.page.click_tab('Data Output') + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + # Search for Actual Rows word in result + canvas.find_element_by_xpath("//*[contains(string(),'Actual Rows')]") + + def _query_tool_explain_analyze_buffers(self): + query = """-- Explain analyze query with buffers +SELECT generate_series(1, 1000) as id order by id desc""" + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + query_op = self.page.find_by_id("btn-query-dropdown") + query_op.click() + + ActionChains(self.driver).move_to_element( + query_op.find_element_by_xpath( + "//li[contains(.,'Explain Options')]")).perform() + + self.page.find_by_id("btn-explain-buffers").click() + + self.page.find_by_id("btn-explain-analyze").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + self.page.click_tab('Data Output') + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + # Search for 'Shared Read Blocks' word in result + canvas.find_element_by_xpath("//*[contains(string(), 'Shared Read Blocks')]") + + def _query_tool_explain_analyze_timing(self): + query = """-- Explain analyze query with timing +SELECT generate_series(1, 1000) as id order by id desc""" + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + query_op = self.page.find_by_id("btn-query-dropdown") + query_op.click() + + ActionChains(self.driver).move_to_element( + query_op.find_element_by_xpath( + "//li[contains(.,'Explain Options')]")).perform() + + self.page.find_by_id("btn-explain-timing").click() + + self.page.find_by_id("btn-explain-analyze").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + self.page.click_tab('Data Output') + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + # Search for 'Actual Total Time' word in result + canvas.find_element_by_xpath("//*[contains(string(), 'Actual Total Time')]") + + def _query_tool_auto_commit_disabled(self): + table_name = 'query_tool_auto_commit_disabled_table' + query = """-- 1. Disable auto commit. +-- 2. Create table in public schema. +-- 3. ROLLBACK transaction. +-- 4. Check if table is *NOT* created. +CREATE TABLE public.{}();""".format(table_name) + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-query-dropdown").click() + + auto_commit_btn = self.page.find_by_id("btn-auto-commit") + + auto_commit_check = auto_commit_btn.find_element_by_tag_name("i") + + # if auto commit is enabled then 'i' element will + # have 'auto-commit fa fa-check' classes + # if auto commit is disabled then 'i' element will + # have 'auto-commit fa fa-check visibility-hidden' classes + + if 'auto-commit fa fa-check' == str(auto_commit_check.get_attribute( + 'class')): + auto_commit_btn.click() + + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "CREATE TABLE")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) Disable auto commit. +-- 2. (Done) Create table in public schema. +-- 3. ROLLBACK transaction. +-- 4. Check if table is *NOT* created. +ROLLBACK;""" + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "ROLLBACK")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) Disable auto commit. +-- 2. (Done) Create table in public schema. +-- 3. (Done) ROLLBACK transaction. +-- 4. Check if table is *NOT* created. +SELECT relname FROM pg_class WHERE relkind IN ('r','s','t') and relnamespace = 2200::oid;""" + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Data Output') + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas"))) + + el = canvas.find_elements_by_xpath("//div[contains(@class, 'slick-cell') and contains(text(), '{}')]".format(table_name)) + + assert len(el) == 0, "Table '{}' created with auto commit disabled and without any explicit commit.".format(table_name) + + def _query_tool_auto_commit_enabled(self): + table_name = 'query_tool_auto_commit_enabled_table' + query = """-- 1. END any open transaction. +-- 2. Enable auto commit. +-- 3. Create table in public schema. +-- 4. ROLLBACK transaction +-- 5. Check if table is created event after ROLLBACK. +END; +CREATE TABLE public.{}();""".format(table_name) + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-query-dropdown").click() + + auto_commit_btn = self.page.find_by_id("btn-auto-commit") + + auto_commit_check = auto_commit_btn.find_element_by_tag_name("i") + + # if auto commit is enabled then 'i' element will + # have 'auto-commit fa fa-check' classes + # if auto commit is disabled then 'i' element will + # have 'auto-commit fa fa-check visibility-hidden' classes + + if 'auto-commit fa fa-check visibility-hidden' == str(auto_commit_check.get_attribute( + 'class')): + auto_commit_btn.click() + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "CREATE TABLE")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) END any open transaction if any. +-- 2. (Done) Enable auto commit. +-- 3. (Done) Create table in public schema. +-- 4. ROLLBACK transaction +-- 5. Check if table is created event after ROLLBACK. +ROLLBACK;""" + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "ROLLBACK")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) END any open transaction if any. +-- 2. (Done) Enable auto commit. +-- 3. (Done) Create table in public schema. +-- 4. (Done) ROLLBACK transaction +-- 5. Check if table is created event after ROLLBACK. +SELECT relname FROM pg_class WHERE relkind IN ('r','s','t') and relnamespace = 2200::oid;""" + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.click_tab('Data Output') + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas"))) + + el = canvas.find_elements_by_xpath("//div[contains(@class, 'slick-cell') and contains(text(), '{}')]".format(table_name)) + + assert len(el) != 0, "Table '{}' is not created with auto commit enabled.".format(table_name) + + def _query_tool_auto_rollback_enabled(self): + table_name = 'query_tool_auto_rollback_enabled_table' + query = """-- 1. END any open transaction. +-- 2. Enable auto rollback and disable auto commit. +-- 3. Create table in public schema. +-- 4. Generate error in transaction. +-- 5. END transaction. +-- 6. Check if table is *NOT* created after ending transaction. +END;""" + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self._clear_query_tool() + + query = """-- 1. (Done) END any open transaction. +-- 2. Enable auto rollback and disable auto commit. +-- 3. Create table in public schema. +-- 4. Generate error in transaction. +-- 5. END transaction. +-- 6. Check if table is *NOT* created after ending transaction. +CREATE TABLE public.{}();""".format(table_name) + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-query-dropdown").click() + + auto_rollback_btn = self.page.find_by_id("btn-auto-rollback") + + auto_rollback_check = auto_rollback_btn.find_element_by_tag_name("i") + + # if auto rollback is enabled then 'i' element will + # have 'auto-rollback fa fa-check' classes + # if auto rollback is disabled then 'i' element will + # have 'auto-rollback fa fa-check visibility-hidden' classes + + if 'auto-rollback fa fa-check visibility-hidden' == str(auto_rollback_check.get_attribute( + 'class')): + auto_rollback_btn.click() + + auto_commit_btn = self.page.find_by_id("btn-auto-commit") + + auto_commit_check = auto_commit_btn.find_element_by_tag_name("i") + + # if auto commit is enabled then 'i' element will + # have 'auto-commit fa fa-check' classes + # if auto commit is disabled then 'i' element will + # have 'auto-commit fa fa-check visibility-hidden' classes + + if 'auto-commit fa fa-check' == str(auto_commit_check.get_attribute( + 'class')): + auto_commit_btn.click() + + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "CREATE TABLE")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) END any open transaction. +-- 2. (Done) Enable auto rollback and disable auto commit. +-- 3. (Done) Create table in public schema. +-- 4. Generate error in transaction. +-- 5. END transaction. +-- 6. Check if table is *NOT* created after ending transaction. +SELECT 1/0;""" + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "division by zero")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) END any open transaction. +-- 2. (Done) Enable auto rollback and disable auto commit. +-- 3. (Done) Create table in public schema. +-- 4. (Done) Generate error in transaction. +-- 5. END transaction. +-- 6. Check if table is *NOT* created after ending transaction. +END;""" + + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "Query returned successfully")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) END any open transaction. +-- 2. (Done) Enable auto rollback and disable auto commit. +-- 3. (Done) Create table in public schema. +-- 4. (Done) Generate error in transaction. +-- 5. (Done) END transaction. +-- 6. Check if table is *NOT* created after ending transaction. +SELECT relname FROM pg_class WHERE relkind IN ('r','s','t') and relnamespace = 2200::oid;""" + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Data Output') + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas"))) + + el = canvas.find_elements_by_xpath("//div[contains(@class, 'slick-cell') and contains(text(), '{}')]".format(table_name)) + + assert len(el) == 0, "Table '{}' created even after ROLLBACK due to sql error.".format(table_name) + + def _query_tool_cancel_query(self): + query = """-- 1. END any open transaction. +-- 2. Enable auto commit and Disable auto rollback. +-- 3. Execute long running query. +-- 4. Cancel long running query execution. +END; +SELECT 1, pg_sleep(10)""" + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-query-dropdown").click() + + auto_rollback_btn = self.page.find_by_id("btn-auto-rollback") + + auto_rollback_check = auto_rollback_btn.find_element_by_tag_name("i") + + # if auto rollback is enabled then 'i' element will + # have 'auto-rollback fa fa-check' classes + # if auto rollback is disabled then 'i' element will + # have 'auto-rollback fa fa-check visibility-hidden' classes + + if 'auto-rollback fa fa-check' == str(auto_rollback_check.get_attribute( + 'class')): + auto_rollback_btn.click() + + auto_commit_btn = self.page.find_by_id("btn-auto-commit") + + auto_commit_check = auto_commit_btn.find_element_by_tag_name("i") + + # if auto commit is enabled then 'i' element will + # have 'auto-commit fa fa-check' classes + # if auto commit is disabled then 'i' element will + # have 'auto-commit fa fa-check visibility-hidden' classes + + if 'auto-commit fa fa-check visibility-hidden' == str(auto_commit_check.get_attribute( + 'class')): + auto_commit_btn.click() + + self.page.find_by_id("btn-flash").click() + self.driver.find_element_by_xpath("//*[@id='fetching_data']") + self.page.find_by_id("btn-cancel-query").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "canceling statement due to user request")]' + ) diff --git a/web/pgadmin/feature_tests/view_data_dml_queries.py b/web/pgadmin/feature_tests/view_data_dml_queries.py index 55b45ad..a6bc5aa 100644 --- a/web/pgadmin/feature_tests/view_data_dml_queries.py +++ b/web/pgadmin/feature_tests/view_data_dml_queries.py @@ -9,6 +9,7 @@ import json import os +import time from selenium.webdriver import ActionChains from regression.python_test_utils import test_utils from regression.feature_utils.base_feature_test import BaseFeatureTest @@ -205,6 +206,10 @@ CREATE TABLE public.defaults self.page.driver.find_element_by_link_text("View Data")) \ .perform() self.page.find_by_partial_link_text("View All Rows").click() + + # wait until datagrid frame is loaded. + self.page.click_tab('Edit Data -') + self.wait.until( EC.visibility_of_element_located( (By.CSS_SELECTOR, 'iframe') @@ -242,6 +247,11 @@ CREATE TABLE public.defaults self._update_cell(row1_cell2_xpath, ["1", "", "int"]) self.page.find_by_id("btn-save").click() # Save data + # There should be some delay after save button is clicked, as it + # takes some time to complete save ajax call otherwise discard unsaved + # changes dialog will appear if we try to execute query before previous + # save ajax is completed. + time.sleep(2) # Verify row 1 and row 2 data self._verify_row_data(False) @@ -254,6 +264,11 @@ CREATE TABLE public.defaults self._update_cell(cell_xpath, config_data[str(idx)]) self.page.find_by_id("btn-save").click() # Save data + # There should be some delay after save button is clicked, as it + # takes some time to complete save ajax call otherwise discard unsaved + # changes dialog will appear if we try to execute query before previous + # save ajax is completed. + time.sleep(2) def _verify_row_data(self, is_new_row): self.page.find_by_id("btn-flash").click() @@ -264,17 +279,17 @@ CREATE TABLE public.defaults xpath = "//*[contains(@class, 'ui-widget-content') and " \ "contains(@style, 'top:" + str(row_height) + "px')]" - # wait for stale element reference exception - self.page.wait_for_element_to_stale(xpath) + self.page.wait_for_query_tool_loading_indicator_to_disappear() + result_row = self.page.find_by_xpath(xpath) # List of row values in an array cells = [el.text for el in result_row.find_elements_by_tag_name('div')] for idx in range(1, len(config_data.keys())): - # # after copy & paste row, the first cell of row 1 and - # # row 2(being primary keys) won't match - # # see if cell values matched to actual value + # after copy & paste row, the first cell of row 1 and + # row 2(being primary keys) won't match + # see if cell values matched to actual value if idx != 1 and not is_new_row: self.assertEquals(cells[idx], config_data[str(idx)][1]) elif is_new_row: diff --git a/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py b/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py index 4fd0783..75b4222 100644 --- a/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py +++ b/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py @@ -10,6 +10,9 @@ from selenium.webdriver import ActionChains from regression.python_test_utils import test_utils from regression.feature_utils.base_feature_test import BaseFeatureTest +from selenium.webdriver.support.ui import WebDriverWait +from selenium.webdriver.support import expected_conditions as EC +from selenium.webdriver.common.by import By import time class CheckForXssFeatureTest(BaseFeatureTest): @@ -72,6 +75,7 @@ class CheckForXssFeatureTest(BaseFeatureTest): def _connects_to_server(self): self.page.find_by_xpath("//*[@class='aciTreeText' and .='Servers']").click() + time.sleep(2) self.page.driver.find_element_by_link_text("Object").click() ActionChains(self.page.driver) \ .move_to_element(self.page.driver.find_element_by_link_text("Create")) \ @@ -152,11 +156,16 @@ class CheckForXssFeatureTest(BaseFeatureTest): self.page.fill_codemirror_area_with("select '<img src=\"x\" onerror=\"console.log(1)\">'") time.sleep(1) self.page.find_by_id("btn-flash").click() - time.sleep(2) + wait = WebDriverWait(self.page.driver, 5) - source_code = self.page.find_by_xpath( - "//*[@id='0']//*[@id='datagrid']/div[5]/div/div[1]/div[2]" - ).get_attribute('innerHTML') + result_row = self.page.find_by_xpath( + "//*[contains(@class, 'ui-widget-content') and contains(@style, 'top:0px')]" + ) + + cells = result_row.find_elements_by_tag_name('div') + + # remove first element as it is row number. + source_code = cells[1].get_attribute('innerHTML') self._check_escaped_characters( source_code, diff --git a/web/pgadmin/feature_tests/xss_checks_pgadmin_debugger_test.py b/web/pgadmin/feature_tests/xss_checks_pgadmin_debugger_test.py index 094dfed..e847040 100644 --- a/web/pgadmin/feature_tests/xss_checks_pgadmin_debugger_test.py +++ b/web/pgadmin/feature_tests/xss_checks_pgadmin_debugger_test.py @@ -42,6 +42,7 @@ class CheckDebuggerForXssFeatureTest(BaseFeatureTest): def _connects_to_server(self): self.page.find_by_xpath("//*[@class='aciTreeText' and .='Servers']").click() + time.sleep(2) self.page.driver.find_element_by_link_text("Object").click() ActionChains(self.page.driver) \ .move_to_element(self.page.driver.find_element_by_link_text("Create")) \ diff --git a/web/pgadmin/static/bundle/slickgrid.js b/web/pgadmin/static/bundle/slickgrid.js index 0d82a6b..dd7f870 100644 --- a/web/pgadmin/static/bundle/slickgrid.js +++ b/web/pgadmin/static/bundle/slickgrid.js @@ -3,6 +3,7 @@ import 'slickgrid/slick-default-theme.css'; import 'slickgrid/css/smoothness/jquery-ui-1.11.3.custom.css'; import 'slickgrid/slick.core'; import 'slickgrid/slick.grid'; +import 'slickgrid/slick.dataview'; import 'slickgrid/slick.editors'; import 'slickgrid/slick.formatters'; import 'slickgrid/plugins/slick.autotooltips'; diff --git a/web/pgadmin/static/js/selection/column_selector.js b/web/pgadmin/static/js/selection/column_selector.js index 305b438..f3cb727 100644 --- a/web/pgadmin/static/js/selection/column_selector.js +++ b/web/pgadmin/static/js/selection/column_selector.js @@ -4,13 +4,18 @@ define([ 'slickgrid', ], function ($, RangeSelectionHelper) { var ColumnSelector = function () { - var Slick = window.Slick; - var gridEventBus = new Slick.EventHandler(); + var Slick = window.Slick, + gridEventBus = new Slick.EventHandler(), + onBeforeColumnSelectAll = new Slick.Event(), + onColumnSelectAll = new Slick.Event(); var init = function (grid) { gridEventBus.subscribe(grid.onHeaderClick, handleHeaderClick.bind(null, grid)); grid.getSelectionModel().onSelectedRangesChanged .subscribe(handleSelectedRangesChanged.bind(null, grid)); + onColumnSelectAll.subscribe(function(e, args) { + updateRanges(args.grid, args.column.id); + }); }; var handleHeaderClick = function (grid, event, args) { @@ -21,11 +26,20 @@ define([ if (isColumnSelectable(columnDefinition)) { var $columnHeader = $(event.target); if (hasClickedChildOfColumnHeader(event)) { + if ($(event.target).hasClass('slick-resizable-handle')) { + return; + } $columnHeader = $(event.target).parents('.slick-header-column'); } $columnHeader.toggleClass('selected'); - updateRanges(grid, columnDefinition.id); + if ($columnHeader.hasClass('selected')) { + onBeforeColumnSelectAll.notify(args, event); + } + + if (!(event.isPropagationStopped() || event.isImmediatePropagationStopped())) { + updateRanges(grid, columnDefinition.id); + } } }; @@ -107,6 +121,8 @@ define([ $.extend(this, { 'init': init, 'getColumnDefinitions': getColumnDefinitions, + 'onBeforeColumnSelectAll': onBeforeColumnSelectAll, + 'onColumnSelectAll': onColumnSelectAll, }); }; return ColumnSelector; diff --git a/web/pgadmin/static/js/selection/copy_data.js b/web/pgadmin/static/js/selection/copy_data.js index c226323..3525966 100644 --- a/web/pgadmin/static/js/selection/copy_data.js +++ b/web/pgadmin/static/js/selection/copy_data.js @@ -12,19 +12,19 @@ function ($, _, clipboard, RangeSelectionHelper, rangeBoundaryNavigator) { var grid = self.slickgrid; var columnDefinitions = grid.getColumns(); var selectedRanges = grid.getSelectionModel().getSelectedRanges(); - var data = grid.getData(); + var dataView = grid.getData(); var rows = grid.getSelectedRows(); if (RangeSelectionHelper.areAllRangesCompleteRows(grid, selectedRanges)) { self.copied_rows = rows.map(function (rowIndex) { - return data[rowIndex]; + return grid.getDataItem(rowIndex); }); setPasteRowButtonEnablement(self.can_edit, true); } else { self.copied_rows = []; setPasteRowButtonEnablement(self.can_edit, false); } - var csvText = rangeBoundaryNavigator.rangesToCsv(data, columnDefinitions, selectedRanges); + var csvText = rangeBoundaryNavigator.rangesToCsv(dataView.getItems(), columnDefinitions, selectedRanges); if (csvText) { clipboard.copyTextToClipboard(csvText); } diff --git a/web/pgadmin/static/js/selection/grid_selector.js b/web/pgadmin/static/js/selection/grid_selector.js index 24fd9d6..ddbbe23 100644 --- a/web/pgadmin/static/js/selection/grid_selector.js +++ b/web/pgadmin/static/js/selection/grid_selector.js @@ -6,21 +6,31 @@ define(['jquery', 'sources/url_for', ], function ($, gettext, ColumnSelector, RowSelector, RangeSelectionHelper, url_for) { var GridSelector = function (columnDefinitions) { - var rowSelector = new RowSelector(columnDefinitions); - var columnSelector = new ColumnSelector(columnDefinitions); + var Slick = window.Slick, + rowSelector = new RowSelector(columnDefinitions), + columnSelector = new ColumnSelector(columnDefinitions), + onBeforeGridSelectAll = new Slick.Event(), + onGridSelectAll = new Slick.Event(), + onBeforeGridColumnSelectAll = columnSelector.onBeforeColumnSelectAll, + onGridColumnSelectAll = columnSelector.onColumnSelectAll; var init = function (grid) { this.grid = grid; grid.onHeaderClick.subscribe(function (event, eventArguments) { - if (eventArguments.column.selectAllOnClick) { - toggleSelectAll(grid); + if (eventArguments.column.selectAllOnClick && !$(event.target).hasClass('slick-resizable-handle')) { + toggleSelectAll(grid, event, eventArguments); } }); grid.getSelectionModel().onSelectedRangesChanged - .subscribe(handleSelectedRangesChanged.bind(null, grid)); + .subscribe(handleSelectedRangesChanged.bind(null, grid)); + grid.registerPlugin(rowSelector); grid.registerPlugin(columnSelector); + + onGridSelectAll.subscribe(function(e, args) { + RangeSelectionHelper.selectAll(args.grid); + }); }; var getColumnDefinitions = function (columnDefinitions) { @@ -45,11 +55,14 @@ define(['jquery', } } - function toggleSelectAll(grid) { + function toggleSelectAll(grid, event, eventArguments) { if (RangeSelectionHelper.isEntireGridSelected(grid)) { selectNone(grid); } else { - RangeSelectionHelper.selectAll(grid); + onBeforeGridSelectAll.notify(eventArguments, event); + if (!(event.isPropagationStopped() || event.isImmediatePropagationStopped())) { + RangeSelectionHelper.selectAll(grid); + } } } @@ -61,6 +74,10 @@ define(['jquery', $.extend(this, { 'init': init, 'getColumnDefinitions': getColumnDefinitions, + 'onBeforeGridSelectAll': onBeforeGridSelectAll, + 'onGridSelectAll': onGridSelectAll, + 'onBeforeGridColumnSelectAll': onBeforeGridColumnSelectAll, + 'onGridColumnSelectAll': onGridColumnSelectAll, }); }; diff --git a/web/pgadmin/static/js/selection/range_boundary_navigator.js b/web/pgadmin/static/js/selection/range_boundary_navigator.js index 803f75a..274b621 100644 --- a/web/pgadmin/static/js/selection/range_boundary_navigator.js +++ b/web/pgadmin/static/js/selection/range_boundary_navigator.js @@ -58,6 +58,7 @@ function (RangeSelectionHelper) { }, rangesToCsv: function (data, columnDefinitions, selectedRanges) { + var rowRangeBounds = selectedRanges.map(function (range) { return [range.fromRow, range.toRow]; }); @@ -72,6 +73,7 @@ function (RangeSelectionHelper) { var csvRows = this.mapOver2DArray(rowRangeBounds, colRangeBounds, this.csvCell.bind(this, data, columnDefinitions), function (rowData) { return rowData.join(','); }); + return csvRows.join('\n'); }, @@ -101,7 +103,7 @@ function (RangeSelectionHelper) { }, csvCell: function (data, columnDefinitions, rowId, colId) { - var val = data[rowId][columnDefinitions[colId].pos]; + var val = data[rowId][columnDefinitions[colId].field]; if (val && _.isObject(val)) { val = '\'' + JSON.stringify(val) + '\''; diff --git a/web/pgadmin/static/js/selection/row_selector.js b/web/pgadmin/static/js/selection/row_selector.js index 236ed62..59d2fa2 100644 --- a/web/pgadmin/static/js/selection/row_selector.js +++ b/web/pgadmin/static/js/selection/row_selector.js @@ -82,7 +82,8 @@ define([ formatter: function (rowIndex) { return '<span ' + 'data-row="' + rowIndex + '" ' + - 'data-cell-type="row-header-selector"/>'; + 'data-cell-type="row-header-selector">' + + (rowIndex+1) + '</span>'; }, width: 30, }); diff --git a/web/pgadmin/static/js/selection/set_staged_rows.js b/web/pgadmin/static/js/selection/set_staged_rows.js index 46977e3..e7a529c 100644 --- a/web/pgadmin/static/js/selection/set_staged_rows.js +++ b/web/pgadmin/static/js/selection/set_staged_rows.js @@ -22,53 +22,44 @@ define( $(selector).prop('disabled', false); } - function getRowPrimaryKeyValuesToStage(selectedRows, primaryKeyColumnIndices, gridData) { + function getRowPrimaryKeyValuesToStage(selectedRows, primaryKeys, dataView, client_primary_key) { return _.reduce(selectedRows, function (primaryKeyValuesToStage, dataGridRowIndex) { - var gridRow = gridData[dataGridRowIndex]; - - if (isRowMissingPrimaryKeys(gridRow, primaryKeyColumnIndices)) { + var gridRow = dataView.getItem(dataGridRowIndex); + if (isRowMissingPrimaryKeys(gridRow, primaryKeys)) { return primaryKeyValuesToStage; } - - var tempPK = gridRow.__temp_PK; - primaryKeyValuesToStage[tempPK] = getSingleRowPrimaryKeyValueToStage(primaryKeyColumnIndices, gridRow); - + var tempPK = gridRow[client_primary_key]; + primaryKeyValuesToStage[tempPK] = getSingleRowPrimaryKeyValueToStage(primaryKeys, gridRow); return primaryKeyValuesToStage; }, {}); } - function isRowMissingPrimaryKeys(gridRow, primaryKeyColumnIndices) { + function isRowMissingPrimaryKeys(gridRow, primaryKeys) { if (_.isUndefined(gridRow)) { return true; } return !_.isUndefined( - _.find(primaryKeyColumnIndices, function (pkIndex) { - return _.isUndefined(gridRow[pkIndex]); + _.find(primaryKeys , function (pk) { + return _.isUndefined(gridRow[pk]); }) ); } - function getSingleRowPrimaryKeyValueToStage(primaryKeyColumnIndices, gridRow) { + function getSingleRowPrimaryKeyValueToStage(primaryKeys, gridRow) { var rowToStage = {}; - if (primaryKeyColumnIndices.length) { - _.each(_.keys(gridRow), function (columnPos) { - if (_.contains(primaryKeyColumnIndices, Number(columnPos))) - rowToStage[columnPos] = gridRow[columnPos]; + if (primaryKeys && primaryKeys.length) { + _.each(_.keys(gridRow), function (columnNames) { + if (_.contains(primaryKeys, columnNames)) + rowToStage[columnNames] = gridRow[columnNames]; }); } return rowToStage; } function getPrimaryKeysForSelectedRows(self, selectedRows) { - var primaryKeyColumnIndices = _.map(_.keys(self.keys), function (columnName) { - var columnInfo = _.findWhere(self.columns, {name: columnName}); - return columnInfo['pos']; - }); - - var gridData = self.grid.getData(); - var stagedRows = getRowPrimaryKeyValuesToStage(selectedRows, primaryKeyColumnIndices, gridData); - + var dataView = self.grid.getData(); + var stagedRows = getRowPrimaryKeyValuesToStage(selectedRows, _.keys(self.keys), dataView, self.client_primary_key); return stagedRows; } diff --git a/web/pgadmin/static/js/slickgrid/slick.pgadmin.editors.js b/web/pgadmin/static/js/slickgrid/slick.pgadmin.editors.js index f2dabe0..15b5fe9 100644 --- a/web/pgadmin/static/js/slickgrid/slick.pgadmin.editors.js +++ b/web/pgadmin/static/js/slickgrid/slick.pgadmin.editors.js @@ -76,18 +76,18 @@ last_value = (column_type === 'number') ? (_.isEmpty(last_value) || last_value) : last_value; - item[args.column.pos] = state; + item[args.column.field] = state; if (last_value && _.isNull(state) && (_.isUndefined(grid.copied_rows[row]) || _.isUndefined(grid.copied_rows[row][cell])) ) { - item[args.column.pos] = undefined; + item[args.column.field] = undefined; if (grid.copied_rows[row] == undefined) grid.copied_rows[row] = []; grid.copied_rows[row][cell] = 1; } } else { - item[args.column.pos] = state; + item[args.column.field] = state; } } @@ -189,14 +189,14 @@ this.loadValue = function (item) { var col = args.column; - if (_.isUndefined(item[args.column.pos]) && col.has_default_val) { + if (_.isUndefined(item[args.column.field]) && col.has_default_val) { $input.val(defaultValue = ""); } - else if (item[args.column.pos] === "") { + else if (item[args.column.field] === "") { $input.val(defaultValue = "''"); } else { - $input.val(defaultValue = item[args.column.pos]); + $input.val(defaultValue = item[args.column.field]); $input.select(); } }; @@ -323,7 +323,7 @@ }; this.loadValue = function (item) { - var data = defaultValue = item[args.column.pos]; + var data = defaultValue = item[args.column.field]; if (data && typeof data === "object" && !Array.isArray(data)) { data = JSON.stringify(data); } else if (Array.isArray(data)) { @@ -443,7 +443,7 @@ }; this.loadValue = function (item) { - $input.val(defaultValue = item[args.column.pos]); + $input.val(defaultValue = item[args.column.field]); $input.select(); }; @@ -452,7 +452,7 @@ }; this.applyValue = function (item, state) { - item[args.column.pos] = state; + item[args.column.field] = state; }; this.isValueChanged = function () { @@ -531,13 +531,13 @@ }; this.loadValue = function (item) { - defaultValue = item[args.column.pos]; - if (_.isNull(defaultValue)|| _.isUndefined(defaultValue)) { + defaultValue = item[args.column.field]; + if (_.isNull(defaultValue)||_.isUndefined(defaultValue)) { $select.prop('indeterminate', true); $select.data('checked', 2); } else { - defaultValue = !!item[args.column.pos]; + defaultValue = !!item[args.column.field]; if (defaultValue) { $select.prop('checked', true); $select.data('checked', 0); @@ -556,7 +556,7 @@ }; this.applyValue = function (item, state) { - item[args.column.pos] = state; + item[args.column.field] = state; }; this.isValueChanged = function () { @@ -648,7 +648,7 @@ }; this.loadValue = function (item) { - var data = defaultValue = item[args.column.pos]; + var data = defaultValue = item[args.column.field]; if (typeof data === "object" && !Array.isArray(data)) { data = JSON.stringify(data); } else if (Array.isArray(data)) { @@ -671,7 +671,7 @@ }; this.applyValue = function (item, state) { - item[args.column.pos] = state; + item[args.column.field] = state; }; this.isValueChanged = function () { @@ -725,7 +725,7 @@ }; this.loadValue = function (item) { - var value = item[args.column.pos]; + var value = item[args.column.field]; // Check if value is null or undefined if (value === undefined && typeof value === "undefined") { @@ -858,7 +858,7 @@ }; this.loadValue = function (item) { - defaultValue = item[args.column.pos]; + defaultValue = item[args.column.field]; $input.val(defaultValue); $input[0].defaultValue = defaultValue; $input.select(); diff --git a/web/pgadmin/tools/sqleditor/__init__.py b/web/pgadmin/tools/sqleditor/__init__.py index 828cb99..709332c 100644 --- a/web/pgadmin/tools/sqleditor/__init__.py +++ b/web/pgadmin/tools/sqleditor/__init__.py @@ -27,7 +27,7 @@ from pgadmin.utils.sqlautocomplete.autocomplete import SQLAutoComplete from pgadmin.misc.file_manager import Filemanager -from config import PG_DEFAULT_DRIVER +from config import PG_DEFAULT_DRIVER, ON_DEMAND_RECORD_COUNT MODULE_NAME = 'sqleditor' @@ -229,13 +229,32 @@ def start_view_data(trans_id): # Check the transaction and connection status status, error_msg, conn, trans_obj, session_obj = check_transaction_status(trans_id) + + # get the default connection as current connection which is attached to + # trans id holds the cursor which has query result so we cannot use that + # connection to execute another query otherwise we'll lose query result. + + manager = get_driver(PG_DEFAULT_DRIVER).connection_manager(trans_obj.sid) + default_conn = manager.connection(did=trans_obj.did) + + # Connect to the Server if not connected. + if not default_conn.connected(): + status, msg = default_conn.connect() + if not status: + return make_json_response( + data={'status': status, 'result': u"{}".format(msg)} + ) + if status and conn is not None \ and trans_obj is not None and session_obj is not None: try: + # set fetched row count to 0 as we are executing query again. + trans_obj.update_fetched_row_cnt(0) + session_obj['command_obj'] = pickle.dumps(trans_obj, -1) # Fetch the sql and primary_keys from the object sql = trans_obj.get_sql() - pk_names, primary_keys = trans_obj.get_primary_keys() + pk_names, primary_keys = trans_obj.get_primary_keys(default_conn) # Fetch the applied filter. filter_applied = trans_obj.is_filter_applied() @@ -303,6 +322,8 @@ def start_query_tool(trans_id): # Use pickle.loads function to get the command object session_obj = grid_data[str(trans_id)] trans_obj = pickle.loads(session_obj['command_obj']) + # set fetched row count to 0 as we are executing query again. + trans_obj.update_fetched_row_cnt(0) can_edit = False can_filter = False @@ -429,65 +450,6 @@ def preferences(trans_id): return success_return() [email protected]('/columns/<int:trans_id>', methods=["GET"]) -@login_required -def get_columns(trans_id): - """ - This method will returns list of columns of last async query. - - Args: - trans_id: unique transaction id - """ - columns = dict() - columns_info = None - primary_keys = None - rset = None - status, error_msg, conn, trans_obj, session_obj = check_transaction_status(trans_id) - if status and conn is not None and session_obj is not None: - - ver = conn.manager.version - # Get the template path for the column - template_path = 'column/sql/#{0}#'.format(ver) - command_obj = pickle.loads(session_obj['command_obj']) - if hasattr(command_obj, 'obj_id'): - SQL = render_template("/".join([template_path, - 'nodes.sql']), - tid=command_obj.obj_id) - # rows with attribute not_null - status, rset = conn.execute_2darray(SQL) - if not status: - return internal_server_error(errormsg=rset) - - # Check PK column info is available or not - if 'primary_keys' in session_obj: - primary_keys = session_obj['primary_keys'] - - # Fetch column information - columns_info = conn.get_column_info() - if columns_info is not None: - for key, col in enumerate(columns_info): - col_type = dict() - col_type['type_code'] = col['type_code'] - col_type['type_name'] = None - if rset: - col_type['not_null'] = col['not_null'] = \ - rset['rows'][key]['not_null'] - - col_type['has_default_val'] = col['has_default_val'] = \ - rset['rows'][key]['has_default_val'] - - columns[col['name']] = col_type - - # As we changed the transaction object we need to - # restore it and update the session variable. - session_obj['columns_info'] = columns - update_session_grid_transaction(trans_id, session_obj) - - return make_json_response(data={'status': True, - 'columns': columns_info, - 'primary_keys': primary_keys}) - - @blueprint.route('/poll/<int:trans_id>', methods=["GET"]) @login_required def poll(trans_id): @@ -499,12 +461,21 @@ def poll(trans_id): """ result = None rows_affected = 0 + rows_fetched_from = 0 + rows_fetched_to = 0 + has_more_rows = False additional_result = [] + columns = dict() + columns_info = None + primary_keys = None + types = {} + client_primary_key = None + rset = None # Check the transaction and connection status status, error_msg, conn, trans_obj, session_obj = check_transaction_status(trans_id) if status and conn is not None and session_obj is not None: - status, result = conn.poll(formatted_exception_msg=True) + status, result = conn.poll(formatted_exception_msg=True, no_result=True) if not status: return internal_server_error(result) elif status == ASYNC_OK: @@ -519,6 +490,80 @@ def poll(trans_id): if (trans_status == TX_STATUS_INERROR and trans_obj.auto_rollback): conn.execute_void("ROLLBACK;") + + st, result = conn.async_fetchmany_2darray(ON_DEMAND_RECORD_COUNT) + if st: + if 'primary_keys' in session_obj: + primary_keys = session_obj['primary_keys'] + + # Fetch column information + columns_info = conn.get_column_info() + client_primary_key = generate_client_primary_key_name( + columns_info + ) + session_obj['client_primary_key'] = client_primary_key + + if columns_info is not None: + + command_obj = pickle.loads(session_obj['command_obj']) + if hasattr(command_obj, 'obj_id'): + # Get the template path for the column + template_path = 'column/sql/#{0}#'.format( + conn.manager.version + ) + + SQL = render_template("/".join([template_path, + 'nodes.sql']), + tid=command_obj.obj_id) + # rows with attribute not_null + colst, rset = conn.execute_2darray(SQL) + if not colst: + return internal_server_error(errormsg=rset) + + for key, col in enumerate(columns_info): + col_type = dict() + col_type['type_code'] = col['type_code'] + col_type['type_name'] = None + columns[col['name']] = col_type + + if rset: + col_type['not_null'] = col['not_null'] = \ + rset['rows'][key]['not_null'] + + col_type['has_default_val'] = \ + col['has_default_val'] = \ + rset['rows'][key]['has_default_val'] + + if columns: + st, types = fetch_pg_types(columns, trans_obj) + + if not st: + return internal_server_error(types) + + for col_info in columns.values(): + for col_type in types: + if col_type['oid'] == col_info['type_code']: + col_info['type_name'] = col_type['typname'] + + session_obj['columns_info'] = columns + # status of async_fetchmany_2darray is True and result is none + # means nothing to fetch + if result and rows_affected > -1: + res_len = len(result) + if res_len == ON_DEMAND_RECORD_COUNT: + has_more_rows = True + + if res_len > 0: + rows_fetched_from = trans_obj.get_fetched_row_cnt() + trans_obj.update_fetched_row_cnt(rows_fetched_from + res_len) + rows_fetched_from += 1 + rows_fetched_to = trans_obj.get_fetched_row_cnt() + session_obj['command_obj'] = pickle.dumps(trans_obj, -1) + + # As we changed the transaction object we need to + # restore it and update the session variable. + update_session_grid_transaction(trans_id, session_obj) + elif status == ASYNC_EXECUTION_ABORTED: status = 'Cancel' else: @@ -559,51 +604,123 @@ def poll(trans_id): data={ 'status': status, 'result': result, 'rows_affected': rows_affected, - 'additional_messages': additional_messages + 'rows_fetched_from': rows_fetched_from, + 'rows_fetched_to': rows_fetched_to, + 'additional_messages': additional_messages, + 'has_more_rows': has_more_rows, + 'colinfo': columns_info, + 'primary_keys': primary_keys, + 'types': types, + 'client_primary_key': client_primary_key } ) [email protected]('/fetch/types/<int:trans_id>', methods=["GET"]) [email protected]('/fetch/<int:trans_id>', methods=["GET"]) [email protected]('/fetch/<int:trans_id>/<int:fetch_all>', methods=["GET"]) @login_required -def fetch_pg_types(trans_id): +def fetch(trans_id, fetch_all=None): + result = None + has_more_rows = False + rows_fetched_from = 0 + rows_fetched_to = 0 + fetch_row_cnt = -1 if fetch_all == 1 else ON_DEMAND_RECORD_COUNT + + # Check the transaction and connection status + status, error_msg, conn, trans_obj, session_obj = check_transaction_status(trans_id) + if status and conn is not None and session_obj is not None: + status, result = conn.async_fetchmany_2darray(fetch_row_cnt) + if not status: + status = 'Error' + else: + status = 'Success' + res_len = len(result) + if fetch_row_cnt != -1 and res_len == ON_DEMAND_RECORD_COUNT: + has_more_rows = True + + if res_len: + rows_fetched_from = trans_obj.get_fetched_row_cnt() + trans_obj.update_fetched_row_cnt(rows_fetched_from + res_len) + rows_fetched_from += 1 + rows_fetched_to = trans_obj.get_fetched_row_cnt() + session_obj['command_obj'] = pickle.dumps(trans_obj, -1) + update_session_grid_transaction(trans_id, session_obj) + else: + status = 'NotConnected' + result = error_msg + + return make_json_response( + data={ + 'status': status, 'result': result, + 'has_more_rows': has_more_rows, + 'rows_fetched_from': rows_fetched_from, + 'rows_fetched_to': rows_fetched_to + } + ) + + +def fetch_pg_types(columns_info, trans_obj): """ This method is used to fetch the pg types, which is required to map the data type comes as a result of the query. Args: - trans_id: unique transaction id + columns_info: """ - # Check the transaction and connection status - status, error_msg, conn, trans_obj, session_obj = check_transaction_status(trans_id) - if status and conn is not None \ - and trans_obj is not None and session_obj is not None: - res = {} - if 'columns_info' in session_obj \ - and session_obj['columns_info'] is not None: + # get the default connection as current connection attached to trans id + # holds the cursor which has query result so we cannot use that connection + # to execute another query otherwise we'll lose query result. + + manager = get_driver(PG_DEFAULT_DRIVER).connection_manager(trans_obj.sid) + default_conn = manager.connection(did=trans_obj.did) + + # Connect to the Server if not connected. + res = [] + if not default_conn.connected(): + status, msg = default_conn.connect() + if not status: + return status, msg - oids = [session_obj['columns_info'][col]['type_code'] for col in session_obj['columns_info']] + oids = [columns_info[col]['type_code'] for col in columns_info] - if oids: - status, res = conn.execute_dict( - u"""SELECT oid, format_type(oid,null) as typname FROM pg_type WHERE oid IN %s ORDER BY oid; + if oids: + status, res = default_conn.execute_dict( + u"""SELECT oid, format_type(oid,null) as typname FROM pg_type WHERE oid IN %s ORDER BY oid; """, [tuple(oids)]) - if status: - # iterate through pg_types and update the type name in session object - for record in res['rows']: - for col in session_obj['columns_info']: - type_obj = session_obj['columns_info'][col] - if type_obj['type_code'] == record['oid']: - type_obj['type_name'] = record['typname'] + if not status: + return False, res - update_session_grid_transaction(trans_id, session_obj) + return status, res['rows'] else: - status = False - res = error_msg - - return make_json_response(data={'status': status, 'result': res}) + return True, [] + + +def generate_client_primary_key_name(columns_info): + temp_key = '__temp_PK' + if not columns_info: + return temp_key + + initial_temp_key_len = len(temp_key) + duplicate = False + suffix = 1 + while 1: + for col in columns_info: + if col['name'] == temp_key: + duplicate = True + break + if duplicate: + if initial_temp_key_len == len(temp_key): + temp_key += str(suffix) + suffix += 1 + else: + temp_key = temp_key[:-1] + str(suffix) + suffix += 1 + duplicate = False + else: + break + return temp_key @blueprint.route('/save/<int:trans_id>', methods=["PUT", "POST"]) @@ -615,7 +732,6 @@ def save(trans_id): Args: trans_id: unique transaction id """ - if request.data: changed_data = json.loads(request.data, encoding='utf-8') else: @@ -625,7 +741,6 @@ def save(trans_id): status, error_msg, conn, trans_obj, session_obj = check_transaction_status(trans_id) if status and conn is not None \ and trans_obj is not None and session_obj is not None: - setattr(trans_obj, 'columns_info', session_obj['columns_info']) # If there is no primary key found then return from the function. if len(session_obj['primary_keys']) <= 0 or len(changed_data) <= 0: @@ -636,7 +751,22 @@ def save(trans_id): } ) - status, res, query_res, _rowid = trans_obj.save(changed_data) + manager = get_driver(PG_DEFAULT_DRIVER).connection_manager(trans_obj.sid) + default_conn = manager.connection(did=trans_obj.did) + + # Connect to the Server if not connected. + if not default_conn.connected(): + status, msg = default_conn.connect() + if not status: + return make_json_response( + data={'status': status, 'result': u"{}".format(msg)} + ) + + status, res, query_res, _rowid = trans_obj.save( + changed_data, + session_obj['columns_info'], + session_obj['client_primary_key'], + default_conn) else: status = False res = error_msg diff --git a/web/pgadmin/tools/sqleditor/command.py b/web/pgadmin/tools/sqleditor/command.py index b7d8a78..3b7efd0 100644 --- a/web/pgadmin/tools/sqleditor/command.py +++ b/web/pgadmin/tools/sqleditor/command.py @@ -258,7 +258,21 @@ class SQLFilter(object): return status, result -class GridCommand(BaseCommand, SQLFilter): +class FetchedRowTracker(object): + """ + Keeps track of fetched row count. + """ + def __init__(self, **kwargs): + self.fetched_rows = 0 + + def get_fetched_row_cnt(self): + return self.fetched_rows + + def update_fetched_row_cnt(self, rows_cnt): + self.fetched_rows = rows_cnt + + +class GridCommand(BaseCommand, SQLFilter, FetchedRowTracker): """ class GridCommand(object) @@ -290,6 +304,7 @@ class GridCommand(BaseCommand, SQLFilter): """ BaseCommand.__init__(self, **kwargs) SQLFilter.__init__(self, **kwargs) + FetchedRowTracker.__init__(self, **kwargs) # Save the connection id, command type self.conn_id = kwargs['conn_id'] if 'conn_id' in kwargs else None @@ -299,10 +314,10 @@ class GridCommand(BaseCommand, SQLFilter): if self.cmd_type == VIEW_FIRST_100_ROWS or self.cmd_type == VIEW_LAST_100_ROWS: self.limit = 100 - def get_primary_keys(self): + def get_primary_keys(self, *args, **kwargs): return None, None - def save(self, changed_data): + def save(self, changed_data, default_conn=None): return forbidden(errmsg=gettext("Data cannot be saved for the current object.")) def get_limit(self): @@ -340,14 +355,14 @@ class TableCommand(GridCommand): # call base class init to fetch the table name super(TableCommand, self).__init__(**kwargs) - def get_sql(self): + def get_sql(self, default_conn=None): """ This method is used to create a proper SQL query to fetch the data for the specified table """ # Fetch the primary keys for the table - pk_names, primary_keys = self.get_primary_keys() + pk_names, primary_keys = self.get_primary_keys(default_conn) sql_filter = self.get_filter() @@ -362,13 +377,16 @@ class TableCommand(GridCommand): return sql - def get_primary_keys(self): + def get_primary_keys(self, default_conn=None): """ This function is used to fetch the primary key columns. """ driver = get_driver(PG_DEFAULT_DRIVER) - manager = driver.connection_manager(self.sid) - conn = manager.connection(did=self.did, conn_id=self.conn_id) + if default_conn is None: + manager = driver.connection_manager(self.sid) + conn = manager.connection(did=self.did, conn_id=self.conn_id) + else: + conn = default_conn pk_names = '' primary_keys = OrderedDict() @@ -400,7 +418,11 @@ class TableCommand(GridCommand): def can_filter(self): return True - def save(self, changed_data): + def save(self, + changed_data, + columns_info, + client_primary_key='__temp_PK', + default_conn=None): """ This function is used to save the data into the database. Depending on condition it will either update or insert the @@ -408,10 +430,16 @@ class TableCommand(GridCommand): Args: changed_data: Contains data to be saved + columns_info: + default_conn: + client_primary_key: """ - - manager = get_driver(PG_DEFAULT_DRIVER).connection_manager(self.sid) - conn = manager.connection(did=self.did, conn_id=self.conn_id) + driver = get_driver(PG_DEFAULT_DRIVER) + if default_conn is None: + manager = driver.connection_manager(self.sid) + conn = manager.connection(did=self.did, conn_id=self.conn_id) + else: + conn = default_conn status = False res = None @@ -421,14 +449,6 @@ class TableCommand(GridCommand): list_of_sql = [] _rowid = None - # Replace column positions with names - def set_column_names(data): - new_data = {} - for key in data: - new_data[changed_data['columns'][int(key)]['name']] = data[key] - - return new_data - if conn.connected(): # Start the transaction @@ -443,6 +463,20 @@ class TableCommand(GridCommand): if len(changed_data[of_type]) < 1: continue + column_type = {} + for each_col in columns_info: + if ( + columns_info[each_col]['not_null'] and + not columns_info[each_col][ + 'has_default_val'] + ): + column_data[each_col] = None + column_type[each_col] =\ + columns_info[each_col]['type_name'] + else: + column_type[each_col] = \ + columns_info[each_col]['type_name'] + # For newly added rows if of_type == 'added': @@ -451,37 +485,18 @@ class TableCommand(GridCommand): # no_default_value, set column to blank, instead # of not null which is set by default. column_data = {} - column_type = {} pk_names, primary_keys = self.get_primary_keys() - for each_col in self.columns_info: - if ( - self.columns_info[each_col]['not_null'] and - not self.columns_info[each_col][ - 'has_default_val'] - ): - column_data[each_col] = None - column_type[each_col] =\ - self.columns_info[each_col]['type_name'] - else: - column_type[each_col] = \ - self.columns_info[each_col]['type_name'] - - for each_row in changed_data[of_type]: data = changed_data[of_type][each_row]['data'] # Remove our unique tracking key - data.pop('__temp_PK', None) + data.pop(client_primary_key, None) data.pop('is_row_copied', None) - data = set_column_names(data) - data_type = set_column_names(changed_data[of_type][each_row]['data_type']) - list_of_rowid.append(data.get('__temp_PK')) + list_of_rowid.append(data.get(client_primary_key)) - # Update columns value and data type - # with columns having not_null=False and has - # no default value + # Update columns value with columns having + # not_null=False and has no default value column_data.update(data) - column_type.update(data_type) sql = render_template("/".join([self.sql_path, 'insert.sql']), data_to_be_saved=column_data, @@ -497,15 +512,14 @@ class TableCommand(GridCommand): # For updated rows elif of_type == 'updated': for each_row in changed_data[of_type]: - data = set_column_names(changed_data[of_type][each_row]['data']) - pk = set_column_names(changed_data[of_type][each_row]['primary_keys']) - data_type = set_column_names(changed_data[of_type][each_row]['data_type']) + data = changed_data[of_type][each_row]['data'] + pk = changed_data[of_type][each_row]['primary_keys'] sql = render_template("/".join([self.sql_path, 'update.sql']), data_to_be_saved=data, primary_keys=pk, object_name=self.object_name, nsp_name=self.nsp_name, - data_type=data_type) + data_type=column_type) list_of_sql.append(sql) list_of_rowid.append(data) @@ -519,18 +533,19 @@ class TableCommand(GridCommand): rows_to_delete.append(changed_data[of_type][each_row]) # Fetch the keys for SQL generation if is_first: - # We need to covert dict_keys to normal list in Python3 - # In Python2, it's already a list & We will also fetch column names using index - keys = [ - changed_data['columns'][int(k)]['name'] - for k in list(changed_data[of_type][each_row].keys()) - ] + # We need to covert dict_keys to normal list in + # Python3 + # In Python2, it's already a list & We will also + # fetch column names using index + keys = list(changed_data[of_type][each_row].keys()) + no_of_keys = len(keys) is_first = False # Map index with column name for each row for row in rows_to_delete: for k, v in row.items(): - # Set primary key with label & delete index based mapped key + # Set primary key with label & delete index based + # mapped key try: row[changed_data['columns'][int(k)]['name']] = v except ValueError: @@ -597,7 +612,7 @@ class ViewCommand(GridCommand): # call base class init to fetch the table name super(ViewCommand, self).__init__(**kwargs) - def get_sql(self): + def get_sql(self, default_conn=None): """ This method is used to create a proper SQL query to fetch the data for the specified view @@ -652,7 +667,7 @@ class ForeignTableCommand(GridCommand): # call base class init to fetch the table name super(ForeignTableCommand, self).__init__(**kwargs) - def get_sql(self): + def get_sql(self, default_conn=None): """ This method is used to create a proper SQL query to fetch the data for the specified foreign table @@ -697,7 +712,7 @@ class CatalogCommand(GridCommand): # call base class init to fetch the table name super(CatalogCommand, self).__init__(**kwargs) - def get_sql(self): + def get_sql(self, default_conn=None): """ This method is used to create a proper SQL query to fetch the data for the specified catalog object @@ -722,7 +737,7 @@ class CatalogCommand(GridCommand): return True -class QueryToolCommand(BaseCommand): +class QueryToolCommand(BaseCommand, FetchedRowTracker): """ class QueryToolCommand(BaseCommand) @@ -732,13 +747,15 @@ class QueryToolCommand(BaseCommand): def __init__(self, **kwargs): # call base class init to fetch the table name - super(QueryToolCommand, self).__init__(**kwargs) + + BaseCommand.__init__(self, **kwargs) + FetchedRowTracker.__init__(self, **kwargs) self.conn_id = None self.auto_rollback = False self.auto_commit = True - def get_sql(self): + def get_sql(self, default_conn=None): return None def can_edit(self): diff --git a/web/pgadmin/tools/sqleditor/static/css/sqleditor.css b/web/pgadmin/tools/sqleditor/static/css/sqleditor.css index 7266b34..d8f4881 100644 --- a/web/pgadmin/tools/sqleditor/static/css/sqleditor.css +++ b/web/pgadmin/tools/sqleditor/static/css/sqleditor.css @@ -423,7 +423,7 @@ input.editor-checkbox:focus { /* To highlight all newly inserted rows */ .grid-canvas .new_row { - background: #dff0d7; + background: #dff0d7 !important; } /* To highlight all the updated rows */ @@ -433,7 +433,7 @@ input.editor-checkbox:focus { /* To highlight row at fault */ .grid-canvas .new_row.error, .grid-canvas .updated_row.error { - background: #f2dede; + background: #f2dede !important; } /* Disabled row */ @@ -460,6 +460,11 @@ input.editor-checkbox:focus { background-color: #2C76B4; } +.slick-cell span[data-cell-type="row-header-selector"] { + display: block; + text-align: right; +} + #datagrid div.slick-header.ui-state-default { background: #ffffff; border-bottom: none; diff --git a/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js b/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js index e6c97ec..5e773bf 100644 --- a/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js +++ b/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js @@ -464,7 +464,7 @@ define([ - staged_rows: This will hold all the data which user copies/pastes/deletes in grid - deleted: - This will hold all the data which user delets in grid + This will hold all the data which user deletes in grid Events handling: ---------------- @@ -480,34 +480,10 @@ define([ - We are using this event for Copy operation on grid */ - // Get the item column value using a custom 'fieldIdx' column param - get_item_column_value: function (item, column) { - if (column.pos !== undefined) { - return item[column.pos]; - } else { - return null; - } - }, - // This function is responsible to create and render the SlickGrid. - render_grid: function(collection, columns, is_editable) { + render_grid: function(collection, columns, is_editable, client_primary_key, rows_affected) { var self = this; - // returns primary keys - self.handler.get_row_primary_key = function() { - var self = this, - tmp_keys = []; - _.each(self.primary_keys, function(p, idx) { - // For each columns search primary key position - _.each(self.columns, function(c) { - if(c.name == idx) { - tmp_keys.push(c.pos); - } - }); - }); - return tmp_keys; - }; - // This will work as data store and holds all the // inserted/updated/deleted data from grid self.handler.data_store = { @@ -522,8 +498,9 @@ define([ // To store primary keys before they gets changed self.handler.primary_keys_data = {}; - // Add getItemMetadata into handler for later use - self.handler.data_view = collection; + self.client_primary_key = client_primary_key; + + self.client_primary_key_counter = 0; // Remove any existing grid first if (self.handler.slickgrid) { @@ -601,7 +578,12 @@ define([ }); var gridSelector = new GridSelector(); - grid_columns = gridSelector.getColumnDefinitions(grid_columns); + grid_columns = self.grid_columns = gridSelector.getColumnDefinitions(grid_columns); + + if (rows_affected) { + // calculate with for header row column. + grid_columns[0]['width'] = SqlEditorUtils.calculateColumnWidth(rows_affected); + } var grid_options = { editable: true, @@ -609,8 +591,7 @@ define([ enableCellNavigation: true, enableColumnReorder: false, asyncEditorLoading: false, - autoEdit: false, - dataItemColumnValueExtractor: this.get_item_column_value + autoEdit: false }; var $data_grid = self.$el.find('#datagrid'); @@ -618,17 +599,15 @@ define([ var grid_height = $($('#editor-panel').find('.wcFrame')[1]).height() - 35; $data_grid.height(grid_height); - // Add our own custom primary key to keep track of changes - _.each(collection, function(row){ - row['__temp_PK'] = SqlEditorUtils.epicRandomString(15); - }); + var dataView = self.dataView = new Slick.Data.DataView(), + grid = self.grid = new Slick.Grid($data_grid, dataView, grid_columns, grid_options); // Add-on function which allow us to identify the faulty row after insert/update // and apply css accordingly - collection.getItemMetadata = function(i) { - var res = {}, - cssClass = '', - data_store = self.handler.data_store; + + dataView.getItemMetadata = function(i) { + var res = {}, cssClass = '', + data_store = self.handler.data_store; if (_.has(self.handler, 'data_store')) { if (i in data_store.added_index && @@ -651,9 +630,8 @@ define([ cssClass += ' disabled_row'; } return {'cssClasses': cssClass}; - } + }; - var grid = new Slick.Grid($data_grid, collection, grid_columns, grid_options); grid.registerPlugin( new Slick.AutoTooltips({ enableForHeaderCells: false }) ); grid.registerPlugin(new ActiveCellCapture()); grid.setSelectionModel(new XCellSelectionModel()); @@ -665,7 +643,8 @@ define([ columns: columns, grid: grid, selection: grid.getSelectionModel(), - editor: self + editor: self, + client_primary_key: self.client_primary_key }; self.handler.slickgrid = grid; @@ -684,6 +663,42 @@ define([ }); }); + gridSelector.onBeforeGridSelectAll.subscribe(function(e, args) { + if (self.handler.has_more_rows) { + // this will prevent selection un-till we load all data + e.stopImmediatePropagation(); + self.fetch_next_all(function() { + // since we've stopped event propagation we need to + // trigger onGridSelectAll manually with new event data. + gridSelector.onGridSelectAll.notify(args, new Slick.EventData()); + }); + } + }); + + gridSelector.onBeforeGridColumnSelectAll.subscribe(function(e, args) { + if (self.handler.has_more_rows) { + // this will prevent selection un-till we load all data + e.stopImmediatePropagation(); + self.fetch_next_all(function() { + // since we've stopped event propagation we need to + // trigger onGridColumnSelectAll manually with new event data. + gridSelector.onGridColumnSelectAll.notify(args, new Slick.EventData()); + }); + } + }); + + // listen for row count change. + dataView.onRowCountChanged.subscribe(function (e, args) { + grid.updateRowCount(); + grid.render(); + }); + + // listen for rows change. + dataView.onRowsChanged.subscribe(function (e, args) { + grid.invalidateRows(args.rows); + grid.render(); + }); + // Listener function which will be called before user updates existing cell // This will be used to collect primary key for that row grid.onBeforeEditCell.subscribe(function (e, args) { @@ -696,8 +711,8 @@ define([ return false; } - if(self.handler.can_edit && before_data && '__temp_PK' in before_data) { - var _pk = before_data.__temp_PK, + if(self.handler.can_edit && before_data && self.client_primary_key in before_data) { + var _pk = before_data[self.client_primary_key], _keys = self.handler.primary_keys, current_pk = {}, each_pk_key = {}; @@ -709,22 +724,34 @@ define([ // Fetch primary keys for the row before they gets modified var _columns = self.handler.columns; _.each(_keys, function(value, key) { - pos = _.where(_columns, {name: key})[0]['pos'] - current_pk[pos] = before_data[pos]; + current_pk[key] = before_data[key]; }); // Place it in main variable for later use self.handler.primary_keys_data[_pk] = current_pk } }); - grid.onKeyDown.subscribe(handleQueryOutputKeyboardEvent); + grid.onKeyDown.subscribe(function(event, args) { + var KEY_A = 65; + var modifiedKey = event.keyCode; + var isModifierDown = event.ctrlKey || event.metaKey; + // Intercept Ctrl/Cmd + A key board event. + // As we might want to load all rows before selecting all. + if (isModifierDown && modifiedKey == KEY_A && self.handler.has_more_rows) { + self.fetch_next_all(function() { + handleQueryOutputKeyboardEvent(event, args); + }); + } else { + handleQueryOutputKeyboardEvent(event, args); + } + }); // Listener function which will be called when user updates existing rows grid.onCellChange.subscribe(function (e, args) { // self.handler.data_store.updated will holds all the updated data - var changed_column = args.grid.getColumns()[args.cell].pos, // Current field pos + var changed_column = args.grid.getColumns()[args.cell].field, updated_data = args.item[changed_column], // New value for current field - _pk = args.item.__temp_PK || null, // Unique key to identify row + _pk = args.item[self.client_primary_key] || null, // Unique key to identify row column_data = {}, _type; @@ -734,11 +761,16 @@ define([ // so that cell edit is enabled for that row. var grid = args.grid, row_data = grid.getDataItem(args.row), - p_keys_list = _.pick( - row_data, self.handler.get_row_primary_key() - ), - is_primary_key = Object.keys(p_keys_list).length ? - p_keys_list[0] : undefined; + is_primary_key = _.all( + _.values( + _.pick( + row_data, self.primary_keys + ) + ), + function(val) { + return val != undefined + } + ); // temp_new_rows is available only for view data. if (is_primary_key && self.handler.temp_new_rows) { @@ -758,7 +790,6 @@ define([ column_data); //Find type for current column self.handler.data_store.added[_pk]['err'] = false - self.handler.data_store.added[_pk]['data_type'][changed_column] = _.where(this.columns, {pos: changed_column})[0]['type']; // Check if it is updated data from existing rows? } else if(_pk in self.handler.data_store.updated) { _.extend( @@ -766,9 +797,6 @@ define([ column_data ); self.handler.data_store.updated[_pk]['err'] = false - - //Find type for current column - self.handler.data_store.updated[_pk]['data_type'][changed_column] = _.where(this.columns, {pos: changed_column})[0]['type']; } else { // First updated data for this primary key self.handler.data_store.updated[_pk] = { @@ -776,32 +804,19 @@ define([ 'primary_keys': self.handler.primary_keys_data[_pk] }; self.handler.data_store.updated_index[args.row] = _pk; - // Find & add column data type for current changed column - var temp = {}; - temp[changed_column] = _.where(this.columns, {pos: changed_column})[0]['type']; - self.handler.data_store.updated[_pk]['data_type'] = temp; } } // Enable save button $("#btn-save").prop('disabled', false); }.bind(editor_data)); - grid.addBlankRow = function() { - // Add a blank row in the end of grid - this.setData(this.getData(), true); - this.updateRowCount(); - this.invalidateAllRows(); - this.render(); - }; - // Listener function which will be called when user adds new rows grid.onAddNewRow.subscribe(function (e, args) { // self.handler.data_store.added will holds all the newly added rows/data - var _key = SqlEditorUtils.epicRandomString(10), - column = args.column, - item = args.item, - data_length = this.grid.getDataLength(), - new_collection = args.grid.getData(); + var column = args.column, + item = args.item, data_length = this.grid.getDataLength(), + _key = (self.client_primary_key_counter++).toString(), + dataView = this.grid.getData(); // Add new row in list to keep track of it if (_.isUndefined(item[0])) { @@ -810,29 +825,32 @@ define([ // If copied item has already primary key, use it. if(item) { - item.__temp_PK = _key; + item[self.client_primary_key] = _key; } - new_collection.push(item); + dataView.addItem(item); self.handler.data_store.added[_key] = {'err': false, 'data': item}; self.handler.data_store.added_index[data_length] = _key; // Fetch data type & add it for the column var temp = {}; - temp[column.pos] = _.where(this.columns, {pos: column.pos})[0]['type']; - self.handler.data_store.added[_key]['data_type'] = temp; - grid.invalidateRows([new_collection.length - 1]); + temp[column.name] = _.where(this.columns, {pos: column.pos})[0]['type']; grid.updateRowCount(); grid.render(); - // Add a blank row after add row - if (!args.is_copy_row) { - grid.addBlankRow(); - } - // Enable save button $("#btn-save").prop('disabled', false); }.bind(editor_data)); + // Listen grid viewportChanged event to load next chunk of data. + grid.onViewportChanged.subscribe(function(e, args) { + var rendered_range = args.grid.getRenderedRange(), + data_len = args.grid.getDataLength(); + // start fetching next batch of records before reaching to bottom. + if (self.handler.has_more_rows && !self.handler.fetching_rows && rendered_range.bottom > data_len - 100) { + // fetch asynchronous + setTimeout(self.fetch_next.bind(self)); + } + }) // Resize SlickGrid when window resize $( window ).resize( function() { // Resize grid only when 'Data Output' panel is visible. @@ -855,6 +873,84 @@ define([ if(self.data_output_panel.isVisible()) self.grid_resize(grid); }); + + for (var i = 0; i < collection.length; i++) { + // Convert to dict from 2darray + var item = {}; + for (var j = 1; j < grid_columns.length; j++) { + item[grid_columns[j]['field']] = collection[i][grid_columns[j]['pos']] + } + + item[self.client_primary_key] = (self.client_primary_key_counter++).toString(); + collection[i] = item; + } + dataView.setItems(collection, self.client_primary_key); + }, + fetch_next_all(cb) { + this.fetch_next(true, cb); + }, + fetch_next: function(fetch_all, cb) { + var self = this; + + // This will prevent fetch operation if previous fetch operation is + // already in progress. + self.handler.fetching_rows = true; + + $("#btn-flash").prop('disabled', true); + + if (fetch_all) { + self.handler.trigger( + 'pgadmin-sqleditor:loading-icon:show', + gettext('Fetching all records...') + ); + } + + $.ajax({ + url: "{{ url_for('sqleditor.index') }}" + "fetch/" + self.transId + (fetch_all ? "/1": ""), + method: 'GET', + success: function(res) { + self.handler.has_more_rows = res.data.has_more_rows; + $("#btn-flash").prop('disabled', false); + self.handler.trigger('pgadmin-sqleditor:loading-icon:hide'); + self.update_grid_data(res.data.result); + self.handler.fetching_rows = false; + if (typeof cb == "function") { + cb(); + } + }, + error: function(e) { + $("#btn-flash").prop('disabled', false); + self.handler.trigger('pgadmin-sqleditor:loading-icon:hide'); + self.handler.has_more_rows = false; + self.handler.fetching_rows = false; + if (typeof cb == "function") { + cb(); + } + if (e.readyState == 0) { + self.update_msg_history(false, + gettext('Not connected to the server or the connection to the server has been closed.') + ); + return; + } + } + }); + }, + + update_grid_data: function(data) { + this.dataView.beginUpdate(); + + for (var i = 0; i < data.length; i++) { + // Convert 2darray to dict. + var item = {}; + for (var j = 1; j < this.grid_columns.length; j++) { + item[this.grid_columns[j]['field']] = data[i][this.grid_columns[j]['pos']] + } + + item[this.client_primary_key] = (this.client_primary_key_counter++).toString(); + this.dataView.addItem(item); + } + + this.dataView.endUpdate(); }, /* This function is responsible to render output grid */ @@ -1396,6 +1492,8 @@ define([ self.explain_buffers = false; self.explain_timing = false; self.is_new_browser_tab = is_new_browser_tab; + self.has_more_rows = false; + self.fetching_rows = false; // We do not allow to call the start multiple times. if (self.gridView) @@ -1503,6 +1601,8 @@ define([ self.rows_to_disable = new Array(); // Temporarily hold new rows added self.temp_new_rows = new Array(); + self.has_more_rows = false; + self.fetching_rows = false; self.trigger( 'pgadmin-sqleditor:loading-icon:show', @@ -1581,45 +1681,14 @@ define([ }); }, - // This function makes the ajax call to fetch columns for last async query, - get_columns: function(poll_result) { - var self = this; - // Check the flag and decide if we need to fetch columns from server - // or use the columns data stored locally from previous call? - if (self.FETCH_COLUMNS_FROM_SERVER) { - $.ajax({ - url: "{{ url_for('sqleditor.index') }}" + "columns/" + self.transId, - method: 'GET', - success: function(res) { - poll_result.colinfo = res.data.columns; - poll_result.primary_keys = res.data.primary_keys; - self.call_render_after_poll(poll_result); - // Set a flag to get columns to false & set the value for future use - self.FETCH_COLUMNS_FROM_SERVER = false; - self.COLUMNS_DATA = res; - }, - error: function(e) { - var msg = e.responseText; - if (e.responseJSON != undefined && e.responseJSON.errormsg != undefined) - msg = e.responseJSON.errormsg; - alertify.error(msg, 5); - } - }); - } else { - // Use the previously saved columns data - poll_result.colinfo = self.COLUMNS_DATA.data.columns; - poll_result.primary_keys = self.COLUMNS_DATA.data.primary_keys; - self.call_render_after_poll(poll_result); - } - }, - // This is a wrapper to call _render function // We need this because we have separated columns route & result route // We need to combine both result here in wrapper before rendering grid call_render_after_poll: function(res) { var self = this; self.query_end_time = new Date(); - self.rows_affected = res.rows_affected; + self.rows_affected = res.rows_affected, + self.has_more_rows = res.has_more_rows; /* If no column information is available it means query runs successfully with no result to display. In this @@ -1668,7 +1737,8 @@ define([ 'pgadmin-sqleditor:loading-icon:message', gettext("Loading data from the database server and rendering...") ); - self.get_columns(res.data); + + self.call_render_after_poll(res.data); } else if (res.data.status === 'Busy') { // If status is Busy then poll the result by recursive call to the poll function @@ -1728,6 +1798,7 @@ define([ var self = this; self.colinfo = data.col_info; self.primary_keys = data.primary_keys; + self.client_primary_key = data.client_primary_key; self.cell_selected = false; self.selected_model = null; self.changedModels = []; @@ -1780,7 +1851,7 @@ define([ self.total_time = self.get_query_run_time(self.query_start_time, self.query_end_time); self.update_msg_history(true, "", false); var msg1 = S(gettext("Total query runtime: %s.")).sprintf(self.total_time).value(); - var msg2 = S(gettext("%s rows retrieved.")).sprintf(self.rows_affected).value(); + var msg2 = S(gettext("%s rows affected.")).sprintf(self.rows_affected).value(); // Display the notifier if the timeout is set to >= 0 if (self.info_notifier_timeout >= 0) { @@ -1816,7 +1887,8 @@ define([ setTimeout( function() { self.gridView.render_grid( - explain_data_array, self.columns, self.can_edit + explain_data_array, self.columns, self.can_edit, + self.client_primary_key ); // Make sure - the 'Explain' panel is visible, before - we // start rendering the grid. @@ -1832,7 +1904,8 @@ define([ self.gridView.data_output_panel.focus(); setTimeout( function() { - self.gridView.render_grid(data.result, self.columns, self.can_edit); + self.gridView.render_grid(data.result, self.columns, + self.can_edit, self.client_primary_key, data.rows_affected); }, 10 ); } @@ -1840,136 +1913,114 @@ define([ // Hide the loading icon self.trigger('pgadmin-sqleditor:loading-icon:hide'); $("#btn-flash").prop('disabled', false); - }.bind(self), - function() { - this.trigger('pgadmin-sqleditor:loading-icon:hide'); - $("#btn-flash").prop('disabled', false); }.bind(self) ); }, // This function creates the columns as required by the backgrid - _fetch_column_metadata: function(data, cb, _fail) { + _fetch_column_metadata: function(data, cb) { var colinfo = data.colinfo, primary_keys = data.primary_keys, result = data.result, columns = [], self = this; + // Store pg_types in an array + var pg_types = new Array(); + _.each(data.types, function(r) { + pg_types[r.oid] = [r.typname]; + }); - self.trigger( - 'pgadmin-sqleditor:loading-icon:message', - gettext("Retrieving information about the columns returned...") - ); - - // Make ajax call to fetch the pg types to map numeric data type - $.ajax({ - url: "{{ url_for('sqleditor.index') }}" + "fetch/types/" + self.transId, - method: 'GET', - success: function(res) { - if (res.data.status) { - // Store pg_types in an array - var pg_types = new Array(); - _.each(res.data.result.rows, function(r) { - pg_types[r.oid] = [r.typname]; - }); - - // Create columns required by backgrid to render - _.each(colinfo, function(c) { - var is_primary_key = false; + // Create columns required by slick grid to render + _.each(colinfo, function(c) { + var is_primary_key = false; - // Check whether table have primary key - if (_.size(primary_keys) > 0) { - _.each(primary_keys, function (value, key) { - if (key === c.name) - is_primary_key = true; - }); - } + // Check whether table have primary key + if (_.size(primary_keys) > 0) { + _.each(primary_keys, function (value, key) { + if (key === c.name) + is_primary_key = true; + }); + } - // To show column label and data type in multiline, - // The elements should be put inside the div. - // Create column label and type. - var col_type = column_label = ''; - var type = pg_types[c.type_code] ? - pg_types[c.type_code][0] : - // This is the case where user might - // have use casting so we will use type - // returned by cast function - pg_types[pg_types.length - 1][0] ? - pg_types[pg_types.length - 1][0] : 'unknown'; - - if (!is_primary_key) - col_type += ' ' + type; - else - col_type += ' [PK] ' + type; + // To show column label and data type in multiline, + // The elements should be put inside the div. + // Create column label and type. + var col_type = column_label = ''; + var type = pg_types[c.type_code] ? + pg_types[c.type_code][0] : + // This is the case where user might + // have use casting so we will use type + // returned by cast function + pg_types[pg_types.length - 1][0] ? + pg_types[pg_types.length - 1][0] : 'unknown'; + + if (!is_primary_key) + col_type += ' ' + type; + else + col_type += ' [PK] ' + type; - if (c.precision && c.precision >= 0 && c.precision != 65535) { - col_type += ' (' + c.precision; - col_type += c.scale && c.scale != 65535 ? - ',' + c.scale + ')': - ')'; - } + if (c.precision && c.precision >= 0 && c.precision != 65535) { + col_type += ' (' + c.precision; + col_type += c.scale && c.scale != 65535 ? + ',' + c.scale + ')': + ')'; + } - // Identify cell type of column. - switch(type) { - case "json": - case "json[]": - case "jsonb": - case "jsonb[]": - col_cell = 'Json'; - break; - case "smallint": - case "integer": - case "bigint": - case "decimal": - case "numeric": - case "real": - case "double precision": - col_cell = 'number'; - break; - case "boolean": - col_cell = 'boolean'; - break; - case "character": - case "character[]": - case "character varying": - case "character varying[]": - if (c.internal_size && c.internal_size >= 0 && c.internal_size != 65535) { - // Update column type to display length on column header - col_type += ' (' + c.internal_size + ')'; - } - col_cell = 'string'; - break; - default: - col_cell = 'string'; - } + // Identify cell type of column. + switch(type) { + case "json": + case "json[]": + case "jsonb": + case "jsonb[]": + col_cell = 'Json'; + break; + case "smallint": + case "integer": + case "bigint": + case "decimal": + case "numeric": + case "real": + case "double precision": + col_cell = 'number'; + break; + case "boolean": + col_cell = 'boolean'; + break; + case "character": + case "character[]": + case "character varying": + case "character varying[]": + if (c.internal_size && c.internal_size >= 0 && c.internal_size != 65535) { + // Update column type to display length on column header + col_type += ' (' + c.internal_size + ')'; + } + col_cell = 'string'; + break; + default: + col_cell = 'string'; + } - column_label = c.display_name + '<br>' + col_type; - - var col = { - 'name': c.name, - 'pos': c.pos, - 'label': column_label, - 'cell': col_cell, - 'can_edit': self.can_edit, - 'type': type, - 'display_name': c.display_name, - 'column_type': col_type, - 'not_null': c.not_null, - 'has_default_val': c.has_default_val - }; - columns.push(col); - }); - } - else { - alertify.alert('Fetching Type Error', res.data.result); - } - self.columns = columns; - if (cb && typeof(cb) == 'function') { - cb(); - } - }, - fail: _fail + column_label = c.display_name + '<br>' + col_type; + + var col = { + 'name': c.name, + 'display_name': c.display_name, + 'column_type': col_type, + 'pos': c.pos, + 'label': column_label, + 'cell': col_cell, + 'can_edit': self.can_edit, + 'type': type, + 'not_null': c.not_null, + 'has_default_val': c.has_default_val + }; + columns.push(col); }); + + self.columns = columns; + if (cb && typeof(cb) == 'function') { + cb(); + } }, // This function is used to raise appropriate message. @@ -1980,22 +2031,21 @@ define([ self.gridView.messages_panel.focus(); - if (self.is_query_tool) { - if (clear_grid) { - // Delete grid - if (self.gridView.handler.slickgrid) { - self.gridView.handler.slickgrid.destroy(); - - } - // Misc cleaning - self.columns = undefined; - self.collection = undefined; + if (clear_grid) { + // Delete grid + if (self.gridView.handler.slickgrid) { + self.gridView.handler.slickgrid.destroy(); - $('.sql-editor-message').text(msg); - } else { - $('.sql-editor-message').append(msg); } + // Misc cleaning + self.columns = undefined; + self.collection = undefined; + + $('.sql-editor-message').text(msg); + } else { + $('.sql-editor-message').append(msg); } + // Scroll automatically when msgs appends to element setTimeout(function(){ $(".sql-editor-message").scrollTop($(".sql-editor-message")[0].scrollHeight);; @@ -2049,7 +2099,7 @@ define([ rows_to_delete: function(data) { var self = this, - tmp_keys = self.get_row_primary_key.call(self); + tmp_keys = self.primary_keys; // re-calculate rows with no primary keys self.temp_new_rows = []; @@ -2062,7 +2112,6 @@ define([ self.temp_new_rows.push(idx); } }); - data.getItemMetadata = self.data_view.getItemMetadata; self.rows_to_disable = _.clone(self.temp_new_rows); }, @@ -2073,69 +2122,73 @@ define([ is_added = _.size(self.data_store.added), is_updated = _.size(self.data_store.updated); - // Remove newly added rows from staged rows as we don't want to send them on server - if(is_added) { - _.each(self.data_store.added, function(val, key) { - if(key in self.data_store.staged_rows) { - // Remove the row from data store so that we do not send it on server - deleted_keys.push(key); - delete self.data_store.staged_rows[key]; - delete self.data_store.added[key] - } - }); + // Remove newly added rows from staged rows as we don't want to send them on server + if(is_added) { + _.each(self.data_store.added, function(val, key) { + if(key in self.data_store.staged_rows) { + // Remove the row from data store so that we do not send it on server + deleted_keys.push(key); + delete self.data_store.staged_rows[key]; + delete self.data_store.added[key]; + delete self.data_store.added_index[key]; } - - // If only newly rows to delete and no data is there to send on server - // then just re-render the grid - if(_.size(self.data_store.staged_rows) == 0) { - var grid = self.slickgrid, data = grid.getData(), idx = 0; - if(deleted_keys.length){ - // Remove new rows from grid data using deleted keys - data = _.reject(data, function(d){ - return (d && _.indexOf(deleted_keys, d.__temp_PK) > -1) - }); - } - self.rows_to_delete.apply(self, [data]); - grid.resetActiveCell(); - grid.setData(data, true); - grid.setSelectedRows([]); - grid.invalidate(); - // Nothing to copy or delete here - $("#btn-delete-row").prop('disabled', true); - $("#btn-copy-row").prop('disabled', true); - if(_.size(self.data_store.added) || is_updated) { - // Do not disable save button if there are - // any other changes present in grid data - $("#btn-save").prop('disabled', false); - } else { - $("#btn-save").prop('disabled', true); - } - alertify.success(gettext("Row(s) deleted")); + }); + } + // If only newly rows to delete and no data is there to send on server + // then just re-render the grid + if(_.size(self.data_store.staged_rows) == 0) { + var grid = self.slickgrid, + dataView = grid.getData(), + data = dataView.getItems(), + idx = 0; + + grid.resetActiveCell(); + + dataView.beginUpdate(); + for (var i = 0; i < deleted_keys.length; i++) { + dataView.deleteItem(deleted_keys[i]); + } + dataView.endUpdate(); + self.rows_to_delete.apply(self, [dataView.getItems()]); + grid.resetActiveCell(); + grid.setSelectedRows([]); + grid.invalidate(); + + // Nothing to copy or delete here + $("#btn-delete-row").prop('disabled', true); + $("#btn-copy-row").prop('disabled', true); + if(_.size(self.data_store.added) || is_updated) { + // Do not disable save button if there are + // any other changes present in grid data + $("#btn-save").prop('disabled', false); } else { - // There are other data to needs to be updated on server - if(is_updated) { - alertify.alert(gettext("Operation failed"), + $("#btn-save").prop('disabled', true); + } + alertify.success(gettext("Row(s) deleted")); + } else { + // There are other data to needs to be updated on server + if(is_updated) { + alertify.alert(gettext("Operation failed"), gettext("There are unsaved changes in grid, Please save them first to avoid inconsistency in data") ); - return; - } - alertify.confirm(gettext("Delete Row(s)"), + return; + } + alertify.confirm(gettext("Delete Row(s)"), gettext("Are you sure you wish to delete selected row(s)?"), - function() { - $("#btn-delete-row").prop('disabled', true); - $("#btn-copy-row").prop('disabled', true); - // Change the state - self.data_store.deleted = self.data_store.staged_rows; - self.data_store.staged_rows = {}; - // Save the changes on server - self._save(); - }, - function() { - // Do nothing as user canceled the operation. - } - ).set('labels', {ok:'Yes', cancel:'No'}); + function() { + $("#btn-delete-row").prop('disabled', true); + $("#btn-copy-row").prop('disabled', true); + // Change the state + self.data_store.deleted = self.data_store.staged_rows; + self.data_store.staged_rows = {}; + // Save the changes on server + self._save(); + }, + function() { + // Do nothing as user canceled the operation. } - + ).set('labels', {ok: gettext("Yes"), cancel:gettext("No")}); + } }, /* This function will fetch the list of changed models and make @@ -2197,7 +2250,9 @@ define([ data: JSON.stringify(req_data), success: function(res) { var grid = self.slickgrid, - data = grid.getData(); + dataView = grid.getData(), + data_length = dataView.getLength(), + data = []; if (res.data.status) { // Remove flag is_row_copied from copied rows _.each(data, function(row, idx) { @@ -2214,22 +2269,20 @@ define([ // Remove deleted rows from client as well if(is_deleted) { var rows = grid.getSelectedRows(); - /* In JavaScript sorting by default is lexical, - * To make sorting numerical we need to pass function - * After that we will Reverse the order of sorted array - * so that when we remove it does not affect array index - */ - if(data.length == rows.length) { + if(data_length == rows.length) { // This means all the rows are selected, clear all data data = []; + dataView.setItems(data, self.client_primary_key); } else { - rows = rows.sort(function(a,b){return a - b}).reverse(); - rows.forEach(function(idx) { - data.splice(idx, 1); - }); + dataView.beginUpdate(); + for (var i = 0; i < rows.length; i++) { + item = grid.getDataItem(rows[i]); + data.push(item); + dataView.deleteItem(item[self.client_primary_key]); + } + dataView.endUpdate(); } self.rows_to_delete.apply(self, [data]); - grid.setData(data, true); grid.setSelectedRows([]); } @@ -2242,6 +2295,7 @@ define([ self.rows_to_disable = _.clone(self.temp_new_rows); } + grid.setSelectedRows([]); // Reset data store self.data_store = { 'added': {}, @@ -2262,7 +2316,7 @@ define([ $('.sql-editor-message').text(res.data.result); var err_msg = S(gettext("%s.")).sprintf(res.data.result).value(); alertify.notify(err_msg, 'error', 20); - + grid.setSelectedRows([]); // To highlight the row at fault if(_.has(res.data, '_rowid') && (!_.isUndefined(res.data._rowid)|| !_.isNull(res.data._rowid))) { @@ -2318,14 +2372,21 @@ define([ // Find index of row at fault from grid data _find_rowindex: function(rowid) { - var self = this; - var grid = self.slickgrid, - data = grid.getData(), _rowid, count = 0, _idx = -1; + var self = this, + grid = self.slickgrid, + dataView = grid.getData(), + data = dataView.getItems(), + _rowid, + count = 0, + _idx = -1; + // If _rowid is object then it's update/delete operation if(_.isObject(rowid)) { _rowid = rowid; - } else if (_.isString(rowid)) { // Insert opration - _rowid = { '__temp_PK': rowid }; + } else if (_.isString(rowid)) { // Insert operation + var rowid = {}; + rowid[self.client_primary_key]= rowid; + _rowid = rowid; } else { // Something is wrong with unique id return _idx; @@ -2547,11 +2608,6 @@ define([ // This function will set the required flag for polling response data _init_polling_flags: function() { var self = this; - // Set a flag to get columns - self.FETCH_COLUMNS_FROM_SERVER = true; - // We will set columns data in this variable for future use once we fetch it - // from server - self.COLUMNS_DATA = {}; // To get a timeout for polling fallback timer in seconds in // regards to elapsed time @@ -2647,7 +2703,7 @@ define([ return; // Add column position and it's value to data - data[column_info.field] = _values[column_info.pos] || ''; + data[column_info.field] = _values[column_info.field] || ''; self.trigger( 'pgadmin-sqleditor:loading-icon:show', @@ -2717,7 +2773,7 @@ define([ return; // Add column position and it's value to data - data[column_info.field] = _values[column_info.pos] || ''; + data[column_info.field] = _values[column_info.field] || ''; self.trigger( 'pgadmin-sqleditor:loading-icon:show', @@ -2880,16 +2936,18 @@ define([ _paste_row: function() { var self = this, col_info = {}, grid = self.slickgrid, - data = grid.getData(), - count = Object.keys(data).length-1; - - var rows = grid.getSelectedRows().sort( + dataView = grid.getData(), + data = dataView.getItems(), + count = dataView.getLength(), + rows = grid.getSelectedRows().sort( function (a, b) { return a - b; } ), - rows = rows.length == 0 ? self.last_copied_rows : rows, copied_rows = rows.map(function (rowIndex) { return data[rowIndex]; }); + + rows = rows.length == 0 ? self.last_copied_rows : rows + self.last_copied_rows = rows; // If there are rows to paste? @@ -2922,22 +2980,22 @@ define([ // Add index of copied row into temp_new_rows // Trigger grid.onAddNewRow when a row is copied // Reset selection + + dataView.beginUpdate(); _.each(copied_rows, function(row) { - var new_row = arr_to_object(row); + var new_row = arr_to_object(row), + _key = (self.gridView.client_primary_key_counter++).toString(); new_row.is_row_copied = true; - row = new_row; self.temp_new_rows.push(count); - grid.onAddNewRow.notify( - { item: new_row, - column: self.columns[0], - grid: grid, - is_copy_row: true - } - ) + new_row[self.client_primary_key] = _key; + dataView.addItem(new_row); + self.data_store.added[_key] = {'err': false, 'data': new_row}; + self.data_store.added_index[count] = _key; count++; }); - // Add a blank row after copy/paste row - grid.addBlankRow(); + dataView.endUpdate(); + grid.updateRowCount(); + grid.render(); grid.setSelectedRows([]); } }, @@ -3009,6 +3067,9 @@ define([ sql = '', history_msg = ''; + self.has_more_rows = false; + self.fetching_rows = false; + /* If code is selected in the code mirror then execute * the selected part else execute the complete code. */ diff --git a/web/pgadmin/utils/driver/abstract.py b/web/pgadmin/utils/driver/abstract.py index 9b2363c..7db3e37 100644 --- a/web/pgadmin/utils/driver/abstract.py +++ b/web/pgadmin/utils/driver/abstract.py @@ -101,6 +101,12 @@ class BaseConnection(object): - Implement this method to execute the given query and returns the result as an array of dict (column name -> value) format. + * def async_fetchmany_2darray(records=-1, formatted_exception_msg=False): + - Implement this method to retrieve result of asynchronous connection and + polling with no_result flag set to True. + This returns the result as a 2 dimensional array. + If records is -1 then fetchmany will behave as fetchall. + * connected() - Implement this method to get the status of the connection. It should return True for connected, otherwise False @@ -133,7 +139,7 @@ class BaseConnection(object): - Implement this method to wait for asynchronous connection with timeout. This must be a non blocking call. - * poll(formatted_exception_msg) + * poll(formatted_exception_msg, no_result) - Implement this method to poll the data of query running on asynchronous connection. @@ -180,6 +186,10 @@ class BaseConnection(object): pass @abstractmethod + def async_fetchmany_2darray(self, records=-1, formatted_exception_msg=False): + pass + + @abstractmethod def connected(self): pass @@ -208,7 +218,7 @@ class BaseConnection(object): pass @abstractmethod - def poll(self, formatted_exception_msg=True): + def poll(self, formatted_exception_msg=True, no_result=False): pass @abstractmethod diff --git a/web/pgadmin/utils/driver/psycopg2/__init__.py b/web/pgadmin/utils/driver/psycopg2/__init__.py index 876b887..848058d 100644 --- a/web/pgadmin/utils/driver/psycopg2/__init__.py +++ b/web/pgadmin/utils/driver/psycopg2/__init__.py @@ -1078,6 +1078,55 @@ Failed to execute query (execute_void) for the server #{server_id} - {conn_id} return True, {'columns': columns, 'rows': rows} + def async_fetchmany_2darray(self, records=2000, formatted_exception_msg=False): + """ + User should poll and check if status is ASYNC_OK before calling this + function + Args: + records: no of records to fetch. use -1 to fetchall. + formatted_exception_msg: + + Returns: + + """ + cur = self.__async_cursor + if not cur: + return False, gettext( + "Cursor could not be found for the async connection." + ) + + if self.conn.isexecuting(): + return False, gettext( + "Asynchronous query execution/operation underway." + ) + + if self.row_count > 0: + result = [] + # For DDL operation, we may not have result. + # + # Because - there is not direct way to differentiate DML and + # DDL operations, we need to rely on exception to figure + # that out at the moment. + try: + if records == -1: + res = cur.fetchall() + else: + res = cur.fetchmany(records) + for row in res: + new_row = [] + for col in self.column_info: + new_row.append(row[col['name']]) + result.append(new_row) + except psycopg2.ProgrammingError as e: + result = None + else: + # User performed operation which dose not produce record/s as + # result. + # for eg. DDL operations. + return True, None + + return True, result + def connected(self): if self.conn: if not self.conn.closed: @@ -1224,7 +1273,7 @@ Failed to reset the connection to the server due to following error: "poll() returned %s from _wait_timeout function" % state ) - def poll(self, formatted_exception_msg=False): + def poll(self, formatted_exception_msg=False, no_result=False): """ This function is a wrapper around connection's poll function. It internally uses the _wait_timeout method to poll the @@ -1234,6 +1283,7 @@ Failed to reset the connection to the server due to following error: Args: formatted_exception_msg: if True then function return the formatted exception message, otherwise error string. + no_result: If True then only poll status will be returned. """ cur = self.__async_cursor @@ -1289,23 +1339,23 @@ Failed to reset the connection to the server due to following error: pos += 1 self.row_count = cur.rowcount - - if cur.rowcount > 0: - result = [] - # For DDL operation, we may not have result. - # - # Because - there is not direct way to differentiate DML and - # DDL operations, we need to rely on exception to figure that - # out at the moment. - try: - for row in cur: - new_row = [] - for col in self.column_info: - new_row.append(row[col['name']]) - result.append(new_row) - - except psycopg2.ProgrammingError: - result = None + if not no_result: + if cur.rowcount > 0: + result = [] + # For DDL operation, we may not have result. + # + # Because - there is not direct way to differentiate DML and + # DDL operations, we need to rely on exception to figure + # that out at the moment. + try: + for row in cur: + new_row = [] + for col in self.column_info: + new_row.append(row[col['name']]) + result.append(new_row) + + except psycopg2.ProgrammingError: + result = None return status, result diff --git a/web/regression/feature_utils/pgadmin_page.py b/web/regression/feature_utils/pgadmin_page.py index c61663a..424ff56 100644 --- a/web/regression/feature_utils/pgadmin_page.py +++ b/web/regression/feature_utils/pgadmin_page.py @@ -85,7 +85,7 @@ class PgadminPage: if 'menu-item' == str(menu_item.get_attribute('class')): break - time.sleep(0.1) + time.sleep(0.1) else: assert False, "'Tools -> Query Tool' menu did not enable." @@ -251,6 +251,20 @@ class PgadminPage: self._wait_for("spinner to disappear", spinner_has_disappeared) + def wait_for_query_tool_loading_indicator_to_disappear(self): + def spinner_has_disappeared(driver): + try: + driver.find_element_by_xpath( + "//*[@id='fetching_data' and @class='hide']" + ) + return False + except NoSuchElementException: + # wait for loading indicator disappear animation to complete. + time.sleep(0.5) + return True + + self._wait_for("spinner to disappear", spinner_has_disappeared) + def wait_for_app(self): def page_shows_app(driver): if driver.title == self.app_config.APP_NAME: @@ -266,19 +280,3 @@ class PgadminPage: timeout = self.timeout return WebDriverWait(self.driver, timeout, 0.01).until(condition_met_function, "Timed out waiting for " + waiting_for_message) - - def wait_for_element_to_stale(self, xpath): - # Reference: http://www.obeythetestinggoat.com/ - # how-to-get-selenium-to-wait-for-page-load-after-a-click.html - el = self.driver.find_element_by_xpath(xpath) - - def element_has_gone_stale(driver): - try: - # poll an arbitrary element - el.find_elements_by_id('element-dont-exist') - return False - except StaleElementReferenceException: - return True - - self._wait_for("element to attach to the page document", - element_has_gone_stale) diff --git a/web/regression/javascript/selection/copy_data_spec.js b/web/regression/javascript/selection/copy_data_spec.js index 9e05e84..2b623b3 100644 --- a/web/regression/javascript/selection/copy_data_spec.js +++ b/web/regression/javascript/selection/copy_data_spec.js @@ -16,16 +16,16 @@ import clipboard from '../../../pgadmin/static/js/selection/clipboard'; import copyData from '../../../pgadmin/static/js/selection/copy_data'; import RangeSelectionHelper from 'sources/selection/range_selection_helper'; import XCellSelectionModel from 'sources/selection/xcell_selection_model'; - describe('copyData', function () { var grid, sqlEditor, gridContainer, buttonPasteRow; var SlickGrid; beforeEach(function () { SlickGrid = Slick.Grid; - var data = [[1, 'leopord', '12'], - [2, 'lion', '13'], - [3, 'puma', '9']]; + var data = [{'id': 1, 'brand':'leopord', 'size':'12', '__temp_PK': '123'}, + {'id': 2, 'brand':'lion', 'size':'13', '__temp_PK': '456'}, + {'id': 3, 'brand':'puma', 'size':'9', '__temp_PK': '789'}], + dataView = new Slick.Data.DataView(); var columns = [ { @@ -37,6 +37,7 @@ describe('copyData', function () { }, { name: 'id', + field: 'id', pos: 0, label: 'id<br> numeric', cell: 'number', @@ -44,6 +45,7 @@ describe('copyData', function () { type: 'numeric', }, { name: 'brand', + field: 'brand', pos: 1, label: 'flavor<br> character varying', cell: 'string', @@ -51,24 +53,26 @@ describe('copyData', function () { type: 'character varying', }, { name: 'size', + field: 'size', pos: 2, label: 'size<br> numeric', cell: 'number', can_edit: false, type: 'numeric', }, - ] - ; - gridContainer = $('<div id=\'grid\'></div>'); + ]; + gridContainer = $('<div id="grid"></div>'); $('body').append(gridContainer); - buttonPasteRow = $('<button id=\'btn-paste-row\' disabled></button>'); + buttonPasteRow = $('<button id="btn-paste-row" disabled></button>'); $('body').append(buttonPasteRow); - grid = new SlickGrid('#grid', data, columns, {}); + grid = new SlickGrid('#grid', dataView, columns, {}); + dataView.setItems(data, '__temp_PK'); grid.setSelectionModel(new XCellSelectionModel()); sqlEditor = {slickgrid: grid}; }); - afterEach(function () { + afterEach(function() { + grid.destroy(); gridContainer.remove(); buttonPasteRow.remove(); }); diff --git a/web/regression/javascript/selection/range_boundary_navigator_spec.js b/web/regression/javascript/selection/range_boundary_navigator_spec.js index 1de6d81..2d56b05 100644 --- a/web/regression/javascript/selection/range_boundary_navigator_spec.js +++ b/web/regression/javascript/selection/range_boundary_navigator_spec.js @@ -134,17 +134,19 @@ describe('RangeBoundaryNavigator', function () { describe('#rangesToCsv', function () { var data, columnDefinitions, ranges; beforeEach(function () { - data = [[1, 'leopard', '12'], - [2, 'lion', '13'], - [3, 'cougar', '9'], - [4, 'tiger', '10']]; - columnDefinitions = [{name: 'id', pos: 0}, {name: 'animal', pos: 1}, {name: 'size', pos: 2}]; + data = [{'id':1, 'animal':'leopard', 'size':'12'}, + {'id':2, 'animal':'lion', 'size':'13'}, + {'id':3, 'animal':'cougar', 'size':'9'}, + {'id':4, 'animal':'tiger', 'size':'10'}]; + + columnDefinitions = [{name: 'id', field: 'id', pos: 0}, + {name: 'animal', field: 'animal', pos: 1}, + {name: 'size', field: 'size', pos: 2}]; ranges = [new Slick.Range(0, 0, 0, 2), new Slick.Range(3, 0, 3, 2)]; }); it('returns csv for the provided ranges', function () { var csvResult = rangeBoundaryNavigator.rangesToCsv(data, columnDefinitions, ranges); - expect(csvResult).toEqual('1,\'leopard\',\'12\'\n4,\'tiger\',\'10\''); }); @@ -158,10 +160,10 @@ describe('RangeBoundaryNavigator', function () { describe('when there is an extra column with checkboxes', function () { beforeEach(function () { - columnDefinitions = [{name: 'not-a-data-column'}, {name: 'id', pos: 0}, {name: 'animal', pos: 1}, { - name: 'size', - pos: 2, - }]; + columnDefinitions = [{name: 'not-a-data-column'}, + {name: 'id', field: 'id', pos: 0}, + {name: 'animal', field: 'animal', pos: 1}, + {name: 'size', field: 'size',pos: 2}]; ranges = [new Slick.Range(0, 0, 0, 3), new Slick.Range(3, 0, 3, 3)]; }); diff --git a/web/regression/javascript/selection/row_selector_spec.js b/web/regression/javascript/selection/row_selector_spec.js index db3c0b4..b6d4d01 100644 --- a/web/regression/javascript/selection/row_selector_spec.js +++ b/web/regression/javascript/selection/row_selector_spec.js @@ -6,7 +6,6 @@ // This software is released under the PostgreSQL Licence // ////////////////////////////////////////////////////////////////////////// - import $ from 'jquery'; import Slick from 'slickgrid'; @@ -23,7 +22,7 @@ describe('RowSelector', function () { UP: 38, DOWN: 40, }; - var container, data, columnDefinitions, grid, cellSelectionModel; + var container, dataView, columnDefinitions, grid, cellSelectionModel; var SlickGrid = Slick.Grid; beforeEach(function () { @@ -43,14 +42,15 @@ describe('RowSelector', function () { pos: 1, }]; + dataView = new Slick.Data.DataView(); var rowSelector = new RowSelector(); - data = []; + var data = []; for (var i = 0; i < 10; i++) { - data.push(['some-value-' + i, 'second value ' + i]); + data.push({'some-column-name':'some-value-' + i, 'second column':'second value ' + i}); } columnDefinitions = rowSelector.getColumnDefinitions(columnDefinitions); - grid = new SlickGrid(container, data, columnDefinitions); - + dataView.setItems(data, 'some-column-name'); + grid = new SlickGrid(container, dataView, columnDefinitions); grid.registerPlugin(new ActiveCellCapture()); cellSelectionModel = new XCellSelectionModel(); grid.setSelectionModel(cellSelectionModel); diff --git a/web/regression/javascript/selection/set_staged_rows_spec.js b/web/regression/javascript/selection/set_staged_rows_spec.js index 6735314..6fd780a 100644 --- a/web/regression/javascript/selection/set_staged_rows_spec.js +++ b/web/regression/javascript/selection/set_staged_rows_spec.js @@ -7,236 +7,247 @@ // ////////////////////////////////////////////////////////////// -define([ - 'jquery', - 'underscore', - 'sources/selection/set_staged_rows', -], function ($, _, SetStagedRows) { - describe('set_staged_rows', function () { - var sqlEditorObj, gridSpy, deleteButton, copyButton, selectionSpy; - beforeEach(function () { - gridSpy = jasmine.createSpyObj('gridSpy', ['getData', 'getCellNode', 'getColumns']); - gridSpy.getData.and.returnValue([ - {0: 'one', 1: 'two', __temp_PK: '123'}, - {0: 'three', 1: 'four', __temp_PK: '456'}, - {0: 'five', 1: 'six', __temp_PK: '789'}, - {0: 'seven', 1: 'eight', __temp_PK: '432'}, - ]); - gridSpy.getColumns.and.returnValue([ +import $ from 'jquery'; +import 'slickgrid.grid'; +import Slick from 'slickgrid'; +import SetStagedRows from 'sources/selection/set_staged_rows'; + +describe('set_staged_rows', function () { + var sqlEditorObj, gridSpy, deleteButton, copyButton, selectionSpy; + beforeEach(function () { + var data = [{'a pk column': 'one', 'some column': 'two', '__temp_PK': '123'}, + {'a pk column': 'three', 'some column': 'four', '__temp_PK': '456'}, + {'a pk column': 'five', 'some column': 'six', '__temp_PK': '789'}, + {'a pk column': 'seven', 'some column': 'eight', '__temp_PK': '432'}], + dataView = new Slick.Data.DataView(); + + dataView.setItems(data, '__temp_PK'); + + gridSpy = jasmine.createSpyObj('gridSpy', ['getData', 'getCellNode', 'getColumns']); + gridSpy.getData.and.returnValue(dataView); + gridSpy.getColumns.and.returnValue([ + { + name: 'a pk column', + field: 'a pk column', + pos: 0, + selectable: true, + }, { + name: 'some column', + field: 'some column', + pos: 1, + selectable: true, + }, + ]); + selectionSpy = jasmine.createSpyObj('selectionSpy', ['setSelectedRows', 'getSelectedRanges']); + deleteButton = $('<button id="btn-delete-row"></button>'); + copyButton = $('<button id="btn-copy-row"></button>'); + + sqlEditorObj = { + grid: gridSpy, + editor: { + handler: { + data_store: { + staged_rows: {'456': {}}, + }, + can_edit: false, + }, + }, + keys: null, + selection: selectionSpy, + columns: [ { + name: 'a pk column', + field: 'a pk column', pos: 0, - selectable: true, - }, { + }, + { + name: 'some column', + field: 'some column', pos: 1, - selectable: true, }, - ]); + ], + client_primary_key: '__temp_PK', + }; - selectionSpy = jasmine.createSpyObj('selectionSpy', ['setSelectedRows', 'getSelectedRanges']); + $('body').append(deleteButton); + $('body').append(copyButton); - deleteButton = $('<button id="btn-delete-row"></button>'); - copyButton = $('<button id="btn-copy-row"></button>'); + deleteButton.prop('disabled', true); + copyButton.prop('disabled', true); - sqlEditorObj = { - grid: gridSpy, - editor: { - handler: { - data_store: { - staged_rows: {'456': {}}, - }, - can_edit: false, - }, - }, - keys: null, - selection: selectionSpy, - columns: [ - { - name: 'a pk column', - pos: 0, - }, - { - name: 'some column', - pos: 1, - }, - ], - }; + selectionSpy = jasmine.createSpyObj('selectionSpy', [ + 'setSelectedRows', + 'getSelectedRanges', + ]); + }); - $('body').append(deleteButton); - $('body').append(copyButton); + afterEach(function () { + copyButton.remove(); + deleteButton.remove(); + }); + describe('when no full rows are selected', function () { + describe('when nothing is selected', function () { + beforeEach(function () { + selectionSpy.getSelectedRanges.and.returnValue([]); + sqlEditorObj.selection = selectionSpy; + SetStagedRows.call(sqlEditorObj, {}, {}); + }); - deleteButton.prop('disabled', true); - copyButton.prop('disabled', true); + it('should disable the delete row button', function () { + expect($('#btn-delete-row').prop('disabled')).toBeTruthy(); + }); - selectionSpy = jasmine.createSpyObj('selectionSpy', [ - 'setSelectedRows', - 'getSelectedRanges', - ]); - }); + it('should disable the copy row button', function () { + expect($('#btn-copy-row').prop('disabled')).toBeTruthy(); + }); - afterEach(function () { - copyButton.remove(); - deleteButton.remove(); + it('should clear staged rows', function () { + expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({}); + }); }); - describe('when no full rows are selected', function () { - describe('when nothing is selected', function () { - beforeEach(function () { - selectionSpy.getSelectedRanges.and.returnValue([]); - sqlEditorObj.selection = selectionSpy; - SetStagedRows.call(sqlEditorObj, {}, {}); - }); - it('should disable the delete row button', function () { - expect($('#btn-delete-row').prop('disabled')).toBeTruthy(); - }); + describe('when there is a selection', function () { + beforeEach(function () { + var range = { + fromCell: 0, + toCell: 0, + fromRow: 1, + toRow: 1, + }; - it('should disable the copy row button', function () { - expect($('#btn-copy-row').prop('disabled')).toBeTruthy(); - }); + selectionSpy.getSelectedRanges.and.returnValue([range]); + sqlEditorObj.selection = selectionSpy; + SetStagedRows.call(sqlEditorObj, {}, {}); + }); - it('should clear staged rows', function () { - expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({}); - }); + it('should disable the delete row button', function () { + expect($('#btn-delete-row').prop('disabled')).toBeTruthy(); }); - describe('when there is a selection', function () { - beforeEach(function () { - var range = { - fromCell: 0, - toCell: 0, - fromRow: 1, - toRow: 1, - }; - - selectionSpy.getSelectedRanges.and.returnValue([range]); - sqlEditorObj.selection = selectionSpy; - SetStagedRows.call(sqlEditorObj, {}, {}); - }); + it('should disable the copy row button', function () { + expect($('#btn-copy-row').prop('disabled')).toBeFalsy(); + }); - it('should disable the delete row button', function () { - expect($('#btn-delete-row').prop('disabled')).toBeTruthy(); - }); + it('should clear staged rows', function () { + expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({}); + }); + }); + }); - it('should disable the copy row button', function () { - expect($('#btn-copy-row').prop('disabled')).toBeFalsy(); - }); + describe('when 2 full rows are selected', function () { + beforeEach(function () { + var range1 = { + fromCell: 0, + toCell: 1, + fromRow: 1, + toRow: 1, + }; + var range2 = { + fromCell: 0, + toCell: 1, + fromRow: 2, + toRow: 2, + }; - it('should clear staged rows', function () { - expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({}); - }); - }); + selectionSpy.getSelectedRanges.and.returnValue([range1, range2]); + sqlEditorObj.selection = selectionSpy; }); - describe('when 2 full rows are selected', function () { - beforeEach(function () { - var range1 = { - fromCell: 0, - toCell: 1, - fromRow: 1, - toRow: 1, - }; - var range2 = { - fromCell: 0, - toCell: 1, - fromRow: 2, - toRow: 2, - }; + describe('when table does not have primary keys', function () { + it('should enable the copy row button', function () { + SetStagedRows.call(sqlEditorObj, {}, {}); + expect($('#btn-copy-row').prop('disabled')).toBeFalsy(); + }); - selectionSpy.getSelectedRanges.and.returnValue([range1, range2]); - sqlEditorObj.selection = selectionSpy; + it('should not enable the delete row button', function () { + SetStagedRows.call(sqlEditorObj, {}, {}); + expect($('#btn-delete-row').prop('disabled')).toBeTruthy(); }); - describe('when table does not have primary keys', function () { - it('should enable the copy row button', function () { - SetStagedRows.call(sqlEditorObj, {}, {}); - expect($('#btn-copy-row').prop('disabled')).toBeFalsy(); - }); + it('should update staged rows with the __temp_PK value of the new Selected Rows', function () { + SetStagedRows.call(sqlEditorObj, {}, {}); + expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({'456': {}, '789': {}}); + }); - it('should not enable the delete row button', function () { + describe('the user can edit', function () { + it('should enable the delete row button', function () { + sqlEditorObj.editor.handler.can_edit = true; SetStagedRows.call(sqlEditorObj, {}, {}); - expect($('#btn-delete-row').prop('disabled')).toBeTruthy(); + expect($('#btn-delete-row').prop('disabled')).toBeFalsy(); }); + }); + }); - it('should update staged rows with the __temp_PK value of the new Selected Rows', function () { + describe('when table has primary keys', function () { + beforeEach(function () { + sqlEditorObj.keys = {'a pk column': 'varchar'}; + sqlEditorObj.editor.handler.data_store.staged_rows = {'456': {'a pk column': 'three'}}; + }); + + describe('selected rows have primary key', function () { + it('should set the staged rows correctly', function () { SetStagedRows.call(sqlEditorObj, {}, {}); - expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({'456': {}, '789': {}}); + expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual( + {'456': {'a pk column': 'three'}, '789': {'a pk column': 'five'}}); }); - describe('the user can edit', function () { - it('should enable the delete row button', function () { - sqlEditorObj.editor.handler.can_edit = true; - SetStagedRows.call(sqlEditorObj, {}, {}); - expect($('#btn-delete-row').prop('disabled')).toBeFalsy(); - }); + it('should not clear selected rows in Cell Selection Model', function () { + SetStagedRows.call(sqlEditorObj, {}, {}); + expect(sqlEditorObj.selection.setSelectedRows).not.toHaveBeenCalledWith(); }); }); - describe('when table has primary keys', function () { + describe('selected rows missing primary key', function () { beforeEach(function () { - sqlEditorObj.keys = {'a pk column': 'varchar'}; - sqlEditorObj.editor.handler.data_store.staged_rows = {'456': {0: 'three'}}; - }); + var data = [{'a pk column': 'one', 'some column': 'two', '__temp_PK': '123'}, + {'some column': 'four', '__temp_PK': '456'}, + {'some column': 'six', '__temp_PK': '789'}, + {'a pk column': 'seven', 'some column': 'eight', '__temp_PK': '432'}], + dataView = new Slick.Data.DataView(); - describe('selected rows have primary key', function () { - it('should set the staged rows correctly', function () { - SetStagedRows.call(sqlEditorObj, {}, {}); - expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual( - {'456': {0: 'three'}, '789': {0: 'five'}}); - }); + dataView.setItems(data, '__temp_PK'); - it('should not clear selected rows in Cell Selection Model', function () { - SetStagedRows.call(sqlEditorObj, {}, {}); - expect(sqlEditorObj.selection.setSelectedRows).not.toHaveBeenCalledWith(); - }); + gridSpy.getData.and.returnValue(dataView); }); - describe('selected rows missing primary key', function () { - beforeEach(function () { - gridSpy.getData.and.returnValue([ - {0: 'one', 1: 'two', __temp_PK: '123'}, - {1: 'four', __temp_PK: '456'}, - {1: 'six', __temp_PK: '789'}, - {0: 'seven', 1: 'eight', __temp_PK: '432'}, - ]); - }); - - it('should clear the staged rows', function () { - SetStagedRows.call(sqlEditorObj, {}, {}); - expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({}); - }); + it('should clear the staged rows', function () { + SetStagedRows.call(sqlEditorObj, {}, {}); + expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({}); + }); - it('should clear selected rows in Cell Selection Model', function () { - SetStagedRows.call(sqlEditorObj, {}, {}); - expect(sqlEditorObj.selection.setSelectedRows).toHaveBeenCalledWith([]); - }); + it('should clear selected rows in Cell Selection Model', function () { + SetStagedRows.call(sqlEditorObj, {}, {}); + expect(sqlEditorObj.selection.setSelectedRows).toHaveBeenCalledWith([]); }); + }); - describe('when the selected row is a new row', function () { - var parentDiv; - beforeEach(function () { - var childDiv = $('<div></div>'); - parentDiv = $('<div class="new_row"></div>'); - parentDiv.append(childDiv); - $('body').append(parentDiv); - gridSpy.getCellNode.and.returnValue(childDiv); - SetStagedRows.call(sqlEditorObj, {}, {}); - }); + describe('when the selected row is a new row', function () { + var parentDiv; + beforeEach(function () { + var childDiv = $('<div></div>'); + parentDiv = $('<div class="new_row"></div>'); + parentDiv.append(childDiv); + $('body').append(parentDiv); + gridSpy.getCellNode.and.returnValue(childDiv); + SetStagedRows.call(sqlEditorObj, {}, {}); + }); - afterEach(function () { - parentDiv.remove(); - }); + afterEach(function () { + parentDiv.remove(); + }); - it('should not clear the staged rows', function () { - expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({ - '456': {0: 'three'}, - '789': {0: 'five'}, - }); + it('should not clear the staged rows', function () { + expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({ + '456': {'a pk column': 'three'}, + '789': {'a pk column': 'five'}, }); + }); - it('should not clear selected rows in Cell Selection Model', function () { - expect(sqlEditorObj.selection.setSelectedRows).not.toHaveBeenCalled(); - }); + it('should not clear selected rows in Cell Selection Model', function () { + expect(sqlEditorObj.selection.setSelectedRows).not.toHaveBeenCalled(); }); }); }); }); }); + diff --git a/web/regression/javascript/selection/xcell_selection_model_spec.js b/web/regression/javascript/selection/xcell_selection_model_spec.js index f41d96c..a7f19aa 100644 --- a/web/regression/javascript/selection/xcell_selection_model_spec.js +++ b/web/regression/javascript/selection/xcell_selection_model_spec.js @@ -33,14 +33,17 @@ describe('XCellSelectionModel', function () { }, { id: '1', name: 'some-column-name', + field: 'some-column-name', pos: 0, }, { id: 'second-column-id', name: 'second column', + field: 'second column', pos: 1, }, { id: 'third-column-id', name: 'third column', + field: 'third column', pos: 2, }, ]; @@ -52,13 +55,15 @@ describe('XCellSelectionModel', function () { 'second column': 'second value ' + i, 'third column': 'third value ' + i, 'fourth column': 'fourth value ' + i, + '__temp_PK': '123' + i, }); } container = $('<div></div>'); + var dataView = new Slick.Data.DataView(); container.height(9999); container.width(9999); - - grid = new SlickGrid(container, data, columns); + dataView.setItems(data, '__temp_PK'); + grid = new SlickGrid(container, dataView, columns); grid.setSelectionModel(new XCellSelectionModel()); $('body').append(container); }); diff --git a/web/regression/javascript/slickgrid/event_handlers/handle_query_output_keyboard_event_spec.js b/web/regression/javascript/slickgrid/event_handlers/handle_query_output_keyboard_event_spec.js index e2edaf7..807e468 100644 --- a/web/regression/javascript/slickgrid/event_handlers/handle_query_output_keyboard_event_spec.js +++ b/web/regression/javascript/slickgrid/event_handlers/handle_query_output_keyboard_event_spec.js @@ -28,21 +28,22 @@ describe('#handleQueryOutputKeyboardEvent', function () { metaKey: false, which: -1, keyCode: -1, - preventDefault: jasmine.createSpy('preventDefault'), + preventDefault: jasmine.createSpy('preventDefault') }; - var data = [['', '0,0-cell-content', '0,1-cell-content'], - ['', '1,0-cell-content', '1,1-cell-content'], - ['', '2,0-cell-content', '2,1-cell-content']]; - var columnDefinitions = [{name: 'checkboxColumn'}, {pos: 1, name: 'firstColumn'}, { - pos: 2, - name: 'secondColumn', - }]; - grid = new SlickGrid($('<div></div>'), data, columnDefinitions); - grid.setSelectionModel(new XCellSelectionModel()); + var data = [{'checkboxColumn': '', 'firstColumn': '0,0-cell-content', 'secondColumn': '0,1-cell-content', '__temp_PK': '123'}, + {'checkboxColumn': '', 'firstColumn': '1,0-cell-content', 'secondColumn': '1,1-cell-content', '__temp_PK': '456'}, + {'checkboxColumn': '', 'firstColumn': '2,0-cell-content', 'secondColumn': '2,1-cell-content', '__temp_PK': '789'}], + columnDefinitions = [{name: 'checkboxColumn'}, + {pos: 1, name: 'firstColumn', field: 'firstColumn'}, + { pos: 2, name: 'secondColumn', field: 'secondColumn'}], + dataView = new Slick.Data.DataView(); + grid = new Slick.Grid($('<div></div>'), dataView, columnDefinitions); + grid.setSelectionModel(new XCellSelectionModel()); + dataView.setItems(data, '__temp_PK'); slickEvent = { - grid: grid, + grid: grid }; spyOn(clipboard, 'copyTextToClipboard'); ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: [pgadmin-hackers] Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] @ 2017-06-26 11:52 Dave Page <[email protected]> parent: Harshal Dhumal <[email protected]> 0 siblings, 1 reply; 24+ messages in thread From: Dave Page @ 2017-06-26 11:52 UTC (permalink / raw) To: Harshal Dhumal <[email protected]>; +Cc: Harshal Dhumal <[email protected]>; Ashesh Vashi <[email protected]>; pgadmin-hackers I'm sorry, it needs rebasing again. If you can do it quickly, I'll make sure it's the next patch I work on in that area. Thanks. On Mon, Jun 26, 2017 at 5:16 AM, Harshal Dhumal <[email protected]> wrote: > Hi Dave, > > Please find updated rebased patch for RM2137 > > On Fri, Jun 23, 2017 at 9:00 PM, Dave Page <[email protected]> wrote: >> >> Hi Harshal, >> >> When can we expect an updated version of this patch? I think it's >> important to get this into the next release. >> >> Thanks! >> >> On Fri, Jun 16, 2017 at 10:55 AM, Dave Page <[email protected]> wrote: >> > Hi, >> > >> > That's better - the failures are far less random now :-). I got the >> > following two though, on both PG and EPAS 9.5: >> > >> > ====================================================================== >> > ERROR: runTest >> > (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >> > Query tool feature test >> > ---------------------------------------------------------------------- >> > Traceback (most recent call last): >> > File >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", >> > line 95, in runTest >> > self._query_tool_explain_analyze_buffers() >> > File >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", >> > line 443, in _query_tool_explain_analyze_buffers >> > canvas.find_element_by_xpath("//*[contains(string(), 'Shared Read >> > Blocks')]") >> > File >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", >> > line 260, in find_element_by_xpath >> > return self.find_element(by=By.XPATH, value=xpath) >> > File >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", >> > line 508, in find_element >> > {"using": by, "value": value})['value'] >> > File >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", >> > line 491, in _execute >> > return self._parent.execute(command, params) >> > File >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> > line 238, in execute >> > self.error_handler.check_response(response) >> > File >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", >> > line 193, in check_response >> > raise exception_class(message, screen, stacktrace) >> > NoSuchElementException: Message: no such element: Unable to locate >> > element: {"method":"xpath","selector":"//*[contains(string(), 'Shared >> > Read Blocks')]"} >> > (Session info: chrome=58.0.3029.110) >> > (Driver info: chromedriver=2.29.461585 >> > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 >> > x86_64) >> > >> > >> > ====================================================================== >> > ERROR: runTest >> > (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest) >> > Validate Insert, Update operations in View data with given test data >> > ---------------------------------------------------------------------- >> > Traceback (most recent call last): >> > File >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py", >> > line 104, in runTest >> > self._add_row() >> > File >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py", >> > line 255, in _add_row >> > self._update_cell(cell_xpath, config_data[str(idx)]) >> > File >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py", >> > line 164, in _update_cell >> > cell_el = self.page.find_by_xpath(xpath) >> > File >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py", >> > line 122, in find_by_xpath >> > return self.wait_for_element(lambda driver: >> > driver.find_element_by_xpath(xpath)) >> > File >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py", >> > line 205, in wait_for_element >> > return self._wait_for("element to exist", element_if_it_exists) >> > File >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py", >> > line 255, in _wait_for >> > "Timed out waiting for " + waiting_for_message) >> > File >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/support/wait.py", >> > line 71, in until >> > value = method(self._driver) >> > File >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgadmin_page.py", >> > line 200, in element_if_it_exists >> > if element.is_displayed() and element.is_enabled(): >> > File >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", >> > line 358, in is_displayed >> > return self._execute(Command.IS_ELEMENT_DISPLAYED)['value'] >> > File >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", >> > line 491, in _execute >> > return self._parent.execute(command, params) >> > File >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> > line 238, in execute >> > self.error_handler.check_response(response) >> > File >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", >> > line 193, in check_response >> > raise exception_class(message, screen, stacktrace) >> > StaleElementReferenceException: Message: stale element reference: >> > element is not attached to the page document >> > (Session info: chrome=58.0.3029.110) >> > (Driver info: chromedriver=2.29.461585 >> > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 >> > x86_64) >> > > > > I checked my feature test cases for any database server version/type (PG, > EPAS) specific failures and I found that test cases are working fine on all > of them. However I have slightly modified test case to overcome above > mentioned failures. > >> >> > >> > On 9.4 everything passes. On 9.6, I get one failure: >> > >> > ====================================================================== >> > FAIL: runTest >> > (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest) >> > Validate Insert, Update operations in View data with given test data >> > ---------------------------------------------------------------------- >> > Traceback (most recent call last): >> > File >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py", >> > line 105, in runTest >> > self._verify_row_data(True) >> > File >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_data_dml_queries.py", >> > line 282, in _verify_row_data >> > self.assertEquals(cells[idx], config_data[str(idx)][1]) >> > AssertionError: u'[default]' != u'1' >> > - [default] >> > + 1 >> > > > I guess Khushboo has fixed this recently. > > >> >> > Screenshots attached. >> > >> > On Fri, Jun 16, 2017 at 7:54 AM, Harshal Dhumal >> > <[email protected]> wrote: >> >> Hi Dave, >> >> >> >> Please find attached patch where I have added timeout of 2 seconds >> >> before >> >> selecting object menu in connect to server function. >> >> >> >> Note: Apply this patch on top of previous patch for on demand loading >> >> feature. >> >> >> >> >> >> -- >> >> Harshal Dhumal >> >> Sr. Software Engineer >> >> >> >> EnterpriseDB India: http://www.enterprisedb.com >> >> The Enterprise PostgreSQL Company >> >> >> >> On Fri, Jun 16, 2017 at 2:25 AM, Dave Page <[email protected]> wrote: >> >>> >> >>> Sounds good, thanks. >> >>> >> >>> On Thu, Jun 15, 2017 at 9:54 PM, Harshal Dhumal >> >>> <[email protected]> wrote: >> >>> > Hi >> >>> > >> >>> > On Fri, Jun 16, 2017 at 2:07 AM, Dave Page <[email protected]> >> >>> > wrote: >> >>> >> >> >>> >> Hi >> >>> >> >> >>> >> On Thu, Jun 15, 2017 at 9:30 PM, Harshal Dhumal >> >>> >> <[email protected]> wrote: >> >>> >> > Hi Dave, >> >>> >> > >> >>> >> > Please find attached updated patch. >> >>> >> > >> >>> >> > On Thu, Jun 15, 2017 at 3:58 PM, Dave Page <[email protected]> >> >>> >> > wrote: >> >>> >> >> >> >>> >> >> Hi >> >>> >> >> >> >>> >> >> On Wed, Jun 14, 2017 at 11:36 PM, Harshal Dhumal >> >>> >> >> <[email protected]> wrote: >> >>> >> >>> >> >>> >> >>> Hi Dave, >> >>> >> >>> >> >>> >> >>> Please find rebased patch for RM2137. >> >>> >> >> >> >>> >> >> >> >>> >> >> Looking very good. The only issues I see are: >> >>> >> >> >> >>> >> >> - The row headers should auto-size such that they can display >> >>> >> >> the >> >>> >> >> row >> >>> >> >> numbers if the last row was displayed. E.g. if there are 12345 >> >>> >> >> rows >> >>> >> >> in >> >>> >> >> total, then the row header should be sized to display 5 digits. >> >>> >> >> >> >>> >> > Fixed. >> >>> >> > >> >>> >> > >> >>> >> >> >> >>> >> >> - The tests are comprehensive, which is awesome. However, every >> >>> >> >> time >> >>> >> >> I >> >>> >> >> ran >> >>> >> >> them, at least one of the feature tests failed. Unfortunately, >> >>> >> >> it >> >>> >> >> was a >> >>> >> >> different one each time. In the last two runs, I got: >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> ====================================================================== >> >>> >> >> ERROR: runTest >> >>> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >> >>> >> >> Query tool feature test >> >>> >> >> >> >>> >> >> >> >>> >> >> ---------------------------------------------------------------------- >> >>> >> >> Traceback (most recent call last): >> >>> >> >> File >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> "/Users/dpage/git/pgadmin4/web/regression/feature_utils/base_feature_test.py", >> >>> >> >> line 40, in setUp >> >>> >> >> self.before() >> >>> >> >> File >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", >> >>> >> >> line 40, in before >> >>> >> >> self._connects_to_server() >> >>> >> >> File >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", >> >>> >> >> line 144, in _connects_to_server >> >>> >> >> self.page.driver.find_element_by_link_text("Create"))\ >> >>> >> >> File >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> >>> >> >> line 319, in find_element_by_link_text >> >>> >> >> return self.find_element(by=By.LINK_TEXT, value=link_text) >> >>> >> >> File >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> >>> >> >> line 756, in find_element >> >>> >> >> 'value': value})['value'] >> >>> >> >> File >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> >>> >> >> line 238, in execute >> >>> >> >> self.error_handler.check_response(response) >> >>> >> >> File >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", >> >>> >> >> line 193, in check_response >> >>> >> >> raise exception_class(message, screen, stacktrace) >> >>> >> >> NoSuchElementException: Message: no such element: Unable to >> >>> >> >> locate >> >>> >> >> element: {"method":"link text","selector":"Create"} >> >>> >> >> (Session info: chrome=58.0.3029.110) >> >>> >> >> (Driver info: chromedriver=2.29.461585 >> >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X >> >>> >> >> 10.12.3 >> >>> >> >> x86_64) >> >>> >> >> >> >>> >> > I checked code from _connects_to_server function which is common >> >>> >> > in >> >>> >> > all >> >>> >> > features test cases. I didn't find anything wrong with this. If >> >>> >> > there >> >>> >> > is >> >>> >> > a >> >>> >> > bug in this function then all feature test must fail. >> >>> >> > Let me know if you are getting failure consistently in >> >>> >> > _connects_to_server >> >>> >> > function. >> >>> >> >> >>> >> I wondered if that one is a race condition. Do we need a short >> >>> >> delay >> >>> >> before clicking the Object menu? I have seen this occasionally >> >>> >> before. >> >>> > >> >>> > >> >>> > OK. In that case let's try putting 1-2 second delay and observer >> >>> > behaviour. >> >>> > I'll send separate patch for this tomorrow as this is not related to >> >>> > on >> >>> > demand query result feature or its test cases. >> >>> > >> >>> >> >> >>> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> ====================================================================== >> >>> >> >> ERROR: runTest >> >>> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >> >>> >> >> Query tool feature test >> >>> >> >> >> >>> >> >> >> >>> >> >> ---------------------------------------------------------------------- >> >>> >> >> Traceback (most recent call last): >> >>> >> >> File >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", >> >>> >> >> line 119, in runTest >> >>> >> >> self._query_tool_auto_rollback_enabled() >> >>> >> >> File >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_tool_tests.py", >> >>> >> >> line 697, in _query_tool_auto_rollback_enabled >> >>> >> >> '//div[contains(@class, "sql-editor-message") and >> >>> >> >> contains(string(), >> >>> >> >> "COMMIT")]' >> >>> >> >> File >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> >>> >> >> line 295, in find_element_by_xpath >> >>> >> >> return self.find_element(by=By.XPATH, value=xpath) >> >>> >> >> File >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> >>> >> >> line 756, in find_element >> >>> >> >> 'value': value})['value'] >> >>> >> >> File >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", >> >>> >> >> line 238, in execute >> >>> >> >> self.error_handler.check_response(response) >> >>> >> >> File >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", >> >>> >> >> line 193, in check_response >> >>> >> >> raise exception_class(message, screen, stacktrace) >> >>> >> >> NoSuchElementException: Message: no such element: Unable to >> >>> >> >> locate >> >>> >> >> element: {"method":"xpath","selector":"//div[contains(@class, >> >>> >> >> "sql-editor-message") and contains(string(), "COMMIT")]"} >> >>> >> >> (Session info: chrome=58.0.3029.110) >> >>> >> >> (Driver info: chromedriver=2.29.461585 >> >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X >> >>> >> >> 10.12.3 >> >>> >> >> x86_64) >> >>> >> >> >> >>> >> > I have updated Auto rollback enabled test in this patch. >> >>> >> > >> >>> >> >> >> >>> >> >> Relevant screenshots attached. >> >>> >> >> >> >>> >> >> - Can you tidy up the regression output a little please? Instead >> >>> >> >> of: >> >>> >> >> >> >>> >> >> ------- >> >>> >> >> runTest >> >>> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >> >>> >> >> Query tool feature test ... On demand result set on scrolling... >> >>> >> >> OK. >> >>> >> >> >> >>> >> >> On demand result set on grid select all... >> >>> >> >> OK. >> >>> >> >> >> >>> >> >> On demand result set on column select all... >> >>> >> >> OK. >> >>> >> >> >> >>> >> >> Explain query... >> >>> >> >> OK. >> >>> >> >> >> >>> >> >> Explain query with verbose... >> >>> >> >> OK. >> >>> >> >> >> >>> >> >> Explain query with costs... >> >>> >> >> OK. >> >>> >> >> >> >>> >> >> Explain analyze query... >> >>> >> >> OK. >> >>> >> >> >> >>> >> >> Explain analyze query with buffers... >> >>> >> >> OK. >> >>> >> >> >> >>> >> >> Explain analyze query with timing... >> >>> >> >> OK. >> >>> >> >> >> >>> >> >> Auto commit disabled... >> >>> >> >> OK. >> >>> >> >> >> >>> >> >> Auto commit enabled... >> >>> >> >> OK. >> >>> >> >> >> >>> >> >> Auto rollback enabled... >> >>> >> >> ERROR >> >>> >> >> ------- >> >>> >> >> >> >>> >> >> Something like: >> >>> >> >> >> >>> >> >> ------- >> >>> >> >> runTest >> >>> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >> >>> >> >> Query tool feature test ... >> >>> >> >> On demand result set on scrolling... OK. >> >>> >> >> On demand result set on grid select all... OK. >> >>> >> >> On demand result set on column select all... OK. >> >>> >> >> Explain query... OK. >> >>> >> >> Explain query with verbose... OK. >> >>> >> >> Explain query with costs... OK. >> >>> >> >> Explain analyze query... OK. >> >>> >> >> Explain analyze query with buffers... OK. >> >>> >> >> Explain analyze query with timing... OK. >> >>> >> >> Auto commit disabled... OK. >> >>> >> >> Auto commit enabled... OK. >> >>> >> >> Auto rollback enabled... ERROR >> >>> >> >> -------- >> >>> >> >> >> >>> >> > Fixed. >> >>> >> > >> >>> >> > >> >>> >> >> >> >>> >> >> Thanks! >> >>> >> >> >> >>> >> >> -- >> >>> >> >> Dave Page >> >>> >> >> Blog: http://pgsnake.blogspot.com >> >>> >> >> Twitter: @pgsnake >> >>> >> >> >> >>> >> >> EnterpriseDB UK: http://www.enterprisedb.com >> >>> >> >> The Enterprise PostgreSQL Company >> >>> >> > >> >>> >> > >> >>> >> >> >>> >> >> >>> >> >> >>> >> -- >> >>> >> Dave Page >> >>> >> Blog: http://pgsnake.blogspot.com >> >>> >> Twitter: @pgsnake >> >>> >> >> >>> >> EnterpriseDB UK: http://www.enterprisedb.com >> >>> >> The Enterprise PostgreSQL Company >> >>> > >> >>> > >> >>> >> >>> >> >>> >> >>> -- >> >>> Dave Page >> >>> Blog: http://pgsnake.blogspot.com >> >>> Twitter: @pgsnake >> >>> >> >>> EnterpriseDB UK: http://www.enterprisedb.com >> >>> The Enterprise PostgreSQL Company >> >> >> >> >> > >> > >> > >> > -- >> > Dave Page >> > Blog: http://pgsnake.blogspot.com >> > Twitter: @pgsnake >> > >> > EnterpriseDB UK: http://www.enterprisedb.com >> > The Enterprise PostgreSQL Company >> >> >> >> -- >> Dave Page >> Blog: http://pgsnake.blogspot.com >> Twitter: @pgsnake >> >> EnterpriseDB UK: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company > > -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: [pgadmin-hackers] Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] @ 2017-06-26 11:54 Harshal Dhumal <[email protected]> parent: Dave Page <[email protected]> 0 siblings, 1 reply; 24+ messages in thread From: Harshal Dhumal @ 2017-06-26 11:54 UTC (permalink / raw) To: Dave Page <[email protected]>; +Cc: Harshal Dhumal <[email protected]>; Ashesh Vashi <[email protected]>; pgadmin-hackers yes i'm working on that only :) -- *Harshal Dhumal* *Sr. Software Engineer* EnterpriseDB India: http://www.enterprisedb.com The Enterprise PostgreSQL Company On Mon, Jun 26, 2017 at 5:22 PM, Dave Page <[email protected]> wrote: > I'm sorry, it needs rebasing again. If you can do it quickly, I'll > make sure it's the next patch I work on in that area. > > Thanks. > > On Mon, Jun 26, 2017 at 5:16 AM, Harshal Dhumal > <[email protected]> wrote: > > Hi Dave, > > > > Please find updated rebased patch for RM2137 > > > > On Fri, Jun 23, 2017 at 9:00 PM, Dave Page <[email protected]> wrote: > >> > >> Hi Harshal, > >> > >> When can we expect an updated version of this patch? I think it's > >> important to get this into the next release. > >> > >> Thanks! > >> > >> On Fri, Jun 16, 2017 at 10:55 AM, Dave Page <[email protected]> wrote: > >> > Hi, > >> > > >> > That's better - the failures are far less random now :-). I got the > >> > following two though, on both PG and EPAS 9.5: > >> > > >> > ============================================================ > ========== > >> > ERROR: runTest > >> > (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) > >> > Query tool feature test > >> > ------------------------------------------------------------ > ---------- > >> > Traceback (most recent call last): > >> > File > >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > query_tool_tests.py", > >> > line 95, in runTest > >> > self._query_tool_explain_analyze_buffers() > >> > File > >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > query_tool_tests.py", > >> > line 443, in _query_tool_explain_analyze_buffers > >> > canvas.find_element_by_xpath("//*[contains(string(), 'Shared Read > >> > Blocks')]") > >> > File > >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webelement.py", > >> > line 260, in find_element_by_xpath > >> > return self.find_element(by=By.XPATH, value=xpath) > >> > File > >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webelement.py", > >> > line 508, in find_element > >> > {"using": by, "value": value})['value'] > >> > File > >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webelement.py", > >> > line 491, in _execute > >> > return self._parent.execute(command, params) > >> > File > >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> > line 238, in execute > >> > self.error_handler.check_response(response) > >> > File > >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/errorhandler.py", > >> > line 193, in check_response > >> > raise exception_class(message, screen, stacktrace) > >> > NoSuchElementException: Message: no such element: Unable to locate > >> > element: {"method":"xpath","selector":"//*[contains(string(), 'Shared > >> > Read Blocks')]"} > >> > (Session info: chrome=58.0.3029.110) > >> > (Driver info: chromedriver=2.29.461585 > >> > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 > >> > x86_64) > >> > > >> > > >> > ============================================================ > ========== > >> > ERROR: runTest > >> > (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest) > >> > Validate Insert, Update operations in View data with given test data > >> > ------------------------------------------------------------ > ---------- > >> > Traceback (most recent call last): > >> > File > >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > view_data_dml_queries.py", > >> > line 104, in runTest > >> > self._add_row() > >> > File > >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > view_data_dml_queries.py", > >> > line 255, in _add_row > >> > self._update_cell(cell_xpath, config_data[str(idx)]) > >> > File > >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > view_data_dml_queries.py", > >> > line 164, in _update_cell > >> > cell_el = self.page.find_by_xpath(xpath) > >> > File > >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/ > pgadmin_page.py", > >> > line 122, in find_by_xpath > >> > return self.wait_for_element(lambda driver: > >> > driver.find_element_by_xpath(xpath)) > >> > File > >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/ > pgadmin_page.py", > >> > line 205, in wait_for_element > >> > return self._wait_for("element to exist", element_if_it_exists) > >> > File > >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/ > pgadmin_page.py", > >> > line 255, in _wait_for > >> > "Timed out waiting for " + waiting_for_message) > >> > File > >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/support/wait.py", > >> > line 71, in until > >> > value = method(self._driver) > >> > File > >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/ > pgadmin_page.py", > >> > line 200, in element_if_it_exists > >> > if element.is_displayed() and element.is_enabled(): > >> > File > >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webelement.py", > >> > line 358, in is_displayed > >> > return self._execute(Command.IS_ELEMENT_DISPLAYED)['value'] > >> > File > >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webelement.py", > >> > line 491, in _execute > >> > return self._parent.execute(command, params) > >> > File > >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> > line 238, in execute > >> > self.error_handler.check_response(response) > >> > File > >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/errorhandler.py", > >> > line 193, in check_response > >> > raise exception_class(message, screen, stacktrace) > >> > StaleElementReferenceException: Message: stale element reference: > >> > element is not attached to the page document > >> > (Session info: chrome=58.0.3029.110) > >> > (Driver info: chromedriver=2.29.461585 > >> > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 > >> > x86_64) > >> > > > > > > > I checked my feature test cases for any database server version/type (PG, > > EPAS) specific failures and I found that test cases are working fine on > all > > of them. However I have slightly modified test case to overcome above > > mentioned failures. > > > >> > >> > > >> > On 9.4 everything passes. On 9.6, I get one failure: > >> > > >> > ============================================================ > ========== > >> > FAIL: runTest > >> > (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest) > >> > Validate Insert, Update operations in View data with given test data > >> > ------------------------------------------------------------ > ---------- > >> > Traceback (most recent call last): > >> > File > >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > view_data_dml_queries.py", > >> > line 105, in runTest > >> > self._verify_row_data(True) > >> > File > >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > view_data_dml_queries.py", > >> > line 282, in _verify_row_data > >> > self.assertEquals(cells[idx], config_data[str(idx)][1]) > >> > AssertionError: u'[default]' != u'1' > >> > - [default] > >> > + 1 > >> > > > > > I guess Khushboo has fixed this recently. > > > > > >> > >> > Screenshots attached. > >> > > >> > On Fri, Jun 16, 2017 at 7:54 AM, Harshal Dhumal > >> > <[email protected]> wrote: > >> >> Hi Dave, > >> >> > >> >> Please find attached patch where I have added timeout of 2 seconds > >> >> before > >> >> selecting object menu in connect to server function. > >> >> > >> >> Note: Apply this patch on top of previous patch for on demand loading > >> >> feature. > >> >> > >> >> > >> >> -- > >> >> Harshal Dhumal > >> >> Sr. Software Engineer > >> >> > >> >> EnterpriseDB India: http://www.enterprisedb.com > >> >> The Enterprise PostgreSQL Company > >> >> > >> >> On Fri, Jun 16, 2017 at 2:25 AM, Dave Page <[email protected]> > wrote: > >> >>> > >> >>> Sounds good, thanks. > >> >>> > >> >>> On Thu, Jun 15, 2017 at 9:54 PM, Harshal Dhumal > >> >>> <[email protected]> wrote: > >> >>> > Hi > >> >>> > > >> >>> > On Fri, Jun 16, 2017 at 2:07 AM, Dave Page <[email protected]> > >> >>> > wrote: > >> >>> >> > >> >>> >> Hi > >> >>> >> > >> >>> >> On Thu, Jun 15, 2017 at 9:30 PM, Harshal Dhumal > >> >>> >> <[email protected]> wrote: > >> >>> >> > Hi Dave, > >> >>> >> > > >> >>> >> > Please find attached updated patch. > >> >>> >> > > >> >>> >> > On Thu, Jun 15, 2017 at 3:58 PM, Dave Page <[email protected]> > >> >>> >> > wrote: > >> >>> >> >> > >> >>> >> >> Hi > >> >>> >> >> > >> >>> >> >> On Wed, Jun 14, 2017 at 11:36 PM, Harshal Dhumal > >> >>> >> >> <[email protected]> wrote: > >> >>> >> >>> > >> >>> >> >>> Hi Dave, > >> >>> >> >>> > >> >>> >> >>> Please find rebased patch for RM2137. > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> Looking very good. The only issues I see are: > >> >>> >> >> > >> >>> >> >> - The row headers should auto-size such that they can display > >> >>> >> >> the > >> >>> >> >> row > >> >>> >> >> numbers if the last row was displayed. E.g. if there are 12345 > >> >>> >> >> rows > >> >>> >> >> in > >> >>> >> >> total, then the row header should be sized to display 5 > digits. > >> >>> >> >> > >> >>> >> > Fixed. > >> >>> >> > > >> >>> >> > > >> >>> >> >> > >> >>> >> >> - The tests are comprehensive, which is awesome. However, > every > >> >>> >> >> time > >> >>> >> >> I > >> >>> >> >> ran > >> >>> >> >> them, at least one of the feature tests failed. Unfortunately, > >> >>> >> >> it > >> >>> >> >> was a > >> >>> >> >> different one each time. In the last two runs, I got: > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> ============================================================ > ========== > >> >>> >> >> ERROR: runTest > >> >>> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) > >> >>> >> >> Query tool feature test > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> ------------------------------------------------------------ > ---------- > >> >>> >> >> Traceback (most recent call last): > >> >>> >> >> File > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> "/Users/dpage/git/pgadmin4/web/regression/feature_utils/ > base_feature_test.py", > >> >>> >> >> line 40, in setUp > >> >>> >> >> self.before() > >> >>> >> >> File > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > query_tool_tests.py", > >> >>> >> >> line 40, in before > >> >>> >> >> self._connects_to_server() > >> >>> >> >> File > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > query_tool_tests.py", > >> >>> >> >> line 144, in _connects_to_server > >> >>> >> >> self.page.driver.find_element_by_link_text("Create"))\ > >> >>> >> >> File > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> >>> >> >> line 319, in find_element_by_link_text > >> >>> >> >> return self.find_element(by=By.LINK_TEXT, > value=link_text) > >> >>> >> >> File > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> >>> >> >> line 756, in find_element > >> >>> >> >> 'value': value})['value'] > >> >>> >> >> File > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> >>> >> >> line 238, in execute > >> >>> >> >> self.error_handler.check_response(response) > >> >>> >> >> File > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/errorhandler.py", > >> >>> >> >> line 193, in check_response > >> >>> >> >> raise exception_class(message, screen, stacktrace) > >> >>> >> >> NoSuchElementException: Message: no such element: Unable to > >> >>> >> >> locate > >> >>> >> >> element: {"method":"link text","selector":"Create"} > >> >>> >> >> (Session info: chrome=58.0.3029.110) > >> >>> >> >> (Driver info: chromedriver=2.29.461585 > >> >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X > >> >>> >> >> 10.12.3 > >> >>> >> >> x86_64) > >> >>> >> >> > >> >>> >> > I checked code from _connects_to_server function which is > common > >> >>> >> > in > >> >>> >> > all > >> >>> >> > features test cases. I didn't find anything wrong with this. If > >> >>> >> > there > >> >>> >> > is > >> >>> >> > a > >> >>> >> > bug in this function then all feature test must fail. > >> >>> >> > Let me know if you are getting failure consistently in > >> >>> >> > _connects_to_server > >> >>> >> > function. > >> >>> >> > >> >>> >> I wondered if that one is a race condition. Do we need a short > >> >>> >> delay > >> >>> >> before clicking the Object menu? I have seen this occasionally > >> >>> >> before. > >> >>> > > >> >>> > > >> >>> > OK. In that case let's try putting 1-2 second delay and observer > >> >>> > behaviour. > >> >>> > I'll send separate patch for this tomorrow as this is not related > to > >> >>> > on > >> >>> > demand query result feature or its test cases. > >> >>> > > >> >>> >> > >> >>> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> ============================================================ > ========== > >> >>> >> >> ERROR: runTest > >> >>> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) > >> >>> >> >> Query tool feature test > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> ------------------------------------------------------------ > ---------- > >> >>> >> >> Traceback (most recent call last): > >> >>> >> >> File > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > query_tool_tests.py", > >> >>> >> >> line 119, in runTest > >> >>> >> >> self._query_tool_auto_rollback_enabled() > >> >>> >> >> File > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/ > query_tool_tests.py", > >> >>> >> >> line 697, in _query_tool_auto_rollback_enabled > >> >>> >> >> '//div[contains(@class, "sql-editor-message") and > >> >>> >> >> contains(string(), > >> >>> >> >> "COMMIT")]' > >> >>> >> >> File > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> >>> >> >> line 295, in find_element_by_xpath > >> >>> >> >> return self.find_element(by=By.XPATH, value=xpath) > >> >>> >> >> File > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> >>> >> >> line 756, in find_element > >> >>> >> >> 'value': value})['value'] > >> >>> >> >> File > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/webdriver.py", > >> >>> >> >> line 238, in execute > >> >>> >> >> self.error_handler.check_response(response) > >> >>> >> >> File > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site- > packages/selenium/webdriver/remote/errorhandler.py", > >> >>> >> >> line 193, in check_response > >> >>> >> >> raise exception_class(message, screen, stacktrace) > >> >>> >> >> NoSuchElementException: Message: no such element: Unable to > >> >>> >> >> locate > >> >>> >> >> element: {"method":"xpath","selector":"//div[contains(@class, > >> >>> >> >> "sql-editor-message") and contains(string(), "COMMIT")]"} > >> >>> >> >> (Session info: chrome=58.0.3029.110) > >> >>> >> >> (Driver info: chromedriver=2.29.461585 > >> >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X > >> >>> >> >> 10.12.3 > >> >>> >> >> x86_64) > >> >>> >> >> > >> >>> >> > I have updated Auto rollback enabled test in this patch. > >> >>> >> > > >> >>> >> >> > >> >>> >> >> Relevant screenshots attached. > >> >>> >> >> > >> >>> >> >> - Can you tidy up the regression output a little please? > Instead > >> >>> >> >> of: > >> >>> >> >> > >> >>> >> >> ------- > >> >>> >> >> runTest > >> >>> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) > >> >>> >> >> Query tool feature test ... On demand result set on > scrolling... > >> >>> >> >> OK. > >> >>> >> >> > >> >>> >> >> On demand result set on grid select all... > >> >>> >> >> OK. > >> >>> >> >> > >> >>> >> >> On demand result set on column select all... > >> >>> >> >> OK. > >> >>> >> >> > >> >>> >> >> Explain query... > >> >>> >> >> OK. > >> >>> >> >> > >> >>> >> >> Explain query with verbose... > >> >>> >> >> OK. > >> >>> >> >> > >> >>> >> >> Explain query with costs... > >> >>> >> >> OK. > >> >>> >> >> > >> >>> >> >> Explain analyze query... > >> >>> >> >> OK. > >> >>> >> >> > >> >>> >> >> Explain analyze query with buffers... > >> >>> >> >> OK. > >> >>> >> >> > >> >>> >> >> Explain analyze query with timing... > >> >>> >> >> OK. > >> >>> >> >> > >> >>> >> >> Auto commit disabled... > >> >>> >> >> OK. > >> >>> >> >> > >> >>> >> >> Auto commit enabled... > >> >>> >> >> OK. > >> >>> >> >> > >> >>> >> >> Auto rollback enabled... > >> >>> >> >> ERROR > >> >>> >> >> ------- > >> >>> >> >> > >> >>> >> >> Something like: > >> >>> >> >> > >> >>> >> >> ------- > >> >>> >> >> runTest > >> >>> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) > >> >>> >> >> Query tool feature test ... > >> >>> >> >> On demand result set on scrolling... OK. > >> >>> >> >> On demand result set on grid select all... OK. > >> >>> >> >> On demand result set on column select all... OK. > >> >>> >> >> Explain query... OK. > >> >>> >> >> Explain query with verbose... OK. > >> >>> >> >> Explain query with costs... OK. > >> >>> >> >> Explain analyze query... OK. > >> >>> >> >> Explain analyze query with buffers... OK. > >> >>> >> >> Explain analyze query with timing... OK. > >> >>> >> >> Auto commit disabled... OK. > >> >>> >> >> Auto commit enabled... OK. > >> >>> >> >> Auto rollback enabled... ERROR > >> >>> >> >> -------- > >> >>> >> >> > >> >>> >> > Fixed. > >> >>> >> > > >> >>> >> > > >> >>> >> >> > >> >>> >> >> Thanks! > >> >>> >> >> > >> >>> >> >> -- > >> >>> >> >> Dave Page > >> >>> >> >> Blog: http://pgsnake.blogspot.com > >> >>> >> >> Twitter: @pgsnake > >> >>> >> >> > >> >>> >> >> EnterpriseDB UK: http://www.enterprisedb.com > >> >>> >> >> The Enterprise PostgreSQL Company > >> >>> >> > > >> >>> >> > > >> >>> >> > >> >>> >> > >> >>> >> > >> >>> >> -- > >> >>> >> Dave Page > >> >>> >> Blog: http://pgsnake.blogspot.com > >> >>> >> Twitter: @pgsnake > >> >>> >> > >> >>> >> EnterpriseDB UK: http://www.enterprisedb.com > >> >>> >> The Enterprise PostgreSQL Company > >> >>> > > >> >>> > > >> >>> > >> >>> > >> >>> > >> >>> -- > >> >>> Dave Page > >> >>> Blog: http://pgsnake.blogspot.com > >> >>> Twitter: @pgsnake > >> >>> > >> >>> EnterpriseDB UK: http://www.enterprisedb.com > >> >>> The Enterprise PostgreSQL Company > >> >> > >> >> > >> > > >> > > >> > > >> > -- > >> > Dave Page > >> > Blog: http://pgsnake.blogspot.com > >> > Twitter: @pgsnake > >> > > >> > EnterpriseDB UK: http://www.enterprisedb.com > >> > The Enterprise PostgreSQL Company > >> > >> > >> > >> -- > >> Dave Page > >> Blog: http://pgsnake.blogspot.com > >> Twitter: @pgsnake > >> > >> EnterpriseDB UK: http://www.enterprisedb.com > >> The Enterprise PostgreSQL Company > > > > > > > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: [pgadmin-hackers] Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] @ 2017-06-27 07:26 Harshal Dhumal <[email protected]> parent: Harshal Dhumal <[email protected]> 0 siblings, 1 reply; 24+ messages in thread From: Harshal Dhumal @ 2017-06-27 07:26 UTC (permalink / raw) To: Dave Page <[email protected]>; +Cc: Harshal Dhumal <[email protected]>; Ashesh Vashi <[email protected]>; pgadmin-hackers Hi, Please find rebased patch. -- *Harshal Dhumal* *Sr. Software Engineer* EnterpriseDB India: http://www.enterprisedb.com The Enterprise PostgreSQL Company On Mon, Jun 26, 2017 at 5:24 PM, Harshal Dhumal < [email protected]> wrote: > yes i'm working on that only :) > > > -- > *Harshal Dhumal* > *Sr. Software Engineer* > > EnterpriseDB India: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > > On Mon, Jun 26, 2017 at 5:22 PM, Dave Page <[email protected]> wrote: > >> I'm sorry, it needs rebasing again. If you can do it quickly, I'll >> make sure it's the next patch I work on in that area. >> >> Thanks. >> >> On Mon, Jun 26, 2017 at 5:16 AM, Harshal Dhumal >> <[email protected]> wrote: >> > Hi Dave, >> > >> > Please find updated rebased patch for RM2137 >> > >> > On Fri, Jun 23, 2017 at 9:00 PM, Dave Page <[email protected]> wrote: >> >> >> >> Hi Harshal, >> >> >> >> When can we expect an updated version of this patch? I think it's >> >> important to get this into the next release. >> >> >> >> Thanks! >> >> >> >> On Fri, Jun 16, 2017 at 10:55 AM, Dave Page <[email protected]> wrote: >> >> > Hi, >> >> > >> >> > That's better - the failures are far less random now :-). I got the >> >> > following two though, on both PG and EPAS 9.5: >> >> > >> >> > ============================================================ >> ========== >> >> > ERROR: runTest >> >> > (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >> >> > Query tool feature test >> >> > ------------------------------------------------------------ >> ---------- >> >> > Traceback (most recent call last): >> >> > File >> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_ >> tool_tests.py", >> >> > line 95, in runTest >> >> > self._query_tool_explain_analyze_buffers() >> >> > File >> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_ >> tool_tests.py", >> >> > line 443, in _query_tool_explain_analyze_buffers >> >> > canvas.find_element_by_xpath("//*[contains(string(), 'Shared >> Read >> >> > Blocks')]") >> >> > File >> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >> ges/selenium/webdriver/remote/webelement.py", >> >> > line 260, in find_element_by_xpath >> >> > return self.find_element(by=By.XPATH, value=xpath) >> >> > File >> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >> ges/selenium/webdriver/remote/webelement.py", >> >> > line 508, in find_element >> >> > {"using": by, "value": value})['value'] >> >> > File >> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >> ges/selenium/webdriver/remote/webelement.py", >> >> > line 491, in _execute >> >> > return self._parent.execute(command, params) >> >> > File >> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >> ges/selenium/webdriver/remote/webdriver.py", >> >> > line 238, in execute >> >> > self.error_handler.check_response(response) >> >> > File >> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >> ges/selenium/webdriver/remote/errorhandler.py", >> >> > line 193, in check_response >> >> > raise exception_class(message, screen, stacktrace) >> >> > NoSuchElementException: Message: no such element: Unable to locate >> >> > element: {"method":"xpath","selector":"//*[contains(string(), >> 'Shared >> >> > Read Blocks')]"} >> >> > (Session info: chrome=58.0.3029.110) >> >> > (Driver info: chromedriver=2.29.461585 >> >> > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 >> >> > x86_64) >> >> > >> >> > >> >> > ============================================================ >> ========== >> >> > ERROR: runTest >> >> > (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest) >> >> > Validate Insert, Update operations in View data with given test data >> >> > ------------------------------------------------------------ >> ---------- >> >> > Traceback (most recent call last): >> >> > File >> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_ >> data_dml_queries.py", >> >> > line 104, in runTest >> >> > self._add_row() >> >> > File >> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_ >> data_dml_queries.py", >> >> > line 255, in _add_row >> >> > self._update_cell(cell_xpath, config_data[str(idx)]) >> >> > File >> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_ >> data_dml_queries.py", >> >> > line 164, in _update_cell >> >> > cell_el = self.page.find_by_xpath(xpath) >> >> > File >> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >> min_page.py", >> >> > line 122, in find_by_xpath >> >> > return self.wait_for_element(lambda driver: >> >> > driver.find_element_by_xpath(xpath)) >> >> > File >> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >> min_page.py", >> >> > line 205, in wait_for_element >> >> > return self._wait_for("element to exist", element_if_it_exists) >> >> > File >> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >> min_page.py", >> >> > line 255, in _wait_for >> >> > "Timed out waiting for " + waiting_for_message) >> >> > File >> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >> ges/selenium/webdriver/support/wait.py", >> >> > line 71, in until >> >> > value = method(self._driver) >> >> > File >> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >> min_page.py", >> >> > line 200, in element_if_it_exists >> >> > if element.is_displayed() and element.is_enabled(): >> >> > File >> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >> ges/selenium/webdriver/remote/webelement.py", >> >> > line 358, in is_displayed >> >> > return self._execute(Command.IS_ELEMENT_DISPLAYED)['value'] >> >> > File >> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >> ges/selenium/webdriver/remote/webelement.py", >> >> > line 491, in _execute >> >> > return self._parent.execute(command, params) >> >> > File >> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >> ges/selenium/webdriver/remote/webdriver.py", >> >> > line 238, in execute >> >> > self.error_handler.check_response(response) >> >> > File >> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >> ges/selenium/webdriver/remote/errorhandler.py", >> >> > line 193, in check_response >> >> > raise exception_class(message, screen, stacktrace) >> >> > StaleElementReferenceException: Message: stale element reference: >> >> > element is not attached to the page document >> >> > (Session info: chrome=58.0.3029.110) >> >> > (Driver info: chromedriver=2.29.461585 >> >> > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X 10.12.3 >> >> > x86_64) >> >> > >> > >> > >> > I checked my feature test cases for any database server version/type >> (PG, >> > EPAS) specific failures and I found that test cases are working fine on >> all >> > of them. However I have slightly modified test case to overcome above >> > mentioned failures. >> > >> >> >> >> > >> >> > On 9.4 everything passes. On 9.6, I get one failure: >> >> > >> >> > ============================================================ >> ========== >> >> > FAIL: runTest >> >> > (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest) >> >> > Validate Insert, Update operations in View data with given test data >> >> > ------------------------------------------------------------ >> ---------- >> >> > Traceback (most recent call last): >> >> > File >> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_ >> data_dml_queries.py", >> >> > line 105, in runTest >> >> > self._verify_row_data(True) >> >> > File >> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_ >> data_dml_queries.py", >> >> > line 282, in _verify_row_data >> >> > self.assertEquals(cells[idx], config_data[str(idx)][1]) >> >> > AssertionError: u'[default]' != u'1' >> >> > - [default] >> >> > + 1 >> >> > >> > >> > I guess Khushboo has fixed this recently. >> > >> > >> >> >> >> > Screenshots attached. >> >> > >> >> > On Fri, Jun 16, 2017 at 7:54 AM, Harshal Dhumal >> >> > <[email protected]> wrote: >> >> >> Hi Dave, >> >> >> >> >> >> Please find attached patch where I have added timeout of 2 seconds >> >> >> before >> >> >> selecting object menu in connect to server function. >> >> >> >> >> >> Note: Apply this patch on top of previous patch for on demand >> loading >> >> >> feature. >> >> >> >> >> >> >> >> >> -- >> >> >> Harshal Dhumal >> >> >> Sr. Software Engineer >> >> >> >> >> >> EnterpriseDB India: http://www.enterprisedb.com >> >> >> The Enterprise PostgreSQL Company >> >> >> >> >> >> On Fri, Jun 16, 2017 at 2:25 AM, Dave Page <[email protected]> >> wrote: >> >> >>> >> >> >>> Sounds good, thanks. >> >> >>> >> >> >>> On Thu, Jun 15, 2017 at 9:54 PM, Harshal Dhumal >> >> >>> <[email protected]> wrote: >> >> >>> > Hi >> >> >>> > >> >> >>> > On Fri, Jun 16, 2017 at 2:07 AM, Dave Page <[email protected]> >> >> >>> > wrote: >> >> >>> >> >> >> >>> >> Hi >> >> >>> >> >> >> >>> >> On Thu, Jun 15, 2017 at 9:30 PM, Harshal Dhumal >> >> >>> >> <[email protected]> wrote: >> >> >>> >> > Hi Dave, >> >> >>> >> > >> >> >>> >> > Please find attached updated patch. >> >> >>> >> > >> >> >>> >> > On Thu, Jun 15, 2017 at 3:58 PM, Dave Page <[email protected] >> > >> >> >>> >> > wrote: >> >> >>> >> >> >> >> >>> >> >> Hi >> >> >>> >> >> >> >> >>> >> >> On Wed, Jun 14, 2017 at 11:36 PM, Harshal Dhumal >> >> >>> >> >> <[email protected]> wrote: >> >> >>> >> >>> >> >> >>> >> >>> Hi Dave, >> >> >>> >> >>> >> >> >>> >> >>> Please find rebased patch for RM2137. >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> Looking very good. The only issues I see are: >> >> >>> >> >> >> >> >>> >> >> - The row headers should auto-size such that they can display >> >> >>> >> >> the >> >> >>> >> >> row >> >> >>> >> >> numbers if the last row was displayed. E.g. if there are >> 12345 >> >> >>> >> >> rows >> >> >>> >> >> in >> >> >>> >> >> total, then the row header should be sized to display 5 >> digits. >> >> >>> >> >> >> >> >>> >> > Fixed. >> >> >>> >> > >> >> >>> >> > >> >> >>> >> >> >> >> >>> >> >> - The tests are comprehensive, which is awesome. However, >> every >> >> >>> >> >> time >> >> >>> >> >> I >> >> >>> >> >> ran >> >> >>> >> >> them, at least one of the feature tests failed. >> Unfortunately, >> >> >>> >> >> it >> >> >>> >> >> was a >> >> >>> >> >> different one each time. In the last two runs, I got: >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> ============================================================ >> ========== >> >> >>> >> >> ERROR: runTest >> >> >>> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest >> ) >> >> >>> >> >> Query tool feature test >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> ------------------------------------------------------------ >> ---------- >> >> >>> >> >> Traceback (most recent call last): >> >> >>> >> >> File >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> "/Users/dpage/git/pgadmin4/web/regression/feature_utils/base >> _feature_test.py", >> >> >>> >> >> line 40, in setUp >> >> >>> >> >> self.before() >> >> >>> >> >> File >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_ >> tool_tests.py", >> >> >>> >> >> line 40, in before >> >> >>> >> >> self._connects_to_server() >> >> >>> >> >> File >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_ >> tool_tests.py", >> >> >>> >> >> line 144, in _connects_to_server >> >> >>> >> >> self.page.driver.find_element_by_link_text("Create"))\ >> >> >>> >> >> File >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >> ges/selenium/webdriver/remote/webdriver.py", >> >> >>> >> >> line 319, in find_element_by_link_text >> >> >>> >> >> return self.find_element(by=By.LINK_TEXT, >> value=link_text) >> >> >>> >> >> File >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >> ges/selenium/webdriver/remote/webdriver.py", >> >> >>> >> >> line 756, in find_element >> >> >>> >> >> 'value': value})['value'] >> >> >>> >> >> File >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >> ges/selenium/webdriver/remote/webdriver.py", >> >> >>> >> >> line 238, in execute >> >> >>> >> >> self.error_handler.check_response(response) >> >> >>> >> >> File >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >> ges/selenium/webdriver/remote/errorhandler.py", >> >> >>> >> >> line 193, in check_response >> >> >>> >> >> raise exception_class(message, screen, stacktrace) >> >> >>> >> >> NoSuchElementException: Message: no such element: Unable to >> >> >>> >> >> locate >> >> >>> >> >> element: {"method":"link text","selector":"Create"} >> >> >>> >> >> (Session info: chrome=58.0.3029.110) >> >> >>> >> >> (Driver info: chromedriver=2.29.461585 >> >> >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X >> >> >>> >> >> 10.12.3 >> >> >>> >> >> x86_64) >> >> >>> >> >> >> >> >>> >> > I checked code from _connects_to_server function which is >> common >> >> >>> >> > in >> >> >>> >> > all >> >> >>> >> > features test cases. I didn't find anything wrong with this. >> If >> >> >>> >> > there >> >> >>> >> > is >> >> >>> >> > a >> >> >>> >> > bug in this function then all feature test must fail. >> >> >>> >> > Let me know if you are getting failure consistently in >> >> >>> >> > _connects_to_server >> >> >>> >> > function. >> >> >>> >> >> >> >>> >> I wondered if that one is a race condition. Do we need a short >> >> >>> >> delay >> >> >>> >> before clicking the Object menu? I have seen this occasionally >> >> >>> >> before. >> >> >>> > >> >> >>> > >> >> >>> > OK. In that case let's try putting 1-2 second delay and observer >> >> >>> > behaviour. >> >> >>> > I'll send separate patch for this tomorrow as this is not >> related to >> >> >>> > on >> >> >>> > demand query result feature or its test cases. >> >> >>> > >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> ============================================================ >> ========== >> >> >>> >> >> ERROR: runTest >> >> >>> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest >> ) >> >> >>> >> >> Query tool feature test >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> ------------------------------------------------------------ >> ---------- >> >> >>> >> >> Traceback (most recent call last): >> >> >>> >> >> File >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_ >> tool_tests.py", >> >> >>> >> >> line 119, in runTest >> >> >>> >> >> self._query_tool_auto_rollback_enabled() >> >> >>> >> >> File >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_ >> tool_tests.py", >> >> >>> >> >> line 697, in _query_tool_auto_rollback_enabled >> >> >>> >> >> '//div[contains(@class, "sql-editor-message") and >> >> >>> >> >> contains(string(), >> >> >>> >> >> "COMMIT")]' >> >> >>> >> >> File >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >> ges/selenium/webdriver/remote/webdriver.py", >> >> >>> >> >> line 295, in find_element_by_xpath >> >> >>> >> >> return self.find_element(by=By.XPATH, value=xpath) >> >> >>> >> >> File >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >> ges/selenium/webdriver/remote/webdriver.py", >> >> >>> >> >> line 756, in find_element >> >> >>> >> >> 'value': value})['value'] >> >> >>> >> >> File >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >> ges/selenium/webdriver/remote/webdriver.py", >> >> >>> >> >> line 238, in execute >> >> >>> >> >> self.error_handler.check_response(response) >> >> >>> >> >> File >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >> ges/selenium/webdriver/remote/errorhandler.py", >> >> >>> >> >> line 193, in check_response >> >> >>> >> >> raise exception_class(message, screen, stacktrace) >> >> >>> >> >> NoSuchElementException: Message: no such element: Unable to >> >> >>> >> >> locate >> >> >>> >> >> element: {"method":"xpath","selector":" >> //div[contains(@class, >> >> >>> >> >> "sql-editor-message") and contains(string(), "COMMIT")]"} >> >> >>> >> >> (Session info: chrome=58.0.3029.110) >> >> >>> >> >> (Driver info: chromedriver=2.29.461585 >> >> >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X >> >> >>> >> >> 10.12.3 >> >> >>> >> >> x86_64) >> >> >>> >> >> >> >> >>> >> > I have updated Auto rollback enabled test in this patch. >> >> >>> >> > >> >> >>> >> >> >> >> >>> >> >> Relevant screenshots attached. >> >> >>> >> >> >> >> >>> >> >> - Can you tidy up the regression output a little please? >> Instead >> >> >>> >> >> of: >> >> >>> >> >> >> >> >>> >> >> ------- >> >> >>> >> >> runTest >> >> >>> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest >> ) >> >> >>> >> >> Query tool feature test ... On demand result set on >> scrolling... >> >> >>> >> >> OK. >> >> >>> >> >> >> >> >>> >> >> On demand result set on grid select all... >> >> >>> >> >> OK. >> >> >>> >> >> >> >> >>> >> >> On demand result set on column select all... >> >> >>> >> >> OK. >> >> >>> >> >> >> >> >>> >> >> Explain query... >> >> >>> >> >> OK. >> >> >>> >> >> >> >> >>> >> >> Explain query with verbose... >> >> >>> >> >> OK. >> >> >>> >> >> >> >> >>> >> >> Explain query with costs... >> >> >>> >> >> OK. >> >> >>> >> >> >> >> >>> >> >> Explain analyze query... >> >> >>> >> >> OK. >> >> >>> >> >> >> >> >>> >> >> Explain analyze query with buffers... >> >> >>> >> >> OK. >> >> >>> >> >> >> >> >>> >> >> Explain analyze query with timing... >> >> >>> >> >> OK. >> >> >>> >> >> >> >> >>> >> >> Auto commit disabled... >> >> >>> >> >> OK. >> >> >>> >> >> >> >> >>> >> >> Auto commit enabled... >> >> >>> >> >> OK. >> >> >>> >> >> >> >> >>> >> >> Auto rollback enabled... >> >> >>> >> >> ERROR >> >> >>> >> >> ------- >> >> >>> >> >> >> >> >>> >> >> Something like: >> >> >>> >> >> >> >> >>> >> >> ------- >> >> >>> >> >> runTest >> >> >>> >> >> (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest >> ) >> >> >>> >> >> Query tool feature test ... >> >> >>> >> >> On demand result set on scrolling... OK. >> >> >>> >> >> On demand result set on grid select all... OK. >> >> >>> >> >> On demand result set on column select all... OK. >> >> >>> >> >> Explain query... OK. >> >> >>> >> >> Explain query with verbose... OK. >> >> >>> >> >> Explain query with costs... OK. >> >> >>> >> >> Explain analyze query... OK. >> >> >>> >> >> Explain analyze query with buffers... OK. >> >> >>> >> >> Explain analyze query with timing... OK. >> >> >>> >> >> Auto commit disabled... OK. >> >> >>> >> >> Auto commit enabled... OK. >> >> >>> >> >> Auto rollback enabled... ERROR >> >> >>> >> >> -------- >> >> >>> >> >> >> >> >>> >> > Fixed. >> >> >>> >> > >> >> >>> >> > >> >> >>> >> >> >> >> >>> >> >> Thanks! >> >> >>> >> >> >> >> >>> >> >> -- >> >> >>> >> >> Dave Page >> >> >>> >> >> Blog: http://pgsnake.blogspot.com >> >> >>> >> >> Twitter: @pgsnake >> >> >>> >> >> >> >> >>> >> >> EnterpriseDB UK: http://www.enterprisedb.com >> >> >>> >> >> The Enterprise PostgreSQL Company >> >> >>> >> > >> >> >>> >> > >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> -- >> >> >>> >> Dave Page >> >> >>> >> Blog: http://pgsnake.blogspot.com >> >> >>> >> Twitter: @pgsnake >> >> >>> >> >> >> >>> >> EnterpriseDB UK: http://www.enterprisedb.com >> >> >>> >> The Enterprise PostgreSQL Company >> >> >>> > >> >> >>> > >> >> >>> >> >> >>> >> >> >>> >> >> >>> -- >> >> >>> Dave Page >> >> >>> Blog: http://pgsnake.blogspot.com >> >> >>> Twitter: @pgsnake >> >> >>> >> >> >>> EnterpriseDB UK: http://www.enterprisedb.com >> >> >>> The Enterprise PostgreSQL Company >> >> >> >> >> >> >> >> > >> >> > >> >> > >> >> > -- >> >> > Dave Page >> >> > Blog: http://pgsnake.blogspot.com >> >> > Twitter: @pgsnake >> >> > >> >> > EnterpriseDB UK: http://www.enterprisedb.com >> >> > The Enterprise PostgreSQL Company >> >> >> >> >> >> >> >> -- >> >> Dave Page >> >> Blog: http://pgsnake.blogspot.com >> >> Twitter: @pgsnake >> >> >> >> EnterpriseDB UK: http://www.enterprisedb.com >> >> The Enterprise PostgreSQL Company >> > >> > >> >> >> >> -- >> Dave Page >> Blog: http://pgsnake.blogspot.com >> Twitter: @pgsnake >> >> EnterpriseDB UK: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> > > Attachments: [text/x-patch] RM2137_query_tool_on_demand_result_V9.patch (165.4K, 3-RM2137_query_tool_on_demand_result_V9.patch) download | inline diff: diff --git a/web/config.py b/web/config.py index 6a3128e..e40201d 100644 --- a/web/config.py +++ b/web/config.py @@ -322,6 +322,12 @@ THREADED_MODE = True SQLALCHEMY_TRACK_MODIFICATIONS = False ########################################################################## +# Number of records to fetch in one batch in query tool when query result +# set is large. +########################################################################## +ON_DEMAND_RECORD_COUNT = 1000 + +########################################################################## # Local config settings ########################################################################## diff --git a/web/pgadmin/feature_tests/connect_to_server_feature_test.py b/web/pgadmin/feature_tests/connect_to_server_feature_test.py index e8de161..28a6df6 100644 --- a/web/pgadmin/feature_tests/connect_to_server_feature_test.py +++ b/web/pgadmin/feature_tests/connect_to_server_feature_test.py @@ -7,6 +7,7 @@ # ########################################################################## +import time from selenium.webdriver import ActionChains import config as app_config @@ -53,6 +54,7 @@ class ConnectsToServerFeatureTest(BaseFeatureTest): def _connects_to_server(self): self.page.find_by_xpath("//*[@class='aciTreeText' and .='Servers']").click() + time.sleep(2) self.page.driver.find_element_by_link_text("Object").click() ActionChains(self.page.driver) \ .move_to_element(self.page.driver.find_element_by_link_text("Create")) \ @@ -72,6 +74,8 @@ class ConnectsToServerFeatureTest(BaseFeatureTest): self.page.toggle_open_server(self.server['name']) self.page.toggle_open_tree_item('Databases') self.page.toggle_open_tree_item('acceptance_test_db') + # wait until all database dependant modules/js are loaded. + time.sleep(5) self.page.toggle_open_tree_item('Schemas') self.page.toggle_open_tree_item('public') self.page.toggle_open_tree_item('Tables') diff --git a/web/pgadmin/feature_tests/pg_datatype_validation_test.py b/web/pgadmin/feature_tests/pg_datatype_validation_test.py index 8483876..26c787b 100644 --- a/web/pgadmin/feature_tests/pg_datatype_validation_test.py +++ b/web/pgadmin/feature_tests/pg_datatype_validation_test.py @@ -6,6 +6,7 @@ # This software is released under the PostgreSQL Licence # ########################################################################## +import time from selenium.webdriver import ActionChains from selenium.common.exceptions import TimeoutException from selenium.webdriver.support.ui import WebDriverWait @@ -56,6 +57,7 @@ class PGDataypeFeatureTest(BaseFeatureTest): self.page.find_by_xpath( "//*[@class='aciTreeText' and .='Servers']" ).click() + time.sleep(2) self.page.driver.find_element_by_link_text("Object").click() ActionChains(self.page.driver) \ .move_to_element( @@ -106,45 +108,19 @@ class PGDataypeFeatureTest(BaseFeatureTest): self.page.fill_codemirror_area_with(query) self.page.find_by_id("btn-flash").click() wait = WebDriverWait(self.page.driver, 5) - wait.until(EC.presence_of_element_located( - (By.XPATH, "//*[@id='0']//*[@id='datagrid']/div[5]/div/div[1]/" - "div[2]/span"))) + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) # For every sample data-type value, check the expected output. cnt = 2 - for val in expected_output[:10]: - try: - source_code = self.page.find_by_xpath( - "//*[@id='0']//*[@id='datagrid']/div[5]/div/div[1]/div[" - + str(cnt) - + "]/span" - ).get_attribute('innerHTML') - - PGDataypeFeatureTest.check_result( - source_code, - expected_output[cnt - 2] - ) - cnt += 1 - except TimeoutException: - assert False, "{0} does not match with {1}".format( - val, expected_output[cnt] - ) - - cnt = 12 - for val in expected_output[10:]: + cells = canvas.find_elements_by_css_selector('.slick-cell') + # remove first element as it is row number. + cells.pop(0) + for val, cell in zip(expected_output, cells): try: - if cnt == 14: - xpath = "//*[@id='0']//*[@id='datagrid']/div[5]/div/div[1]/div[" \ - + str(cnt) \ - + "]/span" - else: - xpath = "//*[@id='0']//*[@id='datagrid']/div[5]/div/div/div[" \ - + str(cnt) \ - + "]" - - source_code = self.page.find_by_xpath( - xpath - ).get_attribute('innerHTML') + source_code = cell.get_attribute('innerHTML') PGDataypeFeatureTest.check_result( source_code, diff --git a/web/pgadmin/feature_tests/query_tool_tests.py b/web/pgadmin/feature_tests/query_tool_tests.py new file mode 100644 index 0000000..c058287 --- /dev/null +++ b/web/pgadmin/feature_tests/query_tool_tests.py @@ -0,0 +1,734 @@ +########################################################################## +# +# pgAdmin 4 - PostgreSQL Tools +# +# Copyright (C) 2013 - 2017, The pgAdmin Development Team +# This software is released under the PostgreSQL Licence +# +########################################################################## + +from __future__ import print_function +import time +import sys +import config +from selenium.webdriver import ActionChains +from selenium.webdriver.support.ui import WebDriverWait +from selenium.webdriver.support import expected_conditions as EC +from selenium.webdriver.common.by import By +from regression.python_test_utils import test_utils +from regression.feature_utils.base_feature_test import BaseFeatureTest + + +class QueryToolFeatureTest(BaseFeatureTest): + """ + This feature test will test the different query tool features. + """ + + scenarios = [ + ("Query tool feature test", dict()) + ] + + def before(self): + connection = test_utils.get_db_connection(self.server['db'], + self.server['username'], + self.server['db_password'], + self.server['host'], + self.server['port']) + test_utils.drop_database(connection, "acceptance_test_db") + test_utils.create_database(self.server, "acceptance_test_db") + self.page.wait_for_spinner_to_disappear() + self._connects_to_server() + self._locate_database_tree_node() + self.page.open_query_tool() + + def runTest(self): + # on demand result set on scrolling. + print("\nOn demand result set on scrolling... ", + file=sys.stderr, end="") + self._on_demand_result() + print("OK.", + file=sys.stderr) + self._clear_query_tool() + + # on demand result set on grid select all. + print("On demand result set on grid select all... ", + file=sys.stderr, end="") + self._on_demand_result_select_all_grid() + print("OK.", + file=sys.stderr) + self._clear_query_tool() + + # on demand result set on column select all. + print("On demand result set on column select all... ", + file=sys.stderr, end="") + self._on_demand_result_select_all_column() + print("OK.", + file=sys.stderr) + self._clear_query_tool() + + # explain query + print("Explain query... ", file=sys.stderr, end="") + self._query_tool_explain() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + # explain query with verbose + print("Explain query with verbose... ", file=sys.stderr, end="") + self._query_tool_explain_verbose() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + # explain query with costs + print("Explain query with costs... ", file=sys.stderr, end="") + self._query_tool_explain_cost() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + # explain analyze query + print("Explain analyze query... ", file=sys.stderr, end="") + self._query_tool_explain_analyze() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + # explain analyze query with buffers + print("Explain analyze query with buffers... ", file=sys.stderr, end="") + self._query_tool_explain_analyze_buffers() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + # explain analyze query with timing + print("Explain analyze query with timing... ", file=sys.stderr, end="") + self._query_tool_explain_analyze_timing() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + # auto commit disabled. + print("Auto commit disabled... ", file=sys.stderr, end="") + self._query_tool_auto_commit_disabled() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + # auto commit enabled. + print("Auto commit enabled... ", file=sys.stderr, end="") + self._query_tool_auto_commit_enabled() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + # auto rollback enabled. + print("Auto rollback enabled...", file=sys.stderr, end="") + self._query_tool_auto_rollback_enabled() + print(" OK.", file=sys.stderr) + self._clear_query_tool() + + # cancel query. + print("Cancel query... ", file=sys.stderr, end="") + self._query_tool_cancel_query() + print("OK.", file=sys.stderr) + self._clear_query_tool() + + def after(self): + self.page.remove_server(self.server) + connection = test_utils.get_db_connection(self.server['db'], + self.server['username'], + self.server['db_password'], + self.server['host'], + self.server['port']) + test_utils.drop_database(connection, "acceptance_test_db") + + def _connects_to_server(self): + self.page.find_by_xpath( + "//*[@class='aciTreeText' and .='Servers']").click() + time.sleep(2) + self.page.driver.find_element_by_link_text("Object").click() + ActionChains(self.page.driver) \ + .move_to_element( + self.page.driver.find_element_by_link_text("Create"))\ + .perform() + self.page.find_by_partial_link_text("Server...").click() + + server_config = self.server + self.page.fill_input_by_field_name("name", server_config['name']) + self.page.find_by_partial_link_text("Connection").click() + self.page.fill_input_by_field_name("host", server_config['host']) + self.page.fill_input_by_field_name("port", server_config['port']) + self.page.fill_input_by_field_name( + "username", + server_config['username'] + ) + self.page.fill_input_by_field_name( + "password", + server_config['db_password'] + ) + self.page.find_by_xpath("//button[contains(.,'Save')]").click() + + def _locate_database_tree_node(self): + self.page.toggle_open_tree_item(self.server['name']) + self.page.toggle_open_tree_item('Databases') + self.page.toggle_open_tree_item('acceptance_test_db') + + def _clear_query_tool(self): + # clear codemirror. + self.page.find_by_id("btn-edit").click() + # wait for alertify dialog open animation to complete. + time.sleep(1) + + self.page.click_element(self.page.find_by_xpath("//button[contains(.,'Yes')]")) + # wait for alertify dialog close animation to complete. + time.sleep(1) + + def _on_demand_result(self): + ON_DEMAND_CHUNKS = 2 + query = """-- On demand query result on scroll +SELECT generate_series(1, {}) as id""".format( + config.ON_DEMAND_RECORD_COUNT * ON_DEMAND_CHUNKS) + + wait = WebDriverWait(self.page.driver, 10) + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-flash").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas"))) + + # scroll to bottom to fetch next chunk of result set. + self.driver.execute_script( + "$('.slick-viewport').scrollTop($('.grid-canvas').height());" + ) + + # wait for ajax to complete. + time.sleep(1) + + # again scroll to bottom to bring last row of next chunk in + # viewport. + self.driver.execute_script( + "$('.slick-viewport').scrollTop($('.grid-canvas').height());" + ) + + row_id_to_find = config.ON_DEMAND_RECORD_COUNT * ON_DEMAND_CHUNKS + + canvas.find_element_by_xpath( + '//span[text()="{}"]'.format(row_id_to_find) + ) + + def _on_demand_result_select_all_grid(self): + ON_DEMAND_CHUNKS = 3 + query = """-- On demand query result on grid select all +SELECT generate_series(1, {}) as id""".format( + config.ON_DEMAND_RECORD_COUNT * ON_DEMAND_CHUNKS) + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-flash").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, ".slick-header-column"))).click() + + # wait for until all records are fetched and selected. + time.sleep(1) + # scroll to bottom to bring last row of next chunk in + # viewport. + self.driver.execute_script( + "$('.slick-viewport').scrollTop($('.grid-canvas').height());" + ) + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + + row_id_to_find = config.ON_DEMAND_RECORD_COUNT * ON_DEMAND_CHUNKS + + canvas.find_element_by_xpath( + '//span[text()="{}"]'.format(row_id_to_find) + ) + + def _on_demand_result_select_all_column(self): + ON_DEMAND_CHUNKS = 4 + query = """-- On demand query result on column select all +SELECT generate_series(1, {}) as id1, 'dummy' as id2""".format( + config.ON_DEMAND_RECORD_COUNT * ON_DEMAND_CHUNKS) + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-flash").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + # click on first data column to select all column. + + wait.until(EC.presence_of_element_located( + ( + By.XPATH, + "//span[contains(@class, 'column-name') and contains(., 'id1')]")) + ).click() + + # wait for until all records are fetched and selected. + time.sleep(1) + # scroll to bottom to bring last row of next chunk in + # viewport. + self.driver.execute_script( + "$('.slick-viewport').scrollTop($('.grid-canvas').height());" + ) + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + + row_id_to_find = config.ON_DEMAND_RECORD_COUNT * ON_DEMAND_CHUNKS + + canvas.find_element_by_xpath( + '//span[text()="{}"]'.format(row_id_to_find) + ) + + def _query_tool_explain(self): + query = """-- Explain query +SELECT generate_series(1, 1000) as id order by id desc""" + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-query-dropdown").click() + self.page.find_by_id("btn-explain").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + self.page.click_tab('Data Output') + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + # Search for Plan word in result + canvas.find_element_by_xpath("//*[contains(string(),'Plan')]") + + def _query_tool_explain_verbose(self): + query = """-- Explain query with verbose +SELECT generate_series(1, 1000) as id order by id desc""" + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + query_op = self.page.find_by_id("btn-query-dropdown") + query_op.click() + + ActionChains(self.driver).move_to_element( + query_op.find_element_by_xpath( + "//li[contains(.,'Explain Options')]")).perform() + + self.page.find_by_id("btn-explain-verbose").click() + + self.page.find_by_id("btn-explain").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + self.page.click_tab('Data Output') + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + # Search for 'Output' word in result + canvas.find_element_by_xpath("//*[contains(string(), 'Output')]") + + def _query_tool_explain_cost(self): + query = """-- Explain query with costs +SELECT generate_series(1, 1000) as id order by id desc""" + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + query_op = self.page.find_by_id("btn-query-dropdown") + query_op.click() + + ActionChains(self.driver).move_to_element( + query_op.find_element_by_xpath( + "//li[contains(.,'Explain Options')]")).perform() + + self.page.find_by_id("btn-explain-costs").click() + + self.page.find_by_id("btn-explain").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + self.page.click_tab('Data Output') + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + # Search for 'Total Cost word in result + canvas.find_element_by_xpath("//*[contains(string(),'Total Cost')]") + + def _query_tool_explain_analyze(self): + query = """-- Explain analyze query +SELECT generate_series(1, 1000) as id order by id desc""" + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-query-dropdown").click() + self.page.find_by_id("btn-explain-analyze").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + self.page.click_tab('Data Output') + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + # Search for Actual Rows word in result + canvas.find_element_by_xpath("//*[contains(string(),'Actual Rows')]") + + def _query_tool_explain_analyze_buffers(self): + query = """-- Explain analyze query with buffers +SELECT generate_series(1, 1000) as id order by id desc""" + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + query_op = self.page.find_by_id("btn-query-dropdown") + query_op.click() + + ActionChains(self.driver).move_to_element( + query_op.find_element_by_xpath( + "//li[contains(.,'Explain Options')]")).perform() + + self.page.find_by_id("btn-explain-buffers").click() + + self.page.find_by_id("btn-explain-analyze").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + self.page.click_tab('Data Output') + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + # Search for 'Shared Read Blocks' word in result + canvas.find_element_by_xpath("//*[contains(string(), 'Shared Read Blocks')]") + + def _query_tool_explain_analyze_timing(self): + query = """-- Explain analyze query with timing +SELECT generate_series(1, 1000) as id order by id desc""" + + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + query_op = self.page.find_by_id("btn-query-dropdown") + query_op.click() + + ActionChains(self.driver).move_to_element( + query_op.find_element_by_xpath( + "//li[contains(.,'Explain Options')]")).perform() + + self.page.find_by_id("btn-explain-timing").click() + + self.page.find_by_id("btn-explain-analyze").click() + + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + self.page.click_tab('Data Output') + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas")) + ) + # Search for 'Actual Total Time' word in result + canvas.find_element_by_xpath("//*[contains(string(), 'Actual Total Time')]") + + def _query_tool_auto_commit_disabled(self): + table_name = 'query_tool_auto_commit_disabled_table' + query = """-- 1. Disable auto commit. +-- 2. Create table in public schema. +-- 3. ROLLBACK transaction. +-- 4. Check if table is *NOT* created. +CREATE TABLE public.{}();""".format(table_name) + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-query-dropdown").click() + + auto_commit_btn = self.page.find_by_id("btn-auto-commit") + + auto_commit_check = auto_commit_btn.find_element_by_tag_name("i") + + # if auto commit is enabled then 'i' element will + # have 'auto-commit fa fa-check' classes + # if auto commit is disabled then 'i' element will + # have 'auto-commit fa fa-check visibility-hidden' classes + + if 'auto-commit fa fa-check' == str(auto_commit_check.get_attribute( + 'class')): + auto_commit_btn.click() + + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "CREATE TABLE")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) Disable auto commit. +-- 2. (Done) Create table in public schema. +-- 3. ROLLBACK transaction. +-- 4. Check if table is *NOT* created. +ROLLBACK;""" + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "ROLLBACK")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) Disable auto commit. +-- 2. (Done) Create table in public schema. +-- 3. (Done) ROLLBACK transaction. +-- 4. Check if table is *NOT* created. +SELECT relname FROM pg_class WHERE relkind IN ('r','s','t') and relnamespace = 2200::oid;""" + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Data Output') + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas"))) + + el = canvas.find_elements_by_xpath("//div[contains(@class, 'slick-cell') and contains(text(), '{}')]".format(table_name)) + + assert len(el) == 0, "Table '{}' created with auto commit disabled and without any explicit commit.".format(table_name) + + def _query_tool_auto_commit_enabled(self): + table_name = 'query_tool_auto_commit_enabled_table' + query = """-- 1. END any open transaction. +-- 2. Enable auto commit. +-- 3. Create table in public schema. +-- 4. ROLLBACK transaction +-- 5. Check if table is created event after ROLLBACK. +END; +CREATE TABLE public.{}();""".format(table_name) + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-query-dropdown").click() + + auto_commit_btn = self.page.find_by_id("btn-auto-commit") + + auto_commit_check = auto_commit_btn.find_element_by_tag_name("i") + + # if auto commit is enabled then 'i' element will + # have 'auto-commit fa fa-check' classes + # if auto commit is disabled then 'i' element will + # have 'auto-commit fa fa-check visibility-hidden' classes + + if 'auto-commit fa fa-check visibility-hidden' == str(auto_commit_check.get_attribute( + 'class')): + auto_commit_btn.click() + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "CREATE TABLE")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) END any open transaction if any. +-- 2. (Done) Enable auto commit. +-- 3. (Done) Create table in public schema. +-- 4. ROLLBACK transaction +-- 5. Check if table is created event after ROLLBACK. +ROLLBACK;""" + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "ROLLBACK")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) END any open transaction if any. +-- 2. (Done) Enable auto commit. +-- 3. (Done) Create table in public schema. +-- 4. (Done) ROLLBACK transaction +-- 5. Check if table is created event after ROLLBACK. +SELECT relname FROM pg_class WHERE relkind IN ('r','s','t') and relnamespace = 2200::oid;""" + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.click_tab('Data Output') + self.page.wait_for_query_tool_loading_indicator_to_disappear() + + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas"))) + + el = canvas.find_elements_by_xpath("//div[contains(@class, 'slick-cell') and contains(text(), '{}')]".format(table_name)) + + assert len(el) != 0, "Table '{}' is not created with auto commit enabled.".format(table_name) + + def _query_tool_auto_rollback_enabled(self): + table_name = 'query_tool_auto_rollback_enabled_table' + query = """-- 1. END any open transaction. +-- 2. Enable auto rollback and disable auto commit. +-- 3. Create table in public schema. +-- 4. Generate error in transaction. +-- 5. END transaction. +-- 6. Check if table is *NOT* created after ending transaction. +END;""" + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self._clear_query_tool() + + query = """-- 1. (Done) END any open transaction. +-- 2. Enable auto rollback and disable auto commit. +-- 3. Create table in public schema. +-- 4. Generate error in transaction. +-- 5. END transaction. +-- 6. Check if table is *NOT* created after ending transaction. +CREATE TABLE public.{}();""".format(table_name) + wait = WebDriverWait(self.page.driver, 10) + + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-query-dropdown").click() + + auto_rollback_btn = self.page.find_by_id("btn-auto-rollback") + + auto_rollback_check = auto_rollback_btn.find_element_by_tag_name("i") + + # if auto rollback is enabled then 'i' element will + # have 'auto-rollback fa fa-check' classes + # if auto rollback is disabled then 'i' element will + # have 'auto-rollback fa fa-check visibility-hidden' classes + + if 'auto-rollback fa fa-check visibility-hidden' == str(auto_rollback_check.get_attribute( + 'class')): + auto_rollback_btn.click() + + auto_commit_btn = self.page.find_by_id("btn-auto-commit") + + auto_commit_check = auto_commit_btn.find_element_by_tag_name("i") + + # if auto commit is enabled then 'i' element will + # have 'auto-commit fa fa-check' classes + # if auto commit is disabled then 'i' element will + # have 'auto-commit fa fa-check visibility-hidden' classes + + if 'auto-commit fa fa-check' == str(auto_commit_check.get_attribute( + 'class')): + auto_commit_btn.click() + + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "CREATE TABLE")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) END any open transaction. +-- 2. (Done) Enable auto rollback and disable auto commit. +-- 3. (Done) Create table in public schema. +-- 4. Generate error in transaction. +-- 5. END transaction. +-- 6. Check if table is *NOT* created after ending transaction. +SELECT 1/0;""" + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "division by zero")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) END any open transaction. +-- 2. (Done) Enable auto rollback and disable auto commit. +-- 3. (Done) Create table in public schema. +-- 4. (Done) Generate error in transaction. +-- 5. END transaction. +-- 6. Check if table is *NOT* created after ending transaction. +END;""" + + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "Query returned successfully")]' + ) + + self._clear_query_tool() + query = """-- 1. (Done) END any open transaction. +-- 2. (Done) Enable auto rollback and disable auto commit. +-- 3. (Done) Create table in public schema. +-- 4. (Done) Generate error in transaction. +-- 5. (Done) END transaction. +-- 6. Check if table is *NOT* created after ending transaction. +SELECT relname FROM pg_class WHERE relkind IN ('r','s','t') and relnamespace = 2200::oid;""" + self.page.fill_codemirror_area_with(query) + self.page.find_by_id("btn-flash").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Data Output') + canvas = wait.until(EC.presence_of_element_located( + (By.CSS_SELECTOR, "#datagrid .slick-viewport .grid-canvas"))) + + el = canvas.find_elements_by_xpath("//div[contains(@class, 'slick-cell') and contains(text(), '{}')]".format(table_name)) + + assert len(el) == 0, "Table '{}' created even after ROLLBACK due to sql error.".format(table_name) + + def _query_tool_cancel_query(self): + query = """-- 1. END any open transaction. +-- 2. Enable auto commit and Disable auto rollback. +-- 3. Execute long running query. +-- 4. Cancel long running query execution. +END; +SELECT 1, pg_sleep(10)""" + self.page.fill_codemirror_area_with(query) + + self.page.find_by_id("btn-query-dropdown").click() + + auto_rollback_btn = self.page.find_by_id("btn-auto-rollback") + + auto_rollback_check = auto_rollback_btn.find_element_by_tag_name("i") + + # if auto rollback is enabled then 'i' element will + # have 'auto-rollback fa fa-check' classes + # if auto rollback is disabled then 'i' element will + # have 'auto-rollback fa fa-check visibility-hidden' classes + + if 'auto-rollback fa fa-check' == str(auto_rollback_check.get_attribute( + 'class')): + auto_rollback_btn.click() + + auto_commit_btn = self.page.find_by_id("btn-auto-commit") + + auto_commit_check = auto_commit_btn.find_element_by_tag_name("i") + + # if auto commit is enabled then 'i' element will + # have 'auto-commit fa fa-check' classes + # if auto commit is disabled then 'i' element will + # have 'auto-commit fa fa-check visibility-hidden' classes + + if 'auto-commit fa fa-check visibility-hidden' == str(auto_commit_check.get_attribute( + 'class')): + auto_commit_btn.click() + + self.page.find_by_id("btn-flash").click() + self.driver.find_element_by_xpath("//*[@id='fetching_data']") + self.page.find_by_id("btn-cancel-query").click() + self.page.wait_for_query_tool_loading_indicator_to_disappear() + self.page.click_tab('Messages') + self.driver.find_element_by_xpath( + '//div[contains(@class, "sql-editor-message") and contains(string(), "canceling statement due to user request")]' + ) diff --git a/web/pgadmin/feature_tests/view_data_dml_queries.py b/web/pgadmin/feature_tests/view_data_dml_queries.py index 55b45ad..a6bc5aa 100644 --- a/web/pgadmin/feature_tests/view_data_dml_queries.py +++ b/web/pgadmin/feature_tests/view_data_dml_queries.py @@ -9,6 +9,7 @@ import json import os +import time from selenium.webdriver import ActionChains from regression.python_test_utils import test_utils from regression.feature_utils.base_feature_test import BaseFeatureTest @@ -205,6 +206,10 @@ CREATE TABLE public.defaults self.page.driver.find_element_by_link_text("View Data")) \ .perform() self.page.find_by_partial_link_text("View All Rows").click() + + # wait until datagrid frame is loaded. + self.page.click_tab('Edit Data -') + self.wait.until( EC.visibility_of_element_located( (By.CSS_SELECTOR, 'iframe') @@ -242,6 +247,11 @@ CREATE TABLE public.defaults self._update_cell(row1_cell2_xpath, ["1", "", "int"]) self.page.find_by_id("btn-save").click() # Save data + # There should be some delay after save button is clicked, as it + # takes some time to complete save ajax call otherwise discard unsaved + # changes dialog will appear if we try to execute query before previous + # save ajax is completed. + time.sleep(2) # Verify row 1 and row 2 data self._verify_row_data(False) @@ -254,6 +264,11 @@ CREATE TABLE public.defaults self._update_cell(cell_xpath, config_data[str(idx)]) self.page.find_by_id("btn-save").click() # Save data + # There should be some delay after save button is clicked, as it + # takes some time to complete save ajax call otherwise discard unsaved + # changes dialog will appear if we try to execute query before previous + # save ajax is completed. + time.sleep(2) def _verify_row_data(self, is_new_row): self.page.find_by_id("btn-flash").click() @@ -264,17 +279,17 @@ CREATE TABLE public.defaults xpath = "//*[contains(@class, 'ui-widget-content') and " \ "contains(@style, 'top:" + str(row_height) + "px')]" - # wait for stale element reference exception - self.page.wait_for_element_to_stale(xpath) + self.page.wait_for_query_tool_loading_indicator_to_disappear() + result_row = self.page.find_by_xpath(xpath) # List of row values in an array cells = [el.text for el in result_row.find_elements_by_tag_name('div')] for idx in range(1, len(config_data.keys())): - # # after copy & paste row, the first cell of row 1 and - # # row 2(being primary keys) won't match - # # see if cell values matched to actual value + # after copy & paste row, the first cell of row 1 and + # row 2(being primary keys) won't match + # see if cell values matched to actual value if idx != 1 and not is_new_row: self.assertEquals(cells[idx], config_data[str(idx)][1]) elif is_new_row: diff --git a/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py b/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py index 4fd0783..75b4222 100644 --- a/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py +++ b/web/pgadmin/feature_tests/xss_checks_panels_and_query_tool_test.py @@ -10,6 +10,9 @@ from selenium.webdriver import ActionChains from regression.python_test_utils import test_utils from regression.feature_utils.base_feature_test import BaseFeatureTest +from selenium.webdriver.support.ui import WebDriverWait +from selenium.webdriver.support import expected_conditions as EC +from selenium.webdriver.common.by import By import time class CheckForXssFeatureTest(BaseFeatureTest): @@ -72,6 +75,7 @@ class CheckForXssFeatureTest(BaseFeatureTest): def _connects_to_server(self): self.page.find_by_xpath("//*[@class='aciTreeText' and .='Servers']").click() + time.sleep(2) self.page.driver.find_element_by_link_text("Object").click() ActionChains(self.page.driver) \ .move_to_element(self.page.driver.find_element_by_link_text("Create")) \ @@ -152,11 +156,16 @@ class CheckForXssFeatureTest(BaseFeatureTest): self.page.fill_codemirror_area_with("select '<img src=\"x\" onerror=\"console.log(1)\">'") time.sleep(1) self.page.find_by_id("btn-flash").click() - time.sleep(2) + wait = WebDriverWait(self.page.driver, 5) - source_code = self.page.find_by_xpath( - "//*[@id='0']//*[@id='datagrid']/div[5]/div/div[1]/div[2]" - ).get_attribute('innerHTML') + result_row = self.page.find_by_xpath( + "//*[contains(@class, 'ui-widget-content') and contains(@style, 'top:0px')]" + ) + + cells = result_row.find_elements_by_tag_name('div') + + # remove first element as it is row number. + source_code = cells[1].get_attribute('innerHTML') self._check_escaped_characters( source_code, diff --git a/web/pgadmin/feature_tests/xss_checks_pgadmin_debugger_test.py b/web/pgadmin/feature_tests/xss_checks_pgadmin_debugger_test.py index 094dfed..e847040 100644 --- a/web/pgadmin/feature_tests/xss_checks_pgadmin_debugger_test.py +++ b/web/pgadmin/feature_tests/xss_checks_pgadmin_debugger_test.py @@ -42,6 +42,7 @@ class CheckDebuggerForXssFeatureTest(BaseFeatureTest): def _connects_to_server(self): self.page.find_by_xpath("//*[@class='aciTreeText' and .='Servers']").click() + time.sleep(2) self.page.driver.find_element_by_link_text("Object").click() ActionChains(self.page.driver) \ .move_to_element(self.page.driver.find_element_by_link_text("Create")) \ diff --git a/web/pgadmin/static/bundle/slickgrid.js b/web/pgadmin/static/bundle/slickgrid.js index 0d82a6b..dd7f870 100644 --- a/web/pgadmin/static/bundle/slickgrid.js +++ b/web/pgadmin/static/bundle/slickgrid.js @@ -3,6 +3,7 @@ import 'slickgrid/slick-default-theme.css'; import 'slickgrid/css/smoothness/jquery-ui-1.11.3.custom.css'; import 'slickgrid/slick.core'; import 'slickgrid/slick.grid'; +import 'slickgrid/slick.dataview'; import 'slickgrid/slick.editors'; import 'slickgrid/slick.formatters'; import 'slickgrid/plugins/slick.autotooltips'; diff --git a/web/pgadmin/static/js/selection/column_selector.js b/web/pgadmin/static/js/selection/column_selector.js index 305b438..f3cb727 100644 --- a/web/pgadmin/static/js/selection/column_selector.js +++ b/web/pgadmin/static/js/selection/column_selector.js @@ -4,13 +4,18 @@ define([ 'slickgrid', ], function ($, RangeSelectionHelper) { var ColumnSelector = function () { - var Slick = window.Slick; - var gridEventBus = new Slick.EventHandler(); + var Slick = window.Slick, + gridEventBus = new Slick.EventHandler(), + onBeforeColumnSelectAll = new Slick.Event(), + onColumnSelectAll = new Slick.Event(); var init = function (grid) { gridEventBus.subscribe(grid.onHeaderClick, handleHeaderClick.bind(null, grid)); grid.getSelectionModel().onSelectedRangesChanged .subscribe(handleSelectedRangesChanged.bind(null, grid)); + onColumnSelectAll.subscribe(function(e, args) { + updateRanges(args.grid, args.column.id); + }); }; var handleHeaderClick = function (grid, event, args) { @@ -21,11 +26,20 @@ define([ if (isColumnSelectable(columnDefinition)) { var $columnHeader = $(event.target); if (hasClickedChildOfColumnHeader(event)) { + if ($(event.target).hasClass('slick-resizable-handle')) { + return; + } $columnHeader = $(event.target).parents('.slick-header-column'); } $columnHeader.toggleClass('selected'); - updateRanges(grid, columnDefinition.id); + if ($columnHeader.hasClass('selected')) { + onBeforeColumnSelectAll.notify(args, event); + } + + if (!(event.isPropagationStopped() || event.isImmediatePropagationStopped())) { + updateRanges(grid, columnDefinition.id); + } } }; @@ -107,6 +121,8 @@ define([ $.extend(this, { 'init': init, 'getColumnDefinitions': getColumnDefinitions, + 'onBeforeColumnSelectAll': onBeforeColumnSelectAll, + 'onColumnSelectAll': onColumnSelectAll, }); }; return ColumnSelector; diff --git a/web/pgadmin/static/js/selection/copy_data.js b/web/pgadmin/static/js/selection/copy_data.js index c226323..3525966 100644 --- a/web/pgadmin/static/js/selection/copy_data.js +++ b/web/pgadmin/static/js/selection/copy_data.js @@ -12,19 +12,19 @@ function ($, _, clipboard, RangeSelectionHelper, rangeBoundaryNavigator) { var grid = self.slickgrid; var columnDefinitions = grid.getColumns(); var selectedRanges = grid.getSelectionModel().getSelectedRanges(); - var data = grid.getData(); + var dataView = grid.getData(); var rows = grid.getSelectedRows(); if (RangeSelectionHelper.areAllRangesCompleteRows(grid, selectedRanges)) { self.copied_rows = rows.map(function (rowIndex) { - return data[rowIndex]; + return grid.getDataItem(rowIndex); }); setPasteRowButtonEnablement(self.can_edit, true); } else { self.copied_rows = []; setPasteRowButtonEnablement(self.can_edit, false); } - var csvText = rangeBoundaryNavigator.rangesToCsv(data, columnDefinitions, selectedRanges); + var csvText = rangeBoundaryNavigator.rangesToCsv(dataView.getItems(), columnDefinitions, selectedRanges); if (csvText) { clipboard.copyTextToClipboard(csvText); } diff --git a/web/pgadmin/static/js/selection/grid_selector.js b/web/pgadmin/static/js/selection/grid_selector.js index 24fd9d6..ddbbe23 100644 --- a/web/pgadmin/static/js/selection/grid_selector.js +++ b/web/pgadmin/static/js/selection/grid_selector.js @@ -6,21 +6,31 @@ define(['jquery', 'sources/url_for', ], function ($, gettext, ColumnSelector, RowSelector, RangeSelectionHelper, url_for) { var GridSelector = function (columnDefinitions) { - var rowSelector = new RowSelector(columnDefinitions); - var columnSelector = new ColumnSelector(columnDefinitions); + var Slick = window.Slick, + rowSelector = new RowSelector(columnDefinitions), + columnSelector = new ColumnSelector(columnDefinitions), + onBeforeGridSelectAll = new Slick.Event(), + onGridSelectAll = new Slick.Event(), + onBeforeGridColumnSelectAll = columnSelector.onBeforeColumnSelectAll, + onGridColumnSelectAll = columnSelector.onColumnSelectAll; var init = function (grid) { this.grid = grid; grid.onHeaderClick.subscribe(function (event, eventArguments) { - if (eventArguments.column.selectAllOnClick) { - toggleSelectAll(grid); + if (eventArguments.column.selectAllOnClick && !$(event.target).hasClass('slick-resizable-handle')) { + toggleSelectAll(grid, event, eventArguments); } }); grid.getSelectionModel().onSelectedRangesChanged - .subscribe(handleSelectedRangesChanged.bind(null, grid)); + .subscribe(handleSelectedRangesChanged.bind(null, grid)); + grid.registerPlugin(rowSelector); grid.registerPlugin(columnSelector); + + onGridSelectAll.subscribe(function(e, args) { + RangeSelectionHelper.selectAll(args.grid); + }); }; var getColumnDefinitions = function (columnDefinitions) { @@ -45,11 +55,14 @@ define(['jquery', } } - function toggleSelectAll(grid) { + function toggleSelectAll(grid, event, eventArguments) { if (RangeSelectionHelper.isEntireGridSelected(grid)) { selectNone(grid); } else { - RangeSelectionHelper.selectAll(grid); + onBeforeGridSelectAll.notify(eventArguments, event); + if (!(event.isPropagationStopped() || event.isImmediatePropagationStopped())) { + RangeSelectionHelper.selectAll(grid); + } } } @@ -61,6 +74,10 @@ define(['jquery', $.extend(this, { 'init': init, 'getColumnDefinitions': getColumnDefinitions, + 'onBeforeGridSelectAll': onBeforeGridSelectAll, + 'onGridSelectAll': onGridSelectAll, + 'onBeforeGridColumnSelectAll': onBeforeGridColumnSelectAll, + 'onGridColumnSelectAll': onGridColumnSelectAll, }); }; diff --git a/web/pgadmin/static/js/selection/range_boundary_navigator.js b/web/pgadmin/static/js/selection/range_boundary_navigator.js index 803f75a..274b621 100644 --- a/web/pgadmin/static/js/selection/range_boundary_navigator.js +++ b/web/pgadmin/static/js/selection/range_boundary_navigator.js @@ -58,6 +58,7 @@ function (RangeSelectionHelper) { }, rangesToCsv: function (data, columnDefinitions, selectedRanges) { + var rowRangeBounds = selectedRanges.map(function (range) { return [range.fromRow, range.toRow]; }); @@ -72,6 +73,7 @@ function (RangeSelectionHelper) { var csvRows = this.mapOver2DArray(rowRangeBounds, colRangeBounds, this.csvCell.bind(this, data, columnDefinitions), function (rowData) { return rowData.join(','); }); + return csvRows.join('\n'); }, @@ -101,7 +103,7 @@ function (RangeSelectionHelper) { }, csvCell: function (data, columnDefinitions, rowId, colId) { - var val = data[rowId][columnDefinitions[colId].pos]; + var val = data[rowId][columnDefinitions[colId].field]; if (val && _.isObject(val)) { val = '\'' + JSON.stringify(val) + '\''; diff --git a/web/pgadmin/static/js/selection/row_selector.js b/web/pgadmin/static/js/selection/row_selector.js index 236ed62..59d2fa2 100644 --- a/web/pgadmin/static/js/selection/row_selector.js +++ b/web/pgadmin/static/js/selection/row_selector.js @@ -82,7 +82,8 @@ define([ formatter: function (rowIndex) { return '<span ' + 'data-row="' + rowIndex + '" ' + - 'data-cell-type="row-header-selector"/>'; + 'data-cell-type="row-header-selector">' + + (rowIndex+1) + '</span>'; }, width: 30, }); diff --git a/web/pgadmin/static/js/selection/set_staged_rows.js b/web/pgadmin/static/js/selection/set_staged_rows.js index 46977e3..bc07dd5 100644 --- a/web/pgadmin/static/js/selection/set_staged_rows.js +++ b/web/pgadmin/static/js/selection/set_staged_rows.js @@ -22,53 +22,44 @@ define( $(selector).prop('disabled', false); } - function getRowPrimaryKeyValuesToStage(selectedRows, primaryKeyColumnIndices, gridData) { + function getRowPrimaryKeyValuesToStage(selectedRows, primaryKeys, dataView, client_primary_key) { return _.reduce(selectedRows, function (primaryKeyValuesToStage, dataGridRowIndex) { - var gridRow = gridData[dataGridRowIndex]; - - if (isRowMissingPrimaryKeys(gridRow, primaryKeyColumnIndices)) { + var gridRow = dataView.getItem(dataGridRowIndex); + if (isRowMissingPrimaryKeys(gridRow, primaryKeys)) { return primaryKeyValuesToStage; } - - var tempPK = gridRow.__temp_PK; - primaryKeyValuesToStage[tempPK] = getSingleRowPrimaryKeyValueToStage(primaryKeyColumnIndices, gridRow); - + var tempPK = gridRow[client_primary_key]; + primaryKeyValuesToStage[tempPK] = getSingleRowPrimaryKeyValueToStage(primaryKeys, gridRow); return primaryKeyValuesToStage; }, {}); } - function isRowMissingPrimaryKeys(gridRow, primaryKeyColumnIndices) { + function isRowMissingPrimaryKeys(gridRow, primaryKeys) { if (_.isUndefined(gridRow)) { return true; } return !_.isUndefined( - _.find(primaryKeyColumnIndices, function (pkIndex) { - return _.isUndefined(gridRow[pkIndex]); + _.find(primaryKeys , function (pk) { + return _.isUndefined(gridRow[pk]); }) ); } - function getSingleRowPrimaryKeyValueToStage(primaryKeyColumnIndices, gridRow) { + function getSingleRowPrimaryKeyValueToStage(primaryKeys, gridRow) { var rowToStage = {}; - if (primaryKeyColumnIndices.length) { - _.each(_.keys(gridRow), function (columnPos) { - if (_.contains(primaryKeyColumnIndices, Number(columnPos))) - rowToStage[columnPos] = gridRow[columnPos]; + if (primaryKeys && primaryKeys.length) { + _.each(_.keys(gridRow), function (columnNames) { + if (_.contains(primaryKeys, columnNames)) + rowToStage[columnNames] = gridRow[columnNames]; }); } return rowToStage; } function getPrimaryKeysForSelectedRows(self, selectedRows) { - var primaryKeyColumnIndices = _.map(_.keys(self.keys), function (columnName) { - var columnInfo = _.findWhere(self.columns, {name: columnName}); - return columnInfo['pos']; - }); - - var gridData = self.grid.getData(); - var stagedRows = getRowPrimaryKeyValuesToStage(selectedRows, primaryKeyColumnIndices, gridData); - + var dataView = self.grid.getData(); + var stagedRows = getRowPrimaryKeyValuesToStage(selectedRows, _.keys(self.keys), dataView, self.client_primary_key); return stagedRows; } @@ -114,4 +105,4 @@ define( }; return setStagedRows; } -); +); \ No newline at end of file diff --git a/web/pgadmin/static/js/slickgrid/slick.pgadmin.editors.js b/web/pgadmin/static/js/slickgrid/slick.pgadmin.editors.js index f2dabe0..15b5fe9 100644 --- a/web/pgadmin/static/js/slickgrid/slick.pgadmin.editors.js +++ b/web/pgadmin/static/js/slickgrid/slick.pgadmin.editors.js @@ -76,18 +76,18 @@ last_value = (column_type === 'number') ? (_.isEmpty(last_value) || last_value) : last_value; - item[args.column.pos] = state; + item[args.column.field] = state; if (last_value && _.isNull(state) && (_.isUndefined(grid.copied_rows[row]) || _.isUndefined(grid.copied_rows[row][cell])) ) { - item[args.column.pos] = undefined; + item[args.column.field] = undefined; if (grid.copied_rows[row] == undefined) grid.copied_rows[row] = []; grid.copied_rows[row][cell] = 1; } } else { - item[args.column.pos] = state; + item[args.column.field] = state; } } @@ -189,14 +189,14 @@ this.loadValue = function (item) { var col = args.column; - if (_.isUndefined(item[args.column.pos]) && col.has_default_val) { + if (_.isUndefined(item[args.column.field]) && col.has_default_val) { $input.val(defaultValue = ""); } - else if (item[args.column.pos] === "") { + else if (item[args.column.field] === "") { $input.val(defaultValue = "''"); } else { - $input.val(defaultValue = item[args.column.pos]); + $input.val(defaultValue = item[args.column.field]); $input.select(); } }; @@ -323,7 +323,7 @@ }; this.loadValue = function (item) { - var data = defaultValue = item[args.column.pos]; + var data = defaultValue = item[args.column.field]; if (data && typeof data === "object" && !Array.isArray(data)) { data = JSON.stringify(data); } else if (Array.isArray(data)) { @@ -443,7 +443,7 @@ }; this.loadValue = function (item) { - $input.val(defaultValue = item[args.column.pos]); + $input.val(defaultValue = item[args.column.field]); $input.select(); }; @@ -452,7 +452,7 @@ }; this.applyValue = function (item, state) { - item[args.column.pos] = state; + item[args.column.field] = state; }; this.isValueChanged = function () { @@ -531,13 +531,13 @@ }; this.loadValue = function (item) { - defaultValue = item[args.column.pos]; - if (_.isNull(defaultValue)|| _.isUndefined(defaultValue)) { + defaultValue = item[args.column.field]; + if (_.isNull(defaultValue)||_.isUndefined(defaultValue)) { $select.prop('indeterminate', true); $select.data('checked', 2); } else { - defaultValue = !!item[args.column.pos]; + defaultValue = !!item[args.column.field]; if (defaultValue) { $select.prop('checked', true); $select.data('checked', 0); @@ -556,7 +556,7 @@ }; this.applyValue = function (item, state) { - item[args.column.pos] = state; + item[args.column.field] = state; }; this.isValueChanged = function () { @@ -648,7 +648,7 @@ }; this.loadValue = function (item) { - var data = defaultValue = item[args.column.pos]; + var data = defaultValue = item[args.column.field]; if (typeof data === "object" && !Array.isArray(data)) { data = JSON.stringify(data); } else if (Array.isArray(data)) { @@ -671,7 +671,7 @@ }; this.applyValue = function (item, state) { - item[args.column.pos] = state; + item[args.column.field] = state; }; this.isValueChanged = function () { @@ -725,7 +725,7 @@ }; this.loadValue = function (item) { - var value = item[args.column.pos]; + var value = item[args.column.field]; // Check if value is null or undefined if (value === undefined && typeof value === "undefined") { @@ -858,7 +858,7 @@ }; this.loadValue = function (item) { - defaultValue = item[args.column.pos]; + defaultValue = item[args.column.field]; $input.val(defaultValue); $input[0].defaultValue = defaultValue; $input.select(); diff --git a/web/pgadmin/tools/sqleditor/__init__.py b/web/pgadmin/tools/sqleditor/__init__.py index 9c0ba00..c6fcdc3 100644 --- a/web/pgadmin/tools/sqleditor/__init__.py +++ b/web/pgadmin/tools/sqleditor/__init__.py @@ -27,7 +27,7 @@ from pgadmin.utils.sqlautocomplete.autocomplete import SQLAutoComplete from pgadmin.misc.file_manager import Filemanager -from config import PG_DEFAULT_DRIVER +from config import PG_DEFAULT_DRIVER, ON_DEMAND_RECORD_COUNT MODULE_NAME = 'sqleditor' @@ -82,9 +82,9 @@ class SqlEditorModule(PgAdminModule): 'sqleditor.view_data_start', 'sqleditor.query_tool_start', 'sqleditor.query_tool_preferences', - 'sqleditor.get_columns', 'sqleditor.poll', - 'sqleditor.fetch_types', + 'sqleditor.fetch', + 'sqleditor.fetch_all', 'sqleditor.save', 'sqleditor.get_filter', 'sqleditor.apply_filter', @@ -261,13 +261,32 @@ def start_view_data(trans_id): # Check the transaction and connection status status, error_msg, conn, trans_obj, session_obj = check_transaction_status(trans_id) + + # get the default connection as current connection which is attached to + # trans id holds the cursor which has query result so we cannot use that + # connection to execute another query otherwise we'll lose query result. + + manager = get_driver(PG_DEFAULT_DRIVER).connection_manager(trans_obj.sid) + default_conn = manager.connection(did=trans_obj.did) + + # Connect to the Server if not connected. + if not default_conn.connected(): + status, msg = default_conn.connect() + if not status: + return make_json_response( + data={'status': status, 'result': u"{}".format(msg)} + ) + if status and conn is not None \ and trans_obj is not None and session_obj is not None: try: + # set fetched row count to 0 as we are executing query again. + trans_obj.update_fetched_row_cnt(0) + session_obj['command_obj'] = pickle.dumps(trans_obj, -1) # Fetch the sql and primary_keys from the object sql = trans_obj.get_sql() - pk_names, primary_keys = trans_obj.get_primary_keys() + pk_names, primary_keys = trans_obj.get_primary_keys(default_conn) # Fetch the applied filter. filter_applied = trans_obj.is_filter_applied() @@ -338,6 +357,8 @@ def start_query_tool(trans_id): # Use pickle.loads function to get the command object session_obj = grid_data[str(trans_id)] trans_obj = pickle.loads(session_obj['command_obj']) + # set fetched row count to 0 as we are executing query again. + trans_obj.update_fetched_row_cnt(0) can_edit = False can_filter = False @@ -467,66 +488,6 @@ def preferences(trans_id): return success_return() [email protected]( - '/columns/<int:trans_id>', methods=["GET"], endpoint='get_columns' -) -@login_required -def get_columns(trans_id): - """ - This method will returns list of columns of last async query. - - Args: - trans_id: unique transaction id - """ - columns = dict() - columns_info = None - primary_keys = None - rset = None - status, error_msg, conn, trans_obj, session_obj = check_transaction_status(trans_id) - if status and conn is not None and session_obj is not None: - - ver = conn.manager.version - # Get the template path for the column - template_path = 'column/sql/#{0}#'.format(ver) - command_obj = pickle.loads(session_obj['command_obj']) - if hasattr(command_obj, 'obj_id'): - SQL = render_template("/".join([template_path, - 'nodes.sql']), - tid=command_obj.obj_id) - # rows with attribute not_null - status, rset = conn.execute_2darray(SQL) - if not status: - return internal_server_error(errormsg=rset) - - # Check PK column info is available or not - if 'primary_keys' in session_obj: - primary_keys = session_obj['primary_keys'] - - # Fetch column information - columns_info = conn.get_column_info() - if columns_info is not None: - for key, col in enumerate(columns_info): - col_type = dict() - col_type['type_code'] = col['type_code'] - col_type['type_name'] = None - if rset: - col_type['not_null'] = col['not_null'] = \ - rset['rows'][key]['not_null'] - - col_type['has_default_val'] = col['has_default_val'] = \ - rset['rows'][key]['has_default_val'] - - columns[col['name']] = col_type - - # As we changed the transaction object we need to - # restore it and update the session variable. - session_obj['columns_info'] = columns - update_session_grid_transaction(trans_id, session_obj) - - return make_json_response(data={'status': True, - 'columns': columns_info, - 'primary_keys': primary_keys}) - @blueprint.route('/poll/<int:trans_id>', methods=["GET"], endpoint='poll') @login_required @@ -539,12 +500,21 @@ def poll(trans_id): """ result = None rows_affected = 0 + rows_fetched_from = 0 + rows_fetched_to = 0 + has_more_rows = False additional_result = [] + columns = dict() + columns_info = None + primary_keys = None + types = {} + client_primary_key = None + rset = None # Check the transaction and connection status status, error_msg, conn, trans_obj, session_obj = check_transaction_status(trans_id) if status and conn is not None and session_obj is not None: - status, result = conn.poll(formatted_exception_msg=True) + status, result = conn.poll(formatted_exception_msg=True, no_result=True) if not status: return internal_server_error(result) elif status == ASYNC_OK: @@ -559,6 +529,80 @@ def poll(trans_id): if (trans_status == TX_STATUS_INERROR and trans_obj.auto_rollback): conn.execute_void("ROLLBACK;") + + st, result = conn.async_fetchmany_2darray(ON_DEMAND_RECORD_COUNT) + if st: + if 'primary_keys' in session_obj: + primary_keys = session_obj['primary_keys'] + + # Fetch column information + columns_info = conn.get_column_info() + client_primary_key = generate_client_primary_key_name( + columns_info + ) + session_obj['client_primary_key'] = client_primary_key + + if columns_info is not None: + + command_obj = pickle.loads(session_obj['command_obj']) + if hasattr(command_obj, 'obj_id'): + # Get the template path for the column + template_path = 'column/sql/#{0}#'.format( + conn.manager.version + ) + + SQL = render_template("/".join([template_path, + 'nodes.sql']), + tid=command_obj.obj_id) + # rows with attribute not_null + colst, rset = conn.execute_2darray(SQL) + if not colst: + return internal_server_error(errormsg=rset) + + for key, col in enumerate(columns_info): + col_type = dict() + col_type['type_code'] = col['type_code'] + col_type['type_name'] = None + columns[col['name']] = col_type + + if rset: + col_type['not_null'] = col['not_null'] = \ + rset['rows'][key]['not_null'] + + col_type['has_default_val'] = \ + col['has_default_val'] = \ + rset['rows'][key]['has_default_val'] + + if columns: + st, types = fetch_pg_types(columns, trans_obj) + + if not st: + return internal_server_error(types) + + for col_info in columns.values(): + for col_type in types: + if col_type['oid'] == col_info['type_code']: + col_info['type_name'] = col_type['typname'] + + session_obj['columns_info'] = columns + # status of async_fetchmany_2darray is True and result is none + # means nothing to fetch + if result and rows_affected > -1: + res_len = len(result) + if res_len == ON_DEMAND_RECORD_COUNT: + has_more_rows = True + + if res_len > 0: + rows_fetched_from = trans_obj.get_fetched_row_cnt() + trans_obj.update_fetched_row_cnt(rows_fetched_from + res_len) + rows_fetched_from += 1 + rows_fetched_to = trans_obj.get_fetched_row_cnt() + session_obj['command_obj'] = pickle.dumps(trans_obj, -1) + + # As we changed the transaction object we need to + # restore it and update the session variable. + update_session_grid_transaction(trans_id, session_obj) + elif status == ASYNC_EXECUTION_ABORTED: status = 'Cancel' else: @@ -599,53 +643,123 @@ def poll(trans_id): data={ 'status': status, 'result': result, 'rows_affected': rows_affected, - 'additional_messages': additional_messages + 'rows_fetched_from': rows_fetched_from, + 'rows_fetched_to': rows_fetched_to, + 'additional_messages': additional_messages, + 'has_more_rows': has_more_rows, + 'colinfo': columns_info, + 'primary_keys': primary_keys, + 'types': types, + 'client_primary_key': client_primary_key } ) [email protected]( - '/fetch/types/<int:trans_id>', methods=["GET"], endpoint='fetch_types' -) [email protected]('/fetch/<int:trans_id>', methods=["GET"], endpoint='fetch') [email protected]('/fetch/<int:trans_id>/<int:fetch_all>', methods=["GET"], endpoint='fetch_all') @login_required -def fetch_pg_types(trans_id): +def fetch(trans_id, fetch_all=None): + result = None + has_more_rows = False + rows_fetched_from = 0 + rows_fetched_to = 0 + fetch_row_cnt = -1 if fetch_all == 1 else ON_DEMAND_RECORD_COUNT + + # Check the transaction and connection status + status, error_msg, conn, trans_obj, session_obj = check_transaction_status(trans_id) + if status and conn is not None and session_obj is not None: + status, result = conn.async_fetchmany_2darray(fetch_row_cnt) + if not status: + status = 'Error' + else: + status = 'Success' + res_len = len(result) + if fetch_row_cnt != -1 and res_len == ON_DEMAND_RECORD_COUNT: + has_more_rows = True + + if res_len: + rows_fetched_from = trans_obj.get_fetched_row_cnt() + trans_obj.update_fetched_row_cnt(rows_fetched_from + res_len) + rows_fetched_from += 1 + rows_fetched_to = trans_obj.get_fetched_row_cnt() + session_obj['command_obj'] = pickle.dumps(trans_obj, -1) + update_session_grid_transaction(trans_id, session_obj) + else: + status = 'NotConnected' + result = error_msg + + return make_json_response( + data={ + 'status': status, 'result': result, + 'has_more_rows': has_more_rows, + 'rows_fetched_from': rows_fetched_from, + 'rows_fetched_to': rows_fetched_to + } + ) + + +def fetch_pg_types(columns_info, trans_obj): """ This method is used to fetch the pg types, which is required to map the data type comes as a result of the query. Args: - trans_id: unique transaction id + columns_info: """ - # Check the transaction and connection status - status, error_msg, conn, trans_obj, session_obj = check_transaction_status(trans_id) - if status and conn is not None \ - and trans_obj is not None and session_obj is not None: - res = {} - if 'columns_info' in session_obj \ - and session_obj['columns_info'] is not None: + # get the default connection as current connection attached to trans id + # holds the cursor which has query result so we cannot use that connection + # to execute another query otherwise we'll lose query result. + + manager = get_driver(PG_DEFAULT_DRIVER).connection_manager(trans_obj.sid) + default_conn = manager.connection(did=trans_obj.did) - oids = [session_obj['columns_info'][col]['type_code'] for col in session_obj['columns_info']] + # Connect to the Server if not connected. + res = [] + if not default_conn.connected(): + status, msg = default_conn.connect() + if not status: + return status, msg - if oids: - status, res = conn.execute_dict( - u"""SELECT oid, format_type(oid,null) as typname FROM pg_type WHERE oid IN %s ORDER BY oid; + oids = [columns_info[col]['type_code'] for col in columns_info] + + if oids: + status, res = default_conn.execute_dict( + u"""SELECT oid, format_type(oid,null) as typname FROM pg_type WHERE oid IN %s ORDER BY oid; """, [tuple(oids)]) - if status: - # iterate through pg_types and update the type name in session object - for record in res['rows']: - for col in session_obj['columns_info']: - type_obj = session_obj['columns_info'][col] - if type_obj['type_code'] == record['oid']: - type_obj['type_name'] = record['typname'] + if not status: + return False, res - update_session_grid_transaction(trans_id, session_obj) + return status, res['rows'] else: - status = False - res = error_msg - - return make_json_response(data={'status': status, 'result': res}) + return True, [] + + +def generate_client_primary_key_name(columns_info): + temp_key = '__temp_PK' + if not columns_info: + return temp_key + + initial_temp_key_len = len(temp_key) + duplicate = False + suffix = 1 + while 1: + for col in columns_info: + if col['name'] == temp_key: + duplicate = True + break + if duplicate: + if initial_temp_key_len == len(temp_key): + temp_key += str(suffix) + suffix += 1 + else: + temp_key = temp_key[:-1] + str(suffix) + suffix += 1 + duplicate = False + else: + break + return temp_key @blueprint.route( @@ -659,7 +773,6 @@ def save(trans_id): Args: trans_id: unique transaction id """ - if request.data: changed_data = json.loads(request.data, encoding='utf-8') else: @@ -669,7 +782,6 @@ def save(trans_id): status, error_msg, conn, trans_obj, session_obj = check_transaction_status(trans_id) if status and conn is not None \ and trans_obj is not None and session_obj is not None: - setattr(trans_obj, 'columns_info', session_obj['columns_info']) # If there is no primary key found then return from the function. if len(session_obj['primary_keys']) <= 0 or len(changed_data) <= 0: @@ -680,7 +792,22 @@ def save(trans_id): } ) - status, res, query_res, _rowid = trans_obj.save(changed_data) + manager = get_driver(PG_DEFAULT_DRIVER).connection_manager(trans_obj.sid) + default_conn = manager.connection(did=trans_obj.did) + + # Connect to the Server if not connected. + if not default_conn.connected(): + status, msg = default_conn.connect() + if not status: + return make_json_response( + data={'status': status, 'result': u"{}".format(msg)} + ) + + status, res, query_res, _rowid = trans_obj.save( + changed_data, + session_obj['columns_info'], + session_obj['client_primary_key'], + default_conn) else: status = False res = error_msg diff --git a/web/pgadmin/tools/sqleditor/command.py b/web/pgadmin/tools/sqleditor/command.py index b7d8a78..3b7efd0 100644 --- a/web/pgadmin/tools/sqleditor/command.py +++ b/web/pgadmin/tools/sqleditor/command.py @@ -258,7 +258,21 @@ class SQLFilter(object): return status, result -class GridCommand(BaseCommand, SQLFilter): +class FetchedRowTracker(object): + """ + Keeps track of fetched row count. + """ + def __init__(self, **kwargs): + self.fetched_rows = 0 + + def get_fetched_row_cnt(self): + return self.fetched_rows + + def update_fetched_row_cnt(self, rows_cnt): + self.fetched_rows = rows_cnt + + +class GridCommand(BaseCommand, SQLFilter, FetchedRowTracker): """ class GridCommand(object) @@ -290,6 +304,7 @@ class GridCommand(BaseCommand, SQLFilter): """ BaseCommand.__init__(self, **kwargs) SQLFilter.__init__(self, **kwargs) + FetchedRowTracker.__init__(self, **kwargs) # Save the connection id, command type self.conn_id = kwargs['conn_id'] if 'conn_id' in kwargs else None @@ -299,10 +314,10 @@ class GridCommand(BaseCommand, SQLFilter): if self.cmd_type == VIEW_FIRST_100_ROWS or self.cmd_type == VIEW_LAST_100_ROWS: self.limit = 100 - def get_primary_keys(self): + def get_primary_keys(self, *args, **kwargs): return None, None - def save(self, changed_data): + def save(self, changed_data, default_conn=None): return forbidden(errmsg=gettext("Data cannot be saved for the current object.")) def get_limit(self): @@ -340,14 +355,14 @@ class TableCommand(GridCommand): # call base class init to fetch the table name super(TableCommand, self).__init__(**kwargs) - def get_sql(self): + def get_sql(self, default_conn=None): """ This method is used to create a proper SQL query to fetch the data for the specified table """ # Fetch the primary keys for the table - pk_names, primary_keys = self.get_primary_keys() + pk_names, primary_keys = self.get_primary_keys(default_conn) sql_filter = self.get_filter() @@ -362,13 +377,16 @@ class TableCommand(GridCommand): return sql - def get_primary_keys(self): + def get_primary_keys(self, default_conn=None): """ This function is used to fetch the primary key columns. """ driver = get_driver(PG_DEFAULT_DRIVER) - manager = driver.connection_manager(self.sid) - conn = manager.connection(did=self.did, conn_id=self.conn_id) + if default_conn is None: + manager = driver.connection_manager(self.sid) + conn = manager.connection(did=self.did, conn_id=self.conn_id) + else: + conn = default_conn pk_names = '' primary_keys = OrderedDict() @@ -400,7 +418,11 @@ class TableCommand(GridCommand): def can_filter(self): return True - def save(self, changed_data): + def save(self, + changed_data, + columns_info, + client_primary_key='__temp_PK', + default_conn=None): """ This function is used to save the data into the database. Depending on condition it will either update or insert the @@ -408,10 +430,16 @@ class TableCommand(GridCommand): Args: changed_data: Contains data to be saved + columns_info: + default_conn: + client_primary_key: """ - - manager = get_driver(PG_DEFAULT_DRIVER).connection_manager(self.sid) - conn = manager.connection(did=self.did, conn_id=self.conn_id) + driver = get_driver(PG_DEFAULT_DRIVER) + if default_conn is None: + manager = driver.connection_manager(self.sid) + conn = manager.connection(did=self.did, conn_id=self.conn_id) + else: + conn = default_conn status = False res = None @@ -421,14 +449,6 @@ class TableCommand(GridCommand): list_of_sql = [] _rowid = None - # Replace column positions with names - def set_column_names(data): - new_data = {} - for key in data: - new_data[changed_data['columns'][int(key)]['name']] = data[key] - - return new_data - if conn.connected(): # Start the transaction @@ -443,6 +463,20 @@ class TableCommand(GridCommand): if len(changed_data[of_type]) < 1: continue + column_type = {} + for each_col in columns_info: + if ( + columns_info[each_col]['not_null'] and + not columns_info[each_col][ + 'has_default_val'] + ): + column_data[each_col] = None + column_type[each_col] =\ + columns_info[each_col]['type_name'] + else: + column_type[each_col] = \ + columns_info[each_col]['type_name'] + # For newly added rows if of_type == 'added': @@ -451,37 +485,18 @@ class TableCommand(GridCommand): # no_default_value, set column to blank, instead # of not null which is set by default. column_data = {} - column_type = {} pk_names, primary_keys = self.get_primary_keys() - for each_col in self.columns_info: - if ( - self.columns_info[each_col]['not_null'] and - not self.columns_info[each_col][ - 'has_default_val'] - ): - column_data[each_col] = None - column_type[each_col] =\ - self.columns_info[each_col]['type_name'] - else: - column_type[each_col] = \ - self.columns_info[each_col]['type_name'] - - for each_row in changed_data[of_type]: data = changed_data[of_type][each_row]['data'] # Remove our unique tracking key - data.pop('__temp_PK', None) + data.pop(client_primary_key, None) data.pop('is_row_copied', None) - data = set_column_names(data) - data_type = set_column_names(changed_data[of_type][each_row]['data_type']) - list_of_rowid.append(data.get('__temp_PK')) + list_of_rowid.append(data.get(client_primary_key)) - # Update columns value and data type - # with columns having not_null=False and has - # no default value + # Update columns value with columns having + # not_null=False and has no default value column_data.update(data) - column_type.update(data_type) sql = render_template("/".join([self.sql_path, 'insert.sql']), data_to_be_saved=column_data, @@ -497,15 +512,14 @@ class TableCommand(GridCommand): # For updated rows elif of_type == 'updated': for each_row in changed_data[of_type]: - data = set_column_names(changed_data[of_type][each_row]['data']) - pk = set_column_names(changed_data[of_type][each_row]['primary_keys']) - data_type = set_column_names(changed_data[of_type][each_row]['data_type']) + data = changed_data[of_type][each_row]['data'] + pk = changed_data[of_type][each_row]['primary_keys'] sql = render_template("/".join([self.sql_path, 'update.sql']), data_to_be_saved=data, primary_keys=pk, object_name=self.object_name, nsp_name=self.nsp_name, - data_type=data_type) + data_type=column_type) list_of_sql.append(sql) list_of_rowid.append(data) @@ -519,18 +533,19 @@ class TableCommand(GridCommand): rows_to_delete.append(changed_data[of_type][each_row]) # Fetch the keys for SQL generation if is_first: - # We need to covert dict_keys to normal list in Python3 - # In Python2, it's already a list & We will also fetch column names using index - keys = [ - changed_data['columns'][int(k)]['name'] - for k in list(changed_data[of_type][each_row].keys()) - ] + # We need to covert dict_keys to normal list in + # Python3 + # In Python2, it's already a list & We will also + # fetch column names using index + keys = list(changed_data[of_type][each_row].keys()) + no_of_keys = len(keys) is_first = False # Map index with column name for each row for row in rows_to_delete: for k, v in row.items(): - # Set primary key with label & delete index based mapped key + # Set primary key with label & delete index based + # mapped key try: row[changed_data['columns'][int(k)]['name']] = v except ValueError: @@ -597,7 +612,7 @@ class ViewCommand(GridCommand): # call base class init to fetch the table name super(ViewCommand, self).__init__(**kwargs) - def get_sql(self): + def get_sql(self, default_conn=None): """ This method is used to create a proper SQL query to fetch the data for the specified view @@ -652,7 +667,7 @@ class ForeignTableCommand(GridCommand): # call base class init to fetch the table name super(ForeignTableCommand, self).__init__(**kwargs) - def get_sql(self): + def get_sql(self, default_conn=None): """ This method is used to create a proper SQL query to fetch the data for the specified foreign table @@ -697,7 +712,7 @@ class CatalogCommand(GridCommand): # call base class init to fetch the table name super(CatalogCommand, self).__init__(**kwargs) - def get_sql(self): + def get_sql(self, default_conn=None): """ This method is used to create a proper SQL query to fetch the data for the specified catalog object @@ -722,7 +737,7 @@ class CatalogCommand(GridCommand): return True -class QueryToolCommand(BaseCommand): +class QueryToolCommand(BaseCommand, FetchedRowTracker): """ class QueryToolCommand(BaseCommand) @@ -732,13 +747,15 @@ class QueryToolCommand(BaseCommand): def __init__(self, **kwargs): # call base class init to fetch the table name - super(QueryToolCommand, self).__init__(**kwargs) + + BaseCommand.__init__(self, **kwargs) + FetchedRowTracker.__init__(self, **kwargs) self.conn_id = None self.auto_rollback = False self.auto_commit = True - def get_sql(self): + def get_sql(self, default_conn=None): return None def can_edit(self): diff --git a/web/pgadmin/tools/sqleditor/static/css/sqleditor.css b/web/pgadmin/tools/sqleditor/static/css/sqleditor.css index 7266b34..d8f4881 100644 --- a/web/pgadmin/tools/sqleditor/static/css/sqleditor.css +++ b/web/pgadmin/tools/sqleditor/static/css/sqleditor.css @@ -423,7 +423,7 @@ input.editor-checkbox:focus { /* To highlight all newly inserted rows */ .grid-canvas .new_row { - background: #dff0d7; + background: #dff0d7 !important; } /* To highlight all the updated rows */ @@ -433,7 +433,7 @@ input.editor-checkbox:focus { /* To highlight row at fault */ .grid-canvas .new_row.error, .grid-canvas .updated_row.error { - background: #f2dede; + background: #f2dede !important; } /* Disabled row */ @@ -460,6 +460,11 @@ input.editor-checkbox:focus { background-color: #2C76B4; } +.slick-cell span[data-cell-type="row-header-selector"] { + display: block; + text-align: right; +} + #datagrid div.slick-header.ui-state-default { background: #ffffff; border-bottom: none; diff --git a/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js b/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js index cac2e55..eeb320e 100644 --- a/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js +++ b/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js @@ -463,7 +463,7 @@ define([ - staged_rows: This will hold all the data which user copies/pastes/deletes in grid - deleted: - This will hold all the data which user delets in grid + This will hold all the data which user deletes in grid Events handling: ---------------- @@ -479,34 +479,10 @@ define([ - We are using this event for Copy operation on grid */ - // Get the item column value using a custom 'fieldIdx' column param - get_item_column_value: function (item, column) { - if (column.pos !== undefined) { - return item[column.pos]; - } else { - return null; - } - }, - // This function is responsible to create and render the SlickGrid. - render_grid: function(collection, columns, is_editable) { + render_grid: function(collection, columns, is_editable, client_primary_key, rows_affected) { var self = this; - // returns primary keys - self.handler.get_row_primary_key = function() { - var self = this, - tmp_keys = []; - _.each(self.primary_keys, function(p, idx) { - // For each columns search primary key position - _.each(self.columns, function(c) { - if(c.name == idx) { - tmp_keys.push(c.pos); - } - }); - }); - return tmp_keys; - }; - // This will work as data store and holds all the // inserted/updated/deleted data from grid self.handler.data_store = { @@ -521,8 +497,9 @@ define([ // To store primary keys before they gets changed self.handler.primary_keys_data = {}; - // Add getItemMetadata into handler for later use - self.handler.data_view = collection; + self.client_primary_key = client_primary_key; + + self.client_primary_key_counter = 0; // Remove any existing grid first if (self.handler.slickgrid) { @@ -601,7 +578,12 @@ define([ }); var gridSelector = new GridSelector(); - grid_columns = gridSelector.getColumnDefinitions(grid_columns); + grid_columns = self.grid_columns = gridSelector.getColumnDefinitions(grid_columns); + + if (rows_affected) { + // calculate with for header row column. + grid_columns[0]['width'] = SqlEditorUtils.calculateColumnWidth(rows_affected); + } var grid_options = { editable: true, @@ -609,8 +591,7 @@ define([ enableCellNavigation: true, enableColumnReorder: false, asyncEditorLoading: false, - autoEdit: false, - dataItemColumnValueExtractor: this.get_item_column_value + autoEdit: false }; var $data_grid = self.$el.find('#datagrid'); @@ -618,17 +599,15 @@ define([ var grid_height = $($('#editor-panel').find('.wcFrame')[1]).height() - 35; $data_grid.height(grid_height); - // Add our own custom primary key to keep track of changes - _.each(collection, function(row){ - row['__temp_PK'] = SqlEditorUtils.epicRandomString(15); - }); + var dataView = self.dataView = new Slick.Data.DataView(), + grid = self.grid = new Slick.Grid($data_grid, dataView, grid_columns, grid_options); // Add-on function which allow us to identify the faulty row after insert/update // and apply css accordingly - collection.getItemMetadata = function(i) { - var res = {}, - cssClass = '', - data_store = self.handler.data_store; + + dataView.getItemMetadata = function(i) { + var res = {}, cssClass = '', + data_store = self.handler.data_store; if (_.has(self.handler, 'data_store')) { if (i in data_store.added_index && @@ -651,9 +630,8 @@ define([ cssClass += ' disabled_row'; } return {'cssClasses': cssClass}; - } + }; - var grid = new Slick.Grid($data_grid, collection, grid_columns, grid_options); grid.registerPlugin( new Slick.AutoTooltips({ enableForHeaderCells: false }) ); grid.registerPlugin(new ActiveCellCapture()); grid.setSelectionModel(new XCellSelectionModel()); @@ -665,7 +643,8 @@ define([ columns: columns, grid: grid, selection: grid.getSelectionModel(), - editor: self + editor: self, + client_primary_key: self.client_primary_key }; self.handler.slickgrid = grid; @@ -684,6 +663,42 @@ define([ }); }); + gridSelector.onBeforeGridSelectAll.subscribe(function(e, args) { + if (self.handler.has_more_rows) { + // this will prevent selection un-till we load all data + e.stopImmediatePropagation(); + self.fetch_next_all(function() { + // since we've stopped event propagation we need to + // trigger onGridSelectAll manually with new event data. + gridSelector.onGridSelectAll.notify(args, new Slick.EventData()); + }); + } + }); + + gridSelector.onBeforeGridColumnSelectAll.subscribe(function(e, args) { + if (self.handler.has_more_rows) { + // this will prevent selection un-till we load all data + e.stopImmediatePropagation(); + self.fetch_next_all(function() { + // since we've stopped event propagation we need to + // trigger onGridColumnSelectAll manually with new event data. + gridSelector.onGridColumnSelectAll.notify(args, new Slick.EventData()); + }); + } + }); + + // listen for row count change. + dataView.onRowCountChanged.subscribe(function (e, args) { + grid.updateRowCount(); + grid.render(); + }); + + // listen for rows change. + dataView.onRowsChanged.subscribe(function (e, args) { + grid.invalidateRows(args.rows); + grid.render(); + }); + // Listener function which will be called before user updates existing cell // This will be used to collect primary key for that row grid.onBeforeEditCell.subscribe(function (e, args) { @@ -696,8 +711,8 @@ define([ return false; } - if(self.handler.can_edit && before_data && '__temp_PK' in before_data) { - var _pk = before_data.__temp_PK, + if(self.handler.can_edit && before_data && self.client_primary_key in before_data) { + var _pk = before_data[self.client_primary_key], _keys = self.handler.primary_keys, current_pk = {}, each_pk_key = {}; @@ -709,22 +724,34 @@ define([ // Fetch primary keys for the row before they gets modified var _columns = self.handler.columns; _.each(_keys, function(value, key) { - pos = _.where(_columns, {name: key})[0]['pos'] - current_pk[pos] = before_data[pos]; + current_pk[key] = before_data[key]; }); // Place it in main variable for later use self.handler.primary_keys_data[_pk] = current_pk } }); - grid.onKeyDown.subscribe(handleQueryOutputKeyboardEvent); + grid.onKeyDown.subscribe(function(event, args) { + var KEY_A = 65; + var modifiedKey = event.keyCode; + var isModifierDown = event.ctrlKey || event.metaKey; + // Intercept Ctrl/Cmd + A key board event. + // As we might want to load all rows before selecting all. + if (isModifierDown && modifiedKey == KEY_A && self.handler.has_more_rows) { + self.fetch_next_all(function() { + handleQueryOutputKeyboardEvent(event, args); + }); + } else { + handleQueryOutputKeyboardEvent(event, args); + } + }); // Listener function which will be called when user updates existing rows grid.onCellChange.subscribe(function (e, args) { // self.handler.data_store.updated will holds all the updated data - var changed_column = args.grid.getColumns()[args.cell].pos, // Current field pos + var changed_column = args.grid.getColumns()[args.cell].field, updated_data = args.item[changed_column], // New value for current field - _pk = args.item.__temp_PK || null, // Unique key to identify row + _pk = args.item[self.client_primary_key] || null, // Unique key to identify row column_data = {}, _type; @@ -734,11 +761,16 @@ define([ // so that cell edit is enabled for that row. var grid = args.grid, row_data = grid.getDataItem(args.row), - p_keys_list = _.pick( - row_data, self.handler.get_row_primary_key() - ), - is_primary_key = Object.keys(p_keys_list).length ? - p_keys_list[0] : undefined; + is_primary_key = _.all( + _.values( + _.pick( + row_data, self.primary_keys + ) + ), + function(val) { + return val != undefined + } + ); // temp_new_rows is available only for view data. if (is_primary_key && self.handler.temp_new_rows) { @@ -758,7 +790,6 @@ define([ column_data); //Find type for current column self.handler.data_store.added[_pk]['err'] = false - self.handler.data_store.added[_pk]['data_type'][changed_column] = _.where(this.columns, {pos: changed_column})[0]['type']; // Check if it is updated data from existing rows? } else if(_pk in self.handler.data_store.updated) { _.extend( @@ -766,9 +797,6 @@ define([ column_data ); self.handler.data_store.updated[_pk]['err'] = false - - //Find type for current column - self.handler.data_store.updated[_pk]['data_type'][changed_column] = _.where(this.columns, {pos: changed_column})[0]['type']; } else { // First updated data for this primary key self.handler.data_store.updated[_pk] = { @@ -776,32 +804,19 @@ define([ 'primary_keys': self.handler.primary_keys_data[_pk] }; self.handler.data_store.updated_index[args.row] = _pk; - // Find & add column data type for current changed column - var temp = {}; - temp[changed_column] = _.where(this.columns, {pos: changed_column})[0]['type']; - self.handler.data_store.updated[_pk]['data_type'] = temp; } } // Enable save button $("#btn-save").prop('disabled', false); }.bind(editor_data)); - grid.addBlankRow = function() { - // Add a blank row in the end of grid - this.setData(this.getData(), true); - this.updateRowCount(); - this.invalidateAllRows(); - this.render(); - }; - // Listener function which will be called when user adds new rows grid.onAddNewRow.subscribe(function (e, args) { // self.handler.data_store.added will holds all the newly added rows/data - var _key = SqlEditorUtils.epicRandomString(10), - column = args.column, - item = args.item, - data_length = this.grid.getDataLength(), - new_collection = args.grid.getData(); + var column = args.column, + item = args.item, data_length = this.grid.getDataLength(), + _key = (self.client_primary_key_counter++).toString(), + dataView = this.grid.getData(); // Add new row in list to keep track of it if (_.isUndefined(item[0])) { @@ -810,29 +825,32 @@ define([ // If copied item has already primary key, use it. if(item) { - item.__temp_PK = _key; + item[self.client_primary_key] = _key; } - new_collection.push(item); + dataView.addItem(item); self.handler.data_store.added[_key] = {'err': false, 'data': item}; self.handler.data_store.added_index[data_length] = _key; // Fetch data type & add it for the column var temp = {}; - temp[column.pos] = _.where(this.columns, {pos: column.pos})[0]['type']; - self.handler.data_store.added[_key]['data_type'] = temp; - grid.invalidateRows([new_collection.length - 1]); + temp[column.name] = _.where(this.columns, {pos: column.pos})[0]['type']; grid.updateRowCount(); grid.render(); - // Add a blank row after add row - if (!args.is_copy_row) { - grid.addBlankRow(); - } - // Enable save button $("#btn-save").prop('disabled', false); }.bind(editor_data)); + // Listen grid viewportChanged event to load next chunk of data. + grid.onViewportChanged.subscribe(function(e, args) { + var rendered_range = args.grid.getRenderedRange(), + data_len = args.grid.getDataLength(); + // start fetching next batch of records before reaching to bottom. + if (self.handler.has_more_rows && !self.handler.fetching_rows && rendered_range.bottom > data_len - 100) { + // fetch asynchronous + setTimeout(self.fetch_next.bind(self)); + } + }) // Resize SlickGrid when window resize $( window ).resize( function() { // Resize grid only when 'Data Output' panel is visible. @@ -855,6 +873,87 @@ define([ if(self.data_output_panel.isVisible()) self.grid_resize(grid); }); + + for (var i = 0; i < collection.length; i++) { + // Convert to dict from 2darray + var item = {}; + for (var j = 1; j < grid_columns.length; j++) { + item[grid_columns[j]['field']] = collection[i][grid_columns[j]['pos']] + } + + item[self.client_primary_key] = (self.client_primary_key_counter++).toString(); + collection[i] = item; + } + dataView.setItems(collection, self.client_primary_key); + }, + fetch_next_all(cb) { + this.fetch_next(true, cb); + }, + fetch_next: function(fetch_all, cb) { + var self = this, url = ''; + + // This will prevent fetch operation if previous fetch operation is + // already in progress. + self.handler.fetching_rows = true; + + $("#btn-flash").prop('disabled', true); + + if (fetch_all) { + self.handler.trigger( + 'pgadmin-sqleditor:loading-icon:show', + gettext('Fetching all records...') + ); + url = url_for('sqleditor.fetch_all', {'trans_id': self.transId, 'fetch_all': 1}); + } else { + url = url = url_for('sqleditor.fetch', {'trans_id': self.transId}); + } + + $.ajax({ + url: url, + method: 'GET', + success: function(res) { + self.handler.has_more_rows = res.data.has_more_rows; + $("#btn-flash").prop('disabled', false); + self.handler.trigger('pgadmin-sqleditor:loading-icon:hide'); + self.update_grid_data(res.data.result); + self.handler.fetching_rows = false; + if (typeof cb == "function") { + cb(); + } + }, + error: function(e) { + $("#btn-flash").prop('disabled', false); + self.handler.trigger('pgadmin-sqleditor:loading-icon:hide'); + self.handler.has_more_rows = false; + self.handler.fetching_rows = false; + if (typeof cb == "function") { + cb(); + } + if (e.readyState == 0) { + self.update_msg_history(false, + gettext('Not connected to the server or the connection to the server has been closed.') + ); + return; + } + } + }); + }, + + update_grid_data: function(data) { + this.dataView.beginUpdate(); + + for (var i = 0; i < data.length; i++) { + // Convert 2darray to dict. + var item = {}; + for (var j = 1; j < this.grid_columns.length; j++) { + item[this.grid_columns[j]['field']] = data[i][this.grid_columns[j]['pos']] + } + + item[this.client_primary_key] = (this.client_primary_key_counter++).toString(); + this.dataView.addItem(item); + } + + this.dataView.endUpdate(); }, /* This function is responsible to render output grid */ @@ -1396,6 +1495,8 @@ define([ self.explain_buffers = false; self.explain_timing = false; self.is_new_browser_tab = is_new_browser_tab; + self.has_more_rows = false; + self.fetching_rows = false; // We do not allow to call the start multiple times. if (self.gridView) @@ -1503,6 +1604,8 @@ define([ self.rows_to_disable = new Array(); // Temporarily hold new rows added self.temp_new_rows = new Array(); + self.has_more_rows = false; + self.fetching_rows = false; self.trigger( 'pgadmin-sqleditor:loading-icon:show', @@ -1581,45 +1684,14 @@ define([ }); }, - // This function makes the ajax call to fetch columns for last async query, - get_columns: function(poll_result) { - var self = this; - // Check the flag and decide if we need to fetch columns from server - // or use the columns data stored locally from previous call? - if (self.FETCH_COLUMNS_FROM_SERVER) { - $.ajax({ - url: url_for('sqleditor.get_columns', {'trans_id': self.transId}), - method: 'GET', - success: function(res) { - poll_result.colinfo = res.data.columns; - poll_result.primary_keys = res.data.primary_keys; - self.call_render_after_poll(poll_result); - // Set a flag to get columns to false & set the value for future use - self.FETCH_COLUMNS_FROM_SERVER = false; - self.COLUMNS_DATA = res; - }, - error: function(e) { - var msg = e.responseText; - if (e.responseJSON != undefined && e.responseJSON.errormsg != undefined) - msg = e.responseJSON.errormsg; - alertify.error(msg, 5); - } - }); - } else { - // Use the previously saved columns data - poll_result.colinfo = self.COLUMNS_DATA.data.columns; - poll_result.primary_keys = self.COLUMNS_DATA.data.primary_keys; - self.call_render_after_poll(poll_result); - } - }, - // This is a wrapper to call _render function // We need this because we have separated columns route & result route // We need to combine both result here in wrapper before rendering grid call_render_after_poll: function(res) { var self = this; self.query_end_time = new Date(); - self.rows_affected = res.rows_affected; + self.rows_affected = res.rows_affected, + self.has_more_rows = res.has_more_rows; /* If no column information is available it means query runs successfully with no result to display. In this @@ -1668,7 +1740,8 @@ define([ 'pgadmin-sqleditor:loading-icon:message', gettext("Loading data from the database server and rendering...") ); - self.get_columns(res.data); + + self.call_render_after_poll(res.data); } else if (res.data.status === 'Busy') { // If status is Busy then poll the result by recursive call to the poll function @@ -1728,6 +1801,7 @@ define([ var self = this; self.colinfo = data.col_info; self.primary_keys = data.primary_keys; + self.client_primary_key = data.client_primary_key; self.cell_selected = false; self.selected_model = null; self.changedModels = []; @@ -1780,7 +1854,7 @@ define([ self.total_time = self.get_query_run_time(self.query_start_time, self.query_end_time); self.update_msg_history(true, "", false); var msg1 = S(gettext("Total query runtime: %s.")).sprintf(self.total_time).value(); - var msg2 = S(gettext("%s rows retrieved.")).sprintf(self.rows_affected).value(); + var msg2 = S(gettext("%s rows affected.")).sprintf(self.rows_affected).value(); // Display the notifier if the timeout is set to >= 0 if (self.info_notifier_timeout >= 0) { @@ -1816,7 +1890,8 @@ define([ setTimeout( function() { self.gridView.render_grid( - explain_data_array, self.columns, self.can_edit + explain_data_array, self.columns, self.can_edit, + self.client_primary_key ); // Make sure - the 'Explain' panel is visible, before - we // start rendering the grid. @@ -1832,7 +1907,8 @@ define([ self.gridView.data_output_panel.focus(); setTimeout( function() { - self.gridView.render_grid(data.result, self.columns, self.can_edit); + self.gridView.render_grid(data.result, self.columns, + self.can_edit, self.client_primary_key, data.rows_affected); }, 10 ); } @@ -1840,138 +1916,114 @@ define([ // Hide the loading icon self.trigger('pgadmin-sqleditor:loading-icon:hide'); $("#btn-flash").prop('disabled', false); - }.bind(self), - function() { - this.trigger('pgadmin-sqleditor:loading-icon:hide'); - $("#btn-flash").prop('disabled', false); }.bind(self) ); }, // This function creates the columns as required by the backgrid - _fetch_column_metadata: function(data, cb, _fail) { + _fetch_column_metadata: function(data, cb) { var colinfo = data.colinfo, primary_keys = data.primary_keys, result = data.result, columns = [], self = this; + // Store pg_types in an array + var pg_types = new Array(); + _.each(data.types, function(r) { + pg_types[r.oid] = [r.typname]; + }); - self.trigger( - 'pgadmin-sqleditor:loading-icon:message', - gettext("Retrieving information about the columns returned...") - ); - - // Make ajax call to fetch the pg types to map numeric data type - $.ajax({ - url: url_for('sqleditor.fetch_types', {'trans_id': self.transId}), - method: 'GET', - success: function(res) { - if (res.data.status) { - // Store pg_types in an array - var pg_types = new Array(); - _.each(res.data.result.rows, function(r) { - pg_types[r.oid] = [r.typname]; - }); - - // Create columns required by backgrid to render - _.each(colinfo, function(c) { - var is_primary_key = false; + // Create columns required by slick grid to render + _.each(colinfo, function(c) { + var is_primary_key = false; - // Check whether table have primary key - if (_.size(primary_keys) > 0) { - _.each(primary_keys, function (value, key) { - if (key === c.name) - is_primary_key = true; - }); - } + // Check whether table have primary key + if (_.size(primary_keys) > 0) { + _.each(primary_keys, function (value, key) { + if (key === c.name) + is_primary_key = true; + }); + } - // To show column label and data type in multiline, - // The elements should be put inside the div. - // Create column label and type. - var col_type = '', - column_label = '', - col_cell; - var type = pg_types[c.type_code] ? - pg_types[c.type_code][0] : - // This is the case where user might - // have use casting so we will use type - // returned by cast function - pg_types[pg_types.length - 1][0] ? - pg_types[pg_types.length - 1][0] : 'unknown'; - - if (!is_primary_key) - col_type += ' ' + type; - else - col_type += ' [PK] ' + type; + // To show column label and data type in multiline, + // The elements should be put inside the div. + // Create column label and type. + var col_type = column_label = ''; + var type = pg_types[c.type_code] ? + pg_types[c.type_code][0] : + // This is the case where user might + // have use casting so we will use type + // returned by cast function + pg_types[pg_types.length - 1][0] ? + pg_types[pg_types.length - 1][0] : 'unknown'; + + if (!is_primary_key) + col_type += ' ' + type; + else + col_type += ' [PK] ' + type; - if (c.precision && c.precision >= 0 && c.precision != 65535) { - col_type += ' (' + c.precision; - col_type += c.scale && c.scale != 65535 ? - ',' + c.scale + ')': - ')'; - } + if (c.precision && c.precision >= 0 && c.precision != 65535) { + col_type += ' (' + c.precision; + col_type += c.scale && c.scale != 65535 ? + ',' + c.scale + ')': + ')'; + } - // Identify cell type of column. - switch(type) { - case "json": - case "json[]": - case "jsonb": - case "jsonb[]": - col_cell = 'Json'; - break; - case "smallint": - case "integer": - case "bigint": - case "decimal": - case "numeric": - case "real": - case "double precision": - col_cell = 'number'; - break; - case "boolean": - col_cell = 'boolean'; - break; - case "character": - case "character[]": - case "character varying": - case "character varying[]": - if (c.internal_size && c.internal_size >= 0 && c.internal_size != 65535) { - // Update column type to display length on column header - col_type += ' (' + c.internal_size + ')'; - } - col_cell = 'string'; - break; - default: - col_cell = 'string'; - } + // Identify cell type of column. + switch(type) { + case "json": + case "json[]": + case "jsonb": + case "jsonb[]": + col_cell = 'Json'; + break; + case "smallint": + case "integer": + case "bigint": + case "decimal": + case "numeric": + case "real": + case "double precision": + col_cell = 'number'; + break; + case "boolean": + col_cell = 'boolean'; + break; + case "character": + case "character[]": + case "character varying": + case "character varying[]": + if (c.internal_size && c.internal_size >= 0 && c.internal_size != 65535) { + // Update column type to display length on column header + col_type += ' (' + c.internal_size + ')'; + } + col_cell = 'string'; + break; + default: + col_cell = 'string'; + } - column_label = c.display_name + '<br>' + col_type; - - var col = { - 'name': c.name, - 'pos': c.pos, - 'label': column_label, - 'cell': col_cell, - 'can_edit': self.can_edit, - 'type': type, - 'display_name': c.display_name, - 'column_type': col_type, - 'not_null': c.not_null, - 'has_default_val': c.has_default_val - }; - columns.push(col); - }); - } - else { - alertify.alert('Fetching Type Error', res.data.result); - } - self.columns = columns; - if (cb && typeof(cb) == 'function') { - cb(); - } - }, - fail: _fail + column_label = c.display_name + '<br>' + col_type; + + var col = { + 'name': c.name, + 'display_name': c.display_name, + 'column_type': col_type, + 'pos': c.pos, + 'label': column_label, + 'cell': col_cell, + 'can_edit': self.can_edit, + 'type': type, + 'not_null': c.not_null, + 'has_default_val': c.has_default_val + }; + columns.push(col); }); + + self.columns = columns; + if (cb && typeof(cb) == 'function') { + cb(); + } }, // This function is used to raise appropriate message. @@ -1982,22 +2034,21 @@ define([ self.gridView.messages_panel.focus(); - if (self.is_query_tool) { - if (clear_grid) { - // Delete grid - if (self.gridView.handler.slickgrid) { - self.gridView.handler.slickgrid.destroy(); - - } - // Misc cleaning - self.columns = undefined; - self.collection = undefined; + if (clear_grid) { + // Delete grid + if (self.gridView.handler.slickgrid) { + self.gridView.handler.slickgrid.destroy(); - $('.sql-editor-message').text(msg); - } else { - $('.sql-editor-message').append(msg); } + // Misc cleaning + self.columns = undefined; + self.collection = undefined; + + $('.sql-editor-message').text(msg); + } else { + $('.sql-editor-message').append(msg); } + // Scroll automatically when msgs appends to element setTimeout(function(){ $(".sql-editor-message").scrollTop($(".sql-editor-message")[0].scrollHeight);; @@ -2052,7 +2103,7 @@ define([ rows_to_delete: function(data) { var self = this, - tmp_keys = self.get_row_primary_key.call(self); + tmp_keys = self.primary_keys; // re-calculate rows with no primary keys self.temp_new_rows = []; @@ -2065,7 +2116,6 @@ define([ self.temp_new_rows.push(idx); } }); - data.getItemMetadata = self.data_view.getItemMetadata; self.rows_to_disable = _.clone(self.temp_new_rows); }, @@ -2076,69 +2126,73 @@ define([ is_added = _.size(self.data_store.added), is_updated = _.size(self.data_store.updated); - // Remove newly added rows from staged rows as we don't want to send them on server - if(is_added) { - _.each(self.data_store.added, function(val, key) { - if(key in self.data_store.staged_rows) { - // Remove the row from data store so that we do not send it on server - deleted_keys.push(key); - delete self.data_store.staged_rows[key]; - delete self.data_store.added[key] - } - }); + // Remove newly added rows from staged rows as we don't want to send them on server + if(is_added) { + _.each(self.data_store.added, function(val, key) { + if(key in self.data_store.staged_rows) { + // Remove the row from data store so that we do not send it on server + deleted_keys.push(key); + delete self.data_store.staged_rows[key]; + delete self.data_store.added[key]; + delete self.data_store.added_index[key]; } - - // If only newly rows to delete and no data is there to send on server - // then just re-render the grid - if(_.size(self.data_store.staged_rows) == 0) { - var grid = self.slickgrid, data = grid.getData(), idx = 0; - if(deleted_keys.length){ - // Remove new rows from grid data using deleted keys - data = _.reject(data, function(d){ - return (d && _.indexOf(deleted_keys, d.__temp_PK) > -1) - }); - } - self.rows_to_delete.apply(self, [data]); - grid.resetActiveCell(); - grid.setData(data, true); - grid.setSelectedRows([]); - grid.invalidate(); - // Nothing to copy or delete here - $("#btn-delete-row").prop('disabled', true); - $("#btn-copy-row").prop('disabled', true); - if(_.size(self.data_store.added) || is_updated) { - // Do not disable save button if there are - // any other changes present in grid data - $("#btn-save").prop('disabled', false); - } else { - $("#btn-save").prop('disabled', true); - } - alertify.success(gettext("Row(s) deleted")); + }); + } + // If only newly rows to delete and no data is there to send on server + // then just re-render the grid + if(_.size(self.data_store.staged_rows) == 0) { + var grid = self.slickgrid, + dataView = grid.getData(), + data = dataView.getItems(), + idx = 0; + + grid.resetActiveCell(); + + dataView.beginUpdate(); + for (var i = 0; i < deleted_keys.length; i++) { + dataView.deleteItem(deleted_keys[i]); + } + dataView.endUpdate(); + self.rows_to_delete.apply(self, [dataView.getItems()]); + grid.resetActiveCell(); + grid.setSelectedRows([]); + grid.invalidate(); + + // Nothing to copy or delete here + $("#btn-delete-row").prop('disabled', true); + $("#btn-copy-row").prop('disabled', true); + if(_.size(self.data_store.added) || is_updated) { + // Do not disable save button if there are + // any other changes present in grid data + $("#btn-save").prop('disabled', false); } else { - // There are other data to needs to be updated on server - if(is_updated) { - alertify.alert(gettext("Operation failed"), + $("#btn-save").prop('disabled', true); + } + alertify.success(gettext("Row(s) deleted")); + } else { + // There are other data to needs to be updated on server + if(is_updated) { + alertify.alert(gettext("Operation failed"), gettext("There are unsaved changes in grid, Please save them first to avoid inconsistency in data") ); - return; - } - alertify.confirm(gettext("Delete Row(s)"), + return; + } + alertify.confirm(gettext("Delete Row(s)"), gettext("Are you sure you wish to delete selected row(s)?"), - function() { - $("#btn-delete-row").prop('disabled', true); - $("#btn-copy-row").prop('disabled', true); - // Change the state - self.data_store.deleted = self.data_store.staged_rows; - self.data_store.staged_rows = {}; - // Save the changes on server - self._save(); - }, - function() { - // Do nothing as user canceled the operation. - } - ).set('labels', {ok:'Yes', cancel:'No'}); + function() { + $("#btn-delete-row").prop('disabled', true); + $("#btn-copy-row").prop('disabled', true); + // Change the state + self.data_store.deleted = self.data_store.staged_rows; + self.data_store.staged_rows = {}; + // Save the changes on server + self._save(); + }, + function() { + // Do nothing as user canceled the operation. } - + ).set('labels', {ok: gettext("Yes"), cancel:gettext("No")}); + } }, /* This function will fetch the list of changed models and make @@ -2200,7 +2254,9 @@ define([ data: JSON.stringify(req_data), success: function(res) { var grid = self.slickgrid, - data = grid.getData(); + dataView = grid.getData(), + data_length = dataView.getLength(), + data = []; if (res.data.status) { // Remove flag is_row_copied from copied rows _.each(data, function(row, idx) { @@ -2217,22 +2273,20 @@ define([ // Remove deleted rows from client as well if(is_deleted) { var rows = grid.getSelectedRows(); - /* In JavaScript sorting by default is lexical, - * To make sorting numerical we need to pass function - * After that we will Reverse the order of sorted array - * so that when we remove it does not affect array index - */ - if(data.length == rows.length) { + if(data_length == rows.length) { // This means all the rows are selected, clear all data data = []; + dataView.setItems(data, self.client_primary_key); } else { - rows = rows.sort(function(a,b){return a - b}).reverse(); - rows.forEach(function(idx) { - data.splice(idx, 1); - }); + dataView.beginUpdate(); + for (var i = 0; i < rows.length; i++) { + item = grid.getDataItem(rows[i]); + data.push(item); + dataView.deleteItem(item[self.client_primary_key]); + } + dataView.endUpdate(); } self.rows_to_delete.apply(self, [data]); - grid.setData(data, true); grid.setSelectedRows([]); } @@ -2245,6 +2299,7 @@ define([ self.rows_to_disable = _.clone(self.temp_new_rows); } + grid.setSelectedRows([]); // Reset data store self.data_store = { 'added': {}, @@ -2265,7 +2320,7 @@ define([ $('.sql-editor-message').text(res.data.result); var err_msg = S(gettext("%s.")).sprintf(res.data.result).value(); alertify.notify(err_msg, 'error', 20); - + grid.setSelectedRows([]); // To highlight the row at fault if(_.has(res.data, '_rowid') && (!_.isUndefined(res.data._rowid)|| !_.isNull(res.data._rowid))) { @@ -2321,14 +2376,21 @@ define([ // Find index of row at fault from grid data _find_rowindex: function(rowid) { - var self = this; - var grid = self.slickgrid, - data = grid.getData(), _rowid, count = 0, _idx = -1; + var self = this, + grid = self.slickgrid, + dataView = grid.getData(), + data = dataView.getItems(), + _rowid, + count = 0, + _idx = -1; + // If _rowid is object then it's update/delete operation if(_.isObject(rowid)) { _rowid = rowid; - } else if (_.isString(rowid)) { // Insert opration - _rowid = { '__temp_PK': rowid }; + } else if (_.isString(rowid)) { // Insert operation + var rowid = {}; + rowid[self.client_primary_key]= rowid; + _rowid = rowid; } else { // Something is wrong with unique id return _idx; @@ -2550,11 +2612,6 @@ define([ // This function will set the required flag for polling response data _init_polling_flags: function() { var self = this; - // Set a flag to get columns - self.FETCH_COLUMNS_FROM_SERVER = true; - // We will set columns data in this variable for future use once we fetch it - // from server - self.COLUMNS_DATA = {}; // To get a timeout for polling fallback timer in seconds in // regards to elapsed time @@ -2650,7 +2707,7 @@ define([ return; // Add column position and it's value to data - data[column_info.field] = _values[column_info.pos] || ''; + data[column_info.field] = _values[column_info.field] || ''; self.trigger( 'pgadmin-sqleditor:loading-icon:show', @@ -2720,7 +2777,7 @@ define([ return; // Add column position and it's value to data - data[column_info.field] = _values[column_info.pos] || ''; + data[column_info.field] = _values[column_info.field] || ''; self.trigger( 'pgadmin-sqleditor:loading-icon:show', @@ -2883,16 +2940,18 @@ define([ _paste_row: function() { var self = this, col_info = {}, grid = self.slickgrid, - data = grid.getData(), - count = Object.keys(data).length-1; - - var rows = grid.getSelectedRows().sort( + dataView = grid.getData(), + data = dataView.getItems(), + count = dataView.getLength(), + rows = grid.getSelectedRows().sort( function (a, b) { return a - b; } ), - rows = rows.length == 0 ? self.last_copied_rows : rows, copied_rows = rows.map(function (rowIndex) { return data[rowIndex]; }); + + rows = rows.length == 0 ? self.last_copied_rows : rows + self.last_copied_rows = rows; // If there are rows to paste? @@ -2925,22 +2984,22 @@ define([ // Add index of copied row into temp_new_rows // Trigger grid.onAddNewRow when a row is copied // Reset selection + + dataView.beginUpdate(); _.each(copied_rows, function(row) { - var new_row = arr_to_object(row); + var new_row = arr_to_object(row), + _key = (self.gridView.client_primary_key_counter++).toString(); new_row.is_row_copied = true; - row = new_row; self.temp_new_rows.push(count); - grid.onAddNewRow.notify( - { item: new_row, - column: self.columns[0], - grid: grid, - is_copy_row: true - } - ) + new_row[self.client_primary_key] = _key; + dataView.addItem(new_row); + self.data_store.added[_key] = {'err': false, 'data': new_row}; + self.data_store.added_index[count] = _key; count++; }); - // Add a blank row after copy/paste row - grid.addBlankRow(); + dataView.endUpdate(); + grid.updateRowCount(); + grid.render(); grid.setSelectedRows([]); } }, @@ -3012,6 +3071,9 @@ define([ sql = '', history_msg = ''; + self.has_more_rows = false; + self.fetching_rows = false; + /* If code is selected in the code mirror then execute * the selected part else execute the complete code. */ diff --git a/web/pgadmin/utils/driver/abstract.py b/web/pgadmin/utils/driver/abstract.py index 9b2363c..7db3e37 100644 --- a/web/pgadmin/utils/driver/abstract.py +++ b/web/pgadmin/utils/driver/abstract.py @@ -101,6 +101,12 @@ class BaseConnection(object): - Implement this method to execute the given query and returns the result as an array of dict (column name -> value) format. + * def async_fetchmany_2darray(records=-1, formatted_exception_msg=False): + - Implement this method to retrieve result of asynchronous connection and + polling with no_result flag set to True. + This returns the result as a 2 dimensional array. + If records is -1 then fetchmany will behave as fetchall. + * connected() - Implement this method to get the status of the connection. It should return True for connected, otherwise False @@ -133,7 +139,7 @@ class BaseConnection(object): - Implement this method to wait for asynchronous connection with timeout. This must be a non blocking call. - * poll(formatted_exception_msg) + * poll(formatted_exception_msg, no_result) - Implement this method to poll the data of query running on asynchronous connection. @@ -180,6 +186,10 @@ class BaseConnection(object): pass @abstractmethod + def async_fetchmany_2darray(self, records=-1, formatted_exception_msg=False): + pass + + @abstractmethod def connected(self): pass @@ -208,7 +218,7 @@ class BaseConnection(object): pass @abstractmethod - def poll(self, formatted_exception_msg=True): + def poll(self, formatted_exception_msg=True, no_result=False): pass @abstractmethod diff --git a/web/pgadmin/utils/driver/psycopg2/__init__.py b/web/pgadmin/utils/driver/psycopg2/__init__.py index 16342d4..8a20521 100644 --- a/web/pgadmin/utils/driver/psycopg2/__init__.py +++ b/web/pgadmin/utils/driver/psycopg2/__init__.py @@ -1079,6 +1079,55 @@ Failed to execute query (execute_void) for the server #{server_id} - {conn_id} return True, {'columns': columns, 'rows': rows} + def async_fetchmany_2darray(self, records=2000, formatted_exception_msg=False): + """ + User should poll and check if status is ASYNC_OK before calling this + function + Args: + records: no of records to fetch. use -1 to fetchall. + formatted_exception_msg: + + Returns: + + """ + cur = self.__async_cursor + if not cur: + return False, gettext( + "Cursor could not be found for the async connection." + ) + + if self.conn.isexecuting(): + return False, gettext( + "Asynchronous query execution/operation underway." + ) + + if self.row_count > 0: + result = [] + # For DDL operation, we may not have result. + # + # Because - there is not direct way to differentiate DML and + # DDL operations, we need to rely on exception to figure + # that out at the moment. + try: + if records == -1: + res = cur.fetchall() + else: + res = cur.fetchmany(records) + for row in res: + new_row = [] + for col in self.column_info: + new_row.append(row[col['name']]) + result.append(new_row) + except psycopg2.ProgrammingError as e: + result = None + else: + # User performed operation which dose not produce record/s as + # result. + # for eg. DDL operations. + return True, None + + return True, result + def connected(self): if self.conn: if not self.conn.closed: @@ -1226,7 +1275,7 @@ Failed to reset the connection to the server due to following error: "poll() returned %s from _wait_timeout function" % state ) - def poll(self, formatted_exception_msg=False): + def poll(self, formatted_exception_msg=False, no_result=False): """ This function is a wrapper around connection's poll function. It internally uses the _wait_timeout method to poll the @@ -1236,6 +1285,7 @@ Failed to reset the connection to the server due to following error: Args: formatted_exception_msg: if True then function return the formatted exception message, otherwise error string. + no_result: If True then only poll status will be returned. """ cur = self.__async_cursor @@ -1291,23 +1341,23 @@ Failed to reset the connection to the server due to following error: pos += 1 self.row_count = cur.rowcount - - if cur.rowcount > 0: - result = [] - # For DDL operation, we may not have result. - # - # Because - there is not direct way to differentiate DML and - # DDL operations, we need to rely on exception to figure that - # out at the moment. - try: - for row in cur: - new_row = [] - for col in self.column_info: - new_row.append(row[col['name']]) - result.append(new_row) - - except psycopg2.ProgrammingError: - result = None + if not no_result: + if cur.rowcount > 0: + result = [] + # For DDL operation, we may not have result. + # + # Because - there is not direct way to differentiate DML and + # DDL operations, we need to rely on exception to figure + # that out at the moment. + try: + for row in cur: + new_row = [] + for col in self.column_info: + new_row.append(row[col['name']]) + result.append(new_row) + + except psycopg2.ProgrammingError: + result = None return status, result diff --git a/web/regression/feature_utils/pgadmin_page.py b/web/regression/feature_utils/pgadmin_page.py index c61663a..02c35ed 100644 --- a/web/regression/feature_utils/pgadmin_page.py +++ b/web/regression/feature_utils/pgadmin_page.py @@ -85,7 +85,7 @@ class PgadminPage: if 'menu-item' == str(menu_item.get_attribute('class')): break - time.sleep(0.1) + time.sleep(0.1) else: assert False, "'Tools -> Query Tool' menu did not enable." @@ -144,7 +144,6 @@ class PgadminPage: except WebDriverException: return - def find_by_xpath(self, xpath): return self.wait_for_element(lambda driver: driver.find_element_by_xpath(xpath)) @@ -251,6 +250,20 @@ class PgadminPage: self._wait_for("spinner to disappear", spinner_has_disappeared) + def wait_for_query_tool_loading_indicator_to_disappear(self): + def spinner_has_disappeared(driver): + try: + driver.find_element_by_xpath( + "//*[@id='fetching_data' and @class='hide']" + ) + return False + except NoSuchElementException: + # wait for loading indicator disappear animation to complete. + time.sleep(0.5) + return True + + self._wait_for("spinner to disappear", spinner_has_disappeared) + def wait_for_app(self): def page_shows_app(driver): if driver.title == self.app_config.APP_NAME: @@ -266,19 +279,3 @@ class PgadminPage: timeout = self.timeout return WebDriverWait(self.driver, timeout, 0.01).until(condition_met_function, "Timed out waiting for " + waiting_for_message) - - def wait_for_element_to_stale(self, xpath): - # Reference: http://www.obeythetestinggoat.com/ - # how-to-get-selenium-to-wait-for-page-load-after-a-click.html - el = self.driver.find_element_by_xpath(xpath) - - def element_has_gone_stale(driver): - try: - # poll an arbitrary element - el.find_elements_by_id('element-dont-exist') - return False - except StaleElementReferenceException: - return True - - self._wait_for("element to attach to the page document", - element_has_gone_stale) diff --git a/web/regression/javascript/selection/copy_data_spec.js b/web/regression/javascript/selection/copy_data_spec.js index 9e05e84..2b623b3 100644 --- a/web/regression/javascript/selection/copy_data_spec.js +++ b/web/regression/javascript/selection/copy_data_spec.js @@ -16,16 +16,16 @@ import clipboard from '../../../pgadmin/static/js/selection/clipboard'; import copyData from '../../../pgadmin/static/js/selection/copy_data'; import RangeSelectionHelper from 'sources/selection/range_selection_helper'; import XCellSelectionModel from 'sources/selection/xcell_selection_model'; - describe('copyData', function () { var grid, sqlEditor, gridContainer, buttonPasteRow; var SlickGrid; beforeEach(function () { SlickGrid = Slick.Grid; - var data = [[1, 'leopord', '12'], - [2, 'lion', '13'], - [3, 'puma', '9']]; + var data = [{'id': 1, 'brand':'leopord', 'size':'12', '__temp_PK': '123'}, + {'id': 2, 'brand':'lion', 'size':'13', '__temp_PK': '456'}, + {'id': 3, 'brand':'puma', 'size':'9', '__temp_PK': '789'}], + dataView = new Slick.Data.DataView(); var columns = [ { @@ -37,6 +37,7 @@ describe('copyData', function () { }, { name: 'id', + field: 'id', pos: 0, label: 'id<br> numeric', cell: 'number', @@ -44,6 +45,7 @@ describe('copyData', function () { type: 'numeric', }, { name: 'brand', + field: 'brand', pos: 1, label: 'flavor<br> character varying', cell: 'string', @@ -51,24 +53,26 @@ describe('copyData', function () { type: 'character varying', }, { name: 'size', + field: 'size', pos: 2, label: 'size<br> numeric', cell: 'number', can_edit: false, type: 'numeric', }, - ] - ; - gridContainer = $('<div id=\'grid\'></div>'); + ]; + gridContainer = $('<div id="grid"></div>'); $('body').append(gridContainer); - buttonPasteRow = $('<button id=\'btn-paste-row\' disabled></button>'); + buttonPasteRow = $('<button id="btn-paste-row" disabled></button>'); $('body').append(buttonPasteRow); - grid = new SlickGrid('#grid', data, columns, {}); + grid = new SlickGrid('#grid', dataView, columns, {}); + dataView.setItems(data, '__temp_PK'); grid.setSelectionModel(new XCellSelectionModel()); sqlEditor = {slickgrid: grid}; }); - afterEach(function () { + afterEach(function() { + grid.destroy(); gridContainer.remove(); buttonPasteRow.remove(); }); diff --git a/web/regression/javascript/selection/range_boundary_navigator_spec.js b/web/regression/javascript/selection/range_boundary_navigator_spec.js index 1de6d81..2d56b05 100644 --- a/web/regression/javascript/selection/range_boundary_navigator_spec.js +++ b/web/regression/javascript/selection/range_boundary_navigator_spec.js @@ -134,17 +134,19 @@ describe('RangeBoundaryNavigator', function () { describe('#rangesToCsv', function () { var data, columnDefinitions, ranges; beforeEach(function () { - data = [[1, 'leopard', '12'], - [2, 'lion', '13'], - [3, 'cougar', '9'], - [4, 'tiger', '10']]; - columnDefinitions = [{name: 'id', pos: 0}, {name: 'animal', pos: 1}, {name: 'size', pos: 2}]; + data = [{'id':1, 'animal':'leopard', 'size':'12'}, + {'id':2, 'animal':'lion', 'size':'13'}, + {'id':3, 'animal':'cougar', 'size':'9'}, + {'id':4, 'animal':'tiger', 'size':'10'}]; + + columnDefinitions = [{name: 'id', field: 'id', pos: 0}, + {name: 'animal', field: 'animal', pos: 1}, + {name: 'size', field: 'size', pos: 2}]; ranges = [new Slick.Range(0, 0, 0, 2), new Slick.Range(3, 0, 3, 2)]; }); it('returns csv for the provided ranges', function () { var csvResult = rangeBoundaryNavigator.rangesToCsv(data, columnDefinitions, ranges); - expect(csvResult).toEqual('1,\'leopard\',\'12\'\n4,\'tiger\',\'10\''); }); @@ -158,10 +160,10 @@ describe('RangeBoundaryNavigator', function () { describe('when there is an extra column with checkboxes', function () { beforeEach(function () { - columnDefinitions = [{name: 'not-a-data-column'}, {name: 'id', pos: 0}, {name: 'animal', pos: 1}, { - name: 'size', - pos: 2, - }]; + columnDefinitions = [{name: 'not-a-data-column'}, + {name: 'id', field: 'id', pos: 0}, + {name: 'animal', field: 'animal', pos: 1}, + {name: 'size', field: 'size',pos: 2}]; ranges = [new Slick.Range(0, 0, 0, 3), new Slick.Range(3, 0, 3, 3)]; }); diff --git a/web/regression/javascript/selection/row_selector_spec.js b/web/regression/javascript/selection/row_selector_spec.js index db3c0b4..b6d4d01 100644 --- a/web/regression/javascript/selection/row_selector_spec.js +++ b/web/regression/javascript/selection/row_selector_spec.js @@ -6,7 +6,6 @@ // This software is released under the PostgreSQL Licence // ////////////////////////////////////////////////////////////////////////// - import $ from 'jquery'; import Slick from 'slickgrid'; @@ -23,7 +22,7 @@ describe('RowSelector', function () { UP: 38, DOWN: 40, }; - var container, data, columnDefinitions, grid, cellSelectionModel; + var container, dataView, columnDefinitions, grid, cellSelectionModel; var SlickGrid = Slick.Grid; beforeEach(function () { @@ -43,14 +42,15 @@ describe('RowSelector', function () { pos: 1, }]; + dataView = new Slick.Data.DataView(); var rowSelector = new RowSelector(); - data = []; + var data = []; for (var i = 0; i < 10; i++) { - data.push(['some-value-' + i, 'second value ' + i]); + data.push({'some-column-name':'some-value-' + i, 'second column':'second value ' + i}); } columnDefinitions = rowSelector.getColumnDefinitions(columnDefinitions); - grid = new SlickGrid(container, data, columnDefinitions); - + dataView.setItems(data, 'some-column-name'); + grid = new SlickGrid(container, dataView, columnDefinitions); grid.registerPlugin(new ActiveCellCapture()); cellSelectionModel = new XCellSelectionModel(); grid.setSelectionModel(cellSelectionModel); diff --git a/web/regression/javascript/selection/set_staged_rows_spec.js b/web/regression/javascript/selection/set_staged_rows_spec.js index 6735314..6fd780a 100644 --- a/web/regression/javascript/selection/set_staged_rows_spec.js +++ b/web/regression/javascript/selection/set_staged_rows_spec.js @@ -7,236 +7,247 @@ // ////////////////////////////////////////////////////////////// -define([ - 'jquery', - 'underscore', - 'sources/selection/set_staged_rows', -], function ($, _, SetStagedRows) { - describe('set_staged_rows', function () { - var sqlEditorObj, gridSpy, deleteButton, copyButton, selectionSpy; - beforeEach(function () { - gridSpy = jasmine.createSpyObj('gridSpy', ['getData', 'getCellNode', 'getColumns']); - gridSpy.getData.and.returnValue([ - {0: 'one', 1: 'two', __temp_PK: '123'}, - {0: 'three', 1: 'four', __temp_PK: '456'}, - {0: 'five', 1: 'six', __temp_PK: '789'}, - {0: 'seven', 1: 'eight', __temp_PK: '432'}, - ]); - gridSpy.getColumns.and.returnValue([ +import $ from 'jquery'; +import 'slickgrid.grid'; +import Slick from 'slickgrid'; +import SetStagedRows from 'sources/selection/set_staged_rows'; + +describe('set_staged_rows', function () { + var sqlEditorObj, gridSpy, deleteButton, copyButton, selectionSpy; + beforeEach(function () { + var data = [{'a pk column': 'one', 'some column': 'two', '__temp_PK': '123'}, + {'a pk column': 'three', 'some column': 'four', '__temp_PK': '456'}, + {'a pk column': 'five', 'some column': 'six', '__temp_PK': '789'}, + {'a pk column': 'seven', 'some column': 'eight', '__temp_PK': '432'}], + dataView = new Slick.Data.DataView(); + + dataView.setItems(data, '__temp_PK'); + + gridSpy = jasmine.createSpyObj('gridSpy', ['getData', 'getCellNode', 'getColumns']); + gridSpy.getData.and.returnValue(dataView); + gridSpy.getColumns.and.returnValue([ + { + name: 'a pk column', + field: 'a pk column', + pos: 0, + selectable: true, + }, { + name: 'some column', + field: 'some column', + pos: 1, + selectable: true, + }, + ]); + selectionSpy = jasmine.createSpyObj('selectionSpy', ['setSelectedRows', 'getSelectedRanges']); + deleteButton = $('<button id="btn-delete-row"></button>'); + copyButton = $('<button id="btn-copy-row"></button>'); + + sqlEditorObj = { + grid: gridSpy, + editor: { + handler: { + data_store: { + staged_rows: {'456': {}}, + }, + can_edit: false, + }, + }, + keys: null, + selection: selectionSpy, + columns: [ { + name: 'a pk column', + field: 'a pk column', pos: 0, - selectable: true, - }, { + }, + { + name: 'some column', + field: 'some column', pos: 1, - selectable: true, }, - ]); + ], + client_primary_key: '__temp_PK', + }; - selectionSpy = jasmine.createSpyObj('selectionSpy', ['setSelectedRows', 'getSelectedRanges']); + $('body').append(deleteButton); + $('body').append(copyButton); - deleteButton = $('<button id="btn-delete-row"></button>'); - copyButton = $('<button id="btn-copy-row"></button>'); + deleteButton.prop('disabled', true); + copyButton.prop('disabled', true); - sqlEditorObj = { - grid: gridSpy, - editor: { - handler: { - data_store: { - staged_rows: {'456': {}}, - }, - can_edit: false, - }, - }, - keys: null, - selection: selectionSpy, - columns: [ - { - name: 'a pk column', - pos: 0, - }, - { - name: 'some column', - pos: 1, - }, - ], - }; + selectionSpy = jasmine.createSpyObj('selectionSpy', [ + 'setSelectedRows', + 'getSelectedRanges', + ]); + }); - $('body').append(deleteButton); - $('body').append(copyButton); + afterEach(function () { + copyButton.remove(); + deleteButton.remove(); + }); + describe('when no full rows are selected', function () { + describe('when nothing is selected', function () { + beforeEach(function () { + selectionSpy.getSelectedRanges.and.returnValue([]); + sqlEditorObj.selection = selectionSpy; + SetStagedRows.call(sqlEditorObj, {}, {}); + }); - deleteButton.prop('disabled', true); - copyButton.prop('disabled', true); + it('should disable the delete row button', function () { + expect($('#btn-delete-row').prop('disabled')).toBeTruthy(); + }); - selectionSpy = jasmine.createSpyObj('selectionSpy', [ - 'setSelectedRows', - 'getSelectedRanges', - ]); - }); + it('should disable the copy row button', function () { + expect($('#btn-copy-row').prop('disabled')).toBeTruthy(); + }); - afterEach(function () { - copyButton.remove(); - deleteButton.remove(); + it('should clear staged rows', function () { + expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({}); + }); }); - describe('when no full rows are selected', function () { - describe('when nothing is selected', function () { - beforeEach(function () { - selectionSpy.getSelectedRanges.and.returnValue([]); - sqlEditorObj.selection = selectionSpy; - SetStagedRows.call(sqlEditorObj, {}, {}); - }); - it('should disable the delete row button', function () { - expect($('#btn-delete-row').prop('disabled')).toBeTruthy(); - }); + describe('when there is a selection', function () { + beforeEach(function () { + var range = { + fromCell: 0, + toCell: 0, + fromRow: 1, + toRow: 1, + }; - it('should disable the copy row button', function () { - expect($('#btn-copy-row').prop('disabled')).toBeTruthy(); - }); + selectionSpy.getSelectedRanges.and.returnValue([range]); + sqlEditorObj.selection = selectionSpy; + SetStagedRows.call(sqlEditorObj, {}, {}); + }); - it('should clear staged rows', function () { - expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({}); - }); + it('should disable the delete row button', function () { + expect($('#btn-delete-row').prop('disabled')).toBeTruthy(); }); - describe('when there is a selection', function () { - beforeEach(function () { - var range = { - fromCell: 0, - toCell: 0, - fromRow: 1, - toRow: 1, - }; - - selectionSpy.getSelectedRanges.and.returnValue([range]); - sqlEditorObj.selection = selectionSpy; - SetStagedRows.call(sqlEditorObj, {}, {}); - }); + it('should disable the copy row button', function () { + expect($('#btn-copy-row').prop('disabled')).toBeFalsy(); + }); - it('should disable the delete row button', function () { - expect($('#btn-delete-row').prop('disabled')).toBeTruthy(); - }); + it('should clear staged rows', function () { + expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({}); + }); + }); + }); - it('should disable the copy row button', function () { - expect($('#btn-copy-row').prop('disabled')).toBeFalsy(); - }); + describe('when 2 full rows are selected', function () { + beforeEach(function () { + var range1 = { + fromCell: 0, + toCell: 1, + fromRow: 1, + toRow: 1, + }; + var range2 = { + fromCell: 0, + toCell: 1, + fromRow: 2, + toRow: 2, + }; - it('should clear staged rows', function () { - expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({}); - }); - }); + selectionSpy.getSelectedRanges.and.returnValue([range1, range2]); + sqlEditorObj.selection = selectionSpy; }); - describe('when 2 full rows are selected', function () { - beforeEach(function () { - var range1 = { - fromCell: 0, - toCell: 1, - fromRow: 1, - toRow: 1, - }; - var range2 = { - fromCell: 0, - toCell: 1, - fromRow: 2, - toRow: 2, - }; + describe('when table does not have primary keys', function () { + it('should enable the copy row button', function () { + SetStagedRows.call(sqlEditorObj, {}, {}); + expect($('#btn-copy-row').prop('disabled')).toBeFalsy(); + }); - selectionSpy.getSelectedRanges.and.returnValue([range1, range2]); - sqlEditorObj.selection = selectionSpy; + it('should not enable the delete row button', function () { + SetStagedRows.call(sqlEditorObj, {}, {}); + expect($('#btn-delete-row').prop('disabled')).toBeTruthy(); }); - describe('when table does not have primary keys', function () { - it('should enable the copy row button', function () { - SetStagedRows.call(sqlEditorObj, {}, {}); - expect($('#btn-copy-row').prop('disabled')).toBeFalsy(); - }); + it('should update staged rows with the __temp_PK value of the new Selected Rows', function () { + SetStagedRows.call(sqlEditorObj, {}, {}); + expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({'456': {}, '789': {}}); + }); - it('should not enable the delete row button', function () { + describe('the user can edit', function () { + it('should enable the delete row button', function () { + sqlEditorObj.editor.handler.can_edit = true; SetStagedRows.call(sqlEditorObj, {}, {}); - expect($('#btn-delete-row').prop('disabled')).toBeTruthy(); + expect($('#btn-delete-row').prop('disabled')).toBeFalsy(); }); + }); + }); - it('should update staged rows with the __temp_PK value of the new Selected Rows', function () { + describe('when table has primary keys', function () { + beforeEach(function () { + sqlEditorObj.keys = {'a pk column': 'varchar'}; + sqlEditorObj.editor.handler.data_store.staged_rows = {'456': {'a pk column': 'three'}}; + }); + + describe('selected rows have primary key', function () { + it('should set the staged rows correctly', function () { SetStagedRows.call(sqlEditorObj, {}, {}); - expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({'456': {}, '789': {}}); + expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual( + {'456': {'a pk column': 'three'}, '789': {'a pk column': 'five'}}); }); - describe('the user can edit', function () { - it('should enable the delete row button', function () { - sqlEditorObj.editor.handler.can_edit = true; - SetStagedRows.call(sqlEditorObj, {}, {}); - expect($('#btn-delete-row').prop('disabled')).toBeFalsy(); - }); + it('should not clear selected rows in Cell Selection Model', function () { + SetStagedRows.call(sqlEditorObj, {}, {}); + expect(sqlEditorObj.selection.setSelectedRows).not.toHaveBeenCalledWith(); }); }); - describe('when table has primary keys', function () { + describe('selected rows missing primary key', function () { beforeEach(function () { - sqlEditorObj.keys = {'a pk column': 'varchar'}; - sqlEditorObj.editor.handler.data_store.staged_rows = {'456': {0: 'three'}}; - }); + var data = [{'a pk column': 'one', 'some column': 'two', '__temp_PK': '123'}, + {'some column': 'four', '__temp_PK': '456'}, + {'some column': 'six', '__temp_PK': '789'}, + {'a pk column': 'seven', 'some column': 'eight', '__temp_PK': '432'}], + dataView = new Slick.Data.DataView(); - describe('selected rows have primary key', function () { - it('should set the staged rows correctly', function () { - SetStagedRows.call(sqlEditorObj, {}, {}); - expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual( - {'456': {0: 'three'}, '789': {0: 'five'}}); - }); + dataView.setItems(data, '__temp_PK'); - it('should not clear selected rows in Cell Selection Model', function () { - SetStagedRows.call(sqlEditorObj, {}, {}); - expect(sqlEditorObj.selection.setSelectedRows).not.toHaveBeenCalledWith(); - }); + gridSpy.getData.and.returnValue(dataView); }); - describe('selected rows missing primary key', function () { - beforeEach(function () { - gridSpy.getData.and.returnValue([ - {0: 'one', 1: 'two', __temp_PK: '123'}, - {1: 'four', __temp_PK: '456'}, - {1: 'six', __temp_PK: '789'}, - {0: 'seven', 1: 'eight', __temp_PK: '432'}, - ]); - }); - - it('should clear the staged rows', function () { - SetStagedRows.call(sqlEditorObj, {}, {}); - expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({}); - }); + it('should clear the staged rows', function () { + SetStagedRows.call(sqlEditorObj, {}, {}); + expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({}); + }); - it('should clear selected rows in Cell Selection Model', function () { - SetStagedRows.call(sqlEditorObj, {}, {}); - expect(sqlEditorObj.selection.setSelectedRows).toHaveBeenCalledWith([]); - }); + it('should clear selected rows in Cell Selection Model', function () { + SetStagedRows.call(sqlEditorObj, {}, {}); + expect(sqlEditorObj.selection.setSelectedRows).toHaveBeenCalledWith([]); }); + }); - describe('when the selected row is a new row', function () { - var parentDiv; - beforeEach(function () { - var childDiv = $('<div></div>'); - parentDiv = $('<div class="new_row"></div>'); - parentDiv.append(childDiv); - $('body').append(parentDiv); - gridSpy.getCellNode.and.returnValue(childDiv); - SetStagedRows.call(sqlEditorObj, {}, {}); - }); + describe('when the selected row is a new row', function () { + var parentDiv; + beforeEach(function () { + var childDiv = $('<div></div>'); + parentDiv = $('<div class="new_row"></div>'); + parentDiv.append(childDiv); + $('body').append(parentDiv); + gridSpy.getCellNode.and.returnValue(childDiv); + SetStagedRows.call(sqlEditorObj, {}, {}); + }); - afterEach(function () { - parentDiv.remove(); - }); + afterEach(function () { + parentDiv.remove(); + }); - it('should not clear the staged rows', function () { - expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({ - '456': {0: 'three'}, - '789': {0: 'five'}, - }); + it('should not clear the staged rows', function () { + expect(sqlEditorObj.editor.handler.data_store.staged_rows).toEqual({ + '456': {'a pk column': 'three'}, + '789': {'a pk column': 'five'}, }); + }); - it('should not clear selected rows in Cell Selection Model', function () { - expect(sqlEditorObj.selection.setSelectedRows).not.toHaveBeenCalled(); - }); + it('should not clear selected rows in Cell Selection Model', function () { + expect(sqlEditorObj.selection.setSelectedRows).not.toHaveBeenCalled(); }); }); }); }); }); + diff --git a/web/regression/javascript/selection/xcell_selection_model_spec.js b/web/regression/javascript/selection/xcell_selection_model_spec.js index f41d96c..a7f19aa 100644 --- a/web/regression/javascript/selection/xcell_selection_model_spec.js +++ b/web/regression/javascript/selection/xcell_selection_model_spec.js @@ -33,14 +33,17 @@ describe('XCellSelectionModel', function () { }, { id: '1', name: 'some-column-name', + field: 'some-column-name', pos: 0, }, { id: 'second-column-id', name: 'second column', + field: 'second column', pos: 1, }, { id: 'third-column-id', name: 'third column', + field: 'third column', pos: 2, }, ]; @@ -52,13 +55,15 @@ describe('XCellSelectionModel', function () { 'second column': 'second value ' + i, 'third column': 'third value ' + i, 'fourth column': 'fourth value ' + i, + '__temp_PK': '123' + i, }); } container = $('<div></div>'); + var dataView = new Slick.Data.DataView(); container.height(9999); container.width(9999); - - grid = new SlickGrid(container, data, columns); + dataView.setItems(data, '__temp_PK'); + grid = new SlickGrid(container, dataView, columns); grid.setSelectionModel(new XCellSelectionModel()); $('body').append(container); }); diff --git a/web/regression/javascript/slickgrid/event_handlers/handle_query_output_keyboard_event_spec.js b/web/regression/javascript/slickgrid/event_handlers/handle_query_output_keyboard_event_spec.js index e2edaf7..807e468 100644 --- a/web/regression/javascript/slickgrid/event_handlers/handle_query_output_keyboard_event_spec.js +++ b/web/regression/javascript/slickgrid/event_handlers/handle_query_output_keyboard_event_spec.js @@ -28,21 +28,22 @@ describe('#handleQueryOutputKeyboardEvent', function () { metaKey: false, which: -1, keyCode: -1, - preventDefault: jasmine.createSpy('preventDefault'), + preventDefault: jasmine.createSpy('preventDefault') }; - var data = [['', '0,0-cell-content', '0,1-cell-content'], - ['', '1,0-cell-content', '1,1-cell-content'], - ['', '2,0-cell-content', '2,1-cell-content']]; - var columnDefinitions = [{name: 'checkboxColumn'}, {pos: 1, name: 'firstColumn'}, { - pos: 2, - name: 'secondColumn', - }]; - grid = new SlickGrid($('<div></div>'), data, columnDefinitions); - grid.setSelectionModel(new XCellSelectionModel()); + var data = [{'checkboxColumn': '', 'firstColumn': '0,0-cell-content', 'secondColumn': '0,1-cell-content', '__temp_PK': '123'}, + {'checkboxColumn': '', 'firstColumn': '1,0-cell-content', 'secondColumn': '1,1-cell-content', '__temp_PK': '456'}, + {'checkboxColumn': '', 'firstColumn': '2,0-cell-content', 'secondColumn': '2,1-cell-content', '__temp_PK': '789'}], + columnDefinitions = [{name: 'checkboxColumn'}, + {pos: 1, name: 'firstColumn', field: 'firstColumn'}, + { pos: 2, name: 'secondColumn', field: 'secondColumn'}], + dataView = new Slick.Data.DataView(); + grid = new Slick.Grid($('<div></div>'), dataView, columnDefinitions); + grid.setSelectionModel(new XCellSelectionModel()); + dataView.setItems(data, '__temp_PK'); slickEvent = { - grid: grid, + grid: grid }; spyOn(clipboard, 'copyTextToClipboard'); ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: [pgadmin-hackers] Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] @ 2017-06-27 13:03 Dave Page <[email protected]> parent: Harshal Dhumal <[email protected]> 0 siblings, 2 replies; 24+ messages in thread From: Dave Page @ 2017-06-27 13:03 UTC (permalink / raw) To: Harshal Dhumal <[email protected]>; +Cc: Harshal Dhumal <[email protected]>; Ashesh Vashi <[email protected]>; pgadmin-hackers Thanks - patch committed! Awsome job :-) On Tue, Jun 27, 2017 at 3:26 AM, Harshal Dhumal < [email protected]> wrote: > Hi, > > Please find rebased patch. > > -- > *Harshal Dhumal* > *Sr. Software Engineer* > > EnterpriseDB India: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > > On Mon, Jun 26, 2017 at 5:24 PM, Harshal Dhumal < > [email protected]> wrote: > >> yes i'm working on that only :) >> >> >> -- >> *Harshal Dhumal* >> *Sr. Software Engineer* >> >> EnterpriseDB India: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> >> On Mon, Jun 26, 2017 at 5:22 PM, Dave Page <[email protected]> wrote: >> >>> I'm sorry, it needs rebasing again. If you can do it quickly, I'll >>> make sure it's the next patch I work on in that area. >>> >>> Thanks. >>> >>> On Mon, Jun 26, 2017 at 5:16 AM, Harshal Dhumal >>> <[email protected]> wrote: >>> > Hi Dave, >>> > >>> > Please find updated rebased patch for RM2137 >>> > >>> > On Fri, Jun 23, 2017 at 9:00 PM, Dave Page <[email protected]> wrote: >>> >> >>> >> Hi Harshal, >>> >> >>> >> When can we expect an updated version of this patch? I think it's >>> >> important to get this into the next release. >>> >> >>> >> Thanks! >>> >> >>> >> On Fri, Jun 16, 2017 at 10:55 AM, Dave Page <[email protected]> >>> wrote: >>> >> > Hi, >>> >> > >>> >> > That's better - the failures are far less random now :-). I got the >>> >> > following two though, on both PG and EPAS 9.5: >>> >> > >>> >> > ============================================================ >>> ========== >>> >> > ERROR: runTest >>> >> > (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >>> >> > Query tool feature test >>> >> > ------------------------------------------------------------ >>> ---------- >>> >> > Traceback (most recent call last): >>> >> > File >>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_t >>> ool_tests.py", >>> >> > line 95, in runTest >>> >> > self._query_tool_explain_analyze_buffers() >>> >> > File >>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_t >>> ool_tests.py", >>> >> > line 443, in _query_tool_explain_analyze_buffers >>> >> > canvas.find_element_by_xpath("//*[contains(string(), 'Shared >>> Read >>> >> > Blocks')]") >>> >> > File >>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>> ges/selenium/webdriver/remote/webelement.py", >>> >> > line 260, in find_element_by_xpath >>> >> > return self.find_element(by=By.XPATH, value=xpath) >>> >> > File >>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>> ges/selenium/webdriver/remote/webelement.py", >>> >> > line 508, in find_element >>> >> > {"using": by, "value": value})['value'] >>> >> > File >>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>> ges/selenium/webdriver/remote/webelement.py", >>> >> > line 491, in _execute >>> >> > return self._parent.execute(command, params) >>> >> > File >>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>> ges/selenium/webdriver/remote/webdriver.py", >>> >> > line 238, in execute >>> >> > self.error_handler.check_response(response) >>> >> > File >>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>> ges/selenium/webdriver/remote/errorhandler.py", >>> >> > line 193, in check_response >>> >> > raise exception_class(message, screen, stacktrace) >>> >> > NoSuchElementException: Message: no such element: Unable to locate >>> >> > element: {"method":"xpath","selector":"//*[contains(string(), >>> 'Shared >>> >> > Read Blocks')]"} >>> >> > (Session info: chrome=58.0.3029.110) >>> >> > (Driver info: chromedriver=2.29.461585 >>> >> > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X >>> 10.12.3 >>> >> > x86_64) >>> >> > >>> >> > >>> >> > ============================================================ >>> ========== >>> >> > ERROR: runTest >>> >> > (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest) >>> >> > Validate Insert, Update operations in View data with given test data >>> >> > ------------------------------------------------------------ >>> ---------- >>> >> > Traceback (most recent call last): >>> >> > File >>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>> ta_dml_queries.py", >>> >> > line 104, in runTest >>> >> > self._add_row() >>> >> > File >>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>> ta_dml_queries.py", >>> >> > line 255, in _add_row >>> >> > self._update_cell(cell_xpath, config_data[str(idx)]) >>> >> > File >>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>> ta_dml_queries.py", >>> >> > line 164, in _update_cell >>> >> > cell_el = self.page.find_by_xpath(xpath) >>> >> > File >>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>> min_page.py", >>> >> > line 122, in find_by_xpath >>> >> > return self.wait_for_element(lambda driver: >>> >> > driver.find_element_by_xpath(xpath)) >>> >> > File >>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>> min_page.py", >>> >> > line 205, in wait_for_element >>> >> > return self._wait_for("element to exist", element_if_it_exists) >>> >> > File >>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>> min_page.py", >>> >> > line 255, in _wait_for >>> >> > "Timed out waiting for " + waiting_for_message) >>> >> > File >>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>> ges/selenium/webdriver/support/wait.py", >>> >> > line 71, in until >>> >> > value = method(self._driver) >>> >> > File >>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>> min_page.py", >>> >> > line 200, in element_if_it_exists >>> >> > if element.is_displayed() and element.is_enabled(): >>> >> > File >>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>> ges/selenium/webdriver/remote/webelement.py", >>> >> > line 358, in is_displayed >>> >> > return self._execute(Command.IS_ELEMENT_DISPLAYED)['value'] >>> >> > File >>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>> ges/selenium/webdriver/remote/webelement.py", >>> >> > line 491, in _execute >>> >> > return self._parent.execute(command, params) >>> >> > File >>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>> ges/selenium/webdriver/remote/webdriver.py", >>> >> > line 238, in execute >>> >> > self.error_handler.check_response(response) >>> >> > File >>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>> ges/selenium/webdriver/remote/errorhandler.py", >>> >> > line 193, in check_response >>> >> > raise exception_class(message, screen, stacktrace) >>> >> > StaleElementReferenceException: Message: stale element reference: >>> >> > element is not attached to the page document >>> >> > (Session info: chrome=58.0.3029.110) >>> >> > (Driver info: chromedriver=2.29.461585 >>> >> > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X >>> 10.12.3 >>> >> > x86_64) >>> >> > >>> > >>> > >>> > I checked my feature test cases for any database server version/type >>> (PG, >>> > EPAS) specific failures and I found that test cases are working fine >>> on all >>> > of them. However I have slightly modified test case to overcome above >>> > mentioned failures. >>> > >>> >> >>> >> > >>> >> > On 9.4 everything passes. On 9.6, I get one failure: >>> >> > >>> >> > ============================================================ >>> ========== >>> >> > FAIL: runTest >>> >> > (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest) >>> >> > Validate Insert, Update operations in View data with given test data >>> >> > ------------------------------------------------------------ >>> ---------- >>> >> > Traceback (most recent call last): >>> >> > File >>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>> ta_dml_queries.py", >>> >> > line 105, in runTest >>> >> > self._verify_row_data(True) >>> >> > File >>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>> ta_dml_queries.py", >>> >> > line 282, in _verify_row_data >>> >> > self.assertEquals(cells[idx], config_data[str(idx)][1]) >>> >> > AssertionError: u'[default]' != u'1' >>> >> > - [default] >>> >> > + 1 >>> >> > >>> > >>> > I guess Khushboo has fixed this recently. >>> > >>> > >>> >> >>> >> > Screenshots attached. >>> >> > >>> >> > On Fri, Jun 16, 2017 at 7:54 AM, Harshal Dhumal >>> >> > <[email protected]> wrote: >>> >> >> Hi Dave, >>> >> >> >>> >> >> Please find attached patch where I have added timeout of 2 seconds >>> >> >> before >>> >> >> selecting object menu in connect to server function. >>> >> >> >>> >> >> Note: Apply this patch on top of previous patch for on demand >>> loading >>> >> >> feature. >>> >> >> >>> >> >> >>> >> >> -- >>> >> >> Harshal Dhumal >>> >> >> Sr. Software Engineer >>> >> >> >>> >> >> EnterpriseDB India: http://www.enterprisedb.com >>> >> >> The Enterprise PostgreSQL Company >>> >> >> >>> >> >> On Fri, Jun 16, 2017 at 2:25 AM, Dave Page <[email protected]> >>> wrote: >>> >> >>> >>> >> >>> Sounds good, thanks. >>> >> >>> >>> >> >>> On Thu, Jun 15, 2017 at 9:54 PM, Harshal Dhumal >>> >> >>> <[email protected]> wrote: >>> >> >>> > Hi >>> >> >>> > >>> >> >>> > On Fri, Jun 16, 2017 at 2:07 AM, Dave Page <[email protected]> >>> >> >>> > wrote: >>> >> >>> >> >>> >> >>> >> Hi >>> >> >>> >> >>> >> >>> >> On Thu, Jun 15, 2017 at 9:30 PM, Harshal Dhumal >>> >> >>> >> <[email protected]> wrote: >>> >> >>> >> > Hi Dave, >>> >> >>> >> > >>> >> >>> >> > Please find attached updated patch. >>> >> >>> >> > >>> >> >>> >> > On Thu, Jun 15, 2017 at 3:58 PM, Dave Page < >>> [email protected]> >>> >> >>> >> > wrote: >>> >> >>> >> >> >>> >> >>> >> >> Hi >>> >> >>> >> >> >>> >> >>> >> >> On Wed, Jun 14, 2017 at 11:36 PM, Harshal Dhumal >>> >> >>> >> >> <[email protected]> wrote: >>> >> >>> >> >>> >>> >> >>> >> >>> Hi Dave, >>> >> >>> >> >>> >>> >> >>> >> >>> Please find rebased patch for RM2137. >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> Looking very good. The only issues I see are: >>> >> >>> >> >> >>> >> >>> >> >> - The row headers should auto-size such that they can >>> display >>> >> >>> >> >> the >>> >> >>> >> >> row >>> >> >>> >> >> numbers if the last row was displayed. E.g. if there are >>> 12345 >>> >> >>> >> >> rows >>> >> >>> >> >> in >>> >> >>> >> >> total, then the row header should be sized to display 5 >>> digits. >>> >> >>> >> >> >>> >> >>> >> > Fixed. >>> >> >>> >> > >>> >> >>> >> > >>> >> >>> >> >> >>> >> >>> >> >> - The tests are comprehensive, which is awesome. However, >>> every >>> >> >>> >> >> time >>> >> >>> >> >> I >>> >> >>> >> >> ran >>> >> >>> >> >> them, at least one of the feature tests failed. >>> Unfortunately, >>> >> >>> >> >> it >>> >> >>> >> >> was a >>> >> >>> >> >> different one each time. In the last two runs, I got: >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> ============================== >>> ======================================== >>> >> >>> >> >> ERROR: runTest >>> >> >>> >> >> (pgadmin.feature_tests.query_t >>> ool_tests.QueryToolFeatureTest) >>> >> >>> >> >> Query tool feature test >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> ------------------------------ >>> ---------------------------------------- >>> >> >>> >> >> Traceback (most recent call last): >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>> /regression/feature_utils/base_feature_test.py", >>> >> >>> >> >> line 40, in setUp >>> >> >>> >> >> self.before() >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>> /pgadmin/feature_tests/query_tool_tests.py", >>> >> >>> >> >> line 40, in before >>> >> >>> >> >> self._connects_to_server() >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>> /pgadmin/feature_tests/query_tool_tests.py", >>> >> >>> >> >> line 144, in _connects_to_server >>> >> >>> >> >> self.page.driver.find_element_by_link_text("Create"))\ >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>> webdriver.py", >>> >> >>> >> >> line 319, in find_element_by_link_text >>> >> >>> >> >> return self.find_element(by=By.LINK_TEXT, >>> value=link_text) >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>> webdriver.py", >>> >> >>> >> >> line 756, in find_element >>> >> >>> >> >> 'value': value})['value'] >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>> webdriver.py", >>> >> >>> >> >> line 238, in execute >>> >> >>> >> >> self.error_handler.check_response(response) >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>> errorhandler.py", >>> >> >>> >> >> line 193, in check_response >>> >> >>> >> >> raise exception_class(message, screen, stacktrace) >>> >> >>> >> >> NoSuchElementException: Message: no such element: Unable to >>> >> >>> >> >> locate >>> >> >>> >> >> element: {"method":"link text","selector":"Create"} >>> >> >>> >> >> (Session info: chrome=58.0.3029.110) >>> >> >>> >> >> (Driver info: chromedriver=2.29.461585 >>> >> >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS >>> X >>> >> >>> >> >> 10.12.3 >>> >> >>> >> >> x86_64) >>> >> >>> >> >> >>> >> >>> >> > I checked code from _connects_to_server function which is >>> common >>> >> >>> >> > in >>> >> >>> >> > all >>> >> >>> >> > features test cases. I didn't find anything wrong with this. >>> If >>> >> >>> >> > there >>> >> >>> >> > is >>> >> >>> >> > a >>> >> >>> >> > bug in this function then all feature test must fail. >>> >> >>> >> > Let me know if you are getting failure consistently in >>> >> >>> >> > _connects_to_server >>> >> >>> >> > function. >>> >> >>> >> >>> >> >>> >> I wondered if that one is a race condition. Do we need a short >>> >> >>> >> delay >>> >> >>> >> before clicking the Object menu? I have seen this occasionally >>> >> >>> >> before. >>> >> >>> > >>> >> >>> > >>> >> >>> > OK. In that case let's try putting 1-2 second delay and observer >>> >> >>> > behaviour. >>> >> >>> > I'll send separate patch for this tomorrow as this is not >>> related to >>> >> >>> > on >>> >> >>> > demand query result feature or its test cases. >>> >> >>> > >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> ============================== >>> ======================================== >>> >> >>> >> >> ERROR: runTest >>> >> >>> >> >> (pgadmin.feature_tests.query_t >>> ool_tests.QueryToolFeatureTest) >>> >> >>> >> >> Query tool feature test >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> ------------------------------ >>> ---------------------------------------- >>> >> >>> >> >> Traceback (most recent call last): >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>> /pgadmin/feature_tests/query_tool_tests.py", >>> >> >>> >> >> line 119, in runTest >>> >> >>> >> >> self._query_tool_auto_rollback_enabled() >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>> /pgadmin/feature_tests/query_tool_tests.py", >>> >> >>> >> >> line 697, in _query_tool_auto_rollback_enabled >>> >> >>> >> >> '//div[contains(@class, "sql-editor-message") and >>> >> >>> >> >> contains(string(), >>> >> >>> >> >> "COMMIT")]' >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>> webdriver.py", >>> >> >>> >> >> line 295, in find_element_by_xpath >>> >> >>> >> >> return self.find_element(by=By.XPATH, value=xpath) >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>> webdriver.py", >>> >> >>> >> >> line 756, in find_element >>> >> >>> >> >> 'value': value})['value'] >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>> webdriver.py", >>> >> >>> >> >> line 238, in execute >>> >> >>> >> >> self.error_handler.check_response(response) >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>> errorhandler.py", >>> >> >>> >> >> line 193, in check_response >>> >> >>> >> >> raise exception_class(message, screen, stacktrace) >>> >> >>> >> >> NoSuchElementException: Message: no such element: Unable to >>> >> >>> >> >> locate >>> >> >>> >> >> element: {"method":"xpath","selector":" >>> //div[contains(@class, >>> >> >>> >> >> "sql-editor-message") and contains(string(), "COMMIT")]"} >>> >> >>> >> >> (Session info: chrome=58.0.3029.110) >>> >> >>> >> >> (Driver info: chromedriver=2.29.461585 >>> >> >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS >>> X >>> >> >>> >> >> 10.12.3 >>> >> >>> >> >> x86_64) >>> >> >>> >> >> >>> >> >>> >> > I have updated Auto rollback enabled test in this patch. >>> >> >>> >> > >>> >> >>> >> >> >>> >> >>> >> >> Relevant screenshots attached. >>> >> >>> >> >> >>> >> >>> >> >> - Can you tidy up the regression output a little please? >>> Instead >>> >> >>> >> >> of: >>> >> >>> >> >> >>> >> >>> >> >> ------- >>> >> >>> >> >> runTest >>> >> >>> >> >> (pgadmin.feature_tests.query_t >>> ool_tests.QueryToolFeatureTest) >>> >> >>> >> >> Query tool feature test ... On demand result set on >>> scrolling... >>> >> >>> >> >> OK. >>> >> >>> >> >> >>> >> >>> >> >> On demand result set on grid select all... >>> >> >>> >> >> OK. >>> >> >>> >> >> >>> >> >>> >> >> On demand result set on column select all... >>> >> >>> >> >> OK. >>> >> >>> >> >> >>> >> >>> >> >> Explain query... >>> >> >>> >> >> OK. >>> >> >>> >> >> >>> >> >>> >> >> Explain query with verbose... >>> >> >>> >> >> OK. >>> >> >>> >> >> >>> >> >>> >> >> Explain query with costs... >>> >> >>> >> >> OK. >>> >> >>> >> >> >>> >> >>> >> >> Explain analyze query... >>> >> >>> >> >> OK. >>> >> >>> >> >> >>> >> >>> >> >> Explain analyze query with buffers... >>> >> >>> >> >> OK. >>> >> >>> >> >> >>> >> >>> >> >> Explain analyze query with timing... >>> >> >>> >> >> OK. >>> >> >>> >> >> >>> >> >>> >> >> Auto commit disabled... >>> >> >>> >> >> OK. >>> >> >>> >> >> >>> >> >>> >> >> Auto commit enabled... >>> >> >>> >> >> OK. >>> >> >>> >> >> >>> >> >>> >> >> Auto rollback enabled... >>> >> >>> >> >> ERROR >>> >> >>> >> >> ------- >>> >> >>> >> >> >>> >> >>> >> >> Something like: >>> >> >>> >> >> >>> >> >>> >> >> ------- >>> >> >>> >> >> runTest >>> >> >>> >> >> (pgadmin.feature_tests.query_t >>> ool_tests.QueryToolFeatureTest) >>> >> >>> >> >> Query tool feature test ... >>> >> >>> >> >> On demand result set on scrolling... OK. >>> >> >>> >> >> On demand result set on grid select all... OK. >>> >> >>> >> >> On demand result set on column select all... OK. >>> >> >>> >> >> Explain query... OK. >>> >> >>> >> >> Explain query with verbose... OK. >>> >> >>> >> >> Explain query with costs... OK. >>> >> >>> >> >> Explain analyze query... OK. >>> >> >>> >> >> Explain analyze query with buffers... OK. >>> >> >>> >> >> Explain analyze query with timing... OK. >>> >> >>> >> >> Auto commit disabled... OK. >>> >> >>> >> >> Auto commit enabled... OK. >>> >> >>> >> >> Auto rollback enabled... ERROR >>> >> >>> >> >> -------- >>> >> >>> >> >> >>> >> >>> >> > Fixed. >>> >> >>> >> > >>> >> >>> >> > >>> >> >>> >> >> >>> >> >>> >> >> Thanks! >>> >> >>> >> >> >>> >> >>> >> >> -- >>> >> >>> >> >> Dave Page >>> >> >>> >> >> Blog: http://pgsnake.blogspot.com >>> >> >>> >> >> Twitter: @pgsnake >>> >> >>> >> >> >>> >> >>> >> >> EnterpriseDB UK: http://www.enterprisedb.com >>> >> >>> >> >> The Enterprise PostgreSQL Company >>> >> >>> >> > >>> >> >>> >> > >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> -- >>> >> >>> >> Dave Page >>> >> >>> >> Blog: http://pgsnake.blogspot.com >>> >> >>> >> Twitter: @pgsnake >>> >> >>> >> >>> >> >>> >> EnterpriseDB UK: http://www.enterprisedb.com >>> >> >>> >> The Enterprise PostgreSQL Company >>> >> >>> > >>> >> >>> > >>> >> >>> >>> >> >>> >>> >> >>> >>> >> >>> -- >>> >> >>> Dave Page >>> >> >>> Blog: http://pgsnake.blogspot.com >>> >> >>> Twitter: @pgsnake >>> >> >>> >>> >> >>> EnterpriseDB UK: http://www.enterprisedb.com >>> >> >>> The Enterprise PostgreSQL Company >>> >> >> >>> >> >> >>> >> > >>> >> > >>> >> > >>> >> > -- >>> >> > Dave Page >>> >> > Blog: http://pgsnake.blogspot.com >>> >> > Twitter: @pgsnake >>> >> > >>> >> > EnterpriseDB UK: http://www.enterprisedb.com >>> >> > The Enterprise PostgreSQL Company >>> >> >>> >> >>> >> >>> >> -- >>> >> Dave Page >>> >> Blog: http://pgsnake.blogspot.com >>> >> Twitter: @pgsnake >>> >> >>> >> EnterpriseDB UK: http://www.enterprisedb.com >>> >> The Enterprise PostgreSQL Company >>> > >>> > >>> >>> >>> >>> -- >>> Dave Page >>> Blog: http://pgsnake.blogspot.com >>> Twitter: @pgsnake >>> >>> EnterpriseDB UK: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >> >> > -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: [pgadmin-hackers] Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] @ 2017-06-27 13:07 Ashesh Vashi <[email protected]> parent: Dave Page <[email protected]> 1 sibling, 0 replies; 24+ messages in thread From: Ashesh Vashi @ 2017-06-27 13:07 UTC (permalink / raw) To: Dave Page <[email protected]>; +Cc: Harshal Dhumal <[email protected]>; Harshal Dhumal <[email protected]>; pgadmin-hackers On Tue, Jun 27, 2017 at 6:33 PM, Dave Page <[email protected]> wrote: > Thanks - patch committed! > > Awsome job :-) > Woohoo! -- Thanks & Regards, Ashesh Vashi > > On Tue, Jun 27, 2017 at 3:26 AM, Harshal Dhumal < > [email protected]> wrote: > >> Hi, >> >> Please find rebased patch. >> >> -- >> *Harshal Dhumal* >> *Sr. Software Engineer* >> >> EnterpriseDB India: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> >> On Mon, Jun 26, 2017 at 5:24 PM, Harshal Dhumal < >> [email protected]> wrote: >> >>> yes i'm working on that only :) >>> >>> >>> -- >>> *Harshal Dhumal* >>> *Sr. Software Engineer* >>> >>> EnterpriseDB India: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >>> On Mon, Jun 26, 2017 at 5:22 PM, Dave Page <[email protected]> wrote: >>> >>>> I'm sorry, it needs rebasing again. If you can do it quickly, I'll >>>> make sure it's the next patch I work on in that area. >>>> >>>> Thanks. >>>> >>>> On Mon, Jun 26, 2017 at 5:16 AM, Harshal Dhumal >>>> <[email protected]> wrote: >>>> > Hi Dave, >>>> > >>>> > Please find updated rebased patch for RM2137 >>>> > >>>> > On Fri, Jun 23, 2017 at 9:00 PM, Dave Page <[email protected]> wrote: >>>> >> >>>> >> Hi Harshal, >>>> >> >>>> >> When can we expect an updated version of this patch? I think it's >>>> >> important to get this into the next release. >>>> >> >>>> >> Thanks! >>>> >> >>>> >> On Fri, Jun 16, 2017 at 10:55 AM, Dave Page <[email protected]> >>>> wrote: >>>> >> > Hi, >>>> >> > >>>> >> > That's better - the failures are far less random now :-). I got the >>>> >> > following two though, on both PG and EPAS 9.5: >>>> >> > >>>> >> > ============================================================ >>>> ========== >>>> >> > ERROR: runTest >>>> >> > (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >>>> >> > Query tool feature test >>>> >> > ------------------------------------------------------------ >>>> ---------- >>>> >> > Traceback (most recent call last): >>>> >> > File >>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_t >>>> ool_tests.py", >>>> >> > line 95, in runTest >>>> >> > self._query_tool_explain_analyze_buffers() >>>> >> > File >>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_t >>>> ool_tests.py", >>>> >> > line 443, in _query_tool_explain_analyze_buffers >>>> >> > canvas.find_element_by_xpath("//*[contains(string(), 'Shared >>>> Read >>>> >> > Blocks')]") >>>> >> > File >>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>> ges/selenium/webdriver/remote/webelement.py", >>>> >> > line 260, in find_element_by_xpath >>>> >> > return self.find_element(by=By.XPATH, value=xpath) >>>> >> > File >>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>> ges/selenium/webdriver/remote/webelement.py", >>>> >> > line 508, in find_element >>>> >> > {"using": by, "value": value})['value'] >>>> >> > File >>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>> ges/selenium/webdriver/remote/webelement.py", >>>> >> > line 491, in _execute >>>> >> > return self._parent.execute(command, params) >>>> >> > File >>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>> ges/selenium/webdriver/remote/webdriver.py", >>>> >> > line 238, in execute >>>> >> > self.error_handler.check_response(response) >>>> >> > File >>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>> ges/selenium/webdriver/remote/errorhandler.py", >>>> >> > line 193, in check_response >>>> >> > raise exception_class(message, screen, stacktrace) >>>> >> > NoSuchElementException: Message: no such element: Unable to locate >>>> >> > element: {"method":"xpath","selector":"//*[contains(string(), >>>> 'Shared >>>> >> > Read Blocks')]"} >>>> >> > (Session info: chrome=58.0.3029.110) >>>> >> > (Driver info: chromedriver=2.29.461585 >>>> >> > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X >>>> 10.12.3 >>>> >> > x86_64) >>>> >> > >>>> >> > >>>> >> > ============================================================ >>>> ========== >>>> >> > ERROR: runTest >>>> >> > (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest) >>>> >> > Validate Insert, Update operations in View data with given test >>>> data >>>> >> > ------------------------------------------------------------ >>>> ---------- >>>> >> > Traceback (most recent call last): >>>> >> > File >>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>> ta_dml_queries.py", >>>> >> > line 104, in runTest >>>> >> > self._add_row() >>>> >> > File >>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>> ta_dml_queries.py", >>>> >> > line 255, in _add_row >>>> >> > self._update_cell(cell_xpath, config_data[str(idx)]) >>>> >> > File >>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>> ta_dml_queries.py", >>>> >> > line 164, in _update_cell >>>> >> > cell_el = self.page.find_by_xpath(xpath) >>>> >> > File >>>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>>> min_page.py", >>>> >> > line 122, in find_by_xpath >>>> >> > return self.wait_for_element(lambda driver: >>>> >> > driver.find_element_by_xpath(xpath)) >>>> >> > File >>>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>>> min_page.py", >>>> >> > line 205, in wait_for_element >>>> >> > return self._wait_for("element to exist", element_if_it_exists) >>>> >> > File >>>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>>> min_page.py", >>>> >> > line 255, in _wait_for >>>> >> > "Timed out waiting for " + waiting_for_message) >>>> >> > File >>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>> ges/selenium/webdriver/support/wait.py", >>>> >> > line 71, in until >>>> >> > value = method(self._driver) >>>> >> > File >>>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>>> min_page.py", >>>> >> > line 200, in element_if_it_exists >>>> >> > if element.is_displayed() and element.is_enabled(): >>>> >> > File >>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>> ges/selenium/webdriver/remote/webelement.py", >>>> >> > line 358, in is_displayed >>>> >> > return self._execute(Command.IS_ELEMENT_DISPLAYED)['value'] >>>> >> > File >>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>> ges/selenium/webdriver/remote/webelement.py", >>>> >> > line 491, in _execute >>>> >> > return self._parent.execute(command, params) >>>> >> > File >>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>> ges/selenium/webdriver/remote/webdriver.py", >>>> >> > line 238, in execute >>>> >> > self.error_handler.check_response(response) >>>> >> > File >>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>> ges/selenium/webdriver/remote/errorhandler.py", >>>> >> > line 193, in check_response >>>> >> > raise exception_class(message, screen, stacktrace) >>>> >> > StaleElementReferenceException: Message: stale element reference: >>>> >> > element is not attached to the page document >>>> >> > (Session info: chrome=58.0.3029.110) >>>> >> > (Driver info: chromedriver=2.29.461585 >>>> >> > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X >>>> 10.12.3 >>>> >> > x86_64) >>>> >> > >>>> > >>>> > >>>> > I checked my feature test cases for any database server version/type >>>> (PG, >>>> > EPAS) specific failures and I found that test cases are working fine >>>> on all >>>> > of them. However I have slightly modified test case to overcome above >>>> > mentioned failures. >>>> > >>>> >> >>>> >> > >>>> >> > On 9.4 everything passes. On 9.6, I get one failure: >>>> >> > >>>> >> > ============================================================ >>>> ========== >>>> >> > FAIL: runTest >>>> >> > (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest) >>>> >> > Validate Insert, Update operations in View data with given test >>>> data >>>> >> > ------------------------------------------------------------ >>>> ---------- >>>> >> > Traceback (most recent call last): >>>> >> > File >>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>> ta_dml_queries.py", >>>> >> > line 105, in runTest >>>> >> > self._verify_row_data(True) >>>> >> > File >>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>> ta_dml_queries.py", >>>> >> > line 282, in _verify_row_data >>>> >> > self.assertEquals(cells[idx], config_data[str(idx)][1]) >>>> >> > AssertionError: u'[default]' != u'1' >>>> >> > - [default] >>>> >> > + 1 >>>> >> > >>>> > >>>> > I guess Khushboo has fixed this recently. >>>> > >>>> > >>>> >> >>>> >> > Screenshots attached. >>>> >> > >>>> >> > On Fri, Jun 16, 2017 at 7:54 AM, Harshal Dhumal >>>> >> > <[email protected]> wrote: >>>> >> >> Hi Dave, >>>> >> >> >>>> >> >> Please find attached patch where I have added timeout of 2 seconds >>>> >> >> before >>>> >> >> selecting object menu in connect to server function. >>>> >> >> >>>> >> >> Note: Apply this patch on top of previous patch for on demand >>>> loading >>>> >> >> feature. >>>> >> >> >>>> >> >> >>>> >> >> -- >>>> >> >> Harshal Dhumal >>>> >> >> Sr. Software Engineer >>>> >> >> >>>> >> >> EnterpriseDB India: http://www.enterprisedb.com >>>> >> >> The Enterprise PostgreSQL Company >>>> >> >> >>>> >> >> On Fri, Jun 16, 2017 at 2:25 AM, Dave Page <[email protected]> >>>> wrote: >>>> >> >>> >>>> >> >>> Sounds good, thanks. >>>> >> >>> >>>> >> >>> On Thu, Jun 15, 2017 at 9:54 PM, Harshal Dhumal >>>> >> >>> <[email protected]> wrote: >>>> >> >>> > Hi >>>> >> >>> > >>>> >> >>> > On Fri, Jun 16, 2017 at 2:07 AM, Dave Page <[email protected]> >>>> >> >>> > wrote: >>>> >> >>> >> >>>> >> >>> >> Hi >>>> >> >>> >> >>>> >> >>> >> On Thu, Jun 15, 2017 at 9:30 PM, Harshal Dhumal >>>> >> >>> >> <[email protected]> wrote: >>>> >> >>> >> > Hi Dave, >>>> >> >>> >> > >>>> >> >>> >> > Please find attached updated patch. >>>> >> >>> >> > >>>> >> >>> >> > On Thu, Jun 15, 2017 at 3:58 PM, Dave Page < >>>> [email protected]> >>>> >> >>> >> > wrote: >>>> >> >>> >> >> >>>> >> >>> >> >> Hi >>>> >> >>> >> >> >>>> >> >>> >> >> On Wed, Jun 14, 2017 at 11:36 PM, Harshal Dhumal >>>> >> >>> >> >> <[email protected]> wrote: >>>> >> >>> >> >>> >>>> >> >>> >> >>> Hi Dave, >>>> >> >>> >> >>> >>>> >> >>> >> >>> Please find rebased patch for RM2137. >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> Looking very good. The only issues I see are: >>>> >> >>> >> >> >>>> >> >>> >> >> - The row headers should auto-size such that they can >>>> display >>>> >> >>> >> >> the >>>> >> >>> >> >> row >>>> >> >>> >> >> numbers if the last row was displayed. E.g. if there are >>>> 12345 >>>> >> >>> >> >> rows >>>> >> >>> >> >> in >>>> >> >>> >> >> total, then the row header should be sized to display 5 >>>> digits. >>>> >> >>> >> >> >>>> >> >>> >> > Fixed. >>>> >> >>> >> > >>>> >> >>> >> > >>>> >> >>> >> >> >>>> >> >>> >> >> - The tests are comprehensive, which is awesome. However, >>>> every >>>> >> >>> >> >> time >>>> >> >>> >> >> I >>>> >> >>> >> >> ran >>>> >> >>> >> >> them, at least one of the feature tests failed. >>>> Unfortunately, >>>> >> >>> >> >> it >>>> >> >>> >> >> was a >>>> >> >>> >> >> different one each time. In the last two runs, I got: >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> ============================== >>>> ======================================== >>>> >> >>> >> >> ERROR: runTest >>>> >> >>> >> >> (pgadmin.feature_tests.query_t >>>> ool_tests.QueryToolFeatureTest) >>>> >> >>> >> >> Query tool feature test >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> ------------------------------ >>>> ---------------------------------------- >>>> >> >>> >> >> Traceback (most recent call last): >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>> /regression/feature_utils/base_feature_test.py", >>>> >> >>> >> >> line 40, in setUp >>>> >> >>> >> >> self.before() >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>> /pgadmin/feature_tests/query_tool_tests.py", >>>> >> >>> >> >> line 40, in before >>>> >> >>> >> >> self._connects_to_server() >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>> /pgadmin/feature_tests/query_tool_tests.py", >>>> >> >>> >> >> line 144, in _connects_to_server >>>> >> >>> >> >> self.page.driver.find_element_by_link_text("Create"))\ >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>> webdriver.py", >>>> >> >>> >> >> line 319, in find_element_by_link_text >>>> >> >>> >> >> return self.find_element(by=By.LINK_TEXT, >>>> value=link_text) >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>> webdriver.py", >>>> >> >>> >> >> line 756, in find_element >>>> >> >>> >> >> 'value': value})['value'] >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>> webdriver.py", >>>> >> >>> >> >> line 238, in execute >>>> >> >>> >> >> self.error_handler.check_response(response) >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>> errorhandler.py", >>>> >> >>> >> >> line 193, in check_response >>>> >> >>> >> >> raise exception_class(message, screen, stacktrace) >>>> >> >>> >> >> NoSuchElementException: Message: no such element: Unable to >>>> >> >>> >> >> locate >>>> >> >>> >> >> element: {"method":"link text","selector":"Create"} >>>> >> >>> >> >> (Session info: chrome=58.0.3029.110) >>>> >> >>> >> >> (Driver info: chromedriver=2.29.461585 >>>> >> >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac >>>> OS X >>>> >> >>> >> >> 10.12.3 >>>> >> >>> >> >> x86_64) >>>> >> >>> >> >> >>>> >> >>> >> > I checked code from _connects_to_server function which is >>>> common >>>> >> >>> >> > in >>>> >> >>> >> > all >>>> >> >>> >> > features test cases. I didn't find anything wrong with >>>> this. If >>>> >> >>> >> > there >>>> >> >>> >> > is >>>> >> >>> >> > a >>>> >> >>> >> > bug in this function then all feature test must fail. >>>> >> >>> >> > Let me know if you are getting failure consistently in >>>> >> >>> >> > _connects_to_server >>>> >> >>> >> > function. >>>> >> >>> >> >>>> >> >>> >> I wondered if that one is a race condition. Do we need a short >>>> >> >>> >> delay >>>> >> >>> >> before clicking the Object menu? I have seen this occasionally >>>> >> >>> >> before. >>>> >> >>> > >>>> >> >>> > >>>> >> >>> > OK. In that case let's try putting 1-2 second delay and >>>> observer >>>> >> >>> > behaviour. >>>> >> >>> > I'll send separate patch for this tomorrow as this is not >>>> related to >>>> >> >>> > on >>>> >> >>> > demand query result feature or its test cases. >>>> >> >>> > >>>> >> >>> >> >>>> >> >>> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> ============================== >>>> ======================================== >>>> >> >>> >> >> ERROR: runTest >>>> >> >>> >> >> (pgadmin.feature_tests.query_t >>>> ool_tests.QueryToolFeatureTest) >>>> >> >>> >> >> Query tool feature test >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> ------------------------------ >>>> ---------------------------------------- >>>> >> >>> >> >> Traceback (most recent call last): >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>> /pgadmin/feature_tests/query_tool_tests.py", >>>> >> >>> >> >> line 119, in runTest >>>> >> >>> >> >> self._query_tool_auto_rollback_enabled() >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>> /pgadmin/feature_tests/query_tool_tests.py", >>>> >> >>> >> >> line 697, in _query_tool_auto_rollback_enabled >>>> >> >>> >> >> '//div[contains(@class, "sql-editor-message") and >>>> >> >>> >> >> contains(string(), >>>> >> >>> >> >> "COMMIT")]' >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>> webdriver.py", >>>> >> >>> >> >> line 295, in find_element_by_xpath >>>> >> >>> >> >> return self.find_element(by=By.XPATH, value=xpath) >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>> webdriver.py", >>>> >> >>> >> >> line 756, in find_element >>>> >> >>> >> >> 'value': value})['value'] >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>> webdriver.py", >>>> >> >>> >> >> line 238, in execute >>>> >> >>> >> >> self.error_handler.check_response(response) >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>> errorhandler.py", >>>> >> >>> >> >> line 193, in check_response >>>> >> >>> >> >> raise exception_class(message, screen, stacktrace) >>>> >> >>> >> >> NoSuchElementException: Message: no such element: Unable to >>>> >> >>> >> >> locate >>>> >> >>> >> >> element: {"method":"xpath","selector":" >>>> //div[contains(@class, >>>> >> >>> >> >> "sql-editor-message") and contains(string(), "COMMIT")]"} >>>> >> >>> >> >> (Session info: chrome=58.0.3029.110) >>>> >> >>> >> >> (Driver info: chromedriver=2.29.461585 >>>> >> >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac >>>> OS X >>>> >> >>> >> >> 10.12.3 >>>> >> >>> >> >> x86_64) >>>> >> >>> >> >> >>>> >> >>> >> > I have updated Auto rollback enabled test in this patch. >>>> >> >>> >> > >>>> >> >>> >> >> >>>> >> >>> >> >> Relevant screenshots attached. >>>> >> >>> >> >> >>>> >> >>> >> >> - Can you tidy up the regression output a little please? >>>> Instead >>>> >> >>> >> >> of: >>>> >> >>> >> >> >>>> >> >>> >> >> ------- >>>> >> >>> >> >> runTest >>>> >> >>> >> >> (pgadmin.feature_tests.query_t >>>> ool_tests.QueryToolFeatureTest) >>>> >> >>> >> >> Query tool feature test ... On demand result set on >>>> scrolling... >>>> >> >>> >> >> OK. >>>> >> >>> >> >> >>>> >> >>> >> >> On demand result set on grid select all... >>>> >> >>> >> >> OK. >>>> >> >>> >> >> >>>> >> >>> >> >> On demand result set on column select all... >>>> >> >>> >> >> OK. >>>> >> >>> >> >> >>>> >> >>> >> >> Explain query... >>>> >> >>> >> >> OK. >>>> >> >>> >> >> >>>> >> >>> >> >> Explain query with verbose... >>>> >> >>> >> >> OK. >>>> >> >>> >> >> >>>> >> >>> >> >> Explain query with costs... >>>> >> >>> >> >> OK. >>>> >> >>> >> >> >>>> >> >>> >> >> Explain analyze query... >>>> >> >>> >> >> OK. >>>> >> >>> >> >> >>>> >> >>> >> >> Explain analyze query with buffers... >>>> >> >>> >> >> OK. >>>> >> >>> >> >> >>>> >> >>> >> >> Explain analyze query with timing... >>>> >> >>> >> >> OK. >>>> >> >>> >> >> >>>> >> >>> >> >> Auto commit disabled... >>>> >> >>> >> >> OK. >>>> >> >>> >> >> >>>> >> >>> >> >> Auto commit enabled... >>>> >> >>> >> >> OK. >>>> >> >>> >> >> >>>> >> >>> >> >> Auto rollback enabled... >>>> >> >>> >> >> ERROR >>>> >> >>> >> >> ------- >>>> >> >>> >> >> >>>> >> >>> >> >> Something like: >>>> >> >>> >> >> >>>> >> >>> >> >> ------- >>>> >> >>> >> >> runTest >>>> >> >>> >> >> (pgadmin.feature_tests.query_t >>>> ool_tests.QueryToolFeatureTest) >>>> >> >>> >> >> Query tool feature test ... >>>> >> >>> >> >> On demand result set on scrolling... OK. >>>> >> >>> >> >> On demand result set on grid select all... OK. >>>> >> >>> >> >> On demand result set on column select all... OK. >>>> >> >>> >> >> Explain query... OK. >>>> >> >>> >> >> Explain query with verbose... OK. >>>> >> >>> >> >> Explain query with costs... OK. >>>> >> >>> >> >> Explain analyze query... OK. >>>> >> >>> >> >> Explain analyze query with buffers... OK. >>>> >> >>> >> >> Explain analyze query with timing... OK. >>>> >> >>> >> >> Auto commit disabled... OK. >>>> >> >>> >> >> Auto commit enabled... OK. >>>> >> >>> >> >> Auto rollback enabled... ERROR >>>> >> >>> >> >> -------- >>>> >> >>> >> >> >>>> >> >>> >> > Fixed. >>>> >> >>> >> > >>>> >> >>> >> > >>>> >> >>> >> >> >>>> >> >>> >> >> Thanks! >>>> >> >>> >> >> >>>> >> >>> >> >> -- >>>> >> >>> >> >> Dave Page >>>> >> >>> >> >> Blog: http://pgsnake.blogspot.com >>>> >> >>> >> >> Twitter: @pgsnake >>>> >> >>> >> >> >>>> >> >>> >> >> EnterpriseDB UK: http://www.enterprisedb.com >>>> >> >>> >> >> The Enterprise PostgreSQL Company >>>> >> >>> >> > >>>> >> >>> >> > >>>> >> >>> >> >>>> >> >>> >> >>>> >> >>> >> >>>> >> >>> >> -- >>>> >> >>> >> Dave Page >>>> >> >>> >> Blog: http://pgsnake.blogspot.com >>>> >> >>> >> Twitter: @pgsnake >>>> >> >>> >> >>>> >> >>> >> EnterpriseDB UK: http://www.enterprisedb.com >>>> >> >>> >> The Enterprise PostgreSQL Company >>>> >> >>> > >>>> >> >>> > >>>> >> >>> >>>> >> >>> >>>> >> >>> >>>> >> >>> -- >>>> >> >>> Dave Page >>>> >> >>> Blog: http://pgsnake.blogspot.com >>>> >> >>> Twitter: @pgsnake >>>> >> >>> >>>> >> >>> EnterpriseDB UK: http://www.enterprisedb.com >>>> >> >>> The Enterprise PostgreSQL Company >>>> >> >> >>>> >> >> >>>> >> > >>>> >> > >>>> >> > >>>> >> > -- >>>> >> > Dave Page >>>> >> > Blog: http://pgsnake.blogspot.com >>>> >> > Twitter: @pgsnake >>>> >> > >>>> >> > EnterpriseDB UK: http://www.enterprisedb.com >>>> >> > The Enterprise PostgreSQL Company >>>> >> >>>> >> >>>> >> >>>> >> -- >>>> >> Dave Page >>>> >> Blog: http://pgsnake.blogspot.com >>>> >> Twitter: @pgsnake >>>> >> >>>> >> EnterpriseDB UK: http://www.enterprisedb.com >>>> >> The Enterprise PostgreSQL Company >>>> > >>>> > >>>> >>>> >>>> >>>> -- >>>> Dave Page >>>> Blog: http://pgsnake.blogspot.com >>>> Twitter: @pgsnake >>>> >>>> EnterpriseDB UK: http://www.enterprisedb.com >>>> The Enterprise PostgreSQL Company >>>> >>> >>> >> > > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: [pgadmin-hackers] Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] @ 2017-06-27 16:16 Khushboo Vashi <[email protected]> parent: Dave Page <[email protected]> 1 sibling, 1 reply; 24+ messages in thread From: Khushboo Vashi @ 2017-06-27 16:16 UTC (permalink / raw) To: Dave Page <[email protected]>; +Cc: Ashesh Vashi <[email protected]>; Harshal Dhumal <[email protected]>; Harshal Dhumal <[email protected]>; pgadmin-hackers On 27 Jun 2017 18:33, "Dave Page" <[email protected]> wrote: Thanks - patch committed! Awsome job :-) Gr8. Finally no more rebase request for Harshal. :) On Tue, Jun 27, 2017 at 3:26 AM, Harshal Dhumal < [email protected]> wrote: > Hi, > > Please find rebased patch. > > -- > *Harshal Dhumal* > *Sr. Software Engineer* > > EnterpriseDB India: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > > On Mon, Jun 26, 2017 at 5:24 PM, Harshal Dhumal < > [email protected]> wrote: > >> yes i'm working on that only :) >> >> >> -- >> *Harshal Dhumal* >> *Sr. Software Engineer* >> >> EnterpriseDB India: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> >> On Mon, Jun 26, 2017 at 5:22 PM, Dave Page <[email protected]> wrote: >> >>> I'm sorry, it needs rebasing again. If you can do it quickly, I'll >>> make sure it's the next patch I work on in that area. >>> >>> Thanks. >>> >>> On Mon, Jun 26, 2017 at 5:16 AM, Harshal Dhumal >>> <[email protected]> wrote: >>> > Hi Dave, >>> > >>> > Please find updated rebased patch for RM2137 >>> > >>> > On Fri, Jun 23, 2017 at 9:00 PM, Dave Page <[email protected]> wrote: >>> >> >>> >> Hi Harshal, >>> >> >>> >> When can we expect an updated version of this patch? I think it's >>> >> important to get this into the next release. >>> >> >>> >> Thanks! >>> >> >>> >> On Fri, Jun 16, 2017 at 10:55 AM, Dave Page <[email protected]> >>> wrote: >>> >> > Hi, >>> >> > >>> >> > That's better - the failures are far less random now :-). I got the >>> >> > following two though, on both PG and EPAS 9.5: >>> >> > >>> >> > ============================================================ >>> ========== >>> >> > ERROR: runTest >>> >> > (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >>> >> > Query tool feature test >>> >> > ------------------------------------------------------------ >>> ---------- >>> >> > Traceback (most recent call last): >>> >> > File >>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_t >>> ool_tests.py", >>> >> > line 95, in runTest >>> >> > self._query_tool_explain_analyze_buffers() >>> >> > File >>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_t >>> ool_tests.py", >>> >> > line 443, in _query_tool_explain_analyze_buffers >>> >> > canvas.find_element_by_xpath("//*[contains(string(), 'Shared >>> Read >>> >> > Blocks')]") >>> >> > File >>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>> ges/selenium/webdriver/remote/webelement.py", >>> >> > line 260, in find_element_by_xpath >>> >> > return self.find_element(by=By.XPATH, value=xpath) >>> >> > File >>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>> ges/selenium/webdriver/remote/webelement.py", >>> >> > line 508, in find_element >>> >> > {"using": by, "value": value})['value'] >>> >> > File >>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>> ges/selenium/webdriver/remote/webelement.py", >>> >> > line 491, in _execute >>> >> > return self._parent.execute(command, params) >>> >> > File >>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>> ges/selenium/webdriver/remote/webdriver.py", >>> >> > line 238, in execute >>> >> > self.error_handler.check_response(response) >>> >> > File >>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>> ges/selenium/webdriver/remote/errorhandler.py", >>> >> > line 193, in check_response >>> >> > raise exception_class(message, screen, stacktrace) >>> >> > NoSuchElementException: Message: no such element: Unable to locate >>> >> > element: {"method":"xpath","selector":"//*[contains(string(), >>> 'Shared >>> >> > Read Blocks')]"} >>> >> > (Session info: chrome=58.0.3029.110) >>> >> > (Driver info: chromedriver=2.29.461585 >>> >> > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X >>> 10.12.3 >>> >> > x86_64) >>> >> > >>> >> > >>> >> > ============================================================ >>> ========== >>> >> > ERROR: runTest >>> >> > (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest) >>> >> > Validate Insert, Update operations in View data with given test data >>> >> > ------------------------------------------------------------ >>> ---------- >>> >> > Traceback (most recent call last): >>> >> > File >>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>> ta_dml_queries.py", >>> >> > line 104, in runTest >>> >> > self._add_row() >>> >> > File >>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>> ta_dml_queries.py", >>> >> > line 255, in _add_row >>> >> > self._update_cell(cell_xpath, config_data[str(idx)]) >>> >> > File >>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>> ta_dml_queries.py", >>> >> > line 164, in _update_cell >>> >> > cell_el = self.page.find_by_xpath(xpath) >>> >> > File >>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>> min_page.py", >>> >> > line 122, in find_by_xpath >>> >> > return self.wait_for_element(lambda driver: >>> >> > driver.find_element_by_xpath(xpath)) >>> >> > File >>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>> min_page.py", >>> >> > line 205, in wait_for_element >>> >> > return self._wait_for("element to exist", element_if_it_exists) >>> >> > File >>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>> min_page.py", >>> >> > line 255, in _wait_for >>> >> > "Timed out waiting for " + waiting_for_message) >>> >> > File >>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>> ges/selenium/webdriver/support/wait.py", >>> >> > line 71, in until >>> >> > value = method(self._driver) >>> >> > File >>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>> min_page.py", >>> >> > line 200, in element_if_it_exists >>> >> > if element.is_displayed() and element.is_enabled(): >>> >> > File >>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>> ges/selenium/webdriver/remote/webelement.py", >>> >> > line 358, in is_displayed >>> >> > return self._execute(Command.IS_ELEMENT_DISPLAYED)['value'] >>> >> > File >>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>> ges/selenium/webdriver/remote/webelement.py", >>> >> > line 491, in _execute >>> >> > return self._parent.execute(command, params) >>> >> > File >>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>> ges/selenium/webdriver/remote/webdriver.py", >>> >> > line 238, in execute >>> >> > self.error_handler.check_response(response) >>> >> > File >>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>> ges/selenium/webdriver/remote/errorhandler.py", >>> >> > line 193, in check_response >>> >> > raise exception_class(message, screen, stacktrace) >>> >> > StaleElementReferenceException: Message: stale element reference: >>> >> > element is not attached to the page document >>> >> > (Session info: chrome=58.0.3029.110) >>> >> > (Driver info: chromedriver=2.29.461585 >>> >> > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X >>> 10.12.3 >>> >> > x86_64) >>> >> > >>> > >>> > >>> > I checked my feature test cases for any database server version/type >>> (PG, >>> > EPAS) specific failures and I found that test cases are working fine >>> on all >>> > of them. However I have slightly modified test case to overcome above >>> > mentioned failures. >>> > >>> >> >>> >> > >>> >> > On 9.4 everything passes. On 9.6, I get one failure: >>> >> > >>> >> > ============================================================ >>> ========== >>> >> > FAIL: runTest >>> >> > (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest) >>> >> > Validate Insert, Update operations in View data with given test data >>> >> > ------------------------------------------------------------ >>> ---------- >>> >> > Traceback (most recent call last): >>> >> > File >>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>> ta_dml_queries.py", >>> >> > line 105, in runTest >>> >> > self._verify_row_data(True) >>> >> > File >>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>> ta_dml_queries.py", >>> >> > line 282, in _verify_row_data >>> >> > self.assertEquals(cells[idx], config_data[str(idx)][1]) >>> >> > AssertionError: u'[default]' != u'1' >>> >> > - [default] >>> >> > + 1 >>> >> > >>> > >>> > I guess Khushboo has fixed this recently. >>> > >>> > >>> >> >>> >> > Screenshots attached. >>> >> > >>> >> > On Fri, Jun 16, 2017 at 7:54 AM, Harshal Dhumal >>> >> > <[email protected]> wrote: >>> >> >> Hi Dave, >>> >> >> >>> >> >> Please find attached patch where I have added timeout of 2 seconds >>> >> >> before >>> >> >> selecting object menu in connect to server function. >>> >> >> >>> >> >> Note: Apply this patch on top of previous patch for on demand >>> loading >>> >> >> feature. >>> >> >> >>> >> >> >>> >> >> -- >>> >> >> Harshal Dhumal >>> >> >> Sr. Software Engineer >>> >> >> >>> >> >> EnterpriseDB India: http://www.enterprisedb.com >>> >> >> The Enterprise PostgreSQL Company >>> >> >> >>> >> >> On Fri, Jun 16, 2017 at 2:25 AM, Dave Page <[email protected]> >>> wrote: >>> >> >>> >>> >> >>> Sounds good, thanks. >>> >> >>> >>> >> >>> On Thu, Jun 15, 2017 at 9:54 PM, Harshal Dhumal >>> >> >>> <[email protected]> wrote: >>> >> >>> > Hi >>> >> >>> > >>> >> >>> > On Fri, Jun 16, 2017 at 2:07 AM, Dave Page <[email protected]> >>> >> >>> > wrote: >>> >> >>> >> >>> >> >>> >> Hi >>> >> >>> >> >>> >> >>> >> On Thu, Jun 15, 2017 at 9:30 PM, Harshal Dhumal >>> >> >>> >> <[email protected]> wrote: >>> >> >>> >> > Hi Dave, >>> >> >>> >> > >>> >> >>> >> > Please find attached updated patch. >>> >> >>> >> > >>> >> >>> >> > On Thu, Jun 15, 2017 at 3:58 PM, Dave Page < >>> [email protected]> >>> >> >>> >> > wrote: >>> >> >>> >> >> >>> >> >>> >> >> Hi >>> >> >>> >> >> >>> >> >>> >> >> On Wed, Jun 14, 2017 at 11:36 PM, Harshal Dhumal >>> >> >>> >> >> <[email protected]> wrote: >>> >> >>> >> >>> >>> >> >>> >> >>> Hi Dave, >>> >> >>> >> >>> >>> >> >>> >> >>> Please find rebased patch for RM2137. >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> Looking very good. The only issues I see are: >>> >> >>> >> >> >>> >> >>> >> >> - The row headers should auto-size such that they can >>> display >>> >> >>> >> >> the >>> >> >>> >> >> row >>> >> >>> >> >> numbers if the last row was displayed. E.g. if there are >>> 12345 >>> >> >>> >> >> rows >>> >> >>> >> >> in >>> >> >>> >> >> total, then the row header should be sized to display 5 >>> digits. >>> >> >>> >> >> >>> >> >>> >> > Fixed. >>> >> >>> >> > >>> >> >>> >> > >>> >> >>> >> >> >>> >> >>> >> >> - The tests are comprehensive, which is awesome. However, >>> every >>> >> >>> >> >> time >>> >> >>> >> >> I >>> >> >>> >> >> ran >>> >> >>> >> >> them, at least one of the feature tests failed. >>> Unfortunately, >>> >> >>> >> >> it >>> >> >>> >> >> was a >>> >> >>> >> >> different one each time. In the last two runs, I got: >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> ============================== >>> ======================================== >>> >> >>> >> >> ERROR: runTest >>> >> >>> >> >> (pgadmin.feature_tests.query_t >>> ool_tests.QueryToolFeatureTest) >>> >> >>> >> >> Query tool feature test >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> ------------------------------ >>> ---------------------------------------- >>> >> >>> >> >> Traceback (most recent call last): >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>> /regression/feature_utils/base_feature_test.py", >>> >> >>> >> >> line 40, in setUp >>> >> >>> >> >> self.before() >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>> /pgadmin/feature_tests/query_tool_tests.py", >>> >> >>> >> >> line 40, in before >>> >> >>> >> >> self._connects_to_server() >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>> /pgadmin/feature_tests/query_tool_tests.py", >>> >> >>> >> >> line 144, in _connects_to_server >>> >> >>> >> >> self.page.driver.find_element_by_link_text("Create"))\ >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>> webdriver.py", >>> >> >>> >> >> line 319, in find_element_by_link_text >>> >> >>> >> >> return self.find_element(by=By.LINK_TEXT, >>> value=link_text) >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>> webdriver.py", >>> >> >>> >> >> line 756, in find_element >>> >> >>> >> >> 'value': value})['value'] >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>> webdriver.py", >>> >> >>> >> >> line 238, in execute >>> >> >>> >> >> self.error_handler.check_response(response) >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>> errorhandler.py", >>> >> >>> >> >> line 193, in check_response >>> >> >>> >> >> raise exception_class(message, screen, stacktrace) >>> >> >>> >> >> NoSuchElementException: Message: no such element: Unable to >>> >> >>> >> >> locate >>> >> >>> >> >> element: {"method":"link text","selector":"Create"} >>> >> >>> >> >> (Session info: chrome=58.0.3029.110) >>> >> >>> >> >> (Driver info: chromedriver=2.29.461585 >>> >> >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS >>> X >>> >> >>> >> >> 10.12.3 >>> >> >>> >> >> x86_64) >>> >> >>> >> >> >>> >> >>> >> > I checked code from _connects_to_server function which is >>> common >>> >> >>> >> > in >>> >> >>> >> > all >>> >> >>> >> > features test cases. I didn't find anything wrong with this. >>> If >>> >> >>> >> > there >>> >> >>> >> > is >>> >> >>> >> > a >>> >> >>> >> > bug in this function then all feature test must fail. >>> >> >>> >> > Let me know if you are getting failure consistently in >>> >> >>> >> > _connects_to_server >>> >> >>> >> > function. >>> >> >>> >> >>> >> >>> >> I wondered if that one is a race condition. Do we need a short >>> >> >>> >> delay >>> >> >>> >> before clicking the Object menu? I have seen this occasionally >>> >> >>> >> before. >>> >> >>> > >>> >> >>> > >>> >> >>> > OK. In that case let's try putting 1-2 second delay and observer >>> >> >>> > behaviour. >>> >> >>> > I'll send separate patch for this tomorrow as this is not >>> related to >>> >> >>> > on >>> >> >>> > demand query result feature or its test cases. >>> >> >>> > >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> ============================== >>> ======================================== >>> >> >>> >> >> ERROR: runTest >>> >> >>> >> >> (pgadmin.feature_tests.query_t >>> ool_tests.QueryToolFeatureTest) >>> >> >>> >> >> Query tool feature test >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> ------------------------------ >>> ---------------------------------------- >>> >> >>> >> >> Traceback (most recent call last): >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>> /pgadmin/feature_tests/query_tool_tests.py", >>> >> >>> >> >> line 119, in runTest >>> >> >>> >> >> self._query_tool_auto_rollback_enabled() >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>> /pgadmin/feature_tests/query_tool_tests.py", >>> >> >>> >> >> line 697, in _query_tool_auto_rollback_enabled >>> >> >>> >> >> '//div[contains(@class, "sql-editor-message") and >>> >> >>> >> >> contains(string(), >>> >> >>> >> >> "COMMIT")]' >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>> webdriver.py", >>> >> >>> >> >> line 295, in find_element_by_xpath >>> >> >>> >> >> return self.find_element(by=By.XPATH, value=xpath) >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>> webdriver.py", >>> >> >>> >> >> line 756, in find_element >>> >> >>> >> >> 'value': value})['value'] >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>> webdriver.py", >>> >> >>> >> >> line 238, in execute >>> >> >>> >> >> self.error_handler.check_response(response) >>> >> >>> >> >> File >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>> errorhandler.py", >>> >> >>> >> >> line 193, in check_response >>> >> >>> >> >> raise exception_class(message, screen, stacktrace) >>> >> >>> >> >> NoSuchElementException: Message: no such element: Unable to >>> >> >>> >> >> locate >>> >> >>> >> >> element: {"method":"xpath","selector":" >>> //div[contains(@class, >>> >> >>> >> >> "sql-editor-message") and contains(string(), "COMMIT")]"} >>> >> >>> >> >> (Session info: chrome=58.0.3029.110) >>> >> >>> >> >> (Driver info: chromedriver=2.29.461585 >>> >> >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS >>> X >>> >> >>> >> >> 10.12.3 >>> >> >>> >> >> x86_64) >>> >> >>> >> >> >>> >> >>> >> > I have updated Auto rollback enabled test in this patch. >>> >> >>> >> > >>> >> >>> >> >> >>> >> >>> >> >> Relevant screenshots attached. >>> >> >>> >> >> >>> >> >>> >> >> - Can you tidy up the regression output a little please? >>> Instead >>> >> >>> >> >> of: >>> >> >>> >> >> >>> >> >>> >> >> ------- >>> >> >>> >> >> runTest >>> >> >>> >> >> (pgadmin.feature_tests.query_t >>> ool_tests.QueryToolFeatureTest) >>> >> >>> >> >> Query tool feature test ... On demand result set on >>> scrolling... >>> >> >>> >> >> OK. >>> >> >>> >> >> >>> >> >>> >> >> On demand result set on grid select all... >>> >> >>> >> >> OK. >>> >> >>> >> >> >>> >> >>> >> >> On demand result set on column select all... >>> >> >>> >> >> OK. >>> >> >>> >> >> >>> >> >>> >> >> Explain query... >>> >> >>> >> >> OK. >>> >> >>> >> >> >>> >> >>> >> >> Explain query with verbose... >>> >> >>> >> >> OK. >>> >> >>> >> >> >>> >> >>> >> >> Explain query with costs... >>> >> >>> >> >> OK. >>> >> >>> >> >> >>> >> >>> >> >> Explain analyze query... >>> >> >>> >> >> OK. >>> >> >>> >> >> >>> >> >>> >> >> Explain analyze query with buffers... >>> >> >>> >> >> OK. >>> >> >>> >> >> >>> >> >>> >> >> Explain analyze query with timing... >>> >> >>> >> >> OK. >>> >> >>> >> >> >>> >> >>> >> >> Auto commit disabled... >>> >> >>> >> >> OK. >>> >> >>> >> >> >>> >> >>> >> >> Auto commit enabled... >>> >> >>> >> >> OK. >>> >> >>> >> >> >>> >> >>> >> >> Auto rollback enabled... >>> >> >>> >> >> ERROR >>> >> >>> >> >> ------- >>> >> >>> >> >> >>> >> >>> >> >> Something like: >>> >> >>> >> >> >>> >> >>> >> >> ------- >>> >> >>> >> >> runTest >>> >> >>> >> >> (pgadmin.feature_tests.query_t >>> ool_tests.QueryToolFeatureTest) >>> >> >>> >> >> Query tool feature test ... >>> >> >>> >> >> On demand result set on scrolling... OK. >>> >> >>> >> >> On demand result set on grid select all... OK. >>> >> >>> >> >> On demand result set on column select all... OK. >>> >> >>> >> >> Explain query... OK. >>> >> >>> >> >> Explain query with verbose... OK. >>> >> >>> >> >> Explain query with costs... OK. >>> >> >>> >> >> Explain analyze query... OK. >>> >> >>> >> >> Explain analyze query with buffers... OK. >>> >> >>> >> >> Explain analyze query with timing... OK. >>> >> >>> >> >> Auto commit disabled... OK. >>> >> >>> >> >> Auto commit enabled... OK. >>> >> >>> >> >> Auto rollback enabled... ERROR >>> >> >>> >> >> -------- >>> >> >>> >> >> >>> >> >>> >> > Fixed. >>> >> >>> >> > >>> >> >>> >> > >>> >> >>> >> >> >>> >> >>> >> >> Thanks! >>> >> >>> >> >> >>> >> >>> >> >> -- >>> >> >>> >> >> Dave Page >>> >> >>> >> >> Blog: http://pgsnake.blogspot.com >>> >> >>> >> >> Twitter: @pgsnake >>> >> >>> >> >> >>> >> >>> >> >> EnterpriseDB UK: http://www.enterprisedb.com >>> >> >>> >> >> The Enterprise PostgreSQL Company >>> >> >>> >> > >>> >> >>> >> > >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> -- >>> >> >>> >> Dave Page >>> >> >>> >> Blog: http://pgsnake.blogspot.com >>> >> >>> >> Twitter: @pgsnake >>> >> >>> >> >>> >> >>> >> EnterpriseDB UK: http://www.enterprisedb.com >>> >> >>> >> The Enterprise PostgreSQL Company >>> >> >>> > >>> >> >>> > >>> >> >>> >>> >> >>> >>> >> >>> >>> >> >>> -- >>> >> >>> Dave Page >>> >> >>> Blog: http://pgsnake.blogspot.com >>> >> >>> Twitter: @pgsnake >>> >> >>> >>> >> >>> EnterpriseDB UK: http://www.enterprisedb.com >>> >> >>> The Enterprise PostgreSQL Company >>> >> >> >>> >> >> >>> >> > >>> >> > >>> >> > >>> >> > -- >>> >> > Dave Page >>> >> > Blog: http://pgsnake.blogspot.com >>> >> > Twitter: @pgsnake >>> >> > >>> >> > EnterpriseDB UK: http://www.enterprisedb.com >>> >> > The Enterprise PostgreSQL Company >>> >> >>> >> >>> >> >>> >> -- >>> >> Dave Page >>> >> Blog: http://pgsnake.blogspot.com >>> >> Twitter: @pgsnake >>> >> >>> >> EnterpriseDB UK: http://www.enterprisedb.com >>> >> The Enterprise PostgreSQL Company >>> > >>> > >>> >>> >>> >>> -- >>> Dave Page >>> Blog: http://pgsnake.blogspot.com >>> Twitter: @pgsnake >>> >>> EnterpriseDB UK: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >> >> > -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: [pgadmin-hackers] Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] @ 2017-06-27 16:19 Robert Eckhardt <[email protected]> parent: Khushboo Vashi <[email protected]> 0 siblings, 1 reply; 24+ messages in thread From: Robert Eckhardt @ 2017-06-27 16:19 UTC (permalink / raw) To: Khushboo Vashi <[email protected]>; +Cc: Dave Page <[email protected]>; Ashesh Vashi <[email protected]>; Harshal Dhumal <[email protected]>; Harshal Dhumal <[email protected]>; pgadmin-hackers On Tue, Jun 27, 2017 at 12:16 PM, Khushboo Vashi < [email protected]> wrote: > > > On 27 Jun 2017 18:33, "Dave Page" <[email protected]> wrote: > > Thanks - patch committed! > > Awsome job :-) > > Gr8. Finally no more rebase request for Harshal. :) > +1 > On Tue, Jun 27, 2017 at 3:26 AM, Harshal Dhumal < > [email protected]> wrote: > >> Hi, >> >> Please find rebased patch. >> >> -- >> *Harshal Dhumal* >> *Sr. Software Engineer* >> >> EnterpriseDB India: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> >> On Mon, Jun 26, 2017 at 5:24 PM, Harshal Dhumal < >> [email protected]> wrote: >> >>> yes i'm working on that only :) >>> >>> >>> -- >>> *Harshal Dhumal* >>> *Sr. Software Engineer* >>> >>> EnterpriseDB India: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >>> On Mon, Jun 26, 2017 at 5:22 PM, Dave Page <[email protected]> wrote: >>> >>>> I'm sorry, it needs rebasing again. If you can do it quickly, I'll >>>> make sure it's the next patch I work on in that area. >>>> >>>> Thanks. >>>> >>>> On Mon, Jun 26, 2017 at 5:16 AM, Harshal Dhumal >>>> <[email protected]> wrote: >>>> > Hi Dave, >>>> > >>>> > Please find updated rebased patch for RM2137 >>>> > >>>> > On Fri, Jun 23, 2017 at 9:00 PM, Dave Page <[email protected]> wrote: >>>> >> >>>> >> Hi Harshal, >>>> >> >>>> >> When can we expect an updated version of this patch? I think it's >>>> >> important to get this into the next release. >>>> >> >>>> >> Thanks! >>>> >> >>>> >> On Fri, Jun 16, 2017 at 10:55 AM, Dave Page <[email protected]> >>>> wrote: >>>> >> > Hi, >>>> >> > >>>> >> > That's better - the failures are far less random now :-). I got the >>>> >> > following two though, on both PG and EPAS 9.5: >>>> >> > >>>> >> > ============================================================ >>>> ========== >>>> >> > ERROR: runTest >>>> >> > (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >>>> >> > Query tool feature test >>>> >> > ------------------------------------------------------------ >>>> ---------- >>>> >> > Traceback (most recent call last): >>>> >> > File >>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_t >>>> ool_tests.py", >>>> >> > line 95, in runTest >>>> >> > self._query_tool_explain_analyze_buffers() >>>> >> > File >>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_t >>>> ool_tests.py", >>>> >> > line 443, in _query_tool_explain_analyze_buffers >>>> >> > canvas.find_element_by_xpath("//*[contains(string(), 'Shared >>>> Read >>>> >> > Blocks')]") >>>> >> > File >>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>> ges/selenium/webdriver/remote/webelement.py", >>>> >> > line 260, in find_element_by_xpath >>>> >> > return self.find_element(by=By.XPATH, value=xpath) >>>> >> > File >>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>> ges/selenium/webdriver/remote/webelement.py", >>>> >> > line 508, in find_element >>>> >> > {"using": by, "value": value})['value'] >>>> >> > File >>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>> ges/selenium/webdriver/remote/webelement.py", >>>> >> > line 491, in _execute >>>> >> > return self._parent.execute(command, params) >>>> >> > File >>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>> ges/selenium/webdriver/remote/webdriver.py", >>>> >> > line 238, in execute >>>> >> > self.error_handler.check_response(response) >>>> >> > File >>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>> ges/selenium/webdriver/remote/errorhandler.py", >>>> >> > line 193, in check_response >>>> >> > raise exception_class(message, screen, stacktrace) >>>> >> > NoSuchElementException: Message: no such element: Unable to locate >>>> >> > element: {"method":"xpath","selector":"//*[contains(string(), >>>> 'Shared >>>> >> > Read Blocks')]"} >>>> >> > (Session info: chrome=58.0.3029.110) >>>> >> > (Driver info: chromedriver=2.29.461585 >>>> >> > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X >>>> 10.12.3 >>>> >> > x86_64) >>>> >> > >>>> >> > >>>> >> > ============================================================ >>>> ========== >>>> >> > ERROR: runTest >>>> >> > (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest) >>>> >> > Validate Insert, Update operations in View data with given test >>>> data >>>> >> > ------------------------------------------------------------ >>>> ---------- >>>> >> > Traceback (most recent call last): >>>> >> > File >>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>> ta_dml_queries.py", >>>> >> > line 104, in runTest >>>> >> > self._add_row() >>>> >> > File >>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>> ta_dml_queries.py", >>>> >> > line 255, in _add_row >>>> >> > self._update_cell(cell_xpath, config_data[str(idx)]) >>>> >> > File >>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>> ta_dml_queries.py", >>>> >> > line 164, in _update_cell >>>> >> > cell_el = self.page.find_by_xpath(xpath) >>>> >> > File >>>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>>> min_page.py", >>>> >> > line 122, in find_by_xpath >>>> >> > return self.wait_for_element(lambda driver: >>>> >> > driver.find_element_by_xpath(xpath)) >>>> >> > File >>>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>>> min_page.py", >>>> >> > line 205, in wait_for_element >>>> >> > return self._wait_for("element to exist", element_if_it_exists) >>>> >> > File >>>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>>> min_page.py", >>>> >> > line 255, in _wait_for >>>> >> > "Timed out waiting for " + waiting_for_message) >>>> >> > File >>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>> ges/selenium/webdriver/support/wait.py", >>>> >> > line 71, in until >>>> >> > value = method(self._driver) >>>> >> > File >>>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>>> min_page.py", >>>> >> > line 200, in element_if_it_exists >>>> >> > if element.is_displayed() and element.is_enabled(): >>>> >> > File >>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>> ges/selenium/webdriver/remote/webelement.py", >>>> >> > line 358, in is_displayed >>>> >> > return self._execute(Command.IS_ELEMENT_DISPLAYED)['value'] >>>> >> > File >>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>> ges/selenium/webdriver/remote/webelement.py", >>>> >> > line 491, in _execute >>>> >> > return self._parent.execute(command, params) >>>> >> > File >>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>> ges/selenium/webdriver/remote/webdriver.py", >>>> >> > line 238, in execute >>>> >> > self.error_handler.check_response(response) >>>> >> > File >>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>> ges/selenium/webdriver/remote/errorhandler.py", >>>> >> > line 193, in check_response >>>> >> > raise exception_class(message, screen, stacktrace) >>>> >> > StaleElementReferenceException: Message: stale element reference: >>>> >> > element is not attached to the page document >>>> >> > (Session info: chrome=58.0.3029.110) >>>> >> > (Driver info: chromedriver=2.29.461585 >>>> >> > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X >>>> 10.12.3 >>>> >> > x86_64) >>>> >> > >>>> > >>>> > >>>> > I checked my feature test cases for any database server version/type >>>> (PG, >>>> > EPAS) specific failures and I found that test cases are working fine >>>> on all >>>> > of them. However I have slightly modified test case to overcome above >>>> > mentioned failures. >>>> > >>>> >> >>>> >> > >>>> >> > On 9.4 everything passes. On 9.6, I get one failure: >>>> >> > >>>> >> > ============================================================ >>>> ========== >>>> >> > FAIL: runTest >>>> >> > (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDataTest) >>>> >> > Validate Insert, Update operations in View data with given test >>>> data >>>> >> > ------------------------------------------------------------ >>>> ---------- >>>> >> > Traceback (most recent call last): >>>> >> > File >>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>> ta_dml_queries.py", >>>> >> > line 105, in runTest >>>> >> > self._verify_row_data(True) >>>> >> > File >>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>> ta_dml_queries.py", >>>> >> > line 282, in _verify_row_data >>>> >> > self.assertEquals(cells[idx], config_data[str(idx)][1]) >>>> >> > AssertionError: u'[default]' != u'1' >>>> >> > - [default] >>>> >> > + 1 >>>> >> > >>>> > >>>> > I guess Khushboo has fixed this recently. >>>> > >>>> > >>>> >> >>>> >> > Screenshots attached. >>>> >> > >>>> >> > On Fri, Jun 16, 2017 at 7:54 AM, Harshal Dhumal >>>> >> > <[email protected]> wrote: >>>> >> >> Hi Dave, >>>> >> >> >>>> >> >> Please find attached patch where I have added timeout of 2 seconds >>>> >> >> before >>>> >> >> selecting object menu in connect to server function. >>>> >> >> >>>> >> >> Note: Apply this patch on top of previous patch for on demand >>>> loading >>>> >> >> feature. >>>> >> >> >>>> >> >> >>>> >> >> -- >>>> >> >> Harshal Dhumal >>>> >> >> Sr. Software Engineer >>>> >> >> >>>> >> >> EnterpriseDB India: http://www.enterprisedb.com >>>> >> >> The Enterprise PostgreSQL Company >>>> >> >> >>>> >> >> On Fri, Jun 16, 2017 at 2:25 AM, Dave Page <[email protected]> >>>> wrote: >>>> >> >>> >>>> >> >>> Sounds good, thanks. >>>> >> >>> >>>> >> >>> On Thu, Jun 15, 2017 at 9:54 PM, Harshal Dhumal >>>> >> >>> <[email protected]> wrote: >>>> >> >>> > Hi >>>> >> >>> > >>>> >> >>> > On Fri, Jun 16, 2017 at 2:07 AM, Dave Page <[email protected]> >>>> >> >>> > wrote: >>>> >> >>> >> >>>> >> >>> >> Hi >>>> >> >>> >> >>>> >> >>> >> On Thu, Jun 15, 2017 at 9:30 PM, Harshal Dhumal >>>> >> >>> >> <[email protected]> wrote: >>>> >> >>> >> > Hi Dave, >>>> >> >>> >> > >>>> >> >>> >> > Please find attached updated patch. >>>> >> >>> >> > >>>> >> >>> >> > On Thu, Jun 15, 2017 at 3:58 PM, Dave Page < >>>> [email protected]> >>>> >> >>> >> > wrote: >>>> >> >>> >> >> >>>> >> >>> >> >> Hi >>>> >> >>> >> >> >>>> >> >>> >> >> On Wed, Jun 14, 2017 at 11:36 PM, Harshal Dhumal >>>> >> >>> >> >> <[email protected]> wrote: >>>> >> >>> >> >>> >>>> >> >>> >> >>> Hi Dave, >>>> >> >>> >> >>> >>>> >> >>> >> >>> Please find rebased patch for RM2137. >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> Looking very good. The only issues I see are: >>>> >> >>> >> >> >>>> >> >>> >> >> - The row headers should auto-size such that they can >>>> display >>>> >> >>> >> >> the >>>> >> >>> >> >> row >>>> >> >>> >> >> numbers if the last row was displayed. E.g. if there are >>>> 12345 >>>> >> >>> >> >> rows >>>> >> >>> >> >> in >>>> >> >>> >> >> total, then the row header should be sized to display 5 >>>> digits. >>>> >> >>> >> >> >>>> >> >>> >> > Fixed. >>>> >> >>> >> > >>>> >> >>> >> > >>>> >> >>> >> >> >>>> >> >>> >> >> - The tests are comprehensive, which is awesome. However, >>>> every >>>> >> >>> >> >> time >>>> >> >>> >> >> I >>>> >> >>> >> >> ran >>>> >> >>> >> >> them, at least one of the feature tests failed. >>>> Unfortunately, >>>> >> >>> >> >> it >>>> >> >>> >> >> was a >>>> >> >>> >> >> different one each time. In the last two runs, I got: >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> ============================== >>>> ======================================== >>>> >> >>> >> >> ERROR: runTest >>>> >> >>> >> >> (pgadmin.feature_tests.query_t >>>> ool_tests.QueryToolFeatureTest) >>>> >> >>> >> >> Query tool feature test >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> ------------------------------ >>>> ---------------------------------------- >>>> >> >>> >> >> Traceback (most recent call last): >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>> /regression/feature_utils/base_feature_test.py", >>>> >> >>> >> >> line 40, in setUp >>>> >> >>> >> >> self.before() >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>> /pgadmin/feature_tests/query_tool_tests.py", >>>> >> >>> >> >> line 40, in before >>>> >> >>> >> >> self._connects_to_server() >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>> /pgadmin/feature_tests/query_tool_tests.py", >>>> >> >>> >> >> line 144, in _connects_to_server >>>> >> >>> >> >> self.page.driver.find_element_by_link_text("Create"))\ >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>> webdriver.py", >>>> >> >>> >> >> line 319, in find_element_by_link_text >>>> >> >>> >> >> return self.find_element(by=By.LINK_TEXT, >>>> value=link_text) >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>> webdriver.py", >>>> >> >>> >> >> line 756, in find_element >>>> >> >>> >> >> 'value': value})['value'] >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>> webdriver.py", >>>> >> >>> >> >> line 238, in execute >>>> >> >>> >> >> self.error_handler.check_response(response) >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>> errorhandler.py", >>>> >> >>> >> >> line 193, in check_response >>>> >> >>> >> >> raise exception_class(message, screen, stacktrace) >>>> >> >>> >> >> NoSuchElementException: Message: no such element: Unable to >>>> >> >>> >> >> locate >>>> >> >>> >> >> element: {"method":"link text","selector":"Create"} >>>> >> >>> >> >> (Session info: chrome=58.0.3029.110) >>>> >> >>> >> >> (Driver info: chromedriver=2.29.461585 >>>> >> >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac >>>> OS X >>>> >> >>> >> >> 10.12.3 >>>> >> >>> >> >> x86_64) >>>> >> >>> >> >> >>>> >> >>> >> > I checked code from _connects_to_server function which is >>>> common >>>> >> >>> >> > in >>>> >> >>> >> > all >>>> >> >>> >> > features test cases. I didn't find anything wrong with >>>> this. If >>>> >> >>> >> > there >>>> >> >>> >> > is >>>> >> >>> >> > a >>>> >> >>> >> > bug in this function then all feature test must fail. >>>> >> >>> >> > Let me know if you are getting failure consistently in >>>> >> >>> >> > _connects_to_server >>>> >> >>> >> > function. >>>> >> >>> >> >>>> >> >>> >> I wondered if that one is a race condition. Do we need a short >>>> >> >>> >> delay >>>> >> >>> >> before clicking the Object menu? I have seen this occasionally >>>> >> >>> >> before. >>>> >> >>> > >>>> >> >>> > >>>> >> >>> > OK. In that case let's try putting 1-2 second delay and >>>> observer >>>> >> >>> > behaviour. >>>> >> >>> > I'll send separate patch for this tomorrow as this is not >>>> related to >>>> >> >>> > on >>>> >> >>> > demand query result feature or its test cases. >>>> >> >>> > >>>> >> >>> >> >>>> >> >>> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> ============================== >>>> ======================================== >>>> >> >>> >> >> ERROR: runTest >>>> >> >>> >> >> (pgadmin.feature_tests.query_t >>>> ool_tests.QueryToolFeatureTest) >>>> >> >>> >> >> Query tool feature test >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> ------------------------------ >>>> ---------------------------------------- >>>> >> >>> >> >> Traceback (most recent call last): >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>> /pgadmin/feature_tests/query_tool_tests.py", >>>> >> >>> >> >> line 119, in runTest >>>> >> >>> >> >> self._query_tool_auto_rollback_enabled() >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>> /pgadmin/feature_tests/query_tool_tests.py", >>>> >> >>> >> >> line 697, in _query_tool_auto_rollback_enabled >>>> >> >>> >> >> '//div[contains(@class, "sql-editor-message") and >>>> >> >>> >> >> contains(string(), >>>> >> >>> >> >> "COMMIT")]' >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>> webdriver.py", >>>> >> >>> >> >> line 295, in find_element_by_xpath >>>> >> >>> >> >> return self.find_element(by=By.XPATH, value=xpath) >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>> webdriver.py", >>>> >> >>> >> >> line 756, in find_element >>>> >> >>> >> >> 'value': value})['value'] >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>> webdriver.py", >>>> >> >>> >> >> line 238, in execute >>>> >> >>> >> >> self.error_handler.check_response(response) >>>> >> >>> >> >> File >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>> errorhandler.py", >>>> >> >>> >> >> line 193, in check_response >>>> >> >>> >> >> raise exception_class(message, screen, stacktrace) >>>> >> >>> >> >> NoSuchElementException: Message: no such element: Unable to >>>> >> >>> >> >> locate >>>> >> >>> >> >> element: {"method":"xpath","selector":" >>>> //div[contains(@class, >>>> >> >>> >> >> "sql-editor-message") and contains(string(), "COMMIT")]"} >>>> >> >>> >> >> (Session info: chrome=58.0.3029.110) >>>> >> >>> >> >> (Driver info: chromedriver=2.29.461585 >>>> >> >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac >>>> OS X >>>> >> >>> >> >> 10.12.3 >>>> >> >>> >> >> x86_64) >>>> >> >>> >> >> >>>> >> >>> >> > I have updated Auto rollback enabled test in this patch. >>>> >> >>> >> > >>>> >> >>> >> >> >>>> >> >>> >> >> Relevant screenshots attached. >>>> >> >>> >> >> >>>> >> >>> >> >> - Can you tidy up the regression output a little please? >>>> Instead >>>> >> >>> >> >> of: >>>> >> >>> >> >> >>>> >> >>> >> >> ------- >>>> >> >>> >> >> runTest >>>> >> >>> >> >> (pgadmin.feature_tests.query_t >>>> ool_tests.QueryToolFeatureTest) >>>> >> >>> >> >> Query tool feature test ... On demand result set on >>>> scrolling... >>>> >> >>> >> >> OK. >>>> >> >>> >> >> >>>> >> >>> >> >> On demand result set on grid select all... >>>> >> >>> >> >> OK. >>>> >> >>> >> >> >>>> >> >>> >> >> On demand result set on column select all... >>>> >> >>> >> >> OK. >>>> >> >>> >> >> >>>> >> >>> >> >> Explain query... >>>> >> >>> >> >> OK. >>>> >> >>> >> >> >>>> >> >>> >> >> Explain query with verbose... >>>> >> >>> >> >> OK. >>>> >> >>> >> >> >>>> >> >>> >> >> Explain query with costs... >>>> >> >>> >> >> OK. >>>> >> >>> >> >> >>>> >> >>> >> >> Explain analyze query... >>>> >> >>> >> >> OK. >>>> >> >>> >> >> >>>> >> >>> >> >> Explain analyze query with buffers... >>>> >> >>> >> >> OK. >>>> >> >>> >> >> >>>> >> >>> >> >> Explain analyze query with timing... >>>> >> >>> >> >> OK. >>>> >> >>> >> >> >>>> >> >>> >> >> Auto commit disabled... >>>> >> >>> >> >> OK. >>>> >> >>> >> >> >>>> >> >>> >> >> Auto commit enabled... >>>> >> >>> >> >> OK. >>>> >> >>> >> >> >>>> >> >>> >> >> Auto rollback enabled... >>>> >> >>> >> >> ERROR >>>> >> >>> >> >> ------- >>>> >> >>> >> >> >>>> >> >>> >> >> Something like: >>>> >> >>> >> >> >>>> >> >>> >> >> ------- >>>> >> >>> >> >> runTest >>>> >> >>> >> >> (pgadmin.feature_tests.query_t >>>> ool_tests.QueryToolFeatureTest) >>>> >> >>> >> >> Query tool feature test ... >>>> >> >>> >> >> On demand result set on scrolling... OK. >>>> >> >>> >> >> On demand result set on grid select all... OK. >>>> >> >>> >> >> On demand result set on column select all... OK. >>>> >> >>> >> >> Explain query... OK. >>>> >> >>> >> >> Explain query with verbose... OK. >>>> >> >>> >> >> Explain query with costs... OK. >>>> >> >>> >> >> Explain analyze query... OK. >>>> >> >>> >> >> Explain analyze query with buffers... OK. >>>> >> >>> >> >> Explain analyze query with timing... OK. >>>> >> >>> >> >> Auto commit disabled... OK. >>>> >> >>> >> >> Auto commit enabled... OK. >>>> >> >>> >> >> Auto rollback enabled... ERROR >>>> >> >>> >> >> -------- >>>> >> >>> >> >> >>>> >> >>> >> > Fixed. >>>> >> >>> >> > >>>> >> >>> >> > >>>> >> >>> >> >> >>>> >> >>> >> >> Thanks! >>>> >> >>> >> >> >>>> >> >>> >> >> -- >>>> >> >>> >> >> Dave Page >>>> >> >>> >> >> Blog: http://pgsnake.blogspot.com >>>> >> >>> >> >> Twitter: @pgsnake >>>> >> >>> >> >> >>>> >> >>> >> >> EnterpriseDB UK: http://www.enterprisedb.com >>>> >> >>> >> >> The Enterprise PostgreSQL Company >>>> >> >>> >> > >>>> >> >>> >> > >>>> >> >>> >> >>>> >> >>> >> >>>> >> >>> >> >>>> >> >>> >> -- >>>> >> >>> >> Dave Page >>>> >> >>> >> Blog: http://pgsnake.blogspot.com >>>> >> >>> >> Twitter: @pgsnake >>>> >> >>> >> >>>> >> >>> >> EnterpriseDB UK: http://www.enterprisedb.com >>>> >> >>> >> The Enterprise PostgreSQL Company >>>> >> >>> > >>>> >> >>> > >>>> >> >>> >>>> >> >>> >>>> >> >>> >>>> >> >>> -- >>>> >> >>> Dave Page >>>> >> >>> Blog: http://pgsnake.blogspot.com >>>> >> >>> Twitter: @pgsnake >>>> >> >>> >>>> >> >>> EnterpriseDB UK: http://www.enterprisedb.com >>>> >> >>> The Enterprise PostgreSQL Company >>>> >> >> >>>> >> >> >>>> >> > >>>> >> > >>>> >> > >>>> >> > -- >>>> >> > Dave Page >>>> >> > Blog: http://pgsnake.blogspot.com >>>> >> > Twitter: @pgsnake >>>> >> > >>>> >> > EnterpriseDB UK: http://www.enterprisedb.com >>>> >> > The Enterprise PostgreSQL Company >>>> >> >>>> >> >>>> >> >>>> >> -- >>>> >> Dave Page >>>> >> Blog: http://pgsnake.blogspot.com >>>> >> Twitter: @pgsnake >>>> >> >>>> >> EnterpriseDB UK: http://www.enterprisedb.com >>>> >> The Enterprise PostgreSQL Company >>>> > >>>> > >>>> >>>> >>>> >>>> -- >>>> Dave Page >>>> Blog: http://pgsnake.blogspot.com >>>> Twitter: @pgsnake >>>> >>>> EnterpriseDB UK: http://www.enterprisedb.com >>>> The Enterprise PostgreSQL Company >>>> >>> >>> >> > > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > > > ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: [pgadmin-hackers] Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] @ 2017-06-27 21:03 Joao Pedro De Almeida Pereira <[email protected]> parent: Robert Eckhardt <[email protected]> 0 siblings, 2 replies; 24+ messages in thread From: Joao Pedro De Almeida Pereira @ 2017-06-27 21:03 UTC (permalink / raw) To: pgadmin-hackers; +Cc: Dave Page <[email protected]>; Ashesh Vashi <[email protected]>; Harshal Dhumal <[email protected]>; Robert Eckhardt <[email protected]>; Shirley Wang <[email protected]> Hello Hackers, When we started the app we noticed some change in the front end. The line numbers in the Editor: - We noticed a bug with the numbers where at 10000 rows, the numbers would be cut off. - When the row is selected the color of the text should be white - Centering the number column would differentiate it from the rest of the table and prevent any confusion [image: Inline image 1] Select all triangle: - This seems to have shifted up and to the left slightly. While editing data, when you paste a new row, that line should come into view. Issue in the code: - In sqleditor/command.py:473 the variable `column_data` is not initialized We were looking into the menu's to access and Edit Data editor, and we noticed that in the menu it is still called "View Data". We are aware that this change was not introduced in this patch, but we just noticed it. It should be consistent. [image: Inline image 4] Thanks Shirley & Joao On Tue, Jun 27, 2017 at 12:19 PM, Robert Eckhardt <[email protected]> wrote: > > > On Tue, Jun 27, 2017 at 12:16 PM, Khushboo Vashi < > [email protected]> wrote: > >> >> >> On 27 Jun 2017 18:33, "Dave Page" <[email protected]> wrote: >> >> Thanks - patch committed! >> >> Awsome job :-) >> >> Gr8. Finally no more rebase request for Harshal. :) >> > > +1 > > >> On Tue, Jun 27, 2017 at 3:26 AM, Harshal Dhumal < >> [email protected]> wrote: >> >>> Hi, >>> >>> Please find rebased patch. >>> >>> -- >>> *Harshal Dhumal* >>> *Sr. Software Engineer* >>> >>> EnterpriseDB India: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >>> On Mon, Jun 26, 2017 at 5:24 PM, Harshal Dhumal < >>> [email protected]> wrote: >>> >>>> yes i'm working on that only :) >>>> >>>> >>>> -- >>>> *Harshal Dhumal* >>>> *Sr. Software Engineer* >>>> >>>> EnterpriseDB India: http://www.enterprisedb.com >>>> The Enterprise PostgreSQL Company >>>> >>>> On Mon, Jun 26, 2017 at 5:22 PM, Dave Page <[email protected]> wrote: >>>> >>>>> I'm sorry, it needs rebasing again. If you can do it quickly, I'll >>>>> make sure it's the next patch I work on in that area. >>>>> >>>>> Thanks. >>>>> >>>>> On Mon, Jun 26, 2017 at 5:16 AM, Harshal Dhumal >>>>> <[email protected]> wrote: >>>>> > Hi Dave, >>>>> > >>>>> > Please find updated rebased patch for RM2137 >>>>> > >>>>> > On Fri, Jun 23, 2017 at 9:00 PM, Dave Page <[email protected]> >>>>> wrote: >>>>> >> >>>>> >> Hi Harshal, >>>>> >> >>>>> >> When can we expect an updated version of this patch? I think it's >>>>> >> important to get this into the next release. >>>>> >> >>>>> >> Thanks! >>>>> >> >>>>> >> On Fri, Jun 16, 2017 at 10:55 AM, Dave Page <[email protected]> >>>>> wrote: >>>>> >> > Hi, >>>>> >> > >>>>> >> > That's better - the failures are far less random now :-). I got >>>>> the >>>>> >> > following two though, on both PG and EPAS 9.5: >>>>> >> > >>>>> >> > ============================================================ >>>>> ========== >>>>> >> > ERROR: runTest >>>>> >> > (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >>>>> >> > Query tool feature test >>>>> >> > ------------------------------------------------------------ >>>>> ---------- >>>>> >> > Traceback (most recent call last): >>>>> >> > File >>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_t >>>>> ool_tests.py", >>>>> >> > line 95, in runTest >>>>> >> > self._query_tool_explain_analyze_buffers() >>>>> >> > File >>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_t >>>>> ool_tests.py", >>>>> >> > line 443, in _query_tool_explain_analyze_buffers >>>>> >> > canvas.find_element_by_xpath("//*[contains(string(), 'Shared >>>>> Read >>>>> >> > Blocks')]") >>>>> >> > File >>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>> ges/selenium/webdriver/remote/webelement.py", >>>>> >> > line 260, in find_element_by_xpath >>>>> >> > return self.find_element(by=By.XPATH, value=xpath) >>>>> >> > File >>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>> ges/selenium/webdriver/remote/webelement.py", >>>>> >> > line 508, in find_element >>>>> >> > {"using": by, "value": value})['value'] >>>>> >> > File >>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>> ges/selenium/webdriver/remote/webelement.py", >>>>> >> > line 491, in _execute >>>>> >> > return self._parent.execute(command, params) >>>>> >> > File >>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>> ges/selenium/webdriver/remote/webdriver.py", >>>>> >> > line 238, in execute >>>>> >> > self.error_handler.check_response(response) >>>>> >> > File >>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>> ges/selenium/webdriver/remote/errorhandler.py", >>>>> >> > line 193, in check_response >>>>> >> > raise exception_class(message, screen, stacktrace) >>>>> >> > NoSuchElementException: Message: no such element: Unable to locate >>>>> >> > element: {"method":"xpath","selector":"//*[contains(string(), >>>>> 'Shared >>>>> >> > Read Blocks')]"} >>>>> >> > (Session info: chrome=58.0.3029.110) >>>>> >> > (Driver info: chromedriver=2.29.461585 >>>>> >> > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X >>>>> 10.12.3 >>>>> >> > x86_64) >>>>> >> > >>>>> >> > >>>>> >> > ============================================================ >>>>> ========== >>>>> >> > ERROR: runTest >>>>> >> > (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDat >>>>> aTest) >>>>> >> > Validate Insert, Update operations in View data with given test >>>>> data >>>>> >> > ------------------------------------------------------------ >>>>> ---------- >>>>> >> > Traceback (most recent call last): >>>>> >> > File >>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>>> ta_dml_queries.py", >>>>> >> > line 104, in runTest >>>>> >> > self._add_row() >>>>> >> > File >>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>>> ta_dml_queries.py", >>>>> >> > line 255, in _add_row >>>>> >> > self._update_cell(cell_xpath, config_data[str(idx)]) >>>>> >> > File >>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>>> ta_dml_queries.py", >>>>> >> > line 164, in _update_cell >>>>> >> > cell_el = self.page.find_by_xpath(xpath) >>>>> >> > File >>>>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>>>> min_page.py", >>>>> >> > line 122, in find_by_xpath >>>>> >> > return self.wait_for_element(lambda driver: >>>>> >> > driver.find_element_by_xpath(xpath)) >>>>> >> > File >>>>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>>>> min_page.py", >>>>> >> > line 205, in wait_for_element >>>>> >> > return self._wait_for("element to exist", >>>>> element_if_it_exists) >>>>> >> > File >>>>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>>>> min_page.py", >>>>> >> > line 255, in _wait_for >>>>> >> > "Timed out waiting for " + waiting_for_message) >>>>> >> > File >>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>> ges/selenium/webdriver/support/wait.py", >>>>> >> > line 71, in until >>>>> >> > value = method(self._driver) >>>>> >> > File >>>>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>>>> min_page.py", >>>>> >> > line 200, in element_if_it_exists >>>>> >> > if element.is_displayed() and element.is_enabled(): >>>>> >> > File >>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>> ges/selenium/webdriver/remote/webelement.py", >>>>> >> > line 358, in is_displayed >>>>> >> > return self._execute(Command.IS_ELEMENT_DISPLAYED)['value'] >>>>> >> > File >>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>> ges/selenium/webdriver/remote/webelement.py", >>>>> >> > line 491, in _execute >>>>> >> > return self._parent.execute(command, params) >>>>> >> > File >>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>> ges/selenium/webdriver/remote/webdriver.py", >>>>> >> > line 238, in execute >>>>> >> > self.error_handler.check_response(response) >>>>> >> > File >>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>> ges/selenium/webdriver/remote/errorhandler.py", >>>>> >> > line 193, in check_response >>>>> >> > raise exception_class(message, screen, stacktrace) >>>>> >> > StaleElementReferenceException: Message: stale element reference: >>>>> >> > element is not attached to the page document >>>>> >> > (Session info: chrome=58.0.3029.110) >>>>> >> > (Driver info: chromedriver=2.29.461585 >>>>> >> > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X >>>>> 10.12.3 >>>>> >> > x86_64) >>>>> >> > >>>>> > >>>>> > >>>>> > I checked my feature test cases for any database server version/type >>>>> (PG, >>>>> > EPAS) specific failures and I found that test cases are working fine >>>>> on all >>>>> > of them. However I have slightly modified test case to overcome above >>>>> > mentioned failures. >>>>> > >>>>> >> >>>>> >> > >>>>> >> > On 9.4 everything passes. On 9.6, I get one failure: >>>>> >> > >>>>> >> > ============================================================ >>>>> ========== >>>>> >> > FAIL: runTest >>>>> >> > (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDat >>>>> aTest) >>>>> >> > Validate Insert, Update operations in View data with given test >>>>> data >>>>> >> > ------------------------------------------------------------ >>>>> ---------- >>>>> >> > Traceback (most recent call last): >>>>> >> > File >>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>>> ta_dml_queries.py", >>>>> >> > line 105, in runTest >>>>> >> > self._verify_row_data(True) >>>>> >> > File >>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>>> ta_dml_queries.py", >>>>> >> > line 282, in _verify_row_data >>>>> >> > self.assertEquals(cells[idx], config_data[str(idx)][1]) >>>>> >> > AssertionError: u'[default]' != u'1' >>>>> >> > - [default] >>>>> >> > + 1 >>>>> >> > >>>>> > >>>>> > I guess Khushboo has fixed this recently. >>>>> > >>>>> > >>>>> >> >>>>> >> > Screenshots attached. >>>>> >> > >>>>> >> > On Fri, Jun 16, 2017 at 7:54 AM, Harshal Dhumal >>>>> >> > <[email protected]> wrote: >>>>> >> >> Hi Dave, >>>>> >> >> >>>>> >> >> Please find attached patch where I have added timeout of 2 >>>>> seconds >>>>> >> >> before >>>>> >> >> selecting object menu in connect to server function. >>>>> >> >> >>>>> >> >> Note: Apply this patch on top of previous patch for on demand >>>>> loading >>>>> >> >> feature. >>>>> >> >> >>>>> >> >> >>>>> >> >> -- >>>>> >> >> Harshal Dhumal >>>>> >> >> Sr. Software Engineer >>>>> >> >> >>>>> >> >> EnterpriseDB India: http://www.enterprisedb.com >>>>> >> >> The Enterprise PostgreSQL Company >>>>> >> >> >>>>> >> >> On Fri, Jun 16, 2017 at 2:25 AM, Dave Page <[email protected]> >>>>> wrote: >>>>> >> >>> >>>>> >> >>> Sounds good, thanks. >>>>> >> >>> >>>>> >> >>> On Thu, Jun 15, 2017 at 9:54 PM, Harshal Dhumal >>>>> >> >>> <[email protected]> wrote: >>>>> >> >>> > Hi >>>>> >> >>> > >>>>> >> >>> > On Fri, Jun 16, 2017 at 2:07 AM, Dave Page <[email protected] >>>>> > >>>>> >> >>> > wrote: >>>>> >> >>> >> >>>>> >> >>> >> Hi >>>>> >> >>> >> >>>>> >> >>> >> On Thu, Jun 15, 2017 at 9:30 PM, Harshal Dhumal >>>>> >> >>> >> <[email protected]> wrote: >>>>> >> >>> >> > Hi Dave, >>>>> >> >>> >> > >>>>> >> >>> >> > Please find attached updated patch. >>>>> >> >>> >> > >>>>> >> >>> >> > On Thu, Jun 15, 2017 at 3:58 PM, Dave Page < >>>>> [email protected]> >>>>> >> >>> >> > wrote: >>>>> >> >>> >> >> >>>>> >> >>> >> >> Hi >>>>> >> >>> >> >> >>>>> >> >>> >> >> On Wed, Jun 14, 2017 at 11:36 PM, Harshal Dhumal >>>>> >> >>> >> >> <[email protected]> wrote: >>>>> >> >>> >> >>> >>>>> >> >>> >> >>> Hi Dave, >>>>> >> >>> >> >>> >>>>> >> >>> >> >>> Please find rebased patch for RM2137. >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> Looking very good. The only issues I see are: >>>>> >> >>> >> >> >>>>> >> >>> >> >> - The row headers should auto-size such that they can >>>>> display >>>>> >> >>> >> >> the >>>>> >> >>> >> >> row >>>>> >> >>> >> >> numbers if the last row was displayed. E.g. if there are >>>>> 12345 >>>>> >> >>> >> >> rows >>>>> >> >>> >> >> in >>>>> >> >>> >> >> total, then the row header should be sized to display 5 >>>>> digits. >>>>> >> >>> >> >> >>>>> >> >>> >> > Fixed. >>>>> >> >>> >> > >>>>> >> >>> >> > >>>>> >> >>> >> >> >>>>> >> >>> >> >> - The tests are comprehensive, which is awesome. However, >>>>> every >>>>> >> >>> >> >> time >>>>> >> >>> >> >> I >>>>> >> >>> >> >> ran >>>>> >> >>> >> >> them, at least one of the feature tests failed. >>>>> Unfortunately, >>>>> >> >>> >> >> it >>>>> >> >>> >> >> was a >>>>> >> >>> >> >> different one each time. In the last two runs, I got: >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> ============================== >>>>> ======================================== >>>>> >> >>> >> >> ERROR: runTest >>>>> >> >>> >> >> (pgadmin.feature_tests.query_t >>>>> ool_tests.QueryToolFeatureTest) >>>>> >> >>> >> >> Query tool feature test >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> ------------------------------ >>>>> ---------------------------------------- >>>>> >> >>> >> >> Traceback (most recent call last): >>>>> >> >>> >> >> File >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>>> /regression/feature_utils/base_feature_test.py", >>>>> >> >>> >> >> line 40, in setUp >>>>> >> >>> >> >> self.before() >>>>> >> >>> >> >> File >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>>> /pgadmin/feature_tests/query_tool_tests.py", >>>>> >> >>> >> >> line 40, in before >>>>> >> >>> >> >> self._connects_to_server() >>>>> >> >>> >> >> File >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>>> /pgadmin/feature_tests/query_tool_tests.py", >>>>> >> >>> >> >> line 144, in _connects_to_server >>>>> >> >>> >> >> self.page.driver.find_element >>>>> _by_link_text("Create"))\ >>>>> >> >>> >> >> File >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>> webdriver.py", >>>>> >> >>> >> >> line 319, in find_element_by_link_text >>>>> >> >>> >> >> return self.find_element(by=By.LINK_TEXT, >>>>> value=link_text) >>>>> >> >>> >> >> File >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>> webdriver.py", >>>>> >> >>> >> >> line 756, in find_element >>>>> >> >>> >> >> 'value': value})['value'] >>>>> >> >>> >> >> File >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>> webdriver.py", >>>>> >> >>> >> >> line 238, in execute >>>>> >> >>> >> >> self.error_handler.check_response(response) >>>>> >> >>> >> >> File >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>> errorhandler.py", >>>>> >> >>> >> >> line 193, in check_response >>>>> >> >>> >> >> raise exception_class(message, screen, stacktrace) >>>>> >> >>> >> >> NoSuchElementException: Message: no such element: Unable >>>>> to >>>>> >> >>> >> >> locate >>>>> >> >>> >> >> element: {"method":"link text","selector":"Create"} >>>>> >> >>> >> >> (Session info: chrome=58.0.3029.110) >>>>> >> >>> >> >> (Driver info: chromedriver=2.29.461585 >>>>> >> >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac >>>>> OS X >>>>> >> >>> >> >> 10.12.3 >>>>> >> >>> >> >> x86_64) >>>>> >> >>> >> >> >>>>> >> >>> >> > I checked code from _connects_to_server function which is >>>>> common >>>>> >> >>> >> > in >>>>> >> >>> >> > all >>>>> >> >>> >> > features test cases. I didn't find anything wrong with >>>>> this. If >>>>> >> >>> >> > there >>>>> >> >>> >> > is >>>>> >> >>> >> > a >>>>> >> >>> >> > bug in this function then all feature test must fail. >>>>> >> >>> >> > Let me know if you are getting failure consistently in >>>>> >> >>> >> > _connects_to_server >>>>> >> >>> >> > function. >>>>> >> >>> >> >>>>> >> >>> >> I wondered if that one is a race condition. Do we need a >>>>> short >>>>> >> >>> >> delay >>>>> >> >>> >> before clicking the Object menu? I have seen this >>>>> occasionally >>>>> >> >>> >> before. >>>>> >> >>> > >>>>> >> >>> > >>>>> >> >>> > OK. In that case let's try putting 1-2 second delay and >>>>> observer >>>>> >> >>> > behaviour. >>>>> >> >>> > I'll send separate patch for this tomorrow as this is not >>>>> related to >>>>> >> >>> > on >>>>> >> >>> > demand query result feature or its test cases. >>>>> >> >>> > >>>>> >> >>> >> >>>>> >> >>> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> ============================== >>>>> ======================================== >>>>> >> >>> >> >> ERROR: runTest >>>>> >> >>> >> >> (pgadmin.feature_tests.query_t >>>>> ool_tests.QueryToolFeatureTest) >>>>> >> >>> >> >> Query tool feature test >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> ------------------------------ >>>>> ---------------------------------------- >>>>> >> >>> >> >> Traceback (most recent call last): >>>>> >> >>> >> >> File >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>>> /pgadmin/feature_tests/query_tool_tests.py", >>>>> >> >>> >> >> line 119, in runTest >>>>> >> >>> >> >> self._query_tool_auto_rollback_enabled() >>>>> >> >>> >> >> File >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>>> /pgadmin/feature_tests/query_tool_tests.py", >>>>> >> >>> >> >> line 697, in _query_tool_auto_rollback_enabled >>>>> >> >>> >> >> '//div[contains(@class, "sql-editor-message") and >>>>> >> >>> >> >> contains(string(), >>>>> >> >>> >> >> "COMMIT")]' >>>>> >> >>> >> >> File >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>> webdriver.py", >>>>> >> >>> >> >> line 295, in find_element_by_xpath >>>>> >> >>> >> >> return self.find_element(by=By.XPATH, value=xpath) >>>>> >> >>> >> >> File >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>> webdriver.py", >>>>> >> >>> >> >> line 756, in find_element >>>>> >> >>> >> >> 'value': value})['value'] >>>>> >> >>> >> >> File >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>> webdriver.py", >>>>> >> >>> >> >> line 238, in execute >>>>> >> >>> >> >> self.error_handler.check_response(response) >>>>> >> >>> >> >> File >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> >>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>> errorhandler.py", >>>>> >> >>> >> >> line 193, in check_response >>>>> >> >>> >> >> raise exception_class(message, screen, stacktrace) >>>>> >> >>> >> >> NoSuchElementException: Message: no such element: Unable >>>>> to >>>>> >> >>> >> >> locate >>>>> >> >>> >> >> element: {"method":"xpath","selector":" >>>>> //div[contains(@class, >>>>> >> >>> >> >> "sql-editor-message") and contains(string(), "COMMIT")]"} >>>>> >> >>> >> >> (Session info: chrome=58.0.3029.110) >>>>> >> >>> >> >> (Driver info: chromedriver=2.29.461585 >>>>> >> >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac >>>>> OS X >>>>> >> >>> >> >> 10.12.3 >>>>> >> >>> >> >> x86_64) >>>>> >> >>> >> >> >>>>> >> >>> >> > I have updated Auto rollback enabled test in this patch. >>>>> >> >>> >> > >>>>> >> >>> >> >> >>>>> >> >>> >> >> Relevant screenshots attached. >>>>> >> >>> >> >> >>>>> >> >>> >> >> - Can you tidy up the regression output a little please? >>>>> Instead >>>>> >> >>> >> >> of: >>>>> >> >>> >> >> >>>>> >> >>> >> >> ------- >>>>> >> >>> >> >> runTest >>>>> >> >>> >> >> (pgadmin.feature_tests.query_t >>>>> ool_tests.QueryToolFeatureTest) >>>>> >> >>> >> >> Query tool feature test ... On demand result set on >>>>> scrolling... >>>>> >> >>> >> >> OK. >>>>> >> >>> >> >> >>>>> >> >>> >> >> On demand result set on grid select all... >>>>> >> >>> >> >> OK. >>>>> >> >>> >> >> >>>>> >> >>> >> >> On demand result set on column select all... >>>>> >> >>> >> >> OK. >>>>> >> >>> >> >> >>>>> >> >>> >> >> Explain query... >>>>> >> >>> >> >> OK. >>>>> >> >>> >> >> >>>>> >> >>> >> >> Explain query with verbose... >>>>> >> >>> >> >> OK. >>>>> >> >>> >> >> >>>>> >> >>> >> >> Explain query with costs... >>>>> >> >>> >> >> OK. >>>>> >> >>> >> >> >>>>> >> >>> >> >> Explain analyze query... >>>>> >> >>> >> >> OK. >>>>> >> >>> >> >> >>>>> >> >>> >> >> Explain analyze query with buffers... >>>>> >> >>> >> >> OK. >>>>> >> >>> >> >> >>>>> >> >>> >> >> Explain analyze query with timing... >>>>> >> >>> >> >> OK. >>>>> >> >>> >> >> >>>>> >> >>> >> >> Auto commit disabled... >>>>> >> >>> >> >> OK. >>>>> >> >>> >> >> >>>>> >> >>> >> >> Auto commit enabled... >>>>> >> >>> >> >> OK. >>>>> >> >>> >> >> >>>>> >> >>> >> >> Auto rollback enabled... >>>>> >> >>> >> >> ERROR >>>>> >> >>> >> >> ------- >>>>> >> >>> >> >> >>>>> >> >>> >> >> Something like: >>>>> >> >>> >> >> >>>>> >> >>> >> >> ------- >>>>> >> >>> >> >> runTest >>>>> >> >>> >> >> (pgadmin.feature_tests.query_t >>>>> ool_tests.QueryToolFeatureTest) >>>>> >> >>> >> >> Query tool feature test ... >>>>> >> >>> >> >> On demand result set on scrolling... OK. >>>>> >> >>> >> >> On demand result set on grid select all... OK. >>>>> >> >>> >> >> On demand result set on column select all... OK. >>>>> >> >>> >> >> Explain query... OK. >>>>> >> >>> >> >> Explain query with verbose... OK. >>>>> >> >>> >> >> Explain query with costs... OK. >>>>> >> >>> >> >> Explain analyze query... OK. >>>>> >> >>> >> >> Explain analyze query with buffers... OK. >>>>> >> >>> >> >> Explain analyze query with timing... OK. >>>>> >> >>> >> >> Auto commit disabled... OK. >>>>> >> >>> >> >> Auto commit enabled... OK. >>>>> >> >>> >> >> Auto rollback enabled... ERROR >>>>> >> >>> >> >> -------- >>>>> >> >>> >> >> >>>>> >> >>> >> > Fixed. >>>>> >> >>> >> > >>>>> >> >>> >> > >>>>> >> >>> >> >> >>>>> >> >>> >> >> Thanks! >>>>> >> >>> >> >> >>>>> >> >>> >> >> -- >>>>> >> >>> >> >> Dave Page >>>>> >> >>> >> >> Blog: http://pgsnake.blogspot.com >>>>> >> >>> >> >> Twitter: @pgsnake >>>>> >> >>> >> >> >>>>> >> >>> >> >> EnterpriseDB UK: http://www.enterprisedb.com >>>>> >> >>> >> >> The Enterprise PostgreSQL Company >>>>> >> >>> >> > >>>>> >> >>> >> > >>>>> >> >>> >> >>>>> >> >>> >> >>>>> >> >>> >> >>>>> >> >>> >> -- >>>>> >> >>> >> Dave Page >>>>> >> >>> >> Blog: http://pgsnake.blogspot.com >>>>> >> >>> >> Twitter: @pgsnake >>>>> >> >>> >> >>>>> >> >>> >> EnterpriseDB UK: http://www.enterprisedb.com >>>>> >> >>> >> The Enterprise PostgreSQL Company >>>>> >> >>> > >>>>> >> >>> > >>>>> >> >>> >>>>> >> >>> >>>>> >> >>> >>>>> >> >>> -- >>>>> >> >>> Dave Page >>>>> >> >>> Blog: http://pgsnake.blogspot.com >>>>> >> >>> Twitter: @pgsnake >>>>> >> >>> >>>>> >> >>> EnterpriseDB UK: http://www.enterprisedb.com >>>>> >> >>> The Enterprise PostgreSQL Company >>>>> >> >> >>>>> >> >> >>>>> >> > >>>>> >> > >>>>> >> > >>>>> >> > -- >>>>> >> > Dave Page >>>>> >> > Blog: http://pgsnake.blogspot.com >>>>> >> > Twitter: @pgsnake >>>>> >> > >>>>> >> > EnterpriseDB UK: http://www.enterprisedb.com >>>>> >> > The Enterprise PostgreSQL Company >>>>> >> >>>>> >> >>>>> >> >>>>> >> -- >>>>> >> Dave Page >>>>> >> Blog: http://pgsnake.blogspot.com >>>>> >> Twitter: @pgsnake >>>>> >> >>>>> >> EnterpriseDB UK: http://www.enterprisedb.com >>>>> >> The Enterprise PostgreSQL Company >>>>> > >>>>> > >>>>> >>>>> >>>>> >>>>> -- >>>>> Dave Page >>>>> Blog: http://pgsnake.blogspot.com >>>>> Twitter: @pgsnake >>>>> >>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>> The Enterprise PostgreSQL Company >>>>> >>>> >>>> >>> >> >> >> -- >> Dave Page >> Blog: http://pgsnake.blogspot.com >> Twitter: @pgsnake >> >> EnterpriseDB UK: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> >> >> > Attachments: [image/png] Screen Shot 2017-06-27 at 5.00.35 PM.png (126.8K, 3-Screen%20Shot%202017-06-27%20at%205.00.35%20PM.png) download | view image [image/png] Screen Shot 2017-06-27 at 4.55.16 PM.png (38.4K, 4-Screen%20Shot%202017-06-27%20at%204.55.16%20PM.png) download | view image ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: [pgadmin-hackers] Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] @ 2017-06-27 21:13 Dave Page <[email protected]> parent: Joao Pedro De Almeida Pereira <[email protected]> 1 sibling, 1 reply; 24+ messages in thread From: Dave Page @ 2017-06-27 21:13 UTC (permalink / raw) To: Joao Pedro De Almeida Pereira <[email protected]>; +Cc: pgadmin-hackers; Ashesh Vashi <[email protected]>; Harshal Dhumal <[email protected]>; Robert Eckhardt <[email protected]>; Shirley Wang <[email protected]> Harshal, can you look at these comments please, and patch as appropriate? On Tue, Jun 27, 2017 at 5:03 PM, Joao Pedro De Almeida Pereira < [email protected]> wrote: > Hello Hackers, > > When we started the app we noticed some change in the front end. > > The line numbers in the Editor: > - We noticed a bug with the numbers where at 10000 rows, the numbers would > be cut off. > - When the row is selected the color of the text should be white > - Centering the number column would differentiate it from the rest of the > table and prevent any confusion > [image: Inline image 1] > > Select all triangle: > - This seems to have shifted up and to the left slightly. > > While editing data, when you paste a new row, that line should come into > view. > > Issue in the code: > - In sqleditor/command.py:473 the variable `column_data` is not > initialized > > > > We were looking into the menu's to access and Edit Data editor, and we > noticed that in the menu it is still called "View Data". We are aware that > this change was not introduced in this patch, but we just noticed it. It > should be consistent. > [image: Inline image 4] > > > Thanks > Shirley & Joao > > > On Tue, Jun 27, 2017 at 12:19 PM, Robert Eckhardt <[email protected]> > wrote: > >> >> >> On Tue, Jun 27, 2017 at 12:16 PM, Khushboo Vashi < >> [email protected]> wrote: >> >>> >>> >>> On 27 Jun 2017 18:33, "Dave Page" <[email protected]> wrote: >>> >>> Thanks - patch committed! >>> >>> Awsome job :-) >>> >>> Gr8. Finally no more rebase request for Harshal. :) >>> >> >> +1 >> >> >>> On Tue, Jun 27, 2017 at 3:26 AM, Harshal Dhumal < >>> [email protected]> wrote: >>> >>>> Hi, >>>> >>>> Please find rebased patch. >>>> >>>> -- >>>> *Harshal Dhumal* >>>> *Sr. Software Engineer* >>>> >>>> EnterpriseDB India: http://www.enterprisedb.com >>>> The Enterprise PostgreSQL Company >>>> >>>> On Mon, Jun 26, 2017 at 5:24 PM, Harshal Dhumal < >>>> [email protected]> wrote: >>>> >>>>> yes i'm working on that only :) >>>>> >>>>> >>>>> -- >>>>> *Harshal Dhumal* >>>>> *Sr. Software Engineer* >>>>> >>>>> EnterpriseDB India: http://www.enterprisedb.com >>>>> The Enterprise PostgreSQL Company >>>>> >>>>> On Mon, Jun 26, 2017 at 5:22 PM, Dave Page <[email protected]> wrote: >>>>> >>>>>> I'm sorry, it needs rebasing again. If you can do it quickly, I'll >>>>>> make sure it's the next patch I work on in that area. >>>>>> >>>>>> Thanks. >>>>>> >>>>>> On Mon, Jun 26, 2017 at 5:16 AM, Harshal Dhumal >>>>>> <[email protected]> wrote: >>>>>> > Hi Dave, >>>>>> > >>>>>> > Please find updated rebased patch for RM2137 >>>>>> > >>>>>> > On Fri, Jun 23, 2017 at 9:00 PM, Dave Page <[email protected]> >>>>>> wrote: >>>>>> >> >>>>>> >> Hi Harshal, >>>>>> >> >>>>>> >> When can we expect an updated version of this patch? I think it's >>>>>> >> important to get this into the next release. >>>>>> >> >>>>>> >> Thanks! >>>>>> >> >>>>>> >> On Fri, Jun 16, 2017 at 10:55 AM, Dave Page <[email protected]> >>>>>> wrote: >>>>>> >> > Hi, >>>>>> >> > >>>>>> >> > That's better - the failures are far less random now :-). I got >>>>>> the >>>>>> >> > following two though, on both PG and EPAS 9.5: >>>>>> >> > >>>>>> >> > ============================================================ >>>>>> ========== >>>>>> >> > ERROR: runTest >>>>>> >> > (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >>>>>> >> > Query tool feature test >>>>>> >> > ------------------------------------------------------------ >>>>>> ---------- >>>>>> >> > Traceback (most recent call last): >>>>>> >> > File >>>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_t >>>>>> ool_tests.py", >>>>>> >> > line 95, in runTest >>>>>> >> > self._query_tool_explain_analyze_buffers() >>>>>> >> > File >>>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_t >>>>>> ool_tests.py", >>>>>> >> > line 443, in _query_tool_explain_analyze_buffers >>>>>> >> > canvas.find_element_by_xpath("//*[contains(string(), >>>>>> 'Shared Read >>>>>> >> > Blocks')]") >>>>>> >> > File >>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>> ges/selenium/webdriver/remote/webelement.py", >>>>>> >> > line 260, in find_element_by_xpath >>>>>> >> > return self.find_element(by=By.XPATH, value=xpath) >>>>>> >> > File >>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>> ges/selenium/webdriver/remote/webelement.py", >>>>>> >> > line 508, in find_element >>>>>> >> > {"using": by, "value": value})['value'] >>>>>> >> > File >>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>> ges/selenium/webdriver/remote/webelement.py", >>>>>> >> > line 491, in _execute >>>>>> >> > return self._parent.execute(command, params) >>>>>> >> > File >>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>> ges/selenium/webdriver/remote/webdriver.py", >>>>>> >> > line 238, in execute >>>>>> >> > self.error_handler.check_response(response) >>>>>> >> > File >>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>> ges/selenium/webdriver/remote/errorhandler.py", >>>>>> >> > line 193, in check_response >>>>>> >> > raise exception_class(message, screen, stacktrace) >>>>>> >> > NoSuchElementException: Message: no such element: Unable to >>>>>> locate >>>>>> >> > element: {"method":"xpath","selector":"//*[contains(string(), >>>>>> 'Shared >>>>>> >> > Read Blocks')]"} >>>>>> >> > (Session info: chrome=58.0.3029.110) >>>>>> >> > (Driver info: chromedriver=2.29.461585 >>>>>> >> > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X >>>>>> 10.12.3 >>>>>> >> > x86_64) >>>>>> >> > >>>>>> >> > >>>>>> >> > ============================================================ >>>>>> ========== >>>>>> >> > ERROR: runTest >>>>>> >> > (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDat >>>>>> aTest) >>>>>> >> > Validate Insert, Update operations in View data with given test >>>>>> data >>>>>> >> > ------------------------------------------------------------ >>>>>> ---------- >>>>>> >> > Traceback (most recent call last): >>>>>> >> > File >>>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>>>> ta_dml_queries.py", >>>>>> >> > line 104, in runTest >>>>>> >> > self._add_row() >>>>>> >> > File >>>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>>>> ta_dml_queries.py", >>>>>> >> > line 255, in _add_row >>>>>> >> > self._update_cell(cell_xpath, config_data[str(idx)]) >>>>>> >> > File >>>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>>>> ta_dml_queries.py", >>>>>> >> > line 164, in _update_cell >>>>>> >> > cell_el = self.page.find_by_xpath(xpath) >>>>>> >> > File >>>>>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>>>>> min_page.py", >>>>>> >> > line 122, in find_by_xpath >>>>>> >> > return self.wait_for_element(lambda driver: >>>>>> >> > driver.find_element_by_xpath(xpath)) >>>>>> >> > File >>>>>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>>>>> min_page.py", >>>>>> >> > line 205, in wait_for_element >>>>>> >> > return self._wait_for("element to exist", >>>>>> element_if_it_exists) >>>>>> >> > File >>>>>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>>>>> min_page.py", >>>>>> >> > line 255, in _wait_for >>>>>> >> > "Timed out waiting for " + waiting_for_message) >>>>>> >> > File >>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>> ges/selenium/webdriver/support/wait.py", >>>>>> >> > line 71, in until >>>>>> >> > value = method(self._driver) >>>>>> >> > File >>>>>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>>>>> min_page.py", >>>>>> >> > line 200, in element_if_it_exists >>>>>> >> > if element.is_displayed() and element.is_enabled(): >>>>>> >> > File >>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>> ges/selenium/webdriver/remote/webelement.py", >>>>>> >> > line 358, in is_displayed >>>>>> >> > return self._execute(Command.IS_ELEMENT_DISPLAYED)['value'] >>>>>> >> > File >>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>> ges/selenium/webdriver/remote/webelement.py", >>>>>> >> > line 491, in _execute >>>>>> >> > return self._parent.execute(command, params) >>>>>> >> > File >>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>> ges/selenium/webdriver/remote/webdriver.py", >>>>>> >> > line 238, in execute >>>>>> >> > self.error_handler.check_response(response) >>>>>> >> > File >>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>> ges/selenium/webdriver/remote/errorhandler.py", >>>>>> >> > line 193, in check_response >>>>>> >> > raise exception_class(message, screen, stacktrace) >>>>>> >> > StaleElementReferenceException: Message: stale element >>>>>> reference: >>>>>> >> > element is not attached to the page document >>>>>> >> > (Session info: chrome=58.0.3029.110) >>>>>> >> > (Driver info: chromedriver=2.29.461585 >>>>>> >> > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X >>>>>> 10.12.3 >>>>>> >> > x86_64) >>>>>> >> > >>>>>> > >>>>>> > >>>>>> > I checked my feature test cases for any database server >>>>>> version/type (PG, >>>>>> > EPAS) specific failures and I found that test cases are working >>>>>> fine on all >>>>>> > of them. However I have slightly modified test case to overcome >>>>>> above >>>>>> > mentioned failures. >>>>>> > >>>>>> >> >>>>>> >> > >>>>>> >> > On 9.4 everything passes. On 9.6, I get one failure: >>>>>> >> > >>>>>> >> > ============================================================ >>>>>> ========== >>>>>> >> > FAIL: runTest >>>>>> >> > (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDat >>>>>> aTest) >>>>>> >> > Validate Insert, Update operations in View data with given test >>>>>> data >>>>>> >> > ------------------------------------------------------------ >>>>>> ---------- >>>>>> >> > Traceback (most recent call last): >>>>>> >> > File >>>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>>>> ta_dml_queries.py", >>>>>> >> > line 105, in runTest >>>>>> >> > self._verify_row_data(True) >>>>>> >> > File >>>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>>>> ta_dml_queries.py", >>>>>> >> > line 282, in _verify_row_data >>>>>> >> > self.assertEquals(cells[idx], config_data[str(idx)][1]) >>>>>> >> > AssertionError: u'[default]' != u'1' >>>>>> >> > - [default] >>>>>> >> > + 1 >>>>>> >> > >>>>>> > >>>>>> > I guess Khushboo has fixed this recently. >>>>>> > >>>>>> > >>>>>> >> >>>>>> >> > Screenshots attached. >>>>>> >> > >>>>>> >> > On Fri, Jun 16, 2017 at 7:54 AM, Harshal Dhumal >>>>>> >> > <[email protected]> wrote: >>>>>> >> >> Hi Dave, >>>>>> >> >> >>>>>> >> >> Please find attached patch where I have added timeout of 2 >>>>>> seconds >>>>>> >> >> before >>>>>> >> >> selecting object menu in connect to server function. >>>>>> >> >> >>>>>> >> >> Note: Apply this patch on top of previous patch for on demand >>>>>> loading >>>>>> >> >> feature. >>>>>> >> >> >>>>>> >> >> >>>>>> >> >> -- >>>>>> >> >> Harshal Dhumal >>>>>> >> >> Sr. Software Engineer >>>>>> >> >> >>>>>> >> >> EnterpriseDB India: http://www.enterprisedb.com >>>>>> >> >> The Enterprise PostgreSQL Company >>>>>> >> >> >>>>>> >> >> On Fri, Jun 16, 2017 at 2:25 AM, Dave Page <[email protected]> >>>>>> wrote: >>>>>> >> >>> >>>>>> >> >>> Sounds good, thanks. >>>>>> >> >>> >>>>>> >> >>> On Thu, Jun 15, 2017 at 9:54 PM, Harshal Dhumal >>>>>> >> >>> <[email protected]> wrote: >>>>>> >> >>> > Hi >>>>>> >> >>> > >>>>>> >> >>> > On Fri, Jun 16, 2017 at 2:07 AM, Dave Page < >>>>>> [email protected]> >>>>>> >> >>> > wrote: >>>>>> >> >>> >> >>>>>> >> >>> >> Hi >>>>>> >> >>> >> >>>>>> >> >>> >> On Thu, Jun 15, 2017 at 9:30 PM, Harshal Dhumal >>>>>> >> >>> >> <[email protected]> wrote: >>>>>> >> >>> >> > Hi Dave, >>>>>> >> >>> >> > >>>>>> >> >>> >> > Please find attached updated patch. >>>>>> >> >>> >> > >>>>>> >> >>> >> > On Thu, Jun 15, 2017 at 3:58 PM, Dave Page < >>>>>> [email protected]> >>>>>> >> >>> >> > wrote: >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> Hi >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> On Wed, Jun 14, 2017 at 11:36 PM, Harshal Dhumal >>>>>> >> >>> >> >> <[email protected]> wrote: >>>>>> >> >>> >> >>> >>>>>> >> >>> >> >>> Hi Dave, >>>>>> >> >>> >> >>> >>>>>> >> >>> >> >>> Please find rebased patch for RM2137. >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> Looking very good. The only issues I see are: >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> - The row headers should auto-size such that they can >>>>>> display >>>>>> >> >>> >> >> the >>>>>> >> >>> >> >> row >>>>>> >> >>> >> >> numbers if the last row was displayed. E.g. if there are >>>>>> 12345 >>>>>> >> >>> >> >> rows >>>>>> >> >>> >> >> in >>>>>> >> >>> >> >> total, then the row header should be sized to display 5 >>>>>> digits. >>>>>> >> >>> >> >> >>>>>> >> >>> >> > Fixed. >>>>>> >> >>> >> > >>>>>> >> >>> >> > >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> - The tests are comprehensive, which is awesome. >>>>>> However, every >>>>>> >> >>> >> >> time >>>>>> >> >>> >> >> I >>>>>> >> >>> >> >> ran >>>>>> >> >>> >> >> them, at least one of the feature tests failed. >>>>>> Unfortunately, >>>>>> >> >>> >> >> it >>>>>> >> >>> >> >> was a >>>>>> >> >>> >> >> different one each time. In the last two runs, I got: >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> ============================== >>>>>> ======================================== >>>>>> >> >>> >> >> ERROR: runTest >>>>>> >> >>> >> >> (pgadmin.feature_tests.query_t >>>>>> ool_tests.QueryToolFeatureTest) >>>>>> >> >>> >> >> Query tool feature test >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> ------------------------------ >>>>>> ---------------------------------------- >>>>>> >> >>> >> >> Traceback (most recent call last): >>>>>> >> >>> >> >> File >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>>>> /regression/feature_utils/base_feature_test.py", >>>>>> >> >>> >> >> line 40, in setUp >>>>>> >> >>> >> >> self.before() >>>>>> >> >>> >> >> File >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>>>> /pgadmin/feature_tests/query_tool_tests.py", >>>>>> >> >>> >> >> line 40, in before >>>>>> >> >>> >> >> self._connects_to_server() >>>>>> >> >>> >> >> File >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>>>> /pgadmin/feature_tests/query_tool_tests.py", >>>>>> >> >>> >> >> line 144, in _connects_to_server >>>>>> >> >>> >> >> self.page.driver.find_element >>>>>> _by_link_text("Create"))\ >>>>>> >> >>> >> >> File >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>>> webdriver.py", >>>>>> >> >>> >> >> line 319, in find_element_by_link_text >>>>>> >> >>> >> >> return self.find_element(by=By.LINK_TEXT, >>>>>> value=link_text) >>>>>> >> >>> >> >> File >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>>> webdriver.py", >>>>>> >> >>> >> >> line 756, in find_element >>>>>> >> >>> >> >> 'value': value})['value'] >>>>>> >> >>> >> >> File >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>>> webdriver.py", >>>>>> >> >>> >> >> line 238, in execute >>>>>> >> >>> >> >> self.error_handler.check_response(response) >>>>>> >> >>> >> >> File >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>>> errorhandler.py", >>>>>> >> >>> >> >> line 193, in check_response >>>>>> >> >>> >> >> raise exception_class(message, screen, stacktrace) >>>>>> >> >>> >> >> NoSuchElementException: Message: no such element: Unable >>>>>> to >>>>>> >> >>> >> >> locate >>>>>> >> >>> >> >> element: {"method":"link text","selector":"Create"} >>>>>> >> >>> >> >> (Session info: chrome=58.0.3029.110) >>>>>> >> >>> >> >> (Driver info: chromedriver=2.29.461585 >>>>>> >> >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac >>>>>> OS X >>>>>> >> >>> >> >> 10.12.3 >>>>>> >> >>> >> >> x86_64) >>>>>> >> >>> >> >> >>>>>> >> >>> >> > I checked code from _connects_to_server function which is >>>>>> common >>>>>> >> >>> >> > in >>>>>> >> >>> >> > all >>>>>> >> >>> >> > features test cases. I didn't find anything wrong with >>>>>> this. If >>>>>> >> >>> >> > there >>>>>> >> >>> >> > is >>>>>> >> >>> >> > a >>>>>> >> >>> >> > bug in this function then all feature test must fail. >>>>>> >> >>> >> > Let me know if you are getting failure consistently in >>>>>> >> >>> >> > _connects_to_server >>>>>> >> >>> >> > function. >>>>>> >> >>> >> >>>>>> >> >>> >> I wondered if that one is a race condition. Do we need a >>>>>> short >>>>>> >> >>> >> delay >>>>>> >> >>> >> before clicking the Object menu? I have seen this >>>>>> occasionally >>>>>> >> >>> >> before. >>>>>> >> >>> > >>>>>> >> >>> > >>>>>> >> >>> > OK. In that case let's try putting 1-2 second delay and >>>>>> observer >>>>>> >> >>> > behaviour. >>>>>> >> >>> > I'll send separate patch for this tomorrow as this is not >>>>>> related to >>>>>> >> >>> > on >>>>>> >> >>> > demand query result feature or its test cases. >>>>>> >> >>> > >>>>>> >> >>> >> >>>>>> >> >>> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> ============================== >>>>>> ======================================== >>>>>> >> >>> >> >> ERROR: runTest >>>>>> >> >>> >> >> (pgadmin.feature_tests.query_t >>>>>> ool_tests.QueryToolFeatureTest) >>>>>> >> >>> >> >> Query tool feature test >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> ------------------------------ >>>>>> ---------------------------------------- >>>>>> >> >>> >> >> Traceback (most recent call last): >>>>>> >> >>> >> >> File >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>>>> /pgadmin/feature_tests/query_tool_tests.py", >>>>>> >> >>> >> >> line 119, in runTest >>>>>> >> >>> >> >> self._query_tool_auto_rollback_enabled() >>>>>> >> >>> >> >> File >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>>>> /pgadmin/feature_tests/query_tool_tests.py", >>>>>> >> >>> >> >> line 697, in _query_tool_auto_rollback_enabled >>>>>> >> >>> >> >> '//div[contains(@class, "sql-editor-message") and >>>>>> >> >>> >> >> contains(string(), >>>>>> >> >>> >> >> "COMMIT")]' >>>>>> >> >>> >> >> File >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>>> webdriver.py", >>>>>> >> >>> >> >> line 295, in find_element_by_xpath >>>>>> >> >>> >> >> return self.find_element(by=By.XPATH, value=xpath) >>>>>> >> >>> >> >> File >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>>> webdriver.py", >>>>>> >> >>> >> >> line 756, in find_element >>>>>> >> >>> >> >> 'value': value})['value'] >>>>>> >> >>> >> >> File >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>>> webdriver.py", >>>>>> >> >>> >> >> line 238, in execute >>>>>> >> >>> >> >> self.error_handler.check_response(response) >>>>>> >> >>> >> >> File >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>>> errorhandler.py", >>>>>> >> >>> >> >> line 193, in check_response >>>>>> >> >>> >> >> raise exception_class(message, screen, stacktrace) >>>>>> >> >>> >> >> NoSuchElementException: Message: no such element: Unable >>>>>> to >>>>>> >> >>> >> >> locate >>>>>> >> >>> >> >> element: {"method":"xpath","selector":" >>>>>> //div[contains(@class, >>>>>> >> >>> >> >> "sql-editor-message") and contains(string(), "COMMIT")]"} >>>>>> >> >>> >> >> (Session info: chrome=58.0.3029.110) >>>>>> >> >>> >> >> (Driver info: chromedriver=2.29.461585 >>>>>> >> >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac >>>>>> OS X >>>>>> >> >>> >> >> 10.12.3 >>>>>> >> >>> >> >> x86_64) >>>>>> >> >>> >> >> >>>>>> >> >>> >> > I have updated Auto rollback enabled test in this patch. >>>>>> >> >>> >> > >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> Relevant screenshots attached. >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> - Can you tidy up the regression output a little please? >>>>>> Instead >>>>>> >> >>> >> >> of: >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> ------- >>>>>> >> >>> >> >> runTest >>>>>> >> >>> >> >> (pgadmin.feature_tests.query_t >>>>>> ool_tests.QueryToolFeatureTest) >>>>>> >> >>> >> >> Query tool feature test ... On demand result set on >>>>>> scrolling... >>>>>> >> >>> >> >> OK. >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> On demand result set on grid select all... >>>>>> >> >>> >> >> OK. >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> On demand result set on column select all... >>>>>> >> >>> >> >> OK. >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> Explain query... >>>>>> >> >>> >> >> OK. >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> Explain query with verbose... >>>>>> >> >>> >> >> OK. >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> Explain query with costs... >>>>>> >> >>> >> >> OK. >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> Explain analyze query... >>>>>> >> >>> >> >> OK. >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> Explain analyze query with buffers... >>>>>> >> >>> >> >> OK. >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> Explain analyze query with timing... >>>>>> >> >>> >> >> OK. >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> Auto commit disabled... >>>>>> >> >>> >> >> OK. >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> Auto commit enabled... >>>>>> >> >>> >> >> OK. >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> Auto rollback enabled... >>>>>> >> >>> >> >> ERROR >>>>>> >> >>> >> >> ------- >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> Something like: >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> ------- >>>>>> >> >>> >> >> runTest >>>>>> >> >>> >> >> (pgadmin.feature_tests.query_t >>>>>> ool_tests.QueryToolFeatureTest) >>>>>> >> >>> >> >> Query tool feature test ... >>>>>> >> >>> >> >> On demand result set on scrolling... OK. >>>>>> >> >>> >> >> On demand result set on grid select all... OK. >>>>>> >> >>> >> >> On demand result set on column select all... OK. >>>>>> >> >>> >> >> Explain query... OK. >>>>>> >> >>> >> >> Explain query with verbose... OK. >>>>>> >> >>> >> >> Explain query with costs... OK. >>>>>> >> >>> >> >> Explain analyze query... OK. >>>>>> >> >>> >> >> Explain analyze query with buffers... OK. >>>>>> >> >>> >> >> Explain analyze query with timing... OK. >>>>>> >> >>> >> >> Auto commit disabled... OK. >>>>>> >> >>> >> >> Auto commit enabled... OK. >>>>>> >> >>> >> >> Auto rollback enabled... ERROR >>>>>> >> >>> >> >> -------- >>>>>> >> >>> >> >> >>>>>> >> >>> >> > Fixed. >>>>>> >> >>> >> > >>>>>> >> >>> >> > >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> Thanks! >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> -- >>>>>> >> >>> >> >> Dave Page >>>>>> >> >>> >> >> Blog: http://pgsnake.blogspot.com >>>>>> >> >>> >> >> Twitter: @pgsnake >>>>>> >> >>> >> >> >>>>>> >> >>> >> >> EnterpriseDB UK: http://www.enterprisedb.com >>>>>> >> >>> >> >> The Enterprise PostgreSQL Company >>>>>> >> >>> >> > >>>>>> >> >>> >> > >>>>>> >> >>> >> >>>>>> >> >>> >> >>>>>> >> >>> >> >>>>>> >> >>> >> -- >>>>>> >> >>> >> Dave Page >>>>>> >> >>> >> Blog: http://pgsnake.blogspot.com >>>>>> >> >>> >> Twitter: @pgsnake >>>>>> >> >>> >> >>>>>> >> >>> >> EnterpriseDB UK: http://www.enterprisedb.com >>>>>> >> >>> >> The Enterprise PostgreSQL Company >>>>>> >> >>> > >>>>>> >> >>> > >>>>>> >> >>> >>>>>> >> >>> >>>>>> >> >>> >>>>>> >> >>> -- >>>>>> >> >>> Dave Page >>>>>> >> >>> Blog: http://pgsnake.blogspot.com >>>>>> >> >>> Twitter: @pgsnake >>>>>> >> >>> >>>>>> >> >>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>> >> >>> The Enterprise PostgreSQL Company >>>>>> >> >> >>>>>> >> >> >>>>>> >> > >>>>>> >> > >>>>>> >> > >>>>>> >> > -- >>>>>> >> > Dave Page >>>>>> >> > Blog: http://pgsnake.blogspot.com >>>>>> >> > Twitter: @pgsnake >>>>>> >> > >>>>>> >> > EnterpriseDB UK: http://www.enterprisedb.com >>>>>> >> > The Enterprise PostgreSQL Company >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> -- >>>>>> >> Dave Page >>>>>> >> Blog: http://pgsnake.blogspot.com >>>>>> >> Twitter: @pgsnake >>>>>> >> >>>>>> >> EnterpriseDB UK: http://www.enterprisedb.com >>>>>> >> The Enterprise PostgreSQL Company >>>>>> > >>>>>> > >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Dave Page >>>>>> Blog: http://pgsnake.blogspot.com >>>>>> Twitter: @pgsnake >>>>>> >>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>> The Enterprise PostgreSQL Company >>>>>> >>>>> >>>>> >>>> >>> >>> >>> -- >>> Dave Page >>> Blog: http://pgsnake.blogspot.com >>> Twitter: @pgsnake >>> >>> EnterpriseDB UK: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >>> >>> >> > -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company Attachments: [image/png] Screen Shot 2017-06-27 at 5.00.35 PM.png (126.8K, 3-Screen%20Shot%202017-06-27%20at%205.00.35%20PM.png) download | view image [image/png] Screen Shot 2017-06-27 at 4.55.16 PM.png (38.4K, 4-Screen%20Shot%202017-06-27%20at%204.55.16%20PM.png) download | view image ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: [pgadmin-hackers] Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] @ 2017-06-27 21:18 Robert Eckhardt <[email protected]> parent: Joao Pedro De Almeida Pereira <[email protected]> 1 sibling, 1 reply; 24+ messages in thread From: Robert Eckhardt @ 2017-06-27 21:18 UTC (permalink / raw) To: Joao Pedro De Almeida Pereira <[email protected]>; +Cc: pgadmin-hackers; Dave Page <[email protected]>; Ashesh Vashi <[email protected]>; Harshal Dhumal <[email protected]>; Shirley Wang <[email protected]> On Tue, Jun 27, 2017 at 5:03 PM, Joao Pedro De Almeida Pereira < [email protected]> wrote: > Hello Hackers, > > When we started the app we noticed some change in the front end. > > The line numbers in the Editor: > - We noticed a bug with the numbers where at 10000 rows, the numbers would > be cut off. > - When the row is selected the color of the text should be white > - Centering the number column would differentiate it from the rest of the > table and prevent any confusion > > Do we want line numbers? I'm of mixed opinion. On the one hand it implies and ordered set of results when Postgres inherently doesn't order results. On the other hand it nicely tells you where you are in the results set if you are scrolling around. -- Rob ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: [pgadmin-hackers] Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] @ 2017-06-27 21:21 Dave Page <[email protected]> parent: Robert Eckhardt <[email protected]> 0 siblings, 0 replies; 24+ messages in thread From: Dave Page @ 2017-06-27 21:21 UTC (permalink / raw) To: Robert Eckhardt <[email protected]>; +Cc: Joao Pedro De Almeida Pereira <[email protected]>; pgadmin-hackers; Ashesh Vashi <[email protected]>; Harshal Dhumal <[email protected]>; Shirley Wang <[email protected]> > On 27 Jun 2017, at 17:18, Robert Eckhardt <[email protected]> wrote: > > > >> On Tue, Jun 27, 2017 at 5:03 PM, Joao Pedro De Almeida Pereira <[email protected]> wrote: >> Hello Hackers, >> >> When we started the app we noticed some change in the front end. >> >> The line numbers in the Editor: >> - We noticed a bug with the numbers where at 10000 rows, the numbers would be cut off. >> - When the row is selected the color of the text should be white >> - Centering the number column would differentiate it from the rest of the table and prevent any confusion >> > > Do we want line numbers? I'm of mixed opinion. On the one hand it implies and ordered set of results when Postgres inherently doesn't order results. On the other hand it nicely tells you where you are in the results set if you are scrolling around. Yes, they are incredibly useful. ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: [pgadmin-hackers] Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] @ 2017-06-28 11:04 Harshal Dhumal <[email protected]> parent: Dave Page <[email protected]> 0 siblings, 1 reply; 24+ messages in thread From: Harshal Dhumal @ 2017-06-28 11:04 UTC (permalink / raw) To: Dave Page <[email protected]>; +Cc: Joao Pedro De Almeida Pereira <[email protected]>; pgadmin-hackers; Ashesh Vashi <[email protected]>; Robert Eckhardt <[email protected]>; Shirley Wang <[email protected]> Hi, Please find attached patch to fix minor issues/improvements suggested. On Wed, Jun 28, 2017 at 2:43 AM, Dave Page <[email protected]> wrote: > Harshal, can you look at these comments please, and patch as appropriate? > > On Tue, Jun 27, 2017 at 5:03 PM, Joao Pedro De Almeida Pereira < > [email protected]> wrote: > >> Hello Hackers, >> >> When we started the app we noticed some change in the front end. >> >> The line numbers in the Editor: >> - We noticed a bug with the numbers where at 10000 rows, the numbers >> would be cut off. >> > No. Row number column automatically adjust its width depending on maximum row count to be shown. (see attached screenshot ) [image: Inline image 1] > - When the row is selected the color of the text should be white >> > Fixed > - Centering the number column would differentiate it from the rest of the >> table and prevent any confusion >> > Fixed. > >> >> Select all triangle: >> - This seems to have shifted up and to the left slightly. >> > Fixed alignment. > >> While editing data, when you paste a new row, that line should come into >> view. >> > Fixed. > >> Issue in the code: >> - In sqleditor/command.py:473 the variable `column_data` is not >> initialized >> > Fixed. > >> >> >> We were looking into the menu's to access and Edit Data editor, and we >> noticed that in the menu it is still called "View Data". We are aware that >> this change was not introduced in this patch, but we just noticed it. It >> should be consistent. >> > Fixed. (see attached screenshot) [image: Inline image 2] > >> >> >> Thanks >> Shirley & Joao >> >> >> On Tue, Jun 27, 2017 at 12:19 PM, Robert Eckhardt <[email protected]> >> wrote: >> >>> >>> >>> On Tue, Jun 27, 2017 at 12:16 PM, Khushboo Vashi < >>> [email protected]> wrote: >>> >>>> >>>> >>>> On 27 Jun 2017 18:33, "Dave Page" <[email protected]> wrote: >>>> >>>> Thanks - patch committed! >>>> >>>> Awsome job :-) >>>> >>>> Gr8. Finally no more rebase request for Harshal. :) >>>> >>> >>> +1 >>> >>> >>>> On Tue, Jun 27, 2017 at 3:26 AM, Harshal Dhumal < >>>> [email protected]> wrote: >>>> >>>>> Hi, >>>>> >>>>> Please find rebased patch. >>>>> >>>>> -- >>>>> *Harshal Dhumal* >>>>> *Sr. Software Engineer* >>>>> >>>>> EnterpriseDB India: http://www.enterprisedb.com >>>>> The Enterprise PostgreSQL Company >>>>> >>>>> On Mon, Jun 26, 2017 at 5:24 PM, Harshal Dhumal < >>>>> [email protected]> wrote: >>>>> >>>>>> yes i'm working on that only :) >>>>>> >>>>>> >>>>>> -- >>>>>> *Harshal Dhumal* >>>>>> *Sr. Software Engineer* >>>>>> >>>>>> EnterpriseDB India: http://www.enterprisedb.com >>>>>> The Enterprise PostgreSQL Company >>>>>> >>>>>> On Mon, Jun 26, 2017 at 5:22 PM, Dave Page <[email protected]> wrote: >>>>>> >>>>>>> I'm sorry, it needs rebasing again. If you can do it quickly, I'll >>>>>>> make sure it's the next patch I work on in that area. >>>>>>> >>>>>>> Thanks. >>>>>>> >>>>>>> On Mon, Jun 26, 2017 at 5:16 AM, Harshal Dhumal >>>>>>> <[email protected]> wrote: >>>>>>> > Hi Dave, >>>>>>> > >>>>>>> > Please find updated rebased patch for RM2137 >>>>>>> > >>>>>>> > On Fri, Jun 23, 2017 at 9:00 PM, Dave Page <[email protected]> >>>>>>> wrote: >>>>>>> >> >>>>>>> >> Hi Harshal, >>>>>>> >> >>>>>>> >> When can we expect an updated version of this patch? I think it's >>>>>>> >> important to get this into the next release. >>>>>>> >> >>>>>>> >> Thanks! >>>>>>> >> >>>>>>> >> On Fri, Jun 16, 2017 at 10:55 AM, Dave Page <[email protected]> >>>>>>> wrote: >>>>>>> >> > Hi, >>>>>>> >> > >>>>>>> >> > That's better - the failures are far less random now :-). I got >>>>>>> the >>>>>>> >> > following two though, on both PG and EPAS 9.5: >>>>>>> >> > >>>>>>> >> > ============================================================ >>>>>>> ========== >>>>>>> >> > ERROR: runTest >>>>>>> >> > (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >>>>>>> >> > Query tool feature test >>>>>>> >> > ------------------------------------------------------------ >>>>>>> ---------- >>>>>>> >> > Traceback (most recent call last): >>>>>>> >> > File >>>>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_t >>>>>>> ool_tests.py", >>>>>>> >> > line 95, in runTest >>>>>>> >> > self._query_tool_explain_analyze_buffers() >>>>>>> >> > File >>>>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_t >>>>>>> ool_tests.py", >>>>>>> >> > line 443, in _query_tool_explain_analyze_buffers >>>>>>> >> > canvas.find_element_by_xpath("//*[contains(string(), >>>>>>> 'Shared Read >>>>>>> >> > Blocks')]") >>>>>>> >> > File >>>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>>> ges/selenium/webdriver/remote/webelement.py", >>>>>>> >> > line 260, in find_element_by_xpath >>>>>>> >> > return self.find_element(by=By.XPATH, value=xpath) >>>>>>> >> > File >>>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>>> ges/selenium/webdriver/remote/webelement.py", >>>>>>> >> > line 508, in find_element >>>>>>> >> > {"using": by, "value": value})['value'] >>>>>>> >> > File >>>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>>> ges/selenium/webdriver/remote/webelement.py", >>>>>>> >> > line 491, in _execute >>>>>>> >> > return self._parent.execute(command, params) >>>>>>> >> > File >>>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>>> ges/selenium/webdriver/remote/webdriver.py", >>>>>>> >> > line 238, in execute >>>>>>> >> > self.error_handler.check_response(response) >>>>>>> >> > File >>>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>>> ges/selenium/webdriver/remote/errorhandler.py", >>>>>>> >> > line 193, in check_response >>>>>>> >> > raise exception_class(message, screen, stacktrace) >>>>>>> >> > NoSuchElementException: Message: no such element: Unable to >>>>>>> locate >>>>>>> >> > element: {"method":"xpath","selector":"//*[contains(string(), >>>>>>> 'Shared >>>>>>> >> > Read Blocks')]"} >>>>>>> >> > (Session info: chrome=58.0.3029.110) >>>>>>> >> > (Driver info: chromedriver=2.29.461585 >>>>>>> >> > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X >>>>>>> 10.12.3 >>>>>>> >> > x86_64) >>>>>>> >> > >>>>>>> >> > >>>>>>> >> > ============================================================ >>>>>>> ========== >>>>>>> >> > ERROR: runTest >>>>>>> >> > (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDat >>>>>>> aTest) >>>>>>> >> > Validate Insert, Update operations in View data with given test >>>>>>> data >>>>>>> >> > ------------------------------------------------------------ >>>>>>> ---------- >>>>>>> >> > Traceback (most recent call last): >>>>>>> >> > File >>>>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>>>>> ta_dml_queries.py", >>>>>>> >> > line 104, in runTest >>>>>>> >> > self._add_row() >>>>>>> >> > File >>>>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>>>>> ta_dml_queries.py", >>>>>>> >> > line 255, in _add_row >>>>>>> >> > self._update_cell(cell_xpath, config_data[str(idx)]) >>>>>>> >> > File >>>>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>>>>> ta_dml_queries.py", >>>>>>> >> > line 164, in _update_cell >>>>>>> >> > cell_el = self.page.find_by_xpath(xpath) >>>>>>> >> > File >>>>>>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>>>>>> min_page.py", >>>>>>> >> > line 122, in find_by_xpath >>>>>>> >> > return self.wait_for_element(lambda driver: >>>>>>> >> > driver.find_element_by_xpath(xpath)) >>>>>>> >> > File >>>>>>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>>>>>> min_page.py", >>>>>>> >> > line 205, in wait_for_element >>>>>>> >> > return self._wait_for("element to exist", >>>>>>> element_if_it_exists) >>>>>>> >> > File >>>>>>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>>>>>> min_page.py", >>>>>>> >> > line 255, in _wait_for >>>>>>> >> > "Timed out waiting for " + waiting_for_message) >>>>>>> >> > File >>>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>>> ges/selenium/webdriver/support/wait.py", >>>>>>> >> > line 71, in until >>>>>>> >> > value = method(self._driver) >>>>>>> >> > File >>>>>>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>>>>>> min_page.py", >>>>>>> >> > line 200, in element_if_it_exists >>>>>>> >> > if element.is_displayed() and element.is_enabled(): >>>>>>> >> > File >>>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>>> ges/selenium/webdriver/remote/webelement.py", >>>>>>> >> > line 358, in is_displayed >>>>>>> >> > return self._execute(Command.IS_ELEMENT_DISPLAYED)['value'] >>>>>>> >> > File >>>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>>> ges/selenium/webdriver/remote/webelement.py", >>>>>>> >> > line 491, in _execute >>>>>>> >> > return self._parent.execute(command, params) >>>>>>> >> > File >>>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>>> ges/selenium/webdriver/remote/webdriver.py", >>>>>>> >> > line 238, in execute >>>>>>> >> > self.error_handler.check_response(response) >>>>>>> >> > File >>>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>>> ges/selenium/webdriver/remote/errorhandler.py", >>>>>>> >> > line 193, in check_response >>>>>>> >> > raise exception_class(message, screen, stacktrace) >>>>>>> >> > StaleElementReferenceException: Message: stale element >>>>>>> reference: >>>>>>> >> > element is not attached to the page document >>>>>>> >> > (Session info: chrome=58.0.3029.110) >>>>>>> >> > (Driver info: chromedriver=2.29.461585 >>>>>>> >> > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X >>>>>>> 10.12.3 >>>>>>> >> > x86_64) >>>>>>> >> > >>>>>>> > >>>>>>> > >>>>>>> > I checked my feature test cases for any database server >>>>>>> version/type (PG, >>>>>>> > EPAS) specific failures and I found that test cases are working >>>>>>> fine on all >>>>>>> > of them. However I have slightly modified test case to overcome >>>>>>> above >>>>>>> > mentioned failures. >>>>>>> > >>>>>>> >> >>>>>>> >> > >>>>>>> >> > On 9.4 everything passes. On 9.6, I get one failure: >>>>>>> >> > >>>>>>> >> > ============================================================ >>>>>>> ========== >>>>>>> >> > FAIL: runTest >>>>>>> >> > (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDat >>>>>>> aTest) >>>>>>> >> > Validate Insert, Update operations in View data with given test >>>>>>> data >>>>>>> >> > ------------------------------------------------------------ >>>>>>> ---------- >>>>>>> >> > Traceback (most recent call last): >>>>>>> >> > File >>>>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>>>>> ta_dml_queries.py", >>>>>>> >> > line 105, in runTest >>>>>>> >> > self._verify_row_data(True) >>>>>>> >> > File >>>>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>>>>> ta_dml_queries.py", >>>>>>> >> > line 282, in _verify_row_data >>>>>>> >> > self.assertEquals(cells[idx], config_data[str(idx)][1]) >>>>>>> >> > AssertionError: u'[default]' != u'1' >>>>>>> >> > - [default] >>>>>>> >> > + 1 >>>>>>> >> > >>>>>>> > >>>>>>> > I guess Khushboo has fixed this recently. >>>>>>> > >>>>>>> > >>>>>>> >> >>>>>>> >> > Screenshots attached. >>>>>>> >> > >>>>>>> >> > On Fri, Jun 16, 2017 at 7:54 AM, Harshal Dhumal >>>>>>> >> > <[email protected]> wrote: >>>>>>> >> >> Hi Dave, >>>>>>> >> >> >>>>>>> >> >> Please find attached patch where I have added timeout of 2 >>>>>>> seconds >>>>>>> >> >> before >>>>>>> >> >> selecting object menu in connect to server function. >>>>>>> >> >> >>>>>>> >> >> Note: Apply this patch on top of previous patch for on demand >>>>>>> loading >>>>>>> >> >> feature. >>>>>>> >> >> >>>>>>> >> >> >>>>>>> >> >> -- >>>>>>> >> >> Harshal Dhumal >>>>>>> >> >> Sr. Software Engineer >>>>>>> >> >> >>>>>>> >> >> EnterpriseDB India: http://www.enterprisedb.com >>>>>>> >> >> The Enterprise PostgreSQL Company >>>>>>> >> >> >>>>>>> >> >> On Fri, Jun 16, 2017 at 2:25 AM, Dave Page <[email protected]> >>>>>>> wrote: >>>>>>> >> >>> >>>>>>> >> >>> Sounds good, thanks. >>>>>>> >> >>> >>>>>>> >> >>> On Thu, Jun 15, 2017 at 9:54 PM, Harshal Dhumal >>>>>>> >> >>> <[email protected]> wrote: >>>>>>> >> >>> > Hi >>>>>>> >> >>> > >>>>>>> >> >>> > On Fri, Jun 16, 2017 at 2:07 AM, Dave Page < >>>>>>> [email protected]> >>>>>>> >> >>> > wrote: >>>>>>> >> >>> >> >>>>>>> >> >>> >> Hi >>>>>>> >> >>> >> >>>>>>> >> >>> >> On Thu, Jun 15, 2017 at 9:30 PM, Harshal Dhumal >>>>>>> >> >>> >> <[email protected]> wrote: >>>>>>> >> >>> >> > Hi Dave, >>>>>>> >> >>> >> > >>>>>>> >> >>> >> > Please find attached updated patch. >>>>>>> >> >>> >> > >>>>>>> >> >>> >> > On Thu, Jun 15, 2017 at 3:58 PM, Dave Page < >>>>>>> [email protected]> >>>>>>> >> >>> >> > wrote: >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> Hi >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> On Wed, Jun 14, 2017 at 11:36 PM, Harshal Dhumal >>>>>>> >> >>> >> >> <[email protected]> wrote: >>>>>>> >> >>> >> >>> >>>>>>> >> >>> >> >>> Hi Dave, >>>>>>> >> >>> >> >>> >>>>>>> >> >>> >> >>> Please find rebased patch for RM2137. >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> Looking very good. The only issues I see are: >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> - The row headers should auto-size such that they can >>>>>>> display >>>>>>> >> >>> >> >> the >>>>>>> >> >>> >> >> row >>>>>>> >> >>> >> >> numbers if the last row was displayed. E.g. if there >>>>>>> are 12345 >>>>>>> >> >>> >> >> rows >>>>>>> >> >>> >> >> in >>>>>>> >> >>> >> >> total, then the row header should be sized to display 5 >>>>>>> digits. >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> > Fixed. >>>>>>> >> >>> >> > >>>>>>> >> >>> >> > >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> - The tests are comprehensive, which is awesome. >>>>>>> However, every >>>>>>> >> >>> >> >> time >>>>>>> >> >>> >> >> I >>>>>>> >> >>> >> >> ran >>>>>>> >> >>> >> >> them, at least one of the feature tests failed. >>>>>>> Unfortunately, >>>>>>> >> >>> >> >> it >>>>>>> >> >>> >> >> was a >>>>>>> >> >>> >> >> different one each time. In the last two runs, I got: >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> ============================== >>>>>>> ======================================== >>>>>>> >> >>> >> >> ERROR: runTest >>>>>>> >> >>> >> >> (pgadmin.feature_tests.query_t >>>>>>> ool_tests.QueryToolFeatureTest) >>>>>>> >> >>> >> >> Query tool feature test >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> ------------------------------ >>>>>>> ---------------------------------------- >>>>>>> >> >>> >> >> Traceback (most recent call last): >>>>>>> >> >>> >> >> File >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>>>>> /regression/feature_utils/base_feature_test.py", >>>>>>> >> >>> >> >> line 40, in setUp >>>>>>> >> >>> >> >> self.before() >>>>>>> >> >>> >> >> File >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>>>>> /pgadmin/feature_tests/query_tool_tests.py", >>>>>>> >> >>> >> >> line 40, in before >>>>>>> >> >>> >> >> self._connects_to_server() >>>>>>> >> >>> >> >> File >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>>>>> /pgadmin/feature_tests/query_tool_tests.py", >>>>>>> >> >>> >> >> line 144, in _connects_to_server >>>>>>> >> >>> >> >> self.page.driver.find_element >>>>>>> _by_link_text("Create"))\ >>>>>>> >> >>> >> >> File >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>>>> webdriver.py", >>>>>>> >> >>> >> >> line 319, in find_element_by_link_text >>>>>>> >> >>> >> >> return self.find_element(by=By.LINK_TEXT, >>>>>>> value=link_text) >>>>>>> >> >>> >> >> File >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>>>> webdriver.py", >>>>>>> >> >>> >> >> line 756, in find_element >>>>>>> >> >>> >> >> 'value': value})['value'] >>>>>>> >> >>> >> >> File >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>>>> webdriver.py", >>>>>>> >> >>> >> >> line 238, in execute >>>>>>> >> >>> >> >> self.error_handler.check_response(response) >>>>>>> >> >>> >> >> File >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>>>> errorhandler.py", >>>>>>> >> >>> >> >> line 193, in check_response >>>>>>> >> >>> >> >> raise exception_class(message, screen, stacktrace) >>>>>>> >> >>> >> >> NoSuchElementException: Message: no such element: >>>>>>> Unable to >>>>>>> >> >>> >> >> locate >>>>>>> >> >>> >> >> element: {"method":"link text","selector":"Create"} >>>>>>> >> >>> >> >> (Session info: chrome=58.0.3029.110) >>>>>>> >> >>> >> >> (Driver info: chromedriver=2.29.461585 >>>>>>> >> >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac >>>>>>> OS X >>>>>>> >> >>> >> >> 10.12.3 >>>>>>> >> >>> >> >> x86_64) >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> > I checked code from _connects_to_server function which >>>>>>> is common >>>>>>> >> >>> >> > in >>>>>>> >> >>> >> > all >>>>>>> >> >>> >> > features test cases. I didn't find anything wrong with >>>>>>> this. If >>>>>>> >> >>> >> > there >>>>>>> >> >>> >> > is >>>>>>> >> >>> >> > a >>>>>>> >> >>> >> > bug in this function then all feature test must fail. >>>>>>> >> >>> >> > Let me know if you are getting failure consistently in >>>>>>> >> >>> >> > _connects_to_server >>>>>>> >> >>> >> > function. >>>>>>> >> >>> >> >>>>>>> >> >>> >> I wondered if that one is a race condition. Do we need a >>>>>>> short >>>>>>> >> >>> >> delay >>>>>>> >> >>> >> before clicking the Object menu? I have seen this >>>>>>> occasionally >>>>>>> >> >>> >> before. >>>>>>> >> >>> > >>>>>>> >> >>> > >>>>>>> >> >>> > OK. In that case let's try putting 1-2 second delay and >>>>>>> observer >>>>>>> >> >>> > behaviour. >>>>>>> >> >>> > I'll send separate patch for this tomorrow as this is not >>>>>>> related to >>>>>>> >> >>> > on >>>>>>> >> >>> > demand query result feature or its test cases. >>>>>>> >> >>> > >>>>>>> >> >>> >> >>>>>>> >> >>> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> ============================== >>>>>>> ======================================== >>>>>>> >> >>> >> >> ERROR: runTest >>>>>>> >> >>> >> >> (pgadmin.feature_tests.query_t >>>>>>> ool_tests.QueryToolFeatureTest) >>>>>>> >> >>> >> >> Query tool feature test >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> ------------------------------ >>>>>>> ---------------------------------------- >>>>>>> >> >>> >> >> Traceback (most recent call last): >>>>>>> >> >>> >> >> File >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>>>>> /pgadmin/feature_tests/query_tool_tests.py", >>>>>>> >> >>> >> >> line 119, in runTest >>>>>>> >> >>> >> >> self._query_tool_auto_rollback_enabled() >>>>>>> >> >>> >> >> File >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>>>>> /pgadmin/feature_tests/query_tool_tests.py", >>>>>>> >> >>> >> >> line 697, in _query_tool_auto_rollback_enabled >>>>>>> >> >>> >> >> '//div[contains(@class, "sql-editor-message") and >>>>>>> >> >>> >> >> contains(string(), >>>>>>> >> >>> >> >> "COMMIT")]' >>>>>>> >> >>> >> >> File >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>>>> webdriver.py", >>>>>>> >> >>> >> >> line 295, in find_element_by_xpath >>>>>>> >> >>> >> >> return self.find_element(by=By.XPATH, value=xpath) >>>>>>> >> >>> >> >> File >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>>>> webdriver.py", >>>>>>> >> >>> >> >> line 756, in find_element >>>>>>> >> >>> >> >> 'value': value})['value'] >>>>>>> >> >>> >> >> File >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>>>> webdriver.py", >>>>>>> >> >>> >> >> line 238, in execute >>>>>>> >> >>> >> >> self.error_handler.check_response(response) >>>>>>> >> >>> >> >> File >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>>>> errorhandler.py", >>>>>>> >> >>> >> >> line 193, in check_response >>>>>>> >> >>> >> >> raise exception_class(message, screen, stacktrace) >>>>>>> >> >>> >> >> NoSuchElementException: Message: no such element: >>>>>>> Unable to >>>>>>> >> >>> >> >> locate >>>>>>> >> >>> >> >> element: {"method":"xpath","selector":" >>>>>>> //div[contains(@class, >>>>>>> >> >>> >> >> "sql-editor-message") and contains(string(), >>>>>>> "COMMIT")]"} >>>>>>> >> >>> >> >> (Session info: chrome=58.0.3029.110) >>>>>>> >> >>> >> >> (Driver info: chromedriver=2.29.461585 >>>>>>> >> >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac >>>>>>> OS X >>>>>>> >> >>> >> >> 10.12.3 >>>>>>> >> >>> >> >> x86_64) >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> > I have updated Auto rollback enabled test in this patch. >>>>>>> >> >>> >> > >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> Relevant screenshots attached. >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> - Can you tidy up the regression output a little >>>>>>> please? Instead >>>>>>> >> >>> >> >> of: >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> ------- >>>>>>> >> >>> >> >> runTest >>>>>>> >> >>> >> >> (pgadmin.feature_tests.query_t >>>>>>> ool_tests.QueryToolFeatureTest) >>>>>>> >> >>> >> >> Query tool feature test ... On demand result set on >>>>>>> scrolling... >>>>>>> >> >>> >> >> OK. >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> On demand result set on grid select all... >>>>>>> >> >>> >> >> OK. >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> On demand result set on column select all... >>>>>>> >> >>> >> >> OK. >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> Explain query... >>>>>>> >> >>> >> >> OK. >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> Explain query with verbose... >>>>>>> >> >>> >> >> OK. >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> Explain query with costs... >>>>>>> >> >>> >> >> OK. >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> Explain analyze query... >>>>>>> >> >>> >> >> OK. >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> Explain analyze query with buffers... >>>>>>> >> >>> >> >> OK. >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> Explain analyze query with timing... >>>>>>> >> >>> >> >> OK. >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> Auto commit disabled... >>>>>>> >> >>> >> >> OK. >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> Auto commit enabled... >>>>>>> >> >>> >> >> OK. >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> Auto rollback enabled... >>>>>>> >> >>> >> >> ERROR >>>>>>> >> >>> >> >> ------- >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> Something like: >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> ------- >>>>>>> >> >>> >> >> runTest >>>>>>> >> >>> >> >> (pgadmin.feature_tests.query_t >>>>>>> ool_tests.QueryToolFeatureTest) >>>>>>> >> >>> >> >> Query tool feature test ... >>>>>>> >> >>> >> >> On demand result set on scrolling... OK. >>>>>>> >> >>> >> >> On demand result set on grid select all... OK. >>>>>>> >> >>> >> >> On demand result set on column select all... OK. >>>>>>> >> >>> >> >> Explain query... OK. >>>>>>> >> >>> >> >> Explain query with verbose... OK. >>>>>>> >> >>> >> >> Explain query with costs... OK. >>>>>>> >> >>> >> >> Explain analyze query... OK. >>>>>>> >> >>> >> >> Explain analyze query with buffers... OK. >>>>>>> >> >>> >> >> Explain analyze query with timing... OK. >>>>>>> >> >>> >> >> Auto commit disabled... OK. >>>>>>> >> >>> >> >> Auto commit enabled... OK. >>>>>>> >> >>> >> >> Auto rollback enabled... ERROR >>>>>>> >> >>> >> >> -------- >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> > Fixed. >>>>>>> >> >>> >> > >>>>>>> >> >>> >> > >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> Thanks! >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> -- >>>>>>> >> >>> >> >> Dave Page >>>>>>> >> >>> >> >> Blog: http://pgsnake.blogspot.com >>>>>>> >> >>> >> >> Twitter: @pgsnake >>>>>>> >> >>> >> >> >>>>>>> >> >>> >> >> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>> >> >>> >> >> The Enterprise PostgreSQL Company >>>>>>> >> >>> >> > >>>>>>> >> >>> >> > >>>>>>> >> >>> >> >>>>>>> >> >>> >> >>>>>>> >> >>> >> >>>>>>> >> >>> >> -- >>>>>>> >> >>> >> Dave Page >>>>>>> >> >>> >> Blog: http://pgsnake.blogspot.com >>>>>>> >> >>> >> Twitter: @pgsnake >>>>>>> >> >>> >> >>>>>>> >> >>> >> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>> >> >>> >> The Enterprise PostgreSQL Company >>>>>>> >> >>> > >>>>>>> >> >>> > >>>>>>> >> >>> >>>>>>> >> >>> >>>>>>> >> >>> >>>>>>> >> >>> -- >>>>>>> >> >>> Dave Page >>>>>>> >> >>> Blog: http://pgsnake.blogspot.com >>>>>>> >> >>> Twitter: @pgsnake >>>>>>> >> >>> >>>>>>> >> >>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>> >> >>> The Enterprise PostgreSQL Company >>>>>>> >> >> >>>>>>> >> >> >>>>>>> >> > >>>>>>> >> > >>>>>>> >> > >>>>>>> >> > -- >>>>>>> >> > Dave Page >>>>>>> >> > Blog: http://pgsnake.blogspot.com >>>>>>> >> > Twitter: @pgsnake >>>>>>> >> > >>>>>>> >> > EnterpriseDB UK: http://www.enterprisedb.com >>>>>>> >> > The Enterprise PostgreSQL Company >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> -- >>>>>>> >> Dave Page >>>>>>> >> Blog: http://pgsnake.blogspot.com >>>>>>> >> Twitter: @pgsnake >>>>>>> >> >>>>>>> >> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>> >> The Enterprise PostgreSQL Company >>>>>>> > >>>>>>> > >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Dave Page >>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>> Twitter: @pgsnake >>>>>>> >>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>> The Enterprise PostgreSQL Company >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> Dave Page >>>> Blog: http://pgsnake.blogspot.com >>>> Twitter: @pgsnake >>>> >>>> EnterpriseDB UK: http://www.enterprisedb.com >>>> The Enterprise PostgreSQL Company >>>> >>>> >>>> >>> >> > > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > Attachments: [image/png] View_Edit_menu.png (47.3K, 3-View_Edit_menu.png) download | view image [image/png] auto_resize_row_number_column.png (32.3K, 4-auto_resize_row_number_column.png) download | view image [text/x-patch] RM2137_minor_issue_fixes.patch (5.7K, 5-RM2137_minor_issue_fixes.patch) download | inline diff: diff --git a/web/pgadmin/feature_tests/view_data_dml_queries.py b/web/pgadmin/feature_tests/view_data_dml_queries.py index a6bc5aa..6454a3e 100644 --- a/web/pgadmin/feature_tests/view_data_dml_queries.py +++ b/web/pgadmin/feature_tests/view_data_dml_queries.py @@ -44,7 +44,7 @@ class CheckForViewDataTest(BaseFeatureTest): """ scenarios = [ - ("Validate Insert, Update operations in View data with given test " + ("Validate Insert, Update operations in View/Edit data with given test " "data", dict()) ] @@ -97,7 +97,7 @@ CREATE TABLE public.defaults self.page.add_server(self.server) self._tables_node_expandable() self.page.select_tree_item('defaults') - # Open Object -> View data + # Open Object -> View/Edit data self._view_data_grid() # Run test to insert a new row in table with default values @@ -203,9 +203,9 @@ CREATE TABLE public.defaults self.page.driver.find_element_by_link_text("Object").click() ActionChains(self.page.driver) \ .move_to_element( - self.page.driver.find_element_by_link_text("View Data")) \ + self.page.driver.find_element_by_link_text("View/Edit Data")) \ .perform() - self.page.find_by_partial_link_text("View All Rows").click() + self.page.find_by_partial_link_text("All Rows").click() # wait until datagrid frame is loaded. self.page.click_tab('Edit Data -') diff --git a/web/pgadmin/tools/datagrid/templates/datagrid/js/datagrid.js b/web/pgadmin/tools/datagrid/templates/datagrid/js/datagrid.js index aa04c2f..2d5717d 100644 --- a/web/pgadmin/tools/datagrid/templates/datagrid/js/datagrid.js +++ b/web/pgadmin/tools/datagrid/templates/datagrid/js/datagrid.js @@ -87,26 +87,26 @@ define([ name: 'view_all_rows_context_' + supported_nodes[idx], node: supported_nodes[idx], module: this, data: {mnuid: 3}, applies: ['context', 'object'], callback: 'show_data_grid', enable: view_menu_enabled, - category: 'view_data', priority: 101, label: gettext('View All Rows') + category: 'view_data', priority: 101, label: gettext('All Rows') },{ name: 'view_first_100_rows_context_' + supported_nodes[idx], node: supported_nodes[idx], module: this, data: {mnuid: 1}, applies: ['context', 'object'], callback: 'show_data_grid', enable: view_menu_enabled, - category: 'view_data', priority: 102, label: gettext('View First 100 Rows') + category: 'view_data', priority: 102, label: gettext('First 100 Rows') },{ name: 'view_last_100_rows_context_' + supported_nodes[idx], node: supported_nodes[idx], module: this, data: {mnuid: 2}, applies: ['context', 'object'], callback: 'show_data_grid', enable: view_menu_enabled, - category: 'view_data', priority: 103, label: gettext('View Last 100 Rows') + category: 'view_data', priority: 103, label: gettext('Last 100 Rows') },{ name: 'view_filtered_rows_context_' + supported_nodes[idx], node: supported_nodes[idx], module: this, data: {mnuid: 4}, applies: ['context', 'object'], callback: 'show_filtered_row', enable: view_menu_enabled, - category: 'view_data', priority: 104, label: gettext('View Filtered Rows...') + category: 'view_data', priority: 104, label: gettext('Filtered Rows...') }); } - pgAdmin.Browser.add_menu_category('view_data', gettext('View Data'), 100, 'fa fa-th'); + pgAdmin.Browser.add_menu_category('view_data', gettext('View/Edit Data'), 100, 'fa fa-th'); pgAdmin.Browser.add_menus(menus); // Creating a new pgAdmin.Browser frame to show the data. @@ -263,7 +263,7 @@ define([ this.setContent($content.get(0)); // Apply CodeMirror to filter text area. - this.filter_obj = CodeMirror.fromTextArea($sql_filter.get(0), { + this.filter_obj = codemirror.fromTextArea($sql_filter.get(0), { lineNumbers: true, indentUnit: 4, mode: "text/x-pgsql", diff --git a/web/pgadmin/tools/sqleditor/static/css/sqleditor.css b/web/pgadmin/tools/sqleditor/static/css/sqleditor.css index 90ae255..528541d 100644 --- a/web/pgadmin/tools/sqleditor/static/css/sqleditor.css +++ b/web/pgadmin/tools/sqleditor/static/css/sqleditor.css @@ -462,7 +462,11 @@ input.editor-checkbox:focus { .slick-cell span[data-cell-type="row-header-selector"] { display: block; - text-align: right; + text-align: center; +} + +.slick-cell.selected span[data-cell-type="row-header-selector"] { + color: white; } #datagrid div.slick-header.ui-state-default { @@ -486,9 +490,12 @@ input.editor-checkbox:focus { .select-all-icon { margin-left: 9px; - margin-right: 9px; + margin-right: 4px; vertical-align: bottom; - float: right; + position: absolute; + bottom: 4px; + right: 0; +} } .slick-cell, .slick-headerrow-column { diff --git a/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js b/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js index 8d76fed..44c9f25 100644 --- a/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js +++ b/web/pgadmin/tools/sqleditor/templates/sqleditor/js/sqleditor.js @@ -3003,7 +3003,8 @@ define([ }); dataView.endUpdate(); grid.updateRowCount(); - grid.render(); + // Pasted row/s always append so bring last row in view port. + grid.scrollRowIntoView(dataView.getLength()); grid.setSelectedRows([]); } }, ^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: [pgadmin-hackers] Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] @ 2017-06-28 13:49 Dave Page <[email protected]> parent: Harshal Dhumal <[email protected]> 0 siblings, 0 replies; 24+ messages in thread From: Dave Page @ 2017-06-28 13:49 UTC (permalink / raw) To: Harshal Dhumal <[email protected]>; +Cc: Joao Pedro De Almeida Pereira <[email protected]>; pgadmin-hackers; Ashesh Vashi <[email protected]>; Robert Eckhardt <[email protected]>; Shirley Wang <[email protected]> Thanks, applied. On Wed, Jun 28, 2017 at 7:04 AM, Harshal Dhumal < [email protected]> wrote: > Hi, > > Please find attached patch to fix minor issues/improvements suggested. > > > On Wed, Jun 28, 2017 at 2:43 AM, Dave Page <[email protected]> wrote: > >> Harshal, can you look at these comments please, and patch as appropriate? >> >> On Tue, Jun 27, 2017 at 5:03 PM, Joao Pedro De Almeida Pereira < >> [email protected]> wrote: >> >>> Hello Hackers, >>> >>> When we started the app we noticed some change in the front end. >>> >>> The line numbers in the Editor: >>> - We noticed a bug with the numbers where at 10000 rows, the numbers >>> would be cut off. >>> >> No. Row number column automatically adjust its width depending on maximum > row count to be shown. (see attached screenshot ) > > [image: Inline image 1] > > > >> - When the row is selected the color of the text should be white >>> >> Fixed > > >> - Centering the number column would differentiate it from the rest of the >>> table and prevent any confusion >>> >> > Fixed. > >> >>> >>> Select all triangle: >>> - This seems to have shifted up and to the left slightly. >>> >> Fixed alignment. > > >> >>> While editing data, when you paste a new row, that line should come into >>> view. >>> >> Fixed. > > >> >>> Issue in the code: >>> - In sqleditor/command.py:473 the variable `column_data` is not >>> initialized >>> >> Fixed. > > >> >>> >>> >>> We were looking into the menu's to access and Edit Data editor, and we >>> noticed that in the menu it is still called "View Data". We are aware that >>> this change was not introduced in this patch, but we just noticed it. It >>> should be consistent. >>> >> Fixed. (see attached screenshot) > > [image: Inline image 2] > > >> >>> >>> >>> Thanks >>> Shirley & Joao >>> >>> >>> On Tue, Jun 27, 2017 at 12:19 PM, Robert Eckhardt <[email protected]> >>> wrote: >>> >>>> >>>> >>>> On Tue, Jun 27, 2017 at 12:16 PM, Khushboo Vashi < >>>> [email protected]> wrote: >>>> >>>>> >>>>> >>>>> On 27 Jun 2017 18:33, "Dave Page" <[email protected]> wrote: >>>>> >>>>> Thanks - patch committed! >>>>> >>>>> Awsome job :-) >>>>> >>>>> Gr8. Finally no more rebase request for Harshal. :) >>>>> >>>> >>>> +1 >>>> >>>> >>>>> On Tue, Jun 27, 2017 at 3:26 AM, Harshal Dhumal < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Please find rebased patch. >>>>>> >>>>>> -- >>>>>> *Harshal Dhumal* >>>>>> *Sr. Software Engineer* >>>>>> >>>>>> EnterpriseDB India: http://www.enterprisedb.com >>>>>> The Enterprise PostgreSQL Company >>>>>> >>>>>> On Mon, Jun 26, 2017 at 5:24 PM, Harshal Dhumal < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> yes i'm working on that only :) >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Harshal Dhumal* >>>>>>> *Sr. Software Engineer* >>>>>>> >>>>>>> EnterpriseDB India: http://www.enterprisedb.com >>>>>>> The Enterprise PostgreSQL Company >>>>>>> >>>>>>> On Mon, Jun 26, 2017 at 5:22 PM, Dave Page <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> I'm sorry, it needs rebasing again. If you can do it quickly, I'll >>>>>>>> make sure it's the next patch I work on in that area. >>>>>>>> >>>>>>>> Thanks. >>>>>>>> >>>>>>>> On Mon, Jun 26, 2017 at 5:16 AM, Harshal Dhumal >>>>>>>> <[email protected]> wrote: >>>>>>>> > Hi Dave, >>>>>>>> > >>>>>>>> > Please find updated rebased patch for RM2137 >>>>>>>> > >>>>>>>> > On Fri, Jun 23, 2017 at 9:00 PM, Dave Page <[email protected]> >>>>>>>> wrote: >>>>>>>> >> >>>>>>>> >> Hi Harshal, >>>>>>>> >> >>>>>>>> >> When can we expect an updated version of this patch? I think it's >>>>>>>> >> important to get this into the next release. >>>>>>>> >> >>>>>>>> >> Thanks! >>>>>>>> >> >>>>>>>> >> On Fri, Jun 16, 2017 at 10:55 AM, Dave Page <[email protected]> >>>>>>>> wrote: >>>>>>>> >> > Hi, >>>>>>>> >> > >>>>>>>> >> > That's better - the failures are far less random now :-). I >>>>>>>> got the >>>>>>>> >> > following two though, on both PG and EPAS 9.5: >>>>>>>> >> > >>>>>>>> >> > ============================================================ >>>>>>>> ========== >>>>>>>> >> > ERROR: runTest >>>>>>>> >> > (pgadmin.feature_tests.query_tool_tests.QueryToolFeatureTest) >>>>>>>> >> > Query tool feature test >>>>>>>> >> > ------------------------------------------------------------ >>>>>>>> ---------- >>>>>>>> >> > Traceback (most recent call last): >>>>>>>> >> > File >>>>>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_t >>>>>>>> ool_tests.py", >>>>>>>> >> > line 95, in runTest >>>>>>>> >> > self._query_tool_explain_analyze_buffers() >>>>>>>> >> > File >>>>>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/query_t >>>>>>>> ool_tests.py", >>>>>>>> >> > line 443, in _query_tool_explain_analyze_buffers >>>>>>>> >> > canvas.find_element_by_xpath("//*[contains(string(), >>>>>>>> 'Shared Read >>>>>>>> >> > Blocks')]") >>>>>>>> >> > File >>>>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>>>> ges/selenium/webdriver/remote/webelement.py", >>>>>>>> >> > line 260, in find_element_by_xpath >>>>>>>> >> > return self.find_element(by=By.XPATH, value=xpath) >>>>>>>> >> > File >>>>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>>>> ges/selenium/webdriver/remote/webelement.py", >>>>>>>> >> > line 508, in find_element >>>>>>>> >> > {"using": by, "value": value})['value'] >>>>>>>> >> > File >>>>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>>>> ges/selenium/webdriver/remote/webelement.py", >>>>>>>> >> > line 491, in _execute >>>>>>>> >> > return self._parent.execute(command, params) >>>>>>>> >> > File >>>>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>>>> ges/selenium/webdriver/remote/webdriver.py", >>>>>>>> >> > line 238, in execute >>>>>>>> >> > self.error_handler.check_response(response) >>>>>>>> >> > File >>>>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>>>> ges/selenium/webdriver/remote/errorhandler.py", >>>>>>>> >> > line 193, in check_response >>>>>>>> >> > raise exception_class(message, screen, stacktrace) >>>>>>>> >> > NoSuchElementException: Message: no such element: Unable to >>>>>>>> locate >>>>>>>> >> > element: {"method":"xpath","selector":"//*[contains(string(), >>>>>>>> 'Shared >>>>>>>> >> > Read Blocks')]"} >>>>>>>> >> > (Session info: chrome=58.0.3029.110) >>>>>>>> >> > (Driver info: chromedriver=2.29.461585 >>>>>>>> >> > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X >>>>>>>> 10.12.3 >>>>>>>> >> > x86_64) >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> >> > ============================================================ >>>>>>>> ========== >>>>>>>> >> > ERROR: runTest >>>>>>>> >> > (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDat >>>>>>>> aTest) >>>>>>>> >> > Validate Insert, Update operations in View data with given >>>>>>>> test data >>>>>>>> >> > ------------------------------------------------------------ >>>>>>>> ---------- >>>>>>>> >> > Traceback (most recent call last): >>>>>>>> >> > File >>>>>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>>>>>> ta_dml_queries.py", >>>>>>>> >> > line 104, in runTest >>>>>>>> >> > self._add_row() >>>>>>>> >> > File >>>>>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>>>>>> ta_dml_queries.py", >>>>>>>> >> > line 255, in _add_row >>>>>>>> >> > self._update_cell(cell_xpath, config_data[str(idx)]) >>>>>>>> >> > File >>>>>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>>>>>> ta_dml_queries.py", >>>>>>>> >> > line 164, in _update_cell >>>>>>>> >> > cell_el = self.page.find_by_xpath(xpath) >>>>>>>> >> > File >>>>>>>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>>>>>>> min_page.py", >>>>>>>> >> > line 122, in find_by_xpath >>>>>>>> >> > return self.wait_for_element(lambda driver: >>>>>>>> >> > driver.find_element_by_xpath(xpath)) >>>>>>>> >> > File >>>>>>>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>>>>>>> min_page.py", >>>>>>>> >> > line 205, in wait_for_element >>>>>>>> >> > return self._wait_for("element to exist", >>>>>>>> element_if_it_exists) >>>>>>>> >> > File >>>>>>>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>>>>>>> min_page.py", >>>>>>>> >> > line 255, in _wait_for >>>>>>>> >> > "Timed out waiting for " + waiting_for_message) >>>>>>>> >> > File >>>>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>>>> ges/selenium/webdriver/support/wait.py", >>>>>>>> >> > line 71, in until >>>>>>>> >> > value = method(self._driver) >>>>>>>> >> > File >>>>>>>> >> > "/Users/dpage/git/pgadmin4/web/regression/feature_utils/pgad >>>>>>>> min_page.py", >>>>>>>> >> > line 200, in element_if_it_exists >>>>>>>> >> > if element.is_displayed() and element.is_enabled(): >>>>>>>> >> > File >>>>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>>>> ges/selenium/webdriver/remote/webelement.py", >>>>>>>> >> > line 358, in is_displayed >>>>>>>> >> > return self._execute(Command.IS_ELEME >>>>>>>> NT_DISPLAYED)['value'] >>>>>>>> >> > File >>>>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>>>> ges/selenium/webdriver/remote/webelement.py", >>>>>>>> >> > line 491, in _execute >>>>>>>> >> > return self._parent.execute(command, params) >>>>>>>> >> > File >>>>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>>>> ges/selenium/webdriver/remote/webdriver.py", >>>>>>>> >> > line 238, in execute >>>>>>>> >> > self.error_handler.check_response(response) >>>>>>>> >> > File >>>>>>>> >> > "/Users/dpage/.virtualenvs/pgadmin4/lib/python2.7/site-packa >>>>>>>> ges/selenium/webdriver/remote/errorhandler.py", >>>>>>>> >> > line 193, in check_response >>>>>>>> >> > raise exception_class(message, screen, stacktrace) >>>>>>>> >> > StaleElementReferenceException: Message: stale element >>>>>>>> reference: >>>>>>>> >> > element is not attached to the page document >>>>>>>> >> > (Session info: chrome=58.0.3029.110) >>>>>>>> >> > (Driver info: chromedriver=2.29.461585 >>>>>>>> >> > (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac OS X >>>>>>>> 10.12.3 >>>>>>>> >> > x86_64) >>>>>>>> >> > >>>>>>>> > >>>>>>>> > >>>>>>>> > I checked my feature test cases for any database server >>>>>>>> version/type (PG, >>>>>>>> > EPAS) specific failures and I found that test cases are working >>>>>>>> fine on all >>>>>>>> > of them. However I have slightly modified test case to overcome >>>>>>>> above >>>>>>>> > mentioned failures. >>>>>>>> > >>>>>>>> >> >>>>>>>> >> > >>>>>>>> >> > On 9.4 everything passes. On 9.6, I get one failure: >>>>>>>> >> > >>>>>>>> >> > ============================================================ >>>>>>>> ========== >>>>>>>> >> > FAIL: runTest >>>>>>>> >> > (pgadmin.feature_tests.view_data_dml_queries.CheckForViewDat >>>>>>>> aTest) >>>>>>>> >> > Validate Insert, Update operations in View data with given >>>>>>>> test data >>>>>>>> >> > ------------------------------------------------------------ >>>>>>>> ---------- >>>>>>>> >> > Traceback (most recent call last): >>>>>>>> >> > File >>>>>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>>>>>> ta_dml_queries.py", >>>>>>>> >> > line 105, in runTest >>>>>>>> >> > self._verify_row_data(True) >>>>>>>> >> > File >>>>>>>> >> > "/Users/dpage/git/pgadmin4/web/pgadmin/feature_tests/view_da >>>>>>>> ta_dml_queries.py", >>>>>>>> >> > line 282, in _verify_row_data >>>>>>>> >> > self.assertEquals(cells[idx], config_data[str(idx)][1]) >>>>>>>> >> > AssertionError: u'[default]' != u'1' >>>>>>>> >> > - [default] >>>>>>>> >> > + 1 >>>>>>>> >> > >>>>>>>> > >>>>>>>> > I guess Khushboo has fixed this recently. >>>>>>>> > >>>>>>>> > >>>>>>>> >> >>>>>>>> >> > Screenshots attached. >>>>>>>> >> > >>>>>>>> >> > On Fri, Jun 16, 2017 at 7:54 AM, Harshal Dhumal >>>>>>>> >> > <[email protected]> wrote: >>>>>>>> >> >> Hi Dave, >>>>>>>> >> >> >>>>>>>> >> >> Please find attached patch where I have added timeout of 2 >>>>>>>> seconds >>>>>>>> >> >> before >>>>>>>> >> >> selecting object menu in connect to server function. >>>>>>>> >> >> >>>>>>>> >> >> Note: Apply this patch on top of previous patch for on demand >>>>>>>> loading >>>>>>>> >> >> feature. >>>>>>>> >> >> >>>>>>>> >> >> >>>>>>>> >> >> -- >>>>>>>> >> >> Harshal Dhumal >>>>>>>> >> >> Sr. Software Engineer >>>>>>>> >> >> >>>>>>>> >> >> EnterpriseDB India: http://www.enterprisedb.com >>>>>>>> >> >> The Enterprise PostgreSQL Company >>>>>>>> >> >> >>>>>>>> >> >> On Fri, Jun 16, 2017 at 2:25 AM, Dave Page <[email protected]> >>>>>>>> wrote: >>>>>>>> >> >>> >>>>>>>> >> >>> Sounds good, thanks. >>>>>>>> >> >>> >>>>>>>> >> >>> On Thu, Jun 15, 2017 at 9:54 PM, Harshal Dhumal >>>>>>>> >> >>> <[email protected]> wrote: >>>>>>>> >> >>> > Hi >>>>>>>> >> >>> > >>>>>>>> >> >>> > On Fri, Jun 16, 2017 at 2:07 AM, Dave Page < >>>>>>>> [email protected]> >>>>>>>> >> >>> > wrote: >>>>>>>> >> >>> >> >>>>>>>> >> >>> >> Hi >>>>>>>> >> >>> >> >>>>>>>> >> >>> >> On Thu, Jun 15, 2017 at 9:30 PM, Harshal Dhumal >>>>>>>> >> >>> >> <[email protected]> wrote: >>>>>>>> >> >>> >> > Hi Dave, >>>>>>>> >> >>> >> > >>>>>>>> >> >>> >> > Please find attached updated patch. >>>>>>>> >> >>> >> > >>>>>>>> >> >>> >> > On Thu, Jun 15, 2017 at 3:58 PM, Dave Page < >>>>>>>> [email protected]> >>>>>>>> >> >>> >> > wrote: >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> Hi >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> On Wed, Jun 14, 2017 at 11:36 PM, Harshal Dhumal >>>>>>>> >> >>> >> >> <[email protected]> wrote: >>>>>>>> >> >>> >> >>> >>>>>>>> >> >>> >> >>> Hi Dave, >>>>>>>> >> >>> >> >>> >>>>>>>> >> >>> >> >>> Please find rebased patch for RM2137. >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> Looking very good. The only issues I see are: >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> - The row headers should auto-size such that they can >>>>>>>> display >>>>>>>> >> >>> >> >> the >>>>>>>> >> >>> >> >> row >>>>>>>> >> >>> >> >> numbers if the last row was displayed. E.g. if there >>>>>>>> are 12345 >>>>>>>> >> >>> >> >> rows >>>>>>>> >> >>> >> >> in >>>>>>>> >> >>> >> >> total, then the row header should be sized to display >>>>>>>> 5 digits. >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> > Fixed. >>>>>>>> >> >>> >> > >>>>>>>> >> >>> >> > >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> - The tests are comprehensive, which is awesome. >>>>>>>> However, every >>>>>>>> >> >>> >> >> time >>>>>>>> >> >>> >> >> I >>>>>>>> >> >>> >> >> ran >>>>>>>> >> >>> >> >> them, at least one of the feature tests failed. >>>>>>>> Unfortunately, >>>>>>>> >> >>> >> >> it >>>>>>>> >> >>> >> >> was a >>>>>>>> >> >>> >> >> different one each time. In the last two runs, I got: >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> ============================== >>>>>>>> ======================================== >>>>>>>> >> >>> >> >> ERROR: runTest >>>>>>>> >> >>> >> >> (pgadmin.feature_tests.query_t >>>>>>>> ool_tests.QueryToolFeatureTest) >>>>>>>> >> >>> >> >> Query tool feature test >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> ------------------------------ >>>>>>>> ---------------------------------------- >>>>>>>> >> >>> >> >> Traceback (most recent call last): >>>>>>>> >> >>> >> >> File >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>>>>>> /regression/feature_utils/base_feature_test.py", >>>>>>>> >> >>> >> >> line 40, in setUp >>>>>>>> >> >>> >> >> self.before() >>>>>>>> >> >>> >> >> File >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>>>>>> /pgadmin/feature_tests/query_tool_tests.py", >>>>>>>> >> >>> >> >> line 40, in before >>>>>>>> >> >>> >> >> self._connects_to_server() >>>>>>>> >> >>> >> >> File >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>>>>>> /pgadmin/feature_tests/query_tool_tests.py", >>>>>>>> >> >>> >> >> line 144, in _connects_to_server >>>>>>>> >> >>> >> >> self.page.driver.find_element >>>>>>>> _by_link_text("Create"))\ >>>>>>>> >> >>> >> >> File >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>>>>> webdriver.py", >>>>>>>> >> >>> >> >> line 319, in find_element_by_link_text >>>>>>>> >> >>> >> >> return self.find_element(by=By.LINK_TEXT, >>>>>>>> value=link_text) >>>>>>>> >> >>> >> >> File >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>>>>> webdriver.py", >>>>>>>> >> >>> >> >> line 756, in find_element >>>>>>>> >> >>> >> >> 'value': value})['value'] >>>>>>>> >> >>> >> >> File >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>>>>> webdriver.py", >>>>>>>> >> >>> >> >> line 238, in execute >>>>>>>> >> >>> >> >> self.error_handler.check_response(response) >>>>>>>> >> >>> >> >> File >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>>>>> errorhandler.py", >>>>>>>> >> >>> >> >> line 193, in check_response >>>>>>>> >> >>> >> >> raise exception_class(message, screen, stacktrace) >>>>>>>> >> >>> >> >> NoSuchElementException: Message: no such element: >>>>>>>> Unable to >>>>>>>> >> >>> >> >> locate >>>>>>>> >> >>> >> >> element: {"method":"link text","selector":"Create"} >>>>>>>> >> >>> >> >> (Session info: chrome=58.0.3029.110) >>>>>>>> >> >>> >> >> (Driver info: chromedriver=2.29.461585 >>>>>>>> >> >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac >>>>>>>> OS X >>>>>>>> >> >>> >> >> 10.12.3 >>>>>>>> >> >>> >> >> x86_64) >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> > I checked code from _connects_to_server function which >>>>>>>> is common >>>>>>>> >> >>> >> > in >>>>>>>> >> >>> >> > all >>>>>>>> >> >>> >> > features test cases. I didn't find anything wrong with >>>>>>>> this. If >>>>>>>> >> >>> >> > there >>>>>>>> >> >>> >> > is >>>>>>>> >> >>> >> > a >>>>>>>> >> >>> >> > bug in this function then all feature test must fail. >>>>>>>> >> >>> >> > Let me know if you are getting failure consistently in >>>>>>>> >> >>> >> > _connects_to_server >>>>>>>> >> >>> >> > function. >>>>>>>> >> >>> >> >>>>>>>> >> >>> >> I wondered if that one is a race condition. Do we need a >>>>>>>> short >>>>>>>> >> >>> >> delay >>>>>>>> >> >>> >> before clicking the Object menu? I have seen this >>>>>>>> occasionally >>>>>>>> >> >>> >> before. >>>>>>>> >> >>> > >>>>>>>> >> >>> > >>>>>>>> >> >>> > OK. In that case let's try putting 1-2 second delay and >>>>>>>> observer >>>>>>>> >> >>> > behaviour. >>>>>>>> >> >>> > I'll send separate patch for this tomorrow as this is not >>>>>>>> related to >>>>>>>> >> >>> > on >>>>>>>> >> >>> > demand query result feature or its test cases. >>>>>>>> >> >>> > >>>>>>>> >> >>> >> >>>>>>>> >> >>> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> ============================== >>>>>>>> ======================================== >>>>>>>> >> >>> >> >> ERROR: runTest >>>>>>>> >> >>> >> >> (pgadmin.feature_tests.query_t >>>>>>>> ool_tests.QueryToolFeatureTest) >>>>>>>> >> >>> >> >> Query tool feature test >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> ------------------------------ >>>>>>>> ---------------------------------------- >>>>>>>> >> >>> >> >> Traceback (most recent call last): >>>>>>>> >> >>> >> >> File >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>>>>>> /pgadmin/feature_tests/query_tool_tests.py", >>>>>>>> >> >>> >> >> line 119, in runTest >>>>>>>> >> >>> >> >> self._query_tool_auto_rollback_enabled() >>>>>>>> >> >>> >> >> File >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> "/Users/dpage/git/pgadmin4/web >>>>>>>> /pgadmin/feature_tests/query_tool_tests.py", >>>>>>>> >> >>> >> >> line 697, in _query_tool_auto_rollback_enabled >>>>>>>> >> >>> >> >> '//div[contains(@class, "sql-editor-message") and >>>>>>>> >> >>> >> >> contains(string(), >>>>>>>> >> >>> >> >> "COMMIT")]' >>>>>>>> >> >>> >> >> File >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>>>>> webdriver.py", >>>>>>>> >> >>> >> >> line 295, in find_element_by_xpath >>>>>>>> >> >>> >> >> return self.find_element(by=By.XPATH, value=xpath) >>>>>>>> >> >>> >> >> File >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>>>>> webdriver.py", >>>>>>>> >> >>> >> >> line 756, in find_element >>>>>>>> >> >>> >> >> 'value': value})['value'] >>>>>>>> >> >>> >> >> File >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>>>>> webdriver.py", >>>>>>>> >> >>> >> >> line 238, in execute >>>>>>>> >> >>> >> >> self.error_handler.check_response(response) >>>>>>>> >> >>> >> >> File >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> "/Users/dpage/.virtualenvs/pga >>>>>>>> dmin4/lib/python2.7/site-packages/selenium/webdriver/remote/ >>>>>>>> errorhandler.py", >>>>>>>> >> >>> >> >> line 193, in check_response >>>>>>>> >> >>> >> >> raise exception_class(message, screen, stacktrace) >>>>>>>> >> >>> >> >> NoSuchElementException: Message: no such element: >>>>>>>> Unable to >>>>>>>> >> >>> >> >> locate >>>>>>>> >> >>> >> >> element: {"method":"xpath","selector":" >>>>>>>> //div[contains(@class, >>>>>>>> >> >>> >> >> "sql-editor-message") and contains(string(), >>>>>>>> "COMMIT")]"} >>>>>>>> >> >>> >> >> (Session info: chrome=58.0.3029.110) >>>>>>>> >> >>> >> >> (Driver info: chromedriver=2.29.461585 >>>>>>>> >> >>> >> >> (0be2cd95f834e9ee7c46bcc7cf405b483f5ae83b),platform=Mac >>>>>>>> OS X >>>>>>>> >> >>> >> >> 10.12.3 >>>>>>>> >> >>> >> >> x86_64) >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> > I have updated Auto rollback enabled test in this patch. >>>>>>>> >> >>> >> > >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> Relevant screenshots attached. >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> - Can you tidy up the regression output a little >>>>>>>> please? Instead >>>>>>>> >> >>> >> >> of: >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> ------- >>>>>>>> >> >>> >> >> runTest >>>>>>>> >> >>> >> >> (pgadmin.feature_tests.query_t >>>>>>>> ool_tests.QueryToolFeatureTest) >>>>>>>> >> >>> >> >> Query tool feature test ... On demand result set on >>>>>>>> scrolling... >>>>>>>> >> >>> >> >> OK. >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> On demand result set on grid select all... >>>>>>>> >> >>> >> >> OK. >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> On demand result set on column select all... >>>>>>>> >> >>> >> >> OK. >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> Explain query... >>>>>>>> >> >>> >> >> OK. >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> Explain query with verbose... >>>>>>>> >> >>> >> >> OK. >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> Explain query with costs... >>>>>>>> >> >>> >> >> OK. >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> Explain analyze query... >>>>>>>> >> >>> >> >> OK. >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> Explain analyze query with buffers... >>>>>>>> >> >>> >> >> OK. >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> Explain analyze query with timing... >>>>>>>> >> >>> >> >> OK. >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> Auto commit disabled... >>>>>>>> >> >>> >> >> OK. >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> Auto commit enabled... >>>>>>>> >> >>> >> >> OK. >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> Auto rollback enabled... >>>>>>>> >> >>> >> >> ERROR >>>>>>>> >> >>> >> >> ------- >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> Something like: >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> ------- >>>>>>>> >> >>> >> >> runTest >>>>>>>> >> >>> >> >> (pgadmin.feature_tests.query_t >>>>>>>> ool_tests.QueryToolFeatureTest) >>>>>>>> >> >>> >> >> Query tool feature test ... >>>>>>>> >> >>> >> >> On demand result set on scrolling... OK. >>>>>>>> >> >>> >> >> On demand result set on grid select all... OK. >>>>>>>> >> >>> >> >> On demand result set on column select all... OK. >>>>>>>> >> >>> >> >> Explain query... OK. >>>>>>>> >> >>> >> >> Explain query with verbose... OK. >>>>>>>> >> >>> >> >> Explain query with costs... OK. >>>>>>>> >> >>> >> >> Explain analyze query... OK. >>>>>>>> >> >>> >> >> Explain analyze query with buffers... OK. >>>>>>>> >> >>> >> >> Explain analyze query with timing... OK. >>>>>>>> >> >>> >> >> Auto commit disabled... OK. >>>>>>>> >> >>> >> >> Auto commit enabled... OK. >>>>>>>> >> >>> >> >> Auto rollback enabled... ERROR >>>>>>>> >> >>> >> >> -------- >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> > Fixed. >>>>>>>> >> >>> >> > >>>>>>>> >> >>> >> > >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> Thanks! >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> -- >>>>>>>> >> >>> >> >> Dave Page >>>>>>>> >> >>> >> >> Blog: http://pgsnake.blogspot.com >>>>>>>> >> >>> >> >> Twitter: @pgsnake >>>>>>>> >> >>> >> >> >>>>>>>> >> >>> >> >> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>> >> >>> >> >> The Enterprise PostgreSQL Company >>>>>>>> >> >>> >> > >>>>>>>> >> >>> >> > >>>>>>>> >> >>> >> >>>>>>>> >> >>> >> >>>>>>>> >> >>> >> >>>>>>>> >> >>> >> -- >>>>>>>> >> >>> >> Dave Page >>>>>>>> >> >>> >> Blog: http://pgsnake.blogspot.com >>>>>>>> >> >>> >> Twitter: @pgsnake >>>>>>>> >> >>> >> >>>>>>>> >> >>> >> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>> >> >>> >> The Enterprise PostgreSQL Company >>>>>>>> >> >>> > >>>>>>>> >> >>> > >>>>>>>> >> >>> >>>>>>>> >> >>> >>>>>>>> >> >>> >>>>>>>> >> >>> -- >>>>>>>> >> >>> Dave Page >>>>>>>> >> >>> Blog: http://pgsnake.blogspot.com >>>>>>>> >> >>> Twitter: @pgsnake >>>>>>>> >> >>> >>>>>>>> >> >>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>> >> >>> The Enterprise PostgreSQL Company >>>>>>>> >> >> >>>>>>>> >> >> >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> >> > -- >>>>>>>> >> > Dave Page >>>>>>>> >> > Blog: http://pgsnake.blogspot.com >>>>>>>> >> > Twitter: @pgsnake >>>>>>>> >> > >>>>>>>> >> > EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>> >> > The Enterprise PostgreSQL Company >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> -- >>>>>>>> >> Dave Page >>>>>>>> >> Blog: http://pgsnake.blogspot.com >>>>>>>> >> Twitter: @pgsnake >>>>>>>> >> >>>>>>>> >> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>> >> The Enterprise PostgreSQL Company >>>>>>>> > >>>>>>>> > >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Dave Page >>>>>>>> Blog: http://pgsnake.blogspot.com >>>>>>>> Twitter: @pgsnake >>>>>>>> >>>>>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>>>>> The Enterprise PostgreSQL Company >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Dave Page >>>>> Blog: http://pgsnake.blogspot.com >>>>> Twitter: @pgsnake >>>>> >>>>> EnterpriseDB UK: http://www.enterprisedb.com >>>>> The Enterprise PostgreSQL Company >>>>> >>>>> >>>>> >>>> >>> >> >> >> -- >> Dave Page >> Blog: http://pgsnake.blogspot.com >> Twitter: @pgsnake >> >> EnterpriseDB UK: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> > > -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company Attachments: [image/png] View_Edit_menu.png (47.3K, 3-View_Edit_menu.png) download | view image [image/png] auto_resize_row_number_column.png (32.3K, 4-auto_resize_row_number_column.png) download | view image ^ permalink raw reply [nested|flat] 24+ messages in thread
end of thread, other threads:[~2017-06-28 13:49 UTC | newest] Thread overview: 24+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2017-06-14 14:25 Re: Re: Server side cursor limitations for on demand loading of data in query tool [RM2137] [pgAdmin4] Harshal Dhumal <[email protected]> 2017-06-14 22:36 ` Harshal Dhumal <[email protected]> 2017-06-15 10:28 ` Dave Page <[email protected]> 2017-06-15 20:30 ` Harshal Dhumal <[email protected]> 2017-06-15 20:37 ` Dave Page <[email protected]> 2017-06-15 20:54 ` Harshal Dhumal <[email protected]> 2017-06-15 20:55 ` Dave Page <[email protected]> 2017-06-16 06:54 ` Harshal Dhumal <[email protected]> 2017-06-16 09:55 ` Dave Page <[email protected]> 2017-06-23 15:30 ` Dave Page <[email protected]> 2017-06-26 09:16 ` Harshal Dhumal <[email protected]> 2017-06-26 11:52 ` Dave Page <[email protected]> 2017-06-26 11:54 ` Harshal Dhumal <[email protected]> 2017-06-27 07:26 ` Harshal Dhumal <[email protected]> 2017-06-27 13:03 ` Dave Page <[email protected]> 2017-06-27 13:07 ` Ashesh Vashi <[email protected]> 2017-06-27 16:16 ` Khushboo Vashi <[email protected]> 2017-06-27 16:19 ` Robert Eckhardt <[email protected]> 2017-06-27 21:03 ` Joao Pedro De Almeida Pereira <[email protected]> 2017-06-27 21:13 ` Dave Page <[email protected]> 2017-06-28 11:04 ` Harshal Dhumal <[email protected]> 2017-06-28 13:49 ` Dave Page <[email protected]> 2017-06-27 21:18 ` Robert Eckhardt <[email protected]> 2017-06-27 21:21 ` Dave Page <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox