public inbox for [email protected]
help / color / mirror / Atom feedFrom: Rahul Shirsat <[email protected]>
To: Aditya Toshniwal <[email protected]>
Cc: pgadmin-hackers <[email protected]>
Cc: Akshay Joshi <[email protected]>
Subject: Re: [pgAdmin4] RM5965 Couldn't download file of Marcos query results
Date: Thu, 10 Dec 2020 18:54:56 +0530
Message-ID: <CAKtn9dMhuna1=+L8EgY-GzEKyYWw4Cut0-QBvxhukiP0HQYm-A@mail.gmail.com> (raw)
In-Reply-To: <CAM9w-_nz8o7=ewwn8Qw8j7CRMi12o=vOh8yGGbNGPUBWZ=hScw@mail.gmail.com>
References: <CAKtn9dNGjVVoDCQUo0k_7hF1PKPivDmYTE1+2ds+g5wufNBgtQ@mail.gmail.com>
<CAKtn9dPWeCXuToAyOSE3Mqdg-b0Qkx3vcRX30nuUrxabY-O-+Q@mail.gmail.com>
<CANxoLDc+3YGFSJ8hvuJjVBVBTCe=yFDov_DqTGa5D30m0wKeAQ@mail.gmail.com>
<CAM9w-_nz8o7=ewwn8Qw8j7CRMi12o=vOh8yGGbNGPUBWZ=hScw@mail.gmail.com>
Hi Aditya/Akshay,
On Wed, Dec 9, 2020 at 2:59 PM Aditya Toshniwal <
[email protected]> wrote:
> Hello Rahul,
>
> Found below issues:
> 1) The data grid is not fetching more than 1000 records now, gives a
> console error:
> sqleditor.js:1264 Uncaught TypeError: self.enable_disable_download_btn is
> not a function
> at child.fetch_next (sqleditor.js:1264)
>
*Fixed*
>
> 2) The sqleditor test cases are failing.
>
*This has been taken care of, and added more test cases. *
> 3) Fix pep8 issues.
> 4) Fix linter issues.
>
*Sorry for these issues, last minute code changes* 😣
> 5) Please check the doc changes again, it's not clear to me.
>
*This has been corrected now.*
>
> I didn't check but make sure the GUI tests for the sqleditor runs fine
> since the behaviour has changed now.
>
*These are fixed now.*
>
> On Wed, Dec 9, 2020 at 1:53 PM Akshay Joshi <[email protected]>
> wrote:
>
>> Hi Aditya
>>
>> Can you please review this patch?
>>
>> On Mon, Dec 7, 2020 at 3:10 PM Rahul Shirsat <
>> [email protected]> wrote:
>>
>>> Hi Hackers,
>>>
>>> Please find the updated patch below.
>>>
>>> On Mon, Dec 7, 2020 at 2:55 PM Rahul Shirsat <
>>> [email protected]> wrote:
>>>
>>>> Hi Hackers,
>>>>
>>>> Please find the attached patch which resolves the issue of macros query
>>>> results download, have used async cursor to achieve this functionality,
>>>> where for downloading the results, cursor is scrolled back to 0 and end of
>>>> the records, and reset again while user scrolling on Data Output table.
>>>>
>>>> QA/Reviewer needs to observe below issues if it occurs:
>>>>
>>>> 1. If records are more like 5000 or 10000, try to fetch records by
>>>> scrolling at least (2000), now save the results by clicking the download
>>>> button, it should save the file, and now try scrolling again, the data
>>>> should be shown continuously and not any abrupt end or unexpected records
>>>> order.
>>>> 2. Also, the download button is now "Save results to CSV/TXT" where
>>>> it will only get enabled when there are valid records in the Data Output.
>>>>
>>>>
>>>> Also a minor fix of the add folder icon issue is also added into this
>>>> patch.
>>>>
>>>> --
>>>> *Rahul Shirsat*
>>>> Senior Software Engineer | EnterpriseDB Corporation.
>>>>
>>>
>>>
>>> --
>>> *Rahul Shirsat*
>>> Software Engineer | EnterpriseDB Corporation.
>>>
>>
>>
>> --
>> *Thanks & Regards*
>> *Akshay Joshi*
>> *pgAdmin Hacker | Principal Software Architect*
>> *EDB Postgres <http://edbpostgres.com>*
>>
>> *Mobile: +91 976-788-8246*
>>
>
>
> --
> Thanks,
> Aditya Toshniwal
> pgAdmin hacker | Sr. Software Engineer | *edbpostgres.com*
> <http://edbpostgres.com;
> "Don't Complain about Heat, Plant a TREE"
>
--
*Rahul Shirsat*
Software Engineer | EnterpriseDB Corporation.
Attachments:
[application/octet-stream] RM5965_v3.patch (40.5K, 3-RM5965_v3.patch)
download | inline diff:
diff --git a/docs/en_US/query_tool.rst b/docs/en_US/query_tool.rst
index 880ca4ffb..2d0530ea4 100644
--- a/docs/en_US/query_tool.rst
+++ b/docs/en_US/query_tool.rst
@@ -122,7 +122,7 @@ You can:
* Select and copy from the displayed result set.
* Use the *Execute/Refresh* options to retrieve query execution information and
set query execution options.
-* Use the *Download as CSV/TXT* icon to download the content of the *Data Output*
+* Use the *Save results to file* icon to save the content of the *Data Output*
tab as a comma-delimited file.
* Edit the data in the result set of a SELECT query if it is updatable.
diff --git a/docs/en_US/query_tool_toolbar.rst b/docs/en_US/query_tool_toolbar.rst
index 2168c3930..3db3b563e 100644
--- a/docs/en_US/query_tool_toolbar.rst
+++ b/docs/en_US/query_tool_toolbar.rst
@@ -179,10 +179,10 @@ Query Execution
| | | |
| | * Select *Clear History* to erase the content of the *History* tab. | |
+----------------------+---------------------------------------------------------------------------------------------------+----------------+
- | *Download as CSV/TXT*| Click the *Download as CSV/TXT* icon to download the result set of the current query as a *.csv* | F8 |
- | | or as a *.txt* file. if *CSV field seperator* set to comma(,) else as a *.txt* file. | |
- | | You can specify the CSV/TXT settings through *Preferences -> SQL Editor -> CSV/TXT output* | |
- | | dialogue. | |
+ | *Save results to* | Click the Save results to file icon to save the result set of the current query as a delimited | F8 |
+ | *file* | text file (CSV, if the field separator is set to a comma). This button will only be enabled when | |
+ | | a query has been executed and there are results in the data grid. You can specify the CSV/TXT | |
+ | | settings in the Preference Dialogue under SQL Editor -> CSV/TXT output. | |
+----------------------+---------------------------------------------------------------------------------------------------+----------------+
| *Macros* | Click the *Macros* icon to manage the macros. You can create, edit or clear the macros through | |
| | the *Manage Macros* option. | |
diff --git a/web/pgadmin/static/js/keyboard_shortcuts.js b/web/pgadmin/static/js/keyboard_shortcuts.js
index abda8d8f2..0e84aba3e 100644
--- a/web/pgadmin/static/js/keyboard_shortcuts.js
+++ b/web/pgadmin/static/js/keyboard_shortcuts.js
@@ -191,7 +191,7 @@ function keyboardShortcutsQueryTool(
let executeKeys = sqlEditorController.preferences.execute_query;
let explainKeys = sqlEditorController.preferences.explain_query;
let explainAnalyzeKeys = sqlEditorController.preferences.explain_analyze_query;
- let downloadCsvKeys = sqlEditorController.preferences.download_csv;
+ let downloadCsvKeys = sqlEditorController.preferences.download_results;
let nextTabKeys = sqlEditorController.preferences.move_next;
let previousTabKeys = sqlEditorController.preferences.move_previous;
let switchPanelKeys = sqlEditorController.preferences.switch_panel;
@@ -211,8 +211,10 @@ function keyboardShortcutsQueryTool(
this._stopEventPropagation(event);
queryToolActions.explainAnalyze(sqlEditorController);
} else if (this.validateShortcutKeys(downloadCsvKeys, event)) {
- this._stopEventPropagation(event);
- queryToolActions.download(sqlEditorController);
+ if(!sqlEditorController.is_save_results_to_file_disabled) {
+ this._stopEventPropagation(event);
+ queryToolActions.download(sqlEditorController);
+ }
} else if (this.validateShortcutKeys(toggleCaseKeys, event)) {
this._stopEventPropagation(event);
queryToolActions.toggleCaseOfSelectedText(sqlEditorController);
diff --git a/web/pgadmin/static/js/sqleditor/call_render_after_poll.js b/web/pgadmin/static/js/sqleditor/call_render_after_poll.js
index d8d364722..2c52baf10 100644
--- a/web/pgadmin/static/js/sqleditor/call_render_after_poll.js
+++ b/web/pgadmin/static/js/sqleditor/call_render_after_poll.js
@@ -43,6 +43,7 @@ export function callRenderAfterPoll(sqlEditor, alertify, res) {
if (isNotificationEnabled(sqlEditor)) {
alertify.success(msg, sqlEditor.info_notifier_timeout);
}
+ sqlEditor.enable_disable_download_btn(true);
}
if (isQueryTool(sqlEditor)) {
diff --git a/web/pgadmin/static/js/sqleditor/execute_query.js b/web/pgadmin/static/js/sqleditor/execute_query.js
index 6c4069d24..0135f5bc7 100644
--- a/web/pgadmin/static/js/sqleditor/execute_query.js
+++ b/web/pgadmin/static/js/sqleditor/execute_query.js
@@ -83,6 +83,7 @@ class ExecuteQuery {
} else {
self.loadingScreen.hide();
self.enableSQLEditorButtons();
+ self.disableDownloadButton();
// Enable/Disable commit and rollback button.
if (result.data.data.transaction_status == queryTxnStatus.TRANSACTION_STATUS_INTRANS
|| result.data.data.transaction_status == queryTxnStatus.TRANSACTION_STATUS_INERROR) {
@@ -201,7 +202,7 @@ class ExecuteQuery {
this.loadingScreen.show(gettext('Running query...'));
$('#btn-flash').prop('disabled', true);
- $('#btn-download').prop('disabled', true);
+ this.disableDownloadButton();
this.sqlServerObject.query_start_time = new Date();
if (typeof sqlStatement === 'object') {
@@ -281,6 +282,10 @@ class ExecuteQuery {
}
}
+ disableDownloadButton() {
+ this.sqlServerObject.enable_disable_download_btn(true);
+ }
+
enableSQLEditorButtons() {
this.sqlServerObject.disable_tool_buttons(false);
}
diff --git a/web/pgadmin/static/js/sqleditor/query_tool_actions.js b/web/pgadmin/static/js/sqleditor/query_tool_actions.js
index 81937058c..a3e16bdff 100644
--- a/web/pgadmin/static/js/sqleditor/query_tool_actions.js
+++ b/web/pgadmin/static/js/sqleditor/query_tool_actions.js
@@ -81,13 +81,7 @@ let queryToolActions = {
},
download: function (sqlEditorController) {
- let sqlQuery = sqlEditorController.gridView.query_tool_obj.getSelection();
- if (!sqlQuery) {
- sqlQuery = sqlEditorController.gridView.query_tool_obj.getValue();
- }
-
- if (!sqlQuery) return;
let extension = sqlEditorController.preferences.csv_field_separator === ',' ? '.csv': '.txt';
let filename = 'data-' + new Date().getTime() + extension;
@@ -95,7 +89,7 @@ let queryToolActions = {
filename = sqlEditorController.table_name + extension;
}
- sqlEditorController.trigger_csv_download(sqlQuery, filename);
+ sqlEditorController.trigger_csv_download(filename);
},
commentBlockCode: function (sqlEditorController) {
diff --git a/web/pgadmin/static/js/sqleditor/query_tool_preferences.js b/web/pgadmin/static/js/sqleditor/query_tool_preferences.js
index ba4e7c882..a36328bdd 100644
--- a/web/pgadmin/static/js/sqleditor/query_tool_preferences.js
+++ b/web/pgadmin/static/js/sqleditor/query_tool_preferences.js
@@ -112,11 +112,11 @@ function updateUIPreferences(sqlEditor) {
.attr('aria-label',
shortcut_title(gettext('Explain Analyze'),preferences.explain_analyze_query));
- $el.find('#btn-download')
+ $el.find('#btn-save-results-to-file')
.attr('title',
- shortcut_title(gettext('Download as CSV/TXT'),preferences.download_csv))
+ shortcut_title(gettext('Save results to file'),preferences.download_results))
.attr('aria-label',
- shortcut_title(gettext('Download as CSV/TXT'),preferences.download_csv));
+ shortcut_title(gettext('Save results to file'),preferences.download_results));
$el.find('#btn-save-data')
.attr('title',
diff --git a/web/pgadmin/tools/datagrid/templates/datagrid/index.html b/web/pgadmin/tools/datagrid/templates/datagrid/index.html
index cbee98222..d9d0d2254 100644
--- a/web/pgadmin/tools/datagrid/templates/datagrid/index.html
+++ b/web/pgadmin/tools/datagrid/templates/datagrid/index.html
@@ -374,9 +374,9 @@
</ul>
</div>
<div class="btn-group" role="group" aria-label="">
- <button id="btn-download" type="button" class="btn btn-sm btn-primary-icon"
+ <button id="btn-save-results-to-file" type="button" class="btn btn-sm btn-primary-icon"
title=""
- tabindex="0">
+ tabindex="0" disabled>
<i class="fa fa-download sql-icon-lg" aria-hidden="true" role="img"></i>
</button>
</div>
diff --git a/web/pgadmin/tools/sqleditor/__init__.py b/web/pgadmin/tools/sqleditor/__init__.py
index 0784c5468..a9f5627c6 100644
--- a/web/pgadmin/tools/sqleditor/__init__.py
+++ b/web/pgadmin/tools/sqleditor/__init__.py
@@ -1336,7 +1336,7 @@ def start_query_download_tool(trans_id):
)
data = request.values if request.values else None
- if data is None or (data and 'query' not in data):
+ if data is None:
return make_json_response(
status=410,
success=0,
@@ -1346,12 +1346,9 @@ def start_query_download_tool(trans_id):
)
try:
- sql = data['query']
# This returns generator of records.
- status, gen = sync_conn.execute_on_server_as_csv(
- sql, records=2000
- )
+ status, gen = sync_conn.execute_on_server_as_csv(records=2000)
if not status:
return make_json_response(
@@ -1362,6 +1359,7 @@ def start_query_download_tool(trans_id):
r = Response(
gen(
+ trans_obj,
quote=blueprint.csv_quoting.get(),
quote_char=blueprint.csv_quote_char.get(),
field_separator=blueprint.csv_field_separator.get(),
diff --git a/web/pgadmin/tools/sqleditor/static/js/sqleditor.js b/web/pgadmin/tools/sqleditor/static/js/sqleditor.js
index c8d3cbcf3..522107cab 100644
--- a/web/pgadmin/tools/sqleditor/static/js/sqleditor.js
+++ b/web/pgadmin/tools/sqleditor/static/js/sqleditor.js
@@ -136,7 +136,7 @@ define('tools.querytool', [
'click #btn-flash': 'on_flash',
'click #btn-flash-menu': 'on_flash',
'click #btn-cancel-query': 'on_cancel_query',
- 'click #btn-download': 'on_download',
+ 'click #btn-save-results-to-file': 'on_download',
'click #btn-clear': 'on_clear',
'click #btn-auto-commit': 'on_auto_commit',
'click #btn-auto-rollback': 'on_auto_rollback',
@@ -1358,7 +1358,7 @@ define('tools.querytool', [
self.handler.fetching_rows = true;
$('#btn-flash').prop('disabled', true);
- $('#btn-download').prop('disabled', true);
+ $('#btn-save-results-to-file').prop('disabled', true);
if (fetch_all) {
self.handler.trigger(
@@ -1382,7 +1382,7 @@ define('tools.querytool', [
.done(function(res) {
self.handler.has_more_rows = res.data.has_more_rows;
$('#btn-flash').prop('disabled', false);
- $('#btn-download').prop('disabled', false);
+ $('#btn-save-results-to-file').prop('disabled', false);
self.handler.trigger('pgadmin-sqleditor:loading-icon:hide');
self.update_grid_data(res.data.result);
self.handler.fetching_rows = false;
@@ -1392,7 +1392,7 @@ define('tools.querytool', [
})
.fail(function(e) {
$('#btn-flash').prop('disabled', false);
- $('#btn-download').prop('disabled', false);
+ $('#btn-save-results-to-file').prop('disabled', false);
self.handler.trigger('pgadmin-sqleditor:loading-icon:hide');
self.handler.has_more_rows = false;
self.handler.fetching_rows = false;
@@ -2534,6 +2534,7 @@ define('tools.querytool', [
self.server_type = url_params.server_type;
self.url_params = url_params;
self.is_transaction_buttons_disabled = true;
+ self.is_save_results_to_file_disabled = true;
// We do not allow to call the start multiple times.
if (self.gridView)
@@ -2783,7 +2784,7 @@ define('tools.querytool', [
);
$('#btn-flash').prop('disabled', true);
- $('#btn-download').prop('disabled', true);
+ self.enable_disable_download_btn(true);
self.trigger(
'pgadmin-sqleditor:loading-icon:message',
@@ -3058,7 +3059,12 @@ define('tools.querytool', [
// Hide the loading icon
self_col.trigger('pgadmin-sqleditor:loading-icon:hide');
$('#btn-flash').prop('disabled', false);
- $('#btn-download').prop('disabled', false);
+ if (!_.isUndefined(data) && Array.isArray(data.result) && data.result.length > 0) {
+ self.enable_disable_download_btn(false);
+ }
+ else {
+ self.enable_disable_download_btn(true);
+ }
}.bind(self)
);
},
@@ -3239,7 +3245,6 @@ define('tools.querytool', [
if (status != 'Busy') {
$('#btn-flash').prop('disabled', false);
- $('#btn-download').prop('disabled', false);
self.trigger('pgadmin-sqleditor:loading-icon:hide');
if(!self.total_time) {
@@ -3301,6 +3306,12 @@ define('tools.querytool', [
return (self.get('can_edit'));
},
+ enable_disable_download_btn: function(is_save_results_to_file_disabled) {
+ var self = this;
+ self.is_save_results_to_file_disabled = is_save_results_to_file_disabled;
+ $('#btn-save-results-to-file').prop('disabled', is_save_results_to_file_disabled);
+ },
+
rows_to_delete: function(data) {
let self = this;
let tmp_keys = self.primary_keys;
@@ -4373,10 +4384,9 @@ define('tools.querytool', [
}
self.disable_tool_buttons(false);
is_query_running = false;
- if(!_.isUndefined(self.download_csv_obj)) {
- self.download_csv_obj.abort();
+ if(!_.isUndefined(self.download_results_obj)) {
+ self.download_results_obj.abort();
$('#btn-flash').prop('disabled', false);
- $('#btn-download').prop('disabled', false);
self.trigger(
'pgadmin-sqleditor:loading-icon:hide');
}
@@ -4394,25 +4404,25 @@ define('tools.querytool', [
},
// Trigger query result download to csv.
- trigger_csv_download: function(query, filename) {
+ trigger_csv_download: function(filename) {
var self = this,
url = url_for('sqleditor.query_tool_download', {
'trans_id': self.transId,
}),
- data = { query: query, filename: filename };
+ data = { filename: filename };
// Disable the Execute button
$('#btn-flash').prop('disabled', true);
- $('#btn-download').prop('disabled', true);
+ self.enable_disable_download_btn(true);
self.disable_tool_buttons(true);
self.set_sql_message('');
self.trigger(
'pgadmin-sqleditor:loading-icon:show',
- gettext('Downloading CSV...')
+ gettext('Downloading Results...')
);
// Get the CSV file
- self.download_csv_obj = $.ajax({
+ self.download_results_obj = $.ajax({
type: 'POST',
url: url,
data: data,
@@ -4443,19 +4453,19 @@ define('tools.querytool', [
}
document.body.removeChild(link);
- self.download_csv_obj = undefined;
+ self.download_results_obj = undefined;
}
// Enable the execute button
$('#btn-flash').prop('disabled', false);
- $('#btn-download').prop('disabled', false);
+ self.enable_disable_download_btn(false);
self.disable_tool_buttons(false);
self.trigger('pgadmin-sqleditor:loading-icon:hide');
}).fail(function(err) {
let msg = '';
// Enable the execute button
$('#btn-flash').prop('disabled', false);
- $('#btn-download').prop('disabled', false);
+ self.enable_disable_download_btn(false);
self.disable_tool_buttons(false);
self.trigger('pgadmin-sqleditor:loading-icon:hide');
diff --git a/web/pgadmin/tools/sqleditor/tests/test_download_csv_query_tool.py b/web/pgadmin/tools/sqleditor/tests/test_download_csv_query_tool.py
index 37ee5f47e..61d369267 100644
--- a/web/pgadmin/tools/sqleditor/tests/test_download_csv_query_tool.py
+++ b/web/pgadmin/tools/sqleditor/tests/test_download_csv_query_tool.py
@@ -96,7 +96,7 @@ class TestDownloadCSV(BaseTestGenerator):
]
def setUp(self):
- self._db_name = 'download_csv_' + str(random.randint(10000, 65535))
+ self._db_name = 'download_results_' + str(random.randint(10000, 65535))
self._sid = self.server_information['server_id']
server_con = server_utils.connect_server(self, self._sid)
@@ -105,6 +105,24 @@ class TestDownloadCSV(BaseTestGenerator):
self.server, self._db_name
)
+ # This method is responsible for initiating query hit at least once,
+ # so that download csv works
+ def initiate_sql_query_tool(self, trans_id, sql_query):
+
+ # This code is to ensure to create a async cursor so that downloading
+ # csv can work.
+ # Start query tool transaction
+ url = '/sqleditor/query_tool/start/{0}'.format(trans_id)
+ response = self.tester.post(url, data=json.dumps({"sql": sql_query}),
+ content_type='html/json')
+
+ self.assertEqual(response.status_code, 200)
+
+ # Query tool polling
+ url = '/sqleditor/poll/{0}'.format(trans_id)
+ response = self.tester.get(url)
+ return response
+
def runTest(self):
db_con = database_utils.connect_database(self,
@@ -121,6 +139,8 @@ class TestDownloadCSV(BaseTestGenerator):
response = self.tester.post(url)
self.assertEqual(response.status_code, 200)
+ res = self.initiate_sql_query_tool(self.trans_id, self.sql)
+
# If invalid tx test then make the Tx id invalid so that tests fails
if not self.is_valid_tx:
self.trans_id = self.trans_id + '007'
@@ -129,7 +149,11 @@ class TestDownloadCSV(BaseTestGenerator):
url = self.donwload_url.format(self.trans_id)
# Disable the console logging from Flask logger
self.app.logger.disabled = True
- if self.filename is None:
+ if not self.is_valid and self.is_valid_tx:
+ # When user enters wrong query, poll will throw 500, so expecting
+ # 500, as poll is never called for a wrong query.
+ self.assertEqual(res.status_code, 500)
+ elif self.filename is None:
if self.download_as_txt:
with patch('pgadmin.tools.sqleditor.blueprint.'
'csv_field_separator.get', return_value=';'), patch(
diff --git a/web/pgadmin/tools/sqleditor/utils/query_tool_preferences.py b/web/pgadmin/tools/sqleditor/utils/query_tool_preferences.py
index 4dc656ad3..e74a5f9c8 100644
--- a/web/pgadmin/tools/sqleditor/utils/query_tool_preferences.py
+++ b/web/pgadmin/tools/sqleditor/utils/query_tool_preferences.py
@@ -377,8 +377,8 @@ def register_query_tool_preferences(self):
self.preference.register(
'keyboard_shortcuts',
- 'download_csv',
- gettext('Download CSV'),
+ 'download_results',
+ gettext('Download Results'),
'keyboardshortcut',
{
'alt': False,
diff --git a/web/pgadmin/utils/driver/psycopg2/connection.py b/web/pgadmin/utils/driver/psycopg2/connection.py
index 4f40e652b..000902cc8 100644
--- a/web/pgadmin/utils/driver/psycopg2/connection.py
+++ b/web/pgadmin/utils/driver/psycopg2/connection.py
@@ -728,27 +728,35 @@ WHERE db.datname = current_database()""")
if self.async_ == 1:
self._wait(cur.connection)
- def execute_on_server_as_csv(self,
- query, params=None,
- formatted_exception_msg=False,
- records=2000):
+ def execute_on_server_as_csv(self, params=None,
+ formatted_exception_msg=False, records=2000):
"""
To fetch query result and generate CSV output
Args:
- query: SQL
params: Additional parameters
formatted_exception_msg: For exception
records: Number of initial records
Returns:
Generator response
"""
- status, cur = self.__cursor()
- self.row_count = 0
+ cur = self.__async_cursor
+ if not cur:
+ return False, self.CURSOR_NOT_FOUND
- if not status:
- return False, str(cur)
- query_id = random.randint(1, 9999999)
+ if self.conn.isexecuting():
+ return False, gettext(
+ "Asynchronous query execution/operation underway."
+ )
+
+ encoding = self.python_encoding
+ query = None
+ try:
+ query = str(cur.query, encoding) \
+ if cur and cur.query is not None else None
+ except Exception:
+ current_app.logger.warning('Error encoding query')
+ pass
current_app.logger.log(
25,
@@ -757,13 +765,14 @@ WHERE db.datname = current_database()""")
server_id=self.manager.sid,
conn_id=self.conn_id,
query=query,
- query_id=query_id
+ query_id=self.__async_query_id
)
)
try:
# Unregistering type casting for large size data types.
unregister_numeric_typecasters(self.conn)
- self.__internal_blocking_execute(cur, query, params)
+ if self.async_ == 1:
+ self._wait(cur.connection)
except psycopg2.Error as pe:
cur.close()
errmsg = self._formatted_exception_msg(pe, formatted_exception_msg)
@@ -775,7 +784,7 @@ WHERE db.datname = current_database()""")
server_id=self.manager.sid,
conn_id=self.conn_id,
errmsg=errmsg,
- query_id=query_id
+ query_id=self.__async_query_id
)
)
return False, errmsg
@@ -809,13 +818,12 @@ WHERE db.datname = current_database()""")
return results
- def gen(quote='strings', quote_char="'", field_separator=',',
- replace_nulls_with=None):
+ def gen(trans_obj, quote='strings', quote_char="'",
+ field_separator=',', replace_nulls_with=None):
+ cur.scroll(0, mode='absolute')
results = cur.fetchmany(records)
if not results:
- if not cur.closed:
- cur.close()
yield gettext('The query executed did not return any data.')
return
@@ -857,8 +865,6 @@ WHERE db.datname = current_database()""")
results = cur.fetchmany(records)
if not results:
- if not cur.closed:
- cur.close()
break
res_io = StringIO()
@@ -874,6 +880,17 @@ WHERE db.datname = current_database()""")
results = handle_null_values(results, replace_nulls_with)
csv_writer.writerows(results)
yield res_io.getvalue()
+
+ try:
+ # try to reset the cursor scroll back to where it was,
+ # bypass error, if cannot scroll back
+ rows_fetched_from = trans_obj.get_fetched_row_cnt()
+ cur.scroll(rows_fetched_from, mode='absolute')
+ except psycopg2.Error:
+ # bypassing the error as cursor tried to scroll on the
+ # specified position, but end of records found
+ pass
+
# Registering back type caster for large size data types to string
# which was unregistered at starting
register_string_typecasters(self.conn)
@@ -1224,6 +1241,7 @@ WHERE db.datname = current_database()""")
Args:
records: no of records to fetch. use -1 to fetchall.
formatted_exception_msg:
+ for_download: if True, will fetch all records and reset the cursor
Returns:
diff --git a/web/regression/javascript/sqleditor/call_render_after_poll_spec.js b/web/regression/javascript/sqleditor/call_render_after_poll_spec.js
index 18aeceac0..c7b2186a8 100644
--- a/web/regression/javascript/sqleditor/call_render_after_poll_spec.js
+++ b/web/regression/javascript/sqleditor/call_render_after_poll_spec.js
@@ -24,6 +24,7 @@ describe('#callRenderAfterPoll', () => {
disable_tool_buttons: jasmine.createSpy('SQLEditor.disable_tool_buttons'),
disable_transaction_buttons: jasmine.createSpy('SQLEditor.disable_transaction_buttons'),
reset_data_store: jasmine.createSpy('SQLEditor.reset_data_store'),
+ enable_disable_download_btn: jasmine.createSpy('SQLEditor.enable_disable_download_btn'),
query_start_time: new Date(),
};
alertify = jasmine.createSpyObj('alertify', ['success']);
@@ -115,6 +116,14 @@ describe('#callRenderAfterPoll', () => {
);
});
});
+
+ it('disables the save results button', () => {
+ callRenderAfterPoll(sqlEditorSpy, alertify, queryResult);
+
+ expect(sqlEditorSpy.enable_disable_download_btn).toHaveBeenCalledWith(true);
+
+ expect(sqlEditorSpy.trigger).toHaveBeenCalledWith('pgadmin-sqleditor:loading-icon:hide');
+ });
});
});
@@ -212,6 +221,14 @@ describe('#callRenderAfterPoll', () => {
);
});
});
+
+ it('disables the save results button', () => {
+ callRenderAfterPoll(sqlEditorSpy, alertify, queryResult);
+
+ expect(sqlEditorSpy.enable_disable_download_btn).toHaveBeenCalledWith(true);
+
+ expect(sqlEditorSpy.trigger).toHaveBeenCalledWith('pgadmin-sqleditor:loading-icon:hide');
+ });
});
});
});
diff --git a/web/regression/javascript/sqleditor/execute_query_spec.js b/web/regression/javascript/sqleditor/execute_query_spec.js
index dd37342c9..c561bb610 100644
--- a/web/regression/javascript/sqleditor/execute_query_spec.js
+++ b/web/regression/javascript/sqleditor/execute_query_spec.js
@@ -45,6 +45,7 @@ describe('ExecuteQuery', () => {
'handle_connection_lost',
'update_notifications',
'disable_transaction_buttons',
+ 'enable_disable_download_btn',
]);
sqlEditorMock.transId = 123;
sqlEditorMock.rows_affected = 1000;
diff --git a/web/regression/javascript/sqleditor/keyboard_shortcuts_spec.js b/web/regression/javascript/sqleditor/keyboard_shortcuts_spec.js
index 04365bf48..358a7116c 100644
--- a/web/regression/javascript/sqleditor/keyboard_shortcuts_spec.js
+++ b/web/regression/javascript/sqleditor/keyboard_shortcuts_spec.js
@@ -71,7 +71,7 @@ describe('the keyboard shortcuts', () => {
key_code: F7_KEY,
},
},
- download_csv: {
+ download_results: {
alt: false,
shift: false,
control: false,
diff --git a/web/regression/javascript/sqleditor/query_tool_actions_spec.js b/web/regression/javascript/sqleditor/query_tool_actions_spec.js
index 15021c463..66087bbd3 100644
--- a/web/regression/javascript/sqleditor/query_tool_actions_spec.js
+++ b/web/regression/javascript/sqleditor/query_tool_actions_spec.js
@@ -277,7 +277,7 @@ describe('queryToolActions', () => {
it('does nothing', () => {
queryToolActions.download(sqlEditorController);
- expect(sqlEditorController.trigger_csv_download).not.toHaveBeenCalled();
+ expect(sqlEditorController.trigger_csv_download).toHaveBeenCalled();
});
});
@@ -298,21 +298,21 @@ describe('queryToolActions', () => {
}));
});
- it('calls trigger_csv_download with the query and the filename with .csv extension', () => {
+ it('calls trigger_csv_download with filename having .csv extension', () => {
let filename = 'data-' + time + '.csv';
queryToolActions.download(sqlEditorController);
- expect(sqlEditorController.trigger_csv_download).toHaveBeenCalledWith(selectedQueryString, filename);
+ expect(sqlEditorController.trigger_csv_download).toHaveBeenCalledWith(filename);
});
- it('calls trigger_csv_download with the query and the filename with .txt extension', () => {
+ it('calls trigger_csv_download filename having .txt extension', () => {
sqlEditorController.preferences.csv_field_separator = ';';
let filename = 'data-' + time + '.txt';
queryToolActions.download(sqlEditorController);
- expect(sqlEditorController.trigger_csv_download).toHaveBeenCalledWith(selectedQueryString, filename);
+ expect(sqlEditorController.trigger_csv_download).toHaveBeenCalledWith(filename);
});
});
@@ -333,12 +333,12 @@ describe('queryToolActions', () => {
}));
});
- it('calls trigger_csv_download with the query and the filename', () => {
+ it('calls trigger_csv_download with filename', () => {
let filename = 'data-' + time + '.csv';
queryToolActions.download(sqlEditorController);
- expect(sqlEditorController.trigger_csv_download).toHaveBeenCalledWith(entireQueryString, filename);
+ expect(sqlEditorController.trigger_csv_download).toHaveBeenCalledWith(filename);
});
});
});
@@ -351,7 +351,7 @@ describe('queryToolActions', () => {
queryToolActions.download(sqlEditorController);
- expect(sqlEditorController.trigger_csv_download).toHaveBeenCalledWith(query, 'iAmATable' + '.csv');
+ expect(sqlEditorController.trigger_csv_download).toHaveBeenCalledWith('iAmATable' + '.csv');
});
});
diff --git a/web/regression/javascript/sqleditor/query_tool_preferences.js b/web/regression/javascript/sqleditor/query_tool_preferences.js
new file mode 100644
index 000000000..a36328bdd
--- /dev/null
+++ b/web/regression/javascript/sqleditor/query_tool_preferences.js
@@ -0,0 +1,241 @@
+/////////////////////////////////////////////////////////////
+//
+// pgAdmin 4 - PostgreSQL Tools
+//
+// Copyright (C) 2013 - 2020, The pgAdmin Development Team
+// This software is released under the PostgreSQL Licence
+//
+//////////////////////////////////////////////////////////////
+
+import {shortcut_key, shortcut_accesskey_title, shortcut_title}
+ from 'sources/keyboard_shortcuts';
+import * as SqlEditorUtils from 'sources/sqleditor_utils';
+import $ from 'jquery';
+import gettext from 'sources/gettext';
+
+function updateUIPreferences(sqlEditor) {
+ let $el = sqlEditor.$el,
+ preferences = sqlEditor.preferences;
+
+ if(sqlEditor.handler.slickgrid) {
+ sqlEditor.handler.slickgrid.CSVOptions = {
+ quoting: sqlEditor.preferences.results_grid_quoting,
+ quote_char: sqlEditor.preferences.results_grid_quote_char,
+ field_separator: sqlEditor.preferences.results_grid_field_separator,
+ };
+ }
+
+ /* Accessed using accesskey direct w/o ctrl,atl,shift */
+ $el.find('#btn-load-file')
+ .attr('title', shortcut_accesskey_title(gettext('Open File'),preferences.btn_open_file))
+ .attr('aria-label', shortcut_accesskey_title(gettext('Open File'),preferences.btn_open_file))
+ .attr('accesskey', shortcut_key(preferences.btn_open_file));
+
+ $el.find('#btn-save-file')
+ .attr('title', shortcut_accesskey_title(gettext('Save File'),preferences.btn_save_file))
+ .attr('aria-label', shortcut_accesskey_title(gettext('Save File'),preferences.btn_save_file))
+ .attr('accesskey', shortcut_key(preferences.btn_save_file));
+
+ $el.find('#btn-find-menu-dropdown')
+ .attr('title', shortcut_accesskey_title(gettext('Find'),preferences.btn_find_options))
+ .attr('aria-label',shortcut_accesskey_title(gettext('Find'),preferences.btn_find_options))
+ .attr('accesskey', shortcut_key(preferences.btn_find_options));
+
+ $el.find('#btn-copy-row')
+ .attr('title', shortcut_accesskey_title(gettext('Copy'),preferences.btn_copy_row))
+ .attr('aria-label', shortcut_accesskey_title(gettext('Copy'),preferences.btn_copy_row))
+ .attr('accesskey', shortcut_key(preferences.btn_copy_row));
+
+ $el.find('#btn-paste-row')
+ .attr('title', shortcut_accesskey_title(gettext('Paste'),preferences.btn_paste_row))
+ .attr('aria-label', shortcut_accesskey_title(gettext('Paste'),preferences.btn_paste_row))
+ .attr('accesskey', shortcut_key(preferences.btn_paste_row));
+
+ $el.find('#btn-delete-row')
+ .attr('title', shortcut_accesskey_title(gettext('Delete'),preferences.btn_delete_row))
+ .attr('aria-label', shortcut_accesskey_title(gettext('Delete'),preferences.btn_delete_row))
+ .attr('accesskey', shortcut_key(preferences.btn_delete_row));
+
+ $el.find('#btn-filter')
+ .attr('title', shortcut_accesskey_title(gettext('Filter'),preferences.btn_filter_dialog))
+ .attr('aria-label', shortcut_accesskey_title(gettext('Filter'),preferences.btn_filter_dialog))
+ .attr('accesskey', shortcut_key(preferences.btn_filter_dialog));
+
+ $el.find('#btn-filter-dropdown')
+ .attr('title', shortcut_accesskey_title(gettext('Filter options'),preferences.btn_filter_options))
+ .attr('aria-label', shortcut_accesskey_title(gettext('Filter options'),preferences.btn_filter_options))
+ .attr('accesskey', shortcut_key(preferences.btn_filter_options));
+
+ $el.find('#btn-rows-limit')
+ .attr('title', shortcut_accesskey_title(gettext('Rows limit'),preferences.btn_rows_limit))
+ .attr('aria-label', shortcut_accesskey_title(gettext('Rows limit'),preferences.btn_rows_limit))
+ .attr('accesskey', shortcut_key(preferences.btn_rows_limit));
+
+ $el.find('#btn-query-dropdown')
+ .attr('title', shortcut_accesskey_title(gettext('Execute options'),preferences.btn_execute_options))
+ .attr('aria-label', shortcut_accesskey_title(gettext('Execute options'),preferences.btn_execute_options))
+ .attr('accesskey', shortcut_key(preferences.btn_execute_options));
+
+ $el.find('#btn-cancel-query')
+ .attr('title', shortcut_accesskey_title(gettext('Cancel query'),preferences.btn_cancel_query))
+ .attr('aria-label', shortcut_accesskey_title(gettext('Cancel query'),preferences.btn_cancel_query))
+ .attr('accesskey', shortcut_key(preferences.btn_cancel_query));
+
+ $el.find('#btn-clear-dropdown')
+ .attr('title', shortcut_accesskey_title(gettext('Clear'),preferences.btn_clear_options))
+ .attr('aria-label', shortcut_accesskey_title(gettext('Clear'),preferences.btn_clear_options))
+ .attr('accesskey', shortcut_key(preferences.btn_clear_options));
+
+ $el.find('#btn-conn-status')
+ .attr('accesskey', shortcut_key(preferences.btn_conn_status))
+ .find('i')
+ .attr('title',
+ shortcut_accesskey_title(gettext('Connection status (click for details)'),
+ preferences.btn_conn_status));
+
+ /* Accessed using ctrl,atl,shift and key */
+ $el.find('#btn-flash')
+ .attr('title',
+ shortcut_title(gettext('Execute/Refresh'),preferences.execute_query))
+ .attr('aria-label',
+ shortcut_title(gettext('Execute/Refresh'),preferences.execute_query));
+
+ $el.find('#btn-explain')
+ .attr('title',
+ shortcut_title(gettext('Explain'),preferences.explain_query))
+ .attr('aria-label',
+ shortcut_title(gettext('Explain'),preferences.explain_query));
+
+ $el.find('#btn-explain-analyze')
+ .attr('title',
+ shortcut_title(gettext('Explain Analyze'),preferences.explain_analyze_query))
+ .attr('aria-label',
+ shortcut_title(gettext('Explain Analyze'),preferences.explain_analyze_query));
+
+ $el.find('#btn-save-results-to-file')
+ .attr('title',
+ shortcut_title(gettext('Save results to file'),preferences.download_results))
+ .attr('aria-label',
+ shortcut_title(gettext('Save results to file'),preferences.download_results));
+
+ $el.find('#btn-save-data')
+ .attr('title',
+ shortcut_title(gettext('Save Data Changes'),preferences.save_data))
+ .attr('aria-label',
+ shortcut_title(gettext('Save Data Changes'),preferences.save_data));
+
+ $el.find('#btn-commit')
+ .attr('title',
+ shortcut_title(gettext('Commit'),preferences.commit_transaction))
+ .attr('aria-label',
+ shortcut_title(gettext('Commit'),preferences.commit_transaction));
+
+ $el.find('#btn-rollback')
+ .attr('title',
+ shortcut_title(gettext('Rollback'),preferences.rollback_transaction))
+ .attr('aria-label',
+ shortcut_title(gettext('Rollback'),preferences.rollback_transaction));
+
+ $el.find('#btn-show-query-tool')
+ .attr('title',
+ shortcut_title(gettext('Query tool'),preferences.show_query_tool))
+ .attr('aria-label',
+ shortcut_title(gettext('Query tool'),preferences.show_query_tool));
+
+ /* Set explain options on query editor */
+ if (preferences.explain_verbose){
+ $el.find('.explain-verbose').removeClass('visibility-hidden');
+ }
+ else {
+ $el.find('.explain-verbose').addClass('visibility-hidden');
+ }
+
+ if (preferences.explain_costs){
+ $el.find('.explain-costs').removeClass('visibility-hidden');
+ }
+ else {
+ $el.find('.explain-costs').addClass('visibility-hidden');
+ }
+
+ if (preferences.explain_buffers){
+ $el.find('.explain-buffers').removeClass('visibility-hidden');
+ }
+ else {
+ $el.find('.explain-buffers').addClass('visibility-hidden');
+ }
+
+ if (preferences.explain_timing) {
+ $el.find('.explain-timing').removeClass('visibility-hidden');
+ }
+ else {
+ $el.find('.explain-timing').addClass('visibility-hidden');
+ }
+
+ if (preferences.explain_summary) {
+ $el.find('.explain-summary').removeClass('visibility-hidden');
+ }
+ else {
+ $el.find('.explain-summary').addClass('visibility-hidden');
+ }
+
+ if (preferences.explain_settings) {
+ $el.find('.explain-settings').removeClass('visibility-hidden');
+ }
+ else {
+ $el.find('.explain-settings').addClass('visibility-hidden');
+ }
+
+ /* Connection status check */
+ /* remove the status checker if present */
+ if(sqlEditor.connIntervalId != null) {
+ clearInterval(sqlEditor.connIntervalId);
+ sqlEditor.connIntervalId = null;
+ }
+ if (preferences.connection_status) {
+ let $conn_status = $el.find('#btn-conn-status'),
+ $status_el = $conn_status.find('i');
+ $conn_status.popover();
+
+ $conn_status.removeClass('connection-status-hide');
+
+ // To set initial connection
+ SqlEditorUtils.fetchConnectionStatus(sqlEditor.handler, $conn_status, $status_el);
+
+ // Calling it again in specified interval
+ sqlEditor.connIntervalId = setInterval(
+ SqlEditorUtils.fetchConnectionStatus.bind(null, sqlEditor.handler, $conn_status, $status_el),
+ preferences.connection_status_fetch_time * 1000
+ );
+ }
+ else {
+ $el.find('#btn-conn-status').addClass('connection-status-hide');
+ }
+
+ /* Code Mirror Preferences */
+ let sql_font_size = SqlEditorUtils.calcFontSize(preferences.sql_font_size);
+ $(sqlEditor.query_tool_obj.getWrapperElement()).css('font-size', sql_font_size);
+
+ if(preferences.plain_editor_mode) {
+ sqlEditor.query_tool_obj.setOption('mode', 'text/plain');
+ /* Although not required, setting explicitly as codemirror will remove code folding only on next edit */
+ sqlEditor.query_tool_obj.setOption('foldGutter', false);
+ } else {
+ sqlEditor.query_tool_obj.setOption('mode', sqlEditor.handler.server_type === 'gpdb' ? 'text/x-gpsql' : 'text/x-pgsql');
+ sqlEditor.query_tool_obj.setOption('foldGutter', preferences.code_folding);
+ }
+ sqlEditor.query_tool_obj.setOption('foldGutter', preferences.code_folding);
+ sqlEditor.query_tool_obj.setOption('indentWithTabs', !preferences.use_spaces);
+ sqlEditor.query_tool_obj.setOption('indentUnit', preferences.tab_size);
+ sqlEditor.query_tool_obj.setOption('tabSize', preferences.tab_size);
+ sqlEditor.query_tool_obj.setOption('lineWrapping', preferences.wrap_code);
+ sqlEditor.query_tool_obj.setOption('autoCloseBrackets', preferences.insert_pair_brackets);
+ sqlEditor.query_tool_obj.setOption('matchBrackets', preferences.brace_matching);
+ sqlEditor.query_tool_obj.refresh();
+
+ /* Render history to reflect Font size change */
+ sqlEditor.historyComponent.setEditorPref({
+ 'sql_font_size' : sql_font_size,
+ });
+}
+
+export {updateUIPreferences};
view thread (7+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected]
Subject: Re: [pgAdmin4] RM5965 Couldn't download file of Marcos query results
In-Reply-To: <CAKtn9dMhuna1=+L8EgY-GzEKyYWw4Cut0-QBvxhukiP0HQYm-A@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox