public inbox for [email protected]help / color / mirror / Atom feed
[pgAdmin][RM6520]: pgAdmin4 v 5.3 text export error 3+ messages / 2 participants [nested] [flat]
* [pgAdmin][RM6520]: pgAdmin4 v 5.3 text export error @ 2021-06-11 13:47 Pradip Parkale <[email protected]> 0 siblings, 1 reply; 3+ messages in thread From: Pradip Parkale @ 2021-06-11 13:47 UTC (permalink / raw) To: pgadmin-hackers Hi Hackers, Please find the attached patch for #6520.Added check to typecast the numeric data while downloading the data in CSV format. -- Thanks & Regards, Pradip Parkale Software Engineer | EnterpriseDB Corporation Attachments: [application/octet-stream] RM6520.patch (3.7K, 3-RM6520.patch) download | inline diff: diff --git a/web/pgadmin/tools/sqleditor/__init__.py b/web/pgadmin/tools/sqleditor/__init__.py index 780c0243f..c7ec52277 100644 --- a/web/pgadmin/tools/sqleditor/__init__.py +++ b/web/pgadmin/tools/sqleditor/__init__.py @@ -1357,7 +1357,8 @@ def start_query_download_tool(trans_id): try: # This returns generator of records. - status, gen = sync_conn.execute_on_server_as_csv(records=2000) + status, gen, conn_obj = \ + sync_conn.execute_on_server_as_csv(records=2000) if not status: return make_json_response( @@ -1367,13 +1368,13 @@ def start_query_download_tool(trans_id): ) r = Response( - gen( + gen(conn_obj, trans_obj, quote=blueprint.csv_quoting.get(), quote_char=blueprint.csv_quote_char.get(), field_separator=blueprint.csv_field_separator.get(), replace_nulls_with=blueprint.replace_nulls_with.get() - ), + ), mimetype='text/csv' if blueprint.csv_field_separator.get() == ',' else 'text/plain' diff --git a/web/pgadmin/utils/driver/psycopg2/connection.py b/web/pgadmin/utils/driver/psycopg2/connection.py index 8ff0963f4..3bddb9fa5 100644 --- a/web/pgadmin/utils/driver/psycopg2/connection.py +++ b/web/pgadmin/utils/driver/psycopg2/connection.py @@ -879,7 +879,7 @@ WHERE db.datname = current_database()""") return results - def gen(trans_obj, quote='strings', quote_char="'", + def gen(conn_obj, trans_obj, quote='strings', quote_char="'", field_separator=',', replace_nulls_with=None): cur.scroll(0, mode='absolute') @@ -889,7 +889,7 @@ WHERE db.datname = current_database()""") return # Type cast the numeric values - results = numeric_typecasters(results) + results = numeric_typecasters(results, conn_obj) header = [] json_columns = [] @@ -958,7 +958,7 @@ WHERE db.datname = current_database()""") # Registering back type caster for large size data types to string # which was unregistered at starting register_string_typecasters(self.conn) - return True, gen + return True, gen, self def execute_scalar(self, query, params=None, formatted_exception_msg=False): diff --git a/web/pgadmin/utils/driver/psycopg2/typecast.py b/web/pgadmin/utils/driver/psycopg2/typecast.py index c6d043e7b..0250e9316 100644 --- a/web/pgadmin/utils/driver/psycopg2/typecast.py +++ b/web/pgadmin/utils/driver/psycopg2/typecast.py @@ -198,25 +198,21 @@ def register_string_typecasters(connection): psycopg2.extensions.register_type(unicode_array_type, connection) -def is_numeric(val): - """Check if value is numeric or not""" - try: - if '.' in val: - float(val) - else: - int(val) - except ValueError: - return False - return True - - -def numeric_typecasters(results): +def numeric_typecasters(results, conn_obj): # This function is to convert pg types to numeic type caster + data = [] + for obj_type in conn_obj.column_info: + if obj_type['type_code'] in TO_STRING_NUMERIC_DATATYPES: + data.append(obj_type['name']) + for result in results: for key, value in result.items(): - if isinstance(result[key], str) and is_numeric(result[key]): + if isinstance(result[key], + str) and key in data and not value.isdigit(): result[key] = float(result[key]) + elif isinstance(result[key], str) and key in data: + result[key] = int(result[key]) return results ^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: [pgAdmin][RM6520]: pgAdmin4 v 5.3 text export error @ 2021-06-11 14:07 Aditya Toshniwal <[email protected]> parent: Pradip Parkale <[email protected]> 0 siblings, 1 reply; 3+ messages in thread From: Aditya Toshniwal @ 2021-06-11 14:07 UTC (permalink / raw) To: Pradip Parkale <[email protected]>; +Cc: pgadmin-hackers Hi Pradip, Just did a go through to your patch. The variable "data" which you've formed, will be computed for every row even though columns won't change. You should move it up and pass it to gen directly. This will save repeated computations. Secondly, it should not be named as data. It can be like - numeric_cols or numeric_keys. On Fri, Jun 11, 2021 at 7:17 PM Pradip Parkale < [email protected]> wrote: > Hi Hackers, > > Please find the attached patch for #6520.Added check to typecast the > numeric data while downloading the data in CSV format. > > -- > Thanks & Regards, > Pradip Parkale > Software Engineer | EnterpriseDB Corporation > -- Thanks, Aditya Toshniwal pgAdmin hacker | Sr. Software Engineer | *edbpostgres.com* <http://edbpostgres.com; "Don't Complain about Heat, Plant a TREE" ^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: [pgAdmin][RM6520]: pgAdmin4 v 5.3 text export error @ 2021-06-14 10:07 Pradip Parkale <[email protected]> parent: Aditya Toshniwal <[email protected]> 0 siblings, 0 replies; 3+ messages in thread From: Pradip Parkale @ 2021-06-14 10:07 UTC (permalink / raw) To: Aditya Toshniwal <[email protected]>; Akshay Joshi <[email protected]>; +Cc: pgadmin-hackers Hi Aditya and Akshay, Please find the updated patch. I have changed the variable name "data" to 'numeric_cols'. On Fri, Jun 11, 2021 at 7:38 PM Aditya Toshniwal < [email protected]> wrote: > Hi Pradip, > > Just did a go through to your patch. > The variable "data" which you've formed, will be computed for every row > even though columns won't change. You should move it up and pass it to gen > directly. This will save repeated computations. > Secondly, it should not be named as data. It can be like - numeric_cols or > numeric_keys. > > On Fri, Jun 11, 2021 at 7:17 PM Pradip Parkale < > [email protected]> wrote: > >> Hi Hackers, >> >> Please find the attached patch for #6520.Added check to typecast the >> numeric data while downloading the data in CSV format. >> >> -- >> Thanks & Regards, >> Pradip Parkale >> Software Engineer | EnterpriseDB Corporation >> > > > -- > Thanks, > Aditya Toshniwal > pgAdmin hacker | Sr. Software Engineer | *edbpostgres.com* > <http://edbpostgres.com; > "Don't Complain about Heat, Plant a TREE" > -- Thanks & Regards, Pradip Parkale Software Engineer | EnterpriseDB Corporation Attachments: [application/octet-stream] RM6520_v2.patch (1.2K, 3-RM6520_v2.patch) download | inline diff: diff --git a/web/pgadmin/utils/driver/psycopg2/typecast.py b/web/pgadmin/utils/driver/psycopg2/typecast.py index 0250e9316..34b2779e3 100644 --- a/web/pgadmin/utils/driver/psycopg2/typecast.py +++ b/web/pgadmin/utils/driver/psycopg2/typecast.py @@ -201,17 +201,17 @@ def register_string_typecasters(connection): def numeric_typecasters(results, conn_obj): # This function is to convert pg types to numeic type caster - data = [] + numeric_cols = [] for obj_type in conn_obj.column_info: if obj_type['type_code'] in TO_STRING_NUMERIC_DATATYPES: - data.append(obj_type['name']) + numeric_cols.append(obj_type['name']) for result in results: for key, value in result.items(): if isinstance(result[key], - str) and key in data and not value.isdigit(): + str) and key in numeric_cols and not value.isdigit(): result[key] = float(result[key]) - elif isinstance(result[key], str) and key in data: + elif isinstance(result[key], str) and key in numeric_cols: result[key] = int(result[key]) return results ^ permalink raw reply [nested|flat] 3+ messages in thread
end of thread, other threads:[~2021-06-14 10:07 UTC | newest] Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2021-06-11 13:47 [pgAdmin][RM6520]: pgAdmin4 v 5.3 text export error Pradip Parkale <[email protected]> 2021-06-11 14:07 ` Aditya Toshniwal <[email protected]> 2021-06-14 10:07 ` Pradip Parkale <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox