public inbox for [email protected]help / color / mirror / Atom feed
[pgAdmin][RM6341] CSV download quotes numeric columns 4+ messages / 2 participants [nested] [flat]
* [pgAdmin][RM6341] CSV download quotes numeric columns @ 2021-04-20 05:24 Pradip Parkale <[email protected]> 0 siblings, 1 reply; 4+ messages in thread From: Pradip Parkale @ 2021-04-20 05:24 UTC (permalink / raw) To: pgadmin-hackers Hi Hackers, Please find the attached patch for #6341 CSV download quotes numeric columns. Initially, the double-precision value was getting typecast to a string. -- Thanks & Regards, Pradip Parkale Software Engineer | EnterpriseDB Corporation Attachments: [application/octet-stream] RM6341.patch (2.5K, 3-RM6341.patch) download | inline diff: diff --git a/web/pgadmin/utils/driver/psycopg2/connection.py b/web/pgadmin/utils/driver/psycopg2/connection.py index de9547322..a9d55c637 100644 --- a/web/pgadmin/utils/driver/psycopg2/connection.py +++ b/web/pgadmin/utils/driver/psycopg2/connection.py @@ -30,7 +30,7 @@ from pgadmin.utils.exception import ConnectionLost, CryptKeyMissing from pgadmin.utils import get_complete_file_path from ..abstract import BaseConnection from .cursor import DictCursor -from .typecast import register_global_typecasters, \ +from .typecast import register_float_typecasters, register_global_typecasters,\ register_string_typecasters, register_binary_typecasters, \ unregister_numeric_typecasters, \ register_array_to_string_typecasters, ALL_JSON_TYPES @@ -462,6 +462,7 @@ class Connection(BaseConnection): self._set_auto_commit(kwargs) register_string_typecasters(self.conn) + register_float_typecasters(self.conn) if self.array_to_string: register_array_to_string_typecasters(self.conn) @@ -911,6 +912,8 @@ WHERE db.datname = current_database()""") # Registering back type caster for large size data types to string # which was unregistered at starting + if any(type['type_code'] == 701 for type in self.column_info): + register_float_typecasters(self.conn) register_string_typecasters(self.conn) return True, gen diff --git a/web/pgadmin/utils/driver/psycopg2/typecast.py b/web/pgadmin/utils/driver/psycopg2/typecast.py index 76bb1b81f..a93a53da7 100644 --- a/web/pgadmin/utils/driver/psycopg2/typecast.py +++ b/web/pgadmin/utils/driver/psycopg2/typecast.py @@ -13,6 +13,7 @@ data types. """ from psycopg2 import STRING as _STRING +from psycopg2.extensions import FLOAT as _FLOAT from psycopg2.extensions import DECIMAL as _DECIMAL, encodings import psycopg2 from psycopg2.extras import Json as psycopg2_json @@ -203,6 +204,14 @@ def register_string_typecasters(connection): psycopg2.extensions.register_type(unicode_array_type, connection) +def register_float_typecasters(connection): + # This function is to convert pg types into decimal type + string_type_to_float = \ + psycopg2.extensions.new_type(TO_STRING_NUMERIC_DATATYPES, + 'TYPECAST_TO_DECIMAL', _DECIMAL) + psycopg2.extensions.register_type(string_type_to_float, connection) + + def register_binary_typecasters(connection): psycopg2.extensions.register_type( psycopg2.extensions.new_type( ^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: [pgAdmin][RM6341] CSV download quotes numeric columns @ 2021-04-22 12:12 Akshay Joshi <[email protected]> parent: Pradip Parkale <[email protected]> 0 siblings, 1 reply; 4+ messages in thread From: Akshay Joshi @ 2021-04-22 12:12 UTC (permalink / raw) To: Pradip Parkale <[email protected]>; +Cc: pgadmin-hackers Thanks, patch applied. On Tue, Apr 20, 2021 at 10:54 AM Pradip Parkale < [email protected]> wrote: > Hi Hackers, > > Please find the attached patch for #6341 CSV download quotes numeric > columns. > Initially, the double-precision value was getting typecast to a string. > > > -- > Thanks & Regards, > Pradip Parkale > Software Engineer | EnterpriseDB Corporation > -- *Thanks & Regards* *Akshay Joshi* *pgAdmin Hacker | Principal Software Architect* *EDB Postgres <http://edbpostgres.com>* *Mobile: +91 976-788-8246* ^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: [pgAdmin][RM6341] CSV download quotes numeric columns @ 2021-04-26 11:31 Pradip Parkale <[email protected]> parent: Akshay Joshi <[email protected]> 0 siblings, 1 reply; 4+ messages in thread From: Pradip Parkale @ 2021-04-26 11:31 UTC (permalink / raw) To: Akshay Joshi <[email protected]>; +Cc: pgadmin-hackers Hi Akshay, Please find the updated patch for #6251. I have fixed the issue caused due to my initial patch. On Thu, Apr 22, 2021 at 5:42 PM Akshay Joshi <[email protected]> wrote: > Thanks, patch applied. > > On Tue, Apr 20, 2021 at 10:54 AM Pradip Parkale < > [email protected]> wrote: > >> Hi Hackers, >> >> Please find the attached patch for #6341 CSV download quotes numeric >> columns. >> Initially, the double-precision value was getting typecast to a string. >> >> >> -- >> Thanks & Regards, >> Pradip Parkale >> Software Engineer | EnterpriseDB Corporation >> > > > -- > *Thanks & Regards* > *Akshay Joshi* > *pgAdmin Hacker | Principal Software Architect* > *EDB Postgres <http://edbpostgres.com>* > > *Mobile: +91 976-788-8246* > -- Thanks & Regards, Pradip Parkale Software Engineer | EnterpriseDB Corporation Attachments: [application/octet-stream] RM6341_v2.patch (4.5K, 3-RM6341_v2.patch) download | inline diff: diff --git a/web/pgadmin/utils/driver/psycopg2/connection.py b/web/pgadmin/utils/driver/psycopg2/connection.py index a9d55c637..78521072f 100644 --- a/web/pgadmin/utils/driver/psycopg2/connection.py +++ b/web/pgadmin/utils/driver/psycopg2/connection.py @@ -12,7 +12,6 @@ Implementation of Connection. It is a wrapper around the actual psycopg2 driver, and connection object. """ - import random import select import datetime @@ -30,7 +29,7 @@ from pgadmin.utils.exception import ConnectionLost, CryptKeyMissing from pgadmin.utils import get_complete_file_path from ..abstract import BaseConnection from .cursor import DictCursor -from .typecast import register_float_typecasters, register_global_typecasters,\ +from .typecast import numeric_typecasters, register_global_typecasters,\ register_string_typecasters, register_binary_typecasters, \ unregister_numeric_typecasters, \ register_array_to_string_typecasters, ALL_JSON_TYPES @@ -462,7 +461,6 @@ class Connection(BaseConnection): self._set_auto_commit(kwargs) register_string_typecasters(self.conn) - register_float_typecasters(self.conn) if self.array_to_string: register_array_to_string_typecasters(self.conn) @@ -845,6 +843,8 @@ WHERE db.datname = current_database()""") if not results: yield gettext('The query executed did not return any data.') return + # Type cast the numeric values + results = numeric_typecasters(results) header = [] json_columns = [] @@ -912,8 +912,6 @@ WHERE db.datname = current_database()""") # Registering back type caster for large size data types to string # which was unregistered at starting - if any(type['type_code'] == 701 for type in self.column_info): - register_float_typecasters(self.conn) register_string_typecasters(self.conn) return True, gen diff --git a/web/pgadmin/utils/driver/psycopg2/typecast.py b/web/pgadmin/utils/driver/psycopg2/typecast.py index a93a53da7..c6d043e7b 100644 --- a/web/pgadmin/utils/driver/psycopg2/typecast.py +++ b/web/pgadmin/utils/driver/psycopg2/typecast.py @@ -22,7 +22,6 @@ from .encoding import configure_driver_encodings, get_encoding configure_driver_encodings(encodings) - # OIDs of data types which need to typecast as string to avoid JavaScript # compatibility issues. # e.g JavaScript does not support 64 bit integers. It has 64-bit double @@ -67,7 +66,6 @@ TO_ARRAY_OF_STRING_DATATYPES = ( # OID of record array data type RECORD_ARRAY = (2287,) - # OIDs of builtin array datatypes supported by psycopg2 # OID reference psycopg2/psycopg/typecast_builtins.c # @@ -103,22 +101,18 @@ PSYCOPG_SUPPORTED_JSON_ARRAY_TYPES = (199, 3807) ALL_JSON_TYPES = PSYCOPG_SUPPORTED_JSON_TYPES +\ PSYCOPG_SUPPORTED_JSON_ARRAY_TYPES - # INET[], CIDR[] # OID reference psycopg2/lib/_ipaddress.py PSYCOPG_SUPPORTED_IPADDRESS_ARRAY_TYPES = (1041, 651) - # uuid[] # OID reference psycopg2/lib/extras.py PSYCOPG_SUPPORTED_IPADDRESS_ARRAY_TYPES = (2951,) - # int4range, int8range, numrange, daterange tsrange, tstzrange[] # OID reference psycopg2/lib/_range.py PSYCOPG_SUPPORTED_RANGE_TYPES = (3904, 3926, 3906, 3912, 3908, 3910) - # int4range[], int8range[], numrange[], daterange[] tsrange[], tstzrange[] # OID reference psycopg2/lib/_range.py PSYCOPG_SUPPORTED_RANGE_ARRAY_TYPES = (3905, 3927, 3907, 3913, 3909, 3911) @@ -204,12 +198,26 @@ def register_string_typecasters(connection): psycopg2.extensions.register_type(unicode_array_type, connection) -def register_float_typecasters(connection): - # This function is to convert pg types into decimal type - string_type_to_float = \ - psycopg2.extensions.new_type(TO_STRING_NUMERIC_DATATYPES, - 'TYPECAST_TO_DECIMAL', _DECIMAL) - psycopg2.extensions.register_type(string_type_to_float, connection) +def is_numeric(val): + """Check if value is numeric or not""" + try: + if '.' in val: + float(val) + else: + int(val) + except ValueError: + return False + return True + + +def numeric_typecasters(results): + # This function is to convert pg types to numeic type caster + + for result in results: + for key, value in result.items(): + if isinstance(result[key], str) and is_numeric(result[key]): + result[key] = float(result[key]) + return results def register_binary_typecasters(connection): ^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: [pgAdmin][RM6341] CSV download quotes numeric columns @ 2021-04-26 11:48 Akshay Joshi <[email protected]> parent: Pradip Parkale <[email protected]> 0 siblings, 0 replies; 4+ messages in thread From: Akshay Joshi @ 2021-04-26 11:48 UTC (permalink / raw) To: Pradip Parkale <[email protected]>; +Cc: pgadmin-hackers Thanks, patch applied. On Mon, Apr 26, 2021 at 5:01 PM Pradip Parkale < [email protected]> wrote: > Hi Akshay, > Please find the updated patch for #6251. I have fixed the issue caused due > to my initial patch. > > On Thu, Apr 22, 2021 at 5:42 PM Akshay Joshi < > [email protected]> wrote: > >> Thanks, patch applied. >> >> On Tue, Apr 20, 2021 at 10:54 AM Pradip Parkale < >> [email protected]> wrote: >> >>> Hi Hackers, >>> >>> Please find the attached patch for #6341 CSV download quotes numeric >>> columns. >>> Initially, the double-precision value was getting typecast to a string. >>> >>> >>> -- >>> Thanks & Regards, >>> Pradip Parkale >>> Software Engineer | EnterpriseDB Corporation >>> >> >> >> -- >> *Thanks & Regards* >> *Akshay Joshi* >> *pgAdmin Hacker | Principal Software Architect* >> *EDB Postgres <http://edbpostgres.com>* >> >> *Mobile: +91 976-788-8246* >> > > > -- > Thanks & Regards, > Pradip Parkale > Software Engineer | EnterpriseDB Corporation > -- *Thanks & Regards* *Akshay Joshi* *pgAdmin Hacker | Principal Software Architect* *EDB Postgres <http://edbpostgres.com>* *Mobile: +91 976-788-8246* ^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2021-04-26 11:48 UTC | newest] Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2021-04-20 05:24 [pgAdmin][RM6341] CSV download quotes numeric columns Pradip Parkale <[email protected]> 2021-04-22 12:12 ` Akshay Joshi <[email protected]> 2021-04-26 11:31 ` Pradip Parkale <[email protected]> 2021-04-26 11:48 ` Akshay Joshi <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox