public inbox for [email protected]
help / color / mirror / Atom feed[pgAdmin][RM6341] CSV download quotes numeric columns
4+ messages / 2 participants
[nested] [flat]
* [pgAdmin][RM6341] CSV download quotes numeric columns
@ 2021-04-20 05:24 Pradip Parkale <[email protected]>
2021-04-22 12:12 ` Re: [pgAdmin][RM6341] CSV download quotes numeric columns Akshay Joshi <[email protected]>
0 siblings, 1 reply; 4+ messages in thread
From: Pradip Parkale @ 2021-04-20 05:24 UTC (permalink / raw)
To: pgadmin-hackers
Hi Hackers,
Please find the attached patch for #6341 CSV download quotes numeric
columns.
Initially, the double-precision value was getting typecast to a string.
--
Thanks & Regards,
Pradip Parkale
Software Engineer | EnterpriseDB Corporation
Attachments:
[application/octet-stream] RM6341.patch (2.5K, 3-RM6341.patch)
download | inline diff:
diff --git a/web/pgadmin/utils/driver/psycopg2/connection.py b/web/pgadmin/utils/driver/psycopg2/connection.py
index de9547322..a9d55c637 100644
--- a/web/pgadmin/utils/driver/psycopg2/connection.py
+++ b/web/pgadmin/utils/driver/psycopg2/connection.py
@@ -30,7 +30,7 @@ from pgadmin.utils.exception import ConnectionLost, CryptKeyMissing
from pgadmin.utils import get_complete_file_path
from ..abstract import BaseConnection
from .cursor import DictCursor
-from .typecast import register_global_typecasters, \
+from .typecast import register_float_typecasters, register_global_typecasters,\
register_string_typecasters, register_binary_typecasters, \
unregister_numeric_typecasters, \
register_array_to_string_typecasters, ALL_JSON_TYPES
@@ -462,6 +462,7 @@ class Connection(BaseConnection):
self._set_auto_commit(kwargs)
register_string_typecasters(self.conn)
+ register_float_typecasters(self.conn)
if self.array_to_string:
register_array_to_string_typecasters(self.conn)
@@ -911,6 +912,8 @@ WHERE db.datname = current_database()""")
# Registering back type caster for large size data types to string
# which was unregistered at starting
+ if any(type['type_code'] == 701 for type in self.column_info):
+ register_float_typecasters(self.conn)
register_string_typecasters(self.conn)
return True, gen
diff --git a/web/pgadmin/utils/driver/psycopg2/typecast.py b/web/pgadmin/utils/driver/psycopg2/typecast.py
index 76bb1b81f..a93a53da7 100644
--- a/web/pgadmin/utils/driver/psycopg2/typecast.py
+++ b/web/pgadmin/utils/driver/psycopg2/typecast.py
@@ -13,6 +13,7 @@ data types.
"""
from psycopg2 import STRING as _STRING
+from psycopg2.extensions import FLOAT as _FLOAT
from psycopg2.extensions import DECIMAL as _DECIMAL, encodings
import psycopg2
from psycopg2.extras import Json as psycopg2_json
@@ -203,6 +204,14 @@ def register_string_typecasters(connection):
psycopg2.extensions.register_type(unicode_array_type, connection)
+def register_float_typecasters(connection):
+ # This function is to convert pg types into decimal type
+ string_type_to_float = \
+ psycopg2.extensions.new_type(TO_STRING_NUMERIC_DATATYPES,
+ 'TYPECAST_TO_DECIMAL', _DECIMAL)
+ psycopg2.extensions.register_type(string_type_to_float, connection)
+
+
def register_binary_typecasters(connection):
psycopg2.extensions.register_type(
psycopg2.extensions.new_type(
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: [pgAdmin][RM6341] CSV download quotes numeric columns
2021-04-20 05:24 [pgAdmin][RM6341] CSV download quotes numeric columns Pradip Parkale <[email protected]>
@ 2021-04-22 12:12 ` Akshay Joshi <[email protected]>
2021-04-26 11:31 ` Re: [pgAdmin][RM6341] CSV download quotes numeric columns Pradip Parkale <[email protected]>
0 siblings, 1 reply; 4+ messages in thread
From: Akshay Joshi @ 2021-04-22 12:12 UTC (permalink / raw)
To: Pradip Parkale <[email protected]>; +Cc: pgadmin-hackers
Thanks, patch applied.
On Tue, Apr 20, 2021 at 10:54 AM Pradip Parkale <
[email protected]> wrote:
> Hi Hackers,
>
> Please find the attached patch for #6341 CSV download quotes numeric
> columns.
> Initially, the double-precision value was getting typecast to a string.
>
>
> --
> Thanks & Regards,
> Pradip Parkale
> Software Engineer | EnterpriseDB Corporation
>
--
*Thanks & Regards*
*Akshay Joshi*
*pgAdmin Hacker | Principal Software Architect*
*EDB Postgres <http://edbpostgres.com>*
*Mobile: +91 976-788-8246*
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: [pgAdmin][RM6341] CSV download quotes numeric columns
2021-04-20 05:24 [pgAdmin][RM6341] CSV download quotes numeric columns Pradip Parkale <[email protected]>
2021-04-22 12:12 ` Re: [pgAdmin][RM6341] CSV download quotes numeric columns Akshay Joshi <[email protected]>
@ 2021-04-26 11:31 ` Pradip Parkale <[email protected]>
2021-04-26 11:48 ` Re: [pgAdmin][RM6341] CSV download quotes numeric columns Akshay Joshi <[email protected]>
0 siblings, 1 reply; 4+ messages in thread
From: Pradip Parkale @ 2021-04-26 11:31 UTC (permalink / raw)
To: Akshay Joshi <[email protected]>; +Cc: pgadmin-hackers
Hi Akshay,
Please find the updated patch for #6251. I have fixed the issue caused due
to my initial patch.
On Thu, Apr 22, 2021 at 5:42 PM Akshay Joshi <[email protected]>
wrote:
> Thanks, patch applied.
>
> On Tue, Apr 20, 2021 at 10:54 AM Pradip Parkale <
> [email protected]> wrote:
>
>> Hi Hackers,
>>
>> Please find the attached patch for #6341 CSV download quotes numeric
>> columns.
>> Initially, the double-precision value was getting typecast to a string.
>>
>>
>> --
>> Thanks & Regards,
>> Pradip Parkale
>> Software Engineer | EnterpriseDB Corporation
>>
>
>
> --
> *Thanks & Regards*
> *Akshay Joshi*
> *pgAdmin Hacker | Principal Software Architect*
> *EDB Postgres <http://edbpostgres.com>*
>
> *Mobile: +91 976-788-8246*
>
--
Thanks & Regards,
Pradip Parkale
Software Engineer | EnterpriseDB Corporation
Attachments:
[application/octet-stream] RM6341_v2.patch (4.5K, 3-RM6341_v2.patch)
download | inline diff:
diff --git a/web/pgadmin/utils/driver/psycopg2/connection.py b/web/pgadmin/utils/driver/psycopg2/connection.py
index a9d55c637..78521072f 100644
--- a/web/pgadmin/utils/driver/psycopg2/connection.py
+++ b/web/pgadmin/utils/driver/psycopg2/connection.py
@@ -12,7 +12,6 @@ Implementation of Connection.
It is a wrapper around the actual psycopg2 driver, and connection
object.
"""
-
import random
import select
import datetime
@@ -30,7 +29,7 @@ from pgadmin.utils.exception import ConnectionLost, CryptKeyMissing
from pgadmin.utils import get_complete_file_path
from ..abstract import BaseConnection
from .cursor import DictCursor
-from .typecast import register_float_typecasters, register_global_typecasters,\
+from .typecast import numeric_typecasters, register_global_typecasters,\
register_string_typecasters, register_binary_typecasters, \
unregister_numeric_typecasters, \
register_array_to_string_typecasters, ALL_JSON_TYPES
@@ -462,7 +461,6 @@ class Connection(BaseConnection):
self._set_auto_commit(kwargs)
register_string_typecasters(self.conn)
- register_float_typecasters(self.conn)
if self.array_to_string:
register_array_to_string_typecasters(self.conn)
@@ -845,6 +843,8 @@ WHERE db.datname = current_database()""")
if not results:
yield gettext('The query executed did not return any data.')
return
+ # Type cast the numeric values
+ results = numeric_typecasters(results)
header = []
json_columns = []
@@ -912,8 +912,6 @@ WHERE db.datname = current_database()""")
# Registering back type caster for large size data types to string
# which was unregistered at starting
- if any(type['type_code'] == 701 for type in self.column_info):
- register_float_typecasters(self.conn)
register_string_typecasters(self.conn)
return True, gen
diff --git a/web/pgadmin/utils/driver/psycopg2/typecast.py b/web/pgadmin/utils/driver/psycopg2/typecast.py
index a93a53da7..c6d043e7b 100644
--- a/web/pgadmin/utils/driver/psycopg2/typecast.py
+++ b/web/pgadmin/utils/driver/psycopg2/typecast.py
@@ -22,7 +22,6 @@ from .encoding import configure_driver_encodings, get_encoding
configure_driver_encodings(encodings)
-
# OIDs of data types which need to typecast as string to avoid JavaScript
# compatibility issues.
# e.g JavaScript does not support 64 bit integers. It has 64-bit double
@@ -67,7 +66,6 @@ TO_ARRAY_OF_STRING_DATATYPES = (
# OID of record array data type
RECORD_ARRAY = (2287,)
-
# OIDs of builtin array datatypes supported by psycopg2
# OID reference psycopg2/psycopg/typecast_builtins.c
#
@@ -103,22 +101,18 @@ PSYCOPG_SUPPORTED_JSON_ARRAY_TYPES = (199, 3807)
ALL_JSON_TYPES = PSYCOPG_SUPPORTED_JSON_TYPES +\
PSYCOPG_SUPPORTED_JSON_ARRAY_TYPES
-
# INET[], CIDR[]
# OID reference psycopg2/lib/_ipaddress.py
PSYCOPG_SUPPORTED_IPADDRESS_ARRAY_TYPES = (1041, 651)
-
# uuid[]
# OID reference psycopg2/lib/extras.py
PSYCOPG_SUPPORTED_IPADDRESS_ARRAY_TYPES = (2951,)
-
# int4range, int8range, numrange, daterange tsrange, tstzrange[]
# OID reference psycopg2/lib/_range.py
PSYCOPG_SUPPORTED_RANGE_TYPES = (3904, 3926, 3906, 3912, 3908, 3910)
-
# int4range[], int8range[], numrange[], daterange[] tsrange[], tstzrange[]
# OID reference psycopg2/lib/_range.py
PSYCOPG_SUPPORTED_RANGE_ARRAY_TYPES = (3905, 3927, 3907, 3913, 3909, 3911)
@@ -204,12 +198,26 @@ def register_string_typecasters(connection):
psycopg2.extensions.register_type(unicode_array_type, connection)
-def register_float_typecasters(connection):
- # This function is to convert pg types into decimal type
- string_type_to_float = \
- psycopg2.extensions.new_type(TO_STRING_NUMERIC_DATATYPES,
- 'TYPECAST_TO_DECIMAL', _DECIMAL)
- psycopg2.extensions.register_type(string_type_to_float, connection)
+def is_numeric(val):
+ """Check if value is numeric or not"""
+ try:
+ if '.' in val:
+ float(val)
+ else:
+ int(val)
+ except ValueError:
+ return False
+ return True
+
+
+def numeric_typecasters(results):
+ # This function is to convert pg types to numeic type caster
+
+ for result in results:
+ for key, value in result.items():
+ if isinstance(result[key], str) and is_numeric(result[key]):
+ result[key] = float(result[key])
+ return results
def register_binary_typecasters(connection):
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: [pgAdmin][RM6341] CSV download quotes numeric columns
2021-04-20 05:24 [pgAdmin][RM6341] CSV download quotes numeric columns Pradip Parkale <[email protected]>
2021-04-22 12:12 ` Re: [pgAdmin][RM6341] CSV download quotes numeric columns Akshay Joshi <[email protected]>
2021-04-26 11:31 ` Re: [pgAdmin][RM6341] CSV download quotes numeric columns Pradip Parkale <[email protected]>
@ 2021-04-26 11:48 ` Akshay Joshi <[email protected]>
0 siblings, 0 replies; 4+ messages in thread
From: Akshay Joshi @ 2021-04-26 11:48 UTC (permalink / raw)
To: Pradip Parkale <[email protected]>; +Cc: pgadmin-hackers
Thanks, patch applied.
On Mon, Apr 26, 2021 at 5:01 PM Pradip Parkale <
[email protected]> wrote:
> Hi Akshay,
> Please find the updated patch for #6251. I have fixed the issue caused due
> to my initial patch.
>
> On Thu, Apr 22, 2021 at 5:42 PM Akshay Joshi <
> [email protected]> wrote:
>
>> Thanks, patch applied.
>>
>> On Tue, Apr 20, 2021 at 10:54 AM Pradip Parkale <
>> [email protected]> wrote:
>>
>>> Hi Hackers,
>>>
>>> Please find the attached patch for #6341 CSV download quotes numeric
>>> columns.
>>> Initially, the double-precision value was getting typecast to a string.
>>>
>>>
>>> --
>>> Thanks & Regards,
>>> Pradip Parkale
>>> Software Engineer | EnterpriseDB Corporation
>>>
>>
>>
>> --
>> *Thanks & Regards*
>> *Akshay Joshi*
>> *pgAdmin Hacker | Principal Software Architect*
>> *EDB Postgres <http://edbpostgres.com>*
>>
>> *Mobile: +91 976-788-8246*
>>
>
>
> --
> Thanks & Regards,
> Pradip Parkale
> Software Engineer | EnterpriseDB Corporation
>
--
*Thanks & Regards*
*Akshay Joshi*
*pgAdmin Hacker | Principal Software Architect*
*EDB Postgres <http://edbpostgres.com>*
*Mobile: +91 976-788-8246*
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2021-04-26 11:48 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-04-20 05:24 [pgAdmin][RM6341] CSV download quotes numeric columns Pradip Parkale <[email protected]>
2021-04-22 12:12 ` Akshay Joshi <[email protected]>
2021-04-26 11:31 ` Pradip Parkale <[email protected]>
2021-04-26 11:48 ` Akshay Joshi <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox