public inbox for [email protected]help / color / mirror / Atom feed
[Patch] Fix Unicode in errmsg 9+ messages / 4 participants [nested] [flat]
* [Patch] Fix Unicode in errmsg @ 2016-07-06 18:55 Magnun Leno (Pessoal) <[email protected]> 0 siblings, 1 reply; 9+ messages in thread From: Magnun Leno (Pessoal) @ 2016-07-06 18:55 UTC (permalink / raw) To: pgadmin-hackers Hi All, Just a small patch to fix server addition/connection errors when using non-ascii languages (pt_BR in my case). I haven't delved enough into the codebase to be sure, but there might be more occurrences of errmsg that need decoding from ASCII to UTF-8. Is there any plans to port PgAdmin4 to Python3, since Python 2 will only be maintained until 2020? Python 3 support was added in Flask 0.10 and, as mentioned in the Flask Docs, "Most libraries and Flask extensions have been ported by now and using Flask with Python 3 is generally a smooth ride.". Thanks for this amazing piece of software. -- Magnun Leno -- Sent via pgadmin-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgadmin-hackers Attachments: [application/x-bzip] fix_unicode_pgadmin4.patch.tar.bz2 (713B, 2-fix_unicode_pgadmin4.patch.tar.bz2) download ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: [Patch] Fix Unicode in errmsg @ 2016-07-07 09:01 Dave Page <[email protected]> parent: Magnun Leno (Pessoal) <[email protected]> 0 siblings, 1 reply; 9+ messages in thread From: Dave Page @ 2016-07-07 09:01 UTC (permalink / raw) To: Magnun Leno (Pessoal) <[email protected]>; +Cc: pgadmin-hackers; Ashesh Vashi <[email protected]> Thanks Magnun. Ashesh, can you review this please, and check if similar changes are needed elsewhere? Thanks. On Wed, Jul 6, 2016 at 7:55 PM, Magnun Leno (Pessoal) <[email protected]> wrote: > Hi All, > > Just a small patch to fix server addition/connection errors when using > non-ascii languages (pt_BR in my case). I haven't delved enough into the > codebase to be sure, but there might be more occurrences of errmsg that need > decoding from ASCII to UTF-8. > > Is there any plans to port PgAdmin4 to Python3, since Python 2 will only be > maintained until 2020? Python 3 support was added in Flask 0.10 and, as > mentioned in the Flask Docs, "Most libraries and Flask extensions have been > ported by now and using Flask with Python 3 is generally a smooth ride.". > > Thanks for this amazing piece of software. > > -- > Magnun Leno > > > -- > Sent via pgadmin-hackers mailing list ([email protected]) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgadmin-hackers > -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgadmin-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgadmin-hackers Attachments: [application/x-bzip] fix_unicode_pgadmin4.patch.tar.bz2 (713B, 2-fix_unicode_pgadmin4.patch.tar.bz2) download ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: [Patch] Fix Unicode in errmsg @ 2016-07-07 11:39 Ashesh Vashi <[email protected]> parent: Dave Page <[email protected]> 0 siblings, 1 reply; 9+ messages in thread From: Ashesh Vashi @ 2016-07-07 11:39 UTC (permalink / raw) To: Dave Page <[email protected]>; +Cc: Magnun Leno (Pessoal) <[email protected]>; pgadmin-hackers Sure - I will do. -- Thanks & Regards, Ashesh Vashi EnterpriseDB INDIA: Enterprise PostgreSQL Company <http://www.enterprisedb.com; *http://www.linkedin.com/in/asheshvashi* <http://www.linkedin.com/in/asheshvashi; On Thu, Jul 7, 2016 at 2:31 PM, Dave Page <[email protected]> wrote: > Thanks Magnun. > > Ashesh, can you review this please, and check if similar changes are > needed elsewhere? > > Thanks. > > On Wed, Jul 6, 2016 at 7:55 PM, Magnun Leno (Pessoal) > <[email protected]> wrote: > > Hi All, > > > > Just a small patch to fix server addition/connection errors when using > > non-ascii languages (pt_BR in my case). I haven't delved enough into the > > codebase to be sure, but there might be more occurrences of errmsg that > need > > decoding from ASCII to UTF-8. > > > > Is there any plans to port PgAdmin4 to Python3, since Python 2 will only > be > > maintained until 2020? Python 3 support was added in Flask 0.10 and, as > > mentioned in the Flask Docs, "Most libraries and Flask extensions have > been > > ported by now and using Flask with Python 3 is generally a smooth ride.". > > > > Thanks for this amazing piece of software. > > > > -- > > Magnun Leno > > > > > > -- > > Sent via pgadmin-hackers mailing list ([email protected]) > > To make changes to your subscription: > > http://www.postgresql.org/mailpref/pgadmin-hackers > > > > > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: [Patch] Fix Unicode in errmsg @ 2016-07-08 16:07 Magnun Leno (Pessoal) <[email protected]> parent: Ashesh Vashi <[email protected]> 0 siblings, 2 replies; 9+ messages in thread From: Magnun Leno (Pessoal) @ 2016-07-08 16:07 UTC (permalink / raw) To: pgadmin-hackers Just found another decoding error, this time during server group creation. Path attached. Thanks again. --- Magnun Leno Em 07-07-2016 08:39, Ashesh Vashi escreveu: > Sure - I will do. > > -- > > Thanks & Regards, > > Ashesh Vashi > EnterpriseDB INDIA: Enterprise PostgreSQL Company > <http://www.enterprisedb.com; > > > /http://www.linkedin.com/in/asheshvashi/ > > > On Thu, Jul 7, 2016 at 2:31 PM, Dave Page <[email protected] > <mailto:[email protected]>> wrote: > > Thanks Magnun. > > Ashesh, can you review this please, and check if similar changes are > needed elsewhere? > > Thanks. > > On Wed, Jul 6, 2016 at 7:55 PM, Magnun Leno (Pessoal) > <[email protected] <mailto:[email protected]>> wrote: > > Hi All, > > > > Just a small patch to fix server addition/connection errors when > using > > non-ascii languages (pt_BR in my case). I haven't delved enough > into the > > codebase to be sure, but there might be more occurrences of > errmsg that need > > decoding from ASCII to UTF-8. > > > > Is there any plans to port PgAdmin4 to Python3, since Python 2 > will only be > > maintained until 2020? Python 3 support was added in Flask 0.10 > and, as > > mentioned in the Flask Docs, "Most libraries and Flask > extensions have been > > ported by now and using Flask with Python 3 is generally a > smooth ride.". > > > > Thanks for this amazing piece of software. > > > > -- > > Magnun Leno > > > > > > -- > > Sent via pgadmin-hackers mailing list > ([email protected] > <mailto:[email protected]>) > > To make changes to your subscription: > > http://www.postgresql.org/mailpref/pgadmin-hackers > > > > > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > > -- Sent via pgadmin-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgadmin-hackers Attachments: [application/x-bzip] fix_unicode_servername_input.patch.tar.bz2 (775B, 3-fix_unicode_servername_input.patch.tar.bz2) download ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: [Patch] Fix Unicode in errmsg @ 2016-07-11 11:36 Dave Page <[email protected]> parent: Magnun Leno (Pessoal) <[email protected]> 1 sibling, 0 replies; 9+ messages in thread From: Dave Page @ 2016-07-11 11:36 UTC (permalink / raw) To: Magnun Leno (Pessoal) <[email protected]>; +Cc: pgadmin-hackers; Ashesh Vashi <[email protected]> Ashesh, please review/commit this one too. Thanks. On Fri, Jul 8, 2016 at 5:07 PM, Magnun Leno (Pessoal) <[email protected] > wrote: > Just found another decoding error, this time during server group creation. > Path attached. > > Thanks again. > > --- > > Magnun Leno > > Em 07-07-2016 08:39, Ashesh Vashi escreveu: > > Sure - I will do. > > -- > > Thanks & Regards, > > Ashesh Vashi > EnterpriseDB INDIA: Enterprise PostgreSQL Company > <http://www.enterprisedb.com; > > > *http://www.linkedin.com/in/asheshvashi* > <http://www.linkedin.com/in/asheshvashi; > > On Thu, Jul 7, 2016 at 2:31 PM, Dave Page <[email protected]> wrote: > >> Thanks Magnun. >> >> Ashesh, can you review this please, and check if similar changes are >> needed elsewhere? >> >> Thanks. >> >> On Wed, Jul 6, 2016 at 7:55 PM, Magnun Leno (Pessoal) >> <[email protected]> wrote: >> > Hi All, >> > >> > Just a small patch to fix server addition/connection errors when using >> > non-ascii languages (pt_BR in my case). I haven't delved enough into the >> > codebase to be sure, but there might be more occurrences of errmsg that >> need >> > decoding from ASCII to UTF-8. >> > >> > Is there any plans to port PgAdmin4 to Python3, since Python 2 will >> only be >> > maintained until 2020? Python 3 support was added in Flask 0.10 and, as >> > mentioned in the Flask Docs, "Most libraries and Flask extensions have >> been >> > ported by now and using Flask with Python 3 is generally a smooth >> ride.". >> > >> > Thanks for this amazing piece of software. >> > >> > -- >> > Magnun Leno >> > >> > >> > -- >> > Sent via pgadmin-hackers mailing list ( >> <[email protected]>[email protected]) >> > To make changes to your subscription: >> > http://www.postgresql.org/mailpref/pgadmin-hackers >> > >> >> >> >> -- >> Dave Page >> Blog: http://pgsnake.blogspot.com >> Twitter: @pgsnake >> >> EnterpriseDB UK: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> > > > > > -- > Sent via pgadmin-hackers mailing list ([email protected]) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgadmin-hackers > > -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: [Patch] Fix Unicode in errmsg @ 2016-07-12 05:33 Ashesh Vashi <[email protected]> parent: Magnun Leno (Pessoal) <[email protected]> 1 sibling, 1 reply; 9+ messages in thread From: Ashesh Vashi @ 2016-07-12 05:33 UTC (permalink / raw) To: Magnun Leno (Pessoal) <[email protected]>; +Cc: pgadmin-hackers Hi Magun, Please find my comments inline.. On Fri, Jul 8, 2016 at 9:37 PM, Magnun Leno (Pessoal) <[email protected] > wrote: > Just found another decoding error, this time during server group creation. > What error are you getting? Can you please share the exact error message to understand the issue properly? We will need to change too many places by this logic. I've attached patch for it. Path attached. > Thanks. -- Thanks & Regards, Ashesh Vashi EnterpriseDB INDIA: Enterprise PostgreSQL Company <http://www.enterprisedb.com/; *http://www.linkedin.com/in/asheshvashi* <http://www.linkedin.com/in/asheshvashi; > Thanks again. > > --- > > Magnun Leno > > Em 07-07-2016 08:39, Ashesh Vashi escreveu: > > Sure - I will do. > > -- > > Thanks & Regards, > > Ashesh Vashi > EnterpriseDB INDIA: Enterprise PostgreSQL Company > <http://www.enterprisedb.com; > > > *http://www.linkedin.com/in/asheshvashi* > <http://www.linkedin.com/in/asheshvashi; > > On Thu, Jul 7, 2016 at 2:31 PM, Dave Page <[email protected]> wrote: > >> Thanks Magnun. >> >> Ashesh, can you review this please, and check if similar changes are >> needed elsewhere? >> >> Thanks. >> >> On Wed, Jul 6, 2016 at 7:55 PM, Magnun Leno (Pessoal) >> <[email protected]> wrote: >> > Hi All, >> > >> > Just a small patch to fix server addition/connection errors when using >> > non-ascii languages (pt_BR in my case). I haven't delved enough into the >> > codebase to be sure, but there might be more occurrences of errmsg that >> need >> > decoding from ASCII to UTF-8. >> > >> > Is there any plans to port PgAdmin4 to Python3, since Python 2 will >> only be >> > maintained until 2020? Python 3 support was added in Flask 0.10 and, as >> > mentioned in the Flask Docs, "Most libraries and Flask extensions have >> been >> > ported by now and using Flask with Python 3 is generally a smooth >> ride.". >> > >> > Thanks for this amazing piece of software. >> > >> > -- >> > Magnun Leno >> > >> > >> > -- >> > Sent via pgadmin-hackers mailing list ( >> <[email protected]>[email protected]) >> > To make changes to your subscription: >> > http://www.postgresql.org/mailpref/pgadmin-hackers >> > >> >> >> >> -- >> Dave Page >> Blog: http://pgsnake.blogspot.com >> Twitter: @pgsnake >> >> EnterpriseDB UK: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> > > > > > -- > Sent via pgadmin-hackers mailing list ([email protected]) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgadmin-hackers > > -- Sent via pgadmin-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgadmin-hackers Attachments: [application/octet-stream] request_unicode_transformation.patch (53.4K, 3-request_unicode_transformation.patch) download | inline diff: diff --git a/web/pgadmin/browser/server_groups/__init__.py b/web/pgadmin/browser/server_groups/__init__.py index 8529aee..dc57aa7 100644 --- a/web/pgadmin/browser/server_groups/__init__.py +++ b/web/pgadmin/browser/server_groups/__init__.py @@ -156,7 +156,9 @@ class ServerGroupView(NodeView): user_id=current_user.id, id=gid).first() - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) if servergroup is None: return make_json_response( @@ -201,7 +203,9 @@ class ServerGroupView(NodeView): ) def create(self): - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) if data[u'name'] != '': try: check_sg = ServerGroup.query.filter_by( diff --git a/web/pgadmin/browser/server_groups/servers/__init__.py b/web/pgadmin/browser/server_groups/servers/__init__.py index 329cd7b..568a69e 100644 --- a/web/pgadmin/browser/server_groups/servers/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/__init__.py @@ -400,7 +400,9 @@ class ServerNode(PGChildNodeView): } idx = 0 - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) from pgadmin.utils.driver import get_driver manager = get_driver(PG_DEFAULT_DRIVER).connection_manager(sid) @@ -554,7 +556,9 @@ class ServerNode(PGChildNodeView): u'role' ] - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) for arg in required_args: if arg not in data: @@ -608,10 +612,16 @@ class ServerNode(PGChildNodeView): if not status: db.session.delete(server) db.session.commit() + + if getattr(errmsg, 'decode'): + errmsg = errmsg.decode('utf-8') + return make_json_response( status=401, success=0, - errormsg=gettext("Unable to connect to server:\n\n%s" % errmsg) + errormsg=gettext( + "Unable to connect to server:\n\n{0}" + ).format(errmsg) ) else: user = manager.user_info @@ -740,8 +750,11 @@ class ServerNode(PGChildNodeView): if user is None: return unauthorized(gettext("Unauthorized request.")) - data = request.form if request.form else json.loads(request.data) if \ - request.data else {} + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) if request.data and hasattr( + request.data, 'decode' + ) else request.data or {} password = None save_password = False @@ -795,8 +808,12 @@ class ServerNode(PGChildNodeView): return internal_server_error(errormsg=str(e)) if not status: + if getattr(errmsg, 'decode'): + errmsg = errmsg.decode('utf-8') current_app.logger.error( - "Could not connected to server(#{0}) - '{1}'.\nError: {2}".format( + gettext( + "Could not connected to server(#{0}) - '{1}'.\nError: {2}" + ).format( server.id, server.name, errmsg ) ) @@ -969,7 +986,10 @@ class ServerNode(PGChildNodeView): sid: Server id """ try: - data = json.loads(request.form['data']) + data = request.form['data'] + data = json.loads( + data.decode('utf-8') if hasattr(data, 'decode') else data + ) if data and ('password' not in data or data['password'] == '' or 'newPassword' not in data or diff --git a/web/pgadmin/browser/server_groups/servers/databases/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/__init__.py index 1f7fd07..adea626 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/__init__.py @@ -441,7 +441,9 @@ class DatabaseView(PGChildNodeView): u'name' ] - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) for arg in required_args: if arg not in data: @@ -515,7 +517,7 @@ class DatabaseView(PGChildNodeView): """Update the database.""" data = request.form if request.form else json.loads( - request.data.decode() + request.data.decode('utf-8') ) info = "nothing to update." @@ -632,7 +634,9 @@ class DatabaseView(PGChildNodeView): data = {} for k, v in request.args.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except ValueError: data[k] = v try: diff --git a/web/pgadmin/browser/server_groups/servers/databases/casts/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/casts/__init__.py index 5c5d5b6..289b4ad 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/casts/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/casts/__init__.py @@ -19,7 +19,7 @@ from pgadmin.browser.collection import CollectionNodeModule from pgadmin.browser.utils import PGChildNodeView from pgadmin.utils.ajax import make_json_response, \ make_response as ajax_response, internal_server_error -from pgadmin.utils.ajax import precondition_required +from pgadmin.utils.ajax import precondition_required, gone from pgadmin.utils.driver import get_driver from config import PG_DEFAULT_DRIVER @@ -323,7 +323,6 @@ class CastView(PGChildNodeView): if not status: return internal_server_error(errormsg=res) - result = res['rows'][0] return ajax_response( response=res['rows'][0], @@ -345,7 +344,9 @@ class CastView(PGChildNodeView): 'trgtyp' ] - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) for arg in required_args: if arg not in data: return make_json_response( @@ -396,7 +397,9 @@ class CastView(PGChildNodeView): :param gid: group id :return: """ - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) sql = self.get_sql(gid, sid, did, data, cid) try: if sql and sql.strip('\n') and sql.strip(' '): @@ -549,7 +552,9 @@ class CastView(PGChildNodeView): :return: """ res = [] - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) sql = render_template("/".join([self.template_path, 'functions.sql']), srctyp=data['srctyp'], trgtyp=data['trgtyp']) diff --git a/web/pgadmin/browser/server_groups/servers/databases/event_triggers/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/event_triggers/__init__.py index b105249..bd2102a 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/event_triggers/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/event_triggers/__init__.py @@ -319,7 +319,9 @@ class EventTriggerView(PGChildNodeView): Returns: """ - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) required_args = { 'name': 'Name', 'eventowner': 'Owner', @@ -382,7 +384,9 @@ class EventTriggerView(PGChildNodeView): Returns: """ - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) try: sql = self.get_sql(data, etid) @@ -481,7 +485,9 @@ class EventTriggerView(PGChildNodeView): data = {} for k, v in request.args.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except ValueError: data[k] = v try: diff --git a/web/pgadmin/browser/server_groups/servers/databases/extensions/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/extensions/__init__.py index c4e0b28..a8e14a9 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/extensions/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/extensions/__init__.py @@ -216,8 +216,9 @@ class ExtensionView(PGChildNodeView): 'name' ] - data = request.form if request.form else \ - json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) for arg in required_args: if arg not in data: return make_json_response( @@ -263,8 +264,9 @@ class ExtensionView(PGChildNodeView): """ This function will update an extension object """ - data = request.form if request.form else \ - json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) SQL = self.getSQL(gid, sid, data, did, eid) try: diff --git a/web/pgadmin/browser/server_groups/servers/databases/foreign_data_wrappers/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/foreign_data_wrappers/__init__.py index 8ccce61..57d3ea0 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/foreign_data_wrappers/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/foreign_data_wrappers/__init__.py @@ -355,7 +355,9 @@ class ForeignDataWrapperView(PGChildNodeView): 'name' ] - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) for arg in required_args: if arg not in data: return make_json_response( @@ -422,7 +424,9 @@ class ForeignDataWrapperView(PGChildNodeView): did: Database ID fid: foreign data wrapper ID """ - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) sql = self.get_sql(gid, sid, data, did, fid) try: if sql and sql.strip('\n') and sql.strip(' '): @@ -513,7 +517,9 @@ class ForeignDataWrapperView(PGChildNodeView): data = {} for k, v in request.args.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except ValueError: data[k] = v diff --git a/web/pgadmin/browser/server_groups/servers/databases/foreign_data_wrappers/foreign_servers/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/foreign_data_wrappers/foreign_servers/__init__.py index 65be843..bafc323 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/foreign_data_wrappers/foreign_servers/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/foreign_data_wrappers/foreign_servers/__init__.py @@ -356,7 +356,9 @@ class ForeignServerView(PGChildNodeView): 'name' ] - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) for arg in required_args: if arg not in data: return make_json_response( @@ -428,7 +430,9 @@ class ForeignServerView(PGChildNodeView): fsid: foreign server ID """ - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) sql = self.get_sql(gid, sid, data, did, fid, fsid) try: if sql and sql.strip('\n') and sql.strip(' '): @@ -526,7 +530,9 @@ class ForeignServerView(PGChildNodeView): data = {} for k, v in request.args.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except ValueError: data[k] = v diff --git a/web/pgadmin/browser/server_groups/servers/databases/foreign_data_wrappers/foreign_servers/user_mapping/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/foreign_data_wrappers/foreign_servers/user_mapping/__init__.py index 1f89033..4f966a9 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/foreign_data_wrappers/foreign_servers/user_mapping/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/foreign_data_wrappers/foreign_servers/user_mapping/__init__.py @@ -361,7 +361,9 @@ class UserMappingView(PGChildNodeView): 'name' ] - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) for arg in required_args: if arg not in data: return make_json_response( @@ -435,7 +437,9 @@ class UserMappingView(PGChildNodeView): umid: User mapping ID """ - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) sql = self.get_sql(gid, sid, data, did, fid, fsid, umid) try: if sql and sql.strip('\n') and sql.strip(' '): @@ -546,7 +550,9 @@ class UserMappingView(PGChildNodeView): data = {} for k, v in request.args.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except ValueError: data[k] = v diff --git a/web/pgadmin/browser/server_groups/servers/databases/languages/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/languages/__init__.py index e1c31c4..97a0e7f 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/languages/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/languages/__init__.py @@ -341,7 +341,9 @@ class LanguageView(PGChildNodeView): did: Database ID lid: Language ID """ - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) sql = self.get_sql(data, lid) try: @@ -389,7 +391,9 @@ class LanguageView(PGChildNodeView): data = {} for k, v in request.args.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except ValueError: data[k] = v diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/schemas/__init__.py index 377e6f7..38c9505 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/schemas/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/__init__.py @@ -490,9 +490,9 @@ It may have been removed by another user. sid: Server ID did: Database ID """ - data = request.form if request.form else \ - json.loads(request.data.decode()) - + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) required_args = { 'name': 'Name' } @@ -559,7 +559,7 @@ It may have been removed by another user. scid: Schema ID """ data = request.form if request.form else json.loads( - request.data.decode() + request.data.decode('utf-8') ) try: SQL = self.get_sql(gid, sid, data, scid) @@ -657,7 +657,9 @@ It may have been removed by another user. data = dict() for k, v in request.args.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except ValueError: data[k] = v diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/domains/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/schemas/domains/__init__.py index 9a9afc4..035c5cf 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/schemas/domains/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/domains/__init__.py @@ -189,7 +189,7 @@ class DomainView(PGChildNodeView, DataTypeReader): data = {} if request.data: - req = json.loads(request.data.decode()) + req = json.loads(request.data.decode('utf-8')) else: req = request.args or request.form @@ -218,12 +218,18 @@ class DomainView(PGChildNodeView, DataTypeReader): if key in list_params and req[key] != '' \ and req[key] is not None: # Coverts string into python list as expected. - data[key] = json.loads(req[key]) + data[key] = json.loads( + req[key].decode('utf-8') if hasattr( + req[key], 'decode' + ) else req[key] + ) elif key == 'typnotnull': - data[key] = True if req[key] == 'true' or req[key] is \ - True else \ - (False if req[key] == 'false' or req[key] is - False else '') + data[key] = True if ( + req[key] == 'true' or req[key] is True + ) else False if ( + req[key] == 'false' or req[key] is False + ) else '' + else: data[key] = req[key] @@ -269,7 +275,6 @@ class DomainView(PGChildNodeView, DataTypeReader): ) ver = self.manager.version - server_type = self.manager.server_type # we will set template path for sql scripts if ver >= 90200: diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/domains/domain_constraints/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/schemas/domains/domain_constraints/__init__.py index 9ae3a9b..bf24806 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/schemas/domains/domain_constraints/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/domains/domain_constraints/__init__.py @@ -193,7 +193,7 @@ class DomainConstraintView(PGChildNodeView): data = {} if request.data: - req = json.loads(request.data.decode()) + req = json.loads(request.data.decode('utf-8')) else: req = request.args or request.form diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/foreign_tables/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/schemas/foreign_tables/__init__.py index 8f4af38..362055e 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/schemas/foreign_tables/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/foreign_tables/__init__.py @@ -232,7 +232,7 @@ class ForeignTableView(PGChildNodeView, DataTypeReader): data = {} if request.data: - req = json.loads(request.data.decode()) + req = json.loads(request.data.decode('utf-8')) else: req = request.args or request.form @@ -267,7 +267,11 @@ class ForeignTableView(PGChildNodeView, DataTypeReader): # Coverts string into python list as expected. data[key] = [] if \ type(req[key]) == list and len(req[key]) == 0 else \ - json.loads(req[key]) + json.loads( + req[key].decode('utf-8') if hasattr( + req[key], 'decode' + ) else req[key] + ) if key == 'inherits': # Convert Table ids from unicode/string to int @@ -349,7 +353,6 @@ class ForeignTableView(PGChildNodeView, DataTypeReader): ) ver = self.manager.version - server_type = self.manager.server_type # Set template path for sql scripts depending # on the server version. diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/functions/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/schemas/functions/__init__.py index 5290129..671dbbd 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/schemas/functions/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/functions/__init__.py @@ -252,7 +252,7 @@ class FunctionView(PGChildNodeView, DataTypeReader): data = {} if request.data: - req = json.loads(request.data.decode()) + req = json.loads(request.data.decode('utf-8')) else: req = request.args or request.form @@ -280,7 +280,11 @@ class FunctionView(PGChildNodeView, DataTypeReader): if key in list_params and req[key] != '' \ and req[key] is not None: # Coverts string into python list as expected. - data[key] = json.loads(req[key]) + data[key] = json.loads( + req[key].decode('utf-8') if hasattr( + req[key], 'decode' + ) else req[key] + ) elif ( key == 'proretset' or key == 'proisstrict' or key == 'prosecdef' or key == 'proiswindow' or @@ -469,8 +473,6 @@ class FunctionView(PGChildNodeView, DataTypeReader): proallargtypes = data['proallargtypes'] \ if data['proallargtypes'] else [] - proargout = [] - proargid = [] proargmodenames = {'i': 'IN', 'o': 'OUT', 'b': 'INOUT', 'v': 'VARIADIC', 't': 'TABLE'} diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/sequences/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/schemas/sequences/__init__.py index b5025a0..90e0d2a 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/schemas/sequences/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/sequences/__init__.py @@ -301,7 +301,9 @@ class SequenceView(PGChildNodeView): u'seqowner', ] - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) for arg in required_args: if arg not in data: @@ -417,7 +419,9 @@ class SequenceView(PGChildNodeView): Returns: """ - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) try: SQL = self.getSQL(gid, sid, did, data, scid, seid) SQL = SQL.strip('\n').strip(' ') @@ -469,7 +473,9 @@ class SequenceView(PGChildNodeView): data = {} for k, v in request.args.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except ValueError: data[k] = v diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/__init__.py index 5720f17..b225a13 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/__init__.py @@ -1348,11 +1348,15 @@ class TableView(PGChildNodeView, DataTypeReader, VacuumSettings): did: Database ID scid: Schema ID """ - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) for k, v in data.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except (ValueError, TypeError): data[k] = v @@ -1381,7 +1385,10 @@ class TableView(PGChildNodeView, DataTypeReader, VacuumSettings): # We will convert it again to list if 'coll_inherits' in data and \ isinstance(data['coll_inherits'], str): - data['coll_inherits'] = json.loads(data['coll_inherits']) + d = data['coll_inherits'] + data['coll_inherits'] = json.loads( + d.decode('utf-8') if hasattr(d, 'decode') else d + ) if 'foreign_key' in data: for c in data['foreign_key']: @@ -1434,12 +1441,14 @@ class TableView(PGChildNodeView, DataTypeReader, VacuumSettings): tid: Table ID """ data = request.form if request.form else json.loads( - request.data.decode() + request.data.decode('utf-8') ) for k, v in data.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except (ValueError, TypeError): data[k] = v @@ -1535,7 +1544,9 @@ class TableView(PGChildNodeView, DataTypeReader, VacuumSettings): tid: Table ID """ # Below will decide if it's simple drop or drop with cascade call - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) # Convert str 'true' to boolean type is_cascade = json.loads(data['cascade']) @@ -1581,7 +1592,9 @@ class TableView(PGChildNodeView, DataTypeReader, VacuumSettings): tid: Table ID """ # Below will decide if it's simple drop or drop with cascade call - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) # Convert str 'true' to boolean type is_enable = json.loads(data['enable']) @@ -1663,7 +1676,9 @@ class TableView(PGChildNodeView, DataTypeReader, VacuumSettings): data = dict() for k, v in request.args.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except (ValueError, TypeError): data[k] = v diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/column/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/column/__init__.py index c60bbfb..f0094cb 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/column/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/column/__init__.py @@ -505,12 +505,14 @@ class ColumnsView(PGChildNodeView, DataTypeReader): tid: Table ID """ data = request.form if request.form else json.loads( - request.data.decode() + request.data.decode('utf-8') ) for k, v in data.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except (ValueError, TypeError): data[k] = v @@ -631,7 +633,9 @@ class ColumnsView(PGChildNodeView, DataTypeReader): tid: Table ID clid: Column ID """ - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) # Adding parent into data dict, will be using it while creating sql data['schema'] = self.schema @@ -688,7 +692,9 @@ class ColumnsView(PGChildNodeView, DataTypeReader): data = dict() for k, v in request.args.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except ValueError: data[k] = v diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/constraints/check_constraint/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/constraints/check_constraint/__init__.py index 32ccf23..ad23cc2 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/constraints/check_constraint/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/constraints/check_constraint/__init__.py @@ -396,11 +396,15 @@ class CheckConstraintView(PGChildNodeView): """ required_args = ['consrc'] - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) for k, v in data.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except (ValueError, TypeError): data[k] = v @@ -545,7 +549,9 @@ class CheckConstraintView(PGChildNodeView): tid: Table Id cid: Check Constraint Id """ - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) try: data['schema'] = self.schema @@ -661,7 +667,9 @@ class CheckConstraintView(PGChildNodeView): data = {} for k, v in request.args.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except ValueError: data[k] = v diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/constraints/exclusion_constraint/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/constraints/exclusion_constraint/__init__.py index 686f0b8..0f02422 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/constraints/exclusion_constraint/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/constraints/exclusion_constraint/__init__.py @@ -441,11 +441,15 @@ class ExclusionConstraintView(PGChildNodeView): """ required_args = ['columns'] - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) for k, v in data.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'deocde') else v + ) except (ValueError, TypeError): data[k] = v @@ -546,7 +550,9 @@ class ExclusionConstraintView(PGChildNodeView): Returns: """ - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) try: data['schema'] = self.schema @@ -666,7 +672,9 @@ class ExclusionConstraintView(PGChildNodeView): data = {} for k, v in request.args.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except ValueError: data[k] = v diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/constraints/foreign_key/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/constraints/foreign_key/__init__.py index 75fd0b9..a6e1a73 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/constraints/foreign_key/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/constraints/foreign_key/__init__.py @@ -464,11 +464,15 @@ class ForeignKeyConstraintView(PGChildNodeView): """ required_args = ['columns'] - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) for k, v in data.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except (ValueError, TypeError): data[k] = v @@ -598,7 +602,9 @@ class ForeignKeyConstraintView(PGChildNodeView): Returns: """ - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) try: data['schema'] = self.schema @@ -726,7 +732,9 @@ class ForeignKeyConstraintView(PGChildNodeView): data = {} for k, v in request.args.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except ValueError: data[k] = v @@ -1044,7 +1052,10 @@ class ForeignKeyConstraintView(PGChildNodeView): index = None try: if data and 'cols' in data: - cols = set(json.loads(data['cols'])) + cols = data['cols'] + cols = set(json.loads( + cols.decode('utf-8') if hasattr(cols, 'decode') else cols + )) index = self.search_coveringindex(tid, cols) return make_json_response( diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/constraints/index_constraint/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/constraints/index_constraint/__init__.py index 8854de2..e68d7ec 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/constraints/index_constraint/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/constraints/index_constraint/__init__.py @@ -444,11 +444,15 @@ class IndexConstraintView(PGChildNodeView): [u'columns', u'index'] # Either of one should be there. ] - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) for k, v in data.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except (ValueError, TypeError): data[k] = v @@ -563,7 +567,9 @@ class IndexConstraintView(PGChildNodeView): Returns: """ - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) try: data['schema'] = self.schema @@ -688,7 +694,9 @@ class IndexConstraintView(PGChildNodeView): data = {} for k, v in request.args.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except ValueError: data[k] = v diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/indexes/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/indexes/__init__.py index 665d4ae..6ada3cd 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/indexes/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/indexes/__init__.py @@ -440,7 +440,6 @@ class IndexesView(PGChildNodeView): # 'options' we need true/false to render switch ASC(false)/DESC(true) columns = [] cols = [] - cnt = 1 for row in rset['rows']: # We need all data as collection for ColumnsModel cols_data = { @@ -522,12 +521,14 @@ class IndexesView(PGChildNodeView): tid: Table ID """ data = request.form if request.form else json.loads( - request.data.decode() + request.data.decode('utf-8') ) for k, v in data.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except (ValueError, TypeError): data[k] = v @@ -672,7 +673,9 @@ class IndexesView(PGChildNodeView): tid: Table ID idx: Index ID """ - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) data['schema'] = self.schema data['table'] = self.table try: @@ -721,7 +724,9 @@ class IndexesView(PGChildNodeView): data = dict() for k, v in request.args.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except ValueError: data[k] = v diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/rules/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/rules/__init__.py index 1b045ec..10290fd 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/rules/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/rules/__init__.py @@ -290,8 +290,9 @@ class RuleView(PGChildNodeView): 'name', ] - data = request.form if request.form else \ - json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) for arg in required_args: if arg not in data: return make_json_response( @@ -331,8 +332,9 @@ class RuleView(PGChildNodeView): """ This function will update a rule object """ - data = request.form if request.form else \ - json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) SQL = self.getSQL(gid, sid, data, tid, rid) try: if SQL and SQL.strip('\n') and SQL.strip(' '): diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/triggers/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/triggers/__init__.py index acffc96..d2492fb 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/triggers/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/triggers/__init__.py @@ -537,12 +537,14 @@ class TriggerView(PGChildNodeView): tid: Table ID """ data = request.form if request.form else json.loads( - request.data.decode() + request.data.decode('utf-8') ) for k, v in data.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except (ValueError, TypeError): data[k] = v @@ -657,7 +659,9 @@ class TriggerView(PGChildNodeView): tid: Table ID trid: Trigger ID """ - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) try: SQL = self.get_sql(scid, tid, trid, data) @@ -705,7 +709,9 @@ class TriggerView(PGChildNodeView): data = dict() for k, v in request.args.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except ValueError: data[k] = v @@ -847,7 +853,9 @@ class TriggerView(PGChildNodeView): trid: Trigger ID """ - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) # Convert str 'true' to boolean type is_enable_flag = json.loads(data['enable']) diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/types/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/schemas/types/__init__.py index 769ebb2..3520999 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/schemas/types/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/types/__init__.py @@ -783,7 +783,9 @@ class TypeView(PGChildNodeView, DataTypeReader): scid: Schema ID tid: Type ID """ - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) required_args = { 'name': 'Name', 'typtype': 'Type' @@ -888,7 +890,9 @@ class TypeView(PGChildNodeView, DataTypeReader): tid: Type ID """ - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) try: SQL = self.get_sql(gid, sid, data, scid, tid) if SQL and SQL.strip('\n') and SQL.strip(' '): @@ -994,7 +998,9 @@ class TypeView(PGChildNodeView, DataTypeReader): # converting nested request data in proper json format for key, val in req.items(): if key in ['composite', 'enum', 'seclabels', 'typacl']: - data[key] = json.loads(val) + data[key] = json.loads( + val.decode('utf-8') if hasattr(val, 'decode') else val + ) else: data[key] = val diff --git a/web/pgadmin/browser/server_groups/servers/databases/schemas/views/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/schemas/views/__init__.py index 741423f..f0b7b91 100644 --- a/web/pgadmin/browser/server_groups/servers/databases/schemas/views/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/databases/schemas/views/__init__.py @@ -466,8 +466,9 @@ class ViewNode(PGChildNodeView, VacuumSettings): 'definition' ] - data = request.form if request.form else \ - json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) for arg in required_args: if arg not in data: return make_json_response( @@ -511,8 +512,9 @@ class ViewNode(PGChildNodeView, VacuumSettings): """ This function will update a view object """ - data = request.form if request.form else \ - json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) SQL = self.getSQL(gid, sid, data, vid) try: if SQL and SQL.strip('\n') and SQL.strip(' '): @@ -626,7 +628,9 @@ class ViewNode(PGChildNodeView, VacuumSettings): data = {} for k, v in request.args.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except ValueError: data[k] = v @@ -1591,8 +1595,9 @@ class MViewNode(ViewNode, VacuumSettings): """ # Below will decide if it's refresh data or refresh concurrently - data = request.form if request.form else \ - json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) is_concurrent = json.loads(data['concurrent']) with_data = json.loads(data['with_data']) diff --git a/web/pgadmin/browser/server_groups/servers/resource_groups/__init__.py b/web/pgadmin/browser/server_groups/servers/resource_groups/__init__.py index 894c3c5..68d74d3 100644 --- a/web/pgadmin/browser/server_groups/servers/resource_groups/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/resource_groups/__init__.py @@ -312,7 +312,9 @@ class ResourceGroupView(NodeView): 'name' ] - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) for arg in required_args: if arg not in data: return make_json_response( @@ -370,7 +372,9 @@ class ResourceGroupView(NodeView): required_args = [ 'name', 'cpu_rate_limit', 'dirty_rate_limit' ] - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) try: sql = render_template("/".join([self.template_path, 'properties.sql']), rgid=rg_id) @@ -462,7 +466,9 @@ class ResourceGroupView(NodeView): data = dict() for k, v in request.args.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'decode') else v + ) except ValueError: data[k] = v diff --git a/web/pgadmin/browser/server_groups/servers/roles/__init__.py b/web/pgadmin/browser/server_groups/servers/roles/__init__.py index ddb55b9..891c9b3 100644 --- a/web/pgadmin/browser/server_groups/servers/roles/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/roles/__init__.py @@ -111,7 +111,10 @@ class RoleView(PGChildNodeView): data = None if request.data: - data = json.loads(request.data) + data = request.data + data = json.loads( + data.decode('utf-8') if hasattr(data, 'decode') else data + ) else: data = dict() req = request.args or request.form @@ -125,7 +128,9 @@ class RoleView(PGChildNodeView): u'rolcatupdate', u'variables', u'rolmembership', u'seclabels' ]: - data[key] = json.loads(val) + data[key] = json.loads( + val.decode('utf-8') if hasattr(val, 'decode') else val + ) else: data[key] = val @@ -146,7 +151,7 @@ class RoleView(PGChildNodeView): data[u'rolvaliduntil'], '%m/%d/%Y' ) data[u'rolvaliduntil'] = date.strftime("%d-%B-%Y") - except Exception as e: + except Exception: return precondition_required( _("Date format is invalid.") ) diff --git a/web/pgadmin/browser/server_groups/servers/tablespaces/__init__.py b/web/pgadmin/browser/server_groups/servers/tablespaces/__init__.py index b853def..8b65379 100644 --- a/web/pgadmin/browser/server_groups/servers/tablespaces/__init__.py +++ b/web/pgadmin/browser/server_groups/servers/tablespaces/__init__.py @@ -252,7 +252,9 @@ class TablespaceView(PGChildNodeView): 'spclocation': 'Location' } - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) for arg in required_args: if arg not in data: @@ -318,7 +320,9 @@ class TablespaceView(PGChildNodeView): """ This function will update tablespace object """ - data = request.form if request.form else json.loads(request.data.decode()) + data = request.form if request.form else json.loads( + request.data.decode('utf-8') + ) try: SQL = self.get_sql(gid, sid, data, tsid) @@ -405,7 +409,9 @@ class TablespaceView(PGChildNodeView): data = dict() for k, v in request.args.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'utf-8') else v + ) except ValueError as ve: current_app.logger.exception(ve) data[k] = v @@ -723,7 +729,10 @@ class TablespaceView(PGChildNodeView): sid: Server ID tsid: Tablespace ID """ - data = json.loads(request.form['data']) + data = request.form['data'] + data = json.loads( + data.decode('utf-8') if hasattr(data, 'decode') else data + ) try: SQL = render_template("/".join( @@ -762,7 +771,9 @@ class TablespaceView(PGChildNodeView): data = dict() for k, v in request.args.items(): try: - data[k] = json.loads(v) + data[k] = json.loads( + v.decode('utf-8') if hasattr(v, 'utf-8') else v + ) except ValueError as ve: current_app.logger.exception(ve) data[k] = v ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: [Patch] Fix Unicode in errmsg @ 2016-07-12 19:41 Magnun Leno <[email protected]> parent: Ashesh Vashi <[email protected]> 0 siblings, 1 reply; 9+ messages in thread From: Magnun Leno @ 2016-07-12 19:41 UTC (permalink / raw) To: Ashesh Vashi <[email protected]>; +Cc: pgadmin-hackers > What error are you getting? > Can you please share the exact error message to understand the issue > properly? Sorry, my bad. Here is the traceback: 2016-07-12 16:12:22,366: ERROR pgadmin: Exception on /browser/server-group/obj/ [POST] Traceback (most recent call last): File "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app response = self.full_dispatch_request() File "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request rv = self.handle_user_exception(e) File "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception reraise(exc_type, exc_value, tb) File "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request rv = self.dispatch_request() File "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/flask/views.py", line 84, in view return self.dispatch_request(*args, **kwargs) File "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/pgadmin4/pgadmin/browser/utils.py", line 235, in dispatch_request return method(*args, **kwargs) File "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/pgadmin4/pgadmin/browser/server_groups/__init__.py", line 204, in create data = request.form if request.form else json.loads(request.data.decode()) UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 14: ordinal not in range(128) How to reproduce: 1. Start PgAdmin4; 2. Access localhost:5050 3. Login 4. In the left panel, right click in Servers. Then, "Create > Server Group", or "Create > Server" (both share the same error) 5. In the "Name field", write any non ASCII word. In my case I was using the following name: Produção 6. Click "Save" More info about my env: $ env | grep LANG LANG=pt_BR.UTF-8 GDM_LANG=pt_BR.UTF-8 $ python --version Python 2.7.11 $ pip freeze Babel==1.3 beautifulsoup4==4.4.1 blinker==1.3 django-htmlmin==0.8.0 extras==0.0.3 fixtures==2.0.0 Flask==0.10.1 Flask-Babel==0.9 Flask-Gravatar==0.4.2 Flask-Login==0.2.11 Flask-Mail==0.9.1 Flask-Principal==0.4.0 Flask-Security==1.7.4 Flask-SQLAlchemy==2.0 Flask-WTF==0.11 html5lib==1.0b3 importlib==1.0.3 itsdangerous==0.24 Jinja2==2.7.3 linecache2==1.0.0 MarkupSafe==0.23 passlib==1.6.2 pbr==1.9.1 pgadmin4==1.0b2 psycopg2==2.5.2 pycrypto==2.6.1 pyrsistent==0.11.13 python-dateutil==2.5.0 python-mimeparse==1.5.1 pytz==2014.10 simplejson==3.6.5 six==1.9.0 speaklater==1.3 SQLAlchemy==0.9.8 sqlparse==0.1.19 testscenarios==0.5.0 testtools==2.0.0 traceback2==1.4.0 unittest2==1.1.0 Werkzeug==0.9.6 WTForms==2.0.2 > > We will need to change too many places by this logic. > I've attached patch for it. I'm not familiar with the code base yet, but not necessarily all occurrences of "decode()" need to be patched, just the ones that deal with user input. I've done some further testing and noticed that even running an insert or update with non ASCII text runs into failure. This "update users set cidade='Brasília' where id=1;", results in the following traceback: 2016-07-12 16:32:59,964: ERROR pgadmin: Exception on /sqleditor/query_tool/start/2760583 [POST] Traceback (most recent call last): File "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app response = self.full_dispatch_request() File "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request rv = self.handle_user_exception(e) File "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception reraise(exc_type, exc_value, tb) File "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request rv = self.dispatch_request() File "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/flask_login.py", line 758, in decorated_view return func(*args, **kwargs) File "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/pgadmin4/pgadmin/tools/sqleditor/__init__.py", line 241, in start_query_tool sql = json.loads(request.data.decode()) UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 48: ordinal not in range(128) But, the other way around isn't true. Executing the update mentioned above in psql and then running a select in PgAdmin4 (select * from users where id=1) works fine. I'll give it a try with Python 3 as soon as possible, it might solve all the issues since it uses unicode by default. Thanks in advance. --- Magnun Leno ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: [Patch] Fix Unicode in errmsg @ 2016-07-13 03:04 Ashesh Vashi <[email protected]> parent: Magnun Leno <[email protected]> 0 siblings, 1 reply; 9+ messages in thread From: Ashesh Vashi @ 2016-07-13 03:04 UTC (permalink / raw) To: Magnun Leno <[email protected]>; +Cc: pgadmin-hackers On Wed, Jul 13, 2016 at 1:11 AM, Magnun Leno <[email protected]> wrote: > What error are you getting? > Can you please share the exact error message to understand the issue > properly? > > Sorry, my bad. Here is the traceback: > > 2016-07-12 16:12:22,366: ERROR pgadmin: Exception on > /browser/server-group/obj/ [POST] > Traceback (most recent call last): > File > "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/flask/app.py", line > 1817, in wsgi_app > response = self.full_dispatch_request() > File > "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/flask/app.py", line > 1477, in full_dispatch_request > rv = self.handle_user_exception(e) > File > "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/flask/app.py", line > 1381, in handle_user_exception > reraise(exc_type, exc_value, tb) > File > "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/flask/app.py", line > 1475, in full_dispatch_request > rv = self.dispatch_request() > File > "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/flask/app.py", line > 1461, in dispatch_request > return self.view_functions[rule.endpoint](**req.view_args) > File > "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/flask/views.py", > line 84, in view > return self.dispatch_request(*args, **kwargs) > File > "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/pgadmin4/pgadmin/browser/utils.py", > line 235, in dispatch_request > return method(*args, **kwargs) > File > "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/pgadmin4/pgadmin/browser/server_groups/__init__.py", > line 204, in create > data = request.form if request.form else > json.loads(request.data.decode()) > UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 14: > ordinal not in range(128) > > How to reproduce: > > 1. Start PgAdmin4; > 2. Access localhost:5050 > 3. Login > 4. In the left panel, right click in Servers. Then, "Create > Server > Group", or "Create > Server" (both share the same error) > 5. In the "Name field", write any non ASCII word. In my case I was > using the following name: Produção > 6. Click "Save" > > More info about my env: > > $ env | grep LANG > LANG=pt_BR.UTF-8 > GDM_LANG=pt_BR.UTF-8 > > $ python --version > Python 2.7.11 > > $ pip freeze > Babel==1.3 > beautifulsoup4==4.4.1 > blinker==1.3 > django-htmlmin==0.8.0 > extras==0.0.3 > fixtures==2.0.0 > Flask==0.10.1 > Flask-Babel==0.9 > Flask-Gravatar==0.4.2 > Flask-Login==0.2.11 > Flask-Mail==0.9.1 > Flask-Principal==0.4.0 > Flask-Security==1.7.4 > Flask-SQLAlchemy==2.0 > Flask-WTF==0.11 > html5lib==1.0b3 > importlib==1.0.3 > itsdangerous==0.24 > Jinja2==2.7.3 > linecache2==1.0.0 > MarkupSafe==0.23 > passlib==1.6.2 > pbr==1.9.1 > pgadmin4==1.0b2 > psycopg2==2.5.2 > pycrypto==2.6.1 > pyrsistent==0.11.13 > python-dateutil==2.5.0 > python-mimeparse==1.5.1 > pytz==2014.10 > simplejson==3.6.5 > six==1.9.0 > speaklater==1.3 > SQLAlchemy==0.9.8 > sqlparse==0.1.19 > testscenarios==0.5.0 > testtools==2.0.0 > traceback2==1.4.0 > unittest2==1.1.0 > Werkzeug==0.9.6 > WTForms==2.0.2 > > > We will need to change too many places by this logic. > I've attached patch for it. > > I'm not familiar with the code base yet, but not necessarily all > occurrences of "decode()" need to be patched, just the ones that deal with > user input. > I've changed only the data coming as the user input for deferent nodes. :-) > > I've done some further testing and noticed that even running an insert or > update with non ASCII text runs into failure. > I've not yet looked at the other utilities in pgAdmin IV. I'll look in to it too, and share the updated patch. > This "update users set cidade='Brasília' where id=1;", results in the > following traceback: > > 2016-07-12 16:32:59,964: ERROR pgadmin: Exception on > /sqleditor/query_tool/start/2760583 [POST] > Traceback (most recent call last): > File > "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/flask/app.py", line > 1817, in wsgi_app > response = self.full_dispatch_request() > File > "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/flask/app.py", line > 1477, in full_dispatch_request > rv = self.handle_user_exception(e) > File > "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/flask/app.py", line > 1381, in handle_user_exception > reraise(exc_type, exc_value, tb) > File > "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/flask/app.py", line > 1475, in full_dispatch_request > rv = self.dispatch_request() > File > "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/flask/app.py", line > 1461, in dispatch_request > return self.view_functions[rule.endpoint](**req.view_args) > File > "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/flask_login.py", > line 758, in decorated_view > return func(*args, **kwargs) > File > "/home/magnun/venv/pgadmin4/lib/python2.7/site-packages/pgadmin4/pgadmin/tools/sqleditor/__init__.py", > line 241, in start_query_tool > sql = json.loads(request.data.decode()) > UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 48: > ordinal not in range(128) > > But, the other way around isn't true. Executing the update mentioned above > in psql and then running a select in PgAdmin4 (select * from users where > id=1) works fine. > > I'll give it a try with Python 3 as soon as possible, it might solve all > the issues since it uses unicode by default. > It's good to have feedback like above. That will give us the idea, where we need improvements, as all the current developers, working on pgAdmin 4, are from English background. Thanks for the inputs/feedbacks, please keep them coming. -- Thanks & Regards, Ashesh Vashi EnterpriseDB INDIA: Enterprise PostgreSQL Company <http://www.enterprisedb.com/; *http://www.linkedin.com/in/asheshvashi* <http://www.linkedin.com/in/asheshvashi; > > Thanks in advance. > > --- > Magnun Leno > > ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: [Patch] Fix Unicode in errmsg @ 2016-07-13 12:49 Magnun Leno <[email protected]> parent: Ashesh Vashi <[email protected]> 0 siblings, 0 replies; 9+ messages in thread From: Magnun Leno @ 2016-07-13 12:49 UTC (permalink / raw) To: Ashesh Vashi <[email protected]>; +Cc: pgadmin-hackers > I've changed only the data coming as the user input for deferent > nodes. :-) Great! > It's good to have feedback like above. > That will give us the idea, where we need improvements, as all the > current developers, working on pgAdmin 4, are from English background. I'm really interested in contributing to the project. I hope to have some spare time soon, so I can study the source code and send some improvements, not just fixes. > > Thanks for the inputs/feedbacks, please keep them coming. I will. Thanks for being so receptive, generally big open source aren't friendly. --- Magnun Leno ^ permalink raw reply [nested|flat] 9+ messages in thread
end of thread, other threads:[~2016-07-13 12:49 UTC | newest] Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2016-07-06 18:55 [Patch] Fix Unicode in errmsg Magnun Leno (Pessoal) <[email protected]> 2016-07-07 09:01 ` Dave Page <[email protected]> 2016-07-07 11:39 ` Ashesh Vashi <[email protected]> 2016-07-08 16:07 ` Magnun Leno (Pessoal) <[email protected]> 2016-07-11 11:36 ` Dave Page <[email protected]> 2016-07-12 05:33 ` Ashesh Vashi <[email protected]> 2016-07-12 19:41 ` Magnun Leno <[email protected]> 2016-07-13 03:04 ` Ashesh Vashi <[email protected]> 2016-07-13 12:49 ` Magnun Leno <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox