Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bMvFX-0007BQ-0R for pgadmin-hackers@arkaria.postgresql.org; Tue, 12 Jul 2016 10:49:11 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.84_2) (envelope-from ) id 1bMvFW-00088s-JS for pgadmin-hackers@arkaria.postgresql.org; Tue, 12 Jul 2016 10:49:10 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1bMvFW-00088k-1y for pgadmin-hackers@postgresql.org; Tue, 12 Jul 2016 10:49:10 +0000 Received: from mail-wm0-x231.google.com ([2a00:1450:400c:c09::231]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.84_2) (envelope-from ) id 1bMvFR-0006mc-Qw for pgadmin-hackers@postgresql.org; Tue, 12 Jul 2016 10:49:09 +0000 Received: by mail-wm0-x231.google.com with SMTP id o80so19940777wme.1 for ; Tue, 12 Jul 2016 03:49:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=M+RiN9Q5L3Lt5NoDVIZswedRYDRgkhkUhFlFtMgalYE=; b=bofIvB4amgcvVCnRvNGc5FKjnKGP2+DfvY4e7ko8rbKkoUGd8e3JXiFQDTtdeS50Ti 7NrB+/vVjuQ0WZ2YNuyFt/wjgfWXrovxAH1x+CKjHS4LltgFjPmawapyxP+HY7Szx38d iB33mUjIH/Ek8sZ3UDZs9P4hs8cE7hQglN8igPBYMUNDQIllECn+3+pk3VEIkgN31xvo lgQaapHU50f9MigbuZ3hEiBGAFxvxYkE0udvoKP1s8DLcY9KVthWA9+UIuhK3jRFL+CP qVeyS0l/LYK/jfhOoh92D7BTCxPNn9XpvKfWYl82Sgghh0FCzPn7QHslZD7CVJCCIPzz 47YA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=M+RiN9Q5L3Lt5NoDVIZswedRYDRgkhkUhFlFtMgalYE=; b=L4P/opvk6ZJavRTHfe+uD+Hjy6M30SljydnuWtbkIfdwDUMAQ5DVlTp3KAYl5ARp+n BfAkJOWarTykW2fT5Q7Fs5948jtBmzYljcgjxSKJL9BDN4j70pt7MEz7C/hjjQtRuLlA pDz3/J4rC+gx8JbiVWHGW1qdi3qTAMVKME6NqhwaJ9jB9OXbVcbBldYwFMpnId4A05Vj VBXdPoHvRD+yr43ghbZ17k5DHk6GN7hYSJfyQfvos5cx89YF2D5hrlVdWt2EoArhmYwU aviNsljTJ0ku+cMDLF8Bybd3y/cUqn7wPBti4dk9lmOKVs17bnhgHGAdFgSXNj7dKCvr Kk6w== X-Gm-Message-State: ALyK8tLbqNgZVlQ6km0ckP2TTjQF4e3aHIuGhQwGX44QIpvZMC9wBfG0yTpvJOl3F9jQpRWZQBaK71JEO/uf4kS4 X-Received: by 10.28.50.199 with SMTP id y190mr14760498wmy.61.1468320544414; Tue, 12 Jul 2016 03:49:04 -0700 (PDT) MIME-Version: 1.0 Received: by 10.28.54.164 with HTTP; Tue, 12 Jul 2016 03:48:34 -0700 (PDT) In-Reply-To: References: From: Surinder Kumar Date: Tue, 12 Jul 2016 16:18:34 +0530 Message-ID: Subject: Re: [pgAdmin4][Patch]: Regarding RM#1419 - Internal Server Error displayed if create any object through simplified Chinese character To: Ashesh Vashi Cc: Murtuza Zabuawala , pgadmin-hackers Content-Type: multipart/alternative; boundary=001a1147da0c777e9105376e0470 X-Pg-Spam-Score: -2.6 (--) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgadmin-hackers Precedence: bulk Sender: pgadmin-hackers-owner@postgresql.org --001a1147da0c777e9105376e0470 Content-Type: text/plain; charset=UTF-8 On Tue, Jul 12, 2016 at 4:07 PM, Ashesh Vashi wrote: > On Tue, Jul 12, 2016 at 3:58 PM, Murtuza Zabuawala < > murtuza.zabuawala@enterprisedb.com> wrote: > >> May be because of Python2.7 but if you try in Python3. It works. >> >> we can use PEP 0263 to fix >> this issue. // # -*- coding: utf-8 -*- >> > I tried but it didn't worked. *It gives me error:* Traceback (most recent call last): File "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/browser/server_groups/servers/databases/__init__.py", line 639, in msql status, res = self.get_sql(gid, sid, data, did) File "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/browser/server_groups/servers/databases/__init__.py", line 686, in get_sql SQL += self.get_new_sql(gid, sid, data, did) File "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/browser/server_groups/servers/databases/__init__.py", line 721, in get_new_sql data=data, conn=self.conn File "/Users/surinder/Documents/Workspaces/pgAdmin_27/lib/python2.7/site-packages/flask/templating.py", line 128, in render_template context, ctx.app) File "/Users/surinder/Documents/Workspaces/pgAdmin_27/lib/python2.7/site-packages/flask/templating.py", line 110, in _render rv = template.render(context) File "/Users/surinder/Documents/Workspaces/pgAdmin_27/lib/python2.7/site-packages/jinja2/environment.py", line 969, in render return self.environment.handle_exception(exc_info, True) File "/Users/surinder/Documents/Workspaces/pgAdmin_27/lib/python2.7/site-packages/jinja2/environment.py", line 742, in handle_exception reraise(exc_type, exc_value, tb) File "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/browser/server_groups/servers/databases/templates/databases/sql/9.3_plus/create.sql", line 3, in top-level template code CREATE DATABASE {{ conn|qtIdent(data.name) }} File "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/utils/driver/psycopg2/__init__.py", line 1597, in qtIdent val = str(val) UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-8: ordinal not in range(128) > Isn't this about the source code encoding, and not about the input > encoding? > > -- > > Thanks & Regards, > > Ashesh Vashi > EnterpriseDB INDIA: Enterprise PostgreSQL Company > > > > *http://www.linkedin.com/in/asheshvashi* > > >> >> >> -- >> Regards, >> Murtuza Zabuawala >> EnterpriseDB: http://www.enterprisedb.com >> The Enterprise PostgreSQL Company >> >> On Tue, Jul 12, 2016 at 3:48 PM, Surinder Kumar < >> surinder.kumar@enterprisedb.com> wrote: >> >>> On Tue, Jul 12, 2016 at 3:43 PM, Murtuza Zabuawala < >>> murtuza.zabuawala@enterprisedb.com> wrote: >>> >>>> Hi, >>>> >>>> Just a heads up, This issue is occurs only in runtime & If you try in >>>> web everything works fine when you provide unicode characters as object >>>> name. >>>> >>> No, this issue is reported for both web and runtime application. It is >>> reproducible to me in web application. >>> >>>> >>>> Now sure but the issue is when runtime(QtWebkit) sends data it does not >>>> encodes it properly & this should be generic fix in runtime. >>>> >>>> Regards, >>>> Murtuza >>>> >>>> -- >>>> Regards, >>>> Murtuza Zabuawala >>>> EnterpriseDB: http://www.enterprisedb.com >>>> The Enterprise PostgreSQL Company >>>> >>>> On Tue, Jul 12, 2016 at 3:27 PM, Surinder Kumar < >>>> surinder.kumar@enterprisedb.com> wrote: >>>> >>>>> Hi >>>>> >>>>> *What is happening in code?* >>>>> Database is created with Chinese characters successfully with ascii >>>>> values(Chinese characters converted to ascii) before saving internally. >>>>> >>>>> but when database properties is retrieved by name, it fails to fetch >>>>> data and throws UnicodeError. >>>>> >>>>> *Solution:* >>>>> Database name should be encoded in 'utf-8' before storing to database. >>>>> This resolved the issue. >>>>> I have tested this patch by creating database name using Japanese and >>>>> Chinese characters. It works. >>>>> >>>>> Also referenced the code to *set default encoding to 'uff-8' *before >>>>> start of application. >>>>> https://github.com/joeyespo/grip/issues/86 >>>>> >>>>> @Ashesh/@Dave can you please review the patch? >>>>> If it looks good, we can do the same changes in case of other nodes. >>>>> >>>>> Thanks, >>>>> Surinder Kumar >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org) >>>>> To make changes to your subscription: >>>>> http://www.postgresql.org/mailpref/pgadmin-hackers >>>>> >>>>> >>>> >>> >> > --001a1147da0c777e9105376e0470 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
On Tue, Jul 12, 2016 at 4:07 PM, Ashesh Vashi <ashesh.vashi@enterprisedb.com> wrote:

On Tue, Jul 12, 2016 at 3:58 PM, Murtuza Zabuawala <murtuza.zabuawala@enterprisedb.com> wrote:

May be because of Python2.7 bu= t if you try in Python3. It works.

we can use= =C2=A0PEP 0263=C2=A0to fix this issue. =C2= =A0// # -*- coding: utf-8 -*-=C2=A0
I tried but it didn't worked.
= It gives me error:
Traceback (most recent call last):
=C2=A0 File "/Users/surinder/D= ocuments/Projects/pgadmin4/web/pgadmin/browser/server_groups/servers/databa= ses/__init__.py", line 639, in msql
=C2=A0 =C2=A0 status, res =3D self.get_sql= (gid, sid, data, did)
=C2=A0 File "/Users/surinder/Documents/Projects/pgadmin4= /web/pgadmin/browser/server_groups/servers/databases/__init__.py", lin= e 686, in get_sql
=C2=A0 =C2=A0 SQL +=3D self.get_new_sql(gid, sid, data, did)
=C2=A0 File = "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/browser/serve= r_groups/servers/databases/__init__.py", line 721, in get_new_sql
=C2=A0 =C2= =A0 data=3Ddata, conn=3Dself.conn
=C2=A0 File "/Users/surinder/Documents/Works= paces/pgAdmin_27/lib/python2.7/site-packages/flask/templating.py", lin= e 128, in render_template
=C2=A0 =C2=A0 context, ctx.app)
=C2=A0 File "/Users/surinder= /Documents/Workspaces/pgAdmin_27/lib/python2.7/site-packages/flask/templati= ng.py", line 110, in _render
=C2=A0 =C2=A0 rv =3D template.render(context)
=C2=A0 File= "/Users/surinder/Documents/Workspaces/pgAdmin_27/lib/python2.7/site-p= ackages/jinja2/environment.py", line 969, in render
<= span style=3D"background-color:rgb(204,204,204)">=C2=A0 =C2=A0 return self.= environment.handle_exception(exc_info, True)
=C2=A0 File "/Users/surinder/Do= cuments/Workspaces/pgAdmin_27/lib/python2.7/site-packages/jinja2/environmen= t.py", line 742, in handle_exception
=C2=A0 =C2=A0 reraise(exc_type, exc_value= , tb)
= =C2=A0 File "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/b= rowser/server_groups/servers/databases/templates/databases/sql/9.3_plus/cre= ate.sql", line 3, in top-level template code
=C2=A0 =C2=A0 CREATE DATABASE {{ = conn|qtIdent(data.name) }}
=C2=A0 File "/User= s/surinder/Documents/Projects/pgadmin4/web/pgadmin/utils/driver/psycopg2/__= init__.py", line 1597, in qtIdent
=C2=A0 =C2=A0 val =3D str(val)
<= div>UnicodeEncodeError: &= #39;ascii' codec can't encode characters in position 0-8: ordinal n= ot in range(128)
=C2=A0
Isn&#= 39;t this about the source code encoding, and not about the input encoding?=

--

Thanks & Regards,

Ashesh Vashi
EnterpriseDB INDIA:=C2=A0Enterprise PostgreSQL= Company


http://www.linkedin.com/in/asheshvashi<= img src=3D"http://s.wisestamp.com/pixel.png?p=3Dmozilla&v=3D2.0.3&t= =3D1285751372076&u=3D4060523&e=3D777" height=3D"1" width=3D"1" styl= e=3D"font-family: 'courier new', monospace;">=C2=A0
<= span>


--
Regards,
Murtuza Zabuawala
EnterpriseDB:=C2=A0http://ww= w.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Jul 12, 2016 at 3:4= 8 PM, Surinder Kumar <surinder.kumar@enterprisedb.com>= ; wrote:
On Tue, Jul 12, 2016 a= t 3:43 PM, Murtuza Zabuawala <murtuza.zabuawala@enterpris= edb.com> wrote:
Hi,

Just a= heads up, This issue is occurs only in runtime & If you try in web eve= rything works fine when you provide unicode characters as object name.
No, this issue is reported for both web and runtim= e application. It is reproducible to me in web application.

Now sure but the issue is when = runtime(QtWebkit) sends data it does not encodes it properly & this sho= uld be generic fix in runtime.

Regards,
= Murtuza

--
Regards,

Hi
=
What is happening in code?
Database is= created with Chinese characters successfully with ascii values(Chinese cha= racters converted to ascii) before saving internally.

<= div>but when database properties is retrieved by name, it fails to fetch da= ta and throws UnicodeError.

Solution:
=
Database name should be encoded in 'utf-8' before storing to d= atabase.
This resolved the issue.=C2=A0
I have tested t= his patch by creating database name using Japanese and Chinese characters. = It works.

Also referenced the code to set defau= lt encoding to 'uff-8' before start of application.
<= a href=3D"https://github.com/joeyespo/grip/issues/86" target=3D"_blank">htt= ps://github.com/joeyespo/grip/issues/86

@A= shesh/@Dave can you please review the patch?
If it looks good, we= can do the same changes in case of other nodes.

T= hanks,
Surinder Kumar




--
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers=






--001a1147da0c777e9105376e0470--