Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bMvbV-0000dG-6Q for pgadmin-hackers@arkaria.postgresql.org; Tue, 12 Jul 2016 11:11:53 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.84_2) (envelope-from ) id 1bMvbT-0001Fr-09 for pgadmin-hackers@arkaria.postgresql.org; Tue, 12 Jul 2016 11:11:51 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1bMvbF-00011y-Rb for pgadmin-hackers@postgresql.org; Tue, 12 Jul 2016 11:11:37 +0000 Received: from mail-qt0-x236.google.com ([2607:f8b0:400d:c0d::236]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.84_2) (envelope-from ) id 1bMvbA-0007MS-Tq for pgadmin-hackers@postgresql.org; Tue, 12 Jul 2016 11:11:37 +0000 Received: by mail-qt0-x236.google.com with SMTP id w38so5929987qtb.0 for ; Tue, 12 Jul 2016 04:11:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=UzhGBSABBdLTgTHw07quxCcP7hLVg2mPI8a2772vUtk=; b=Is4MG3rc+uURHZwDAMr6fm833HUZMvdzWJVpP3qclheMRCiJL9Ajbtl9kHwa7mNfeq 2f2LzLqbDe4yBxG9ZkcAlgu1YnVcfjrkqY1NRqUqcpM/aL0H87G/4LuBx/4wfOGtLqYg TZGtev9YG0zhfVKoCT6R2FTWGD+kDj8Y7jQE2Vvu5v+iepeuEaBA2TA9lYNDuopqKvrh elY9G/i5J3NR9POOCEx3bcUkc1t6Qd/DN+unxYHugqXnrDv7bGRJjIcvSKnazayY1h4x oIVZ6rhmNdR1Cr4KRMLuKRBSQinbPZ92vwRBOiywM1IL8TXvhr4+Rm8R5xQRvEQbUwyI JMdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=UzhGBSABBdLTgTHw07quxCcP7hLVg2mPI8a2772vUtk=; b=M7kT+sDcZDO1BhRxKmhK2JwR7HZqRy4s/6VXrAGNibwVgLMWFC4niyqb4ZJn+jEfgJ yKl6Bq0soLCHEGpWds8RlIrTK0t6kUnmb5ACcUvR8CaAU5mpR0dq9vrCnk4aa/9/uBKV L6YtTEsLkWG0fYo6mmWT4bRE+/wtlRKaALJEcwxrZV/+IvtiN/dwScLp5FvrvfMW05MN OU1seQiQTJM0oF/1/vP8OElIFGkuAu2auS8pDcgu6NnW51945KXrIPskLf5frSHcZKx3 jHpdSngYrJRUt0TaQhrBTncX8/kT/l+T2de4P5aEO7ysKAgB3vwoQbeJ/Au6xGFr4xeV sC1w== X-Gm-Message-State: ALyK8tIeCz/o5tIUfG77CwEbFUeDgC112ZNxIL4OP16ymkP63GJNFQQ80kwWGyvfBrggG3S82EVj8pO4Df/44Qy1 X-Received: by 10.200.43.97 with SMTP id 30mr2701515qtv.2.1468321891119; Tue, 12 Jul 2016 04:11:31 -0700 (PDT) MIME-Version: 1.0 Received: by 10.55.8.3 with HTTP; Tue, 12 Jul 2016 04:11:11 -0700 (PDT) In-Reply-To: References: From: Murtuza Zabuawala Date: Tue, 12 Jul 2016 16:41:11 +0530 Message-ID: Subject: Re: [pgAdmin4][Patch]: Regarding RM#1419 - Internal Server Error displayed if create any object through simplified Chinese character To: Surinder Kumar Cc: Ashesh Vashi , pgadmin-hackers Content-Type: multipart/alternative; boundary=001a113f183abca39205376e54b9 X-Pg-Spam-Score: -2.6 (--) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgadmin-hackers Precedence: bulk Sender: pgadmin-hackers-owner@postgresql.org --001a113f183abca39205376e54b9 Content-Type: text/plain; charset=UTF-8 Please refer, http://stackoverflow.com/questions/3828723/why-should-we-not-use-sys-setdefaultencodingutf-8-in-a-py-script -- Regards, Murtuza Zabuawala EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company On Tue, Jul 12, 2016 at 4:18 PM, Surinder Kumar < surinder.kumar@enterprisedb.com> wrote: > On Tue, Jul 12, 2016 at 4:07 PM, Ashesh Vashi < > ashesh.vashi@enterprisedb.com> wrote: > >> On Tue, Jul 12, 2016 at 3:58 PM, Murtuza Zabuawala < >> murtuza.zabuawala@enterprisedb.com> wrote: >> >>> May be because of Python2.7 but if you try in Python3. It works. >>> >>> we can use PEP 0263 to fix >>> this issue. // # -*- coding: utf-8 -*- >>> >> I tried but it didn't worked. > *It gives me error:* > Traceback (most recent call last): > File > "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/browser/server_groups/servers/databases/__init__.py", > line 639, in msql > status, res = self.get_sql(gid, sid, data, did) > File > "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/browser/server_groups/servers/databases/__init__.py", > line 686, in get_sql > SQL += self.get_new_sql(gid, sid, data, did) > File > "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/browser/server_groups/servers/databases/__init__.py", > line 721, in get_new_sql > data=data, conn=self.conn > File > "/Users/surinder/Documents/Workspaces/pgAdmin_27/lib/python2.7/site-packages/flask/templating.py", > line 128, in render_template > context, ctx.app) > File > "/Users/surinder/Documents/Workspaces/pgAdmin_27/lib/python2.7/site-packages/flask/templating.py", > line 110, in _render > rv = template.render(context) > File > "/Users/surinder/Documents/Workspaces/pgAdmin_27/lib/python2.7/site-packages/jinja2/environment.py", > line 969, in render > return self.environment.handle_exception(exc_info, True) > File > "/Users/surinder/Documents/Workspaces/pgAdmin_27/lib/python2.7/site-packages/jinja2/environment.py", > line 742, in handle_exception > reraise(exc_type, exc_value, tb) > File > "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/browser/server_groups/servers/databases/templates/databases/sql/9.3_plus/create.sql", > line 3, in top-level template code > CREATE DATABASE {{ conn|qtIdent(data.name) }} > File > "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/utils/driver/psycopg2/__init__.py", > line 1597, in qtIdent > val = str(val) > UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-8: > ordinal not in range(128) > > >> Isn't this about the source code encoding, and not about the input >> encoding? >> >> -- >> >> Thanks & Regards, >> >> Ashesh Vashi >> EnterpriseDB INDIA: Enterprise PostgreSQL Company >> >> >> >> *http://www.linkedin.com/in/asheshvashi* >> >> >>> >>> >>> -- >>> Regards, >>> Murtuza Zabuawala >>> EnterpriseDB: http://www.enterprisedb.com >>> The Enterprise PostgreSQL Company >>> >>> On Tue, Jul 12, 2016 at 3:48 PM, Surinder Kumar < >>> surinder.kumar@enterprisedb.com> wrote: >>> >>>> On Tue, Jul 12, 2016 at 3:43 PM, Murtuza Zabuawala < >>>> murtuza.zabuawala@enterprisedb.com> wrote: >>>> >>>>> Hi, >>>>> >>>>> Just a heads up, This issue is occurs only in runtime & If you try in >>>>> web everything works fine when you provide unicode characters as object >>>>> name. >>>>> >>>> No, this issue is reported for both web and runtime application. It is >>>> reproducible to me in web application. >>>> >>>>> >>>>> Now sure but the issue is when runtime(QtWebkit) sends data it does >>>>> not encodes it properly & this should be generic fix in runtime. >>>>> >>>>> Regards, >>>>> Murtuza >>>>> >>>>> -- >>>>> Regards, >>>>> Murtuza Zabuawala >>>>> EnterpriseDB: http://www.enterprisedb.com >>>>> The Enterprise PostgreSQL Company >>>>> >>>>> On Tue, Jul 12, 2016 at 3:27 PM, Surinder Kumar < >>>>> surinder.kumar@enterprisedb.com> wrote: >>>>> >>>>>> Hi >>>>>> >>>>>> *What is happening in code?* >>>>>> Database is created with Chinese characters successfully with ascii >>>>>> values(Chinese characters converted to ascii) before saving internally. >>>>>> >>>>>> but when database properties is retrieved by name, it fails to fetch >>>>>> data and throws UnicodeError. >>>>>> >>>>>> *Solution:* >>>>>> Database name should be encoded in 'utf-8' before storing to database. >>>>>> This resolved the issue. >>>>>> I have tested this patch by creating database name using Japanese and >>>>>> Chinese characters. It works. >>>>>> >>>>>> Also referenced the code to *set default encoding to 'uff-8' *before >>>>>> start of application. >>>>>> https://github.com/joeyespo/grip/issues/86 >>>>>> >>>>>> @Ashesh/@Dave can you please review the patch? >>>>>> If it looks good, we can do the same changes in case of other nodes. >>>>>> >>>>>> Thanks, >>>>>> Surinder Kumar >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org >>>>>> ) >>>>>> To make changes to your subscription: >>>>>> http://www.postgresql.org/mailpref/pgadmin-hackers >>>>>> >>>>>> >>>>> >>>> >>> >> > --001a113f183abca39205376e54b9 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

--=
Regards,
Murtuza ZabuawalaE= nterpriseDB:=C2=A0http://www.enter= prisedb.com
The Enterprise PostgreSQL Company

On Tue, Jul 12, 2016 at 4:18 PM, Surinder Ku= mar <surinder.kumar@enterprisedb.com> wrote:
On Tue, Jul 12, 2016= at 4:07 PM, Ashesh Vashi <ashesh.vashi@enterprisedb.com&g= t; wrote:

On Tue, Jul 12, 2016 at 3:58 PM, = Murtuza Zabuawala <murtuza.zabuawala@enterprisedb.com= > wrote:

May be because of Pyt= hon2.7 but if you try in Python3. It works.

we= can use=C2=A0PEP 0263=C2=A0to fix this is= sue. =C2=A0// # -*- coding: utf-8 -*-=C2=A0
=
I tried but it didn't worked.
=
It gives me error:
Traceback (most recent call last):
=C2=A0 File "/Users/su= rinder/Documents/Projects/pgadmin4/web/pgadmin/browser/server_groups/server= s/databases/__init__.py", line 639, in msql
=C2=A0 =C2=A0 status, res =3D self= .get_sql(gid, sid, data, did)
=C2=A0 File "/Users/surinder/Documents/Projects/= pgadmin4/web/pgadmin/browser/server_groups/servers/databases/__init__.py&qu= ot;, line 686, in get_sql
=C2=A0 =C2=A0 SQL +=3D self.get_new_sql(gid, sid, data, d= id)
=C2= =A0 File "/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/brow= ser/server_groups/servers/databases/__init__.py", line 721, in get_new= _sql
=C2= =A0 =C2=A0 data=3Ddata, conn=3Dself.conn
=C2=A0 File "/Users/surinder/Document= s/Workspaces/pgAdmin_27/lib/python2.7/site-packages/flask/templating.py&quo= t;, line 128, in render_template
=C2=A0 =C2=A0 context, ctx.app)
<= span style=3D"background-color:rgb(204,204,204)">=C2=A0 File "/Users/s= urinder/Documents/Workspaces/pgAdmin_27/lib/python2.7/site-packages/flask/t= emplating.py", line 110, in _render
=C2=A0 =C2=A0 rv =3D template.render(conte= xt)
=C2= =A0 File "/Users/surinder/Documents/Workspaces/pgAdmin_27/lib/python2.= 7/site-packages/jinja2/environment.py", line 969, in render
=C2=A0 =C2=A0 retu= rn self.environment.handle_exception(exc_info, True)
=C2=A0 File "/Users/surin= der/Documents/Workspaces/pgAdmin_27/lib/python2.7/site-packages/jinja2/envi= ronment.py", line 742, in handle_exception
=C2=A0 =C2=A0 reraise(exc_type, exc= _value, tb)
=C2=A0 File "/Users/surinder/Documents/Projects/pgadmin4/web/pgadm= in/browser/server_groups/servers/databases/templates/databases/sql/9.3_plus= /create.sql", line 3, in top-level template code
=C2=A0 =C2=A0 CREATE DATABASE= {{ conn|qtIdent(data.name) }}
<= div>=C2=A0 =C2=A0 val =3D= str(val)
=
Isn't this about the source code encoding, and not ab= out the input encoding?

--

= Thanks & Regards,

Ashesh Vashi
EnterpriseDB I= NDIA:=C2=A0Enterprise PostgreSQL Company




--
Regards,
Murtuza Zabu= awala
EnterpriseDB:=C2=A0ht= tp://www.enterprisedb.com
The Enterprise PostgreSQL Company
<= /div>

On Tue, Jul 12, 2016 at 3:4= 8 PM, Surinder Kumar <surinder.kumar@enterprisedb.com>= ; wrote:
On Tue, Jul 12, 2016 a= t 3:43 PM, Murtuza Zabuawala <murtuza.zabuawala@enterpris= edb.com> wrote:
Hi,

Just a= heads up, This issue is occurs only in runtime & If you try in web eve= rything works fine when you provide unicode characters as object name.
No, this issue is reported for both web and runtim= e application. It is reproducible to me in web application.

Now sure but the issue is when = runtime(QtWebkit) sends data it does not encodes it properly & this sho= uld be generic fix in runtime.

Regards,
= Murtuza

--
Regards,

Hi
=
What is happening in code?
Database is= created with Chinese characters successfully with ascii values(Chinese cha= racters converted to ascii) before saving internally.

<= div>but when database properties is retrieved by name, it fails to fetch da= ta and throws UnicodeError.

Solution:
=
Database name should be encoded in 'utf-8' before storing to d= atabase.
This resolved the issue.=C2=A0
I have tested t= his patch by creating database name using Japanese and Chinese characters. = It works.

Also referenced the code to set defau= lt encoding to 'uff-8' before start of application.
<= a href=3D"https://github.com/joeyespo/grip/issues/86" target=3D"_blank">htt= ps://github.com/joeyespo/grip/issues/86

@A= shesh/@Dave can you please review the patch?
If it looks good, we= can do the same changes in case of other nodes.

T= hanks,
Surinder Kumar




--
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers=







--001a113f183abca39205376e54b9--