Subject: [pgAdmin4][Patch]: Regarding RM#1419 - Internal Server Error displayed if create any object through simplified Chinese character

From: Surinder Kumar <[email-masked]>
Date: Tue, 12 Jul 2016 15:27:44 +0530

Hi

*What is happening in code?*
Database is created with Chinese characters successfully with ascii
values(Chinese characters converted to ascii) before saving internally.

but when database properties is retrieved by name, it fails to fetch data
and throws UnicodeError.

*Solution:*
Database name should be encoded in 'utf-8' before storing to database.
This resolved the issue.
I have tested this patch by creating database name using Japanese and
Chinese characters. It works.

Also referenced the code to *set default encoding to 'uff-8' *before start
of application.
https://github.com/joeyespo/grip/issues/86

@Ashesh/@Dave can you please review the patch?
If it looks good, we can do the same changes in case of other nodes.

Thanks,
Surinder Kumar

-- 
Sent via pgadmin-hackers mailing list ([email-masked])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers

---

From: Murtuza Zabuawala <[email-masked]>
Date: Tue, 12 Jul 2016 15:43:42 +0530

Hi,

Just a heads up, This issue is occurs only in runtime & If you try in web
everything works fine when you provide unicode characters as object name.

Now sure but the issue is when runtime(QtWebkit) sends data it does not
encodes it properly & this should be generic fix in runtime.

Regards,
Murtuza

--
Regards,
Murtuza Zabuawala
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Jul 12, 2016 at 3:27 PM, Surinder Kumar <
[email-masked]> wrote:

> Also referenced the code to *set default encoding to 'uff-8' *before
> start of application.

---

From: Surinder Kumar <[email-masked]>
Date: Tue, 12 Jul 2016 15:48:19 +0530

> everything works fine when you provide unicode characters as object name.
No, this issue is reported for both web and runtime application. It is
reproducible to me in web application.

---

From: Surinder Kumar <[email-masked]>
Date: Tue, 12 Jul 2016 15:57:19 +0530

> reproducible to me in web application.
It is reproducible for me on
Mac OS X Yosemite version 10.10.2
Python version: 2.7
Browsers: Chrome & Firefox.

>>> but when database properties is retrieved by name, it fails to fetch
>>> data and throws UnicodeError.
>>>
>>> *Solution:*

---

From: Murtuza Zabuawala <[email-masked]>
Date: Tue, 12 Jul 2016 15:58:13 +0530

May be because of Python2.7 but if you try in Python3. It works.

we can use PEP 0263 <http://www.python.org/dev/peps/pep-0263/> to fix this
issue.  // # -*- coding: utf-8 -*-

---

From: Ashesh Vashi <[email-masked]>
Date: Tue, 12 Jul 2016 16:07:21 +0530

On Tue, Jul 12, 2016 at 3:58 PM, Murtuza Zabuawala <
[email-masked]> wrote:

> we can use PEP 0263 <http://www.python.org/dev/peps/pep-0263/> to fix
> this issue.  // # -*- coding: utf-8 -*-
>
Isn't this about the source code encoding, and not about the input encoding?

--

Thanks & Regards,

Ashesh Vashi
EnterpriseDB INDIA: Enterprise PostgreSQL Company
<http://www.enterprisedb.com/>

*http://www.linkedin.com/in/asheshvashi*
<http://www.linkedin.com/in/asheshvashi>

>>> Just a heads up, This issue is occurs only in runtime & If you try in
>>> web everything works fine when you provide unicode characters as object
>>> name.
>>>

---

From: Surinder Kumar <[email-masked]>
Date: Tue, 12 Jul 2016 16:18:34 +0530

On Tue, Jul 12, 2016 at 4:07 PM, Ashesh Vashi <[email-masked]
> wrote:

> I tried but it didn't worked.
*It gives me error:*
Traceback (most recent call last):
  File
"/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/browser/server_groups/servers/databases/__init__.py",
line 639, in msql
    status, res = self.get_sql(gid, sid, data, did)
  File
"/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/browser/server_groups/servers/databases/__init__.py",
line 686, in get_sql
    SQL += self.get_new_sql(gid, sid, data, did)
  File
"/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/browser/server_groups/servers/databases/__init__.py",
line 721, in get_new_sql
    data=data, conn=self.conn
  File
"/Users/surinder/Documents/Workspaces/pgAdmin_27/lib/python2.7/site-packages/flask/templating.py",
line 128, in render_template
    context, ctx.app)
  File
"/Users/surinder/Documents/Workspaces/pgAdmin_27/lib/python2.7/site-packages/flask/templating.py",
line 110, in _render
    rv = template.render(context)
  File
"/Users/surinder/Documents/Workspaces/pgAdmin_27/lib/python2.7/site-packages/jinja2/environment.py",
line 969, in render
    return self.environment.handle_exception(exc_info, True)
  File
"/Users/surinder/Documents/Workspaces/pgAdmin_27/lib/python2.7/site-packages/jinja2/environment.py",
line 742, in handle_exception
    reraise(exc_type, exc_value, tb)
  File
"/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/browser/server_groups/servers/databases/templates/databases/sql/9.3_plus/create.sql",
line 3, in top-level template code
    CREATE DATABASE {{ conn|qtIdent(data.name) }}
  File
"/Users/surinder/Documents/Projects/pgadmin4/web/pgadmin/utils/driver/psycopg2/__init__.py",
line 1597, in qtIdent
    val = str(val)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-8:
ordinal not in range(128)

---

From: Murtuza Zabuawala <[email-masked]>
Date: Tue, 12 Jul 2016 16:41:11 +0530

Please refer,

http://stackoverflow.com/questions/3828723/why-should-we-not-use-sys-setdefaultencodingutf-8-in-a-py-script

On Tue, Jul 12, 2016 at 4:18 PM, Surinder Kumar <
[email-masked]> wrote:

> On Tue, Jul 12, 2016 at 4:07 PM, Ashesh Vashi <
> [email-masked]> wrote:
>
>>>>> Now sure but the issue is when runtime(QtWebkit) sends data it does
>>>>> not encodes it properly & this should be generic fix in runtime.
>>>>>
>>>>> Regards,
>>>>> Murtuza
>>>>>
>>>>> --
>>>>> Regards,
>>>>> Murtuza Zabuawala