Received: from malur.postgresql.org ([2a02:16a8:dc51::56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1fL2w9-0000eS-Bg for pgadmin-hackers@arkaria.postgresql.org; Tue, 22 May 2018 08:46:29 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1fL2w8-0006MB-5X for pgadmin-hackers@arkaria.postgresql.org; Tue, 22 May 2018 08:46:28 +0000 Received: from makus.postgresql.org ([2001:4800:1501:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1fL2w7-0006M1-J5 for pgadmin-hackers@lists.postgresql.org; Tue, 22 May 2018 08:46:28 +0000 Received: from mail-lf0-x244.google.com ([2a00:1450:4010:c07::244]) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1fL2w4-00046m-JC for pgadmin-hackers@postgresql.org; Tue, 22 May 2018 08:46:26 +0000 Received: by mail-lf0-x244.google.com with SMTP id n18-v6so27918729lfh.10 for ; Tue, 22 May 2018 01:46:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=NQkJF45KUpv7DCWOkjPbNgwxa/UpWfG1rQUuk0ktp5s=; b=yRIqzhOUYtr6Yl0s6lTHuHNhX34x7U3H9J9qVuCDjFKMAzP8idRa5mY++sgyHwhFlD hqZiAEcKOdYfOsj8ZlC4u3cV75jDUhJ85ccYoUvQVIjB7KR06pb47rlxWZYzT/SEz5F3 gd2GyRajP33wxhIQIGCG7jP16n4Ftqtd7Z6w5Gs2yQrsCgZ7ORBu4W96wvrAQS8PY+03 PkOfOjG2lus1O+0MJ/yHlNNLJePo27PEYuucnQKv0yN7qqdzT9ELh19G92hSleNFYvh7 TSveWW2ePj2mwMWqdsxVt8/qm9LkFIp3sqNmFKzVYe+Mh6GYaHPnUUcpB11G2Y8MGdsZ L8Qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=NQkJF45KUpv7DCWOkjPbNgwxa/UpWfG1rQUuk0ktp5s=; b=FY+ZKu/r3YPcBhWwofBlOjNgCVTS2yfS3/GFPc9O24NgpqV11PvYJi1EttESIHSQvl 0w9Nd++EXGOcR6DLKpxsldiuaDTWRTIkZ1MmSZnpQHRqbJLc8bcB11dveDLv8breOQo8 falf+1eHiGVtQXGQJCBqJNzjtoAK99FQDDlmpRmT3Iqx2HOgpmcCz0XA1Q1hKMnMcbbR tnsNxmLKeguB7w+xD/UhPiE1H3dCAHtiNfAwzMNzTSYx7TNPTdNskp8ncSrLDz/ptLk4 4XsDlBHAljKm9ZXv6+V3ccdqU7Zf5OqZviVaKaV4Yp0ccFM8xpH55Xjsa3G1/x+tFYNo I5rQ== X-Gm-Message-State: ALKqPweaD45V85ofGTD77yvOssd5MfEai9to22T+ztIeyaTACd3bJg2k 3DVE1wxjxcNMxYeQ9TeNl92P250XZ3pEHB9Ap2QqkA== X-Google-Smtp-Source: AB8JxZrwwN8Fq8Dkurqv83xMA+lEihKPsUSqawNMKah0q0tqDBaXJ8d3U0vWGvbdohj6HcTUEcrEzPKxcHwVW1IZXXA= X-Received: by 2002:a2e:7113:: with SMTP id m19-v6mr14932034ljc.44.1526978782514; Tue, 22 May 2018 01:46:22 -0700 (PDT) MIME-Version: 1.0 Received: by 10.46.158.138 with HTTP; Tue, 22 May 2018 01:46:21 -0700 (PDT) In-Reply-To: References: From: Aditya Toshniwal Date: Tue, 22 May 2018 14:16:21 +0530 Message-ID: Subject: Re: [pgAdmin4][RM#3289] Can't query SQL_ASCII database. To: Dave Page Cc: pgadmin-hackers Content-Type: multipart/mixed; boundary="000000000000e98170056cc773e2" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --000000000000e98170056cc773e2 Content-Type: multipart/alternative; boundary="000000000000e9816d056cc773e0" --000000000000e9816d056cc773e0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi, PFA updated patch. Linter issues are fixed ( we dont have any linter setup for python :-( ) Regarding test cases, they run successfully on my system and the reason it failed for pivotal is timeout exception. I am sorry I can't help with that. Traceback (most recent call last): File "/tmp/build/a453582b/pgadmin-repo/web/pgadmin/feature_tests/keyboard_shortc= ut_test.py", line 52, in runTest self._check_shortcuts() File "/tmp/build/a453582b/pgadmin-repo/web/pgadmin/feature_tests/keyboard_shortc= ut_test.py", line 77, in _check_shortcuts ") and contains(@class, 'open')]") File "/root/.pyenv/versions/pgadmin36/lib/python3.6/site-packages/selenium/webdr= iver/support/wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions.TimeoutException: Message: Thanks and Regards, Aditya Toshniwal Software Engineer | EnterpriseDB Software Solutions | Pune "Don't Complain about Heat, Plant a tree" On Tue, May 22, 2018 at 1:37 PM, Dave Page wrote: > Hi > > Pivotal's buildbot is showing problems with this patch: > > https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/ > pipelines/pgadmin-patch/jobs/run-linter/builds/66 (linter failed) > https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/ > pipelines/pgadmin-patch/jobs/run-tests/builds/84 (tests failed) > > > On Tue, May 22, 2018 at 7:05 AM, Aditya Toshniwal enterprisedb.com> wrote: > >> Hi Hackers, >> >> PFA patch for RM#3289 where decode error was thrown on querying a >> SQL_ASCII database table. Please note, this problem occurs only on windo= ws. >> Sample insert - insert into test_tab values ('=C3=A9'); >> >> psycopg2 has a encodings dictionary where Postgres Database Encodings ar= e >> mapped to python equivalent. It uses 'ascii' decoder of python to decode >> for SQL_ASCII encoding. If data has characters beyond the limit of ascii >> then it failed. The solution would be to use utf_8 decoder instead of >> ascii. I tried setting the client_encoding using >> set_client_encoding('UTF8') method of a psycopg2 connection but no luck >> (also its not allowed for async connection). I also tried executing "SET >> CLIENT_ENCODING=3D'UTF8'" but it didn't work too. >> So, as in the patch, I had to set encodings dict value directly to >> 'utf_8' and it seems to be working. Please note, the same is added to >> psycopg3 milestones >> https://github.com/psycopg/psycopg2/milestone/4 >> >> Also fixed a small glitch for sql editor connection status check. >> >> Kindly review. >> >> Thanks and Regards, >> Aditya Toshniwal >> Software Engineer | EnterpriseDB Software Solutions | Pune >> "Don't Complain about Heat, Plant a tree" >> > > > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EnterpriseDB UK: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > --000000000000e9816d056cc773e0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi,

PFA updated patch. Linter issues ar= e fixed ( we dont have any linter setup for python :-( )
Regardin= g test cases, they run successfully on my system and the reason it failed f= or pivotal is timeout exception. I am sorry I can't help with that.

Traceback (most recent call last):
=C2=A0 File "/t= mp/build/a453582b/pgadmin-repo/web/pgadmin/feature_tests/keyboard_shortcut_= test.py", line 52, in runTest
=C2=A0 =C2=A0 self._check_shortcuts()=
=C2=A0 File "/tmp/build/a453582b/pgadmin-repo/web/pgadmin/feature_= tests/keyboard_shortcut_test.py", line 77, in _check_shortcuts
=C2= =A0 =C2=A0 ") and contains(@class, 'open')]")
=C2=A0 F= ile "/root/.pyenv/versions/pgadmin36/lib/python3.6/site-packages/selen= ium/webdriver/support/wait.py", line 80, in until
=C2=A0 =C2=A0 rai= se TimeoutException(message, screen, stacktrace)
selenium.common.excepti= ons.TimeoutException: Message:

Thanks and Regards,
Aditya Toshniwal
<= div>Software Engineer |=C2=A0En= terpriseDB Software Solutions |=C2=A0Pu= ne
"Don't Complain= about Heat, Plant a tree"
=

On Tue, May 22, 2018 at 1:37 PM, Dave Page <= span dir=3D"ltr"><dpage@pgadmin.org> wrote:
=
Hi

Pivotal's buildbot is showing pr= oblems with this patch:


On Tue, May 22, 2018 at 7:05 AM, Aditya Tos= hniwal <aditya.toshniwal@enterprisedb.com> wrote:
Hi Hacke= rs,

PFA patch for RM#3289 where decode error was t= hrown on querying a SQL_ASCII database table. Please note, this problem occ= urs only on windows.
Sample insert -=C2=A0insert into test_tab va= lues ('=C3=A9');

psycopg2 has a encodings = dictionary where Postgres Database Encodings are mapped to python equivalen= t. It uses 'ascii' decoder of python to decode for SQL_ASCII encodi= ng. If data has characters beyond the limit of ascii then it failed. The so= lution would be to use utf_8 decoder instead of ascii. I tried setting the = client_encoding using set_client_encoding('UTF8') method of a psyco= pg2 connection but no luck (also its not allowed for async connection). I a= lso tried executing "SET CLIENT_ENCODING=3D'UTF8'" but it= didn't work too.
So, as in the patch, I had to set encodings= dict value directly to 'utf_8' and it seems to be working. Please = note, the same is added to psycopg3 milestones

Also fix= ed a small glitch for sql editor connection status check.

Kindly review.

Thanks and Regards,
Aditya Toshniwal
Software Engineer |=C2=A0EnterpriseDB Software Solutions= |=C2=A0Pune
"Don't Complain about Heat, Plant a tree&quo= t;



--
Dave Page
Bl= og: http://pgsnak= e.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com<= /a>
The Enterprise PostgreSQL Company

--000000000000e9816d056cc773e0-- --000000000000e98170056cc773e2 Content-Type: text/x-patch; charset="US-ASCII"; name="RM3289.patch" Content-Disposition: attachment; filename="RM3289.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_jhhfq7ed0 ZGlmZiAtLWdpdCBhL3dlYi9wZ2FkbWluL3Rvb2xzL3NxbGVkaXRvci9fX2luaXRfXy5weSBiL3dl Yi9wZ2FkbWluL3Rvb2xzL3NxbGVkaXRvci9fX2luaXRfXy5weQppbmRleCBjNzI1MDVhLi5mYThi ODNmIDEwMDY0NAotLS0gYS93ZWIvcGdhZG1pbi90b29scy9zcWxlZGl0b3IvX19pbml0X18ucHkK KysrIGIvd2ViL3BnYWRtaW4vdG9vbHMvc3FsZWRpdG9yL19faW5pdF9fLnB5CkBAIC0xNDc0LDcg KzE0NzQsOCBAQCBkZWYgcXVlcnlfdG9vbF9zdGF0dXModHJhbnNfaWQpOgogICAgICAgICAgICAg ZXJyb3Jtc2c9ZXJyb3JfbXNnCiAgICAgICAgICkKIAotICAgIGlmIGNvbm4gYW5kIHRyYW5zX29i aiBhbmQgc2Vzc2lvbl9vYmo6CisgICAgaWYgY29ubiBhbmQgdHJhbnNfb2JqIGFuZCBzZXNzaW9u X29iaiBcCisgICAgICAgIGFuZCBjb25uLnRyYW5zYWN0aW9uX3N0YXR1cygpIGlzIG5vdCBOb25l OgogICAgICAgICBzdGF0dXMgPSBjb25uLnRyYW5zYWN0aW9uX3N0YXR1cygpCiAgICAgICAgIHJl dHVybiBtYWtlX2pzb25fcmVzcG9uc2UoCiAgICAgICAgICAgICBkYXRhPXsKZGlmZiAtLWdpdCBh L3dlYi9wZ2FkbWluL3V0aWxzL2RyaXZlci9wc3ljb3BnMi9jb25uZWN0aW9uLnB5IGIvd2ViL3Bn YWRtaW4vdXRpbHMvZHJpdmVyL3BzeWNvcGcyL2Nvbm5lY3Rpb24ucHkKaW5kZXggMzE1NjMxYy4u OGNhMzBiMyAxMDA2NDQKLS0tIGEvd2ViL3BnYWRtaW4vdXRpbHMvZHJpdmVyL3BzeWNvcGcyL2Nv bm5lY3Rpb24ucHkKKysrIGIvd2ViL3BnYWRtaW4vdXRpbHMvZHJpdmVyL3BzeWNvcGcyL2Nvbm5l Y3Rpb24ucHkKQEAgLTQ5LDYgKzQ5LDEwIEBAIGVsc2U6CiAKIF8gPSBnZXR0ZXh0CiAKKyMgVXNl IHV0Zl84IGRlY29kZXIgaW5zdGVhZCBvZiBwc3ljb3BnMiBkZWZhdWx0IGFzY2lpIGZvcgorIyBT UUxfQVNDSUkgZGF0YWJhc2UgdG8gYXZvaWQgZGVjb2RlIGVycm9ycworZW5jb2RpbmdzWydTUUxf QVNDSUknXSA9ICd1dGZfOCcKK2VuY29kaW5nc1snU1FMQVNDSUknXSA9ICd1dGZfOCcKIAogIyBS ZWdpc3RlciBnbG9iYWwgdHlwZSBjYXN0ZXIgd2hpY2ggd2lsbCBiZSBhcHBsaWNhYmxlIHRvIGFs bCBjb25uZWN0aW9ucy4KIHJlZ2lzdGVyX2dsb2JhbF90eXBlY2FzdGVycygpCg== --000000000000e98170056cc773e2--