Received: from malur.postgresql.org ([2a02:16a8:dc51::56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1fL2L1-0006Qy-Ps for pgadmin-hackers@arkaria.postgresql.org; Tue, 22 May 2018 08:08:08 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1fL2L0-0003zj-D1 for pgadmin-hackers@arkaria.postgresql.org; Tue, 22 May 2018 08:08:06 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1fL2L0-0003zZ-7z for pgadmin-hackers@lists.postgresql.org; Tue, 22 May 2018 08:08:06 +0000 Received: from mail-wr0-x234.google.com ([2a00:1450:400c:c0c::234]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1fL2Kt-0008Ud-7K for pgadmin-hackers@postgresql.org; Tue, 22 May 2018 08:08:05 +0000 Received: by mail-wr0-x234.google.com with SMTP id w3-v6so10781563wrl.12 for ; Tue, 22 May 2018 01:07:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pgadmin-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=H9IpZY1Q3xUPuOfRTyX/gMq5XKHCciU58sJrDKbpU3c=; b=sKFeiJFBKnmOWfSassrFm50Uegs+kWEJgNDWEQM41eduzBcF/fWV20wtWMbdY/PXV9 5/mwRwW7n0zzOPfsaG3GUzj7dIbMEYa1T66CYNBllR78DaZAaBdpU73ED8oVfPwqPWbo IhBI0VbPv8y6MmR7KyFdwX9bFXMY1yf86/V9IPB9tQvbVB4RZMVw0z2inBFsP/ZjAtob kliRZa47YeaWhZWdCp1CSaY96aZ10u6BHjzfY8V+omPFXqDbDV8ZhdyspImfdKaeMM/f fudpFeSNsmSH7hMOS2Q+TOu8izyWQlYENeSfehMdUPNsBGNsGOqcG4tyoOiTxdgOMW12 nYRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=H9IpZY1Q3xUPuOfRTyX/gMq5XKHCciU58sJrDKbpU3c=; b=Go/VX8rdrIG60GF9vQ8M+bT36RM8GI0+Y88KWSdyTFic61Ft/VbaOru1X5cGosqrnW cujwJYxmkMaiFqngTGKZscN08rxOipAnoxSdaPtTUQzNtYRTpOIZZoi4eevJBIOEf/qr IDIfbJtLqb1Mf9b+WNwDAr61XqTMWuDCVn+1NuX8xgeW6woG+dfB37Z9/jEwybWVic8b 7ue8mP4mrlTtcjCxCllLrB75qd5Gh3UU7HauzE1XdR1Cd4MWke6kThbSyynVjj7fBCxk rbm4/aUUYvK5aAOWS/Ep8vPd1O90w4rammYEMSlRiUhsReCNhVObAENVzXtLnD+FfVDH chSA== X-Gm-Message-State: ALKqPwdQ7AXCGc5O61DdqjBftA1+UVLzxgUpP4CioqnkiEflZnUlv4Wy 9CfSm96ckcgCrHu76NQTOYk4swCku7+h/DD2VuPqTQ== X-Google-Smtp-Source: AB8JxZrYIZEs2RU36dVi/mGa4ZUa1M2iW0FdPJ3XE7l/LW1kHnLDMDp5eXEBnDFygtJkfPx8ChUzv2UsE88GeUCf8Nw= X-Received: by 2002:adf:86b2:: with SMTP id 47-v6mr17629827wrx.256.1526976477035; Tue, 22 May 2018 01:07:57 -0700 (PDT) MIME-Version: 1.0 Received: by 10.28.85.194 with HTTP; Tue, 22 May 2018 01:07:56 -0700 (PDT) In-Reply-To: References: From: Dave Page Date: Tue, 22 May 2018 09:07:56 +0100 Message-ID: Subject: Re: [pgAdmin4][RM#3289] Can't query SQL_ASCII database. To: Aditya Toshniwal Cc: pgadmin-hackers Content-Type: multipart/alternative; boundary="0000000000007e8715056cc6ea43" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --0000000000007e8715056cc6ea43 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Pivotal's buildbot is showing problems with this patch: https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/= jobs/run-linter/builds/66 (linter failed) https://gpdb-dev.bosh.pivotalci.info/teams/pgadmin/pipelines/pgadmin-patch/= jobs/run-tests/builds/84 (tests failed) On Tue, May 22, 2018 at 7:05 AM, Aditya Toshniwal < aditya.toshniwal@enterprisedb.com> wrote: > Hi Hackers, > > PFA patch for RM#3289 where decode error was thrown on querying a > SQL_ASCII database table. Please note, this problem occurs only on window= s. > Sample insert - insert into test_tab values ('=C3=A9'); > > psycopg2 has a encodings dictionary where Postgres Database Encodings are > mapped to python equivalent. It uses 'ascii' decoder of python to decode > for SQL_ASCII encoding. If data has characters beyond the limit of ascii > then it failed. The solution would be to use utf_8 decoder instead of > ascii. I tried setting the client_encoding using > set_client_encoding('UTF8') method of a psycopg2 connection but no luck > (also its not allowed for async connection). I also tried executing "SET > CLIENT_ENCODING=3D'UTF8'" but it didn't work too. > So, as in the patch, I had to set encodings dict value directly to 'utf_8= ' > and it seems to be working. Please note, the same is added to psycopg3 > milestones > https://github.com/psycopg/psycopg2/milestone/4 > > Also fixed a small glitch for sql editor connection status check. > > Kindly review. > > Thanks and Regards, > Aditya Toshniwal > Software Engineer | EnterpriseDB Software Solutions | Pune > "Don't Complain about Heat, Plant a tree" > --=20 Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company --0000000000007e8715056cc6ea43 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

On Tue, May 22, 2018 at 7:05 AM, Aditya Toshniwal <aditya.toshniwal@enterprisedb.com> wrote:
Hi Hackers,

=
PFA patch for RM#3289 where decode error was thrown on querying = a SQL_ASCII database table. Please note, this problem occurs only on window= s.
Sample insert -=C2=A0insert into test_tab values ('=C3=A9&= #39;);

psycopg2 has a encodings dictionary where P= ostgres Database Encodings are mapped to python equivalent. It uses 'as= cii' decoder of python to decode for SQL_ASCII encoding. If data has ch= aracters beyond the limit of ascii then it failed. The solution would be to= use utf_8 decoder instead of ascii. I tried setting the client_encoding us= ing set_client_encoding('UTF8') method of a psycopg2 connection but= no luck (also its not allowed for async connection). I also tried executin= g "SET CLIENT_ENCODING=3D'UTF8'" but it didn't work t= oo.
So, as in the patch, I had to set encodings dict value direct= ly to 'utf_8' and it seems to be working. Please note, the same is = added to psycopg3 milestones

Also fixed a small glitch = for sql editor connection status check.

Kindly rev= iew.

Thanks and Regards,
= Aditya Toshniwal<= /div>
Software Engineer |= =C2=A0EnterpriseDB Software Solutions |=C2=A0Pune
"Don't = Complain about Heat, Plant a tree"



--
Dave Page
Blog: = http://pgsnake.bl= ogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com<= br>The Enterprise PostgreSQL Company
--0000000000007e8715056cc6ea43--