Received: from malur.postgresql.org ([2a02:16a8:dc51::56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1fQA6h-0007qL-Mp for pgadmin-hackers@arkaria.postgresql.org; Tue, 05 Jun 2018 11:26:31 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1fQA6f-0003fJ-MV for pgadmin-hackers@arkaria.postgresql.org; Tue, 05 Jun 2018 11:26:29 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1fQA6f-0003f7-Fp for pgadmin-hackers@lists.postgresql.org; Tue, 05 Jun 2018 11:26:29 +0000 Received: from mail-wm0-x242.google.com ([2a00:1450:400c:c09::242]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1fQA6b-0005Ay-5H for pgadmin-hackers@postgresql.org; Tue, 05 Jun 2018 11:26:28 +0000 Received: by mail-wm0-x242.google.com with SMTP id v16-v6so4197610wmh.5 for ; Tue, 05 Jun 2018 04:26:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pgadmin-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=0PAMcmRnPm9DG14t2IRMqwgA6iITQQ5Pq8eo8mHawWs=; b=n+BvGjtITKsmo5MDtaRKWe479zRyIwnEJzZLfxw9xbke9hvDPGo40300HUSRIDqxp/ LcjC0EnjEEyBFd651gq8I/saBi6FSscncW5d0M1UUbU+mAdozXCihpfAONMxAg8pdcdk xbmlE8vQdB2UNiEx0ouQO3kNa456DFsEDCV9VLH6CrOvRQFnhH8+VkIsmsDjpTh+bw/K H+L4rY9ciSR5VjMBpp/TyA6bQf7IdTW3r64l1cN5aU6KQ7YbVPlQTlYWUQ/WUUE2EATZ M8tF8UkvzYRDM6CD5xx4L3PYwQrA2N3v9PuXwJ+K3Eu9qx8Fcvsotf3yTlv9NWR//N7G iFXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=0PAMcmRnPm9DG14t2IRMqwgA6iITQQ5Pq8eo8mHawWs=; b=NsA6ROZ0Ovt71O1FCgzzTugo+vcHGDNK+8qe7+gE7jbTa8ZWnHH6/RtLXLKDoKGGuA ivIv3DRTpAxWkL8JNaYDvm1ZGX2CG0rqp7t8e4cHPT7FAfZaQkmIHoAlnFsGgNHBZtqS lEwYBjNJCJ1HcGhnb5CvksNUevGTsEC0aBqnPU216kp/TmkSKC4XGuwA3q/S+Ft48F7s YgvMa1dlS5YqtP+kUC/wafgzUZNuqZ4PRJwkTDQKYBE18i8ehIgGMXd2/TGcnWYK0lt+ GTGr/s9bYEmzi/ozlhKhdqIhhN+B+J0l6Enr8UnTaacCj1b0uh7jAfq1kUsLMvlP+TFH 7Y3w== X-Gm-Message-State: APt69E2CXwH89ykBldg6JyqjBh8fCjZsDmtJDlJcgVJgQOdQCcRaD6BO CoHpc59UNwV0+O8qRfQM3TkVXJR51lPhQYjKsBwYnQ== X-Google-Smtp-Source: ADUXVKI6CfS0sq/SpHZdaX2eSOTBBYPL7U7D6jHJOEyfPv1P3UH2DVClOtPd5J6IUGhaDLouFokch/oVUxARJXHl3nY= X-Received: by 2002:a1c:ec84:: with SMTP id h4-v6mr12327607wmi.154.1528197984063; Tue, 05 Jun 2018 04:26:24 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a1c:2907:0:0:0:0:0 with HTTP; Tue, 5 Jun 2018 04:26:23 -0700 (PDT) In-Reply-To: References: From: Dave Page Date: Tue, 5 Jun 2018 12:26:23 +0100 Message-ID: Subject: Re: [pgAdmin4][RM#3289] Can't query SQL_ASCII database. To: Aditya Toshniwal Cc: Joao De Almeida Pereira , pgadmin-hackers Content-Type: multipart/alternative; boundary="000000000000fca4f9056de35169" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --000000000000fca4f9056de35169 Content-Type: text/plain; charset="UTF-8" Hi On Tue, Jun 5, 2018 at 9:50 AM, Aditya Toshniwal < aditya.toshniwal@enterprisedb.com> wrote: > Hi Hackers, > > PFA updated patch. The sqleditor change is sent separately and removed > from current patch as suggested. > The test cases were running fine when the module was specified using --pkg > but were failing in complete run. Fixed that. > I did a quick test by creating a SQL_ASCII database containing a simple table: CREATE TABLE sql_ascii (id serial primary key, data text); And then populated it with data: /Library/PostgreSQL/9.4/bin/psql -d sql_ascii -U postgres -c "INSERT INTO sql_acsii (data) VALUES ('[Windows-1252] Euro: \x80 Double dagger: \x87');" /Library/PostgreSQL/9.4/bin/psql -d sql_ascii -U postgres -c "INSERT INTO sql_ascii (data) VALUES ('[Latin-1] Yen: \xa5 Half: \xbd');" /Library/PostgreSQL/9.4/bin/psql -d sql_ascii -U postgres -c "INSERT INTO sql_ascii (data) VALUES ('[Japanese] Ship: \xe8\x88\xb9');" /Library/PostgreSQL/9.4/bin/psql -d sql_ascii -U postgres -c "INSERT INTO sql_ascii (data) VALUES ('[Invalid UTF-8] Blob: \xf4\xa5\xa3\xa5');" I then right-clicked the table in the treeview, and selected the option to view all rows, and immediately saw an error: 2018-06-05 12:23:27,319: SQL pgadmin: Execute (async) for server #1 - CONN:1187535 (Query-id: 8522474): SELECT * FROM public.sql_ascii ORDER BY id ASC 2018-06-05 12:23:27,320: ERROR pgadmin: Failed to execute query (execute_async) for the server #1 - CONN:1187535(Query-id: 8522474): Error Message:ERROR: invalid byte sequence for encoding "UTF8": 0x80 SQL state: 22021 Running "SELECT * FROM sql_ascii" in the query tool resulted in the same error, however, if I ran "SET client_encoding = 'SQL_ASCII';" first, I do see results. I have confirmed that I've restarted the server after applying the patch. What am I missing? Why don't we just set the client_encoding to SQL_ASCII if it's a SQL_ASCII database? Note that this testing was on Python 2.7.10 on MacOS. > > Kindly review. > > Thanks and Regards, > Aditya Toshniwal > Software Engineer | EnterpriseDB Software Solutions | Pune > "Don't Complain about Heat, Plant a tree" > > On Tue, Jun 5, 2018 at 10:15 AM, Aditya Toshniwal enterprisedb.com> wrote: > >> Hi >> >> On Tue, Jun 5, 2018 at 1:08 AM, Joao De Almeida Pereira < >> jdealmeidapereira@pivotal.io> wrote: >> >>> Hello Aditya, >>> >>>> >>>> >>>> There is no change related to notifications in this patch. >>>> The below code is minor fix related to connection status of sql editor. >>>> Can you please share the code snippet if it is not the below. >>>> >>>> - # Check for the asynchronous notifies statements. >>>> - conn.check_notifies(True) >>>> - notifies = conn.get_notifies() >>>> + if status is not None: >>>> + # Check for the asynchronous notifies statements. >>>> + conn.check_notifies(True) >>>> + notifies = conn.get_notifies() >>>> >>>> >>> This is a minor fix, but is it related to querying SQL_ASCII database? >>> >> No its not. It is something I found when I was working on SQL_ASCII >> related changes. >> Well then, will send a separate patch for it. >> >>> >>> Thanks >>> Victoria && Joao >>> >> >> > -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company --000000000000fca4f9056de35169 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi

On Tue, Jun 5, 2018 at 9:50 AM, Aditya Toshniwal &l= t;ad= itya.toshniwal@enterprisedb.com> wrote:
Hi Hackers,

PFA updated patch. The sqleditor change is sent separately and remove= d from current patch as suggested.
The test cases were running fi= ne when the module was specified using --pkg but were failing in complete r= un. Fixed that.

I did a quick t= est by creating a SQL_ASCII database containing a simple table:
<= br>
CREATE TABLE sql_ascii (id serial primary key, data text);

And then populated it with data:

/Library/PostgreSQL/9.4/bin/psql -d sql_ascii -U postgres -c &qu= ot;INSERT INTO sql_acsii (data) VALUES ('[Windows-1252]=C2=A0 =C2=A0Eur= o: \x80=C2=A0 =C2=A0Double dagger: \x87');"
/Library/Pos= tgreSQL/9.4/bin/psql -d sql_ascii -U postgres -c "INSERT INTO sql_asci= i (data) VALUES ('[Latin-1]=C2=A0 =C2=A0Yen: \xa5=C2=A0 =C2=A0Half: \xb= d');"
/Library/PostgreSQL/9.4/bin/psql -d sql_ascii -U p= ostgres -c "INSERT INTO sql_ascii (data) VALUES ('[Japanese]=C2=A0= =C2=A0Ship: \xe8\x88\xb9');"
/Library/PostgreSQL/9.4/bi= n/psql -d sql_ascii -U postgres -c "INSERT INTO sql_ascii (data) VALUE= S ('[Invalid UTF-8]=C2=A0 Blob: \xf4\xa5\xa3\xa5');"

I then right-clicked the table in the treeview, and s= elected the option to view all rows, and immediately saw an error:

2018-06-05 12:23:27,319: SQL pgadmin: Execute (asy= nc) for server #1 - CONN:1187535 (Query-id: 8522474):
SELECT * FR= OM public.sql_ascii
ORDER BY id ASC=C2=A0
2018-06-05 12= :23:27,320: ERROR pgadmin: Failed to execute query (execute_async) for t= he server #1 - CONN:1187535(Query-id: 8522474):
Error Message:ERR= OR:=C2=A0 invalid byte sequence for encoding "UTF8": 0x80
SQL state: 22021

Running "SELECT * F= ROM sql_ascii" in the query tool resulted in the same error, however, = if I ran "SET client_encoding =3D 'SQL_ASCII';" first, I = do see results.

I have confirmed that I've res= tarted the server after applying the patch.

What a= m I missing? Why don't we just set the client_encoding to SQL_ASCII if = it's a SQL_ASCII database?

Note that this test= ing was on Python 2.7.10 on MacOS.
=C2=A0

Kindl= y review.

Thanks and Regards,
Aditya Toshniwal
Software Engineer |=C2=A0Enter= priseDB Software Solutions |=C2=A0Pune<= /span>
"Don't Complain ab= out Heat, Plant a tree"

On Tue, = Jun 5, 2018 at 10:15 AM, Aditya Toshniwal <aditya.toshniwa= l@enterprisedb.com> wrote:
Hi

<= div class=3D"gmail_quote">On Tue, Jun 5, 2018 at 1:08 AM, Joao De Alm= eida Pereira <jdealmeidapereira@pivotal.io> wrote= :
Hel= lo Aditya,


There is no = change related to notifications in this patch.=C2=A0
The below co= de is minor fix related to connection status of sql editor. Can you please = share the code snippet if it is not the below.

-=C2=A0 =C2=A0 =C2=A0 =C2=A0 # Check for the asynchronous notifies statem= ents.
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 conn.check_notifies(True)
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 notifies =3D conn.get_notifies()
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 if status is not None:
+=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 # Check for the asynchronous notifies st= atements.
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 conn.check_n= otifies(True)
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 notifies= =3D conn.get_notifies()

=
This is a minor fix, but is it related to querying SQ= L_ASCII database?
No its not. It = is something I found when I was working on SQL_ASCII related changes.
=
Well then, will send a separate patch for it.

Thanks
Victoria && Joao
=





--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake=

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Compan= y
--000000000000fca4f9056de35169--