Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1gHUNC-0003Bj-BD for psycopg@arkaria.postgresql.org; Tue, 30 Oct 2018 13:47:58 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1gHUNA-0008KS-Iq for psycopg@arkaria.postgresql.org; Tue, 30 Oct 2018 13:47:56 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1gHUNA-0008KL-Ab for psycopg@lists.postgresql.org; Tue, 30 Oct 2018 13:47:56 +0000 Received: from mail-lj1-x244.google.com ([2a00:1450:4864:20::244]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1gHUN7-0007aj-Pb for psycopg@postgresql.org; Tue, 30 Oct 2018 13:47:55 +0000 Received: by mail-lj1-x244.google.com with SMTP id x3-v6so11401940lji.13 for ; Tue, 30 Oct 2018 06:47:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=oApGxWB0HYx2B6gKQbrsYMlmB3ch7XyWbkW6S9H9Pto=; b=PEtyz8VDVhOU/xYQd2WYQO1gQHctqwSKczsOUhnd3wbexd1W7eNlGFM3HibgGuZu9n 4oQgQNYbJNgXoUqCihuQBplgQShW3jiN+pZeKatgle+lL8hAigEY9a+C+4tGCTV28Vk9 u+s5bCQRwyrMRj6pGsflzngDCZx6DuVbXhjzZwxUmeE53QwMWrrE1gHPFAdl0OpOpzyh wjMt+44EcUUYjR1O0it71ZUO1zi7Gk0SdwbB5KVKqX4phWWvfaJhhdOmaLWBa2Ha3ECg FgroXeAxE6HMNc9q2/V8tW9JuHjw2zpMol0ofMpjRQcgHAKPxN4W+8LSwjUzyNngc+Ei 1Lgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=oApGxWB0HYx2B6gKQbrsYMlmB3ch7XyWbkW6S9H9Pto=; b=QzmibfwIbnW3mfwg65Mg5/suJSjwdPLBne2Z8wzLsUL4L1IkO9R45NWGjUyMa2qDw+ BCjxgdICCadwpuT8cqz1JqNbjfotRNtgloQf3QB8kS4tAEujpOZluUoMtUq468hAQE6K Unzz0grascLXZr8OHM6yaoekEAKbwUCgnzKPZ7HXNF4Fwhj8iWhSyZ5rhlb8Dl+LhW3L nvEL4mX4vb2mZSm6BCnVnjrosCWdz9wDgu+WmLQTom5h/HidF7Tnazw+WEKHMZzktD71 ILowLU1Dc30o5VFPvImrsBpMwwywB75Az5CG0jjv9sWLaFHvUkwj4Kpu2gcb7gY6eAu9 umRw== X-Gm-Message-State: AGRZ1gKuT5hFujKRaYlA8gEqJB+RfqMn4QlwJUSTAxps4YdA7VKsnbJJ GfWIcn7oszr5PhHyk+EhDXwrYA+HoQ39ggfPN3A= X-Google-Smtp-Source: AJdET5eMSjMxXCV/MoaPRKQL0ZX9gzkbJ1zGvXJadHODkQRF5IZ9J1i1hFwa9f4iXTECndyQB/Y8C8ifVKmB63aVlrM= X-Received: by 2002:a2e:55d3:: with SMTP id g80-v6mr5719152lje.78.1540907272488; Tue, 30 Oct 2018 06:47:52 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Daniele Varrazzo Date: Tue, 30 Oct 2018 13:47:40 +0000 Message-ID: Subject: Re: psycopg2 conn.poll() hangs when used with sshtunnel To: Akshay Joshi Cc: psycopg@postgresql.org Content-Type: multipart/alternative; boundary="0000000000009bcdf80579726e48" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --0000000000009bcdf80579726e48 Content-Type: text/plain; charset="UTF-8" It seems to me that the problem is in pqpath.c pq_get_last_result() function. The loop assumes PQgetResult won't block, which seems always the case with the direct connection, but PQisBusy() actually returns 1 connecting through the tunnel. So at a first read, the simplification given by pq_get_last_result() is broken: the results should be returned one by one in the normal loop going through PQisBusy(), PQconsumeInput(), and the polling machinery. It seems a chunky refactoring but as it is now I don't like pq_get_last_result() anymore :( -- Daniele On Tue, Oct 30, 2018 at 12:52 PM Daniele Varrazzo < daniele.varrazzo@gmail.com> wrote: > Taking a look. Thank you for the test. > > On Tue, Oct 30, 2018 at 12:36 PM Akshay Joshi < > akshay.joshi@enterprisedb.com> wrote: > >> Hi Team >> >> I have created one sample application to replicate the issue. Attached is >> the sample application, please read "README.txt" file to proceed further. >> Can someone please suggest why poll() function hangs when run wrong query >> with sshtunnel v0.1.4. >> >> On Mon, Oct 8, 2018 at 12:19 PM Akshay Joshi < >> akshay.joshi@enterprisedb.com> wrote: >> >>> Hi Team >>> >>> As one issue(*https://github.com/psycopg/psycopg2/issues/781 >>> *) has already been >>> created for conn.poll() function hangs when executes COPY command in async >>> mode. Similarly conn.poll() functions hangs when we run wrong query in >>> async mode using sshtunnel v0.1.3. >>> >>> Can someone please suggest what went wrong when used sshtunnel, or the >>> pointers/reason where poll() method hangs. >>> >>> On Fri, Sep 28, 2018 at 5:43 PM Akshay Joshi < >>> akshay.joshi@enterprisedb.com> wrote: >>> >>>> Hello >>>> >>>> I am using psycopg2 2.7.4 to connect to the PostgreSQL database server >>>> using asynchronoussupport. It is working absolutely fine. I have used >>>> sshtunnel v0.1.3 in pgAdmin4. >>>> When I connect the PostgreSQL database server using ssh tunnel and run the >>>> valid/correct query it works fine, but when I run any wrong query(invalid >>>> column of table) my application gets hang on conn.poll() function. >>>> >>>> Please refer the code how we use conn.poll() with timeouts >>>> https://git.postgresql.org/gitweb/?p=pgadmin4.git;a=blob;f=web/pgadmin/utils/driver/psycopg2/connection.py;h=4f11c12b30882209c308cb3558e67189c97ea31e;hb=15fe26a7106610b710f3de5b604cd038302c926a#l1363 >>>> >>>> Can anyone please provide some pointers, suggestions? >>>> >>>> -- >>>> *Akshay Joshi* >>>> >>>> *Sr. Software Architect * >>>> >>>> >>>> >>>> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246* >>>> >>> >>> >>> -- >>> *Akshay Joshi* >>> >>> *Sr. Software Architect * >>> >>> >>> >>> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246* >>> >> >> >> -- >> *Akshay Joshi* >> >> *Sr. Software Architect * >> >> >> >> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246* >> > --0000000000009bcdf80579726e48 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
It seems to me that the problem is in pqpath.c pq_get_last= _result() function. The loop assumes PQgetResult won't block, which see= ms always the case with the direct connection, but PQisBusy() actually retu= rns 1 connecting through the tunnel.

So at a first read,= the simplification given by pq_get_last_result() is broken: the results sh= ould be returned one by one in the normal loop going through PQisBusy(), PQ= consumeInput(), and the polling machinery. It seems a chunky refactoring bu= t as it is now I don't like pq_get_last_result() anymore :(
<= br>
-- Daniele=C2=A0


On Tue, Oct 30, 2018 at 12:52 PM Daniele Varr= azzo <daniele.varrazzo@gma= il.com> wrote:
Taking a look. Thank you for the test.

On Tue, Oct 30, 2018 at 12:36 PM Akshay Joshi <akshay.joshi= @enterprisedb.com> wrote:
Hi Team=C2=A0

I have created one sample a= pplication to replicate the issue. Attached is the sample application, plea= se read "README.txt" file to proceed=C2=A0further.
Can = someone please suggest why poll() function hangs when run wrong query with = sshtunnel v0.1.4.

On Mon, Oct 8, 2018 at 12:19 PM Akshay Joshi <akshay.joshi@enterprisedb.com> wrote:
Hi Team=C2=A0

As one issue(https://gith= ub.com/psycopg/psycopg2/issues/781) has already been created for co= nn.poll() function hangs when executes COPY command in async mode. Similarl= y conn.poll() functions hangs when we run wrong query in async mode using= =C2=A0sshtunnel=C2=A0v0.1.3.

Can someone ple= ase suggest what went wrong when used sshtunnel, or the pointers/reason whe= re poll() method hangs.=C2=A0 =C2=A0=C2=A0

On Fri, Sep 28, 2018 at 5:43 PM Akshay Jo= shi <= akshay.joshi@enterprisedb.com> wrote:

= Hello

I am using=C2=A0psycopg2 2.7.4= =C2=A0to connect to the PostgreSQL database server using=C2=A0asynchronoussupport. It i= s working absolutely fine. I have used=C2=A0sshtunnel=C2=A0v0.1.3 in=C2=A0pgAdmin4. When I connect the PostgreSQL dat= abase server using ssh tunnel and run the valid/correct query it works fine= , but when I run any wrong query(invalid column of table) my application ge= ts hang on conn.poll() function.

Please refer the code how we use conn.poll() with timeouts=C2=A0https://gi= t.postgresql.org/gitweb/?p=3Dpgadmin4.git;a=3Dblob;f=3Dweb/pgadmin/utils/dr= iver/psycopg2/connection.py;h=3D4f11c12b30882209c308cb3558e67189c97ea31e;hb= =3D15fe26a7106610b710f3de5b604cd038302c926a#l1363

Can anyone please provide some pointers, sugge= stions?


--
Akshay Joshi
Sr. Software Architect
<= /font>

=

Phone: +91 20-3058= -9517
Mobile: +91 976-788-8246
=


--
Akshay Joshi
Sr. Software Architect
<= br>

Phone: +91 20-3= 058-9517
Mobile: +91 976-788-8246


--
Akshay Joshi=
Sr.= Software Architect


Phone: +91 20-3058-9517
Mobile: +91 976-788-8246
--0000000000009bcdf80579726e48--