Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1gHUUB-0003Uq-8U for psycopg@arkaria.postgresql.org; Tue, 30 Oct 2018 13:55:11 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1gHUU9-0002by-7D for psycopg@arkaria.postgresql.org; Tue, 30 Oct 2018 13:55:09 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1gHUU9-0002br-1I for psycopg@lists.postgresql.org; Tue, 30 Oct 2018 13:55:09 +0000 Received: from mail-qk1-x744.google.com ([2607:f8b0:4864:20::744]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1gHUU6-0007kW-1R for psycopg@postgresql.org; Tue, 30 Oct 2018 13:55:08 +0000 Received: by mail-qk1-x744.google.com with SMTP id n12so7239203qkh.11 for ; Tue, 30 Oct 2018 06:55:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=WmULucUJiQ6mIow4kx9MDinDO4SOjmlr6hz5/Zo2wHw=; b=WzM01YqB6GgceA2/t4FCuojgUZtbMWWRmKFdtvFkH7B5I1Rs5LM4nYpx0SbtK+4jPD JTFgV3DfE+Fxcbt5s9BoyasgoE853X83pdg0srA6AA8cpOAsabnEe7xsl4Mwr2WpElPW cK/5FTUhzlcoDp4+3zfRHwTy5xU5GCMHdfXiWrqH3mf3cZJtFVStMigstwtVjBf+1qeH upNVWgnDPuT+TEcfnekSXMTtF/pCB7tgLXxXWgQuFk9nySsHi7qRpXs4qJuDRQdG5PRT SAj79RZzWtcq5LW1hf9Ho8QIiG6kYKg1pFgtJVqwMva17AEnQ84KeORfrGrVqnt9jIX0 cD2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=WmULucUJiQ6mIow4kx9MDinDO4SOjmlr6hz5/Zo2wHw=; b=j6fuCCG/yrojfM7D3bJlwNTFEiLOebyRbXXPJ14sb5BUm/yyfaWWmXq702YhxuYM6F /nj4/xB0pWyzUQXB0gYWCmJ1tE6/bR9igscaIEBCD+Ja6fhWgEfIA/iHrNU+wGsOUkHE bp+Cz1+jFqdORILXRP8ehOkBNxNiXjEmySsbeeuKKXAO9ku9yWF+pInKiK4kwpqbDpID HgfAen3nKoLLX2ar02OsTE3i6Zu2Zz3m8ciQRHMkgHQfQ7InJz8VAna3ZMpYSu0g/Uh0 7YUixq25d06xS7E83x7GF9rFnpVII2fmgiO2VvDtFM/7+nF31RJm934EvXFgnmDRvQ9s V/FA== X-Gm-Message-State: AGRZ1gKB1wpcASID7WvUrZRIEeYS/FrAkSiBQynZvJwJW0GB4U1l1cKa O8GUxGr0tneuc/DrdCj5tmrx5gcIk6s71aiXPTcJ1A== X-Google-Smtp-Source: AJdET5f7883XLkqlrCUvcKI5TAXZ45RGyd7q/mFHb17X8BqVDYQnUJJ4OFBFFJEub7CNF0u96mYM6SpYbWTLSpPPPRM= X-Received: by 2002:a37:a1c1:: with SMTP id k184mr15371754qke.166.1540907703791; Tue, 30 Oct 2018 06:55:03 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Akshay Joshi Date: Tue, 30 Oct 2018 19:24:51 +0530 Message-ID: Subject: Re: psycopg2 conn.poll() hangs when used with sshtunnel To: Daniele Varrazzo Cc: psycopg@postgresql.org Content-Type: multipart/alternative; boundary="000000000000510dc70579728871" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --000000000000510dc70579728871 Content-Type: text/plain; charset="UTF-8" Hi On Tue, 30 Oct 2018, 19:17 Daniele Varrazzo It seems to me that the problem is in pqpath.c pq_get_last_result() > function. The loop assumes PQgetResult won't block, which seems always the > case with the direct connection, but PQisBusy() actually returns 1 > connecting through the tunnel. > > So at a first read, the simplification given by pq_get_last_result() is > broken: the results should be returned one by one in the normal loop going > through PQisBusy(), PQconsumeInput(), and the polling machinery. It seems a > chunky refactoring but as it is now I don't like pq_get_last_result() > anymore :( > What changes should I made to fix this? Or it should be fixed in psycopg2 itself. > > -- Daniele > > > On Tue, Oct 30, 2018 at 12:52 PM Daniele Varrazzo < > daniele.varrazzo@gmail.com> wrote: > >> Taking a look. Thank you for the test. >> >> On Tue, Oct 30, 2018 at 12:36 PM Akshay Joshi < >> akshay.joshi@enterprisedb.com> wrote: >> >>> Hi Team >>> >>> I have created one sample application to replicate the issue. Attached >>> is the sample application, please read "README.txt" file to proceed further. >>> Can someone please suggest why poll() function hangs when run wrong >>> query with sshtunnel v0.1.4. >>> >>> On Mon, Oct 8, 2018 at 12:19 PM Akshay Joshi < >>> akshay.joshi@enterprisedb.com> wrote: >>> >>>> Hi Team >>>> >>>> As one issue(*https://github.com/psycopg/psycopg2/issues/781 >>>> *) has already been >>>> created for conn.poll() function hangs when executes COPY command in async >>>> mode. Similarly conn.poll() functions hangs when we run wrong query in >>>> async mode using sshtunnel >>>> v0.1.3. >>>> >>>> Can someone please suggest what went wrong when used sshtunnel, or the >>>> pointers/reason where poll() method hangs. >>>> >>>> On Fri, Sep 28, 2018 at 5:43 PM Akshay Joshi < >>>> akshay.joshi@enterprisedb.com> wrote: >>>> >>>>> Hello >>>>> >>>>> I am using psycopg2 2.7.4 to connect to the PostgreSQL database >>>>> server using asynchronoussupport. It is working absolutely fine. I >>>>> have used sshtunnel v0.1.3 in >>>>> pgAdmin4. When I connect the PostgreSQL database server using ssh >>>>> tunnel and run the valid/correct query it works fine, but when I run any >>>>> wrong query(invalid column of table) my application gets hang on >>>>> conn.poll() function. >>>>> >>>>> Please refer the code how we use conn.poll() with timeouts >>>>> https://git.postgresql.org/gitweb/?p=pgadmin4.git;a=blob;f=web/pgadmin/utils/driver/psycopg2/connection.py;h=4f11c12b30882209c308cb3558e67189c97ea31e;hb=15fe26a7106610b710f3de5b604cd038302c926a#l1363 >>>>> >>>>> Can anyone please provide some pointers, suggestions? >>>>> >>>>> -- >>>>> *Akshay Joshi* >>>>> >>>>> *Sr. Software Architect * >>>>> >>>>> >>>>> >>>>> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246* >>>>> >>>> >>>> >>>> -- >>>> *Akshay Joshi* >>>> >>>> *Sr. Software Architect * >>>> >>>> >>>> >>>> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246* >>>> >>> >>> >>> -- >>> *Akshay Joshi* >>> >>> *Sr. Software Architect * >>> >>> >>> >>> *Phone: +91 20-3058-9517Mobile: +91 976-788-8246* >>> >> --000000000000510dc70579728871 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi

On Tue, 30 Oct 2018, 19:17 Daniele Varrazzo <daniele.varrazzo@gmail.com wrote:
It seems to me that the problem is= in pqpath.c pq_get_last_result() function. The loop assumes PQgetResult wo= n't block, which seems always the case with the direct connection, but = PQisBusy() actually returns 1 connecting through the tunnel.

=
So at a first read, the simplification given by pq_get_last_result() i= s broken: the results should be returned one by one in the normal loop goin= g through PQisBusy(), PQconsumeInput(), and the polling machinery. It seems= a chunky refactoring but as it is now I don't like pq_get_last_result(= ) anymore :(

=C2=A0 =C2=A0 What changes should I made to = fix this? Or it should be fixed in psycopg2 itself.
=
=

-- Daniele=C2=A0


On Tue, Oct 30, 2018 at 12:52 PM Daniel= e Varrazzo <daniele.varrazzo@gmail.com> wrote:
<= blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px= #ccc solid;padding-left:1ex">
Taking a look. Thank you for= the test.

On Tue, Oct= 30, 2018 at 12:36 PM Akshay Joshi <akshay.joshi@enterprisedb= .com> wrote:
Hi Team=C2=A0

I have created one sample application to= replicate the issue. Attached is the sample application, please read "= ;README.txt" file to proceed=C2=A0further.
Can someone pleas= e suggest why poll() function hangs when run wrong query with sshtunnel v0.= 1.4.

On Mon, Oct= 8, 2018 at 12:19 PM Akshay Joshi <akshay.joshi@enterprisedb.= com> wrote:
Hi Team=C2=A0

As one issue(https://github.com/psycopg/psycopg2/issues/781) has= already been created for conn.poll() function hangs when executes COPY com= mand in async mode. Similarly conn.poll() functions hangs when we run wrong= query in async mode using=C2=A0sshtunnel=C2=A0v0.1.3.

Can someone please suggest what went wrong when used = sshtunnel, or the pointers/reason where poll() method hangs.=C2=A0 =C2=A0= =C2=A0

On = Fri, Sep 28, 2018 at 5:43 PM Akshay Joshi <akshay.joshi@enter= prisedb.com> wrote:

Hello

I am using=C2=A0psycopg2 2.7.4=C2=A0to connect to= the PostgreSQL database server using=C2=A0asynchronoussupport. It is working absolutely = fine. I have used=C2=A0sshtunnel=C2=A0v0.1.3 in=C2=A0pgAdmin4. When I connect the PostgreSQL database serve= r using ssh tunnel and run the valid/correct query it works fine, but when = I run any wrong query(invalid column of table) my application gets hang on = conn.poll() function.

Pl= ease refer the code how we use conn.poll() with timeouts=C2=A0https://git.= postgresql.org/gitweb/?p=3Dpgadmin4.git;a=3Dblob;f=3Dweb/pgadmin/utils/driv= er/psycopg2/connection.py;h=3D4f11c12b30882209c308cb3558e67189c97ea31e;hb= =3D15fe26a7106610b710f3de5b604cd038302c926a#l1363

Can anyone please provide some pointers, sugge= stions?


--
Akshay Joshi
Sr. Software Archite= ct

=


--
Akshay Joshi
Sr. Software Ar= chitect


Phone: +91 20-3058-9517
Mobile: +91 976-788-8246=


--
Ak= shay Joshi
Sr. Software Architect
<= font color=3D"#3333FF">
=
=
Phone: +91 20-3058-9517
Mobil= e: +91 976-788-8246
--000000000000510dc70579728871--