Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1gHUzO-00050T-4B for psycopg@arkaria.postgresql.org; Tue, 30 Oct 2018 14:27:26 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1gHUzL-00070d-WB for psycopg@arkaria.postgresql.org; Tue, 30 Oct 2018 14:27:23 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1gHUyu-0006CM-94 for psycopg@lists.postgresql.org; Tue, 30 Oct 2018 14:26:56 +0000 Received: from mail-lf1-x135.google.com ([2a00:1450:4864:20::135]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1gHUyr-0000F1-2N for psycopg@postgresql.org; Tue, 30 Oct 2018 14:26:55 +0000 Received: by mail-lf1-x135.google.com with SMTP id p86so6810680lfg.5 for ; Tue, 30 Oct 2018 07:26:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=J030zBLCkveokVByCVik1r+7vKVYkZY44OB87Z8GFNQ=; b=O2O1obCh+2/bbpA66K44qNvCifbzevlk/N7tKpd/HD9jJZS81TqOqnhGSQGcNrGW0Q hf2WasQ0WKFsE25mUhhjpCP8vYC/b5NPZrHA9qqqKDvvafEzlDu9Y+9JvMyo2Drrpowz LFjwtnjlWCD6iV/C+cIxWiruJrbZoMCirdMwK4z7MmbLeF2CwqCIC4peDOwLe3O/EEBl wsrQGFtmPsfLKRjlrD1to/SfX2GmC1vF5aEnFh30JBc7Ml0aJjG0D1vcO1XbVEhCZEzB Jaey/MfzxYq9X4FHVuGHQDDkp3xti6n719bCXsPoHuvqkmSUJg2/APkUMLtLK1lRsH6X 7YFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=J030zBLCkveokVByCVik1r+7vKVYkZY44OB87Z8GFNQ=; b=taV/+zaS1A0PUkPYWot/54ba+9aAdeuBfHX2q9mTVzsRObfSf4Q+3yW0zX5wQMIBss Y7JnNo/hG9IXcsAM6sWPBo25F3A3LYq8DZ1W8vdmlfVbvDR6EI2ZXUHlLgqbVRWs5JWC ajWsLtYzALOUOv7NcAnXABGt2JoKeoGWL4cwz8uxHVtw8XDqgt4tvLH5/A5qb31SZPw/ 2UW3WFsTsit15bF8qxXIsz4Buo5xwHVDHJZxMcVP+YK7411+3Ga+BS0xeDwESvM9TCKw Zej6NQ9y7ktXg2YGL/06/pOweh9RYGNnmneSZ9AF9vjceiv8QDa4FupiVjBFBKpH9n07 ysOw== X-Gm-Message-State: AGRZ1gIdiWYqUDTxmhJiygbrVJeRsu3zBgFmF3iUo4ljESRK2Mu7tH0M cH/lgeatI5I9Q/mcnktt/4wmcpLsBjXxBt3QeBc= X-Google-Smtp-Source: AJdET5cV89IBgCZ0giUa54tEQJD/j5OMeJf1i5xaYKxiAF79ISCksqVLE+s6oFmARlUz+vCcnO8E45LVR1s2TF8FwW4= X-Received: by 2002:a19:690d:: with SMTP id e13mr2023105lfc.84.1540909610728; Tue, 30 Oct 2018 07:26:50 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Daniele Varrazzo Date: Tue, 30 Oct 2018 14:26:38 +0000 Message-ID: Subject: Re: psycopg2 conn.poll() hangs when used with sshtunnel To: Akshay Joshi Cc: psycopg@postgresql.org Content-Type: multipart/alternative; boundary="000000000000fa8532057972f9ea" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --000000000000fa8532057972f9ea Content-Type: text/plain; charset="UTF-8" On Tue, Oct 30, 2018 at 1:55 PM Akshay Joshi wrote: > Hi > > On Tue, 30 Oct 2018, 19:17 Daniele Varrazzo wrote: > >> It seems to me that the problem is in pqpath.c pq_get_last_result() >> function. The loop assumes PQgetResult won't block, which seems always the >> case with the direct connection, but PQisBusy() actually returns 1 >> connecting through the tunnel. >> >> So at a first read, the simplification given by pq_get_last_result() is >> broken: the results should be returned one by one in the normal loop going >> through PQisBusy(), PQconsumeInput(), and the polling machinery. It seems a >> chunky refactoring but as it is now I don't like pq_get_last_result() >> anymore :( >> > > What changes should I made to fix this? Or it should be fixed in > psycopg2 itself. > It is a problem in psycopg. I have a quick fix I can release with the next bugfix if it shows no regression ( https://github.com/psycopg/psycopg2/issues/801). A better fix requires a non trivial rewrite of the async and green paths ( https://github.com/psycopg/psycopg2/issues/802): I'd like to do it but it may happen later if the band aid holds. I was just about to prepare the 2.7.6 release, so if the quick fix is easy and doesn't cause regressions it will be released soon. -- Daniele --000000000000fa8532057972f9ea Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Tue, Oct 30, 2018 at 1:55 PM Akshay Joshi <akshay.joshi@enterprisedb.com> wrote:
Hi

= It seems to me that the problem is in pqpath.c pq_get_last_result() functio= n. The loop assumes PQgetResult won't block, which seems always the cas= e with the direct connection, but PQisBusy() actually returns 1 connecting = through the tunnel.

So at a first read, the simplificati= on given by pq_get_last_result() is broken: the results should be returned = one by one in the normal loop going through PQisBusy(), PQconsumeInput(), a= nd the polling machinery. It seems a chunky refactoring but as it is now I = don't like pq_get_last_result() anymore :(

=C2=A0 =C2= =A0 What changes should I made to fix this? Or it should be fixed in psycop= g2 itself.

It is a problem in p= sycopg. I have a quick fix I can release with the next bugfix if it shows n= o regression (ht= tps://github.com/psycopg/psycopg2/issues/801). A better fix requires a = non trivial rewrite of the async and green paths (https://github.com/psycopg/psycopg2/issue= s/802): I'd like to do it but it may happen later if the band aid h= olds.

I was just about to prepare the 2.7.6 releas= e, so if the quick fix is easy and doesn't cause regressions it will be= released soon.

-- Daniele

--000000000000fa8532057972f9ea--