public inbox for [email protected]
help / color / mirror / Atom feedFrom: Akshay Joshi <[email protected]>
To: Daniele Varrazzo <[email protected]>
Cc: [email protected]
Subject: Re: psycopg2 conn.poll() hangs when used with sshtunnel
Date: Tue, 30 Oct 2018 21:27:39 +0530
Message-ID: <CANxoLDccOOqJk2Q9--SnOhFpaV+r7Aq2euo5-hu-i7D-aeFFpg@mail.gmail.com> (raw)
In-Reply-To: <CANxoLDemxzjQitt_4K_3-Ear08898ug7bryHz0eYyWas5j_xxw@mail.gmail.com>
References: <CANxoLDc4AosChXUhArLh2ufdon1BRwhCNRgGjyeCsYf9vHTmTw@mail.gmail.com>
<CANxoLDf3ra5UaqFw_6A824_hAi5XPykySr+DJre2ZbprbxTCVA@mail.gmail.com>
<CANxoLDesBkV9=qpg22-9nG=JYgvUuo3Xm+iDxSWwLVReTiEC7g@mail.gmail.com>
<CA+mi_8ZBYkPm7Kou8smEnDfXPPjVByw7o3M-nY3PvVEK_TN9nA@mail.gmail.com>
<CA+mi_8boc=n4jZnX+20-UYwZYd4wse+wrnZWf6zG-1nMhDLmnA@mail.gmail.com>
<CANxoLDfDw=9oAePwZk0Su9PLB3ni_NQdJ5Lj1k21GzYe07sY8Q@mail.gmail.com>
<CA+mi_8ZfYM9Mqp18z7cTC5Hh8TEPxS+LrotTmWTnS3=wmcMJnA@mail.gmail.com>
<CANxoLDemxzjQitt_4K_3-Ear08898ug7bryHz0eYyWas5j_xxw@mail.gmail.com>
I have one more observation when i run this application one out of 10 times
it works. I have logged one more issue as COPY command hangs, so thought it
is the problem in psycopg2
On Tue, 30 Oct 2018, 21:22 Akshay Joshi <[email protected]
wrote:
>
>
> On Tue, 30 Oct 2018, 21:06 Daniele Varrazzo <[email protected]
> wrote:
>
>> On Tue, Oct 30, 2018 at 1:55 PM Akshay Joshi <
>> [email protected]> wrote:
>>
>>
>>> What changes should I made to fix this? Or it should be fixed in
>>> psycopg2 itself.
>>>
>>
>> Actually, I have another observation. I was surprised to see that the
>> whole test suite passed running the connection through an ssh tunnel: there
>> are definitely failing queries in the test suite. So I made other tests and
>> it seems the bad condition only happens using the `sshtunnel` module: If I
>> open a tunnel manually with something like:
>>
>> ssh -L 36421:localhost:5432 -N localhost
>>
>> and point your script to port 36421 everything works ok.
>>
>>
>> Conversely, I don't seem able to run the test suite through the tunnel
>> open by the sshtunnel module. It doesn't hang, but the tunnel breaks in a
>> test with the following reported on the sshtunnel side:
>>
>> 2018-10-30 15:24:25,550| ERROR | Socket exception: Bad file
>> descriptor (9)
>> 2018-10-30 15:24:25,551| ERROR | Could not establish connection
>> from ('127.0.0.1', 33743) to remote side of the tunnel
>>
>> and the following tests fail to run as the connection is broken.
>>
>>
>> Con-conversely, the patch I had in mind to fix #801, which very brutally
>> is:
>>
>> ```
>> @@ -1136,6 +1136,13 @@ pq_get_last_result(connectionObject *conn)
>> || status == PGRES_COPY_IN) {
>> break;
>> }
>> + if (PQisBusy(conn->pgconn)) {
>> + /* This happens connecting through ssl tunnel
>> + * TODO: just kill this function. The loop should happen
>> within
>> + * the async/green machinery. */
>> + Dprintf("pq_get_last_result: we are busy");
>> + break;
>> + }
>> }
>>
>> return result;
>> ```
>>
>> doesn't work, and failing queries leave the connection in an inconsistent
>> state.
>>
>>
>> So, wrapping up, I think there is something you can do on your side,
>> checking why the sshtunnel module behaves differently from a normal ssh
>> tunnel, and if you use the latter (or configure sshtunnel to behave so) you
>> shouldn't hit the problem. On our side I don't think we can fix #801 with a
>> quick band aid, and we should rather do #802, but as things stands now I
>> don't trust sshtunnel to do the right thing, and have less of an urgency to
>> do so.
>>
>> Please let us know if you understand what is the difference between the
>> module and the tunnel via `ssh -L`, thank you!
>>
>
> I have send the sample application and created an issue to sshtunnel
> git hub. Not sure where is the problem, as poll() function hangs so i
> thought its issue in psycopg2.
>
>>
>> -- Daniele
>>
>>>
view thread (11+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected]
Subject: Re: psycopg2 conn.poll() hangs when used with sshtunnel
In-Reply-To: <CANxoLDccOOqJk2Q9--SnOhFpaV+r7Aq2euo5-hu-i7D-aeFFpg@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox