Subject: Re: about client-side cursors
To: Christophe Pettus <xof@thebuild.com>
Cc: Daniele Varrazzo <daniele.varrazzo@gmail.com>, psycopg@postgresql.org
References: <20210203161548.vkhu3qkpvmgvyhik@dalibo.com>
 <CA+mi_8a1ery2xvb+AdxMtVXydOdGC7wRMnfXNAvk8v73HsLFyw@mail.gmail.com>
 <20210204111647.4s3qie5cnassksa5@dalibo.com>
 <CA+mi_8ZhJGgZ46POUn2Bujb1e9guiampac0x+S54x05dSOXUJQ@mail.gmail.com>
 <20210204172135.bl2boudpyw4vfahv@dalibo.com>
 <D6A97A99-3F53-43CA-A01A-B72C39C0FA84@thebuild.com>
From: Denis Laxalde <denis.laxalde@dalibo.com>
Message-ID: <e1db73d8-4206-bcba-171b-80bb5cd4c7ad@dalibo.com>
Date: Fri, 5 Feb 2021 10:04:15 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.6.0
MIME-Version: 1.0
In-Reply-To: <D6A97A99-3F53-43CA-A01A-B72C39C0FA84@thebuild.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Precedence: bulk

Christophe Pettus a écrit :
>> On Feb 4, 2021, at 09:21, Denis Laxalde <denis.laxalde@dalibo.com> wrote:
>>
>> Well, maybe I'm missing something... In the examples above, (written
>> down explicitly to understand where IO happens), if I shut down postgres
>> between 'await conn.execute()' and 'await cur.fetchall()', the first
>> example breaks but the second doesn't. Perhaps the autocommit mention
>> was misleading; it's enough to insert 'await conn.commit()' before
>> 'await cur.fetchall()' to reproduce. So (and again, unless I'm missing
>> something), if this is not "by design", maybe this is bug?
> 
> You're relying on private knowledge, not an API guarantee, as to "where I/O happens" here.  Like any expectation based on private knowledge, you can get tripped up by that.

Well, that's just an exercise for me to understand where things happen. 
Of course, I'd expect an 'await' expression to possibly involve I/O in 
real life.


> If it comes down to "cursor isn't a good name for this class," that's probably true, but we're a decade past making that decision.

That, and the fact that fetch*() methods do not fetch actually. (For the 
synchronous case, I'm quite convinced now this is too late to change.)

>> As far as the async interface is concerned, I think there is no adoption
>> issue because there's no precedent use from psycopg2. So we could
>> expose two API: cursorless querying ('await conn.execute()') and have a
>> single server-side cursor class.
> 
> Right now, switching from using just a client-side cursor object to server-side cursor preserves largely preserves the API.  I think that's a valuable feature that's worth retaining.

Worth retaining, maybe. Is it worth imposing the 'await cursor.fetch*()' 
pattern to everybody when it does not appear needed? I'm not so sure, as 
this would be at the cost of readability and clarity. (On the other 
hand, if I were to migrate some code to use server-side cursors, I would 
probably do it on a case-by-case basis; so if the cursor wasn't there in 
the first place, adding it would be little trouble in contrast to 
deciding where a server-side cursor is needed or not.)