Content-Type: text/plain;
	charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.7\))
Subject: Re: about client-side cursors
From: Christophe Pettus <xof@thebuild.com>
In-Reply-To: <20210204172135.bl2boudpyw4vfahv@dalibo.com>
Date: Thu, 4 Feb 2021 09:31:22 -0800
Cc: Daniele Varrazzo <daniele.varrazzo@gmail.com>,
 psycopg@postgresql.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <D6A97A99-3F53-43CA-A01A-B72C39C0FA84@thebuild.com>
References: <20210203161548.vkhu3qkpvmgvyhik@dalibo.com>
 <CA+mi_8a1ery2xvb+AdxMtVXydOdGC7wRMnfXNAvk8v73HsLFyw@mail.gmail.com>
 <20210204111647.4s3qie5cnassksa5@dalibo.com>
 <CA+mi_8ZhJGgZ46POUn2Bujb1e9guiampac0x+S54x05dSOXUJQ@mail.gmail.com>
 <20210204172135.bl2boudpyw4vfahv@dalibo.com>
To: Denis Laxalde <denis.laxalde@dalibo.com>
Precedence: bulk


> On Feb 4, 2021, at 09:21, Denis Laxalde <denis.laxalde@dalibo.com> =
wrote:
>=20
> Well, maybe I'm missing something... In the examples above, (written
> down explicitly to understand where IO happens), if I shut down =
postgres
> between 'await conn.execute()' and 'await cur.fetchall()', the first
> example breaks but the second doesn't. Perhaps the autocommit mention
> was misleading; it's enough to insert 'await conn.commit()' before
> 'await cur.fetchall()' to reproduce. So (and again, unless I'm missing
> something), if this is not "by design", maybe this is bug?

You're relying on private knowledge, not an API guarantee, as to "where =
I/O happens" here.  Like any expectation based on private knowledge, you =
can get tripped up by that.

We shouldn't write into the API contract that either cursor.execute() or =
cursor.fetch*() are guaranteed not to fail with an I/O error.  Any time =
you interact with the cursor object, that can be assumed to be an =
asynchronous operation that can fail due to the remote server not being =
available.  Knowing that, you can reasonably code defensively no matter =
what.

Adding those explicit guarantees about I/O to the API hugely limits =
underlying implementation changes in the future, for as far as I can =
tell no real gain.

If it comes down to "cursor isn't a good name for this class," that's =
probably true, but we're a decade past making that decision.

> As far as the async interface is concerned, I think there is no =
adoption
> issue because there's no precedent use from psycopg2. So we could
> expose two API: cursorless querying ('await conn.execute()') and have =
a
> single server-side cursor class.

Right now, switching from using just a client-side cursor object to =
server-side cursor preserves largely preserves the API.  I think that's =
a valuable feature that's worth retaining.

--
-- Christophe Pettus
   xof@thebuild.com