Date: Tue, 10 Nov 2020 21:24:11 +0000
From: Rory Campbell-Lange <rory@campbell-lange.net>
To: Daniele Varrazzo <daniele.varrazzo@gmail.com>
Cc: Christophe Pettus <xof@thebuild.com>,
	Vladimir Ryabtsev <greatvovan@gmail.com>,
	Federico Di Gregorio <fog@dndg.it>, psycopg@lists.postgresql.org
Subject: Re: psycopg3 and adaptation choices
Message-ID: <20201110212411.GA5668@campbell-lange.net>
References: 
 <CA+mi_8ZfV+rEDKaqM-pcSf43FuN4FUJGoGTxvEEXap-x_GWP0Q@mail.gmail.com>
 <88fb7a7e-a182-a816-c1a7-8a1f54b65215@aklaver.com>
 <CA+mi_8YUEPfTPqD-1Jxm6NUwgJpWxPgejRZ+2-mPQr=0DQWiHA@mail.gmail.com>
 <4830fb8d-fa57-e0f8-0e4f-a96ed040dede@dndg.it>
 <CA+mi_8a+Q0C2LwiedwNkRPVY=-G3q0zW6w19zYvvADtdEd4b_w@mail.gmail.com>
 <1546497c-ff41-9bd0-b4d7-931bd305caac@dndg.it>
 <CAMqTPqkac5xa=+kwKLj8DEr9OdrkJpEzJj4Re8VvKP=JeTdZog@mail.gmail.com>
 <CA+mi_8aRQBG+Lz12oW0Be62RSR7UEEoev5Yn34ZSygcOgxbOCw@mail.gmail.com>
 <9989BCEF-2CC4-4AA4-9E4E-C83740F21414@thebuild.com>
 <CA+mi_8aWb9avqTtm=MZcNzUVvKvvWYS7BzcuCapgA9wnNFRyGQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
 <CA+mi_8aWb9avqTtm=MZcNzUVvKvvWYS7BzcuCapgA9wnNFRyGQ@mail.gmail.com>
User-Agent: Mutt/1.10.1 (2018-07-13)
Precedence: bulk

On 10/11/20, Daniele Varrazzo (daniele.varrazzo@gmail.com) wrote:
> On Tue, 10 Nov 2020 at 03:22, Christophe Pettus <xof@thebuild.com> wrote:
> >
> > > On Nov 9, 2020, at 19:20, Daniele Varrazzo <daniele.varrazzo@gmail.com> wrote:
> > >
> > >    Choices to cast Python ``int`` type:
> >
> > Is it absurd to make the choice at execution time, based on the actual value of the Python int?
> 
> I've been thinking a lot about it. I haven't completely ruled it out,
> but there are a few cases in which having different oids for the same
> query gets in the way. One that comes to mind is with prepared
> statements, either explicit (which I haven't exposed yet, but it's
> like the #1 request for a new feature), or implicit (currently using
> them to implement 'executemany()'). However I might be overestimating
> these issues, yes.
> 
> I guess I should give an overview of the whole adaptation system: I'll
> try and write its documentation in the next few days. I have to start
> with the documentation somewhere...

Apologies for a no-doubt naive suggestion, Daniele, but how about a
postgresql type 'shim' of some sort that only accepts python types on
input and translates output back to only python types.

If such a shim, perhaps a type + C function pair were used, I assume it
would not round-trip per-se, but could cycle through int types from most
restrictive to most lenient on the basis (I'm guessing) that postgresql
will coerce a postgres int4 to and int8 on insertion if necessary, on
the principle that the following works ok:

    test=> create table a (b int8);
    CREATE TABLE
    test=> insert into a values (1::int4);
    INSERT 0 1

    test=> create table b (c numeric);
    CREATE TABLE
    test=> insert into b values (4::int8);
    INSERT 0 1

I assume mapping native postgresql column types to output values would
pass back through such a 'sieve' quite naturally into native python
types.

Whether such a filtering layer should work directly in postgresql or as
a translation (or 'adaptation') layer in psycopg[3]*is perhaps a similar
debate -- although at a lower level -- about Django's ORM. Where should
the logic lie?

By the way I believe this is this 'layer' for the golang pgx module,
which may be of interest:
https://github.com/jackc/pgx/blob/93c6b60429e13e0016665214dca2c6382982cf99/values.go#L28
although golang is of course is more strongly typed than python. I
thought the type switch test for coercion, as Christophe suggests, might
be doable through the 'shim' layer I'm imagining.

Regards
Rory