Subject: Fwd: Re: Integration of Psycopg with XTA

From: Federico Di Gregorio <[email-masked]>
Date: Thu, 4 Oct 2018 12:36:42 +0200

On 10/04/2018 12:30 PM, Daniele Varrazzo wrote:
 > On Thu, Oct 4, 2018 at 11:24 AM Federico Di Gregorio<[email-masked]>  wrote:
 >> On 10/04/2018 12:13 PM, Daniele Varrazzo wrote:
 >>> On Thu, Oct 4, 2018 at 10:33 AM Federico Di Gregorio<[email-masked]> 
wrote:
 >>>> @Daniele, I see that a capsule has an implicit state, "invalid" if the
 >>>> pointer inside it is null. Does it makes sense to reuse the same 
capsule
 >>>> and keep a reference to it to be able to invalidate it when the
 >>>> connection is closed?
 >>> I can take a look at it but I suspect it's a lot of work for a feature
 >>> very rarely used.
 >> Nah, I can do it. I was just asking for your opinion. :D
 > I don't know much about the capsule object, its life cycle etc. Yes,
 > not a huge deal saving it in the connection state and returning always
 > the same. I just wonder if it's worth the effort.

I suppose it helps to debug the situation where the capsule is used 
after the connection has been closed. Instead of a BOOM you get a nice 
exception (if the consumer checks for it, obviously).

I'll work on this in my repo and if everything works we can integrate it 
for 2.8.

federico

-- 
Federico Di Gregorio                         [email-masked]
DNDG srl                                                  http://dndg.it
   Gli avvoltoi cinesi si nutrono di arte, ma possono anche mangiare
    i `domani'.                                        -- Haruki Murakami

---

From: Christian Ferrari <[email-masked]>
Date: Thu, 4 Oct 2018 20:33:31 +0000 (UTC)

Dear Federico,I will test it as soon as it will be released on git.Kind RegardsCh.F.

-------------------------------------------------------------
Good design can't stop bad implementation

    Il giovedì 4 ottobre 2018, 12:36:53 CEST, Federico Di Gregorio <[email-masked]> ha scritto:  
 
capsule

federico

---

From: Christian Ferrari <[email-masked]>
Date: Fri, 5 Oct 2018 16:02:58 +0000 (UTC)

In a previous mail I stated something like "the name of the method doesn't matter"...... after better thinking I realized a decent name might be better than other ones...Let me say:
get_native_conn()get_native_connection()
might be better than "get_something_pgconn" because the same type of method could be useful for other databases in the future...More or less all the databases use a "native connection", only PostgreSQL uses "PGconn connection".

Have a good w.e.Ch.F.

 
capsule

federico

---

From: Federico Di Gregorio <[email-masked]>
Date: Sun, 7 Oct 2018 13:56:37 +0200

On 10/05/2018 06:02 PM, Christian Ferrari wrote:
> In a previous mail I stated something like "the name of the method 
> doesn't matter"...
> ... after better thinking I realized a decent name might be better than 
> other ones...
> Let me say:
> 
> get_native_conn()
> get_native_connection()
> 
> might be better than "get_/something/_pgconn" because the same type of 
> method could be useful for other databases in the future...
> More or less all the databases use a "native connection", only 
> PostgreSQL uses "PGconn connection".

After some more checking I discovered that it isn't possible to 
"invalidate" a PyCapsule so there is no way to tell if the pointer 
inside it is still a valid libpq connection.

The current code (feel free to pull) adds a new method

get_native_connection()

that raise an exception if called on a closed connection. I better like 
a method here because it gives the idea that you need to call it every 
time you want a Capsule to pass to the target C API.

I also added some docs and a dumb test.

https://github.com/fogzot/psycopg2/tree/feature-expose-pgconn

@Daniele, if Christian reports no problems I'll merge this later for 2.8.

federico

-- 
Federico Di Gregorio                         [email-masked]
DNDG srl                                                  http://dndg.it
  Io non sono romantica. La candelina sul tavolo mi vede e si spegne.
                                                       -- sisterconfusion

---

From: Christian Ferrari <[email-masked]>
Date: Sun, 7 Oct 2018 16:42:10 +0000 (UTC)

Hi Federico,I've just tryed out commit https://github.com/fogzot/psycopg2/commit/81addddaee2c690e925bb8f381e7bcb02ca97687 : get_native_connection() is in the interface, but unfortunately it seems to work differently than the old _raw_pgconn.
Here's my code snippet:
pgconn = rm1.get_native_connection()#pgconn = rm1._raw_pgconn                                                       
xar1 = PostgresqlXaResource(pgconn, "PostgreSQL", "dbname=testdb")

using get_native_connection(), I catch an error from XTA, the PGconn pointer is null.
I have reversed back to commit https://github.com/fogzot/psycopg2/commit/ccae5cae34051d6640ec6cbce6d9313778c509d3 : _raw_pgconn works as before.
Here's the code snippet:
#pgconn = rm1.get_native_connection()                                           
pgconn = rm1._raw_pgconn
xar1 = PostgresqlXaResource(pgconn, "PostgreSQL", "dbname=testdb")

It works fine.
Should "get_native_connection()" used differently than "_raw_pgconn" ? What type of data are necessary to investigate the issue?
Thanks in advanceKind RegardsCh.F.

    Il domenica 7 ottobre 2018, 13:56:49 CEST, Federico Di Gregorio <[email-masked]> ha scritto:  

get_native_connection()

federico

---

From: Christian Ferrari <[email-masked]>
Date: Sun, 7 Oct 2018 16:53:15 +0000 (UTC)

Sorry for disturbing with my previous email... :(It was my fault: I didn't remember to check the key associated to the pointer in the capsule, now it's "psycopg2.connection.native_connection"...
Everything works as expected, get_native_connection() can be even inlined like below:
xar1 = PostgresqlXaResource(rm1.get_native_connection(), "PostgreSQL", "dbname=testdb")

Thank you so much for your support.Kind RegardsCh.F.

   Il domenica 7 ottobre 2018, 18:42:31 CEST, Christian Ferrari <[email-masked]> ha scritto:  

It works fine.

 

get_native_connection()

federico

---

From: Daniele Varrazzo <[email-masked]>
Date: Mon, 8 Oct 2018 16:08:41 +0100

> @Daniele, if Christian reports no problems I'll merge this later for 2.8.

LGTM. Do we also want a cursor.get_native_result() to return its PGresult?

-- Daniele

---

From: Christian Ferrari <[email-masked]>
Date: Mon, 8 Oct 2018 19:51:25 +0000 (UTC)

> LGTM. Do we also want a cursor.get_native_result() to return its PGresult?

I have no idea about a use case that can use cursor.get_native_result() ......but who can tell?!

Kind RegardsCh.F.

---

From: Christian Ferrari <[email-masked]>
Date: Wed, 10 Oct 2018 21:10:55 +0000 (UTC)

By the way, PyMySQL/mysqlclient-python project asked me to rename the method from "get_native_conn()" to "_get_native_conn()".
I'm not a "naming addicted person", but from the developer point of view, using two different methods would be for sure not appreciated... 

xar1 = PostgresqlXaResource(rm1.get_native_connection(), "PostgreSQL", "dbname=testdb")
xar2 = MysqlXaResource(rm2._get_native_connection(), "MySQL", "localhost,0,lixa,,lixa")

If you don't mind, I kindly ask you to wait a couple of days: I hope to close the pull request https://github.com/PyMySQL/mysqlclient-python/pull/269 shortly...

Thank you in advanceKind RegardsCh.F.

 

get_native_connection()

federico

---

From: Federico Di Gregorio <[email-masked]>
Date: Wed, 10 Oct 2018 23:19:45 +0200

On 10/10/2018 11:10 PM, Christian Ferrari wrote:
> By the way, PyMySQL/mysqlclient-python project 
> <https://github.com/PyMySQL/mysqlclient-python> asked me to rename the 
> method from "get_native_conn()" to "_get_native_conn()".
> 
> I'm not a "naming addicted person", but from the developer point of 
> view, using two different methods would be for sure not appreciated...
> 
> If you don't mind, I kindly ask you to wait a couple of days: I hope to 
> close the pull request 
> https://github.com/PyMySQL/mysqlclient-python/pull/269 shortly...

I think that _if_ this should go cross-database a method without the 
underscore is better. The underscore usually means private and that's 
good as long as it is just that. An informal "standard" method name 
should not have the "_" in it.

federico

-- 
Federico Di Gregorio                         [email-masked]
DNDG srl                                                  http://dndg.it
   Gli esseri umani, a volte, sono destinati, per il solo fatto di
    esistere, a fare del male a qualcuno.              -- Haruki Murakami

---

From: Daniele Varrazzo <[email-masked]>
Date: Thu, 11 Oct 2018 19:46:33 +0100

> should not have the "_" in it.

+1

-- Daniele