public inbox for [email protected]
help / color / mirror / Atom feedUsing composite types in psycopg3
4+ messages / 2 participants
[nested] [flat]
* Using composite types in psycopg3
@ 2020-11-10 21:59 Vladimir Ryabtsev <[email protected]>
2020-11-11 00:24 ` Re: Using composite types in psycopg3 Daniele Varrazzo <[email protected]>
0 siblings, 1 reply; 4+ messages in thread
From: Vladimir Ryabtsev @ 2020-11-10 21:59 UTC (permalink / raw)
To: psycopg
Hello,
I have tried psycopg3 very briefly and I have a question.
I have a couple of use cases in systems I currently work with
that prevented (or seriously limited) usage of psycopg2, so
I had to use other drivers. This generally relates to accessing
composite types that you can construct on-the-fly in SQL queries.
Consider the following example:
'''
import psycopg2
def get_query(fpath):
with open(fpath, 'rt') as f:
return f.read()
def main():
conn = psycopg2.connect('postgres://user:password@host/db')
cur = conn.cursor()
cur.execute(get_query('query.sql'))
result = cur.fetchone()[0]
print(type(result), result)
cur.close()
conn.close()
if __name__ == '__main__':
main()
'''
Where query.sql is:
'''
with test as (
select 1 as id, 'one' val
union all
select 1, 'one more'
union all
select 2, 'two'
)
select array(
select (id, array_agg(val))
from test
group by id
)
'''
psycopg2 returns the 'result' as a basic string, while
in asyncpg and py-postgresql I have structured data
(roughly 'List[Tuple[int, List[str]]]').
I tried the same in psycopg3 and it is little bit better, but
not entirely: it shows the outer list, the tuples inside it,
but the innermost list is still represented as a basic string:
'{one,"one more"}'.
Is it something you are still working on? Any workarounds?
Vladimir
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Using composite types in psycopg3
2020-11-10 21:59 Using composite types in psycopg3 Vladimir Ryabtsev <[email protected]>
@ 2020-11-11 00:24 ` Daniele Varrazzo <[email protected]>
2020-11-11 00:35 ` Re: Using composite types in psycopg3 Vladimir Ryabtsev <[email protected]>
0 siblings, 1 reply; 4+ messages in thread
From: Daniele Varrazzo @ 2020-11-11 00:24 UTC (permalink / raw)
To: Vladimir Ryabtsev <[email protected]>; +Cc: psycopg
On Tue, 10 Nov 2020 at 21:59, Vladimir Ryabtsev <[email protected]> wrote:
> psycopg2 returns the 'result' as a basic string, while
> in asyncpg and py-postgresql I have structured data
> (roughly 'List[Tuple[int, List[str]]]').
>
> I tried the same in psycopg3 and it is little bit better, but
> not entirely: it shows the outer list, the tuples inside it,
> but the innermost list is still represented as a basic string:
> '{one,"one more"}'.
>
> Is it something you are still working on? Any workarounds?
Yes: by obtaining data from the db in binary mode you can get
information about deeply nested objects. psycopg2 works only in text
mode, psycopg3 in both.
In [1]: query = """
...: with test as (
...: select 1 as id, 'one' val
...: union all
...: select 1, 'one more'
...: union all
...: select 2, 'two'
...: )
...: select array(
...: select (id, array_agg(val))
...: from test
...: group by id
...: )"""
In [2]: import psycopg3
In [3]: from psycopg3.pq import Format
In [4]: cnn = psycopg3.connect("")
In [5]: cnn.cursor().execute(query).fetchone()[0]
Out[5]: [('1', '{one,"one more"}'), ('2', '{two}')]
In [6]: cnn.cursor(format=Format.BINARY).execute(query).fetchone()[0]
Out[6]: [(1, ['one', 'one more']), (2, ['two'])]
Binary loading/dumping is not supported yet for all the data types,
but the plan is to cover all the builtins. Still not sure about the
interface to request text/binary results, or whether binary shouldn't
be the default as opposed to text. There is still ground to cover, but
we are getting there.
-- Daniele
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Using composite types in psycopg3
2020-11-10 21:59 Using composite types in psycopg3 Vladimir Ryabtsev <[email protected]>
2020-11-11 00:24 ` Re: Using composite types in psycopg3 Daniele Varrazzo <[email protected]>
@ 2020-11-11 00:35 ` Vladimir Ryabtsev <[email protected]>
2020-11-11 00:48 ` Re: Using composite types in psycopg3 Daniele Varrazzo <[email protected]>
0 siblings, 1 reply; 4+ messages in thread
From: Vladimir Ryabtsev @ 2020-11-11 00:35 UTC (permalink / raw)
To: Daniele Varrazzo <[email protected]>; +Cc: psycopg
Awesome, thanks.
It would be great if it worked out of the box, as in other drivers.
Appreciate your efforts.
Vladimir
On Tue, 10 Nov 2020 at 16:24, Daniele Varrazzo <[email protected]>
wrote:
> On Tue, 10 Nov 2020 at 21:59, Vladimir Ryabtsev <[email protected]>
> wrote:
>
> > psycopg2 returns the 'result' as a basic string, while
> > in asyncpg and py-postgresql I have structured data
> > (roughly 'List[Tuple[int, List[str]]]').
> >
> > I tried the same in psycopg3 and it is little bit better, but
> > not entirely: it shows the outer list, the tuples inside it,
> > but the innermost list is still represented as a basic string:
> > '{one,"one more"}'.
> >
> > Is it something you are still working on? Any workarounds?
> Yes: by obtaining data from the db in binary mode you can get
> information about deeply nested objects. psycopg2 works only in text
> mode, psycopg3 in both.
>
> In [1]: query = """
> ...: with test as (
> ...: select 1 as id, 'one' val
> ...: union all
> ...: select 1, 'one more'
> ...: union all
> ...: select 2, 'two'
> ...: )
> ...: select array(
> ...: select (id, array_agg(val))
> ...: from test
> ...: group by id
> ...: )"""
>
> In [2]: import psycopg3
>
> In [3]: from psycopg3.pq import Format
>
> In [4]: cnn = psycopg3.connect("")
>
> In [5]: cnn.cursor().execute(query).fetchone()[0]
> Out[5]: [('1', '{one,"one more"}'), ('2', '{two}')]
>
> In [6]: cnn.cursor(format=Format.BINARY).execute(query).fetchone()[0]
> Out[6]: [(1, ['one', 'one more']), (2, ['two'])]
>
> Binary loading/dumping is not supported yet for all the data types,
> but the plan is to cover all the builtins. Still not sure about the
> interface to request text/binary results, or whether binary shouldn't
> be the default as opposed to text. There is still ground to cover, but
> we are getting there.
>
> -- Daniele
>
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Using composite types in psycopg3
2020-11-10 21:59 Using composite types in psycopg3 Vladimir Ryabtsev <[email protected]>
2020-11-11 00:24 ` Re: Using composite types in psycopg3 Daniele Varrazzo <[email protected]>
2020-11-11 00:35 ` Re: Using composite types in psycopg3 Vladimir Ryabtsev <[email protected]>
@ 2020-11-11 00:48 ` Daniele Varrazzo <[email protected]>
0 siblings, 0 replies; 4+ messages in thread
From: Daniele Varrazzo @ 2020-11-11 00:48 UTC (permalink / raw)
To: Vladimir Ryabtsev <[email protected]>; +Cc: psycopg
On Wed, 11 Nov 2020 at 00:36, Vladimir Ryabtsev <[email protected]> wrote:
>
> Awesome, thanks.
>
> It would be great if it worked out of the box, as in other drivers.
I do agree.
I have much more experience with the text format, and in my
understanding not all the types support binary I/O. But my
understanding could be wrong.
After covering the binary format for the missing types (especially
numeric and date/time objects) we can probably do more testing and
check if defaulting to the binary format doesn't have unexpected
consequences.
Cheers!
-- Daniele
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2020-11-11 00:48 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-11-10 21:59 Using composite types in psycopg3 Vladimir Ryabtsev <[email protected]>
2020-11-11 00:24 ` Daniele Varrazzo <[email protected]>
2020-11-11 00:35 ` Vladimir Ryabtsev <[email protected]>
2020-11-11 00:48 ` Daniele Varrazzo <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox