public inbox for [email protected]  

help / color / mirror / Atom feed

From: Daniele Varrazzo <[email protected]>
To: [email protected]
Subject: libpq pipeline/batch mode
Date: Sun, 7 Feb 2021 16:50:39 +0100
Message-ID: <CA+mi_8YkA=c6cDSRPtdDfaTjZHr48vh6Hqkz=iYuUpD6g-egyA@mail.gmail.com> (raw)

Hello,

I am pleased to see that the pipeline/batch mode is progressing and it
might be released in pg14: https://commitfest.postgresql.org/30/2724/

psycopg3 uses only async functions, even for seemingly blocking calls,
so I think it is in a good position to make use of it. I don't know
yet if it should be a special operation mode, or the normal way to
work, replacing the lock to coordinate cursor interaction with the
connection with a pair of send/receive queues, if we could make
execute only send and fetch receive (ending up giving the async fetch
a proper I/O use, to reconnect to the recent conversation about it).

However I am getting stretched pretty thinly with different directions
of work: main tasks in my queue are:

- connection pool
- binary numeric/datetime adapters
- server-side cursors
- jsonb binary protocol (something to design server-side too)

so I doubt I will be able to hack on a branch of psycopg and a branch
of the libpq to study and design the pipeline possibilities too.

If anyone would like to take up this research task, or even would like
to collaborate on any other task listed above (or on the project
board: https://github.com/psycopg/psycopg3/projects/1) I would be
grateful!

Cheers!

-- Daniele

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected]
  Subject: Re: libpq pipeline/batch mode
  In-Reply-To: <CA+mi_8YkA=c6cDSRPtdDfaTjZHr48vh6Hqkz=iYuUpD6g-egyA@mail.gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox