MIME-Version: 1.0
References: <CAO6_XqroE7JuMEm1sWz55rp9fAYX2JwmcP_3m_v51vnOFdsLiQ@mail.gmail.com>
 <CAGECzQTjCEXpJGOq9O6Nf+p4zRwjt-b1p6kACWVNJ+uYcLGvyg@mail.gmail.com>
 <Z0gQ-DYex9dbsBWe@paquier.xyz> <CAGECzQTiCvOxxuA98Ei+qVb1px82eP-iRvJvnvrcKU1de55-Pw@mail.gmail.com>
 <CAO6_XqrseE1UGxMEHu3CARdn5HW_C4rqR3Xy=Vg2sfHjPRa1og@mail.gmail.com>
 <CAO6_Xqr10ncZPmVzn1gUT2nHu8Bd+u3SkbgF4Uw9TL8FUA5fbQ@mail.gmail.com> <CAO6_XqrDPpaHqvKA0r8kfuHBE8BZ5ZqWLwucu30cXobeGtvBMA@mail.gmail.com>
In-Reply-To: <CAO6_XqrDPpaHqvKA0r8kfuHBE8BZ5ZqWLwucu30cXobeGtvBMA@mail.gmail.com>
From: Jelte Fennema-Nio <postgres@jeltef.nl>
Date: Thu, 12 Dec 2024 00:53:27 +0100
Message-ID: <CAGECzQQfjzxhYvH9LpM-4NK=4RjOrQCETeUPeGG_7ftaNEinCA@mail.gmail.com>
Subject: Re: Add Pipelining support in psql
To: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Cc: Michael Paquier <michael@paquier.xyz>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://www.postgresql.org/message-id/CAGECzQQfjzxhYvH9LpM-4NK%3D4RjOrQCETeUPeGG_7ftaNEinCA%40mail.gmail.com>
Precedence: bulk

On Tue, 10 Dec 2024 at 11:44, Anthonin Bonnefoy
<anthonin.bonnefoy@datadoghq.com> wrote:
> num_queries (2nd element in the pipeline status prompt) is now used to
> track queued queries that were not flushed (with a flush request or
> sync) to the server. It used to count both unflushed queries and
> flushed queries.

I skimmed the code a bit, but haven't looked closely at it yet. I did
try the patch out though. My thoughts below:

I think that new prompt is super useful, so useful in fact that I'd
suggest linking to it from the \startpipeline docs. I do think that
the wording in the docs could be a bit more precise:
1. The columns are not necessarily queries, they are messages or
commands. i.e. \parse and \bind_named both count as 1, even though
they form one query together.
2. messages not followed by \sync and \flushrequest, could very well
already "be sent to the server" (if the client buffer got full, or in
case of manual \flush). The main thing that \sync and \flushrequest do
is make sure that the server actually sends its own result back, even
if its buffer is not full yet.

The main feedback I have after playing around with this version is
that I'd like to have a \getresult (without the s), to only get a
single result. So that you can get results one by one, possibly
interleaved with some other queries again.

One thing I'm wondering is how useful the num_syncs count is in the
pipeline prompt, since you never really wait for a sync.

Regarding the usefulness of \flush. I agree it's not as useful as I
thought, because indeed \getresults already flushes everything. But
it's not completely useless either. The main way I was able to use it
interactively in a somewhat interesting way was to send it after a
long running query, and then while that's processing type up the next
query after it. Something like the following:

localhost jelte@postgres:5432-26274=3D
#> \startpipeline
Time: 0.000 ms
localhost jelte@postgres:5432-26274=3D
#|0,0,0|> select pg_sleep(5) \bind \g
Time: 0.000 ms
localhost jelte@postgres:5432-26274=3D
#|0,1,0|*> \flush
Time: 0.000 ms
localhost jelte@postgres:5432-26274=3D
#|0,1,0|*> select 1 \bind \g
Time: 0.000 ms
localhost jelte@postgres:5432-26274=3D
#|0,2,0|*> \syncpipeline
Time: 0.000 ms
localhost jelte@postgres:5432-26274=3D
#|1,0,2|*> \getresults
 pg_sleep
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80

(1 row)

 ?column?
=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=
=94=80=E2=94=80
        1
(1 row)

Time: 0.348 ms