Feedback-ID: i0fe9450f:Fastmail
Date: Tue, 4 Mar 2025 09:56:55 +0900
From: Michael Paquier <michael@paquier.xyz>
To: Daniel Verite <daniel@manitou-mail.org>
Cc: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>,
	Jelte Fennema-Nio <postgres@jeltef.nl>,
	PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Subject: Re: Add Pipelining support in psql
Message-ID: <Z8ZP17bZfwxFV_xJ@paquier.xyz>
References: <CAO6_XqrrjJRhKNhYa9Cg=_BSH8MmS1tAHMFq-H7fH=NcWJ-jvw@mail.gmail.com>
 <ad4b9f1a-f7fe-4ab8-8546-90754726d0be@manitou-mail.org>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="O8MxNDXE8PBhxi6m"
Content-Disposition: inline
In-Reply-To: <ad4b9f1a-f7fe-4ab8-8546-90754726d0be@manitou-mail.org>
Archived-At: <https://www.postgresql.org/message-id/Z8ZP17bZfwxFV_xJ%40paquier.xyz>
Precedence: bulk


--O8MxNDXE8PBhxi6m
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

(My previous message did not reach the lists, so re-sending with some
edits.)

On Fri, Feb 28, 2025 at 05:31:00PM +0100, Daniel Verite wrote:
> The following invocations don't respect the desired output destination
> and formats (ignoring them), when a pipeline is active:
>=20
> select ... \bind \g filename
> select ... \bind \g |program
> select ... \bind \g (format=3Dunaligned tuples_only=3Don)
>=20
> Just like for \gx the problem is that in a pipeline, sending the query
> is not followed by getting the results, and the output properties
> of a query are lost in between.

Right.  I completely forgot that these options could be applied with a
simple \g.  With the results being decoupled from the execution, one
can argue that the options defined at the moment when the query is
sent and executed should be the moment commanding how the result are
shaped when retrieving a batch with \getresult during a pipeline.
However, this means that we would need to save the set of options from
printQueryOpt when running the query depending on the number of
results we expect, and reapply them in order of the results
expected. We have the APIs to do that, with savePsetInfo() and
restorePsetInfo().

Anyway, can we really say that the set of printQueryOpt saved at
execution is the correct set to use?  It is possible to have the
opposite argument and say that we should just apply the printQueryOpt
at the moment where \getresult is run.  A benefit of that it to keep
the loop retrieving results simpler in ExecQueryAndProcessResults(),
because we pass down to this call *one* printQueryOpt as "opt".=20

There's of course the simplicity argument that I do like a lot here,
but applying the printing options at the time of \getresult feels also
more natural to me.

FWIW, I agree that HEAD is unbalanced with its handling of \gx, so we
could do one of the following two things:
1) Ignore any formatting options given to \g, but also allow \gx to
run, documenting that during a pipeline the formatting options are
ignored, and that the set of options defined when doing a \getresult
is the only thing that matters.
2) Invent a new meta-command (suggested by Daniel Verite off-list, and
not published to the lists because I don't know how to do a
reply-all), like a \send, a \push, forbiding entirely \gx and \g when
in a pipeline.  If we were to integrate options into this new
meta-command, a split with \g would make an integration easier to
think about.  One name suggestion I can come up is \sendpipeline.

I'd consider option 2, based on Daniel's concerns.  One line I'm going
to draw here is that we should not go down to manipulations of
printQueryOpt while retrieving batch of results depending on the style
of output that was defined when sending a query in a pipeline.

Anthonin, as the primary author, any thoughts?
--
Michael

--O8MxNDXE8PBhxi6m
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAmfGT9YACgkQnvQgOdby
QH0kYBAAhUGT/FykqU2xPqR4fQ0EYfUPReBqB3GeIAiJV3gRi85P8ufi2LS8l1tc
HnBj9bfLtxN73Wz2zpwvrne7tZ1zeoD/QAnmnhFEWzPbPKjZGppq+ksHcV+a/mql
6dmIh2X7ZemJPwVoBjEOiqyBdv9XooT2KWOosFJdinf3B/V2VeNH102T2urSUmRe
rdcCHkyfiPDczCu0Ir3+Vh/idCqOSn+Ix3Id0Oex13fVCau7HZMVcn9GgT1ildbH
aAXjKuZnua+VdrWKOBfyuWQVm1CqGiRWhR2aBONtvnWhOY//yBCnKaAT7Qh4jR0x
vwNt3Yk00tdQnjqgCnNhzVfGQYvyR6KM5CJUXCD0pRuAXefj2J8AJ7SYh7RYmQMx
htekRM0PlgKS8edLEwa1s/vx2VimcLWC69tmIH1PKewf7Elti5HkV20Byk1jhmdm
Pp19uSokZ4ri0q/8wYmwcwqeZeYqdcBLhv4MjmNSiNB+sKNpzagu7Xn0+XA+OgS8
RsL+lmseYV94i1bJk1hwqYVtuHtArwBGGBEQ1crGEOXVy6gNIJN1EdpQVGwC4Gau
8qty6d2+ZW935BcivcIfcAQ+A4TgcKkem+a4SB4oy83mKcKml4taH3edDI1gpdqO
+3gIOGzx/l5qPD7N6vxx1VEui79x05/IK9pMJJK7okSpR7LL0mg=
=A19p
-----END PGP SIGNATURE-----

--O8MxNDXE8PBhxi6m--