Feedback-ID: i0fe9450f:Fastmail
Date: Wed, 29 Mar 2023 15:59:20 +0900
From: Michael Paquier <michael@paquier.xyz>
To: "Imseih (AWS), Sami" <simseih@amazon.com>
Cc: "Drouvot, Bertrand" <bertranddrouvot.pg@gmail.com>,
	David Zhang <david.zhang@highgo.ca>,
	"pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Subject: Re: [BUG] pg_stat_statements and extended query protocol
Message-ID: <ZCPhyBISjW2OMgrm@paquier.xyz>
References: <ZA0/oM2k2UUOpkLw@paquier.xyz>
 <a9a26eb0-f8c3-0fbd-b7aa-f789da39d189@highgo.ca>
 <EC5F1919-6777-44D2-BB32-87DA503EF614@amazon.com>
 <ZBkCrq05xzlZKkAz@paquier.xyz>
 <3227719C-F7A3-4A94-B0A5-D27673316560@amazon.com>
 <0d583e2a-8214-4975-a4b4-7e9e0b8f8e9c@gmail.com>
 <DF311964-77CB-4027-B00A-0AD7A3DB61D5@amazon.com>
 <472cfda7-1e4f-ed47-0860-78cf685b53b0@gmail.com>
 <ZBwaEbdpVYJPXjT5@paquier.xyz>
 <8FFCD4F0-8B04-4EA2-9190-FEC70C376523@amazon.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="w01Op4GzcqXdEksg"
Content-Disposition: inline
In-Reply-To: <8FFCD4F0-8B04-4EA2-9190-FEC70C376523@amazon.com>
Archived-At: <https://www.postgresql.org/message-id/ZCPhyBISjW2OMgrm%40paquier.xyz>
Precedence: bulk


--w01Op4GzcqXdEksg
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Thu, Mar 23, 2023 at 01:54:05PM +0000, Imseih (AWS), Sami wrote:
> Yes, that is possible but we will need to add a libpq API
> that allows the caller to pass in a "fetch size".
> PQsendQueryParams does not take in a "fetch size",=20
> so it returns all rows, through PQsendQueryParams
>=20
> https://github.com/postgres/postgres/blob/master/src/interfaces/libpq/fe-=
exec.c#L1882
>=20
> Adding such an API that takes in a "fetch size" will be beneficial=20
> not just for this test, but I can see it enabling another psql meta comma=
nd,
> similar to \bind but that takes in a "fetch size".

So...  The idea here is to set a custom fetch size so as the number of
calls can be deterministic in the tests, still more than 1 for the
tests we'd have.  And your point is that libpq enforces always 0 when
sending the EXECUTE message causing it to always return all the rows
for any caller of PQsendQueryGuts().

The extended protocol allows that, so you would like a libpq API to
have more control of what we send with EXECUTE:
https://www.postgresql.org/docs/current/protocol-overview.html#PROTOCOL-QUE=
RY-CONCEPTS

The extended query protocol would require multiple 'E' messages, but
we would not need multiple describe or bind messages, meaning that
this cannot just be an extra flavor PQsendQueryParams().  Am I gettig
that right?  The correct API design seems tricky, to say the least.
Perhaps requiring this much extra work in libpq for the purpose of
having some tests in this thread is not a brilliant idea..  Or perhaps
we could just do it and have something a-la-JDBC with two routines?
That would be one libpq routine for describe/bind and one for execute
where the limit can be given by the caller in the latter case, similar
to sendDescribeStatement() and sendExecute() in
QueryExecutorImpl.java.
--
Michael

--w01Op4GzcqXdEksg
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAmQj4cgACgkQnvQgOdby
QH33HA//eBeOj3MDCm6P/oJIr4FotfBvQsJtWE9mBbqruUUhlAHRenR6vUidbJhv
vTcSnY6Uy0Tgd/njK9/hKff/ULcRGPb60J9/JFe43TtSynZro4u2hONDpUtZhXLU
1OMIIKCey5HLiqlHebc9MmCvzX4zsos9JpIjbN/RH1WCJDB/QGp/KUTQ8AnEgjOT
eoDdo4dF/7JzaKrc+CJ4RMkRFzV145KXmyfkX+92M0QArjxrcGwir7yTZ8o0bAbu
WmovjwIS/1wVZoyqsqz1hfsD6P35LUkQVh3kIAetHTNsP1lX0ZS0cpbAimJDoXiE
2q8X2MGdKiTsIU1bOKN7xWuSFu+rgG5tJJyDdYSnKdGJlHxRrR50dZdY5K0quu+x
y4GGst16n86QqDadRfxN8xaAQaCnMWKwOMa/PL9vHxgES/7KIUVwN4SsjyaSro2L
19jAwmkA5IhRsGROvf4IMtXA/Tgs24iUxxW9X2Y/ZpkxZWSq12NaS98eZO0ETbFy
dLogtllSqkyb8h1FAhI9dw+lJGQcHhPnMrG2InXgMdcDMykdM3zZtfCJ0VQgkPgK
AfY0SPAusjgK//I/hK32wBYgKNyxXlBRB6DzNYWVuYXB1FYC5jlo5+9gDanGnSAI
nHdRu6YQpTDBYMAGC5CIQgRzOG3dDqmufcCTRTrkd39ijBXs10Q=
=8kzG
-----END PGP SIGNATURE-----

--w01Op4GzcqXdEksg--