pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feedAdd ParameterDescription message to libpq frontend long message types
4+ messages / 4 participants
[nested] [flat]
Commits from this thread:
2026-06-15 e0511883ca b1ab4bc52a e4183c667d 8d516d2b99 0f63b74a47 1b79c8d1a5 Fix PQdescribePrepared with more than 7498 params
* Add ParameterDescription message to libpq frontend long message types
@ 2026-04-01 06:32 Ning Sun <classicning@gmail.com>
0 siblings, 2 replies; 4+ messages in thread
From: Ning Sun @ 2026-04-01 06:32 UTC (permalink / raw)
To: pgsql-hackers
Hi,
I'm maintaining a Rust library pgwire to implement postgres wire
protocol in rust. While doing a corner case test, I noticed the
inconsistency for ParameterDescription between backend and frontend.
The backend allows up to 65535 parameters in a prepared statement. But
when running Describe on the statement, there is a size limit of 30000
bytes for ParameterDescription on the frontend. This means we can only
describe statements with at most ~7500 parameters. For statements exceed
the limit, it ends up with error about the message size.
This patch simply adds ParameterDescription to VALID_LONG_MESSAGE_TYPE
whitelist to remove the cap.
Attachments:
0001-add-ParameterDescription-to-long-message-types.patch (text/x-patch, 829B) download
From 6036564be819db0beb3efc1ad52c981922d52529 Mon Sep 17 00:00:00 2001
From: Ning Sun <sunning@greptime.com>
Date: Wed, 1 Apr 2026 11:53:38 +0800
Subject: [PATCH] add ParameterDescription to long message types
---
src/interfaces/libpq/fe-protocol3.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index 8c1fda5caf0..7fd49cb151e 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -42,7 +42,8 @@
(id) == PqMsg_FunctionCallResponse || \
(id) == PqMsg_NoticeResponse || \
(id) == PqMsg_NotificationResponse || \
- (id) == PqMsg_RowDescription)
+ (id) == PqMsg_RowDescription || \
+ (id) == PqMsg_ParameterDescription)
static void handleFatalError(PGconn *conn);
--
2.53.0
* Re: Add ParameterDescription message to libpq frontend long message types
@ 2026-04-02 01:13 jie wang <jugierwang@gmail.com>
parent: Ning Sun <classicning@gmail.com>
1 sibling, 1 reply; 4+ messages in thread
From: jie wang @ 2026-04-02 01:13 UTC (permalink / raw)
To: Ning Sun <classicning@gmail.com>; +Cc: pgsql-hackers
Ning Sun <classicning@gmail.com> 于2026年4月2日周四 08:52写道：
> Hi,
>
> I'm maintaining a Rust library pgwire to implement postgres wire
> protocol in rust. While doing a corner case test, I noticed the
> inconsistency for ParameterDescription between backend and frontend.
>
> The backend allows up to 65535 parameters in a prepared statement. But
> when running Describe on the statement, there is a size limit of 30000
> bytes for ParameterDescription on the frontend. This means we can only
> describe statements with at most ~7500 parameters. For statements exceed
> the limit, it ends up with error about the message size.
>
> This patch simply adds ParameterDescription to VALID_LONG_MESSAGE_TYPE
> whitelist to remove the cap.
>
>
Hi,
Based on my understanding, this approach is safe for the following reasons:
1. The message format is clearly defined and has been properly handled.
2. Similar message types (such as RowDescription) are already allowed to
exceed this limit.
3. No changes have been introduced at the protocol level.
Thanks!
wang jie
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Add ParameterDescription message to libpq frontend long message types
@ 2026-06-14 12:46 ZizhuanLiu X-MAN <44973863@qq.com>
parent: jie wang <jugierwang@gmail.com>
0 siblings, 0 replies; 4+ messages in thread
From: ZizhuanLiu X-MAN @ 2026-06-14 12:46 UTC (permalink / raw)
To: jie wang <jugierwang@gmail.com>; Ning Sun <classicning@gmail.com>; +Cc: pgsql-hackers
>Original
>From: jie wang <jugierwang@gmail.com>
>Date: 2026-04-02 09:13
>To: Ning Sun <classicning@gmail.com>
>Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
>Subject: Re: Add ParameterDescription message to libpq frontend long message types
>
>Ning Sun <classicning@gmail.com> 于2026年4月2日周四 08:52写道：
>Hi,
>
>I'm maintaining a Rust library pgwire to implement postgres wire
>protocol in rust. While doing a corner case test, I noticed the
>inconsistency for ParameterDescription between backend and frontend.
>
>The backend allows up to 65535 parameters in a prepared statement. But
>when running Describe on the statement, there is a size limit of?30000
>bytes?for ParameterDescription on the frontend. This means we can only
>describe statements with at most ~7500 parameters. For statements exceed
>the limit, it ends up with error about the message size.
>
>This patch simply adds ParameterDescription to VALID_LONG_MESSAGE_TYPE
>whitelist to remove the cap.
>
>
>
>
>Hi,
>
>Based on my understanding, this approach is safe for the following reasons:
>
>1. The message format is clearly defined and has been properly handled.
>2. Similar message types (such as RowDescription) are already allowed to exceed this limit.
>3. No changes have been introduced at the protocol level.
>
>Thanks!
>wang jie
Hi, Ning Sun, wang jie,
Thank you very much for your investigation and patch. I have learned a lot about libpq internals from this discussion.
The overall interaction architecture is: CLIENTAPP <--> LIBPQ <--> BACKEND.
The frontend/backend inconsistency we are discussing happens exactly in this workflow, which I summarize below:
1. The client submits a prepared statement with up to PQ_QUERY_PARAM_MAX_LIMIT (65535) parameters.
Libpq accepts this request, which conforms to the official PostgreSQL limit documented in Appendix K,
where the maximum number of query parameters is defined as 65535.
2. The backend successfully prepares the statement with the full parameter set without any truncation.
3. When the client retrieves statement metadata via PQdescribePrepared() and similar libpq APIs, the backend
returns complete parameter information for all 65535 parameters. The resulting ParameterDescription message
totals 262146 bytes. However, libpq enforces a hardcoded 30000-byte limit for any message type outside the
VALID_LONG_MESSAGE_TYPE whitelist. Since ParameterDescription (type t) is not whitelisted, this valid oversized
message is rejected, resulting in a synchronization loss error.
4. After reviewing the libpq source code, I confirmed that ParameterDescription messages only have two consumption paths:
- Debug tracing: pqTraceOutput_ParameterDescription(), which iterates and prints each parameter’s metadata to conn->Pfdebug;
- Upper-layer application access: clients consume the metadata through public APIs like PQnparams() and PQparamtype().
Even for the full 65535 parameters, the maximum payload size is only around 256KB. This size is completely safe and manageable, and allowing longer ParameterDescription messages introduces no negative impact on either the debug tracing or application API consumption paths.
5. I have applied, compiled and tested this patch locally. It works as expected and perfectly resolves the frontend-backend inconsistency.
Thanks again for your great work!
regards,
--
ZizhuanLiu (X-MAN)
44973863@qq.com
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Add ParameterDescription message to libpq frontend long message types
@ 2026-06-15 09:01 Heikki Linnakangas <hlinnaka@iki.fi>
parent: Ning Sun <classicning@gmail.com>
1 sibling, 0 replies; 4+ messages in thread
From: Heikki Linnakangas @ 2026-06-15 09:01 UTC (permalink / raw)
To: Ning Sun <classicning@gmail.com>; pgsql-hackers
On 01/04/2026 09:32, Ning Sun wrote:
> I'm maintaining a Rust library pgwire to implement postgres wire
> protocol in rust. While doing a corner case test, I noticed the
> inconsistency for ParameterDescription between backend and frontend.
>
> The backend allows up to 65535 parameters in a prepared statement. But
> when running Describe on the statement, there is a size limit of 30000
> bytes for ParameterDescription on the frontend. This means we can only
> describe statements with at most ~7500 parameters. For statements exceed
> the limit, it ends up with error about the message size.
>
> This patch simply adds ParameterDescription to VALID_LONG_MESSAGE_TYPE
> whitelist to remove the cap.
Committed and backpatched to all supported versions, thanks!
- Heikki
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2026-06-15 09:01 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-04-01 06:32 Add ParameterDescription message to libpq frontend long message types Ning Sun <classicning@gmail.com>
2026-04-02 01:13 ` jie wang <jugierwang@gmail.com>
2026-06-14 12:46 ` ZizhuanLiu X-MAN <44973863@qq.com>
2026-06-15 09:01 ` Heikki Linnakangas <hlinnaka@iki.fi>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox