pgjdbc/pgjdbc GitHub issues and pull requests (mirror)
help / color / mirror / Atom feed[pgjdbc/pgjdbc] PR #2525: fix: support queries with up to 65535 (inclusive) parameters
13+ messages / 4 participants
[nested] [flat]
* [pgjdbc/pgjdbc] PR #2525: fix: support queries with up to 65535 (inclusive) parameters
@ 2022-05-31 08:15 "vlsi (@vlsi)" <[email protected]>
0 siblings, 0 replies; 13+ messages in thread
From: vlsi (@vlsi) @ 2022-05-31 08:15 UTC (permalink / raw)
To: pgjdbc/pgjdbc <[email protected]>
Previously the execution failed with "Tried to send an out-of-range integer as a 2-byte value"
when the user attempted executing a query with more than 32767 parameters.
Technically speaking, the wire protocol limit is 2-byte-unsigned-int,
so we should support 65535 parameters.
In practice, simple mode (preferQueryMode=simple) allows executing queries
with an arbitrary number of parameters, however, that escape hatch is not recommended
as it still has limits on the SQL length, and it would likely be slow.
fixes #1311
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #2525: fix: support queries with up to 65535 (inclusive) parameters
@ 2022-05-31 12:27 ` "davecramer (@davecramer)" <[email protected]>
11 siblings, 0 replies; 13+ messages in thread
From: davecramer (@davecramer) @ 2022-05-31 12:27 UTC (permalink / raw)
To: pgjdbc/pgjdbc <[email protected]>
(on pgjdbc/src/main/java/org/postgresql/core/PGStream.java)
are we allowed negative values here ? Should this be < 0 || > 65535 ?
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #2525: fix: support queries with up to 65535 (inclusive) parameters
@ 2022-05-31 12:54 ` "vlsi (@vlsi)" <[email protected]>
11 siblings, 0 replies; 13+ messages in thread
From: vlsi (@vlsi) @ 2022-05-31 12:54 UTC (permalink / raw)
To: pgjdbc/pgjdbc <[email protected]>
(on pgjdbc/src/main/java/org/postgresql/core/PGStream.java)
Indeed. Negatives shall not pass
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #2525: fix: support queries with up to 65535 (inclusive) parameters
@ 2022-06-01 10:14 ` "vlsi (@vlsi)" <[email protected]>
11 siblings, 0 replies; 13+ messages in thread
From: vlsi (@vlsi) @ 2022-06-01 10:14 UTC (permalink / raw)
To: pgjdbc/pgjdbc <[email protected]>
I just pushed revert commits to check if the CI duration improves.
In an unlikely case, "fix close refcursors" caused cursor leaks or something like that.
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #2525: fix: support queries with up to 65535 (inclusive) parameters
@ 2022-06-01 11:33 ` "vlsi (@vlsi)" <[email protected]>
11 siblings, 0 replies; 13+ messages in thread
From: vlsi (@vlsi) @ 2022-06-01 11:33 UTC (permalink / raw)
To: pgjdbc/pgjdbc <[email protected]>
Rolling back "fix close refcursors" reduces CI duration from 3h50m to 50min.
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #2525: fix: support queries with up to 65535 (inclusive) parameters
@ 2022-06-01 11:51 ` "vlsi (@vlsi)" <[email protected]>
11 siblings, 0 replies; 13+ messages in thread
From: vlsi (@vlsi) @ 2022-06-01 11:51 UTC (permalink / raw)
To: pgjdbc/pgjdbc <[email protected]>
Ok, the reason is "org.postgresql.test.jdbc2.RefCursorFetchTest | | PASSED | 2h 57m 24.946s"
In other words, it does not look like a code regression, rather it looks like RefCursorFetchTest is extremely slow for unknown reasons.
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #2525: fix: support queries with up to 65535 (inclusive) parameters
@ 2022-06-01 12:20 ` "davecramer (@davecramer)" <[email protected]>
11 siblings, 0 replies; 13+ messages in thread
From: davecramer (@davecramer) @ 2022-06-01 12:20 UTC (permalink / raw)
To: pgjdbc/pgjdbc <[email protected]>
> Ok, the reason is "org.postgresql.test.jdbc2.RefCursorFetchTest | | PASSED | 2h 57m 24.946s"
>
> In other words, it does not look like a code regression, rather it looks like RefCursorFetchTest is extremely slow for unknown reasons.
that seems rather odd ?
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #2525: fix: support queries with up to 65535 (inclusive) parameters
@ 2022-06-01 13:10 ` "vlsi (@vlsi)" <[email protected]>
11 siblings, 0 replies; 13+ messages in thread
From: vlsi (@vlsi) @ 2022-06-01 13:10 UTC (permalink / raw)
To: pgjdbc/pgjdbc <[email protected]>
It all means "fix close refcursors" was OK, and there's something else going slow with CI env.
The worst thing is that if I launch just `org.postgresql.test.jdbc2.RefCursorFetchTest` in Actions, then it takes ~16 or so seconds.
So it looks like some of the tests before RefCursorFetchTest triggers a bad condition, so subsequent tests become slow.
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #2525: fix: support queries with up to 65535 (inclusive) parameters
@ 2022-06-01 13:16 ` "davecramer (@davecramer)" <[email protected]>
11 siblings, 0 replies; 13+ messages in thread
From: davecramer (@davecramer) @ 2022-06-01 13:16 UTC (permalink / raw)
To: pgjdbc/pgjdbc <[email protected]>
we can try to ask for support from github. They have been helpful in the past on security isssues.
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #2525: fix: support queries with up to 65535 (inclusive) parameters
@ 2022-06-01 14:29 ` "vlsi (@vlsi)" <[email protected]>
11 siblings, 0 replies; 13+ messages in thread
From: vlsi (@vlsi) @ 2022-06-01 14:29 UTC (permalink / raw)
To: pgjdbc/pgjdbc <[email protected]>
Finally, I nailed it: https://github.com/pgjdbc/pgjdbc/commit/82dbbe46002d43298ed4ff1df26623f9cb51effd#diff-dd8064c49349ab...
`SendRecvBufferSizeTest` had `System.setProperty("sendBufferSize", "1024"); System.setProperty("receiveBufferSize", "1024");`
That caused all the test after `SendRecvBufferSizeTest` to use the small buffer size, which caused slow processing of large requests/responses.
Just in case you wonder, the trace log does show buffer sizes:
```
2022-05-31 20:10:33 FINE org.postgresql.Driver connect Connecting with URL: jdbc:postgresql://localhost:5432/test?ApplicationName=Driver Tests&loggerLevel=OFF&loggerFile=target/pgjdbc-tests.log
2022-05-31 20:10:33 FINE org.postgresql.jdbc.PgConnection <init> PostgreSQL JDBC Driver 42.3.7-SNAPSHOT
2022-05-31 20:10:33 FINE org.postgresql.jdbc.PgConnection setDefaultFetchSize setDefaultFetchSize = 0
2022-05-31 20:10:33 FINE org.postgresql.jdbc.PgConnection setPrepareThreshold setPrepareThreshold = 5
2022-05-31 20:10:33 FINE org.postgresql.core.v3.ConnectionFactoryImpl openConnectionImpl Trying to establish a protocol version 3 connection to localhost:5432
2022-05-31 20:10:33 FINE org.postgresql.core.v3.ConnectionFactoryImpl tryConnect Receive Buffer Size is 1,152
2022-05-31 20:10:33 FINE org.postgresql.core.v3.ConnectionFactoryImpl tryConnect Send Buffer Size is 2,304
2022-05-31 20:10:33 FINEST org.postgresql.core.v3.ConnectionFactoryImpl enableSSL FE=> SSLRequest
2022-05-31 20:10:33 FINEST org.postgresql.core.v3.ConnectionFactoryImpl enableSSL <=BE SSLOk
2022-05-31 20:10:33 FINE org.postgresql.ssl.MakeSSL convert converting regular socket connection to SSL
```
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #2525: fix: support queries with up to 65535 (inclusive) parameters
@ 2022-06-14 07:20 ` "pwagland (@pwagland)" <[email protected]>
11 siblings, 0 replies; 13+ messages in thread
From: pwagland (@pwagland) @ 2022-06-14 07:20 UTC (permalink / raw)
To: pgjdbc/pgjdbc <[email protected]>
(on pgjdbc/src/main/java/org/postgresql/jdbc/PgPreparedStatement.java:102)
This _should_ have been:
```java
GT.tr("PreparedStatement can have at most {0} parameters. Please consider using arrays, or splitting the query in several ones, or using COPY. Given query has {1} parameters",
```
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #2525: fix: support queries with up to 65535 (inclusive) parameters
@ 2026-01-15 03:49 ` "jarvis24young (@jarvis24young)" <[email protected]>
11 siblings, 0 replies; 13+ messages in thread
From: jarvis24young (@jarvis24young) @ 2026-01-15 03:49 UTC (permalink / raw)
To: pgjdbc/pgjdbc <[email protected]>
im wondering where it shows the the wire protocol limit is 2-byte-unsigned-int instead of 2-byte-int
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #2525: fix: support queries with up to 65535 (inclusive) parameters
@ 2026-01-15 11:02 ` "davecramer (@davecramer)" <[email protected]>
11 siblings, 0 replies; 13+ messages in thread
From: davecramer (@davecramer) @ 2026-01-15 11:02 UTC (permalink / raw)
To: pgjdbc/pgjdbc <[email protected]>
> im wondering where it shows the the wire protocol limit is 2-byte-unsigned-int instead of 2-byte-int
all of the protocol docs are on the postgresql website.
^ permalink raw reply [nested|flat] 13+ messages in thread
end of thread, other threads:[~2026-01-15 11:02 UTC | newest]
Thread overview: 13+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2022-05-31 08:15 [pgjdbc/pgjdbc] PR #2525: fix: support queries with up to 65535 (inclusive) parameters "vlsi (@vlsi)" <[email protected]>
2022-05-31 12:27 ` "davecramer (@davecramer)" <[email protected]>
2022-05-31 12:54 ` "vlsi (@vlsi)" <[email protected]>
2022-06-01 10:14 ` "vlsi (@vlsi)" <[email protected]>
2022-06-01 11:33 ` "vlsi (@vlsi)" <[email protected]>
2022-06-01 11:51 ` "vlsi (@vlsi)" <[email protected]>
2022-06-01 12:20 ` "davecramer (@davecramer)" <[email protected]>
2022-06-01 13:10 ` "vlsi (@vlsi)" <[email protected]>
2022-06-01 13:16 ` "davecramer (@davecramer)" <[email protected]>
2022-06-01 14:29 ` "vlsi (@vlsi)" <[email protected]>
2022-06-14 07:20 ` "pwagland (@pwagland)" <[email protected]>
2026-01-15 03:49 ` "jarvis24young (@jarvis24young)" <[email protected]>
2026-01-15 11:02 ` "davecramer (@davecramer)" <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox