pgjdbc/pgjdbc GitHub issues and pull requests (mirror)help / color / mirror / Atom feed
[pgjdbc/pgjdbc] issue #4015: receiveTupleV3 allocates huge byte[] due to misread 4-byte field length under high concurrency 2+ messages / 2 participants [nested] [flat]
* [pgjdbc/pgjdbc] issue #4015: receiveTupleV3 allocates huge byte[] due to misread 4-byte field length under high concurrency @ 2026-04-10 10:42 "duwenice (@duwenice)" <[email protected]> 0 siblings, 0 replies; 2+ messages in thread From: duwenice (@duwenice) @ 2026-04-10 10:42 UTC (permalink / raw) To: pgjdbc/pgjdbc <[email protected]> ## Describe the issue Under high-concurrency stress testing, `PGStream.receiveTupleV3()` misreads a 4-byte field length from the protocol stream, causing it to allocate a ~2GB byte array (e.g. `new byte[1697905436]`). The actual field data is only ~9KB. The thread then blocks forever in `SocketInputStream.read0` waiting for data that will never arrive, while holding a 2GB allocation. ## Driver Version 42.7.5 (shipped with Spring Boot 3.4.10) ## Java Version 21 ## PostgreSQL Version 17 ## OS Version macOS (observed in test environment) ## To Reproduce Not reliably reproducible. Observed during high-concurrency stress testing with HikariCP connection pooling. No PgBouncer or connection pooler in front of PostgreSQL. **Likely trigger conditions:** - Connection reuse after a query timeout or cancellation - Residual data left in `VisibleBufferedInputStream` buffer from a previous query - When the reused connection is assigned to a new query, the driver reads the wrong position in the protocol stream - A data byte sequence (e.g. `"e6,"` = `0x65 0x36 0x2C 0x22`) is interpreted as a 4-byte signed integer length field, producing `1697905436` (~1.7GB) ## Expected behaviour The driver should either: 1. Validate that the field length is reasonable (e.g. < 100MB for a text field) 2. Detect protocol stream desync and throw a `PSQLException` instead of allocating a huge array ## Evidence **Stack trace at heap dump time:** ``` java.net.SocketInputStream.socketRead0(FileDescriptor, byte[], int, int) java.net.SocketInputStream.read(byte[], int, int) org.postgresql.core.VisibleBufferedInputStream.read(byte[], int, int) org.postgresql.core.PGStream.receive(byte[], int, int) org.postgresql.core.PGStream.receiveTupleV3() org.postgresql.core.v3.QueryExecutorImpl.processResults(...) ``` **Heap dump analysis:** - `byte[1697905436]` allocated by `receiveTupleV3` - First ~9KB contain valid field data (`"test_0fb64","test_0fe51",...`) - Bytes from offset 8192 onward are all `0x00` (uninitialized) - `VisibleBufferedInputStream` buffer was exactly 8192 bytes, `index == endIndex == 0` at dump time - `PGStream.maxRowSizeBytes = 60795` — no historical row exceeded 60KB **Hex analysis of the misread length:** - `1697905436` = `0x65362C22` - Bytes: `0x65 0x36 0x2C 0x22` → ASCII: `e 6 , "` - This is a fragment of the actual FILTER field data (`"test_...e6,","test_..."`) This proves the 4-byte field length was misread from the protocol stream — business data (`e6,"`) was interpreted as a length integer. ## Connection pool config (HikariCP) ```yaml hikari: leak-detection-threshold: 30000 # No socketTimeout configured # No connection-test-query configured # No max-lifetime configured ``` **JDBC URL:** `jdbc:postgresql://...?reWriteBatchedInserts=true` ## Question Is this a known issue? Is there any bounds checking that could be added to `receiveTupleV3` to prevent allocating enormous byte arrays when the protocol stream is desynchronized (similar to the fix in #1592 for copy protocol)? ## Logs No PSQLException was thrown — the thread silently blocked forever waiting for 1.7GB of data that never arrived. The connection was never returned to the pool (blocked on `socketRead0`). ## Test case Unable to provide a standalone test case — this is a race condition that only occurs under specific high-concurrency conditions with connection reuse after query cancellation/timeout. ^ permalink raw reply [nested|flat] 2+ messages in thread
* Re: [pgjdbc/pgjdbc] issue #4015: receiveTupleV3 allocates huge byte[] due to misread 4-byte field length under high concurrency @ 2026-04-10 11:15 "davecramer (@davecramer)" <[email protected]> 0 siblings, 0 replies; 2+ messages in thread From: davecramer (@davecramer) @ 2026-04-10 11:15 UTC (permalink / raw) To: pgjdbc/pgjdbc <[email protected]> Interesting, thanks for the report. Your suggestions make sense ^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2026-04-10 11:15 UTC | newest] Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2026-04-10 10:42 [pgjdbc/pgjdbc] issue #4015: receiveTupleV3 allocates huge byte[] due to misread 4-byte field length under high concurrency "duwenice (@duwenice)" <[email protected]> 2026-04-10 11:15 Re: [pgjdbc/pgjdbc] issue #4015: receiveTupleV3 allocates huge byte[] due to misread 4-byte field length under high concurrency "davecramer (@davecramer)" <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox