pgjdbc/pgjdbc GitHub issues and pull requests (mirror)
help / color / mirror / Atom feed[pgjdbc/pgjdbc] PR #4066: feat: Add pipeline mode with dedicated reader thread
4+ messages / 2 participants
[nested] [flat]
* [pgjdbc/pgjdbc] PR #4066: feat: Add pipeline mode with dedicated reader thread
@ 2026-05-12 14:42 "davecramer (@davecramer)" <[email protected]>
0 siblings, 0 replies; 4+ messages in thread
From: davecramer (@davecramer) @ 2026-05-12 14:42 UTC (permalink / raw)
To: pgjdbc/pgjdbc <[email protected]>
Implement protocol-level pipelining using a dedicated reader thread that decouples send and receive paths. When enabled via the pipelineMode connection property, the driver sends Parse/Bind/Execute messages without waiting for responses, while a background thread reads and dispatches results to ResponseSlots using LockSupport park/unpark signaling.
Key design decisions:
- Dedicated reader thread owns pgInput exclusively after startup
- ConcurrentLinkedDeque for lock-free slot queue between threads
- Volatile signaling fields with parkNanos timeout + liveness check
- SYNC_MARKER sentinels for error cascading across batch boundaries
- Clean shutdown path via close()/abort() overrides
Concurrency fixes applied during review:
- Made ResponseSlot.waiter volatile for cross-thread visibility
- Added reader thread shutdown in close() and abort()
- Added parkNanos(1s) timeout with isAlive() liveness check to prevent indefinite parking if reader thread dies
- Await BEGIN slot and propagate errors before user query
- Used receiveErrorString for notice messages (encoding fallback)
Performance measurements (PG 15, us-east-1 EC2, network RTT ~20ms):
Sequential queries (200 iter): sync 36.22ms, pipeline 54.56ms (0.66x)
Batch insert (500 rows): sync 23.52ms, pipeline 19.07ms (1.23x)
Prepared stmt reuse (500 iter): sync 52.99ms, pipeline 55.83ms (0.95x)
Large result fetch (10k rows): sync 18.39ms, pipeline 10.23ms (1.80x)
Pipeline mode shows clear wins for bulk operations (batch insert 1.23x, large result fetch 1.80x) where the decoupled read path eliminates round-trip stalls. Sequential single-query workloads show slight overhead from thread coordination, as expected — the benefit scales with batch size and result volume.
Test: 22 pipeline functional tests + 4 performance tests all pass.
Full suite: 9367 tests, 0 pipeline-related failures.
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #4066: feat: Add pipeline mode with dedicated reader thread
@ 2026-05-14 06:16 "vlsi (@vlsi)" <[email protected]>
2 siblings, 0 replies; 4+ messages in thread
From: vlsi (@vlsi) @ 2026-05-14 06:16 UTC (permalink / raw)
To: pgjdbc/pgjdbc <[email protected]>
@davecramer , is this still work in progress? WDYT of converting it to draft, and converting back when it is ready for the review?
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #4066: feat: Add pipeline mode with dedicated reader thread
@ 2026-05-14 09:38 "davecramer (@davecramer)" <[email protected]>
2 siblings, 0 replies; 4+ messages in thread
From: davecramer (@davecramer) @ 2026-05-14 09:38 UTC (permalink / raw)
To: pgjdbc/pgjdbc <[email protected]>
> @davecramer , is this still work in progress? WDYT of converting it to draft, and converting back when it is ready for the review?
I don't have any issues with that, but why?
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #4066: feat: Add pipeline mode with dedicated reader thread
@ 2026-05-14 09:49 "vlsi (@vlsi)" <[email protected]>
2 siblings, 0 replies; 4+ messages in thread
From: vlsi (@vlsi) @ 2026-05-14 09:49 UTC (permalink / raw)
To: pgjdbc/pgjdbc <[email protected]>
To let the others know when it is time for the review
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2026-05-14 09:49 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-05-12 14:42 [pgjdbc/pgjdbc] PR #4066: feat: Add pipeline mode with dedicated reader thread "davecramer (@davecramer)" <[email protected]>
2026-05-14 06:16 ` "vlsi (@vlsi)" <[email protected]>
2026-05-14 09:38 ` "davecramer (@davecramer)" <[email protected]>
2026-05-14 09:49 ` "vlsi (@vlsi)" <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox