pgjdbc/pgjdbc GitHub issues and pull requests (mirror)help / color / mirror / Atom feed
[pgjdbc/pgjdbc] PR #3870: feat: Add INFO log when fetchSize is ignored under autocommit=true 7+ messages / 3 participants [nested] [flat]
* [pgjdbc/pgjdbc] PR #3870: feat: Add INFO log when fetchSize is ignored under autocommit=true @ 2025-11-24 15:51 "helloJosh (@helloJosh)" <[email protected]> 0 siblings, 0 replies; 7+ messages in thread From: helloJosh (@helloJosh) @ 2025-11-24 15:51 UTC (permalink / raw) To: pgjdbc/pgjdbc <[email protected]> First of all, Thank you for maintaining pgjdbc. As someone who relies on PostgreSQL heavily, I really appreciate the effort that goes into this project If it's not too much trouble, I would like to get feedback on whether adding this log message is acceptable. ### Motivation While using Spring Batch with JdbcCursorItemReader, our application repeatedly hit OOM because fetchSize was silently ignored when the connection ran with autocommit=true. Although the documentation notes that server-side cursors require autocommit=false, this detail becomes easy to miss when multiple application are involved. I initially considered implementing a WITH HOLD cursor, but the complexity was non-trivial. So, I am proposing to surface this behavior through an INFO-level log. ### Why INFO - Users typically set fetchSize because the result set is large. - Many users expect streaming behavior even when autocommit is true(or may not realize autocommit is enabled). - This log appears only when fetchSize > 0 and autocommit=true, so it does not introduce noisy logging in unrelated cases. ### What is changed - Add an INFO-level log in PgStatement.executeInternal when: - fetchSize > 0 - autocommit = true - when driver connot use a server-side cursor - No behavior change - cursor flags untouched - No new test - tests passed locally ^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #3870: feat: Add INFO log when fetchSize is ignored under autocommit=true @ 2025-11-25 12:11 ` "davecramer (@davecramer)" <[email protected]> 5 siblings, 0 replies; 7+ messages in thread From: davecramer (@davecramer) @ 2025-11-25 12:11 UTC (permalink / raw) To: pgjdbc/pgjdbc <[email protected]> I'm OK with this. @vlsi ? ^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #3870: feat: Add INFO log when fetchSize is ignored under autocommit=true @ 2025-11-25 12:46 ` "vlsi (@vlsi)" <[email protected]> 5 siblings, 0 replies; 7+ messages in thread From: vlsi (@vlsi) @ 2025-11-25 12:46 UTC (permalink / raw) To: pgjdbc/pgjdbc <[email protected]> I am afraid it might result in logging lots of extra log messages (the code would generate a message for every execution), and it will be useless if the query produces just a few rows. It would be false positives. In the meantime, what do you think of the following instead? a) Add a custom property for tracking performance or stability-related issues. For instance `stabilityIssues=ignore|warn|fail`. Then pgjdbc would either ignore (the default), or log a warning (`warn`), or throw an exception so the user knows which exact functionality uses a bad code pattern b) Log a warning (~`INFO`) only in case the number of fetched rows exceeds certain threshold. It would reduce the number of false-positives. I don't like hard-coding the thresholds though. ^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #3870: feat: Add INFO log when fetchSize is ignored under autocommit=true @ 2025-11-25 13:23 ` "helloJosh (@helloJosh)" <[email protected]> 5 siblings, 0 replies; 7+ messages in thread From: helloJosh (@helloJosh) @ 2025-11-25 13:23 UTC (permalink / raw) To: pgjdbc/pgjdbc <[email protected]> Thanks for the helpful feedback. I understand your concern now - there are far more queries that return only a small number of rows than those that actually cause OOM. For option(b), I also think choosing a meaningful row-count is difficult, since too many rows are subjective and depends on local memory. Following your idea in (a), what do you think about the following behavior for a property `stabilityIssues=ignore|warn|fail` - When set to ignore (the default), pgjdbc would not do anything (current behavior). - When set to warn, pgjdbc would log a warning when a stability issue is detected (for example, fetchSize > 0 together with autocommit=true). - When set to fail, pgjdbc would throw an exception (or possibly log at a higher severity), so users can immediately see that a problematic pattern is being used. ^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #3870: feat: Add INFO log when fetchSize is ignored under autocommit=true @ 2025-11-25 13:31 ` "davecramer (@davecramer)" <[email protected]> 5 siblings, 0 replies; 7+ messages in thread From: davecramer (@davecramer) @ 2025-11-25 13:31 UTC (permalink / raw) To: pgjdbc/pgjdbc <[email protected]> I'm wondering if anything can be done in Spring instead ? ^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #3870: feat: Add INFO log when fetchSize is ignored under autocommit=true @ 2025-11-25 14:10 ` "helloJosh (@helloJosh)" <[email protected]> 5 siblings, 0 replies; 7+ messages in thread From: helloJosh (@helloJosh) @ 2025-11-25 14:10 UTC (permalink / raw) To: pgjdbc/pgjdbc <[email protected]> First, I also thought this could be addressed in Spring Batch. But the difficulty is that Spring would have to apply the same logic consistently across all other JDBC-based components as well. That's why I ended up contributing an update to the Spring Batch documentation instead. Also, I think Spring can't fully detect or explain this behavior. Because fetchSize and server-side cursor is determined by the JDBC driver. In my view pgjdbc is the best place to provide a precise diagnostic signal when fetchSize > 0 is combined with autocommit=true. ^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: [pgjdbc/pgjdbc] PR #3870: feat: Add INFO log when fetchSize is ignored under autocommit=true @ 2025-11-25 23:43 ` "helloJosh (@helloJosh)" <[email protected]> 5 siblings, 0 replies; 7+ messages in thread From: helloJosh (@helloJosh) @ 2025-11-25 23:43 UTC (permalink / raw) To: pgjdbc/pgjdbc <[email protected]> On second thought, lowering the log level to FINE might be a simpler option. It would avoid adding a new property and still provide the message for users who enable detailed logging. ^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2025-11-25 23:43 UTC | newest] Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2025-11-24 15:51 [pgjdbc/pgjdbc] PR #3870: feat: Add INFO log when fetchSize is ignored under autocommit=true "helloJosh (@helloJosh)" <[email protected]> 2025-11-25 12:11 ` "davecramer (@davecramer)" <[email protected]> 2025-11-25 12:46 ` "vlsi (@vlsi)" <[email protected]> 2025-11-25 13:23 ` "helloJosh (@helloJosh)" <[email protected]> 2025-11-25 13:31 ` "davecramer (@davecramer)" <[email protected]> 2025-11-25 14:10 ` "helloJosh (@helloJosh)" <[email protected]> 2025-11-25 23:43 ` "helloJosh (@helloJosh)" <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox