Message-ID: <gh-pgjdbc-pgjdbc-3483-c2616818194@github.com>
From: "james-johnston-thumbtack (@james-johnston-thumbtack)" <noreply+james-johnston-thumbtack@github.com>
To: "pgjdbc/pgjdbc" <noreply+pgjdbc-pgjdbc@github.com>
Date: Mon, 27 Jan 2025 20:26:20 +0000
Subject: Re: [pgjdbc/pgjdbc] issue #3483: Support adaptive fetching without enforcing memory limits, and/or have a separate buffer size for it
In-Reply-To: <gh-pgjdbc-pgjdbc-3483@github.com>
References: <gh-pgjdbc-pgjdbc-3483@github.com>
List-Id: <gh-pgjdbc-pgjdbc.github.com>
X-GitHub-Author-Login: james-johnston-thumbtack
X-GitHub-Comment-Id: 2616818194
X-GitHub-Comment-Type: issue_comment
X-GitHub-Issue: 3483
X-GitHub-Repo: pgjdbc/pgjdbc
X-GitHub-Type: comment
X-GitHub-Url: https://github.com/pgjdbc/pgjdbc/issues/3483#issuecomment-2616818194
Content-Type: text/plain; charset=utf-8

In my situation, I'm an end-user using & tuning parameters for the existing open source [Debezium's PostgreSQL connector](https://debezium.io/documentation/reference/stable/connectors/postgresql.html), so changing JDBC drivers isn't practical.  One of the tricky things here is easily setting efficient parameters generally while not running out of memory / making good use of memory, when row sizes and row counts do vary widely from table to table.  Picking an appropriate buffer size measured in bytes, not rows, for the JDBC driver would make this much easier.  I don't think there's a useful need to multiplex multiple queries or fetches over one connection in this use case, and IMHO it's perfectly reasonable to read the rest of the fetch before being able to do something else on the connection.