pgjdbc/pgjdbc GitHub issues and pull requests (mirror)  

help / color / mirror / Atom feed

From: vlsi (@vlsi) <[email protected]>
To: pgjdbc/pgjdbc <[email protected]>
Subject: Re: [pgjdbc/pgjdbc] issue #2656: Adaptive Fetch doesn't enable ResultSet streaming
Date: Thu, 03 Nov 2022 11:41:55 +0000
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>

> it takes a LONG time to load because it is loading the entire table into RAM.

As far as I remember, there's no way to use partial fetches in autocommit=true mode.

We might probably introduce a warning if the user code happens to fetch a huge resultset with `autocommit=true` mode.

For instance:
* `maxResultSizeBeforeWarning=512m`. In other words, if the resultset fetches more than 512M in one go, raise a warning
* `maxResultSizeBeforeError=4g`. In other words, if a resultset fetches more than 4G in one go, throw an error and discard the data. It would protect apps from running out of memory, and the workaround would be using `autoCommit=false`, or raising the limit, or asking PostgreSQL database developers to support "fetch from portal across transaction". AFAIK even named portals do not survive after the transaction commits, so the only option is to fetch all the rows at once. There's another option to "store" the data in the socket buffer (e.g. fetch 10 rows, return control to the user, and then fetch the next rows), however, it would be hard to implement, and it would effectively make the connection unusable to all the other queries.

WDYT?

view thread (12+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: github://pgjdbc/pgjdbc
  Cc: [email protected], [email protected]
  Subject: Re: [pgjdbc/pgjdbc] issue #2656: Adaptive Fetch doesn't enable ResultSet streaming
  In-Reply-To: <<[email protected]>>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox