public inbox for [email protected]  

help / color / mirror / Atom feed

From: Dave Page <[email protected]>
To: Aditya Toshniwal <[email protected]>
Cc: pgadmin-hackers <[email protected]>
Subject: Re: Regarding multiple result set in query tool
Date: Thu, 25 Sep 2025 10:58:54 +0100
Message-ID: <CA+OCxowcJNADQdbrzX9qxtOt+8qAS36tcYyE=-tcdQFWGqzVQw@mail.gmail.com> (raw)
In-Reply-To: <CAM9w-_mQsC--AfLuEbv0NbW_p_b=PTD0ZbQr8J2Vv=2ioDp_gw@mail.gmail.com>
References: <CAM9w-_mQsC--AfLuEbv0NbW_p_b=PTD0ZbQr8J2Vv=2ioDp_gw@mail.gmail.com>

Hi

On Wed, 24 Sept 2025 at 13:43, Aditya Toshniwal <
[email protected]> wrote:

> Hi Dave/Hackers,
>
> I'm working on a feature where the query tool will show separate data
> output for all the select statements run in a single batch. psycopg does
> provide the result sets (as libpq provides) but there is a catch. Let me
> explain how pgAdmin currently works:
> 1. psycopg provides a cursor object on query execution.
> 2. The cursor object has a function called nextset which can be used to
> move to the next result set of queries executed.
> 3. Once you move to the nextset, you cannot get data for the previous set.
> It will only point to the current set.
> 4. Right now, we keep on looping through nextset until it reaches the last
> set and then fetch the data from the last set (using pagination).
> 5. The fetched result is stored in client memory (python process)
>
> So if we need to show the output of all the queries, we'll have to
> fetch the result for each query and store it in python memory before moving
> to the next set.
> psycopg already stores the data on the client side, the only difference
> will be that we'll store all sets and not just the last one.
>

That seems like it's potentially problematic with large result sets, and
workarounds would likely lead to potentially confusing behaviour for end
users (which I really don't like the sound of).


>
> If any one has any suggestions on memory management then please let me
> know.
> Otherwise, I'm proceeding with what is discussed above.
>

I don't have any suggestions regarding memory management here, but I do
wonder if this is something which warrants an enhancement to psycopg, to
allow random access to the result sets. At a quick glance, it looks like
BaseCursor._results is a simple list of PGresult objects, which could be
easily exposed of course. What I haven't checked is whether any witchcraft
happens that would make random access to those objects problematic.

-- 
Dave Page
pgAdmin: https://www.pgadmin.org
PostgreSQL: https://www.postgresql.org
pgEdge: https://www.pgedge.com

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected]
  Subject: Re: Regarding multiple result set in query tool
  In-Reply-To: <CA+OCxowcJNADQdbrzX9qxtOt+8qAS36tcYyE=-tcdQFWGqzVQw@mail.gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox