Subject: Client-server communication for FETCH

From: Tim Fors <[email-masked]>
Date: Wed, 3 Dec 2025 13:02:40 -0500

Hello community,

I have a simple C program using libpq V18 to FETCH one row at a time (see
below). Does the server send multiple rows to the client where they are
cached for satisfying the FETCH statements, or does it just send one row at
a time since that's all that each FETCH statement is asking for? If it's
sending multiple rows at a time, is there some way that I can observe this
e.g. some kind of trace info that would show it? Or where in the libpq
source would I look to see how the client is retrieving rows from the
server?

I have done extensive searching to try and find the definitive answer to
this. The searches indicate that libpq supports the concept of a
client-side cursor, where it has a cache of rows sent by the server and
uses that cache to perform each FETCH, but I'd like to be able to verify
whether this is true or not.

Thanks,

Tim

  res = PQexec(conn, "DECLARE cur CURSOR FOR select * from table1");
  if (PQresultStatus(res) != PGRES_COMMAND_OK)
  {
    printf("DECLARE CURSOR failed: %s", PQerrorMessage(conn));
    PQclear(res);
    exit(1);
  }
  PQclear(res);

  res = PQexec(conn, "FETCH in cur");
  if (PQresultStatus(res) != PGRES_TUPLES_OK)
  {
    printf("FETCH failed: %s", PQerrorMessage(conn));
    PQclear(res);
    exit(1);
  }
  PQclear(res);

  res = PQexec(conn, "FETCH in cur");
  if (PQresultStatus(res) != PGRES_TUPLES_OK)
  {
    printf("FETCH failed: %s", PQerrorMessage(conn));
    PQclear(res);
    exit(1);
  }

---

From: Tom Lane <[email-masked]>
Date: Wed, 03 Dec 2025 13:41:48 -0500

Tim Fors <[email-masked]> writes:
> a time since that's all that each FETCH statement is asking for?

It's sending one row at a time.  You could improve that by asking
for more than one row ("FETCH n" not just "FETCH"), but you'd have
to iterate over the rows returned by each command.

Another alternative is to forget about using a cursor, tell the
server to just execute the query, but use libpq's "chunked results"
mode to process rows before all of the query result has arrived.
See

https://www.postgresql.org/docs/current/libpq-single-row-mode.html

> whether this is true or not.

Don't know where you read that, but it's not true of libpq.  There
might be other Postgres client libraries that can do that.

			regards, tom lane

---

From: Tim Fors <[email-masked]>
Date: Wed, 3 Dec 2025 14:45:54 -0500

Thanks for the quick reply Tom.

Let me give you a bit of background info:

Oracle has a COBOL pre-processor for embedded SQL, and it supports an
option named PREFETCH:

The precompiler option PREFETCH allows for more efficient queries by
> pre-fetching rows. This decreases the number of server round-trips needed
> and reduces memory required. The number of rows set by the PREFETCH option
> value in a configuration file or on the command line is used for all
> queries involving explicit cursors, subject to the standard precedence
> rules. When used inline, the PREFETCH option must precede any of these
> cursor statements: • EXEC SQL OPEN cursor • EXEC SQL OPEN cursor USING
> host_var_list • EXEC SQL OPEN cursor USING DESCRIPTOR desc_name When an
> OPEN is executed, the value of PREFETCH gives the number of rows to be
> prefetched when the query is executed. You can set the value from 0 (no
> pre-fetching) to 9999. The default value is 1.

The IBM compiler for COBOL on Linux x86 recently added support for PGSQL
(as a co-processor, integrated into the compiler). We have a business
partner with a COBOL application that contains a large number of single-row
FETCH statements, and they've noticed a significant performance degradation
when switching from Oracle to PGSQL, and it's almost certainly because
they're using that Oracle option PREFETCH. They are asserting that it's the
large amount of client-server exchanges to retrieve one row at a time that
is the problem in PGSQL.

As you can imagine, they're not wanting to rework their application(s) in
order to achieve the same performance - they want the single-row FETCH to
be as performant as Oracle. And I'm sure this same stance would be adopted
by other users of the compiler who want to use PGSQL as well. They have
loops in their programs which FETCH rows one at a time.  I was hoping that
the "chunked mode" you mentioned would achieve the equivalent of a
client-side cursor, but your answer indicates that it won't.

The IBM co-processor is a rewrite of ECPG, and therefore uses libecpg at
runtime, which of course uses libpq.

Do you have any further thoughts re: addressing this requirement
(client-side cursor support)? Is this a requirement that the PGSQL
community would consider implementing in libpq? Without it, it seems like a
huge impediment to adoption of PGSQL for COBOL applications, and as you
probably know, COBOL applications are still very pervasive. (C programs
using ECPG would of course hit the same issue, so it's not just a
COBOL-specific problem.)

Thanks,

Tim

---

From: Jan Claeys <[email-masked]>
Date: Fri, 05 Dec 2025 10:08:53 +0100

On Wed, 2025-12-03 at 14:45 -0500, Tim Fors wrote:
> community would consider implementing in libpq? Without it, it seems
> like a huge impediment to adoption of PGSQL for COBOL applications,
> and as you probably know, COBOL applications are still very
> pervasive. (C programs using ECPG would of course hit the same issue,
> so it's not just a COBOL-specific problem.)

Sounds to me like this ought to be implemented in ECPG and/or its COBOL
equivalent, not in libpq?

That's where the "Embedded SQL" is implemented after all, and also the
equivalent of where Oracle implements its PREFETCH, I think?

And I suppose nobody would be opposed to such improvements, if someone
contributes them or pays for their development...

-- 
Jan Claeys

(please don't CC me when replying to the list)