public inbox for [email protected]
help / color / mirror / Atom feedClient-server communication for FETCH
4+ messages / 3 participants
[nested] [flat]
* Client-server communication for FETCH
@ 2025-12-03 18:02 Tim Fors <[email protected]>
2025-12-03 18:41 ` Re: Client-server communication for FETCH Tom Lane <[email protected]>
0 siblings, 1 reply; 4+ messages in thread
From: Tim Fors @ 2025-12-03 18:02 UTC (permalink / raw)
To: [email protected]
Hello community,
I have a simple C program using libpq V18 to FETCH one row at a time (see
below). Does the server send multiple rows to the client where they are
cached for satisfying the FETCH statements, or does it just send one row at
a time since that's all that each FETCH statement is asking for? If it's
sending multiple rows at a time, is there some way that I can observe this
e.g. some kind of trace info that would show it? Or where in the libpq
source would I look to see how the client is retrieving rows from the
server?
I have done extensive searching to try and find the definitive answer to
this. The searches indicate that libpq supports the concept of a
client-side cursor, where it has a cache of rows sent by the server and
uses that cache to perform each FETCH, but I'd like to be able to verify
whether this is true or not.
Thanks,
Tim
res = PQexec(conn, "DECLARE cur CURSOR FOR select * from table1");
if (PQresultStatus(res) != PGRES_COMMAND_OK)
{
printf("DECLARE CURSOR failed: %s", PQerrorMessage(conn));
PQclear(res);
exit(1);
}
PQclear(res);
res = PQexec(conn, "FETCH in cur");
if (PQresultStatus(res) != PGRES_TUPLES_OK)
{
printf("FETCH failed: %s", PQerrorMessage(conn));
PQclear(res);
exit(1);
}
PQclear(res);
res = PQexec(conn, "FETCH in cur");
if (PQresultStatus(res) != PGRES_TUPLES_OK)
{
printf("FETCH failed: %s", PQerrorMessage(conn));
PQclear(res);
exit(1);
}
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Client-server communication for FETCH
2025-12-03 18:02 Client-server communication for FETCH Tim Fors <[email protected]>
@ 2025-12-03 18:41 ` Tom Lane <[email protected]>
2025-12-03 19:45 ` Re: Client-server communication for FETCH Tim Fors <[email protected]>
0 siblings, 1 reply; 4+ messages in thread
From: Tom Lane @ 2025-12-03 18:41 UTC (permalink / raw)
To: Tim Fors <[email protected]>; +Cc: [email protected]
Tim Fors <[email protected]> writes:
> I have a simple C program using libpq V18 to FETCH one row at a time (see
> below). Does the server send multiple rows to the client where they are
> cached for satisfying the FETCH statements, or does it just send one row at
> a time since that's all that each FETCH statement is asking for?
It's sending one row at a time. You could improve that by asking
for more than one row ("FETCH n" not just "FETCH"), but you'd have
to iterate over the rows returned by each command.
Another alternative is to forget about using a cursor, tell the
server to just execute the query, but use libpq's "chunked results"
mode to process rows before all of the query result has arrived.
See
https://www.postgresql.org/docs/current/libpq-single-row-mode.html
> I have done extensive searching to try and find the definitive answer to
> this. The searches indicate that libpq supports the concept of a
> client-side cursor, where it has a cache of rows sent by the server and
> uses that cache to perform each FETCH, but I'd like to be able to verify
> whether this is true or not.
Don't know where you read that, but it's not true of libpq. There
might be other Postgres client libraries that can do that.
regards, tom lane
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Client-server communication for FETCH
2025-12-03 18:02 Client-server communication for FETCH Tim Fors <[email protected]>
2025-12-03 18:41 ` Re: Client-server communication for FETCH Tom Lane <[email protected]>
@ 2025-12-03 19:45 ` Tim Fors <[email protected]>
2025-12-05 09:08 ` Re: Client-server communication for FETCH Jan Claeys <[email protected]>
0 siblings, 1 reply; 4+ messages in thread
From: Tim Fors @ 2025-12-03 19:45 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: [email protected]
Thanks for the quick reply Tom.
Let me give you a bit of background info:
Oracle has a COBOL pre-processor for embedded SQL, and it supports an
option named PREFETCH:
The precompiler option PREFETCH allows for more efficient queries by
> pre-fetching rows. This decreases the number of server round-trips needed
> and reduces memory required. The number of rows set by the PREFETCH option
> value in a configuration file or on the command line is used for all
> queries involving explicit cursors, subject to the standard precedence
> rules. When used inline, the PREFETCH option must precede any of these
> cursor statements: • EXEC SQL OPEN cursor • EXEC SQL OPEN cursor USING
> host_var_list • EXEC SQL OPEN cursor USING DESCRIPTOR desc_name When an
> OPEN is executed, the value of PREFETCH gives the number of rows to be
> prefetched when the query is executed. You can set the value from 0 (no
> pre-fetching) to 9999. The default value is 1.
The IBM compiler for COBOL on Linux x86 recently added support for PGSQL
(as a co-processor, integrated into the compiler). We have a business
partner with a COBOL application that contains a large number of single-row
FETCH statements, and they've noticed a significant performance degradation
when switching from Oracle to PGSQL, and it's almost certainly because
they're using that Oracle option PREFETCH. They are asserting that it's the
large amount of client-server exchanges to retrieve one row at a time that
is the problem in PGSQL.
As you can imagine, they're not wanting to rework their application(s) in
order to achieve the same performance - they want the single-row FETCH to
be as performant as Oracle. And I'm sure this same stance would be adopted
by other users of the compiler who want to use PGSQL as well. They have
loops in their programs which FETCH rows one at a time. I was hoping that
the "chunked mode" you mentioned would achieve the equivalent of a
client-side cursor, but your answer indicates that it won't.
The IBM co-processor is a rewrite of ECPG, and therefore uses libecpg at
runtime, which of course uses libpq.
Do you have any further thoughts re: addressing this requirement
(client-side cursor support)? Is this a requirement that the PGSQL
community would consider implementing in libpq? Without it, it seems like a
huge impediment to adoption of PGSQL for COBOL applications, and as you
probably know, COBOL applications are still very pervasive. (C programs
using ECPG would of course hit the same issue, so it's not just a
COBOL-specific problem.)
Thanks,
Tim
On Wed, Dec 3, 2025 at 1:41 PM Tom Lane <[email protected]> wrote:
> Tim Fors <[email protected]> writes:
> > I have a simple C program using libpq V18 to FETCH one row at a time (see
> > below). Does the server send multiple rows to the client where they are
> > cached for satisfying the FETCH statements, or does it just send one row
> at
> > a time since that's all that each FETCH statement is asking for?
>
> It's sending one row at a time. You could improve that by asking
> for more than one row ("FETCH n" not just "FETCH"), but you'd have
> to iterate over the rows returned by each command.
>
> Another alternative is to forget about using a cursor, tell the
> server to just execute the query, but use libpq's "chunked results"
> mode to process rows before all of the query result has arrived.
> See
>
> https://www.postgresql.org/docs/current/libpq-single-row-mode.html
>
> > I have done extensive searching to try and find the definitive answer to
> > this. The searches indicate that libpq supports the concept of a
> > client-side cursor, where it has a cache of rows sent by the server and
> > uses that cache to perform each FETCH, but I'd like to be able to verify
> > whether this is true or not.
>
> Don't know where you read that, but it's not true of libpq. There
> might be other Postgres client libraries that can do that.
>
> regards, tom lane
>
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Client-server communication for FETCH
2025-12-03 18:02 Client-server communication for FETCH Tim Fors <[email protected]>
2025-12-03 18:41 ` Re: Client-server communication for FETCH Tom Lane <[email protected]>
2025-12-03 19:45 ` Re: Client-server communication for FETCH Tim Fors <[email protected]>
@ 2025-12-05 09:08 ` Jan Claeys <[email protected]>
0 siblings, 0 replies; 4+ messages in thread
From: Jan Claeys @ 2025-12-05 09:08 UTC (permalink / raw)
To: [email protected]
On Wed, 2025-12-03 at 14:45 -0500, Tim Fors wrote:
> Do you have any further thoughts re: addressing this requirement
> (client-side cursor support)? Is this a requirement that the PGSQL
> community would consider implementing in libpq? Without it, it seems
> like a huge impediment to adoption of PGSQL for COBOL applications,
> and as you probably know, COBOL applications are still very
> pervasive. (C programs using ECPG would of course hit the same issue,
> so it's not just a COBOL-specific problem.)
Sounds to me like this ought to be implemented in ECPG and/or its COBOL
equivalent, not in libpq?
That's where the "Embedded SQL" is implemented after all, and also the
equivalent of where Oracle implements its PREFETCH, I think?
And I suppose nobody would be opposed to such improvements, if someone
contributes them or pays for their development...
--
Jan Claeys
(please don't CC me when replying to the list)
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2025-12-05 09:08 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2025-12-03 18:02 Client-server communication for FETCH Tim Fors <[email protected]>
2025-12-03 18:41 ` Tom Lane <[email protected]>
2025-12-03 19:45 ` Tim Fors <[email protected]>
2025-12-05 09:08 ` Jan Claeys <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox