From: Greg Stark <gsstark@mit.edu>
To: Alvaro Herrera <alvherre@commandprompt.com>
Cc: Goulet, Dick <DGoulet@vicr.com>
Cc: Frank van Vugt <ftm.van.vugt@foxi.nl>
Cc: Greg Stark <gsstark@mit.edu>
Cc: pgsql-interfaces@postgresql.org
Subject: Re: Incremental results from libpq
Date: 13 Nov 2005 11:13:48 -0500
Message-ID: <87r79kwmo3.fsf@stark.xeocode.com> (raw)
In-Reply-To: <20051113152401.GB31570@surnet.cl>
References: <4001DEAF7DF9BD498B58B45051FBEA6502EF5389@25exch1.vicorpower.vicr.com>
<20051113152401.GB31570@surnet.cl>
Alvaro Herrera <alvherre@commandprompt.com> writes:
> Postgres supports cursors too. The Qt guys, and everyone else, could be
> using it to get incremental results right now, no libpq mods necessary.
Not really since the way Postgres supports cursors is at the SQL level. Users
of Qt and every other driver could be using cursors if their drivers support
them, but Qt can't really be reasonably expected to go into users' SQL and
modify them to use cursors.
Moreover cursors aren't really that great a substitute. With cursors you have
to manually fetch individual records. You're just trading off the
inefficiencies of batching up all the results for entirely different
inefficiencies. Now for every record you retrieve you need a network round
trip as well as a round trip down through your driver, the kernel layers on
both machines, and the backend as well.
The efficient approach as Oracle and other mature network layers implement is
to issue the query once, then pipeline the results back to the application
buffering a substantial amount in the driver. DBD::Oracle goes to some lengths
to ensure the number of records buffered is a reasonable multiple of the
default TCP mss of 1500 bytes.
So even though the application only retrieves one record at a time it's just
pulling it out of an array that's already prefilled. When the array gets low
the next block of records is retrieved from the server (where they're probably
already buffered as well). The result is a constant flow of network traffic
that keeps the application and network as busy as possible.
--
greg
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-interfaces@postgresql.org
Cc: gsstark@mit.edu, alvherre@commandprompt.com, DGoulet@vicr.com, ftm.van.vugt@foxi.nl
Subject: Re: Incremental results from libpq
In-Reply-To: <87r79kwmo3.fsf@stark.xeocode.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox