Date: Fri, 19 Feb 2021 22:36:55 -0300
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
To: Craig Ringer <craig.ringer@enterprisedb.com>
Cc: Matthieu Garrigues <matthieu.garrigues@gmail.com>,
	PostgreSQL-development <pgsql-hackers@lists.postgresql.org>
Subject: Re: PATCH: Batch/pipelining support for libpq
Message-ID: <20210220013655.GA26818@alvherre.pgsql>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: 
 <CAGRY4nxT5v3Q+BD9F9=6dGNz6EnQL-BTP-AYJTxg+wqOXd9PgA@mail.gmail.com>
User-Agent: Mutt/1.10.1 (2018-07-13)
Precedence: bulk

On 2021-Feb-20, Craig Ringer wrote:

> On Wed, 17 Feb 2021, 07:13 Alvaro Herrera, <alvherre@alvh.no-ip.org> wrote:

> > I
> > think the docs could use some additional tweaks now in order to make a
> > coherent story on pipeline mode, how it can be used in a batched
> > fashion, etc.
> 
> I'll do that soon and send an update.

I started to do that, but was sidetracked having a look at error
handling while checking one of the claims.

So now it starts with this:

   <sect3 id="libpq-pipeline-sending">
    <title>Issuing Queries</title>

    <para>
     After entering pipeline mode, the application dispatches requests using
     <xref linkend="libpq-PQsendQueryParams"/>, 
     or its prepared-query sibling
     <xref linkend="libpq-PQsendQueryPrepared"/>.
     These requests are queued on the client-side until flushed to the server;
     this occurs when <xref linkend="libpq-PQsendPipeline"/> is used to
     establish a synchronization point in the pipeline,
     or when <xref linkend="libpq-PQflush"/> is called.
     The functions <xref linkend="libpq-PQsendPrepare"/>,
     <xref linkend="libpq-PQsendDescribePrepared"/>, and
     <xref linkend="libpq-PQsendDescribePortal"/> also work in pipeline mode.
     Result processing is described below.
    </para>

    <para>
     The server executes statements, and returns results, in the order the
     client sends them.  The server will begin executing the commands in the
     pipeline immediately, not waiting for the end of the pipeline.
     If any statement encounters an error, the server aborts the current
     transaction and skips processing commands in the pipeline until the
     next synchronization point established by <function>PQsendPipeline</function>.
     (This remains true even if the commands in the pipeline would rollback
     the transaction.)
     Query processing resumes after the synchronization point.
    </para>

(Note I changed the wording that "the pipeline is ended by
PQsendPipeline" to "a synchronization point is established".  Is this
easily understandable?  On re-reading it, I'm not sure it's really an
improvement.)

BTW we don't explain why doesn't PQsendQuery work (to wit: because it
uses "simple" query protocol and thus doesn't require the Sync message).
I think we should either document that, or change things so that
PQsendQuery no longer uses a 'Q' message when in pipeline mode; instead
use extended query protocol.  I don't see why we'd force people to use
PQsendQueryParams when not necessary.

BTW, in my experimentation, the sequence of PGresult that you get when
handling a pipeline with errors don't make a lot of sense.  I'll spend
some more time on it.

While at it, as for the PGresult sequence and NULL returns: I think for
PQexec and maybe PQsendQuery, it makes sense to loop until PQgetResult
returns NULL, because you never know how many results are you going to
get.  But for PQsendQueryParams, this is no longer true, because you
can't send multiple queries in one query string.  So the only way to get
multiple results, is by using single-row mode.  But that already has its
own protocol for multiple results, namely to get a stream of
PGRES_SINGLE_TUPLE terminated by a zero-rows PGRES_TUPLES_OK.  So I'm
not sure there is a strong need for the mandatory NULL result.

-- 
Álvaro Herrera                            39°49'30"S 73°17'W