Subject: Question about non-blocking mode in libpq

From: Yugo NAGATA <[email-masked]>
Date: Tue, 13 Jul 2021 11:59:49 +0900

Hello,

During reading the documentation of libpq [1] , I found the following
description:

 In the nonblocking state, calls to PQsendQuery, PQputline, PQputnbytes,
 PQputCopyData, and PQendcopy will not block but instead return an error
 if they need to be called again.

[1] https://www.postgresql.org/docs/devel/libpq-async.html

However, looking into the code, PQsendQuery seems not to return an error
in non-bloking mode even if unable to send all data. In such cases,
pqSendSome will return 1 but it doesn't cause an error. Moreover,
we would not need to call PQsendQuery again. Indead, we need to call
PQflush until it returns 0, as documented with regard to PQflush.

Do we need to fix the description of PQsetnonblocking?

Regards,
Yugo Nagata

-- 
Yugo NAGATA <[email-masked]>

---

From: Yugo NAGATA <[email-masked]>
Date: Mon, 19 Jul 2021 23:11:29 +0900

On Tue, 13 Jul 2021 11:59:49 +0900

> Do we need to fix the description of PQsetnonblocking?

I have further questions. Reading the following statement:

 "In the nonblocking state, calls to PQsendQuery, PQputline, PQputnbytes,
 PQputCopyData, and PQendcopy will not block" 

this seems to me that this is a list of functions that could block in blocking
mode, but I wander PQflush also could block because it calls pqSendSome, right?

Also, in the last paragraph of the section, I can find the following:

 "After sending any command or data on a nonblocking connection, call PQflush. ..."

However, ISTM we don't need to call PQflush in non-bloking mode and we can
call PQgetResult immediately because PQgetResult internally calls pqFlush
until it returns 0 (or -1).

        /*
         * If data remains unsent, send it.  Else we might be waiting for the
         * result of a command the backend hasn't even got yet.
         */
        while ((flushResult = pqFlush(conn)) > 0) 
        {
            if (pqWait(false, true, conn))
            {
                flushResult = -1;
                break;
            }
        }

Therefore, I wander the last paragraph of this section is
now unnecessary. right?

---

From: Alvaro Herrera <[email-masked]>
Date: Tue, 20 Jul 2021 12:05:11 -0400

> > Do we need to fix the description of PQsetnonblocking?

Yeah, I think you're right -- these functions don't error out, the
commands are just stored locally in the output buffer.

> mode, but I wander PQflush also could block because it calls pqSendSome, right?

I don't see that.  If pqSendSome can't write anything, it'll just return 1.

> until it returns 0 (or -1).

Well, maybe you don't *need* to PQflush(); but if you don't call it,
then the commands will sit in the output buffer indefinitely, which
means the server won't execute them.  So even if it works to just call
PQgetResult and have it block, surely you would like to only call
PQgetResult when the query has already been executed and the result
already been received and processed; that is, so that you can call
PQgetResult and obtain the result immediately, and avoid (say) blocking
a GUI interface while PQgetResult flushes the commands out, the server
executes the query and sends the results back.

> Therefore, I wander the last paragraph of this section is

Doesn't seem so to me.

-- 
Álvaro Herrera           39°49'30"S 73°17'W  —  https://www.EnterpriseDB.com/

---

From: Yugo NAGATA <[email-masked]>
Date: Wed, 21 Jul 2021 10:15:09 +0900

Hello Alvaro,

On Tue, 20 Jul 2021 12:05:11 -0400

> commands are just stored locally in the output buffer.

Thank you for your explanation!
I attached a patch fix the description.

> I don't see that.  If pqSendSome can't write anything, it'll just return 1.

Well, is this the case of non-blocking mode, nor? If I understood correctly,
pqSendSome could block in blocking mode, so PQflush could block, too.  I thought
we should add PQflush to the list in the description to enphasis that this would
not block  in non-blocking mode. However, now I don't think so because PQflush
seems useful only in non-blocking mode.

> executes the query and sends the results back.

I understood that, although PQgetResult() also flushes the buffer, we still
should call PQflush() beforehand because we would not like get blocked after
calling PQgetResult(). Thanks.

Regards,
Yugo Nagata

-- 
Yugo NAGATA <[email-masked]>
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 56689ba873..03ac480d0c 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -4924,8 +4924,8 @@ int PQsetnonblocking(PGconn *conn, int arg);
        In the nonblocking state, calls to
        <xref linkend="libpq-PQsendQuery"/>, <xref linkend="libpq-PQputline"/>,
        <xref linkend="libpq-PQputnbytes"/>, <xref linkend="libpq-PQputCopyData"/>,
-       and <xref linkend="libpq-PQendcopy"/> will not block but instead return
-       an error if they need to be called again.
+       and <xref linkend="libpq-PQendcopy"/> will not block but the commands are
+       stored locally in the output buffer until it is flushed.
       </para>
 
       <para>