Content-Type: multipart/mixed; boundary="------------aoKvjaTWbhpUcYZaeBAQgM5X"
Message-ID: <4cdfc0dc-9f66-4113-ad00-6e7a2cfa7211@iki.fi>
Date: Thu, 9 Jul 2026 15:55:59 +0300
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: libpq: Process buffered SSL read bytes to support records >8kB on
 async API
To: Peter Eisentraut <peter@eisentraut.org>, solai v <solai.cdac@gmail.com>,
 Jacob Champion <jacob.champion@enterprisedb.com>
Cc: Mark Dilger <mark.dilger@enterprisedb.com>,
 Andres Freund <andres@anarazel.de>, Lars Kanis <lars@greiz-reinsdorf.de>,
 pgsql-hackers@lists.postgresql.org, Merlin Moncure <mmoncure@gmail.com>
References: <2039ac58-d3e0-434b-ac1a-2a987f3b4cb1@greiz-reinsdorf.de>
 <CAOYmi+=jZNU9mZ4Z83aR0RiqZ2pF35u5040gCxBgcFqoA4oNbQ@mail.gmail.com>
 <CAOYmi+mpymrgZ76Jre2dx_PwRniS9YZojwH0rZnTuiGHCsj0rA@mail.gmail.com>
 <CAOYmi+=kbb7pV7k7RfWUcFt=CArJmp1R+R1Xnb8UnZLwpFgOwA@mail.gmail.com>
 <CAOYmi+==Y5ZZiwukN-wzGCyxdrL1dU3wOG-j0nza3MC3R2XhcA@mail.gmail.com>
 <kl6inl3iyycxqpiedmnnzaz2ytid5dw6knwwqk5plg6ej34oi6@fbyu4ll7ebzn>
 <CAOYmi+kZhv_LCg=g8Z6xtiueSYdz1fCvrXwAJ-i9m3VMm9_tXw@mail.gmail.com>
 <CAOYmi+mD6EbDEYfwZON0FCUAvGO+2=jR2V4KQYx+d+g0ap0Amg@mail.gmail.com>
 <CAOYmi+=YJ2J_sQ6yS8y1o9AX1Xgz35_r0hApV8QRWwi1K8gC0g@mail.gmail.com>
 <d987a763-09a1-4c7e-844e-bf8596abde43@iki.fi>
 <CAHgHdKseseJG0j_96xrOuP3LgubvGScvXsQyPCYWANnBxTL4tA@mail.gmail.com>
 <57d1e8b1-d016-4cea-9b60-63cbdb40eb81@iki.fi>
 <CAF0whuf=-G3SwXSTWyZNO148qoaEmth71cM7XBuX4X+2PnQkPQ@mail.gmail.com>
 <990a4710-fed1-411a-aba4-ddc25825ee48@iki.fi>
 <be2f229a-25e8-4d19-a307-e4ce3312f212@iki.fi>
 <34844e8c-267c-4daf-b1e0-f26059a4a7d3@eisentraut.org>
Content-Language: en-US
From: Heikki Linnakangas <hlinnaka@iki.fi>
In-Reply-To: <34844e8c-267c-4daf-b1e0-f26059a4a7d3@eisentraut.org>
Archived-At: 
 <https://www.postgresql.org/message-id/4cdfc0dc-9f66-4113-ad00-6e7a2cfa7211%40iki.fi>
Precedence: bulk

This is a multi-part message in MIME format.
--------------aoKvjaTWbhpUcYZaeBAQgM5X
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 08/07/2026 22:16, Peter Eisentraut wrote:
> In pqDrainPending(), there is
> 
>      nread = pqsecure_read(conn, conn->inBuffer + conn->inEnd,
>                            bytes_pending);
>      conn->inEnd += nread;
> 
> But pqsecure_read() can return -1 for error.  So adding that to conn- 
>  >inEnd at that point seems wrong.
> 
> There is error handling in the following code, but this would still kind 
> of corrupt the conn->inEnd value?
> 
>      /* When there are bytes pending, the read function is not supposed 
> to fail */
>      if (nread != bytes_pending)
>      {
>          libpq_append_conn_error(conn,
>                                  "drained only %zu of %zd pending bytes 
> in transport buffer",
>                                  nread, bytes_pending);
>          return -1;
>      }
> 
> I'm not sure if that comment means that a -1 return cannot happen?  Or 
> just that the whole error check should not happen?

The comment in pgsecure_bytes_pending() says:

>  * If pqsecure_read() is called for this number of bytes, it's guaranteed to
>  * return successfully without reading from the underlying socket.  See
>  * pqDrainPending() for a more complete discussion of the concepts involved.

so as long as pqsecure_read() and pqsecure_read_pending() honor that 
contract, pqsecure_read() should not return an error. I agree that's a 
bit sloppy though, we should still check the return value.

The intention was also that it cannot do a short read, but that wasn't 
quite clear from the comment either.

> Also note that the format placeholder for nread is wrong.  If you do get 
> a -1, it will print some large unsigned value.

Good catch.

Patch attached to clean up all that.

- Heikki

--------------aoKvjaTWbhpUcYZaeBAQgM5X
Content-Type: text/x-patch; charset=UTF-8;
 name="0001-libpq-Make-error-checks-in-the-new-buffer-draining-c.patch"
Content-Disposition: attachment;
 filename*0="0001-libpq-Make-error-checks-in-the-new-buffer-draining-c.pa";
 filename*1="tch"
Content-Transfer-Encoding: base64

RnJvbSBiZTY4ZTk0ZjgxZDhiYjgzMGNmZGE2ZWVlNmEyMGE0MWVjMzM1OTMzIE1vbiBTZXAg
MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBIZWlra2kgTGlubmFrYW5nYXMgPGhlaWtraS5saW5u
YWthbmdhc0Bpa2kuZmk+CkRhdGU6IFRodSwgOSBKdWwgMjAyNiAxNTo1MTozMCArMDMwMApT
dWJqZWN0OiBbUEFUQ0ggMS8xXSBsaWJwcTogTWFrZSBlcnJvciBjaGVja3MgaW4gdGhlIG5l
dyBidWZmZXIgZHJhaW5pbmcgY29kZQogbW9yZSByb2J1c3QKCkNoZWNrIGV4cGxpY2l0bHkg
Zm9yIHBxc2VjdXJlX3JlYWQoKSByZXR1cm5pbmcgYW4gZXJyb3IuIEl0IHNob3VsZG4ndApm
YWlsLCBhbmQgd2Ugd291bGQndmUgY2F1Z2h0IGl0IGluIHRoZSBjaGVjayBmb3IgYSBzaG9y
dCByZWFkLCBidXQKYmV0dGVyIHRvIGJlIGV4cGxpY2l0IHNvIHRoYXQgdGhlIGVycm9yIG1l
c3NhZ2UgaXMgbW9yZSBpbmZvcm1hdGl2ZS4KV2UgYWxzbyBzaG91bGRuJ3QgdXBkYXRlICdp
bkVuZCcgd2hlbiB0aGUgcmVhZCBmYWlscywgYWx0aG91Z2ggdGhhdAp0b28gaXMganVzdCBw
cm8gZm9ybWEgYXMgd2Ugd2lsbCBiYWlsIG91dCBhbmQgY2xvc2UgdGhlIGNvbm5lY3Rpb24g
b24KZXJyb3IuCgpSZXBvcnRlZC1ieTogUGV0ZXIgRWlzZW50cmF1dCA8cGV0ZXJAZWlzZW50
cmF1dC5vcmc+CkRpc2N1c3Npb246IGh0dHBzOi8vd3d3LnBvc3RncmVzcWwub3JnL21lc3Nh
Z2UtaWQvMzQ4NDRlOGMtMjY3Yy00ZGFmLWIxZTAtZjI2MDU5YTRhN2QzQGVpc2VudHJhdXQu
b3JnCi0tLQogc3JjL2ludGVyZmFjZXMvbGlicHEvZmUtbWlzYy5jICAgfCAxMSArKysrKysr
Ky0tLQogc3JjL2ludGVyZmFjZXMvbGlicHEvZmUtc2VjdXJlLmMgfCAgNSArKystLQogMiBm
aWxlcyBjaGFuZ2VkLCAxMSBpbnNlcnRpb25zKCspLCA1IGRlbGV0aW9ucygtKQoKZGlmZiAt
LWdpdCBhL3NyYy9pbnRlcmZhY2VzL2xpYnBxL2ZlLW1pc2MuYyBiL3NyYy9pbnRlcmZhY2Vz
L2xpYnBxL2ZlLW1pc2MuYwppbmRleCBkZDc3MjgzODAwNS4uZjExYjU4YmY5YzcgMTAwNjQ0
Ci0tLSBhL3NyYy9pbnRlcmZhY2VzL2xpYnBxL2ZlLW1pc2MuYworKysgYi9zcmMvaW50ZXJm
YWNlcy9saWJwcS9mZS1taXNjLmMKQEAgLTkyOCwxMyArOTI4LDE4IEBAIHBxRHJhaW5QZW5k
aW5nKFBHY29ubiAqY29ubikKIAogCW5yZWFkID0gcHFzZWN1cmVfcmVhZChjb25uLCBjb25u
LT5pbkJ1ZmZlciArIGNvbm4tPmluRW5kLAogCQkJCQkJICBieXRlc19wZW5kaW5nKTsKLQlj
b25uLT5pbkVuZCArPSBucmVhZDsKIAotCS8qIFdoZW4gdGhlcmUgYXJlIGJ5dGVzIHBlbmRp
bmcsIHRoZSByZWFkIGZ1bmN0aW9uIGlzIG5vdCBzdXBwb3NlZCB0byBmYWlsICovCisJLyoK
KwkgKiBXaGVuIHRoZXJlIGFyZSBieXRlcyBwZW5kaW5nLCBwcXNlY3VyZV9yZWFkKCkgaXMg
bm90IHN1cHBvc2VkIHRvIGZhaWwKKwkgKiBvciBkbyBhIHNob3J0IHJlYWQsIGJ1dCBsZXQn
cyBjaGVjayBhbnl3YXkgdG8gYmUgc2FmZS4KKwkgKi8KKwlpZiAobnJlYWQgPCAwKQorCQly
ZXR1cm4gLTE7CisJY29ubi0+aW5FbmQgKz0gbnJlYWQ7CiAJaWYgKG5yZWFkICE9IGJ5dGVz
X3BlbmRpbmcpCiAJewogCQlsaWJwcV9hcHBlbmRfY29ubl9lcnJvcihjb25uLAotCQkJCQkJ
CQkiZHJhaW5lZCBvbmx5ICV6dSBvZiAlemQgcGVuZGluZyBieXRlcyBpbiB0cmFuc3BvcnQg
YnVmZmVyIiwKKwkJCQkJCQkJImRyYWluZWQgb25seSAlemQgb2YgJXpkIHBlbmRpbmcgYnl0
ZXMgaW4gdHJhbnNwb3J0IGJ1ZmZlciIsCiAJCQkJCQkJCW5yZWFkLCBieXRlc19wZW5kaW5n
KTsKIAkJcmV0dXJuIC0xOwogCX0KZGlmZiAtLWdpdCBhL3NyYy9pbnRlcmZhY2VzL2xpYnBx
L2ZlLXNlY3VyZS5jIGIvc3JjL2ludGVyZmFjZXMvbGlicHEvZmUtc2VjdXJlLmMKaW5kZXgg
NzBmYWY4YjJmZTAuLjFhOGUyZTY3NDZlIDEwMDY0NAotLS0gYS9zcmMvaW50ZXJmYWNlcy9s
aWJwcS9mZS1zZWN1cmUuYworKysgYi9zcmMvaW50ZXJmYWNlcy9saWJwcS9mZS1zZWN1cmUu
YwpAQCAtMjQ3LDggKzI0Nyw5IEBAIHBxc2VjdXJlX3Jhd19yZWFkKFBHY29ubiAqY29ubiwg
dm9pZCAqcHRyLCBzaXplX3QgbGVuKQogICoJUmV0dXJuIHRoZSBudW1iZXIgb2YgYnl0ZXMg
YXZhaWxhYmxlIGluIHRoZSB0cmFuc3BvcnQgYnVmZmVyLgogICoKICAqIElmIHBxc2VjdXJl
X3JlYWQoKSBpcyBjYWxsZWQgZm9yIHRoaXMgbnVtYmVyIG9mIGJ5dGVzLCBpdCdzIGd1YXJh
bnRlZWQgdG8KLSAqIHJldHVybiBzdWNjZXNzZnVsbHkgd2l0aG91dCByZWFkaW5nIGZyb20g
dGhlIHVuZGVybHlpbmcgc29ja2V0LiAgU2VlCi0gKiBwcURyYWluUGVuZGluZygpIGZvciBh
IG1vcmUgY29tcGxldGUgZGlzY3Vzc2lvbiBvZiB0aGUgY29uY2VwdHMgaW52b2x2ZWQuCisg
KiByZXR1cm4gc3VjY2Vzc2Z1bGx5IHdpdGggdGhlIHNhbWUgbnVtYmVyIG9mIGJ5dGVzLCB3
aXRob3V0IHJlYWRpbmcgZnJvbSB0aGUKKyAqIHVuZGVybHlpbmcgc29ja2V0LiAgU2VlIHBx
RHJhaW5QZW5kaW5nKCkgZm9yIGEgbW9yZSBjb21wbGV0ZSBkaXNjdXNzaW9uIG9mCisgKiB0
aGUgY29uY2VwdHMgaW52b2x2ZWQuCiAgKi8KIHNzaXplX3QKIHBxc2VjdXJlX2J5dGVzX3Bl
bmRpbmcoUEdjb25uICpjb25uKQotLSAKMi40Ny4zCgo=

--------------aoKvjaTWbhpUcYZaeBAQgM5X--