pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Don't reduce output request size on non-Unix-socket connections. 6+ messages / 1 participants [nested] [flat]
* pgsql: Don't reduce output request size on non-Unix-socket connections. @ 2025-06-10 22:39 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2025-06-10 22:39 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Don't reduce output request size on non-Unix-socket connections. Traditionally, libpq's pqPutMsgEnd has rounded down the amount-to-send to be a multiple of 8K when it is eagerly writing some data. This still seems like a good idea when sending through a Unix socket, as pipes typically have a buffer size of 8K or some fraction/multiple of that. But there's not much argument for it on a TCP connection, since (a) standard MTU values are not commensurate with that, and (b) the kernel typically applies its own packet splitting/merging logic. Worse, our SSL and GSSAPI code paths both have API stipulations that if they fail to send all the data that was offered in the previous write attempt, we mustn't offer less data in the next attempt; else we may get "SSL error: bad length" or "GSSAPI caller failed to retransmit all data needing to be retried". The previous write attempt might've been pqFlush attempting to send everything in the buffer, so pqPutMsgEnd can't safely write less than the full buffer contents. (Well, we could add some more state to track exactly how much the previous write attempt was, but there's little value evident in such extra complication.) Hence, apply the round-down only on AF_UNIX sockets, where we never use SSL or GSSAPI. Interestingly, we had a very closely related bug report before, which I attempted to fix in commit d053a879b. But the test case we had then seemingly didn't trigger this pqFlush-then-pqPutMsgEnd scenario, or at least we failed to recognize this variant of the bug. Bug: #18907 Reported-by: Dorjpalam Batbaatar <htgn.dbat.95@gmail.com> Author: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/18907-d41b9bcf6f29edda@postgresql.org Backpatch-through: 13 Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/0703c9385ba0ff435efec0673ca9f15bda941317 Modified Files -------------- src/backend/libpq/be-secure-gssapi.c | 6 +++--- src/interfaces/libpq/fe-misc.c | 28 +++++++++++++++++++++++++++- src/interfaces/libpq/fe-secure-gssapi.c | 6 +++--- 3 files changed, 33 insertions(+), 7 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Don't reduce output request size on non-Unix-socket connections. @ 2025-06-10 22:39 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2025-06-10 22:39 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Don't reduce output request size on non-Unix-socket connections. Traditionally, libpq's pqPutMsgEnd has rounded down the amount-to-send to be a multiple of 8K when it is eagerly writing some data. This still seems like a good idea when sending through a Unix socket, as pipes typically have a buffer size of 8K or some fraction/multiple of that. But there's not much argument for it on a TCP connection, since (a) standard MTU values are not commensurate with that, and (b) the kernel typically applies its own packet splitting/merging logic. Worse, our SSL and GSSAPI code paths both have API stipulations that if they fail to send all the data that was offered in the previous write attempt, we mustn't offer less data in the next attempt; else we may get "SSL error: bad length" or "GSSAPI caller failed to retransmit all data needing to be retried". The previous write attempt might've been pqFlush attempting to send everything in the buffer, so pqPutMsgEnd can't safely write less than the full buffer contents. (Well, we could add some more state to track exactly how much the previous write attempt was, but there's little value evident in such extra complication.) Hence, apply the round-down only on AF_UNIX sockets, where we never use SSL or GSSAPI. Interestingly, we had a very closely related bug report before, which I attempted to fix in commit d053a879b. But the test case we had then seemingly didn't trigger this pqFlush-then-pqPutMsgEnd scenario, or at least we failed to recognize this variant of the bug. Bug: #18907 Reported-by: Dorjpalam Batbaatar <htgn.dbat.95@gmail.com> Author: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/18907-d41b9bcf6f29edda@postgresql.org Backpatch-through: 13 Branch ------ REL_17_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/30e0d9ee904d29854c43c15b4ecbe13ceaf2d7d6 Modified Files -------------- src/backend/libpq/be-secure-gssapi.c | 6 +++--- src/interfaces/libpq/fe-misc.c | 28 +++++++++++++++++++++++++++- src/interfaces/libpq/fe-secure-gssapi.c | 6 +++--- 3 files changed, 33 insertions(+), 7 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Don't reduce output request size on non-Unix-socket connections. @ 2025-06-10 22:39 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2025-06-10 22:39 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Don't reduce output request size on non-Unix-socket connections. Traditionally, libpq's pqPutMsgEnd has rounded down the amount-to-send to be a multiple of 8K when it is eagerly writing some data. This still seems like a good idea when sending through a Unix socket, as pipes typically have a buffer size of 8K or some fraction/multiple of that. But there's not much argument for it on a TCP connection, since (a) standard MTU values are not commensurate with that, and (b) the kernel typically applies its own packet splitting/merging logic. Worse, our SSL and GSSAPI code paths both have API stipulations that if they fail to send all the data that was offered in the previous write attempt, we mustn't offer less data in the next attempt; else we may get "SSL error: bad length" or "GSSAPI caller failed to retransmit all data needing to be retried". The previous write attempt might've been pqFlush attempting to send everything in the buffer, so pqPutMsgEnd can't safely write less than the full buffer contents. (Well, we could add some more state to track exactly how much the previous write attempt was, but there's little value evident in such extra complication.) Hence, apply the round-down only on AF_UNIX sockets, where we never use SSL or GSSAPI. Interestingly, we had a very closely related bug report before, which I attempted to fix in commit d053a879b. But the test case we had then seemingly didn't trigger this pqFlush-then-pqPutMsgEnd scenario, or at least we failed to recognize this variant of the bug. Bug: #18907 Reported-by: Dorjpalam Batbaatar <htgn.dbat.95@gmail.com> Author: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/18907-d41b9bcf6f29edda@postgresql.org Backpatch-through: 13 Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/137935bd1167a94b0bfea7239033f1ba1a1d95bb Modified Files -------------- src/backend/libpq/be-secure-gssapi.c | 6 +++--- src/interfaces/libpq/fe-misc.c | 28 +++++++++++++++++++++++++++- src/interfaces/libpq/fe-secure-gssapi.c | 6 +++--- 3 files changed, 33 insertions(+), 7 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Don't reduce output request size on non-Unix-socket connections. @ 2025-06-10 22:39 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2025-06-10 22:39 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Don't reduce output request size on non-Unix-socket connections. Traditionally, libpq's pqPutMsgEnd has rounded down the amount-to-send to be a multiple of 8K when it is eagerly writing some data. This still seems like a good idea when sending through a Unix socket, as pipes typically have a buffer size of 8K or some fraction/multiple of that. But there's not much argument for it on a TCP connection, since (a) standard MTU values are not commensurate with that, and (b) the kernel typically applies its own packet splitting/merging logic. Worse, our SSL and GSSAPI code paths both have API stipulations that if they fail to send all the data that was offered in the previous write attempt, we mustn't offer less data in the next attempt; else we may get "SSL error: bad length" or "GSSAPI caller failed to retransmit all data needing to be retried". The previous write attempt might've been pqFlush attempting to send everything in the buffer, so pqPutMsgEnd can't safely write less than the full buffer contents. (Well, we could add some more state to track exactly how much the previous write attempt was, but there's little value evident in such extra complication.) Hence, apply the round-down only on AF_UNIX sockets, where we never use SSL or GSSAPI. Interestingly, we had a very closely related bug report before, which I attempted to fix in commit d053a879b. But the test case we had then seemingly didn't trigger this pqFlush-then-pqPutMsgEnd scenario, or at least we failed to recognize this variant of the bug. Bug: #18907 Reported-by: Dorjpalam Batbaatar <htgn.dbat.95@gmail.com> Author: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/18907-d41b9bcf6f29edda@postgresql.org Backpatch-through: 13 Branch ------ REL_15_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/6a4d93edad4b0afc1c04ffa76c155648c3b35c08 Modified Files -------------- src/backend/libpq/be-secure-gssapi.c | 6 +++--- src/interfaces/libpq/fe-misc.c | 28 +++++++++++++++++++++++++++- src/interfaces/libpq/fe-secure-gssapi.c | 6 +++--- 3 files changed, 33 insertions(+), 7 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Don't reduce output request size on non-Unix-socket connections. @ 2025-06-10 22:39 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2025-06-10 22:39 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Don't reduce output request size on non-Unix-socket connections. Traditionally, libpq's pqPutMsgEnd has rounded down the amount-to-send to be a multiple of 8K when it is eagerly writing some data. This still seems like a good idea when sending through a Unix socket, as pipes typically have a buffer size of 8K or some fraction/multiple of that. But there's not much argument for it on a TCP connection, since (a) standard MTU values are not commensurate with that, and (b) the kernel typically applies its own packet splitting/merging logic. Worse, our SSL and GSSAPI code paths both have API stipulations that if they fail to send all the data that was offered in the previous write attempt, we mustn't offer less data in the next attempt; else we may get "SSL error: bad length" or "GSSAPI caller failed to retransmit all data needing to be retried". The previous write attempt might've been pqFlush attempting to send everything in the buffer, so pqPutMsgEnd can't safely write less than the full buffer contents. (Well, we could add some more state to track exactly how much the previous write attempt was, but there's little value evident in such extra complication.) Hence, apply the round-down only on AF_UNIX sockets, where we never use SSL or GSSAPI. Interestingly, we had a very closely related bug report before, which I attempted to fix in commit d053a879b. But the test case we had then seemingly didn't trigger this pqFlush-then-pqPutMsgEnd scenario, or at least we failed to recognize this variant of the bug. Bug: #18907 Reported-by: Dorjpalam Batbaatar <htgn.dbat.95@gmail.com> Author: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/18907-d41b9bcf6f29edda@postgresql.org Backpatch-through: 13 Branch ------ REL_13_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/f09fea386ce913d1b67f00fab67e695ecc0f270a Modified Files -------------- src/backend/libpq/be-secure-gssapi.c | 6 +++--- src/interfaces/libpq/fe-misc.c | 28 +++++++++++++++++++++++++++- src/interfaces/libpq/fe-secure-gssapi.c | 6 +++--- 3 files changed, 33 insertions(+), 7 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Don't reduce output request size on non-Unix-socket connections. @ 2025-06-10 22:39 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2025-06-10 22:39 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Don't reduce output request size on non-Unix-socket connections. Traditionally, libpq's pqPutMsgEnd has rounded down the amount-to-send to be a multiple of 8K when it is eagerly writing some data. This still seems like a good idea when sending through a Unix socket, as pipes typically have a buffer size of 8K or some fraction/multiple of that. But there's not much argument for it on a TCP connection, since (a) standard MTU values are not commensurate with that, and (b) the kernel typically applies its own packet splitting/merging logic. Worse, our SSL and GSSAPI code paths both have API stipulations that if they fail to send all the data that was offered in the previous write attempt, we mustn't offer less data in the next attempt; else we may get "SSL error: bad length" or "GSSAPI caller failed to retransmit all data needing to be retried". The previous write attempt might've been pqFlush attempting to send everything in the buffer, so pqPutMsgEnd can't safely write less than the full buffer contents. (Well, we could add some more state to track exactly how much the previous write attempt was, but there's little value evident in such extra complication.) Hence, apply the round-down only on AF_UNIX sockets, where we never use SSL or GSSAPI. Interestingly, we had a very closely related bug report before, which I attempted to fix in commit d053a879b. But the test case we had then seemingly didn't trigger this pqFlush-then-pqPutMsgEnd scenario, or at least we failed to recognize this variant of the bug. Bug: #18907 Reported-by: Dorjpalam Batbaatar <htgn.dbat.95@gmail.com> Author: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/18907-d41b9bcf6f29edda@postgresql.org Backpatch-through: 13 Branch ------ REL_16_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/3f37400cfb87002e48ee98d6c0d34a597d0dddf8 Modified Files -------------- src/backend/libpq/be-secure-gssapi.c | 6 +++--- src/interfaces/libpq/fe-misc.c | 28 +++++++++++++++++++++++++++- src/interfaces/libpq/fe-secure-gssapi.c | 6 +++--- 3 files changed, 33 insertions(+), 7 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2025-06-10 22:39 UTC | newest] Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2025-06-10 22:39 pgsql: Don't reduce output request size on non-Unix-socket connections. Tom Lane <tgl@sss.pgh.pa.us> 2025-06-10 22:39 pgsql: Don't reduce output request size on non-Unix-socket connections. Tom Lane <tgl@sss.pgh.pa.us> 2025-06-10 22:39 pgsql: Don't reduce output request size on non-Unix-socket connections. Tom Lane <tgl@sss.pgh.pa.us> 2025-06-10 22:39 pgsql: Don't reduce output request size on non-Unix-socket connections. Tom Lane <tgl@sss.pgh.pa.us> 2025-06-10 22:39 pgsql: Don't reduce output request size on non-Unix-socket connections. Tom Lane <tgl@sss.pgh.pa.us> 2025-06-10 22:39 pgsql: Don't reduce output request size on non-Unix-socket connections. Tom Lane <tgl@sss.pgh.pa.us>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox