Subject: About default inBufSize (connection read buffer size) in libpq

From: Трофимов Иван <[email-masked]>
Date: Sun, 05 Mar 2023 05:42:06 +0300

Hi!

I was running some benchmarks for PG driver built on top of libpq async functionality,

and noticed that recv syscalls issued by the application are limited by 16Kb, which seems to

be inBufSize coming from makeEmptyPGconn in interfaces/libpq/fe-connect.c.

Hacking that to higher values allowed my benchmarks to issue drastically less syscalls

when running some heavy selects, both in local and cloud environments, which made them

significantly faster.

I believe there is a reason for that value to be 16Kb, but i was wondering if it's safe to change

this default to user-provided value, and if it is - could this functionality be added into API?

---

From: Andres Freund <[email-masked]>
Date: Mon, 6 Mar 2023 10:15:11 -0800

Hi,

> this default to user-provided value, and if it is - could this functionality be

I've observed the small buffer size hurting as well - not just client side,
also on the serve.

But I don't think we necessarily need to make it configurable. From what I can
tell the pain mainly comes using the read/send buffers when they don't even
help, because the message data we're processing is bigger than the buffer
size. When we need to receive / send data that we know is bigger than the
the buffer, we should copy the portion that is still in the buffer, and then
send/receive directly from the data to be sent/received.

Greetings,

Andres Freund