Feedback-ID: id4a34324:Fastmail
Date: Mon, 6 Mar 2023 10:15:11 -0800
From: Andres Freund <andres@anarazel.de>
To: =?utf-8?B?0KLRgNC+0YTQuNC80L7QsiDQmNCy0LDQvQ==?= <i.trofimow@yandex.ru>,
	pgsql-hackers@postgresql.org
Cc: "pgsql-interfaces@lists.postgresql.org"
 <pgsql-interfaces@lists.postgresql.org>
Subject: Re: About default inBufSize (connection read buffer size) in libpq
Message-ID: <20230306181511.kpje2skz2vhpqz3b@awork3.anarazel.de>
References: <298631677983213@mail.yandex.ru>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <298631677983213@mail.yandex.ru>
Archived-At: 
 <https://www.postgresql.org/message-id/20230306181511.kpje2skz2vhpqz3b%40awork3.anarazel.de>
Precedence: bulk

Hi,

On 2023-03-05 05:42:06 +0300, Трофимов Иван wrote:
> I was running some benchmarks for PG driver built on top of libpq async
> functionality,
> and noticed that recv syscalls issued by the application are limited by 16Kb,
> which seems to
> be inBufSize coming from makeEmptyPGconn in interfaces/libpq/fe-connect.c.
>  
> Hacking that to higher values allowed my benchmarks to issue drastically less
> syscalls
> when running some heavy selects, both in local and cloud environments, which
> made them
> significantly faster.
>  
> I believe there is a reason for that value to be 16Kb, but i was wondering if
> it's safe to change
> this default to user-provided value, and if it is - could this functionality be
> added into API?

I've observed the small buffer size hurting as well - not just client side,
also on the serve.

But I don't think we necessarily need to make it configurable. From what I can
tell the pain mainly comes using the read/send buffers when they don't even
help, because the message data we're processing is bigger than the buffer
size. When we need to receive / send data that we know is bigger than the
the buffer, we should copy the portion that is still in the buffer, and then
send/receive directly from the data to be sent/received.

Greetings,

Andres Freund