public inbox for [email protected]
help / color / mirror / Atom feedAbout default inBufSize (connection read buffer size) in libpq
2+ messages / 2 participants
[nested] [flat]
* About default inBufSize (connection read buffer size) in libpq
@ 2023-03-05 02:42 Трофимов Иван <[email protected]>
2023-03-06 18:15 ` Re: About default inBufSize (connection read buffer size) in libpq Andres Freund <[email protected]>
0 siblings, 1 reply; 2+ messages in thread
From: Трофимов Иван @ 2023-03-05 02:42 UTC (permalink / raw)
To: [email protected] <[email protected]>
<div>Hi!</div><div> </div><div>I was running some benchmarks for PG driver built on top of libpq async functionality,</div><div>and noticed that recv syscalls issued by the application are limited by 16Kb, which seems to</div><div>be inBufSize coming from makeEmptyPGconn in interfaces/libpq/fe-connect.c.</div><div> </div><div>Hacking that to higher values allowed my benchmarks to issue drastically less syscalls</div><div>when running some heavy selects, both in local and cloud environments, which made them</div><div>significantly faster.</div><div> </div><div>I believe there is a reason for that value to be 16Kb, but i was wondering if it's safe to change</div><div>this default to user-provided value, and if it is - could this functionality be added into API?</div>
^ permalink raw reply [nested|flat] 2+ messages in thread
* Re: About default inBufSize (connection read buffer size) in libpq
2023-03-05 02:42 About default inBufSize (connection read buffer size) in libpq Трофимов Иван <[email protected]>
@ 2023-03-06 18:15 ` Andres Freund <[email protected]>
0 siblings, 0 replies; 2+ messages in thread
From: Andres Freund @ 2023-03-06 18:15 UTC (permalink / raw)
To: Трофимов Иван <[email protected]>; [email protected]; +Cc: [email protected] <[email protected]>
Hi,
On 2023-03-05 05:42:06 +0300, Трофимов Иван wrote:
> I was running some benchmarks for PG driver built on top of libpq async
> functionality,
> and noticed that recv syscalls issued by the application are limited by 16Kb,
> which seems to
> be inBufSize coming from makeEmptyPGconn in interfaces/libpq/fe-connect.c.
>
> Hacking that to higher values allowed my benchmarks to issue drastically less
> syscalls
> when running some heavy selects, both in local and cloud environments, which
> made them
> significantly faster.
>
> I believe there is a reason for that value to be 16Kb, but i was wondering if
> it's safe to change
> this default to user-provided value, and if it is - could this functionality be
> added into API?
I've observed the small buffer size hurting as well - not just client side,
also on the serve.
But I don't think we necessarily need to make it configurable. From what I can
tell the pain mainly comes using the read/send buffers when they don't even
help, because the message data we're processing is bigger than the buffer
size. When we need to receive / send data that we know is bigger than the
the buffer, we should copy the portion that is still in the buffer, and then
send/receive directly from the data to be sent/received.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2023-03-06 18:15 UTC | newest]
Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-03-05 02:42 About default inBufSize (connection read buffer size) in libpq Трофимов Иван <[email protected]>
2023-03-06 18:15 ` Andres Freund <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox