public inbox for [email protected]  

help / color / mirror / Atom feed

From: Tom Lane <[email protected]>
To: Jelte Fennema-Nio <[email protected]>
Cc: Дмитрий Питаков <[email protected]>
Cc: [email protected]
Subject: Re: Small LO_BUFSIZE slows down lo_import and lo_export in libpq
Date: Fri, 21 Jun 2024 15:43:47 -0400
Message-ID: <[email protected]> (raw)
In-Reply-To: <CAGECzQSdUevo7NrbNvnfrxukfXShJPS-BA+7cuQ-OMcCpz0D=A@mail.gmail.com>
References: <CAN9_0q6SxObnbEdwwmDON_YqhtH2C75z32wKf1wYWh4cKfbo6Q@mail.gmail.com>
	<CAGECzQSdUevo7NrbNvnfrxukfXShJPS-BA+7cuQ-OMcCpz0D=A@mail.gmail.com>

Jelte Fennema-Nio <[email protected]> writes:
> On Fri, 21 Jun 2024 at 10:46, Дмитрий Питаков <[email protected]> wrote:
>> Why not increase the buffer size?

> I think changing the buffer size sounds like a reasonable idea, if
> that speeds stuff up. But I think it would greatly help your case if
> you showed the perf increase using a simple benchmark, especially if
> people could run this benchmark on their own machines to reproduce.

Yeah.  "Why not" is not a patch proposal, mainly because the correct
question is "what other size are you proposing?"

This is not something that we can just randomly whack around, either.
Both lo_import_internal and lo_export assume they can allocate the
buffer on the stack, which means you have to worry about available
stack space.  As a concrete example, I believe that musl still
defaults to 128kB thread stack size, which means that a threaded
client program on that platform would definitely fail with 
LO_BUFSIZE >= 128kB, and even 64kB would be not without risk.

We could dodge that objection by malloc'ing the buffer, which might
be a good thing to do anyway because it'd improve the odds of getting
a nicely-aligned buffer.  But then you have to make the case that the
extra malloc and free isn't a net loss, which it could be for
not-very-large transfers.

So bottom line is that you absolutely need a test case whose
performance can be measured under different conditions.

			regards, tom lane

view thread (4+ messages)

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected], [email protected]
  Subject: Re: Small LO_BUFSIZE slows down lo_import and lo_export in libpq
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox