From: Tom Lane <tgl@sss.pgh.pa.us>
To: Jelte Fennema-Nio <postgres@jeltef.nl>
cc: =?UTF-8?B?0JTQvNC40YLRgNC40Lkg0J/QuNGC0LDQutC+0LI=?= <dvpitakov@gmail.com>,
        pgsql-hackers@lists.postgresql.org
Subject: Re: Small LO_BUFSIZE slows down lo_import and lo_export in libpq
In-reply-to: <CAGECzQSdUevo7NrbNvnfrxukfXShJPS-BA+7cuQ-OMcCpz0D=A@mail.gmail.com>
References: <CAN9_0q6SxObnbEdwwmDON_YqhtH2C75z32wKf1wYWh4cKfbo6Q@mail.gmail.com> <CAGECzQSdUevo7NrbNvnfrxukfXShJPS-BA+7cuQ-OMcCpz0D=A@mail.gmail.com>
Comments: In-reply-to Jelte Fennema-Nio <postgres@jeltef.nl>
	message dated "Fri, 21 Jun 2024 14:07:21 +0200"
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-ID: <640298.1718999027.1@sss.pgh.pa.us>
Content-Transfer-Encoding: quoted-printable
Date: Fri, 21 Jun 2024 15:43:47 -0400
Message-ID: <640299.1718999027@sss.pgh.pa.us>
Archived-At: <https://www.postgresql.org/message-id/640299.1718999027%40sss.pgh.pa.us>
Precedence: bulk

Jelte Fennema-Nio <postgres@jeltef.nl> writes:
> On Fri, 21 Jun 2024 at 10:46, =D0=94=D0=BC=D0=B8=D1=82=D1=80=D0=B8=D0=B9=
 =D0=9F=D0=B8=D1=82=D0=B0=D0=BA=D0=BE=D0=B2 <dvpitakov@gmail.com> wrote:
>> Why not increase the buffer size?

> I think changing the buffer size sounds like a reasonable idea, if
> that speeds stuff up. But I think it would greatly help your case if
> you showed the perf increase using a simple benchmark, especially if
> people could run this benchmark on their own machines to reproduce.

Yeah.  "Why not" is not a patch proposal, mainly because the correct
question is "what other size are you proposing?"

This is not something that we can just randomly whack around, either.
Both lo_import_internal and lo_export assume they can allocate the
buffer on the stack, which means you have to worry about available
stack space.  As a concrete example, I believe that musl still
defaults to 128kB thread stack size, which means that a threaded
client program on that platform would definitely fail with =

LO_BUFSIZE >=3D 128kB, and even 64kB would be not without risk.

We could dodge that objection by malloc'ing the buffer, which might
be a good thing to do anyway because it'd improve the odds of getting
a nicely-aligned buffer.  But then you have to make the case that the
extra malloc and free isn't a net loss, which it could be for
not-very-large transfers.

So bottom line is that you absolutely need a test case whose
performance can be measured under different conditions.

			regards, tom lane