public inbox for [email protected]
help / color / mirror / Atom feedFrom: Ronan Dunklau <[email protected]>
To: Andres Freund <[email protected]>
Cc: [email protected]
Cc: Peter Eisentraut <[email protected]>
Cc: [email protected]
Subject: Re: Add GUC to tune glibc's malloc implementation.
Date: Tue, 27 Jun 2023 08:35:28 +0200
Message-ID: <8235070.T7Z3S40VBb@aivenlaptop> (raw)
In-Reply-To: <[email protected]>
References: <3424675.QJadu78ljV@aivenlaptop>
<2285512.ElGaqSPkdT@aivenlaptop>
<[email protected]>
Le lundi 26 juin 2023, 23:03:48 CEST Andres Freund a écrit :
> Hi,
>
> On 2023-06-26 08:38:35 +0200, Ronan Dunklau wrote:
> > I hope what I'm trying to achieve is clearer that way. Maybe this patch is
> > not the best way to go about this, but since the memory allocator
> > behaviour can have such an impact it's a bit sad we have to leave half
> > the performance on the table because of it when there are easily
> > accessible knobs to avoid it.
> I'm *quite* doubtful this patch is the way to go. If we want to more
> tightly control memory allocation patterns, because we have more
> information than glibc, we should do that, rather than try to nudge glibc's
> malloc in random direction. In contrast a generic malloc() implementation
> we can have much more information about memory lifetimes etc due to memory
> contexts.
Yes this is probably much more appropriate, but a much larger change with
greater risks of regression. Especially as we have to make sure we're not
overfitting our own code for a specific malloc implementation, to the detriment
of others. Except if you hinted we should write our own directly instead ?
>
> We e.g. could keep a larger number of memory blocks reserved
> ourselves. Possibly by delaying the release of additionally held blocks
> until we have been idle for a few seconds or such.
I think keeping work_mem around after it has been used a couple times make
sense. This is the memory a user is willing to dedicate to operations, after
all.
>
>
> WRT to the difference in TPS in the benchmark you mention - I suspect that
> we are doing something bad that needs to be improved regardless of the
> underlying memory allocator implementation. Due to the lack of detailed
> instructions I couldn't reproduce the results immediately.
I re-attached the simple script I used. I've run this script with different
values for glibc_malloc_max_trim_threshold.
Best regards,
--
Ronan Dunklau
Attachments:
[application/x-shellscript] bench.sh (735B, ../8235070.T7Z3S40VBb@aivenlaptop/2-bench.sh)
download
view thread (6+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected]
Subject: Re: Add GUC to tune glibc's malloc implementation.
In-Reply-To: <8235070.T7Z3S40VBb@aivenlaptop>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox