Feedback-ID: id4a34324:Fastmail
Date: Mon, 26 Jun 2023 14:03:48 -0700
From: Andres Freund <andres@anarazel.de>
To: Ronan Dunklau <ronan.dunklau@aiven.io>
Cc: pgsql-hackers@postgresql.org, Peter Eisentraut <peter@eisentraut.org>,
	tomas.vondra@enterprisedb.com
Subject: Re: Add GUC to tune glibc's malloc implementation.
Message-ID: <20230626210348.rk4h3bscdtcwbe7o@awork3.anarazel.de>
References: <3424675.QJadu78ljV@aivenlaptop>
 <e1ce9d7e-745d-5ae9-208a-3e57065a11ec@eisentraut.org>
 <2285512.ElGaqSPkdT@aivenlaptop>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <2285512.ElGaqSPkdT@aivenlaptop>
Archived-At: <https://www.postgresql.org/message-id/20230626210348.rk4h3bscdtcwbe7o%40awork3.anarazel.de>
Precedence: bulk

Hi,

On 2023-06-26 08:38:35 +0200, Ronan Dunklau wrote:
> I hope what I'm trying to achieve is clearer that way. Maybe this patch is not
> the best way to go about this, but since the memory allocator behaviour can
> have such an impact it's a bit sad we have to leave half the performance on
> the table because of it when there are easily accessible knobs to avoid it.

I'm *quite* doubtful this patch is the way to go.  If we want to more tightly
control memory allocation patterns, because we have more information than
glibc, we should do that, rather than try to nudge glibc's malloc in random
direction.  In contrast a generic malloc() implementation we can have much
more information about memory lifetimes etc due to memory contexts.

We e.g. could keep a larger number of memory blocks reserved
ourselves. Possibly by delaying the release of additionally held blocks until
we have been idle for a few seconds or such.


WRT to the difference in TPS in the benchmark you mention - I suspect that we
are doing something bad that needs to be improved regardless of the underlying
memory allocator implementation.  Due to the lack of detailed instructions I
couldn't reproduce the results immediately.

Greetings,

Andres Freund