From: Ronan Dunklau <ronan.dunklau@aiven.io>
To: David Rowley <dgrowleyml@gmail.com>, pgsql-hackers@lists.postgresql.org,
 Tomas Vondra <tomas.vondra@enterprisedb.com>
Cc: Andres Freund <andres@anarazel.de>, Tomas Vondra <tv@fuzzy.cz>
Subject: Re: Use generation context to speed up tuplesorts
Date: Fri, 17 Dec 2021 09:08:06 +0100
Message-ID: <7285172.GXAFRqVoOG@aivenronan>
Organization: aiven
In-Reply-To: <70b689dc-7ad4-138c-2c0b-9fcc131a1aaf@enterprisedb.com>
References: 
 <CAApHDvoH4ASzsAOyHcxkuY01Qf++8JJ0paw+03dk+W25tQEcNQ@mail.gmail.com>
 <36444569.XM6RcZxFsP@aivenronan>
 <70b689dc-7ad4-138c-2c0b-9fcc131a1aaf@enterprisedb.com>
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="iso-8859-1"
Archived-At: 
 <https://www.postgresql.org/message-id/7285172.GXAFRqVoOG%40aivenronan>
Precedence: bulk

Le jeudi 16 d=E9cembre 2021, 18:00:56 CET Tomas Vondra a =E9crit :
> On 12/16/21 17:03, Ronan Dunklau wrote:
> > Le jeudi 16 d=E9cembre 2021, 11:56:15 CET Ronan Dunklau a =E9crit :
> >> I will follow up with a benchmark of the test sorting a table with a
> >> width
> >> varying from 1 to 32 columns.
> >=20
> > So please find attached another benchmark for that case.
> >=20
> > The 3 different patchsets tested are:
> >   - master
> >   - fixed (David's original patch)
> >   - adjust (Thomas growing blocks patch)
>=20
> Presumably Thomas is me, right?

I'm really sorry for this typo... Please accept my apologies.=20

>=20
> > So it looks like tuning malloc for this would be very benificial for any
> > kind of allocation, and by doing so we reduce the problems seen with the
> > growing blocks patch to next to nothing, while keeping the ability to n=
ot
> > allocate too much memory from the get go.
>=20
> Thanks for running those tests and investigating the glibc behavior! I
> find those results very interesting. My conclusions from this is that
> the interaction interaction between "our" allocator and the allocator in
> malloc (e.g. glibc) can be problematic. Which makes benchmarking and
> optimization somewhat tricky because code changes may trigger behavior
> change in glibc (or whatever allocator backs malloc).
>=20
> I think it's worth exploring if we can tune this in a reasonable way,
> but I have a couple concerns related to that:
>=20
> 1) I wonder how glibc-specific this is - I'd bet it applies to other
> allocators (either on another OS or just different allocator on Linux)
> too. Tweaking glibc parameters won't affect those systems, of course,
> but maybe we should allow tweaking those systems too ...

I agree, finding their specific problems and see if we can workaround it wo=
uld=20
be interesting. I suppose glibc's malloc is the most commonly used allocato=
r=20
in production, as it is the default for most Linux distributions.

>=20
> 2) In fact, I wonder if different glibc versions behave differently?
> Hopefully it's not changing that much, though. Ditto kernel versions,
> but the mmap/sbrk interface is likely more stable. We can test this.

That could be tested, yes. As a matter of fact, a commit removing the upper=
=20
limit for MALLOC_MMAP_THRESHOLD has just been committed yesterday to glibc,=
=20
which means we can service much bigger allocation without mmap.


>=20
> 3) If we bump the thresholds, won't that work against reusing the
> memory? I mean, if we free a whole block (from any allocator we have),
> glibc might return it to kernel, depending on mmap threshold value. It's
> not guaranteed, but increasing the malloc thresholds will make that even
> less likely. So we might just as well increase the minimum block size,
> with about the same effect, no?

It is my understanding that malloc will try to compact memory by moving it=
=20
around. So the memory should be actually be released to the kernel at some=
=20
point. In the meantime, malloc can reuse it for our next invocation (which =
can=20
be in a different memory context on our side).=20

If we increase the minimum block size, this is memory we will actually=20
reserve, and it will not protect us against the ramping-up behaviour:
 - the first allocation of a big block may be over mmap_threshold, and serv=
iced=20
by an expensive mmap
 - when it's free, the threshold is doubled
 - next invocation is serviced by an sbrk call
 - freeing it will be above the trim threshold, and it will be returned.

After several "big" allocations, the thresholds will raise to their maximum=
=20
values (well, it used to, I need to check what happens with that latest pat=
ch=20
of glibc...)
=20
This will typically happen several times as malloc doubles the threshold ea=
ch=20
time. This is probably the reason quadrupling the block sizes was more=20
effective.=20


>=20
> > I would like to try to implement some dynamic glibc malloc tuning, if t=
hat
> > is something we don't reject on principle from the get go.
>=20
> +1 to that

Ok, I'll work on a patch for this and submit a new thread.=20


=2D-=20
Ronan Dunklau