pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: David Rowley <drowley@postgresql.org>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Improve the generation memory allocator
Date: Mon, 04 Apr 2022 08:53:46 +0000
Message-ID: <E1nbIT4-000Fxv-An@gemulon.postgresql.org> (raw)

Improve the generation memory allocator

Here we make a series of improvements to the generation memory
allocator, namely:

1. Allow generation contexts to have a minimum, initial and maximum block
sizes. The standard allocator allows this already but when the generation
context was added, it only allowed fixed-sized blocks.  The problem with
fixed-sized blocks is that it's difficult to choose how large to make the
blocks.  If the chosen size is too small then we'd end up with a large
number of blocks and a large number of malloc calls. If the block size is
made too large, then memory is wasted.

2. Add support for "keeper" blocks.  This is a special block that is
allocated along with the context itself but is never freed.  Instead,
when the last chunk in the keeper block is freed, we simply mark the block
as empty to allow new allocations to make use of it.

3. Add facility to "recycle" newly empty blocks instead of freeing them
and having to later malloc an entire new block again.  We do this by
recording a single GenerationBlock which has become empty of any chunks.
When we run out of space in the current block, we check to see if there is
a "freeblock" and use that if it contains enough space for the allocation.

Author: David Rowley, Tomas Vondra
Reviewed-by: Andy Fan
Discussion: https://postgr.es/m/d987fd54-01f8-0f73-af6c-519f799a0ab8@enterprisedb.com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/1b0d9aa4f728edfdff64c3a13fc52bb95f7fb860

Modified Files
--------------
src/backend/access/gist/gistvacuum.c            |   6 +
src/backend/replication/logical/reorderbuffer.c |   7 +
src/backend/utils/mmgr/generation.c             | 385 +++++++++++++++++++-----
src/include/utils/memutils.h                    |   4 +-
4 files changed, 325 insertions(+), 77 deletions(-)



Message-ID: <E1nbIT4-000Fxv-An@gemulon.postgresql.org>
Permalink:  ../E1nbIT4-000Fxv-An@gemulon.postgresql.org/
Also on:    postgresql.org/message-id/E1nbIT4-000Fxv-An@gemulon.postgresql.org

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-committers@postgresql.org
  Cc: drowley@postgresql.org
  Subject: Re: pgsql: Improve the generation memory allocator
  In-Reply-To: <E1nbIT4-000Fxv-An@gemulon.postgresql.org>
  References: <E1nbIT4-000Fxv-An@gemulon.postgresql.org>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox