pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Petr Jelinek <petr@2ndquadrant.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: John Gorman <johngorman2@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: PATCH: two slab-like memory allocators
Date: Tue, 25 Oct 2016 23:48:10 +0200
Message-ID: <4f176a6c-c197-165d-dee4-5de8f279f1f4@2ndquadrant.com> (raw)
In-Reply-To: <f2aaf4a5-412c-a4b9-389d-c9f682fbdc77@2ndquadrant.com>
References: <d15dff83-0b37-28ed-0809-95a5cc7292ad@2ndquadrant.com>
	<c5653004-fe91-5a13-409c-db9f34642b0c@2ndquadrant.com>
	<68b837a3-ce55-f9a2-f685-9eafac536f21@2ndquadrant.com>
	<885540a7-ac72-5a21-6dae-a4d5b50672da@2ndquadrant.com>
	<03110c11-ea72-7b4d-027d-6f19eb300b2a@2ndquadrant.com>
	<CALkS6B9W0xHS25OOq2sWhUWtS79XuPiiAOFpsh6F8v56uAvy_Q@mail.gmail.com>
	<e342a3f5-6fd2-0c1f-ca12-f6b1dd7ad88f@2ndquadrant.com>
	<CALkS6B8hisTEJJ7ZCQ_GiBLNVzKfsevAD+92rYN6myyFK=UQtw@mail.gmail.com>
	<11821eb9-95a5-353e-ccba-13e8e9351e62@2ndquadrant.com>
	<b8ded0aa-2515-7ffd-162b-8f515df5eca1@2ndquadrant.com>
	<CA+Tgmob7u4thiBbdvyseQk1aVApJeG10C_rFFt8LPV03-UzD7g@mail.gmail.com>
	<a4593432-c989-8ec4-2320-943e3f97ce6d@2ndquadrant.com>
	<CA+TgmobJrM74FRTcmn2JDMVbLB0a5pj5e3ieLBMzkDNtrxF4dw@mail.gmail.com>
	<9818b54a-fcb6-f018-0b16-96d6a1d1da27@2ndquadrant.com>
	<34a6d4ea-7f3d-a62c-fff1-7a9108c078d7@2ndquadrant.com>
	<f2aaf4a5-412c-a4b9-389d-c9f682fbdc77@2ndquadrant.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>

On 10/23/2016 05:26 PM, Petr Jelinek wrote:
> On 23/10/16 16:26, Tomas Vondra wrote:
>> On 10/22/2016 08:30 PM, Tomas Vondra wrote:
>> ...
>> Moreover, the slab/gen allocators proposed here seem like a better
>> fit for reorderbuffer, e.g. because they release memory. I haven't
>> looked at sb_alloc too closely, but I think it behaves more like
>> AllocSet in this regard (i.e. keeping the memory indefinitely).
>>
>
> For reorderbuffer, from what I've seen in practice, I'd prefer
> proper freeing to 5% performance gain as I seen walsenders taking GBs
> of memory dues to reoderbuffer allocations that are never properly
> freed.
>

Right.

 >
> About your actual patch. I do like both the Slab and the Gen allocators
> and think that we should proceed with them for the moment. You
> definitely need to rename the Gen one (don't ask me to what though) as
> it sounds like "generic" and do some finishing touches but I think it's
> the way to go. I don't see any point in GenSlab anymore.
>

Attached is a v5 of the patch that does this i.e. throws away the 
GenSlab allocator and modifies reorderbuffer in two steps.

First (0001) it adds Slab allocator for TXN/Change allocations, and 
keeps the local slab cache for TupleBuf allocations (with a separate 
AllocSet context).

Then (in 0002) it adds the Gen allocator for TupleBuf, removing the last 
bits of the local slab cache.

I do think this version is is as simple as it gets - there's not much 
more we could simplify / remove.

The main issue that bugs me is the name of the Gen allocator, but I 
don't have a good naming ideas :( The basic characteristics of Gen is 
that it does not reuse space released by pfree(), relying on the fact 
that the whole block will become free. That should be reflected in the 
name somehow, I guess.

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Attachments:

  [application/x-compressed-tar] slab-allocators-v5.tgz (18.3K, ../4f176a6c-c197-165d-dee4-5de8f279f1f4@2ndquadrant.com/2-slab-allocators-v5.tgz)
  download

view thread (97+ messages)  latest in thread

Message-ID: <4f176a6c-c197-165d-dee4-5de8f279f1f4@2ndquadrant.com>
Permalink:  ../4f176a6c-c197-165d-dee4-5de8f279f1f4@2ndquadrant.com/
Also on:    postgresql.org/message-id/4f176a6c-c197-165d-dee4-5de8f279f1f4@2ndquadrant.com

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: tomas.vondra@2ndquadrant.com, petr@2ndquadrant.com, robertmhaas@gmail.com, johngorman2@gmail.com
  Subject: Re: PATCH: two slab-like memory allocators
  In-Reply-To: <4f176a6c-c197-165d-dee4-5de8f279f1f4@2ndquadrant.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox