Subject: Re: PATCH: two slab-like memory allocators
To: Robert Haas <robertmhaas@gmail.com>,
 Tomas Vondra <tomas.vondra@2ndquadrant.com>
References: <d15dff83-0b37-28ed-0809-95a5cc7292ad@2ndquadrant.com>
 <c5653004-fe91-5a13-409c-db9f34642b0c@2ndquadrant.com>
 <68b837a3-ce55-f9a2-f685-9eafac536f21@2ndquadrant.com>
 <885540a7-ac72-5a21-6dae-a4d5b50672da@2ndquadrant.com>
 <03110c11-ea72-7b4d-027d-6f19eb300b2a@2ndquadrant.com>
 <CALkS6B9W0xHS25OOq2sWhUWtS79XuPiiAOFpsh6F8v56uAvy_Q@mail.gmail.com>
 <e342a3f5-6fd2-0c1f-ca12-f6b1dd7ad88f@2ndquadrant.com>
 <CALkS6B8hisTEJJ7ZCQ_GiBLNVzKfsevAD+92rYN6myyFK=UQtw@mail.gmail.com>
 <11821eb9-95a5-353e-ccba-13e8e9351e62@2ndquadrant.com>
 <b8ded0aa-2515-7ffd-162b-8f515df5eca1@2ndquadrant.com>
 <CA+Tgmob7u4thiBbdvyseQk1aVApJeG10C_rFFt8LPV03-UzD7g@mail.gmail.com>
Cc: John Gorman <johngorman2@gmail.com>,
 pgsql-hackers <pgsql-hackers@postgresql.org>
From: Petr Jelinek <petr@2ndquadrant.com>
Message-ID: <a4593432-c989-8ec4-2320-943e3f97ce6d@2ndquadrant.com>
Date: Wed, 19 Oct 2016 00:27:39 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
 Thunderbird/45.3.0
MIME-Version: 1.0
In-Reply-To: 
 <CA+Tgmob7u4thiBbdvyseQk1aVApJeG10C_rFFt8LPV03-UzD7g@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

On 18/10/16 22:25, Robert Haas wrote:
> On Wed, Oct 5, 2016 at 12:22 AM, Tomas Vondra
> <tomas.vondra@2ndquadrant.com> wrote:
>> attached is v3 of the patches, with a few minor fixes in Slab, and much
>> larger fixes in GenSlab.
>>
>> Slab (minor fixes)
>> ------------------
>> - Removed the unnecessary memset() of new blocks in SlabAlloc(), although we
>> still need to zero the free bitmap at the end of the block.
>> - Renamed minFreeCount to minFreeChunks, added a few comments explaining
>> why/how firstFreeChunk and minFreeChunks are maintained.
>> - Fixed / improved a bunch of additional comments, based on feedback.
> 
> I had a look at 0001 today, but it seems to me that it still needs
> work.  It's still got a lot of remnants of where you've
> copy-and-pasted aset.c.  I dispute this allegation:
> 
> + * SlabContext is our standard implementation of MemoryContext.
> 

Are you looking at old version of the patch? I complained about this as
well and Tomas has changed that.

> And then there's this:
> 
> +#ifdef HAVE_ALLOCINFO
> +#define SlabFreeInfo(_cxt, _chunk) \
> +            fprintf(stderr, "AllocFree: %s: %p, %d\n", \
> +                (_cxt)->header.name, (_chunk), (_chunk)->size)
> +#define SlabAllocInfo(_cxt, _chunk) \
> +            fprintf(stderr, "AllocAlloc: %s: %p, %d\n", \
> +                (_cxt)->header.name, (_chunk), (_chunk)->size)
> 
> Well, it's pretty stupid that AllocSetAlloc is reporting it's name as
> AllocAlloc, a think that, as far as I can tell, is not real. But
> having this new context type also pretend to be AllocAlloc is even
> dumber.

You are definitely looking at old version.

> 
> More broadly, I'm not sure I like this design very much.  The whole
> point of a slab context is that all of the objects are the same size.
> I wouldn't find it too difficult to support this patch if we were
> adding an allocator for fixed-size objects that was then being used to
> allocate objects which are fixed size.  However, what we seem to be
> doing is creating an allocator for fixed-size objects and then using
> it for variable-size tuples.  That's really pretty weird.  Isn't the
> root of this problem that aset.c is utterly terrible at handling large
> number of allocations?  Maybe we should try to attack that problem
> more directly.

It's used for TXNs which are fixed and some tuples (there is assumption
that the decoded tuples have more or less normal distribution).

I agree though that the usability beyond the ReoderBuffer is limited
because everything that will want to use it for part of allocations will
get much more complicated by the fact that it will have to use two
different allocators.

I was wondering if rather than trying to implement new allocator we
should maybe implement palloc_fixed which would use some optimized
algorithm for fixed sized objects in our current allocator. The
advantage of that would be that we could for example use that for things
like ListCell easily (memory management of which I see quite often in
profiles).

-- 
  Petr Jelinek                  http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers