Subject: Re: PATCH: two slab-like memory allocators
To: Tomas Vondra <tomas.vondra@2ndquadrant.com>,
 Robert Haas <robertmhaas@gmail.com>
References: <d15dff83-0b37-28ed-0809-95a5cc7292ad@2ndquadrant.com>
 <c5653004-fe91-5a13-409c-db9f34642b0c@2ndquadrant.com>
 <68b837a3-ce55-f9a2-f685-9eafac536f21@2ndquadrant.com>
 <885540a7-ac72-5a21-6dae-a4d5b50672da@2ndquadrant.com>
 <03110c11-ea72-7b4d-027d-6f19eb300b2a@2ndquadrant.com>
 <CALkS6B9W0xHS25OOq2sWhUWtS79XuPiiAOFpsh6F8v56uAvy_Q@mail.gmail.com>
 <e342a3f5-6fd2-0c1f-ca12-f6b1dd7ad88f@2ndquadrant.com>
 <CALkS6B8hisTEJJ7ZCQ_GiBLNVzKfsevAD+92rYN6myyFK=UQtw@mail.gmail.com>
 <11821eb9-95a5-353e-ccba-13e8e9351e62@2ndquadrant.com>
 <b8ded0aa-2515-7ffd-162b-8f515df5eca1@2ndquadrant.com>
 <CA+Tgmob7u4thiBbdvyseQk1aVApJeG10C_rFFt8LPV03-UzD7g@mail.gmail.com>
 <a4593432-c989-8ec4-2320-943e3f97ce6d@2ndquadrant.com>
 <CA+TgmobJrM74FRTcmn2JDMVbLB0a5pj5e3ieLBMzkDNtrxF4dw@mail.gmail.com>
 <9818b54a-fcb6-f018-0b16-96d6a1d1da27@2ndquadrant.com>
 <34a6d4ea-7f3d-a62c-fff1-7a9108c078d7@2ndquadrant.com>
Cc: John Gorman <johngorman2@gmail.com>,
 pgsql-hackers <pgsql-hackers@postgresql.org>
From: Petr Jelinek <petr@2ndquadrant.com>
Message-ID: <f2aaf4a5-412c-a4b9-389d-c9f682fbdc77@2ndquadrant.com>
Date: Sun, 23 Oct 2016 17:26:18 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
 Thunderbird/45.3.0
MIME-Version: 1.0
In-Reply-To: <34a6d4ea-7f3d-a62c-fff1-7a9108c078d7@2ndquadrant.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

On 23/10/16 16:26, Tomas Vondra wrote:
> On 10/22/2016 08:30 PM, Tomas Vondra wrote:
>> On 10/20/2016 04:43 PM, Robert Haas wrote:
>>>
>>> ...
>>>
>>> The sb_alloc allocator I proposed a couple of years ago would work
>>> well for this case, I think.
>>>
>>
>> Maybe, but it does not follow the Memory Context design at all, if I
>> understand it correctly. I was willing to give it a spin anyway and see
>> how it compares to the two other allocators, but this is a significant
>> paradigm shift and certainly much larger step than what I proposed.
>>
>> I'm not even sure it's possible to implement a MemoryContext based on
>> the same ideas as sb_alloc(), because one of the important points of
>> sb_alloc design seems to be throwing away the chunk header. While that
>> may be possible, it would certainly affect the whole tree (not just the
>> reorderbuffer bit), and it'd require way more work.
>>
>> Moreover, the two allocators I proposed significantly benefit from the
>> "same lifespan" assumption. I don't think sb_alloc can do that.
>>
> 
> I've given the sb_alloc patch another try - essentially hacking it into
> reorderbuffer, ignoring the issues mentioned yesterday. And yes, it's
> faster than the allocators discussed in this thread. Based on a few very
> quick tests on my laptop, the difference is usually ~5-10%.
> 
> That might seem like a significant improvement, but it's negligible
> compared to the "master -> slab/gen" improvement, which improves
> performance by orders of magnitude (at least for the tested cases).
> 
> Moreover, the slab/gen allocators proposed here seem like a better fit
> for reorderbuffer, e.g. because they release memory. I haven't looked at
> sb_alloc too closely, but I think it behaves more like AllocSet in this
> regard (i.e. keeping the memory indefinitely).
> 

For reorderbuffer, from what I've seen in practice, I'd prefer proper
freeing to 5% performance gain as I seen walsenders taking GBs of memory
dues to reoderbuffer allocations that are never properly freed.

About your actual patch. I do like both the Slab and the Gen allocators
and think that we should proceed with them for the moment. You
definitely need to rename the Gen one (don't ask me to what though) as
it sounds like "generic" and do some finishing touches but I think it's
the way to go. I don't see any point in GenSlab anymore.

-- 
  Petr Jelinek                  http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers