Subject: Re: PATCH: two slab-like memory allocators
To: Andres Freund <andres@anarazel.de>
References: <0fdc7920-f1b4-ea5c-0168-ed4f56448e53@2ndquadrant.com>
 <20ca0c69-e8c3-cb50-4456-f03908c672ec@2ndquadrant.com>
 <20170209213732.pykbsixeezcthotr@alap3.anarazel.de>
 <b72dfa3d-c241-8a3e-97ad-bf214b754aca@2ndquadrant.com>
 <20170211013337.iw32bvfqwta634wy@alap3.anarazel.de>
 <53def906-2197-bc06-b987-d237f9574800@2ndquadrant.com>
 <20170214022221.obhdeyfn34h6hkwj@alap3.anarazel.de>
 <19464ac8-e66b-3598-858f-29c749bec35a@2ndquadrant.com>
 <b3b2245c-b37a-e1e5-ebc4-857c914bc747@2ndquadrant.com>
 <20170224221038.43xf2cyonczzr6hb@alap3.anarazel.de>
 <20170227111732.vrx5v72ighehwpkf@alap3.anarazel.de>
Cc: Petr Jelinek <petr.jelinek@2ndquadrant.com>,
 Petr Jelinek <petr@2ndquadrant.com>, Robert Haas <robertmhaas@gmail.com>,
 John Gorman <johngorman2@gmail.com>,
 pgsql-hackers <pgsql-hackers@postgresql.org>
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Message-ID: <f33180b4-2396-68ab-4bfd-cd6fe5578f15@2ndquadrant.com>
Date: Mon, 27 Feb 2017 15:11:40 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
 Thunderbird/45.4.0
MIME-Version: 1.0
In-Reply-To: <20170227111732.vrx5v72ighehwpkf@alap3.anarazel.de>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

On 02/27/2017 12:17 PM, Andres Freund wrote:
> Hi,
>
> On 2017-02-24 14:10:38 -0800, Andres Freund wrote:
>> I've not yet looked a lot at the next type of context - I want to get
>> this much committed first...
>>
>> I plan to give this another pass sometime this weekend and then push
>> soon.
>
> Before committing I wanted to make sure that
> http://archives.postgresql.org/message-id/32354.1487977458%40sss.pgh.pa.us
> isn't a sufficient fix.
>
> With the test of N=1000000 from this thread I measured both runtime and
> memory usage (note that's peak virtual memory which includes 2GB of
> shared_buffers and such), in assert enabled builds.
>
> master:                         doesn't finish reasonably
> master+doubly linked list fix:  9390.805 ms	VmPeak: 10,969,424 kb
> master+this thread:		6500.293 ms	VmPeak:  2,969,528 kB
>
> So the doubly-linked-list fix is great (and much more backpatchable),
> but the patches in this thread are both better runtime *and* peak memory
> usage wise.  So that seems like a clear call.
>

Nice, thanks for doing the test.

> I've not yet reviewed the generational allocator yet, but during these
> measurements I get:
> postgres[3970][1]=# select count(*) FROM pg_logical_slot_get_changes('ttt', NULL, NULL);
> WARNING:  01000: problem in Generation Tuples: number of free chunks 0 in block 0x55d011ef10f0 exceeds 7234 allocated
> LOCATION:  GenerationCheck, generation.c:693
> WARNING:  01000: problem in Generation Tuples: number of free chunks 0 in block 0x55d01023eba0 exceeds 65532 allocated
> LOCATION:  GenerationCheck, generation.c:693
> WARNING:  01000: problem in Generation Tuples: number of free chunks 0 in block 0x55d00d7fb870 exceeds 65532 allocated
> LOCATION:  GenerationCheck, generation.c:693
> WARNING:  01000: problem in Generation Tuples: number of free chunks 0 in block 0x55d00cde17b0 exceeds 65531 allocated
> LOCATION:  GenerationCheck, generation.c:693
>
> that seems to occur when there's currently in-progress transactions when
> finishing decoding:
>
...
>
> could it be that the test's condition is inverted?
>

Yeah, that seems like the culprit - the condition seems wrong. I wonder 
why I haven't seen it during my tests, though ...

> I'll work on getting slab committed first, and then review / edit /
> commit generation.c later.  One first note there is that I'm wondering
> if generation.c is a too generic filename.
>

Naming things is hard.

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers