Subject: Re: PATCH: two slab-like memory allocators
To: Petr Jelinek <petr@2ndquadrant.com>, Andres Freund <andres@anarazel.de>
References: 
 <CALkS6B8hisTEJJ7ZCQ_GiBLNVzKfsevAD+92rYN6myyFK=UQtw@mail.gmail.com>
 <11821eb9-95a5-353e-ccba-13e8e9351e62@2ndquadrant.com>
 <b8ded0aa-2515-7ffd-162b-8f515df5eca1@2ndquadrant.com>
 <CA+Tgmob7u4thiBbdvyseQk1aVApJeG10C_rFFt8LPV03-UzD7g@mail.gmail.com>
 <a4593432-c989-8ec4-2320-943e3f97ce6d@2ndquadrant.com>
 <CA+TgmobJrM74FRTcmn2JDMVbLB0a5pj5e3ieLBMzkDNtrxF4dw@mail.gmail.com>
 <9818b54a-fcb6-f018-0b16-96d6a1d1da27@2ndquadrant.com>
 <34a6d4ea-7f3d-a62c-fff1-7a9108c078d7@2ndquadrant.com>
 <f2aaf4a5-412c-a4b9-389d-c9f682fbdc77@2ndquadrant.com>
 <4f176a6c-c197-165d-dee4-5de8f279f1f4@2ndquadrant.com>
 <20161112191257.3a4n6kzegvu637qn@alap3.anarazel.de>
 <efd47210-cf95-d7ee-e984-a873f76596a8@2ndquadrant.com>
 <0b14e90b-1985-bb04-0261-2fcdc28755ab@2ndquadrant.com>
 <9cdbe694-1b49-0c61-b0b2-d95ef2b3cb31@2ndquadrant.com>
Cc: Robert Haas <robertmhaas@gmail.com>, John Gorman <johngorman2@gmail.com>,
 pgsql-hackers <pgsql-hackers@postgresql.org>
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Message-ID: <98735a6c-c376-86bb-8cb2-1878a0c13bdb@2ndquadrant.com>
Date: Sun, 27 Nov 2016 19:42:58 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
 Thunderbird/45.4.0
MIME-Version: 1.0
In-Reply-To: <9cdbe694-1b49-0c61-b0b2-d95ef2b3cb31@2ndquadrant.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

On 11/27/2016 07:25 PM, Petr Jelinek wrote:
> On 15/11/16 01:44, Tomas Vondra wrote:
>> Attached is v6 of the patch series, fixing most of the points:
>>
>> * common bits (valgrind/randomization/wipe) moved to memdebug.h/c
>>
>> Instead of introducing a new header file, I've added the prototypes to
>> memdebug.h (which was already used for the valgrind stuff anyway), and
>> the implementations to a new memdebug.c file. Not sure what you meant by
>> "static inlines" though.
>
> I think Andres wanted to put the implementation to the static inline
> functions directly in the header (see parts of pg_list or how atomics
> work), but I guess you way works too.
>

I see. Well turning that into static inlines just like in pg_list is 
possible. I guess the main reason is performance - for pg_list that 
probably makes sense, but the memory randomization/valgrind stuff is 
only ever used for debugging, which already does a lot of expensive 
stuff anyway.

>>
>> I've however kept SlabContext->freelist as an array, because there may
>> be many blocks with the same number of free chunks, in which case moving
>> the block in the list would be expensive. This way it's simply
>> dlist_delete + dlist_push.
>>
>
> +1
>
> I get mxact isolation test failures in test_decoding with this version
> of patch:
>   step s0w: INSERT INTO do_write DEFAULT VALUES;
> + WARNING:  problem in slab TXN: number of free chunks 33 in block
> 0x22beba0 does not match bitmap 34
>   step s0start: SELECT data FROM
> pg_logical_slot_get_changes('isolation_slot', NULL, NULL,
> 'include-xids', 'false');
>   data
> and
>   step s0alter: ALTER TABLE do_write ADD column ts timestamptz;
>   step s0w: INSERT INTO do_write DEFAULT VALUES;
> + WARNING:  problem in slab TXN: number of free chunks 33 in block
> 0x227c3f0 does not match bitmap 34
>   step s0start: SELECT data FROM
> pg_logical_slot_get_changes('isolation_slot', NULL, NULL,
> 'include-xids', 'false');
>   data
>

D'oh! I believe this is a simple thinko in SlabCheck, which iterates 
over chunks like this:

     for (j = 0; j <= slab->chunksPerBlock; j++)
     ...

which is of course off-by-one error (and the 33 vs. 34 error message is 
consistent with this theory).

>
> Also, let's just rename the Gen to Generation.
>

OK.

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers