From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Andres Freund <andres@anarazel.de>
Cc: Petr Jelinek <petr.jelinek@2ndquadrant.com>
Cc: Petr Jelinek <petr@2ndquadrant.com>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: John Gorman <johngorman2@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: PATCH: two slab-like memory allocators
Date: Mon, 27 Feb 2017 15:11:40 +0100
Message-ID: <f33180b4-2396-68ab-4bfd-cd6fe5578f15@2ndquadrant.com> (raw)
In-Reply-To: <20170227111732.vrx5v72ighehwpkf@alap3.anarazel.de>
References: <0fdc7920-f1b4-ea5c-0168-ed4f56448e53@2ndquadrant.com>
<20ca0c69-e8c3-cb50-4456-f03908c672ec@2ndquadrant.com>
<20170209213732.pykbsixeezcthotr@alap3.anarazel.de>
<b72dfa3d-c241-8a3e-97ad-bf214b754aca@2ndquadrant.com>
<20170211013337.iw32bvfqwta634wy@alap3.anarazel.de>
<53def906-2197-bc06-b987-d237f9574800@2ndquadrant.com>
<20170214022221.obhdeyfn34h6hkwj@alap3.anarazel.de>
<19464ac8-e66b-3598-858f-29c749bec35a@2ndquadrant.com>
<b3b2245c-b37a-e1e5-ebc4-857c914bc747@2ndquadrant.com>
<20170224221038.43xf2cyonczzr6hb@alap3.anarazel.de>
<20170227111732.vrx5v72ighehwpkf@alap3.anarazel.de>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
On 02/27/2017 12:17 PM, Andres Freund wrote:
> Hi,
>
> On 2017-02-24 14:10:38 -0800, Andres Freund wrote:
>> I've not yet looked a lot at the next type of context - I want to get
>> this much committed first...
>>
>> I plan to give this another pass sometime this weekend and then push
>> soon.
>
> Before committing I wanted to make sure that
> http://archives.postgresql.org/message-id/32354.1487977458%40sss.pgh.pa.us
> isn't a sufficient fix.
>
> With the test of N=1000000 from this thread I measured both runtime and
> memory usage (note that's peak virtual memory which includes 2GB of
> shared_buffers and such), in assert enabled builds.
>
> master: doesn't finish reasonably
> master+doubly linked list fix: 9390.805 ms VmPeak: 10,969,424 kb
> master+this thread: 6500.293 ms VmPeak: 2,969,528 kB
>
> So the doubly-linked-list fix is great (and much more backpatchable),
> but the patches in this thread are both better runtime *and* peak memory
> usage wise. So that seems like a clear call.
>
Nice, thanks for doing the test.
> I've not yet reviewed the generational allocator yet, but during these
> measurements I get:
> postgres[3970][1]=# select count(*) FROM pg_logical_slot_get_changes('ttt', NULL, NULL);
> WARNING: 01000: problem in Generation Tuples: number of free chunks 0 in block 0x55d011ef10f0 exceeds 7234 allocated
> LOCATION: GenerationCheck, generation.c:693
> WARNING: 01000: problem in Generation Tuples: number of free chunks 0 in block 0x55d01023eba0 exceeds 65532 allocated
> LOCATION: GenerationCheck, generation.c:693
> WARNING: 01000: problem in Generation Tuples: number of free chunks 0 in block 0x55d00d7fb870 exceeds 65532 allocated
> LOCATION: GenerationCheck, generation.c:693
> WARNING: 01000: problem in Generation Tuples: number of free chunks 0 in block 0x55d00cde17b0 exceeds 65531 allocated
> LOCATION: GenerationCheck, generation.c:693
>
> that seems to occur when there's currently in-progress transactions when
> finishing decoding:
>
...
>
> could it be that the test's condition is inverted?
>
Yeah, that seems like the culprit - the condition seems wrong. I wonder
why I haven't seen it during my tests, though ...
> I'll work on getting slab committed first, and then review / edit /
> commit generation.c later. One first note there is that I'm wondering
> if generation.c is a too generic filename.
>
Naming things is hard.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tomas.vondra@2ndquadrant.com, andres@anarazel.de, petr.jelinek@2ndquadrant.com, petr@2ndquadrant.com, robertmhaas@gmail.com, johngorman2@gmail.com
Subject: Re: PATCH: two slab-like memory allocators
In-Reply-To: <f33180b4-2396-68ab-4bfd-cd6fe5578f15@2ndquadrant.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox