Message-ID: <6043f5db-d9b1-5fcd-a713-a7ac71fa6fe1@enterprisedb.com>
Date: Wed, 23 Feb 2022 20:19:52 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.5.0
Subject: Re: Use generation context to speed up tuplesorts
Content-Language: en-US
To: Andres Freund <andres@anarazel.de>, David Rowley <dgrowleyml@gmail.com>
Cc: Ronan Dunklau <ronan.dunklau@aiven.io>,
 Julien Rouhaud <rjuju123@gmail.com>,
 PostgreSQL Developers <pgsql-hackers@lists.postgresql.org>,
 Tomas Vondra <tv@fuzzy.cz>
References: 
 <CAApHDvoH4ASzsAOyHcxkuY01Qf++8JJ0paw+03dk+W25tQEcNQ@mail.gmail.com>
 <20220118064539.o3gc257z2vavb66e@jrouhaud>
 <CAApHDvrbe7rwS_pYnT=L7amt0Wp9MtgFLfg2Qwu8Mv1G9tEiow@mail.gmail.com>
 <4377612.LvFx2qVVIh@aivenronan>
 <d24b7d97-7a8f-7416-b932-2c0a781c2409@enterprisedb.com>
 <CAApHDvpnpoGrEcoJT0bQjCR55_zqZgMhd0mmgQ748cyXZoQQ8w@mail.gmail.com>
 <20220223022504.3ghth7eckjnnubxa@alap3.anarazel.de>
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
In-Reply-To: <20220223022504.3ghth7eckjnnubxa@alap3.anarazel.de>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/6043f5db-d9b1-5fcd-a713-a7ac71fa6fe1%40enterprisedb.com>
Precedence: bulk

On 2/23/22 03:25, Andres Freund wrote:
> Hi,
> 
> On 2022-02-18 12:10:51 +1300, David Rowley wrote:
>> The other way I thought to fix it was by changing the logic for when
>> generation blocks are freed.  In the problem case mentioned above, the
>> block being freed is the current block (which was just allocated).  I
>> made some changes to adjust this behaviour so that we no longer free
>> the block when the final chunk is pfree()'d. Instead, that now lingers
>> and can be reused by future allocations, providing they fit inside it.
> 
> That makes sense to me, as long as we keep just one such block.
> 
> 
>> The problem I see with this method is that there still could be some
>> pathological case that causes us to end up storing just a single tuple per
>> generation block.
> 
> Crazy idea: Detect the situation, and recompact. Create a new context, copy
> all the tuples over, delete the old context. That could be a win even in less
> adversarial situations than "a single tuple per generation block".
> 

What about pointers to the chunks in the old memory context?


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company