Subject: Re: Protect syscache from bloating with negative cache entries
To: Bruce Momjian <bruce@momjian.us>, Andres Freund <andres@anarazel.de>
Cc: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>,
 tsunakawa.takay@jp.fujitsu.com, alvherre@2ndquadrant.com,
 robertmhaas@gmail.com, tgl@sss.pgh.pa.us, ideriha.takeshi@jp.fujitsu.com,
 pgsql-hackers@lists.postgresql.org, michael.paquier@gmail.com,
 david@pgmasters.net, craig@2ndquadrant.com
References: <20190207.211845.263152487.horiguchi.kyotaro@lab.ntt.co.jp>
 <0A3221C70F24FB45833433255569204D1FB972A6@G01JPEXMBYT05>
 <20190212.203628.118792892.horiguchi.kyotaro@lab.ntt.co.jp>
 <20190213.153114.239737674.horiguchi.kyotaro@lab.ntt.co.jp>
 <20190214084010.bdn6tmba2j7szo3m@alap3.anarazel.de>
 <20190214144647.GA19578@momjian.us>
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Message-ID: <b79b09bd-fa8f-8501-0781-fa4e3619bd54@2ndquadrant.com>
Date: Fri, 15 Feb 2019 16:58:40 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.4.0
MIME-Version: 1.0
In-Reply-To: <20190214144647.GA19578@momjian.us>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk


On 2/14/19 3:46 PM, Bruce Momjian wrote:
> On Thu, Feb 14, 2019 at 12:40:10AM -0800, Andres Freund wrote:
>> Hi,
>>
>> On 2019-02-13 15:31:14 +0900, Kyotaro HORIGUCHI wrote:
>>> Instead, I added an accounting(?) interface function.
>>>
>>> | MemoryContextGettConsumption(MemoryContext cxt);
>>>
>>> The API returns the current consumption in this memory
>>> context. This allows "real" memory accounting almost without
>>> overhead.
>>
>> That's definitely *NOT* almost without overhead. This adds additional
>> instructions to one postgres' hottest set of codepaths.
>>
>> I think you're not working incrementally enough here. I strongly suggest
>> solving the negative cache entry problem, and then incrementally go from
>> there after that's committed. The likelihood of this patch ever getting
>> merged otherwise seems extremely small.
> 
> Agreed --- the patch is going in the wrong direction.
> 

I recall endless discussions about memory accounting in the
"memory-bounded hash-aggregate" patch a couple of years ago, and the
overhead was one of the main issues there. So yeah, trying to solve that
problem here is likely to kill this patch (or at least significantly
delay it).

ISTM there's a couple of ways to deal with that:

1) Ignore the memory amounts entirely, and do just time-base eviction.

2) If we want some size thresholds (e.g. to disable eviction for
backends with small caches etc.) use the number of entries instead. I
don't think that's particularly worse that specifying size in MB.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services