From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Bruce Momjian <bruce@momjian.us>
To: Andres Freund <andres@anarazel.de>
Cc: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>
Cc: tsunakawa.takay@jp.fujitsu.com
Cc: alvherre@2ndquadrant.com
Cc: robertmhaas@gmail.com
Cc: tgl@sss.pgh.pa.us
Cc: ideriha.takeshi@jp.fujitsu.com
Cc: pgsql-hackers@lists.postgresql.org
Cc: michael.paquier@gmail.com
Cc: david@pgmasters.net
Cc: craig@2ndquadrant.com
Subject: Re: Protect syscache from bloating with negative cache entries
Date: Fri, 15 Feb 2019 16:58:40 +0100
Message-ID: <b79b09bd-fa8f-8501-0781-fa4e3619bd54@2ndquadrant.com> (raw)
In-Reply-To: <20190214144647.GA19578@momjian.us>
References: <20190207.211845.263152487.horiguchi.kyotaro@lab.ntt.co.jp>
<0A3221C70F24FB45833433255569204D1FB972A6@G01JPEXMBYT05>
<20190212.203628.118792892.horiguchi.kyotaro@lab.ntt.co.jp>
<20190213.153114.239737674.horiguchi.kyotaro@lab.ntt.co.jp>
<20190214084010.bdn6tmba2j7szo3m@alap3.anarazel.de>
<20190214144647.GA19578@momjian.us>
On 2/14/19 3:46 PM, Bruce Momjian wrote:
> On Thu, Feb 14, 2019 at 12:40:10AM -0800, Andres Freund wrote:
>> Hi,
>>
>> On 2019-02-13 15:31:14 +0900, Kyotaro HORIGUCHI wrote:
>>> Instead, I added an accounting(?) interface function.
>>>
>>> | MemoryContextGettConsumption(MemoryContext cxt);
>>>
>>> The API returns the current consumption in this memory
>>> context. This allows "real" memory accounting almost without
>>> overhead.
>>
>> That's definitely *NOT* almost without overhead. This adds additional
>> instructions to one postgres' hottest set of codepaths.
>>
>> I think you're not working incrementally enough here. I strongly suggest
>> solving the negative cache entry problem, and then incrementally go from
>> there after that's committed. The likelihood of this patch ever getting
>> merged otherwise seems extremely small.
>
> Agreed --- the patch is going in the wrong direction.
>
I recall endless discussions about memory accounting in the
"memory-bounded hash-aggregate" patch a couple of years ago, and the
overhead was one of the main issues there. So yeah, trying to solve that
problem here is likely to kill this patch (or at least significantly
delay it).
ISTM there's a couple of ways to deal with that:
1) Ignore the memory amounts entirely, and do just time-base eviction.
2) If we want some size thresholds (e.g. to disable eviction for
backends with small caches etc.) use the number of entries instead. I
don't think that's particularly worse that specifying size in MB.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tomas.vondra@2ndquadrant.com, bruce@momjian.us, andres@anarazel.de, horiguchi.kyotaro@lab.ntt.co.jp, tsunakawa.takay@jp.fujitsu.com, alvherre@2ndquadrant.com, robertmhaas@gmail.com, tgl@sss.pgh.pa.us, ideriha.takeshi@jp.fujitsu.com, pgsql-hackers@lists.postgresql.org, michael.paquier@gmail.com, david@pgmasters.net, craig@2ndquadrant.com
Subject: Re: Protect syscache from bloating with negative cache entries
In-Reply-To: <b79b09bd-fa8f-8501-0781-fa4e3619bd54@2ndquadrant.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox