Subject: Re: Protect syscache from bloating with negative cache entries
To: Robert Haas <robertmhaas@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>,
 Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>,
 Andres Freund <andres@anarazel.de>,
 "Tsunakawa, Takayuki" <tsunakawa.takay@jp.fujitsu.com>,
 Alvaro Herrera <alvherre@2ndquadrant.com>, Bruce Momjian <bruce@momjian.us>,
 "Ideriha, Takeshi" <ideriha.takeshi@jp.fujitsu.com>,
 PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
 Michael Paquier <michael.paquier@gmail.com>,
 David Steele <david@pgmasters.net>, Craig Ringer <craig@2ndquadrant.com>
References: 
 <CA+TgmoZXw+SwK_9Tp=wLqZDstW_X+Ant=rd7K+q4zmYONPuL=w@mail.gmail.com>
 <20190225.152653.211323929.horiguchi.kyotaro@lab.ntt.co.jp>
 <CA+Tgmoa2b-LUF9h3wugD9ZA5MP0xyu2kJYHC9L6sdLywNSmhBQ@mail.gmail.com>
 <20190301.173245.25820912.horiguchi.kyotaro@lab.ntt.co.jp>
 <CA+TgmobXDWanVtVjLpayQnWfyTd49ma4j1q71ek_Ce=Z9A2UBg@mail.gmail.com>
 <1752.1551903431@sss.pgh.pa.us>
 <b950e760-8ea3-f2f4-4826-c7c50f6a16e6@2ndquadrant.com>
 <CA+TgmoYyWpw=K6zm1pncwd2jKWU6QifepM9pot1yfZBpYrPz6g@mail.gmail.com>
 <0846ed27-2732-0f1b-e285-6380dfbc17e1@2ndquadrant.com>
 <CA+Tgmobjc7EnHePr4T-gsYdj8tvqY4a3L7wxA7SjH7UC16Q4pw@mail.gmail.com>
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Message-ID: <1d2d403c-a1dd-b6c9-d599-e94204442a15@2ndquadrant.com>
Date: Thu, 7 Mar 2019 16:10:48 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.5.1
MIME-Version: 1.0
In-Reply-To: 
 <CA+Tgmobjc7EnHePr4T-gsYdj8tvqY4a3L7wxA7SjH7UC16Q4pw@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 3/7/19 4:01 PM, Robert Haas wrote:
> On Thu, Mar 7, 2019 at 9:49 AM Tomas Vondra
> <tomas.vondra@2ndquadrant.com> wrote:
>> I don't think this shows any regression, but perhaps we should do a
>> microbenchmark isolating the syscache entirely?
> 
> Well, if we need the LRU list, then yeah I think a microbenchmark
> would be a good idea to make sure we really understand what the impact
> of that is going to be.  But if we don't need it and can just remove
> it then we don't.
> 
>> What I had in mind is more along these lines:
>>
>> (a) track number of active syscache entries (increment when adding a new
>> one, decrement when evicting one)
>>
>> (b) track peak number of active syscache entries
>>
>> (c) after clock-sweep, if (peak > K*active) where K=2 or K=4 or so, do a
>> memory context swap, i.e. create a new context, copy active entries over
>> and destroy the old one
>>
>> That would at least free() the memory. Of course, the syscache entries
>> may have different sizes, so tracking just numbers of entries is just an
>> approximation. But I think it'd be enough.
> 
> Yeah, that could be done.  I'm not sure how expensive it would be, and
> I'm also not sure how much more effective it would be than what's
> currently proposed in terms of actually freeing memory.  If you free
> enough dead syscache entries, you might manage to give some memory
> back to the OS: after all, there may be some locality there.  And even
> if you don't, you'll at least prevent further growth, which might be
> good enough.
> 

I have my doubts about that happening in practice. It might happen for
some workloads, but I think the locality is rather unpredictable.

> We could consider doing some version of what has been proposed here
> and the thing you're proposing here could later be implemented on top
> of that.  I mean, evicting entries at all is a prerequisite to
> copy-and-compact.
> 

Sure. I'm not saying the patch must do this to make it committable.

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services