From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>
Cc: Andres Freund <andres@anarazel.de>
Cc: Tsunakawa, Takayuki <tsunakawa.takay@jp.fujitsu.com>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: Bruce Momjian <bruce@momjian.us>
Cc: Ideriha, Takeshi <ideriha.takeshi@jp.fujitsu.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Cc: Michael Paquier <michael.paquier@gmail.com>
Cc: David Steele <david@pgmasters.net>
Cc: Craig Ringer <craig@2ndquadrant.com>
Subject: Re: Protect syscache from bloating with negative cache entries
Date: Thu, 7 Mar 2019 16:10:48 +0100
Message-ID: <1d2d403c-a1dd-b6c9-d599-e94204442a15@2ndquadrant.com> (raw)
In-Reply-To: <CA+Tgmobjc7EnHePr4T-gsYdj8tvqY4a3L7wxA7SjH7UC16Q4pw@mail.gmail.com>
References: <CA+TgmoZXw+SwK_9Tp=wLqZDstW_X+Ant=rd7K+q4zmYONPuL=w@mail.gmail.com>
<20190225.152653.211323929.horiguchi.kyotaro@lab.ntt.co.jp>
<CA+Tgmoa2b-LUF9h3wugD9ZA5MP0xyu2kJYHC9L6sdLywNSmhBQ@mail.gmail.com>
<20190301.173245.25820912.horiguchi.kyotaro@lab.ntt.co.jp>
<CA+TgmobXDWanVtVjLpayQnWfyTd49ma4j1q71ek_Ce=Z9A2UBg@mail.gmail.com>
<1752.1551903431@sss.pgh.pa.us>
<b950e760-8ea3-f2f4-4826-c7c50f6a16e6@2ndquadrant.com>
<CA+TgmoYyWpw=K6zm1pncwd2jKWU6QifepM9pot1yfZBpYrPz6g@mail.gmail.com>
<0846ed27-2732-0f1b-e285-6380dfbc17e1@2ndquadrant.com>
<CA+Tgmobjc7EnHePr4T-gsYdj8tvqY4a3L7wxA7SjH7UC16Q4pw@mail.gmail.com>
On 3/7/19 4:01 PM, Robert Haas wrote:
> On Thu, Mar 7, 2019 at 9:49 AM Tomas Vondra
> <tomas.vondra@2ndquadrant.com> wrote:
>> I don't think this shows any regression, but perhaps we should do a
>> microbenchmark isolating the syscache entirely?
>
> Well, if we need the LRU list, then yeah I think a microbenchmark
> would be a good idea to make sure we really understand what the impact
> of that is going to be. But if we don't need it and can just remove
> it then we don't.
>
>> What I had in mind is more along these lines:
>>
>> (a) track number of active syscache entries (increment when adding a new
>> one, decrement when evicting one)
>>
>> (b) track peak number of active syscache entries
>>
>> (c) after clock-sweep, if (peak > K*active) where K=2 or K=4 or so, do a
>> memory context swap, i.e. create a new context, copy active entries over
>> and destroy the old one
>>
>> That would at least free() the memory. Of course, the syscache entries
>> may have different sizes, so tracking just numbers of entries is just an
>> approximation. But I think it'd be enough.
>
> Yeah, that could be done. I'm not sure how expensive it would be, and
> I'm also not sure how much more effective it would be than what's
> currently proposed in terms of actually freeing memory. If you free
> enough dead syscache entries, you might manage to give some memory
> back to the OS: after all, there may be some locality there. And even
> if you don't, you'll at least prevent further growth, which might be
> good enough.
>
I have my doubts about that happening in practice. It might happen for
some workloads, but I think the locality is rather unpredictable.
> We could consider doing some version of what has been proposed here
> and the thing you're proposing here could later be implemented on top
> of that. I mean, evicting entries at all is a prerequisite to
> copy-and-compact.
>
Sure. I'm not saying the patch must do this to make it committable.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tomas.vondra@2ndquadrant.com, robertmhaas@gmail.com, tgl@sss.pgh.pa.us, horiguchi.kyotaro@lab.ntt.co.jp, andres@anarazel.de, tsunakawa.takay@jp.fujitsu.com, alvherre@2ndquadrant.com, bruce@momjian.us, ideriha.takeshi@jp.fujitsu.com, pgsql-hackers@lists.postgresql.org, michael.paquier@gmail.com, david@pgmasters.net, craig@2ndquadrant.com
Subject: Re: Protect syscache from bloating with negative cache entries
In-Reply-To: <1d2d403c-a1dd-b6c9-d599-e94204442a15@2ndquadrant.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox