From: Andres Freund <andres@anarazel.de>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Michael Paquier <michael.paquier@gmail.com>
Cc: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>
Cc: David Steele <david@pgmasters.net>
Cc: Jim Nasby <Jim.Nasby@bluetreble.com>
Cc: Craig Ringer <craig@2ndquadrant.com>
Cc: PostgreSQL mailing lists <pgsql-hackers@postgresql.org>
Subject: Re: Protect syscache from bloating with negative cache entries
Date: Fri, 1 Dec 2017 13:45:34 -0800
Message-ID: <20171201214534.dfp2sd427puislll@alap3.anarazel.de> (raw)
In-Reply-To: <3544.1512164423@sss.pgh.pa.us>
References: <20171031.184622.92520047.horiguchi.kyotaro@lab.ntt.co.jp>
<CAB7nPqSGdesvS-odcP+RRJaBOssPazwKURf1GAUYZED4dKSmiQ@mail.gmail.com>
<CA+TgmobPHMmQEEUFDU6XCSj5ArEfyagS1+wisJ8QAY+EsUNbxg@mail.gmail.com>
<CAB7nPqQvbMc5Exx=Qfx=42Osntqrm-thLLt3wCiLvLaokcsg4Q@mail.gmail.com>
<84820.1512015443@sss.pgh.pa.us>
<CA+TgmoY9qNYtdx8xh463f=82NosFw4PgthFV4UeP7WhvWu_cbQ@mail.gmail.com>
<85587.1512101829@sss.pgh.pa.us>
<CA+TgmoYiztC-M7qf-LVGdG10qN2wvyVUgudtcyeMCkn0UATOqw@mail.gmail.com>
<20171201213113.z4gbol66ipvg5b4o@alap3.anarazel.de>
<3544.1512164423@sss.pgh.pa.us>
On 2017-12-01 16:40:23 -0500, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > On 2017-12-01 16:20:44 -0500, Robert Haas wrote:
> >> Well, yeah, that would be insane. But I think even something very
> >> rough could work well enough. I think our goal should be to eliminate
> >> cache entries that are have gone unused for many *minutes*, and
> >> there's no urgency about getting it to any sort of exact value. For
> >> non-idle backends, using the most recent statement start time as a
> >> proxy would probably be plenty good enough. Idle backends might need
> >> a bit more thought.
>
> > Our timer framework is flexible enough that we can install a
> > once-a-minute timer without much overhead. That timer could increment a
> > 'cache generation' integer. Upon cache access we write the current
> > generation into relcache / syscache (and potentially also plancache?)
> > entries. Not entirely free, but cheap enough. In those once-a-minute
> > passes entries that haven't been touched in X cycles get pruned.
>
> I have no faith in either of these proposals, because they both assume
> that the problem only arises over the course of many minutes. In the
> recent complaint about pg_dump causing relcache bloat, it probably does
> not take nearly that long for the bloat to occur.
To me that's a bit of a different problem than what I was discussing
here. It also actually doesn't seem that hard - if your caches are
growing fast, you'll continually get hash-resizing of the
various. Adding cache-pruning to the resizing code doesn't seem hard,
and wouldn't add meaningful overhead.
Greetings,
Andres Freund
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: andres@anarazel.de, tgl@sss.pgh.pa.us, robertmhaas@gmail.com, michael.paquier@gmail.com, horiguchi.kyotaro@lab.ntt.co.jp, david@pgmasters.net, Jim.Nasby@bluetreble.com, craig@2ndquadrant.com
Subject: Re: Protect syscache from bloating with negative cache entries
In-Reply-To: <20171201214534.dfp2sd427puislll@alap3.anarazel.de>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox