From: Andres Freund <andres@2ndquadrant.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Merlin Moncure <mmoncure@gmail.com>
Cc: Jim Nasby <jim@nasby.net>
Cc: Jeff Janes <jeff.janes@gmail.com>
Cc: Ants Aasma <ants@cybertec.at>
Cc: Atri Sharma <atri.jiit@gmail.com>
Cc: Greg Stark <stark@mit.edu>
Cc: Amit Kapila <amit.kapila@huawei.com>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: Page replacement algorithm in buffer cache
Date: Tue, 2 Apr 2013 19:20:41 +0200
Message-ID: <20130402172041.GH2415@alap2.anarazel.de> (raw)
In-Reply-To: <3949.1364921816@sss.pgh.pa.us>
References: <3949.1364921816@sss.pgh.pa.us>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
On 2013-04-02 12:56:56 -0400, Tom Lane wrote:
> Andres Freund <andres@2ndquadrant.com> writes:
> > On 2013-04-02 12:22:03 -0400, Tom Lane wrote:
> >> I agree in general, though I'm not sure the bgwriter process can
> >> reasonably handle this need along with what it's already supposed to be
> >> doing. We may need another background process that is just responsible
> >> for keeping the freelist populated.
>
> > What else is the bgwriter actually doing otherwise? Sure, it doesn't put the
> > pages on the freelist, but otherwise its trying to do the above isn't it?
>
> I think it will be problematic to tie buffer-undirtying to putting both
> clean and dirty buffers into the freelist. It might chance to work all
> right to use one scan process for both, but I'm afraid it's more likely
> that we'd end up either overserving one goal or underserving the other.
Hm. I had imagined that we would only ever put clean buffers into the
freelist and that we would never write out a buffer that we don't need
for a new page. I don't see much point in randomly writing out buffers
that won't be needed for something else soon. Currently we can't do much
better than basically undirtying random buffers since we don't really know
which page will reach a usagecount of zero since bgwriter doesn't
manipulate usagecounts.
One other scenario I can see is the problem of strategy buffer reusage
of dirtied pages (hint bits, pruning) during seqscans where we would
benefit from pages being written out fast, but I can't imagine that that
could be handled very well by something like the bgwriter?
Am I missing something?
> Also note the entire design of BgBufferSync right now is predicated on
> the assumption that the rate of motion of the scan strategy point
> reflects the rate at which new buffers are needed. If backends are
> supposed to always get new buffers from the freelist, that logic becomes
> circular: the bgwriter would be watching itself. Perhaps we can
> refactor the feedback control loop logic so that the buffer scan rate is
> driven by changes in the length of the freelist, but I'm not sure
> exactly what the consequences would be.
Yea, that will definitely need refactoring. What I am imagining is that
that pacing is basically built ontop of a few StragetyControl variables
like:
* number of buffers from the freelist
* number of buffers acquired by backend because freelist was empty
* number of buffers written out by backend because freelist was empty
Those should be pretty cheap to maintain and should be enough to
implement sensible pacing for bgwriter.
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: andres@2ndquadrant.com, tgl@sss.pgh.pa.us, robertmhaas@gmail.com, mmoncure@gmail.com, jim@nasby.net, jeff.janes@gmail.com, ants@cybertec.at, atri.jiit@gmail.com, stark@mit.edu, amit.kapila@huawei.com
Subject: Re: Page replacement algorithm in buffer cache
In-Reply-To: <20130402172041.GH2415@alap2.anarazel.de>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox