From: Jim Nasby <jim@nasby.net>
To: Greg Smith <greg@2ndQuadrant.com>
Cc: pgsql-hackers@postgresql.org
Subject: Re: Page replacement algorithm in buffer cache
Date: Mon, 01 Apr 2013 18:22:04 -0500
Message-ID: <515A169C.7070406@nasby.net> (raw)
In-Reply-To: <514EFB8A.80306@2ndQuadrant.com>
References: <CAOeZVic4HikhmzVD=ZP4JY9g8PgpyiQQOXOELWP=kR+=H1Frgg@mail.gmail.com>
<514bf302.84d5440a.43f2.ffffe411SMTPIN_ADDED_BROKEN@mx.google.com>
<08BBC126-D375-4743-B9A6-D1320D05D5C7@gmail.com>
<514c3395.487a420a.28d1.764eSMTPIN_ADDED_BROKEN@mx.google.com>
<CAOeZVifZ89tOtcKeC6C2917sVfSyR3xiMenorg8sNR2+y2W=Wg@mail.gmail.com>
<CA+CSw_se74iDCLswBfF7_Rd1542nhyw-YSmQ+OqDwEzu0rZkSw@mail.gmail.com>
<514EFB8A.80306@2ndQuadrant.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
On 3/24/13 8:11 AM, Greg Smith wrote:
> On 3/22/13 8:45 AM, Ants Aasma wrote:
>> However, I think the main issue isn't finding new algorithms that are
>> better in some specific circumstances. The hard part is figuring out
>> whether their performance is better in general.
>
> Right. The current page replacement method works as expected. Many frequently accessed pages accumulate a usage count of 5 before the clock sweep hits them. Pages that are accessed once and not again before the clock sweep are evicted. There are several theoretically better ways to approach this. Anyone who hasn't already been working on this for a few years is very unlikely to come up with a brand new idea, one that hasn't already been tested in the academic research.
>
> But the real blocker here isn't ideas, it's creating benchmark workloads to validate any change. Right now I see the most promising work that could lead toward the "performance farm" idea as all of the Jenkins based testing that's been going on recently. Craig Ringer has using that for 2ndQuadrant work here, Peter Eisentraut has been working with it: http://petereisentraut.blogspot.com/2013/01/postgresql-and-jenkins.html and the PostGIS project uses it too. There's some good momentum brewing there.
>
> When we have regular performance testing with a mix of workloads--I have about 10 in mind to start--at that point we could start the testing performance changes to the buffer replacement. Until then this whole area is hard to touch usefully. You have to assume that any tuning you do for one type of workload might accidentally slow another. Starting with a lot of baseline workloads is the only way to move usefully forward when facing that problem.
The other thing I think would be tremendously useful would be the ability to get performance data from systems in the field *without having to install extra stuff or do a special build*. The last point is critical because there are so many places where deviating from a standard package takes an act of Congress.
In this case, if I could run some queries to get stats about clock sweep waits and what-not then I could get our shared buffer size changed on some hosts and see how those changes affect the numbers. But doing this with a non-standard build is pretty much a non-starter.
I know there's been some improvement in this area, but I suspect there's still more to go.
--
Jim C. Nasby, Data Architect jim@nasby.net
512.569.9461 (cell) http://jim.nasby.net
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: jim@nasby.net, greg@2ndQuadrant.com
Subject: Re: Page replacement algorithm in buffer cache
In-Reply-To: <515A169C.7070406@nasby.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox