From: Andres Freund <andres@2ndquadrant.com>
To: Merlin Moncure <mmoncure@gmail.com>
Cc: Jeff Janes <jeff.janes@gmail.com>
Cc: Ants Aasma <ants@cybertec.at>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Atri Sharma <atri.jiit@gmail.com>
Cc: Greg Stark <stark@mit.edu>
Cc: Amit Kapila <amit.kapila@huawei.com>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: Page replacement algorithm in buffer cache
Date: Mon, 1 Apr 2013 23:09:20 +0200
Message-ID: <20130401210920.GC7244@alap2.anarazel.de> (raw)
In-Reply-To: <CAHyXU0xBXzuGAD8jJ82t2OEDyMmuY1=NwWYGwKte6xqsWWdU6w@mail.gmail.com>
References: <CAM-w4HM2A0rAGN-KRyDQfcH+8YsDvCJ4x=XYAsS+G2kp56qOjQ@mail.gmail.com>
<CAOeZVicGOGQZJZKFXFYjm0wDFA63+7cmzzGQ63nP+165-5qiLQ@mail.gmail.com>
<CAHyXU0xMFvr2eiR+2Lv=dj+UON+muei_pjbUP2BMetiR-UTASw@mail.gmail.com>
<29181.1363981925@sss.pgh.pa.us>
<CAHyXU0yKC-HWk9hhBjUCPsnPMqhskvu_MEqB2cvmRu0d0+BsCw@mail.gmail.com>
<29748.1363983378@sss.pgh.pa.us>
<CAHyXU0wKmB5WXS+hAt2QmGfpTNdsz4Dx1nR3GCKvsNBOzt2ZbQ@mail.gmail.com>
<CA+CSw_sMmHt-Up+SiyL4OPkrgfECQVnMeOpyJz+=M7x=DNTzWg@mail.gmail.com>
<CAMkU=1zVSyNRR_AQh4j_w6h37+qyvAz4fY8A+QP8e0dsuBg7Fw@mail.gmail.com>
<CAHyXU0xBXzuGAD8jJ82t2OEDyMmuY1=NwWYGwKte6xqsWWdU6w@mail.gmail.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
On 2013-04-01 08:28:13 -0500, Merlin Moncure wrote:
> On Sun, Mar 31, 2013 at 1:27 PM, Jeff Janes <jeff.janes@gmail.com> wrote:
> > On Friday, March 22, 2013, Ants Aasma wrote:
> >>
> >> On Fri, Mar 22, 2013 at 10:22 PM, Merlin Moncure <mmoncure@gmail.com>
> >> wrote:
> >> > well if you do a non-locking test first you could at least avoid some
> >> > cases (and, if you get the answer wrong, so what?) by jumping to the
> >> > next buffer immediately. if the non locking test comes good, only
> >> > then do you do a hardware TAS.
> >> >
> >> > you could in fact go further and dispense with all locking in front of
> >> > usage_count, on the premise that it's only advisory and not a real
> >> > refcount. so you only then lock if/when it's time to select a
> >> > candidate buffer, and only then when you did a non locking test first.
> >> > this would of course require some amusing adjustments to various
> >> > logical checks (usage_count <= 0, heh).
> >>
> >> Moreover, if the buffer happens to miss a decrement due to a data
> >> race, there's a good chance that the buffer is heavily used and
> >> wouldn't need to be evicted soon anyway. (if you arrange it to be a
> >> read-test-inc/dec-store operation then you will never go out of
> >> bounds) However, clocksweep and usage_count maintenance is not what is
> >> causing contention because that workload is distributed. The issue is
> >> pinning and unpinning.
> >
> >
> > That is one of multiple issues. Contention on the BufFreelistLock is
> > another one. I agree that usage_count maintenance is unlikely to become a
> > bottleneck unless one or both of those is fixed first (and maybe not even
> > then)
>
> usage_count manipulation is not a bottleneck but that is irrelevant.
> It can be affected by other page contention which can lead to priority
> inversion. I don't be believe there is any reasonable argument that
> sitting and spinning while holding the BufFreelistLock is a good idea.
In my experience the mere fact of (unlockedly, but still) accessing all the
buffer headers can cause noticeable slowdowns in write only/mostly workloads with
big amounts of shmem.
Due to the write only nature large amounts of the buffers have a similar
usagecounts (since they are infrequently touched after the initial insertion)
and there are no free ones around so the search for a buffer frequently runs
through *all* buffer headers multiple times till it decremented all usagecounts
to 0. Then comes a period where free buffers are found easily (since all
usagecounts from the current sweep point onwards are zero). After that it
starts all over.
I now have seen that scenario multiple times :(
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: andres@2ndquadrant.com, mmoncure@gmail.com, jeff.janes@gmail.com, ants@cybertec.at, tgl@sss.pgh.pa.us, atri.jiit@gmail.com, stark@mit.edu, amit.kapila@huawei.com
Subject: Re: Page replacement algorithm in buffer cache
In-Reply-To: <20130401210920.GC7244@alap2.anarazel.de>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox