pg.ddx.io pgsql-hackers@postgresql.org mailing list archive help / color / mirror / Atom feed From: Amit Kapila <amit.kapila@huawei.com>
To: 'Robert Haas' <robertmhaas@gmail.com>
Cc: 'Greg Smith' <greg@2ndquadrant.com>
Cc: pgsql-hackers@postgresql.org
Subject: Re: Page replacement algorithm in buffer cache
Date: Tue, 9 Apr 2013 12:37:03 +0530
Message-ID: <006f01ce34f0$d6fa8220$84ef8660$@kapila@huawei.com> (raw )
In-Reply-To: <CA+TgmoYhWsz__KtSxm6BuBirE7VR6Qqc_COkbEZTQpk8oom3CA@mail.gmail.com >
References: <CAM-w4HM2A0rAGN-KRyDQfcH+8YsDvCJ4x=XYAsS+G2kp56qOjQ@mail.gmail.com >
<CAHyXU0xMFvr2eiR+2Lv=dj+UON+muei_pjbUP2BMetiR-UTASw@mail.gmail.com >
<29181.1363981925@sss.pgh.pa.us >
<CAHyXU0yKC-HWk9hhBjUCPsnPMqhskvu_MEqB2cvmRu0d0+BsCw@mail.gmail.com >
<29748.1363983378@sss.pgh.pa.us >
<CAHyXU0wKmB5WXS+hAt2QmGfpTNdsz4Dx1nR3GCKvsNBOzt2ZbQ@mail.gmail.com >
<CA+CSw_sMmHt-Up+SiyL4OPkrgfECQVnMeOpyJz+=M7x=DNTzWg@mail.gmail.com >
<CAMkU=1zVSyNRR_AQh4j_w6h37+qyvAz4fY8A+QP8e0dsuBg7Fw@mail.gmail.com >
<CAHyXU0xBXzuGAD8jJ82t2OEDyMmuY1=NwWYGwKte6xqsWWdU6w@mail.gmail.com >
<20130401210920.GC7244@alap2.anarazel.de >
<CAHyXU0zvW_Yv3Q5i+5DzTjaGmvjUZS+vRiVGq-WwNAj8A=iMmg@mail.gmail.com >
<515A1393.9090908@nasby.net >
<CAHyXU0zFmRFBJeGtWuwxey-8brcxS60Ve0vqT=wW+1v8P3E7UA@mail.gmail.com >
<CA+TgmobVPCZ60eyDvwb6pfNnBuzanAxF8K8ey-Dd3t02E7fD+g@mail.gmail.com >
<CAHyXU0ygrf+Fz52sxYHkRUbZJR+CG4GcK5a14Rsb+zsVh2Q8gg@mail.gmail.com >
<CA+TgmoaB7RyE6stTO45uGT8TiC5VBGZpB+VDrbaeqxm80nT4Fw@mail.gmail.com >
<515CDC20.5090005@2ndQuadrant.com >
<CA+Tgmob37KgvKQCCNiwpNAYPD5fmkyScq4Q9oXCqW+U5P4_L_g@mail.gmail.com >
<515e5d78.037ecd0a.4d15.ffff9299SMTPIN_ADDED_BROKEN@mx.google.com >
<CA+TgmoZ7FrPawgYZ4f0DDUajMSYcFwODhNya74Y7CkudcbqZ_Q@mail.gmail.com >
<515f91fa.c212cd0a.236a.2ddfSMTPIN_ADDED_BROKEN@mx.google.com >
<CA+TgmoYhWsz__KtSxm6BuBirE7VR6Qqc_COkbEZTQpk8oom3CA@mail.gmail.com >
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
> -----Original Message-----
> From: Robert Haas [mailto:robertmhaas@gmail.com]
> Sent: Tuesday, April 09, 2013 9:28 AM
> To: Amit Kapila
> Cc: Greg Smith; pgsql-hackers@postgresql.org
> Subject: Re: [HACKERS] Page replacement algorithm in buffer cache
>
> On Fri, Apr 5, 2013 at 11:08 PM, Amit Kapila <amit.kapila@huawei.com>
> wrote:
> > I still have one more doubt, consider the below scenario for cases
> when we
> > Invalidate buffers during moving to freelist v/s just move to
> freelist
> >
> > Backend got the buffer from freelist for a request of page-9
> (number 9 is
> > random, just to explain), it still have association with another
> page-10
> > It needs to add the buffer with new tag (new page association) in
> bufhash
> > table and remove the buffer with oldTag (old page association).
> >
> > The benefit for just moving to freelist is that if we get request of
> same
> > page until somebody else used it for another page, it will save read
> I/O.
> > However on the other side for many cases
> > Backend will need extra partition lock to remove oldTag (which can
> lead to
> > some bottleneck).
> >
> > I think saving read I/O is more beneficial but just not sure if that
> is best
> > as cases might be less for it.
>
> I think saving read I/O is a lot more beneficial. I haven't seen
> evidence of a severe bottleneck updating the buffer mapping tables. I
> have seen some evidence of spinlock-level contention on read workloads
> that fit in shared buffers, because in that case the system can run
> fast enough for the spinlocks protecting the lwlocks to get pretty
> hot. But if you're doing writes, or if the workload doesn't fit in
> shared buffers, other bottlenecks slow you down enough that this
> doesn't really seem to become much of an issue.
>
> Also, even if you *can* find some scenario where pushing the buffer
> invalidation into the background is a win, I'm not convinced that
> would justify doing it, because the case where it's a huge loss -
> namely, working set just a tiny bit smaller than shared_buffers - is
> pretty obvious. I don't think we dare fool around with that; the
> townspeople will arrive with pitchforks.
>
> I believe that the big win here is getting the clock sweep out of the
> foreground so that BufFreelistLock doesn't catch fire. The buffer
> mapping locks are partitioned and, while it's not like that completely
> gets rid of the contention, it sure does help a lot. So I would view
> that goal as primary, at least for now. If we get a first round of
> optimization done in this area, that doesn't preclude further
> improving it in the future.
I agree with you that this can be first step towards improvement.
> > Last time following tests have been executed to validate the results:
> >
> > Test suite - pgbench
> > DB Size - 16 GB
> > RAM - 24 GB
> > Shared Buffers - 2G, 5G, 7G, 10G
> > Concurrency - 8, 16, 32, 64 clients
> > Pre-warm the buffers before start of test
> >
> > Shall we try for any other scenario's or for initial test of patch
> above are
> > okay.
>
> Seems like a reasonable place to start.
I shall work on this for first CF of 9.4.
With Regards,
Amit Kapila.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
view thread (82+ messages)
Message-ID: <006f01ce34f0$d6fa8220$84ef8660$@kapila@huawei.com>
Permalink: ../006f01ce34f0$d6fa8220$84ef8660$@kapila@huawei.com/
Also on: postgresql.org/message-id/006f01ce34f0$d6fa8220$84ef8660$@kapila@huawei.com
copy link · copy postgr.es
reply Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: amit.kapila@huawei.com, robertmhaas@gmail.com, greg@2ndquadrant.com
Subject: Re: Page replacement algorithm in buffer cache
In-Reply-To: <006f01ce34f0$d6fa8220$84ef8660$@kapila@huawei.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox