From: Amit Kapila <amit.kapila@huawei.com>
To: "'Robert Haas'" <robertmhaas@gmail.com>
CC: "'Greg Smith'" <greg@2ndquadrant.com>, <pgsql-hackers@postgresql.org>
References: 
 <CAM-w4HM2A0rAGN-KRyDQfcH+8YsDvCJ4x=XYAsS+G2kp56qOjQ@mail.gmail.com>
	<CAHyXU0xMFvr2eiR+2Lv=dj+UON+muei_pjbUP2BMetiR-UTASw@mail.gmail.com>
	<29181.1363981925@sss.pgh.pa.us>
	<CAHyXU0yKC-HWk9hhBjUCPsnPMqhskvu_MEqB2cvmRu0d0+BsCw@mail.gmail.com>
	<29748.1363983378@sss.pgh.pa.us>
	<CAHyXU0wKmB5WXS+hAt2QmGfpTNdsz4Dx1nR3GCKvsNBOzt2ZbQ@mail.gmail.com>
	<CA+CSw_sMmHt-Up+SiyL4OPkrgfECQVnMeOpyJz+=M7x=DNTzWg@mail.gmail.com>
	<CAMkU=1zVSyNRR_AQh4j_w6h37+qyvAz4fY8A+QP8e0dsuBg7Fw@mail.gmail.com>
	<CAHyXU0xBXzuGAD8jJ82t2OEDyMmuY1=NwWYGwKte6xqsWWdU6w@mail.gmail.com>
	<20130401210920.GC7244@alap2.anarazel.de>
	<CAHyXU0zvW_Yv3Q5i+5DzTjaGmvjUZS+vRiVGq-WwNAj8A=iMmg@mail.gmail.com>
	<515A1393.9090908@nasby.net>
	<CAHyXU0zFmRFBJeGtWuwxey-8brcxS60Ve0vqT=wW+1v8P3E7UA@mail.gmail.com>
	<CA+TgmobVPCZ60eyDvwb6pfNnBuzanAxF8K8ey-Dd3t02E7fD+g@mail.gmail.com>
	<CAHyXU0ygrf+Fz52sxYHkRUbZJR+CG4GcK5a14Rsb+zsVh2Q8gg@mail.gmail.com>
	<CA+TgmoaB7RyE6stTO45uGT8TiC5VBGZpB+VDrbaeqxm80nT4Fw@mail.gmail.com>
	<515CDC20.5090005@2ndQuadrant.com>
	<CA+Tgmob37KgvKQCCNiwpNAYPD5fmkyScq4Q9oXCqW+U5P4_L_g@mail.gmail.com>
	<515e5d78.037ecd0a.4d15.ffff9299SMTPIN_ADDED_BROKEN@mx.google.com>
	<CA+TgmoZ7FrPawgYZ4f0DDUajMSYcFwODhNya74Y7CkudcbqZ_Q@mail.gmail.com>
	<515f91fa.c212cd0a.236a.2ddfSMTPIN_ADDED_BROKEN@mx.google.com>
	<CA+TgmoYhWsz__KtSxm6BuBirE7VR6Qqc_COkbEZTQpk8oom3CA@mail.gmail.com>
In-Reply-To: 
 <CA+TgmoYhWsz__KtSxm6BuBirE7VR6Qqc_COkbEZTQpk8oom3CA@mail.gmail.com>
Subject: Re: Page replacement algorithm in buffer cache
Date: Tue, 9 Apr 2013 12:37:03 +0530
Message-ID: <006f01ce34f0$d6fa8220$84ef8660$@kapila@huawei.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Thread-Index: Ac401ndUKX/uibyySAycMGGCsubq6AAGYqUA
Content-Language: en-us
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org


> -----Original Message-----
> From: Robert Haas [mailto:robertmhaas@gmail.com]
> Sent: Tuesday, April 09, 2013 9:28 AM
> To: Amit Kapila
> Cc: Greg Smith; pgsql-hackers@postgresql.org
> Subject: Re: [HACKERS] Page replacement algorithm in buffer cache
> 
> On Fri, Apr 5, 2013 at 11:08 PM, Amit Kapila <amit.kapila@huawei.com>
> wrote:
> > I still have one more doubt, consider the below scenario for cases
> when we
> > Invalidate buffers during moving to freelist v/s just move to
> freelist
> >
> >    Backend got the buffer from freelist for a request of page-9
> (number 9 is
> > random, just to explain), it still have association with another
> page-10
> >    It needs to add the buffer with new tag (new page association) in
> bufhash
> > table and remove the buffer with oldTag (old page association).
> >
> > The benefit for just moving to freelist is that if we get request of
> same
> > page until somebody else used it for another page, it will save read
> I/O.
> > However on the other side for many cases
> > Backend will need extra partition lock to remove oldTag (which can
> lead to
> > some bottleneck).
> >
> > I think saving read I/O is more beneficial but just not sure if that
> is best
> > as cases might be less for it.
> 
> I think saving read I/O is a lot more beneficial.  I haven't seen
> evidence of a severe bottleneck updating the buffer mapping tables.  I
> have seen some evidence of spinlock-level contention on read workloads
> that fit in shared buffers, because in that case the system can run
> fast enough for the spinlocks protecting the lwlocks to get pretty
> hot.  But if you're doing writes, or if the workload doesn't fit in
> shared buffers, other bottlenecks slow you down enough that this
> doesn't really seem to become much of an issue.
> 
> Also, even if you *can* find some scenario where pushing the buffer
> invalidation into the background is a win, I'm not convinced that
> would justify doing it, because the case where it's a huge loss -
> namely, working set just a tiny bit smaller than shared_buffers - is
> pretty obvious. I don't think we dare fool around with that; the
> townspeople will arrive with pitchforks.
> 
> I believe that the big win here is getting the clock sweep out of the
> foreground so that BufFreelistLock doesn't catch fire.  The buffer
> mapping locks are partitioned and, while it's not like that completely
> gets rid of the contention, it sure does help a lot.  So I would view
> that goal as primary, at least for now.  If we get a first round of
> optimization done in this area, that doesn't preclude further
> improving it in the future.

I agree with you that this can be first step towards improvement.

> > Last time following tests have been executed to validate the results:
> >
> > Test suite - pgbench
> > DB Size - 16 GB
> > RAM     - 24 GB
> > Shared Buffers - 2G, 5G, 7G, 10G
> > Concurrency - 8, 16, 32, 64 clients
> > Pre-warm the buffers before start of test
> >
> > Shall we try for any other scenario's or for initial test of patch
> above are
> > okay.
> 
> Seems like a reasonable place to start.

I shall work on this for first CF of 9.4.


With Regards,
Amit Kapila.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers