Re: Page replacement algorithm in buffer cache

pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  
help / color / mirror / Atom feed

From: Amit Kapila <amit.kapila@huawei.com>
To: 'Jim Nasby' <jim@nasby.net>
Cc: 'Ants Aasma' <ants@cybertec.at>
Cc: 'Merlin Moncure' <mmoncure@gmail.com>
Cc: 'Tom Lane' <tgl@sss.pgh.pa.us>
Cc: 'Atri Sharma' <atri.jiit@gmail.com>
Cc: 'Greg Stark' <stark@mit.edu>
Cc: 'PostgreSQL-development' <pgsql-hackers@postgresql.org>
Subject: Re: Page replacement algorithm in buffer cache
Date: Tue, 2 Apr 2013 11:27:43 +0530
Message-ID: <006301ce2f66$fe9943f0$fbcbcbd0$@kapila@huawei.com> (raw)
In-Reply-To: <515A1474.7030002@nasby.net>
References: <CAOeZVic4HikhmzVD=ZP4JY9g8PgpyiQQOXOELWP=kR+=H1Frgg@mail.gmail.com>
	<514bf302.84d5440a.43f2.ffffe411SMTPIN_ADDED_BROKEN@mx.google.com>
	<08BBC126-D375-4743-B9A6-D1320D05D5C7@gmail.com>
	<514c3395.487a420a.28d1.764eSMTPIN_ADDED_BROKEN@mx.google.com>
	<CAOeZVifZ89tOtcKeC6C2917sVfSyR3xiMenorg8sNR2+y2W=Wg@mail.gmail.com>
	<CA+CSw_se74iDCLswBfF7_Rd1542nhyw-YSmQ+OqDwEzu0rZkSw@mail.gmail.com>
	<3985.1363960959@sss.pgh.pa.us>
	<CAM-w4HM2A0rAGN-KRyDQfcH+8YsDvCJ4x=XYAsS+G2kp56qOjQ@mail.gmail.com>
	<CAOeZVicGOGQZJZKFXFYjm0wDFA63+7cmzzGQ63nP+165-5qiLQ@mail.gmail.com>
	<CAHyXU0xMFvr2eiR+2Lv=dj+UON+muei_pjbUP2BMetiR-UTASw@mail.gmail.com>
	<29181.1363981925@sss.pgh.pa.us>
	<CAHyXU0yKC-HWk9hhBjUCPsnPMqhskvu_MEqB2cvmRu0d0+BsCw@mail.gmail.com>
	<29748.1363983378@sss.pgh.pa.us>
	<CAHyXU0wKmB5WXS+hAt2QmGfpTNdsz4Dx1nR3GCKvsNBOzt2ZbQ@mail.gmail.com>
	<CA+CSw_sMmHt-Up+SiyL4OPkrgfECQVnMeOpyJz+=M7x=DNTzWg@mail.gmail.com>
	<514D29CB.5080600@nasby.net>
	<000001ce27aa$edf71680$c9e54380$@kapila@huawei.com>
	<515A1474.7030002@nas by.net>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>



> -----Original Message-----
> From: Jim Nasby [mailto:jim@nasby.net]
> Sent: Tuesday, April 02, 2013 4:43 AM
> To: Amit Kapila
> Cc: 'Ants Aasma'; 'Merlin Moncure'; 'Tom Lane'; 'Atri Sharma'; 'Greg
> Stark'; 'PostgreSQL-development'
> Subject: Re: [HACKERS] Page replacement algorithm in buffer cache
> 
> On 3/23/13 4:43 AM, Amit Kapila wrote:
> > I have tried one of the idea's : Adding the buffers background writer
> finds
> > reusable to freelist.
> > http://www.postgresql.org/message-
> id/6C0B27F7206C9E4CA54AE035729E9C382852FF97@szxeml509-mbs
> > This can reduce the clock swipe as it can find buffers from freelist.
> 
> That's a nice potential efficiency gain, but it's not the same as
> having a separate bg process charged with keeping pages on the
> freelist. I believe a separate process would be useful in a wider
> variety of workloads, because it's not dependent on stumbling across 0
> count blocks; it would actively work to "produce" zero count blocks
> when none existed and then free-list them.

If bg process work to produce zero count blocks, then wouldn't it give more
priority to
reduce usage count without any demand, or would you like to do the work in
bg process
based on some statistics for buffer usage.

> > It shows performance improvement for read loads when data can be
> contained
> > in shared buffers,
> > but when the data becomes large and (I/O) is involved, it shows some
> dip as
> > well.
> 
> Do you remember off-hand why it slowed down with I/O so I don't have to
> read the whole thread? :) 

You can go through below mail to read some of my observation regarding the
performance improvement
http://www.postgresql.org/message-id/6C0B27F7206C9E4CA54AE035729E9C383BC76A4
C@szxeml509-mbx

For case when I/O is involved you can once check the readings in below mail:
http://www.postgresql.org/message-id/6C0B27F7206C9E4CA54AE035729E9C383BC76A4
C@szxeml509-mbx


> Was it just a matter of it evicting dirty pages sooner than it would
otherwise?

The exact reason was not nailed down, but as it is I/O heavy test, there is
some chance that OS scheduling also would have played role.
The reason I think why OS can be involved, because when we are flushing
buffers internally OS also might change their usage count based on access of
buffer.
The other could be as pointed out by you, early eviction.


With Regards,
Amit Kapila.



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

view thread (82+ messages)  latest in thread

Message-ID: <006301ce2f66$fe9943f0$fbcbcbd0$@kapila@huawei.com>
Permalink:  ../006301ce2f66$fe9943f0$fbcbcbd0$@kapila@huawei.com/
Also on:    postgresql.org/message-id/006301ce2f66$fe9943f0$fbcbcbd0$@kapila@huawei.com

 ·

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: amit.kapila@huawei.com, jim@nasby.net, ants@cybertec.at, mmoncure@gmail.com, tgl@sss.pgh.pa.us, atri.jiit@gmail.com, stark@mit.edu
  Subject: Re: Page replacement algorithm in buffer cache
  In-Reply-To: <006301ce2f66$fe9943f0$fbcbcbd0$@kapila@huawei.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox