pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Jim Nasby <jim@nasby.net>
To: Ants Aasma <ants@cybertec.at>
Cc: Merlin Moncure <mmoncure@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Atri Sharma <atri.jiit@gmail.com>
Cc: Greg Stark <stark@mit.edu>
Cc: Amit Kapila <amit.kapila@huawei.com>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: Page replacement algorithm in buffer cache
Date: Mon, 01 Apr 2013 17:56:19 -0500
Message-ID: <515A1093.8090403@nasby.net> (raw)
In-Reply-To: <CA+CSw_tHoPr8p0HDBOOCvJeNV7n1AgQMFP8zG=T_5MYbWOKVmw@mail.gmail.com>
References: <CAOeZVic4HikhmzVD=ZP4JY9g8PgpyiQQOXOELWP=kR+=H1Frgg@mail.gmail.com>
	<08BBC126-D375-4743-B9A6-D1320D05D5C7@gmail.com>
	<514c3395.487a420a.28d1.764eSMTPIN_ADDED_BROKEN@mx.google.com>
	<CAOeZVifZ89tOtcKeC6C2917sVfSyR3xiMenorg8sNR2+y2W=Wg@mail.gmail.com>
	<CA+CSw_se74iDCLswBfF7_Rd1542nhyw-YSmQ+OqDwEzu0rZkSw@mail.gmail.com>
	<3985.1363960959@sss.pgh.pa.us>
	<CAM-w4HM2A0rAGN-KRyDQfcH+8YsDvCJ4x=XYAsS+G2kp56qOjQ@mail.gmail.com>
	<CAOeZVicGOGQZJZKFXFYjm0wDFA63+7cmzzGQ63nP+165-5qiLQ@mail.gmail.com>
	<CAHyXU0xMFvr2eiR+2Lv=dj+UON+muei_pjbUP2BMetiR-UTASw@mail.gmail.com>
	<29181.1363981925@sss.pgh.pa.us>
	<CAHyXU0yKC-HWk9hhBjUCPsnPMqhskvu_MEqB2cvmRu0d0+BsCw@mail.gmail.com>
	<29748.1363983378@sss.pgh.pa.us>
	<CAHyXU0wKmB5WXS+hAt2QmGfpTNdsz4Dx1nR3GCKvsNBOzt2ZbQ@mail.gmail.com>
	<CA+CSw_sMmHt-Up+SiyL4OPkrgfECQVnMeOpyJz+=M7x=DNTzWg@mail.gmail.com>
	<514D29CB.5080600@nasby.net>
	<CA+CSw_tHoPr8p0HDBOOCvJeNV7n1AgQMFP8zG=T_5MYbWOKVmw@mail.gmail.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>

On 3/23/13 7:41 AM, Ants Aasma wrote:
> On Sat, Mar 23, 2013 at 6:04 AM, Jim Nasby <jim@nasby.net> wrote:
>> Partitioned clock sweep strikes me as a bad idea... you could certainly get
>> unlucky and end up with a lot of hot stuff in one partition.
>
> Surely that is not worse than having everything in a single partition.
> Given a decent partitioning function it's very highly unlikely to have
> more than a few of the hottest buffers end up in a single partition.

One could argue that it is worse because you've added another layer of unpredictability to performance. If something happens to suddenly put two heavily hit sets in the same partition your previously good performance suddenly tanks.

Maybe that issue isn't real enough to be worth worrying about, but I still think it'd be easier and cleaner to try keeping stuff on the free list first...

>> Another idea that'sbeen broughht up inthe past is to have something in the
>> background keep a minimum number of buffers on the free list. That's how OS
>> VM systems I'm familiar with work, so there's precedent for it.
>>
>> I recall there were at least some theoretical concerns about this, but I
>> don't remember if anyone actually tested the idea.
>
> Yes, having bgwriter do the actual cleaning up seems like a good idea.
> The whole bgwriter infrastructure will need some serious tuning. There
> are many things that could be shifted to background if we knew it
> could keep up, like hint bit setting on dirty buffers being flushed
> out. But again, we have the issue of having good tests to see where
> the changes hurt.

I think at some point we need to stop depending on just bgwriter for all this stuff. I believe it would be much cleaner if we had separate procs for everything we needed (although some synergies might exist; if we wanted to set hint bits during write then bgwriter *is* the logical place to put that).

In this case, I don't think keeping stuff on the free list is close enough to checkpoints that we'd want bgwriter to handle both. At most we might want them to pass some metrics back in forth.
-- 
--
Jim C. Nasby, Data Architect                   jim@nasby.net
512.569.9461 (cell)                         http://jim.nasby.net


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers



view thread (82+ messages)  latest in thread

Message-ID: <515A1093.8090403@nasby.net>
Permalink:  ../515A1093.8090403@nasby.net/
Also on:    postgresql.org/message-id/515A1093.8090403@nasby.net

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: jim@nasby.net, ants@cybertec.at, mmoncure@gmail.com, tgl@sss.pgh.pa.us, atri.jiit@gmail.com, stark@mit.edu, amit.kapila@huawei.com
  Subject: Re: Page replacement algorithm in buffer cache
  In-Reply-To: <515A1093.8090403@nasby.net>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox