pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Amit Kapila <amit.kapila@huawei.com>
To: 'Simon Riggs' <simon@2ndQuadrant.com>
Cc: 'Robert Haas' <robertmhaas@gmail.com>
Cc: 'Andres Freund' <andres@2ndquadrant.com>
Cc: 'Greg Smith' <greg@2ndquadrant.com>
Cc: 'PostgreSQL-development' <pgsql-hackers@postgresql.org>
Subject: Re: Move unused buffers to freelist
Date: Wed, 3 Jul 2013 17:26:02 +0530
Message-ID: <002d01ce77e4$4b015370$e103fa50$@kapila@huawei.com> (raw)
In-Reply-To: <CA+U5nMJNgD351zMNJJLwwK3hAsp4ysWZKiSUuDgohjJn2qgc7g@mail.gmail.com>
References: <51928CDA.7030009@2ndQuadrant.com>
	<5194eb32.85d60e0a.1c17.2253SMTPIN_ADDED_BROKEN@mx.google.com>
	<CA+TgmobBckMYLvqjrsWBfc0KGBp5_eD2sPA_g3KZFN2P8p2JUA@mail.gmail.com>
	<519b1d5f.9025b40a.2d07.ffffed08SMTPIN_ADDED_BROKEN@mx.google.com>
	<CA+TgmobawHrzH0tMCgjC=_zmcJxYUo7ttawm4P2jf_tNi6dhLg@mail.gmail.com>
	<519f060c.cade440a.493f.ffffb080SMTPIN_ADDED_BROKEN@mx.google.com>
	<CA+TgmoYhfcANR4ci01Gtd8YFzca=VTW9dVn9TY4RM+OdYqCPPw@mail.gmail.com>
	<51b033cf.25dc420a.1edd.ffff8fd9SMTPIN_ADDED_BROKEN@mx.google.com>
	<CA+TgmoZVPtSWHRWtgV_TwQE1Su3dT9sJeU+KYq+-XObxjxLbaw@mail.gmail.com>
	<51cada34.c31b0f0a.2927.7bfbSMTPIN_ADDED_BROKEN@mx.google.com>
	<CA+TgmobJm0GHk58nUPRQHCGwY25n1DCkU4ku9aQeczZEjiz9mQ@mail.gmail.com>
	<4174155236987592579@unknownmsgid>
	<CA+U5nMJNgD351zMNJJLwwK3hAsp4ysWZKiSUuDgohjJn2qgc7g@mail.gmail.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>

On Wednesday, July 03, 2013 12:27 PM Simon Riggs wrote:
On 28 June 2013 05:52, Amit Kapila <amit.kapila@huawei.com> wrote:
 
>> As per my understanding Summarization of points raised by you and Andres
>> which this patch should address to have a bigger win:

>> 1. Bgwriter needs to be improved so that it can help in reducing usage
count
>> and finding next victim buffer
>>   (run the clock sweep and add buffers to the free list).
>>2. SetLatch for bgwriter (wakeup bgwriter) when elements in freelist are
>>less.
>>3. Split the workdone globallock (Buffreelist) in StrategyGetBuffer
>>   (a spinlock for the freelist, and an lwlock for the clock sweep).
>>4. Separate processes for writing dirty buffers and moving buffers to
>>freelist
>>5. Bgwriter needs to be more aggressive, logic based on which it
calculates
>>how many buffers it needs to process needs to be improved.
>>6. There can be contention around buffer mapping locks, but we can focus
on
>>it later
>>7. cacheline bouncing around the buffer header spinlocks, is there
anything
>>we can do to reduce this?

>My perspectives here would be

> * BufFreelistLock is a huge issue. Finding a next victim block needs to be
an O(1) operation, yet it is currently much worse than that. Measuring 
> contention on that lock hides that problem, since having shared buffers
lock up for 100ms or more but only occasionally is a huge problem, even if
it 
> doesn't occur frequently enough for the averaged contention to show as an
issue.

  To optimize finding next victim buffer, I am planning to run the clock
sweep in background. Apart from that do you have any idea to make it closer
to O(1)?

With Regards,
Amit Kapila.



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers



view thread (39+ messages)  latest in thread

Message-ID: <002d01ce77e4$4b015370$e103fa50$@kapila@huawei.com>
Permalink:  ../002d01ce77e4$4b015370$e103fa50$@kapila@huawei.com/
Also on:    postgresql.org/message-id/002d01ce77e4$4b015370$e103fa50$@kapila@huawei.com

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: amit.kapila@huawei.com, simon@2ndQuadrant.com, robertmhaas@gmail.com, andres@2ndquadrant.com, greg@2ndquadrant.com
  Subject: Re: Move unused buffers to freelist
  In-Reply-To: <002d01ce77e4$4b015370$e103fa50$@kapila@huawei.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox