pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Amit Kapila <amit.kapila@huawei.com>
To: 'Robert Haas' <robertmhaas@gmail.com>
Cc: 'Andres Freund' <andres@2ndquadrant.com>
Cc: 'Greg Smith' <greg@2ndquadrant.com>
Cc: 'PostgreSQL-development' <pgsql-hackers@postgresql.org>
Subject: Re: Move unused buffers to freelist
Date: Tue, 6 Aug 2013 11:48:32 +0530
Message-ID: <006e01ce926c$c7768680$56639380$@kapila@huawei.com> (raw)
In-Reply-To: <CA+TgmoaETz+Oh-bZcM2DVW6Z3h_bQ7MD0tXbpEbOspJ2AVTzUw@mail.gmail.com>
References: <51928CDA.7030009@2ndQuadrant.com>
	<5194eb32.85d60e0a.1c17.2253SMTPIN_ADDED_BROKEN@mx.google.com>
	<CA+TgmobBckMYLvqjrsWBfc0KGBp5_eD2sPA_g3KZFN2P8p2JUA@mail.gmail.com>
	<519b1d5f.9025b40a.2d07.ffffed08SMTPIN_ADDED_BROKEN@mx.google.com>
	<CA+TgmobawHrzH0tMCgjC=_zmcJxYUo7ttawm4P2jf_tNi6dhLg@mail.gmail.com>
	<519f060c.cade440a.493f.ffffb080SMTPIN_ADDED_BROKEN@mx.google.com>
	<CA+TgmoYhfcANR4ci01Gtd8YFzca=VTW9dVn9TY4RM+OdYqCPPw@mail.gmail.com>
	<51b033cf.25dc420a.1edd.ffff8fd9SMTPIN_ADDED_BROKEN@mx.google.com>
	<CA+TgmoZVPtSWHRWtgV_TwQE1Su3dT9sJeU+KYq+-XObxjxLbaw@mail.gmail.com>
	<51cada34.c31b0f0a.2927.7bfbSMTPIN_ADDED_BROKEN@mx.google.com>
	<CA+TgmobJm0GHk58nUPRQHCGwY25n1DCkU4ku9aQeczZEjiz9mQ@mail.gmail.com>
	<51cd16b6.a2d8420a.2b95.ffffb090SMTPIN_ADDED_BROKEN@mx.google.com>
	<CA+TgmoaETz+Oh-bZcM2DVW6Z3h_bQ7MD0tXbpEbOspJ2AVTzUw@mail.gmail.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>

On Friday, June 28, 2013 6:20 PM Robert Haas wrote:
> On Fri, Jun 28, 2013 at 12:52 AM, Amit Kapila <amit.kapila@huawei.com>
> wrote:
> > Currently it wakes up based on bgwriterdelay config parameter which
> is by
> > default 200ms, so you means we should
> > think of waking up bgwriter based on allocations and number of
> elements left
> > in freelist?
> 
> I think that's what Andres and I are proposing, yes.
> 
> > As per my understanding Summarization of points raised by you and
> Andres
> > which this patch should address to have a bigger win:
> >
> > 1. Bgwriter needs to be improved so that it can help in reducing
> usage count
> > and finding next victim buffer
> >    (run the clock sweep and add buffers to the free list).
> 
> Check.
     I think one way to handle it is that while moving buffers to freelist,
if we find
     that there are not enough buffers (>= high watermark) which have zero
usage count,  
     then move through buffer list and reduce usage count. Now here I think
it is important
     how do we find that how many times we should circulate the buffer list
to reduce usage count.
     Currently I have kept it proportional to number of times it failed to
move enough buffers to freelist.
 
> > 2. SetLatch for bgwriter (wakeup bgwriter) when elements in freelist
> are
> > less.
> 
> Check. The way to do this is to keep a variable in shared memory in
> the same cache line as the spinlock protecting the freelist, and
> update it when you update the free list.


  Added a new variable freelistLatch in BufferStrategyControl

> > 3. Split the workdone globallock (Buffreelist) in StrategyGetBuffer
> >    (a spinlock for the freelist, and an lwlock for the clock sweep).
> 
> Check.

 Added a new variable freelist_lck in BufferStrategyControl which will be
used to protect freelist.
 Still Buffreelist will be used to protect clock sweep part of
StrategyGetBuffer.

     

> > 4. Separate processes for writing dirty buffers and moving buffers to
> > freelist
> 
> I think this part might be best pushed to a separate patch, although I
> agree we probably need it.
> 
> > 5. Bgwriter needs to be more aggressive, logic based on which it
> calculates
> > how many buffers it needs to process needs to be improved.
> 
> This is basically overlapping with points already made.  I suspect we
> could just get rid of bgwriter_delay, bgwriter_lru_maxpages, and
> bgwriter_lru_multiplier altogether.  The background writer would just
> have a high and a low watermark.  When the number of buffers on the
> freelist drops below the low watermark, the allocating backend sets
> the latch and bgwriter wakes up and begins adding buffers to the
> freelist.  When the number of buffers on the free list reaches the
> high watermark, the background writer goes back to sleep.  Some
> experimentation might be needed to figure out what values are
> appropriate for those watermarks.  In theory this could be a
> configuration knob, but I suspect it's better to just make the system
> tune it right automatically.

Currently in Patch I have used low watermark as 1/6 and high watermark as
1/3 of NBuffers.
Values are hardcoded for now, but I will change to guc's or hash defines.
As far as I can think there is no way to find number of buffers on freelist,
so I had added one more variable to maintain it.
Initially I thought that I could use existing variables firstfreebuffer and
lastfreebuffer to calculate it, but it may not be accurate as
once the buffers are moved to freelist, these don't give exact count.

The main doubt here is what if after traversing all buffers, it didn't find
enough buffers to meet 
high watermark?

Currently I just move out of loop to move buffers and just try to reduce
usage count as explained in point-1

> > 6. There can be contention around buffer mapping locks, but we can
> focus on
> > it later
> > 7. cacheline bouncing around the buffer header spinlocks, is there
> anything
> > we can do to reduce this?
> 
> I think these are points that we should leave for the future.

This is just a WIP patch. I have kept older code in comments. I need to
further refine it and collect performance data.
I had prepared one script (perf_buff_mgmt.sh) to collect performance data
for different shared buffers/scalefactor/number_of_clients

Top level points which still needs to be taken care:
1. Choose Optimistically used buffer in StrategyGetBuffer(). Refer Simon's
Patch:
   https://commitfest.postgresql.org/action/patch_view?id=743
2. Don't bump the usage count on every time buffer is pinned. This idea I
got when reading archives about 
   improvements in this area.

With Regards,
Amit Kapila.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Attachments:

  [application/octet-stream] changed_freelist_mgmt.patch (16.9K, ../006e01ce926c$c7768680$56639380$@kapila@huawei.com/2-changed_freelist_mgmt.patch)
  download | inline diff:
*** a/src/backend/postmaster/bgwriter.c
--- b/src/backend/postmaster/bgwriter.c
***************
*** 230,235 **** BackgroundWriterMain(void)
--- 230,242 ----
  	prev_hibernate = false;
  
  	/*
+ 	 * Initialize the freelist latch. ToDo, this needs to be done under
+ 	 * spinlock which will be used to protect freelist.
+ 	 */
+ 
+ 	StrategyInitFreeListLatch(&MyProc->procLatch);
+ 
+ 	/*
  	 * Loop forever
  	 */
  	for (;;)
***************
*** 259,265 **** BackgroundWriterMain(void)
  		/*
  		 * Do one cycle of dirty-buffer writing.
  		 */
! 		can_hibernate = BgBufferSync();
  
  		/*
  		 * Send off activity statistics to the stats collector
--- 266,272 ----
  		/*
  		 * Do one cycle of dirty-buffer writing.
  		 */
! 		/*can_hibernate = BgBufferSync();		*/
  
  		/*
  		 * Send off activity statistics to the stats collector
***************
*** 276,281 **** BackgroundWriterMain(void)
--- 283,296 ----
  		}
  
  		/*
+ 		 * Sleep untill signalled by backend.
+ 		 */
+ 		WaitLatch(&MyProc->procLatch, WL_LATCH_SET | WL_POSTMASTER_DEATH, -1);
+ 
+ 		BgBufferSyncAndMoveBuffersToFreelist();
+ 
+ 
+ 		/*
  		 * Sleep until we are signaled or BgWriterDelay has elapsed.
  		 *
  		 * Note: the feedback control loop in BgBufferSync() expects that we
***************
*** 285,293 **** BackgroundWriterMain(void)
  		 * down with latch events that are likely to happen frequently during
  		 * normal operation.
  		 */
! 		rc = WaitLatch(&MyProc->procLatch,
  					   WL_LATCH_SET | WL_TIMEOUT | WL_POSTMASTER_DEATH,
! 					   BgWriterDelay /* ms */ );
  
  		/*
  		 * If no latch event and BgBufferSync says nothing's happening, extend
--- 300,308 ----
  		 * down with latch events that are likely to happen frequently during
  		 * normal operation.
  		 */
! 		/*rc = WaitLatch(&MyProc->procLatch,
  					   WL_LATCH_SET | WL_TIMEOUT | WL_POSTMASTER_DEATH,
! 					   BgWriterDelay /* ms */ /*);*/
  
  		/*
  		 * If no latch event and BgBufferSync says nothing's happening, extend
***************
*** 307,323 **** BackgroundWriterMain(void)
  		 * for two consecutive cycles.	Also, we mitigate any possible
  		 * consequences of a missed wakeup by not hibernating forever.
  		 */
! 		if (rc == WL_TIMEOUT && can_hibernate && prev_hibernate)
  		{
  			/* Ask for notification at next buffer allocation */
! 			StrategyNotifyBgWriter(&MyProc->procLatch);
  			/* Sleep ... */
! 			rc = WaitLatch(&MyProc->procLatch,
  						   WL_LATCH_SET | WL_TIMEOUT | WL_POSTMASTER_DEATH,
  						   BgWriterDelay * HIBERNATE_FACTOR);
  			/* Reset the notification request in case we timed out */
! 			StrategyNotifyBgWriter(NULL);
! 		}
  
  		/*
  		 * Emergency bailout if postmaster has died.  This is to avoid the
--- 322,338 ----
  		 * for two consecutive cycles.	Also, we mitigate any possible
  		 * consequences of a missed wakeup by not hibernating forever.
  		 */
! 		/*if (rc == WL_TIMEOUT && can_hibernate && prev_hibernate)
  		{
  			/* Ask for notification at next buffer allocation */
! 			/*StrategyNotifyBgWriter(&MyProc->procLatch);
  			/* Sleep ... */
! 			/*rc = WaitLatch(&MyProc->procLatch,
  						   WL_LATCH_SET | WL_TIMEOUT | WL_POSTMASTER_DEATH,
  						   BgWriterDelay * HIBERNATE_FACTOR);
  			/* Reset the notification request in case we timed out */
! 			/*StrategyNotifyBgWriter(NULL);
! 		}*/
  
  		/*
  		 * Emergency bailout if postmaster has died.  This is to avoid the
*** a/src/backend/storage/buffer/bufmgr.c
--- b/src/backend/storage/buffer/bufmgr.c
***************
*** 1633,1638 **** BgBufferSync(void)
--- 1633,1725 ----
  	return (bufs_to_lap == 0 && recent_alloc == 0);
  }
  
+ void
+ BgBufferSyncAndMoveBuffersToFreelist(void)
+ {
+ 	uint32	next_to_clean;
+ 	uint32	save_next_to_clean;
+ 	uint32	num_to_free;
+ 	int			num_written;
+ 	bool		scanned_buflist = false;
+ 	int		i;
+ 	volatile BufferDesc *bufHdr;
+ 	/* 
+ 	 * used to track how many times buffer list is scanned without
+ 	 * freeing required number of buffers
+ 	 */
+ 	static uint16	num_times_buflist_scanned = 0;
+ 
+ 	StrategySyncStartAndEnd(&next_to_clean, &num_to_free);
+ 
+ 	/* 
+ 	 * save the next victim buffer, so that it can be used later
+ 	 * as a start point to reduce the usage count of buffers.
+ 	 */
+ 	save_next_to_clean = next_to_clean;
+ 
+ 	/* Make sure we can handle the pin inside SyncOneBuffer */
+ 	ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
+ 
+ 	num_written = 0;
+ 
+ 	/* Execute the LRU scan */
+ 	while (num_to_free > 0)
+ 	{
+ 		int			buffer_state = SyncOneBuffer(next_to_clean, true);
+ 
+ 		bufHdr = &BufferDescriptors[next_to_clean];
+ 		if (++next_to_clean >= NBuffers)
+ 		{
+ 			next_to_clean = 0;
+ 			/* 
+ 			 * if buflist is scanned and we didn't find enough buffers
+ 			 * exit the loop, it can try to free after some more activity.
+ 			 */
+ 			if (scanned_buflist)
+ 				break;
+ 			scanned_buflist = true;
+ 		}
+ 		if (buffer_state & BUF_WRITTEN)
+ 			++num_written;
+ 		if (buffer_state & BUF_REUSABLE)
+ 		{
+ 			if (StrategyMoveBufferToFreeListEnd (bufHdr))
+ 				num_to_free--;
+ 		}                                                            
+ 	}
+ 
+ 	BgWriterStats.m_buf_written_clean += num_written;
+ 
+ 	/*
+ 	  * try to reduce the usage count if not able to free required
+ 	  * number of buffers. It will traverse proportional to number of times
+ 	  * it failed to free required number of buffers.
+ 	  */
+ 
+ 	if(scanned_buflist && num_to_free != 0)
+ 	{
+ 		++num_times_buflist_scanned;
+ 		for (i = 0; i < num_times_buflist_scanned * NBuffers; i++)
+ 		{
+ 			bufHdr = &BufferDescriptors[save_next_to_clean];
+ 
+ 			if (++save_next_to_clean >= NBuffers)
+ 				save_next_to_clean = 0;
+ 
+ 			 /*
+ 		 	  * If the buffer is not-pinned and has a nonzero usage_count, decrement 
+ 		 	  * the usage_count and keep scanning.
+ 		 	  */
+ 			LockBufHdr(bufHdr);
+ 			if (bufHdr->refcount == 0 && bufHdr->usage_count > 0)
+ 				bufHdr->usage_count--;
+ 			UnlockBufHdr(bufHdr);
+ 		}
+ 	}
+ 	else
+ 		num_times_buflist_scanned = 0;
+ }
+ 
  /*
   * SyncOneBuffer -- process a single buffer during syncing.
   *
*** a/src/backend/storage/buffer/freelist.c
--- b/src/backend/storage/buffer/freelist.c
***************
*** 29,34 **** typedef struct
--- 29,35 ----
  
  	int			firstFreeBuffer;	/* Head of list of unused buffers */
  	int			lastFreeBuffer; /* Tail of list of unused buffers */
+ 	int			numFreeListBuffers; /* number of buffers on freelist */
  
  	/*
  	 * NOTE: lastFreeBuffer is undefined when firstFreeBuffer is -1 (that is,
***************
*** 42,47 **** typedef struct
--- 43,52 ----
  	uint32		completePasses; /* Complete cycles of the clock sweep */
  	uint32		numBufferAllocs;	/* Buffers allocated since last reset */
  
+ 	Latch	   *freelistLatch;  /* Latch to wake bgwriter */
+ 	/* protects freelist variables (firstFreeBuffer, lastFreeBuffer, numFreeListBuffers, BufferDesc->freeNext)*/
+ 	slock_t	     freelist_lck;
+ 
  	/*
  	 * Notification latch, or NULL if none.  See StrategyNotifyBgWriter.
  	 */
***************
*** 112,118 **** volatile BufferDesc *
  StrategyGetBuffer(BufferAccessStrategy strategy, bool *lock_held)
  {
  	volatile BufferDesc *buf;
! 	Latch	   *bgwriterLatch;
  	int			trycounter;
  
  	/*
--- 117,123 ----
  StrategyGetBuffer(BufferAccessStrategy strategy, bool *lock_held)
  {
  	volatile BufferDesc *buf;
! 	/*Latch	   *bgwriterLatch;*/
  	int			trycounter;
  
  	/*
***************
*** 130,144 **** StrategyGetBuffer(BufferAccessStrategy strategy, bool *lock_held)
  	}
  
  	/* Nope, so lock the freelist */
! 	*lock_held = true;
! 	LWLockAcquire(BufFreelistLock, LW_EXCLUSIVE);
  
  	/*
  	 * We count buffer allocation requests so that the bgwriter can estimate
  	 * the rate of buffer consumption.	Note that buffers recycled by a
  	 * strategy object are intentionally not counted here.
  	 */
! 	StrategyControl->numBufferAllocs++;
  
  	/*
  	 * If bgwriterLatch is set, we need to waken the bgwriter, but we should
--- 135,149 ----
  	}
  
  	/* Nope, so lock the freelist */
! 	/**lock_held = true;
! 	LWLockAcquire(BufFreelistLock, LW_EXCLUSIVE);*/
  
  	/*
  	 * We count buffer allocation requests so that the bgwriter can estimate
  	 * the rate of buffer consumption.	Note that buffers recycled by a
  	 * strategy object are intentionally not counted here.
  	 */
! 	/*StrategyControl->numBufferAllocs++;*/
  
  	/*
  	 * If bgwriterLatch is set, we need to waken the bgwriter, but we should
***************
*** 146,159 **** StrategyGetBuffer(BufferAccessStrategy strategy, bool *lock_held)
  	 * is annoyingly tedious, but it happens at most once per bgwriter cycle,
  	 * so the performance hit is minimal.
  	 */
! 	bgwriterLatch = StrategyControl->bgwriterLatch;
  	if (bgwriterLatch)
  	{
  		StrategyControl->bgwriterLatch = NULL;
  		LWLockRelease(BufFreelistLock);
  		SetLatch(bgwriterLatch);
  		LWLockAcquire(BufFreelistLock, LW_EXCLUSIVE);
! 	}
  
  	/*
  	 * Try to get a buffer from the freelist.  Note that the freeNext fields
--- 151,169 ----
  	 * is annoyingly tedious, but it happens at most once per bgwriter cycle,
  	 * so the performance hit is minimal.
  	 */
! 	/*bgwriterLatch = StrategyControl->bgwriterLatch;
  	if (bgwriterLatch)
  	{
  		StrategyControl->bgwriterLatch = NULL;
  		LWLockRelease(BufFreelistLock);
  		SetLatch(bgwriterLatch);
  		LWLockAcquire(BufFreelistLock, LW_EXCLUSIVE);
! 	}*/
! 
! 	SpinLockAcquire(&StrategyControl->freelist_lck);
! 
! 	if (StrategyControl->numFreeListBuffers < NBuffers/6)
! 		SetLatch(StrategyControl->freelistLatch);
  
  	/*
  	 * Try to get a buffer from the freelist.  Note that the freeNext fields
***************
*** 169,174 **** StrategyGetBuffer(BufferAccessStrategy strategy, bool *lock_held)
--- 179,185 ----
  		/* Unconditionally remove buffer from freelist */
  		StrategyControl->firstFreeBuffer = buf->freeNext;
  		buf->freeNext = FREENEXT_NOT_IN_LIST;
+ 		--StrategyControl->numFreeListBuffers;
  
  		/*
  		 * If the buffer is pinned or has a nonzero usage_count, we cannot use
***************
*** 182,194 **** StrategyGetBuffer(BufferAccessStrategy strategy, bool *lock_held)
--- 193,213 ----
  		{
  			if (strategy != NULL)
  				AddBufferToRing(strategy, buf);
+ 			SpinLockRelease(&StrategyControl->freelist_lck);
+ 			*lock_held = false;
  			return buf;
  		}
  		UnlockBufHdr(buf);
  	}
  
+ 	SpinLockRelease(&StrategyControl->freelist_lck);
+ 
  	/* Nothing on the freelist, so run the "clock sweep" algorithm */
  	trycounter = NBuffers;
+ 
+ 	*lock_held = true;
+ 	LWLockAcquire(BufFreelistLock, LW_EXCLUSIVE);
+ 	
  	for (;;)
  	{
  		buf = &BufferDescriptors[StrategyControl->nextVictimBuffer];
***************
*** 196,202 **** StrategyGetBuffer(BufferAccessStrategy strategy, bool *lock_held)
  		if (++StrategyControl->nextVictimBuffer >= NBuffers)
  		{
  			StrategyControl->nextVictimBuffer = 0;
! 			StrategyControl->completePasses++;
  		}
  
  		/*
--- 215,221 ----
  		if (++StrategyControl->nextVictimBuffer >= NBuffers)
  		{
  			StrategyControl->nextVictimBuffer = 0;
! 			/*StrategyControl->completePasses++;*/
  		}
  
  		/*
***************
*** 241,247 **** StrategyGetBuffer(BufferAccessStrategy strategy, bool *lock_held)
  void
  StrategyFreeBuffer(volatile BufferDesc *buf)
  {
! 	LWLockAcquire(BufFreelistLock, LW_EXCLUSIVE);
  
  	/*
  	 * It is possible that we are told to put something in the freelist that
--- 260,266 ----
  void
  StrategyFreeBuffer(volatile BufferDesc *buf)
  {
! 	SpinLockAcquire(&StrategyControl->freelist_lck);
  
  	/*
  	 * It is possible that we are told to put something in the freelist that
***************
*** 253,264 **** StrategyFreeBuffer(volatile BufferDesc *buf)
  		if (buf->freeNext < 0)
  			StrategyControl->lastFreeBuffer = buf->buf_id;
  		StrategyControl->firstFreeBuffer = buf->buf_id;
  	}
  
! 	LWLockRelease(BufFreelistLock);
  }
  
  /*
   * StrategySyncStart -- tell BufferSync where to start syncing
   *
   * The result is the buffer index of the best buffer to sync first.
--- 272,321 ----
  		if (buf->freeNext < 0)
  			StrategyControl->lastFreeBuffer = buf->buf_id;
  		StrategyControl->firstFreeBuffer = buf->buf_id;
+ 		++StrategyControl->numFreeListBuffers;
  	}
  
! 	SpinLockRelease(&StrategyControl->freelist_lck);
  }
  
  /*
+  * StrategyMoveBufferToFreeListEnd: put a buffer on the end of freelist
+  */                                                                                      
+ bool
+ StrategyMoveBufferToFreeListEnd(volatile BufferDesc *buf)                                
+ {
+ 	bool		freed = false;
+ 	SpinLockAcquire(&StrategyControl->freelist_lck);                                    
+  	/*
+  	 * It is possible that we are told to put something in the freelist that
+  	 * is already in it; don't screw up the list if so.
+  	 */                    
+  	if (buf->freeNext == FREENEXT_NOT_IN_LIST)
+  	{
+ 	 	++StrategyControl->numFreeListBuffers;
+ 		freed = true;
+  		/*
+  		 * put the buffer on end of list and if list is empty then
+  		 * assign first and last freebuffer with this buffer id.             
+  		 */     
+  		buf->freeNext = FREENEXT_END_OF_LIST;
+  		if (StrategyControl->firstFreeBuffer < 0)                                
+  		{                
+  			StrategyControl->firstFreeBuffer = buf->buf_id;
+  			StrategyControl->lastFreeBuffer = buf->buf_id;                   
+  			SpinLockRelease(&StrategyControl->freelist_lck);              
+  			return freed; 
+  		}                                                         
+  		BufferDescriptors[StrategyControl->lastFreeBuffer].freeNext = buf->buf_id;
+  		StrategyControl->lastFreeBuffer = buf->buf_id;
+  	}
+  	SpinLockRelease(&StrategyControl->freelist_lck);
+ 
+ 	return freed;
+ }                                                                                        
+ 
+ 
+ /*
   * StrategySyncStart -- tell BufferSync where to start syncing
   *
   * The result is the buffer index of the best buffer to sync first.
***************
*** 287,292 **** StrategySyncStart(uint32 *complete_passes, uint32 *num_buf_alloc)
--- 344,372 ----
  	return result;
  }
  
+ void
+ StrategySyncStartAndEnd(uint32 *start, uint32 *end)
+ {
+ 	int			curfreebuffers;
+ 	int			reqfreebuffers;
+ 
+ 	/* 
+ 	  * ideally numFreeListBuffers should get called under
+ 	  * freelist spin lock, however here we need this number for
+ 	  * estimating approximate number of free buffers required
+ 	  * on freelist, so it would be okay, even if numFreeListBuffers is not exact.
+ 	  */
+ 	
+ 
+ 	LWLockAcquire(BufFreelistLock, LW_EXCLUSIVE);
+ 	*start = StrategyControl->nextVictimBuffer;
+        curfreebuffers = StrategyControl->numFreeListBuffers;
+ 	 reqfreebuffers = NBuffers/3;
+ 	 *end = reqfreebuffers -curfreebuffers;
+ 	LWLockRelease(BufFreelistLock);
+ 	return;
+ }
+ 
  /*
   * StrategyNotifyBgWriter -- set or clear allocation notification latch
   *
***************
*** 309,314 **** StrategyNotifyBgWriter(Latch *bgwriterLatch)
--- 389,407 ----
  }
  
  
+ void
+ StrategyInitFreeListLatch(Latch *bgwriterLatch)
+ {
+ 	/*
+ 	 * We acquire the BufFreelistLock just to ensure that the store appears
+ 	 * atomic to StrategyGetBuffer.  The bgwriter should call this rather
+ 	 * infrequently, so there's no performance penalty from being safe.
+ 	 */
+ 	SpinLockAcquire(&StrategyControl->freelist_lck);
+ 	StrategyControl->freelistLatch= bgwriterLatch;
+ 	SpinLockRelease(&StrategyControl->freelist_lck);
+ }
+ 
  /*
   * StrategyShmemSize
   *
***************
*** 376,381 **** StrategyInitialize(bool init)
--- 469,475 ----
  		 */
  		StrategyControl->firstFreeBuffer = 0;
  		StrategyControl->lastFreeBuffer = NBuffers - 1;
+ 		StrategyControl->numFreeListBuffers = NBuffers;
  
  		/* Initialize the clock sweep pointer */
  		StrategyControl->nextVictimBuffer = 0;
***************
*** 386,391 **** StrategyInitialize(bool init)
--- 480,487 ----
  
  		/* No pending notification */
  		StrategyControl->bgwriterLatch = NULL;
+ 		StrategyControl->freelistLatch = NULL;
+ 		SpinLockInit(&StrategyControl->freelist_lck);
  	}
  	else
  		Assert(!init);
*** a/src/include/storage/buf_internals.h
--- b/src/include/storage/buf_internals.h
***************
*** 185,195 **** extern BufferDesc *LocalBufferDescriptors;
--- 185,198 ----
  extern volatile BufferDesc *StrategyGetBuffer(BufferAccessStrategy strategy,
  				  bool *lock_held);
  extern void StrategyFreeBuffer(volatile BufferDesc *buf);
+ extern bool StrategyMoveBufferToFreeListEnd(volatile BufferDesc *buf);
  extern bool StrategyRejectBuffer(BufferAccessStrategy strategy,
  					 volatile BufferDesc *buf);
  
  extern int	StrategySyncStart(uint32 *complete_passes, uint32 *num_buf_alloc);
+ extern void     StrategySyncStartAndEnd(uint32 *start, uint32 *end);
  extern void StrategyNotifyBgWriter(Latch *bgwriterLatch);
+ extern void StrategyInitFreeListLatch(Latch *bgwriterLatch);
  
  extern Size StrategyShmemSize(void);
  extern void StrategyInitialize(bool init);
*** a/src/include/storage/bufmgr.h
--- b/src/include/storage/bufmgr.h
***************
*** 217,222 **** extern void AbortBufferIO(void);
--- 217,223 ----
  
  extern void BufmgrCommit(void);
  extern bool BgBufferSync(void);
+ extern void BgBufferSyncAndMoveBuffersToFreelist(void);
  
  extern void AtProcExit_LocalBuffers(void);
  
=

  [application/octet-stream] perf_buff_mgmt.sh (2.1K, ../006e01ce926c$c7768680$56639380$@kapila@huawei.com/3-perf_buff_mgmt.sh)
  download

view thread (39+ messages)  latest in thread

Message-ID: <006e01ce926c$c7768680$56639380$@kapila@huawei.com>
Permalink:  ../006e01ce926c$c7768680$56639380$@kapila@huawei.com/
Also on:    postgresql.org/message-id/006e01ce926c$c7768680$56639380$@kapila@huawei.com

 ·  · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: amit.kapila@huawei.com, robertmhaas@gmail.com, andres@2ndquadrant.com, greg@2ndquadrant.com
  Subject: Re: Move unused buffers to freelist
  In-Reply-To: <006e01ce926c$c7768680$56639380$@kapila@huawei.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox