To: Tatsuo Ishii <t-ishii@sra.co.jp>
cc: christian.bucanac@mindark.com, pgsql-general@postgresql.org
Subject: Re: Performance tuning for linux, 1GB RAM, dual CPU? 
In-reply-to: <20010712134958W.t-ishii@sra.co.jp> 
References: <5146853DD571D411AC54000102070D611C7F10@MINGBGNTS02>
	<23761.994870319@sss.pgh.pa.us> <20010712134958W.t-ishii@sra.co.jp>
Comments: In-reply-to Tatsuo Ishii <t-ishii@sra.co.jp>
	message dated "Thu, 12 Jul 2001 13:49:58 +0900"
Date: Thu, 12 Jul 2001 10:20:57 -0400
Message-ID: <16987.994947657@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

Tatsuo Ishii <t-ishii@sra.co.jp> writes:
> In my testing with *particluar* environment (Linux kernel 2.2.x,
> pgbench), it was indicated that too many shared buffers reduced the
> performance even though there was lots of memory, say 1GB. I'm not
> sure why, but I suspect there is a siginificant overhead to lookup
> shared buffers.

Regular lookups use a hash table, and shouldn't suffer much from
degraded performance as NBuffers rises.  However, there are some
operations that do a linear scan of all the buffers --- table deletion
comes to mind.  Perhaps your test was exercising one of these.

pgbench doesn't do table deletions of course... hmm... the only
such loop in bufmgr.c that looks like it would be executed during
normal transactions is BufferPoolCheckLeak().  Maybe we should
make that routine be a no-op unless assert checking is turned on?
Have we reached the point where performance is more interesting than
error checking?  It'd be interesting to retry your results with
BufferPoolCheckLeak() reduced to "return false".

Another factor, not under our control, is that if the shared memory
region gets too large the kernel may decide to swap out portions of
it that haven't been touched lately.  This of course is completely
counterproductive, especially if what gets swapped is a dirty buffer,
which'll eventually have to be read back in and then written to where
it should have gone.  This is the main factor behind my thought that you
don't want to skimp on kernel disk buffer space --- any memory pressure
in the system should be resolvable by dropping kernel disk buffers, not
by starting to swap shmem or user processes.

			regards, tom lane