From: Jan Wieck <JanWieck@Yahoo.com>
To: Kevin Brown <kevin@sysexperts.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: First set of OSDL Shared Mem scalability results, some
Date: Wed, 13 Oct 2004 20:52:43 -0400
Message-ID: <416DCDDB.1010902@Yahoo.com> (raw)
In-Reply-To: <20041009112048.GA665@filer>
References: <200410081443.16109.josh@agliodbs.com>
<m31xg8zlo4.fsf@wolfe.cbbrowne.com>
<20041009112048.GA665@filer>
On 10/9/2004 7:20 AM, Kevin Brown wrote:
> Christopher Browne wrote:
>> Increasing the number of cache buffers _is_ likely to lead to some
>> slowdowns:
>>
>> - Data that passes through the cache also passes through kernel
>> cache, so it's recorded twice, and read twice...
>
> Even worse, memory that's used for the PG cache is memory that's not
> available to the kernel's page cache. Even if the overall memory
Which underlines my previous statement, that a PG shared cache much
larger than the high-frequently accessed data portion of the DB is
counterproductive. Double buffering (kernel-disk-buffer plus shared
buffer) only makes sense for data that would otherwise cause excessive
memory copies in and out of the shared buffer. After that, in only
lowers the memory available for disk buffers.
Jan
> usage in the system isn't enough to cause some paging to disk, most
> modern kernels will adjust the page/disk cache size dynamically to fit
> the memory demands of the system, which in this case means it'll be
> smaller if running programs need more memory for their own use.
>
> This is why I sometimes wonder whether or not it would be a win to use
> mmap() to access the data and index files -- doing so under a truly
> modern OS would surely at the very least save a buffer copy (from the
> page/disk cache to program memory) because the OS could instead
> direcly map the buffer cache pages directly to the program's memory
> space.
>
> Since PG often has to have multiple files open at the same time, and
> in a production database many of those files will be rather large, PG
> would have to limit the size of the mmap()ed region on 32-bit
> platforms, which means that things like the order of mmap() operations
> to access various parts of the file can become just as important in
> the mmap()ed case as it is in the read()/write() case (if not more
> so!). I would imagine that the use of mmap() on a 64-bit platform
> would be a much, much larger win because PG would most likely be able
> to mmap() entire files and let the OS work out how to order disk reads
> and writes.
>
> The biggest problem as I see it is that (I think) mmap() would have to
> be made to cooperate with malloc() for virtual address space. I
> suspect issues like this have already been worked out by others,
> however...
>
>
>
--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck@Yahoo.com #
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: JanWieck@Yahoo.com, kevin@sysexperts.com
Subject: Re: First set of OSDL Shared Mem scalability results, some
In-Reply-To: <416DCDDB.1010902@Yahoo.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox