From: Greg Stark <gsstark@mit.edu>
To: Curt Sampson <cjs@cynic.net>
Cc: Greg Stark <gsstark@mit.edu>
Cc: pgsql-hackers@postgresql.org
Subject: Re: [PATCHES] ARC Memory Usage analysis
Date: 26 Oct 2004 11:30:23 -0400
Message-ID: <87r7nlts34.fsf@stark.xeocode.com> (raw)
In-Reply-To: <Pine.NEB.4.58.0410261502510.13312@angelic-vtfw.cvpn.cynic.net>
References: <1098471059.20926.49.camel@localhost.localdomain>
<41796115.2020501@Yahoo.com>
<20041022200955.GC4382@it.is.rice.edu>
<417D1D01.3090401@Yahoo.com>
<20651.1098720192@sss.pgh.pa.us>
<87mzyavhxc.fsf@stark.xeocode.com>
<87hdoiv62b.fsf@stark.xeocode.com>
<790.1098741205@sss.pgh.pa.us>
<878y9uukgb.fsf@stark.xeocode.com>
<Pine.NEB.4.58.0410261502510.13312@angelic-vtfw.cvpn.cynic.net>
Curt Sampson <cjs@cynic.net> writes:
> On Tue, 26 Oct 2004, Greg Stark wrote:
>
> > I see mmap or O_DIRECT being the only viable long-term stable states. My
> > natural inclination was the former but after the latest thread on the subject
> > I suspect it'll be forever out of reach. That makes O_DIRECT And a Postgres
> > managed cache the only real choice. Having both caches is just a waste of
> > memory and a waste of cpu cycles.
>
> I don't see why mmap is any more out of reach than O_DIRECT; it's not
> all that much harder to implement, and mmap (and madvise!) is more
> widely available.
Because there's no way to prevent a write-out from occurring and no way to be
notified by mmap before a write-out occurs, and Postgres wants to do its WAL
logging at that time if it hasn't already happened.
> But if using two caches is only costing us 1% in performance, there's
> not really much point....
Well firstly it depends on the work profile. It can probably get much higher
than we saw in that profile if your work load is causing more fresh buffers to
be fetched.
Secondly it also reduces the amount of cache available. If you have 256M of
ram with about 200M free, and 40Mb of ram set aside for Postgres's buffer
cache then you really only get 160Mb. It's costing you 20% of your cache, and
reducing the cache hit rate accordingly.
Thirdly the kernel doesn't know as much as Postgres about the load. Postgres
could optimize its use of cache based on whether it knows the data is being
loaded by a vacuum or sequential scan rather than an index lookup. In practice
Postgres has gone with ARC which I suppose a kernel could implement anyways,
but afaik neither linux nor BSD choose to do anything like it.
--
greg
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: gsstark@mit.edu, cjs@cynic.net, pgsql-hackers@postgresql.org
Subject: Re: [PATCHES] ARC Memory Usage analysis
In-Reply-To: <87r7nlts34.fsf@stark.xeocode.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox