pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: J. Andrew Rogers <jrogers@neopolitan.com>
To: josh@agliodbs.com
To: pgsql-performance@postgresql.org
Subject: Re: First set of OSDL Shared Mem scalability results,
Date: 08 Oct 2004 15:13:00 -0700
Message-ID: <1097273580.25588.18.camel@vulture.corp.neopolitan.com> (raw)
In-Reply-To: <200410081443.16109.josh@agliodbs.com>
References: <200410081443.16109.josh@agliodbs.com>

I have an idea that makes some assumptions about internals that I think
are correct.

When you have a huge number of buffers in a list that has to be
traversed to look for things in cache, e.g. 100k, you will generate an
almost equivalent number of cache line misses on the processor to jump
through all those buffers.  As I understand it (and I haven't looked so
I could be wrong), the buffer cache is searched by traversing it
sequentially.  OTOH, it seems reasonable to me that the OS disk cache
may actually be using a tree structure that would generate vastly fewer
cache misses by comparison to find a buffer.  This could mean a
substantial linear search cost as a function of the number of buffers,
big enough to rise above the noise floor when you have hundreds of
thousands of buffers.

Cache misses start to really add up when a code path generates many,
many thousands of them, and differences in the access path between the
buffer cache and disk cache would be reflected when you have that many
buffers.  I've seen these types of unexpected performance anomalies
before that got traced back to code patterns and cache efficiency and
gotten integer factors improvements by making some seemingly irrelevant
code changes.

So I guess my question would be 1) are my assumptions about the
internals correct, and 2) if they are, is there a way to optimize
searching the buffer cache so that a search doesn't iterate over a
really long buffer list that is bottlenecked on cache line replacement. 

My random thought of the day,

j. andrew rogers





view thread (41+ messages)  latest in thread

Message-ID: <1097273580.25588.18.camel@vulture.corp.neopolitan.com>
Permalink:  ../1097273580.25588.18.camel@vulture.corp.neopolitan.com/
Also on:    postgresql.org/message-id/1097273580.25588.18.camel@vulture.corp.neopolitan.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-performance@postgresql.org
  Cc: jrogers@neopolitan.com, josh@agliodbs.com
  Subject: Re: First set of OSDL Shared Mem scalability results,
  In-Reply-To: <1097273580.25588.18.camel@vulture.corp.neopolitan.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox