pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Christopher Browne <cbbrowne@acm.org>
To: pgsql-performance@postgresql.org
Subject: Re: Working on huge RAM based datasets
Date: Sat, 10 Jul 2004 09:06:34 -0400
Message-ID: <m3hdsgovtx.fsf@wolfe.cbbrowne.com> (raw)
References: <20040708101913.GA15871@uio.no>
	<00ac01c46516$3ec859e0$0300a8c0@lappy>
	<40EDF7DE.8080900@familyhealth.com.au>
	<011301c46597$15d145c0$0300a8c0@lappy>

Quoth andy_ballingall@bigfoot.com ("Andy Ballingall"):
> This is the future, isn't it? Each year, a higher percentage of DB
> applications will be able to fit entirely in RAM, and that percentage is
> going to be quite significant in just a few years. The disk system gets
> relegated to a data preload on startup and servicing the writes as the
> server does its stuff.

Regrettably, this may be something that fits better with MySQL, as it
already has an architecture oriented to having different "storage
engines" in behind.

There may be merit to the notion of implementing in-memory databases;
some assumptions change:

 - You might use bitmap indices, although that probably "kills" MVCC;

 - You might use T-trees rather than B-trees for indices, although
   research seems to indicate that B-trees win out if there is a 
   great deal of concurrent access;

 - It can become worthwhile to use compression schemes to fit more
   records into memory that wouldn't be worthwhile if using demand
   paging.

If you really want to try this, then look at Konstantin Knizhnik's
FastDB system:
  http://www.ispras.ru/~knizhnik/fastdb.html

It assumes that your application will be a monolithic C++ process; if
that isn't the case, then performance will probably suffer due to
throwing in context switches.

The changes in assumptions are pretty vital ones, that imply you're
heading in a fairly different direction than that which PostgreSQL
seems to be taking.  

That's not to say that there isn't merit to building a database system
using T-trees and bitmap indices attuned to applications where
main-memory storage is key; it's just that the proposal probably
should go somewhere else.
-- 
output = ("cbbrowne" "@" "cbbrowne.com")
http://www3.sympatico.ca/cbbrowne/languages.html
How does the guy who drives the snowplow get to work in the mornings?



view thread (16+ messages)  latest in thread

Message-ID: <m3hdsgovtx.fsf@wolfe.cbbrowne.com>
Permalink:  ../m3hdsgovtx.fsf@wolfe.cbbrowne.com/
Also on:    postgresql.org/message-id/m3hdsgovtx.fsf@wolfe.cbbrowne.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-performance@postgresql.org
  Cc: cbbrowne@acm.org
  Subject: Re: Working on huge RAM based datasets
  In-Reply-To: <m3hdsgovtx.fsf@wolfe.cbbrowne.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox