From: Mischa Sandberg <mischa_sandberg@telus.net>
To: pgsql-performance@postgresql.org
Subject: Re: Inverted-list databases (was: Working on huge RAM based datasets)
Date: Fri, 09 Jul 2004 22:23:03 GMT
Message-ID: <bVEHc.6984$iw3.4810@clgrps13> (raw)
References: <20040708101913.GA15871@uio.no>
<00ac01c46516$3ec859e0$0300a8c0@lappy>
<40EDF7DE.8080900@familyhealth.com.au>
<011301c46597$15d145c0$0300a8c0@lappy>
""Andy Ballingall"" <andy_ballingall@bigfoot.com> wrote in message
news:011301c46597$15d145c0$0300a8c0@lappy...
> On another thread, (not in this mailing list), someone mentioned that
there
> are a class of databases which, rather than caching bits of database file
> (be it in the OS buffer cache or the postmaster workspace), construct a a
> well indexed memory representation of the entire data in the postmaster
> workspace (or its equivalent), and this, remaining persistent, allows the
DB
> to service backend queries far quicker than if the postmaster was working
> with the assumption that most of the data was on disk (even if, in
practice,
> large amounts or perhaps even all of it resides in OS cache).
As a historical note, System R (grandaddy of all relational dbs) worked this
way.
And it worked under ridiculous memory constraints by modern standards.
Space-conscious MOLAP databases do this, FWIW.
Sybase 11 bitmap indexes pretty much amount to this, too.
I've built a SQL engine that used bitmap indexes within B-Tree indexes,
making it practical to index every field of every table (the purpose of the
engine).
You can also build special-purpose in-memory representations to test for
existence (of a key), when you expect a lot of failures. Google
"superimposed coding" e.g. http://www.dbcsoftware.com/dbcnews/NOV94.TXT
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: mischa_sandberg@telus.net
Subject: Re: Inverted-list databases (was: Working on huge RAM based datasets)
In-Reply-To: <bVEHc.6984$iw3.4810@clgrps13>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox