X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id C4E4CD1B360 for ; Fri, 9 Jul 2004 19:29:03 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 38066-07 for ; Fri, 9 Jul 2004 22:28:57 +0000 (GMT) Received: from news.hub.org (news.hub.org [200.46.204.72]) by svr1.postgresql.org (Postfix) with ESMTP id B096BD1B342 for ; Fri, 9 Jul 2004 19:28:52 -0300 (ADT) Received: from news.hub.org (news.hub.org [200.46.204.72]) by news.hub.org (8.12.9/8.12.9) with ESMTP id i69MSpi7078548 for ; Fri, 9 Jul 2004 22:28:51 GMT (envelope-from news@news.hub.org) Received: (from news@localhost) by news.hub.org (8.12.9/8.12.9/Submit) id i69MN00N077592 for pgsql-performance@postgresql.org; Fri, 9 Jul 2004 22:23:00 GMT From: "Mischa Sandberg" X-Newsgroups: comp.databases.postgresql.performance References: <20040708101913.GA15871@uio.no> <00ac01c46516$3ec859e0$0300a8c0@lappy> <40EDF7DE.8080900@familyhealth.com.au> <011301c46597$15d145c0$0300a8c0@lappy> Subject: Re: Inverted-list databases (was: Working on huge RAM based datasets) Lines: 32 X-Priority: 3 X-MSMail-Priority: Normal X-Newsreader: Microsoft Outlook Express 6.00.2800.1409 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1409 Message-ID: Date: Fri, 09 Jul 2004 22:23:03 GMT To: pgsql-performance@postgresql.org X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.8 tagged_above=0.0 required=5.0 tests=PRIORITY_NO_NAME X-Spam-Level: X-Archive-Number: 200407/76 X-Sequence-Number: 7467 ""Andy Ballingall"" wrote in message news:011301c46597$15d145c0$0300a8c0@lappy... > On another thread, (not in this mailing list), someone mentioned that there > are a class of databases which, rather than caching bits of database file > (be it in the OS buffer cache or the postmaster workspace), construct a a > well indexed memory representation of the entire data in the postmaster > workspace (or its equivalent), and this, remaining persistent, allows the DB > to service backend queries far quicker than if the postmaster was working > with the assumption that most of the data was on disk (even if, in practice, > large amounts or perhaps even all of it resides in OS cache). As a historical note, System R (grandaddy of all relational dbs) worked this way. And it worked under ridiculous memory constraints by modern standards. Space-conscious MOLAP databases do this, FWIW. Sybase 11 bitmap indexes pretty much amount to this, too. I've built a SQL engine that used bitmap indexes within B-Tree indexes, making it practical to index every field of every table (the purpose of the engine). You can also build special-purpose in-memory representations to test for existence (of a key), when you expect a lot of failures. Google "superimposed coding" e.g. http://www.dbcsoftware.com/dbcnews/NOV94.TXT