From: Chris Browne <cbbrowne@acm.org>
Subject: Re: large dataset with write vs read clients
Date: Tue, 12 Oct 2010 12:41:17 -0400
Organization: cbbrowne Computing Inc
Lines: 39
Message-ID: <87aamj9xea.fsf@cbbrowne.afilias-int.info>
References: <AANLkTimNJc=0ffgRO_gO6Vsq06=aqBYXZouhbv0kBe-i@mail.gmail.com>
	<4CAE3FDB.1040909@2ndquadrant.com> <87ocb3yu10.fsf@mid.deneb.enyo.de>
	<4CB0E006.8010905@vmsinfo.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.2 (gnu/linux)
Cancel-Lock: sha1:o6ChNVbY4xFMbuAGAKvQ9tKJibM=
To: pgsql-performance@postgresql.org

mladen.gogala@vmsinfo.com (Mladen Gogala) writes:
> I have a logical problem with asynchronous commit. The "commit"
> command should instruct the database to make the outcome of the
> transaction permanent. The application should wait to see whether the
> commit was successful or not. Asynchronous behavior in the commit
> statement breaks the ACID rules and should not be used in a RDBMS
> system. If you don't need ACID, you may not need RDBMS at all. You may
> try with MongoDB. MongoDB is web scale:
> http://www.youtube.com/watch?v=b2F-DItXtZs

The "client" always has the option of connecting to a set of databases,
and stowing parts of the data hither and thither.  That often leads to
the relaxation called "BASE."  (And IBM has been selling that relaxation
as MQ-Series since the early '90s!)

There often *ARE* cases where it is acceptable for some of the data to
not be as durable, because that data is readily reconstructed.  This is
particularly common for calculated/cached/aggregated data.

Many things can get relaxed for a "data warehouse" data store, where the
database is not authoritative, but rather aggregates data drawn from
other authoritative sources.  In such applications, neither the A, C, I,
nor the D are pointedly crucial, in the DW data store.

- We don't put the original foreign key constraints into the DW
  database; they don't need to be enforced a second time.  Ditto for
  constraints of all sorts.

- Batching of the loading of updates is likely to break several of the
  letters.  And I find it *quite* acceptable to lose "D" if the data may
  be safely reloaded into the DW database.

I don't think this is either cavalier nor that it points to "MongoDB is
web scale."
-- 
"cbbrowne","@","gmail.com"
Rules  of the  Evil Overlord  #181.  "I  will decree  that all  hay be
shipped in tightly-packed bales. Any wagonload of loose hay attempting
to pass through a checkpoint will be set on fire."