Message-ID: <4CC46479.2050206@2ndquadrant.com>
Date: Sun, 24 Oct 2010 12:53:13 -0400
From: Greg Smith <greg@2ndquadrant.com>
User-Agent: Thunderbird 2.0.0.24 (X11/20100317)
MIME-Version: 1.0
To: James Mansion <james@mansionfamily.plus.com>
CC: Kevin Grittner <Kevin.Grittner@wicourts.gov>,
	Bruce Momjian <bruce@momjian.us>,
	jd@commandprompt.com, Scott Marlowe <scott.marlowe@gmail.com>,
	Steve Crawford <scrawford@pinpointresearch.com>,
	pgsql-performance@postgresql.org, Ben Chobot <bench@silentmedia.com>
Subject: Re: BBU Cache vs. spindles
References: <4CC04EA50200002500036C5E@gw.wicourts.gov>
	<201010211935.o9LJZTM21744@momjian.us>
	<4CC0513E0200002500036C68@gw.wicourts.gov>
	<4CC3E8BF.4090409@mansionfamily.plus.com>
In-Reply-To: <4CC3E8BF.4090409@mansionfamily.plus.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

James Mansion wrote:
> When I looked at the internals of TokyoCabinet for example, the design 
> was flawed but
> would be 'fairly robust' so long as mmap'd pages that were dirtied did 
> not get persisted
> until msync, and were then persisted atomically.

If TokyoCabinet presumes that's true and overwrites existing blocks with 
that assumption, it would land onto my list of databases I wouldn't 
trust to hold my TODO list.  Flip off power to a server, and you have no 
idea what portion of the blocks sitting in the drive's cache actually 
made it to disk; that's not even guaranteed atomic to the byte level.  
Torn pages happen all the time unless you either a) put the entire write 
into a non-volatile cache before writing any of it, b) write and sync 
somewhere else first and then do a journaled filesystem pointer swap 
from the old page to the new one, or c) journal the whole write the way 
PostgreSQL does with full_page_writes and the WAL.  The discussion here 
veered off over whether (a) was sufficiently satisfied just by having a 
RAID controller with battery backup, and what I concluded from the dive 
into the details is that it's definitely not true unless the filesystem 
block size exactly matches the database one.  And even then, make sure 
you test heavily.

-- 
Greg Smith   2ndQuadrant US    greg@2ndQuadrant.com   Baltimore, MD
PostgreSQL Training, Services and Support        www.2ndQuadrant.us
"PostgreSQL 9.0 High Performance": http://www.2ndQuadrant.com/books