Date: Tue, 15 Aug 2006 15:02:56 -0400
From: Michael Stone <mstone+postgres@mathom.us>
Subject: Re: Postgresql Performance on an HP DL385 and
In-reply-to: <20060815183327.GC12834@mark.mielke.cc>
To: mark@mark.mielke.cc
Cc: "Jim C. Nasby" <jnasby@pervasive.com>, pgsql-performance@postgresql.org
Mail-followup-to: mark@mark.mielke.cc, "Jim C. Nasby" <jnasby@pervasive.com>,
	pgsql-performance@postgresql.org
Message-id: <20060815190254.GW2900@mathom.us>
MIME-version: 1.0
Content-type: text/plain; charset=us-ascii; format=flowed
Content-disposition: inline
References: <20060809210539.GR40481@pervasive.com>
	<1155180553.13315.0.camel@amd64-gentoo-laptop>
	<20060810110935.GF2900@mathom.us>
	<20060814153841.GF27928@pervasive.com>
	<721b21dc0608140851v2332cdf9g59808004c983540f@mail.gmail.com>
	<20060814170545.GG27928@pervasive.com>
	<20060814170902.GN2900@mathom.us>
	<20060815162926.GS27928@pervasive.com>
	<20060815172645.GU2900@mathom.us>
	<20060815183327.GC12834@mark.mielke.cc>
User-Agent: Mutt/1.5.12-2006-07-14

On Tue, Aug 15, 2006 at 02:33:27PM -0400, mark@mark.mielke.cc wrote:
>On Tue, Aug 15, 2006 at 01:26:46PM -0400, Michael Stone wrote:
>> On Tue, Aug 15, 2006 at 11:29:26AM -0500, Jim C. Nasby wrote:
>> >Are 'we' sure that such a setup can't lose any data?
>> Yes. If you check the archives, you can even find the last time this was 
>> discussed...
>
>I looked last night (coincidence actually) and didn't find proof that
>you cannot lose data.

You aren't going to find proof, any more than you'll find proof that you 
won't lose data if you do lose a journalling fs. (Because there isn't 
any.) Unfortunately, many people misunderstand the what a metadata 
journal does for you, and overstate its importance in this type of 
application.

>How do you deal with the file system structure being updated before the
>data blocks are (re-)written?

*That's what the postgres log is for.* If the latest xlog entries don't 
make it to disk, they won't be replayed; if they didn't make it to 
disk, the transaction would not have been reported as commited. An 
application that understands filesystem semantics can guarantee data 
integrity without metadata journaling.

>> The bottom line is that the only reason you need a metadata journalling 
>> filesystem is to save the fsck time when you come up. On a little 
>> partition like xlog, that's not an issue.
>
>fsck isn't only about time to fix. fsck is needed, because the file system
>is broken. 

fsck is needed to reconcile the metadata with the on-disk allocations. 
To do that, it reads all the inodes and their corresponding directory 
entries. The time to do that is proportional to the size of the 
filesystem, hence the comment about time. fsck is not needed "because 
the filesystem is broken", it's needed because the filesystem is marked 
dirty. 

Mike Stone