Date: Tue, 15 Aug 2006 17:38:43 -0400
From: mark@mark.mielke.cc
To: Michael Stone <mstone+postgres@mathom.us>
Cc: pgsql-performance@postgresql.org
Subject: Re: Postgresql Performance on an HP DL385 and
Message-ID: <20060815213842.GA15210@mark.mielke.cc>
References: <20060814153841.GF27928@pervasive.com>
	<721b21dc0608140851v2332cdf9g59808004c983540f@mail.gmail.com>
	<20060814170545.GG27928@pervasive.com>
	<20060814170902.GN2900@mathom.us>
	<20060815162926.GS27928@pervasive.com>
	<20060815172645.GU2900@mathom.us>
	<20060815183327.GC12834@mark.mielke.cc>
	<20060815190254.GW2900@mathom.us>
	<20060815193951.GA13695@mark.mielke.cc>
	<20060815205857.GY2900@mathom.us>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20060815205857.GY2900@mathom.us>
User-Agent: Mutt/1.4.2.1i

On Tue, Aug 15, 2006 at 04:58:59PM -0400, Michael Stone wrote:
> On Tue, Aug 15, 2006 at 03:39:51PM -0400, mark@mark.mielke.cc wrote:
> >No. This is not true. Updating the file system structure (inodes, indirect
> >blocks) touches a separate part of the disk than the actual data. If
> >the file system structure is modified, say, to extend a file to allow
> >it to contain more data, but the data itself is not written, then upon
> >a restore, with a system such as ext2, or ext3 with writeback, or xfs,
> >it is possible that the end of the file, even the postgres log file,
> >will contain a random block of data from the disk. If this random block
> >of data happens to look like a valid xlog block, it may be played back,
> >and the database corrupted.
> you're conflating a whole lot of different issues here. You're ignoring 
> the fact that postgres preallocates the xlog segment, you're ignoring 
> the fact that you can sync a directory entry, you're ignoring the fact 
> that syncing some metadata (such as atime) doesn't matter (only the 
> block allocation is important in this case, and the blocks are 
> pre-allocated).

Yes, no, no, no. :-)

I didn't know that the xlog segment only uses pre-allocated space. I
ignore mtime/atime as they don't count as file system structure
changes to me. It's updating a field in place. No change to the structure.

With the pre-allocation knowledge, I agree with you. Not sure how I
missed that in my reviewing of the archives... I did know it
pre-allocated once upon a time... Hmm....

> >This is also wrong. fsck is needed because the file system is broken.
> nope, the file system *may* be broken. the dirty flag simply indicates 
> that the filesystem needs to be checked to find out whether or not it is 
> broken.

Ah, but if we knew it wasn't broken, then fsck wouldn't be needed, now
would it? So we assume that it is broken. A little bit of a game, but
it is important to me. If I assumed the file system was not broken, I
wouldn't run fsck. I run fsck, because I assume it may be broken. If
broken, it indicates potential corruption.

The difference for me, is that if you are correct, that the xlog is
safe, than for a disk that only uses xlog, fsck is not ever necessary,
even after a system crash. If fsck is necessary, then there is potential
for a problem.

With the pre-allocation knowledge, I'm tempted to agree with you that
fsck is not ever necessary for partitions that only hold a properly
pre-allocated xlog.

> >I don't mean to be offensive, but I won't accept what you say, as it does
> >not make sense with my understanding of how file systems work. :-)
> <shrug> I'm not getting paid to convince you of anything.

Just getting you to back up your claim a bit... As I said, no intent
to offend. I learned from it.

Thanks,
mark

-- 
mark@mielke.cc / markm@ncf.ca / markm@nortel.com     __________________________
.  .  _  ._  . .   .__    .  . ._. .__ .   . . .__  | Neighbourhood Coder
|\/| |_| |_| |/    |_     |\/|  |  |_  |   |/  |_   | 
|  | | | | \ | \   |__ .  |  | .|. |__ |__ | \ |__  | Ottawa, Ontario, Canada

  One ring to rule them all, one ring to find them, one ring to bring them all
                       and in the darkness bind them...

                           http://mark.mielke.cc/