DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=message-id:date:from:to:subject:cc:in-reply-to:mime-version
	:content-type:content-transfer-encoding:content-disposition
	:references;
	b=B02+2k2BU3I6muAGdMYcM5dmgVHAMRMsIS59Gma8p4F9yQyT4KmdxjqoZftSWqlPAM
	7Z05EGDnaJj8bDGnIcBjWHCH2DSh0Rgzya0uHgUodRU++RfKw9bf9r/YLFBL2F/hqQIo
	NfeNp44Mpyupbt+YtktcfU4AJyhl0JwQdZX2g=
Message-ID: <72dbd3150811061821ub30ec93x4da25b82060e5d29@mail.gmail.com>
Date: Thu, 6 Nov 2008 18:21:59 -0800
From: "David Rees" <drees76@gmail.com>
To: "Scott Marlowe" <scott.marlowe@gmail.com>
Subject: Re: Create and drop temp table in 8.3.4
Cc: "Kevin Grittner" <Kevin.Grittner@wicourts.gov>, jd@commandprompt.com,
	pgsql-performance@postgresql.org, "Scott Carey" <scott@richrelevance.com>
In-Reply-To: <dcc563d10811061603t574c74afj4f651c4ec7dbb88d@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <4911957A.EE98.0025.0@wicourts.gov>
	<4912EAFF.EE98.0025.0@wicourts.gov>
	<1225999071.1204.14.camel@jd-laptop.pragmaticzealot.org>
	<4912F29E.EE98.0025.0@wicourts.gov>
	<a1ec7d000811061305n7b28800od469c1411e1a9928@mail.gmail.com>
	<dcc563d10811061405l76cb59adh58741b227fe9d8a6@mail.gmail.com>
	<49131C67.EE98.0025.0@wicourts.gov>
	<dcc563d10811061445h32c5ad72o84f367d7cc3c6899@mail.gmail.com>
	<49132383.EE98.0025.0@wicourts.gov>
	<dcc563d10811061603t574c74afj4f651c4ec7dbb88d@mail.gmail.com>

On Thu, Nov 6, 2008 at 4:03 PM, Scott Marlowe <scott.marlowe@gmail.com> wrote:
> On Thu, Nov 6, 2008 at 4:04 PM, Kevin Grittner <Kevin.Grittner@wicourts.gov> wrote:
>> "Scott Marlowe" <scott.marlowe@gmail.com> wrote:
>>> Without write barriers in my file system an fsync request will
>>> be immediately returned true, correct?
>>
>> Not as I understand it; although it will be pretty fast if it all fits
>> into the battery backed cache.
>
> OK, thought exercise time.  There's a limited size for the cache.
> Let's assume it's much smaller, say 16Megabytes.  We turn off write
> barriers.  We start writing data to the RAID array faster than the
> disks can write it.  At some point, the data flowing into the cache is
> backing up into the OS.  Without write barriers, the second we call an
> fsync it returns true.  But the data's not in the cache yet, or on the
> disk.  Machine crashes, data is incoherent.
>
> But that's assuming write barriers work as I understand them.

Let's try to clear up a couple things:

1. We are talking about 3 different memory caches in order from low to high:
Disk cache, Controller cache (BBU in this case) and OS cache.

2. A write barrier instructs the lower level hardware that commands
issued before the barrier must be written to disk before commands
issued after the barrier. Write barriers are used to ensure that data
written to disk is written in such a way as to maintain filesystem
consistency, without losing all the benefits of a write cache.

3. A fsync call forces data to be synced to the controller.

This means that whenever you call fsync, at the very minimum, the data
will have made it to the controller. How much further down the line
will depend on whether or not the controller is in WriteBack or
WriteThrough mode and whether or not the disk is also caching writes.

So in your example, if the OS is caching some writes and fsync is
called, it won't be returned until at a minimum the controller has
accepted all the data, regardless of whether or not write barriers are
enabled.

In theory, it should be safe to disable write barriers if you have a
BBU because the BBU should guarantee that all writes will eventually
make it to disk (or at least reduce the risk of that not happening to
an acceptable level).

-Dave