pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Bruce Momjian <bruce@momjian.us>
To: Greg Smith <greg@2ndquadrant.com>
Cc: Kevin Grittner <Kevin.Grittner@wicourts.gov>
Cc: jd@commandprompt.com, Scott Marlowe <scott.marlowe@gmail.com>
Cc: Steve Crawford <scrawford@pinpointresearch.com>
Cc: pgsql-performance@postgresql.org, Ben Chobot <bench@silentmedia.com>
Subject: Re: BBU Cache vs. spindles
Date: Tue, 30 Nov 2010 22:07:18 -0500 (EST)
Message-ID: <201012010307.oB137IA19179@momjian.us> (raw)
In-Reply-To: <4CC1094C.1090306@2ndquadrant.com>

Greg Smith wrote:
> Kevin Grittner wrote:
> > I assume that we send a full
> > 8K to the OS cache, and the file system writes disk sectors
> > according to its own algorithm.  With either platters or BBU cache,
> > the data is persisted on fsync; why do you see a risk with one but
> > not the other
> 
> I'd like a 10 minute argument please.  I started to write something to 
> refute this, only to clarify in my head the sequence of events that 
> leads to the most questionable result, where I feel a bit less certain 
> than I did before of the safety here.  Here is the worst case I believe 
> you're describing:
> 
> 1) Transaction is written to the WAL and sync'd; client receives 
> COMMIT.  Since full_page_writes is off, the data in the WAL consists 
> only of the delta of what changed on the page.
> 2) 8K database page is written to OS cache
> 3) PG calls fsync to force the database block out
> 4) OS writes first 4K block of the change to the BBU write cache.  Worst 
> case, this fills the cache, and it takes a moment for some random writes 
> to process before it has space to buffer again (makes this more likely 
> to happen, but it's not required to see the failure case here)
> 5) Sudden power interruption, second half of the page write is lost
> 6) Server restarts
> 7) That 4K write is now replayed from the battery's cache
> 
> At this point, you now have a torn 8K page, with 1/2 old and 1/2 new 

Based on this report, I think we need to update our documentation and
backpatch removal of text that says that BBU users can safely turn off
full-page writes.  Patch attached.

I think we have fallen into a trap I remember from the late 1990's where
I was assuming that an 8k-block based file system would write to the
disk atomically in 8k segments, which of course it cannot.  My bet is
that even if you write to the kernel in 8k pages, and have an 8k file
system, the disk is still accessed via 512-byte blocks, even with a BBU.
 
-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + It's impossible for everything to be true. +

Attachments:

/pgpatches/bbu (text/x-diff, 1.4K) download diff --git a/doc/src/sgml/wal.sgml b/doc/src/sgml/wal.sgml index a2724fa..1e67bbd 100644 *** /tmp/pgrevert.14281/7sLqTb_wal.sgml Tue Nov 30 21:57:17 2010 --- doc/src/sgml/wal.sgml Tue Nov 30 21:56:49 2010 *************** *** 164,173 **** <productname>PostgreSQL</> periodically writes full page images to permanent WAL storage <emphasis>before</> modifying the actual page on disk. By doing this, during crash recovery <productname>PostgreSQL</> can ! restore partially-written pages. If you have a battery-backed disk ! controller or file-system software that prevents partial page writes ! (e.g., ZFS), you can turn off this page imaging by turning off the ! <xref linkend="guc-full-page-writes"> parameter. </para> </sect1> --- 164,175 ---- <productname>PostgreSQL</> periodically writes full page images to permanent WAL storage <emphasis>before</> modifying the actual page on disk. By doing this, during crash recovery <productname>PostgreSQL</> can ! restore partially-written pages. If you have file-system software ! that prevents partial page writes (e.g., ZFS), you can turn off ! this page imaging by turning off the <xref ! linkend="guc-full-page-writes"> parameter. Battery-Backed unit ! (BBU) disk controllers do not prevent partial page writes unless ! they guarantee that data is written to the BBU as full (8kB) pages. </para> </sect1>

view thread (61+ messages)  latest in thread

Message-ID: <201012010307.oB137IA19179@momjian.us>
Permalink:  ../201012010307.oB137IA19179@momjian.us/
Also on:    postgresql.org/message-id/201012010307.oB137IA19179@momjian.us

 ·  · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-performance@postgresql.org
  Cc: bruce@momjian.us, greg@2ndquadrant.com, Kevin.Grittner@wicourts.gov, scott.marlowe@gmail.com, scrawford@pinpointresearch.com, bench@silentmedia.com
  Subject: Re: BBU Cache vs. spindles
  In-Reply-To: <201012010307.oB137IA19179@momjian.us>
  References: <4CC1094C.1090306@2ndquadrant.com> <201012010307.oB137IA19179@momjian.us>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox