pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Kevin Brown <kevin@sysexperts.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Mount options for Ext3?
Date: Fri, 24 Jan 2003 21:58:55 -0500
Message-ID: <1260.1043463535@sss.pgh.pa.us> (raw)
In-Reply-To: <20030125021159.GC28252@filer>
References: <200301241120.14404.josh@agliodbs.com>
	<20030125003011.GA28252@filer>
	<28176.1043457408@sss.pgh.pa.us>
	<20030125021159.GC28252@filer>

Kevin Brown <kevin@sysexperts.com> writes:
> I was presuming that when a savepoint occurs, a marker is written to
> the log indicating which transactions had been committed to the data
> files, and that this marker was paid attention to during database
> startup.

Not quite.  The marker says that all datafile updates described by
log entries before point X have been flushed to disk by the checkpoint
--- and, therefore, if we need to restart we need only replay log
entries occurring after the last checkpoint's point X.

This has nothing directly to do with which transactions are committed
or not committed.  If we based checkpoint behavior on that, we'd need
to maintain an indefinitely large amount of WAL log to cope with
long-running transactions.

The actual checkpoint algorithm is

	take note of current logical end of WAL (this will be point X)
	write() all dirty buffers in shared buffer arena
	sync() to ensure that above writes, as well as previous ones,
		are on disk
	put checkpoint record referencing point X into WAL; write and
		fsync WAL
	update pg_control with new checkpoint record, fsync it

Since pg_control is what's examined after restart, the checkpoint is
effectively committed when the pg_control write hits disk.  At any
instant before that, a crash would result in replaying from the
prior checkpoint's point X.  The algorithm is correct if and only if
the pg_control write hits disk after all the other writes mentioned.

The key assumption we are making about the filesystem's behavior is that
writes scheduled by the sync() will occur before the pg_control write
that's issued after it.  People have occasionally faulted this algorithm
by quoting the sync() man page, which saith (in the Gospel According To
HP)

     The writing, although scheduled, is not necessarily complete upon
     return from sync.

This, however, is not a problem in itself.  What we need to know is
whether the filesystem will allow writes issued after the sync() to
complete before those "scheduled" by the sync().


> So suppose the marker makes it to the log but not all of the data the
> marker refers to makes it to the data files.  Then the system crashes.

I think that this analysis is not relevant to what we're doing.

			regards, tom lane



view thread (14+ messages)  latest in thread

Message-ID: <1260.1043463535@sss.pgh.pa.us>
Permalink:  ../1260.1043463535@sss.pgh.pa.us/
Also on:    postgresql.org/message-id/1260.1043463535@sss.pgh.pa.us

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-performance@postgresql.org
  Cc: tgl@sss.pgh.pa.us, kevin@sysexperts.com
  Subject: Re: Mount options for Ext3?
  In-Reply-To: <1260.1043463535@sss.pgh.pa.us>
  References: <200301241120.14404.josh@agliodbs.com> <20030125003011.GA28252@filer> <28176.1043457408@sss.pgh.pa.us> <20030125021159.GC28252@filer> <1260.1043463535@sss.pgh.pa.us>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox