Date: Wed, 22 Jan 2014 16:48:58 +0100
From: Andres Freund <andres@2ndquadrant.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Subject: Re: Changeset Extraction v7.0 (was logical changeset
 generation)
Message-ID: <20140122154858.GK21170@alap3.anarazel.de>
References: <20140115002223.GA17204@awork2.anarazel.de>
 <CA+TgmoaWO2xU0LGQ_DFd0RaQM5ZRa8wB6u+-0UAdSrau1i0V=w@mail.gmail.com>
 <20140115203901.GF8653@awork2.anarazel.de>
 <CA+TgmoZt2DhWKWwgbqdUpKJ-tH9V=fD8k9uHZYKVokrUR=Me_A@mail.gmail.com>
 <20140116145432.GD4498@alap3.lan>
 <CA+TgmoasqtaJmgTRBqyRTpQdHDCqjt979W+zY66Ypk5RFEyeoQ@mail.gmail.com>
 <20140122144846.GH21170@alap3.anarazel.de>
 <CA+TgmoZ9yeTiwxc10Y+4+hZ2oAgeEUtPiSSgPTZ5ynODe74+eg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
 <CA+TgmoZ9yeTiwxc10Y+4+hZ2oAgeEUtPiSSgPTZ5ynODe74+eg@mail.gmail.com>
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org

On 2014-01-22 10:14:27 -0500, Robert Haas wrote:
> On Wed, Jan 22, 2014 at 9:48 AM, Andres Freund <andres@2ndquadrant.com> wrote:
> > On 2014-01-18 08:35:47 -0500, Robert Haas wrote:
> >> > I am not sure I understand that point. We can either update the
> >> > in-memory bit before performing the on-disk operations or
> >> > afterwards. Either way, there's a way to be inconsistent if the disk
> >> > operation fails somewhere inbetween (it might fail but still have
> >> > deleted the file/directory!). The normal way to handle that in other
> >> > places is PANICing when we don't know so we recover from the on-disk
> >> > state.
> >> > I really don't see the problem here? Code doesn't get more robust by
> >> > doing s/PANIC/ERROR/, rather the contrary. It takes extra smarts to only
> >> > ERROR, often that's not warranted.
> >>
> >> People get cranky when the database PANICs because of a filesystem
> >> failure.  We should avoid that, especially when it's trivial to do so.
> >>  The update to shared memory should be done second and should be set
> >> up to be no-fail.
> >
> > I don't see how that would help. If we fail during unlink/rmdir, we
> > don't really know at which point we failed.
> 
> This doesn't make sense to me.  unlink/rmdir are atomic operations.

Yes, individual operations should be, but you cannot be sure whether a
rename()/unlink() will survive a crash until the directory is
fsync()ed. So, what is one going to do if the unlink suceeded, but the
fsync didn't?

Deletion currently works like:
    if (rename(path, tmppath) != 0)
        ereport(ERROR,
                (errcode_for_file_access(),
                 errmsg("could not rename \"%s\" to \"%s\": %m",
                        path, tmppath)));

    /* make sure no partial state is visible after a crash */
    fsync_fname(tmppath, false);
    fsync_fname("pg_replslot", true);

    if (!rmtree(tmppath, true))
    {
        ereport(ERROR,
                (errcode_for_file_access(),
                 errmsg("could not remove directory \"%s\": %m",
                        tmppath)));
    }

If we fail between the rename() and the fsync_fname() we don't really
know which state we are in. We'd also have to add code to handle
incomplete slot directories, which currently only exists for startup, to
other places.

Greetings,

Andres Freund

-- 
 Andres Freund	                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers