From: Andres Freund <andres@2ndquadrant.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>
Subject: Re: Changeset Extraction v7.0 (was logical changeset generation)
Date: Thu, 23 Jan 2014 13:05:03 +0100
Message-ID: <20140123120503.GB7182@awork2.anarazel.de> (raw)
In-Reply-To: <CA+Tgmob94d1A5OiwEmsE1aug8g8w4WU2WEbViGPY4F8YPR3g2g@mail.gmail.com>
References: <20140115002223.GA17204@awork2.anarazel.de>
<CA+TgmoaWO2xU0LGQ_DFd0RaQM5ZRa8wB6u+-0UAdSrau1i0V=w@mail.gmail.com>
<20140115203901.GF8653@awork2.anarazel.de>
<CA+TgmoZt2DhWKWwgbqdUpKJ-tH9V=fD8k9uHZYKVokrUR=Me_A@mail.gmail.com>
<20140116145432.GD4498@alap3.lan>
<CA+TgmoasqtaJmgTRBqyRTpQdHDCqjt979W+zY66Ypk5RFEyeoQ@mail.gmail.com>
<20140122144846.GH21170@alap3.anarazel.de>
<CA+TgmoZ9yeTiwxc10Y+4+hZ2oAgeEUtPiSSgPTZ5ynODe74+eg@mail.gmail.com>
<20140122154858.GK21170@alap3.anarazel.de>
<CA+Tgmob94d1A5OiwEmsE1aug8g8w4WU2WEbViGPY4F8YPR3g2g@mail.gmail.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
Hi,
On 2014-01-22 13:00:44 -0500, Robert Haas wrote:
> Well, apparently, one is going to PANIC and reinitialize the system.
> I presume that upon reinitialization we'll decide that the slot is
> gone, and thus won't recreate it in shared memory.
Yea, and if it's half-gone we'll continue deletion. And since yesterday
evening we'll even fsync things during startup to handle scenarios
similar to 20140122162115.GL21170@alap3.anarazel.de .
> Of course, if the entire system suffers a hard power failure after that and before the
> directory is succesfully fsync'd, then the slot could reappear on the
> next startup. Which is also exactly what would happen if we removed
> the slot from shared memory after doing the unlink, and then the
> system suffered a hard power failure before the directory contents
> made it to disk. Except that we also panicked.
Yes, but that could only happen as long as no relevant data has been
lost since we hold relevant locks during this.
> In the case of shared buffers, the way we handle fsync failures is by
> not allowing the system to checkpoint until all of the fsyncs succeed.
I don't think shared buffers fsyncs are the apt comparison. It's more
something like UpdateControlFile(). Which PANICs.
I really don't get why you fight PANICs in general that much. There are
some nasty PANICs in postgres which can happen in legitimate situations,
which should be made to fail more gracefully, but this surely isn't one
of them. We're doing rename(), unlink() and rmdir(). That's it.
We should concentrate on the ones that legitimately can happen, not the
ones created by an admin running a chmod -R 000 . ; rm -rf $PGDATA or
mount -o remount,ro /. We don't increase reliability by a bit adding
codepaths that will never get tested.
> If there's an OS-level reset before that happens, WAL replay will
> perform the same buffer modifications over again and the next
> checkpoint will again try to flush them to disk and will not complete
> unless it does. That forms a closed system where we never advance the
> redo pointer over the covering WAL record until the changes it covers
> are on the disk. But I don't think this code has any similar
> interlock; if it does, I missed it.
No, it doesn't (until the first rename() at least), but the number of
failure scenarios is far smaller.
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: andres@2ndquadrant.com, robertmhaas@gmail.com
Subject: Re: Changeset Extraction v7.0 (was logical changeset generation)
In-Reply-To: <20140123120503.GB7182@awork2.anarazel.de>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox