Date: Sat, 28 Apr 2018 11:21:20 -0400
From: Stephen Frost <sfrost@snowman.net>
To: Craig Ringer <craig@2ndquadrant.com>
Cc: Andres Freund <andres@anarazel.de>,
	PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Subject: Re: Postgres, fsync, and OSs (specifically linux)
Message-ID: <20180428152120.GS27724@tamriel.snowman.net>
References: <20180427222842.in2e4mibx45zdth5@alap3.anarazel.de>
 <CAMsr+YGz+9cEs88aq0hBbFvz1pFNGBVPNd1qsJNLkgJLhpDFag@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="FmBpKfX8H23NJZGP"
Content-Disposition: inline
In-Reply-To: 
 <CAMsr+YGz+9cEs88aq0hBbFvz1pFNGBVPNd1qsJNLkgJLhpDFag@mail.gmail.com>
User-Agent: Mutt/1.5.24 (2015-08-30)
Precedence: bulk


--FmBpKfX8H23NJZGP
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Greetings,

* Craig Ringer (craig@2ndquadrant.com) wrote:
> On 28 April 2018 at 06:28, Andres Freund <andres@anarazel.de> wrote:
> > - Add a pre-checkpoint hook that checks for filesystem errors *after*
> >   fsyncing all the files, but *before* logging the checkpoint completion
> >   record. Operating systems, filesystems, etc. all log the error format
> >   differently, but for larger installations it'd not be too hard to
> >   write code that checks their specific configuration.
>=20
> I looked into using trace event file descriptors for this, btw, but
> we'd need CAP_SYS_ADMIN to create one that captured events for other
> processes. Plus filtering the events to find only events for the files
> / file systems of interest would be far from trivial. And I don't know
> what guarantees we have about when events are delivered.
>=20
> I'd love to be able to use inotify for this, but again, that'd only be
> a new-kernels thing since it'd need an inotify extension to report I/O
> errors.
>=20
> Presumably mostly this check would land up looking at dmesg.
>=20
> I'm not convinced it'd get widely deployed and widely used, or that
> it'd be used correctly when people tried to use it. Look at the
> hideous mess that most backup/standby creation scripts,
> archive_command scripts, etc are.

Agree with more-or-less everything you've said here, but a big +1 on
this.  If we do end up going down this route we have *got* to provide
scripts which we know work and have been tested and are well maintained
on the popular OS's for the popular filesystems and make it clear that
we've tested those and not others.  We definitely shouldn't put
something in our docs that is effectively an example of the interface
but not an actual command that anyone should be using.

Thanks!

Stephen

--FmBpKfX8H23NJZGP
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCgAGBQJa5JFwAAoJEO1sijiDR2RVlCgP/iACuZ/SXlt9oSjWum81a/5C
gMiCUd2tHImjGiNyDj30mUQlZw0vlWeADV/qig1hW28Iy3m58sm7OGhTjXNMGW5G
dqpPU3wYkH9O7mxLjVNZjuBhmn/1kCtvsxlwv4cFh+jIfsWsuMKtHywIkPh3meH4
xiGY4X/SSbBUkvh+jZF/MraHgyGi/2hnYOgAfcSpS9EuNLMXzYa4TwIfHqU4OOao
P4OnhsiWJUUy1oBDfROZQnYmDEWHkt1XEKy7vI9icOcDU8o+dmkR4G5J/Y8pXVoL
KnBL5cFODX7uy/smg4HAlkF5fp+Xd4+WKqu1LIojUyTofMPqiZqID68VFRgeEIXG
lpoQI5qb6lYI8jrW5zq5ze++I30UVqzWFbD5r+sfPVLSpLYcv3D/GGDo4PqrrP2D
3usoTT8rO7aKmtN8uNigSn3/vdIOHKDxglsbdMPBLUXmDIRaQy3wszNRS53ROKXF
8iPCqck0lDlF+8lFB3iRriAIeEueMvrE7j9cJfYqYmPZCOdh3HMGAflbPhoKaX8C
XKmMBA8sdn3bHy5HgJLkeVH8fGgKFHhQNho3T9iJ0b1PXqsFW883kjeJAZ9WETp6
T3UWxis3CnGnLnJ2AzY7870c6097z+99FdXBukDa2mdJGBG4dQ5n3dlMCHBYYy9u
KEXFaP2qW20DFAyv6Bgi
=KDWa
-----END PGP SIGNATURE-----

--FmBpKfX8H23NJZGP--