From: Thomas Munro <tmunro@postgresql.org>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Refactor rmtree() to use get_dirent_type().
Date: Tue, 31 Jan 2023 00:49:58 +0000
Message-ID: <E1pMeqT-000nNW-7u@gemulon.postgresql.org> (raw)
Refactor rmtree() to use get_dirent_type().
Switch to get_dirent_type() instead of lstat() while traversing a
directory tree, to see if that fixes the intermittent ENOTEMPTY failures
seen in recent pg_upgrade tests, on Windows CI. While refactoring, also
use AllocateDir() instead of opendir() in the backend, which knows how
to handle descriptor pressure.
Our CI system currently uses Windows Server 2019, a version known not to
have POSIX unlink semantics enabled by default yet, unlike typical
Windows 10 and 11 systems. That might explain why we see this flapping
on CI but (apparently) not in the build farm, though the frequency is
quite low.
The theory is that some directory entry must be in state
STATUS_DELETE_PENDING, which lstat() would report as ENOENT, though
unfortunately we don't know exactly why yet. With this change, rmtree()
will not skip them, and try to unlink (again). Our unlink() wrapper
should either wait a short time for them to go away when some other
process closes the handle, or log a message to tell us the path of the
problem file if not, so we can dig further.
Discussion: https://postgr.es/m/20220919213217.ptqfdlcc5idk5xup%40awork3.anarazel.de
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/54e72b66ed1a55c2fa558bc60d534bdc61dbb62f
Modified Files
--------------
src/common/rmtree.c | 120 ++++++++++++++++++++++++++++------------------------
1 file changed, 64 insertions(+), 56 deletions(-)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-committers@postgresql.org
Cc: tmunro@postgresql.org
Subject: Re: pgsql: Refactor rmtree() to use get_dirent_type().
In-Reply-To: <E1pMeqT-000nNW-7u@gemulon.postgresql.org>
References: <E1pMeqT-000nNW-7u@gemulon.postgresql.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox