From: Tom Lane <tgl@sss.pgh.pa.us>
To: Justin Pryzby <pryzby@telsasoft.com>
cc: pgsql-hackers@postgresql.org, Fabien COELHO <coelho@cri.ensmp.fr>,
        Alvaro Herrera <alvherre@2ndquadrant.com>,
        David Steele <david@pgmasters.net>,
        "Bossart,
    Nathan" <bossartn@amazon.com>,
        Thomas Munro <thomas.munro@gmail.com>
Subject: Re: pg11+: pg_ls_*dir LIMIT 1: temporary files .. not closed at
 end-of-transaction
In-reply-to: <28906.1585422475@sss.pgh.pa.us>
References: <27334.1583692669@sss.pgh.pa.us>
 <20200308191456.GD1357@telsasoft.com> <5679.1583696409@sss.pgh.pa.us>
 <20200311111921.GQ29065@telsasoft.com> <21724.1583955158@sss.pgh.pa.us>
 <20200312121156.GB29065@telsasoft.com> <20200316155306.GM26184@telsasoft.com>
 <3061.1584409130@sss.pgh.pa.us> <20200317020017.GT26184@telsasoft.com>
 <24244.1585415634@sss.pgh.pa.us> <20200328183904.GI20103@telsasoft.com>
 <28906.1585422475@sss.pgh.pa.us>
Comments: In-reply-to Tom Lane <tgl@sss.pgh.pa.us>
	message dated "Sat, 28 Mar 2020 15:07:55 -0400"
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="----- =_aaaaaaaaaa0"
Content-ID: <27037.1585499800.0@sss.pgh.pa.us>
Date: Sun, 29 Mar 2020 12:37:05 -0400
Message-ID: <27064.1585499825@sss.pgh.pa.us>
Precedence: bulk

------- =_aaaaaaaaaa0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <27037.1585499800.1@sss.pgh.pa.us>
Content-Transfer-Encoding: quoted-printable

I wrote:
> Justin Pryzby <pryzby@telsasoft.com> writes:
>> Maybe we should lstat() the file to determine if it's a dangling link; =
if
>> lstat() fails, then skip it.  Currently, we use stat(), which shows met=
data of
>> a link's *target*.  Maybe we'd change that.

> Hm, good point that ENOENT could refer to a symlink's target.  Still,
> I'm not sure it's worth going out of our way to disambiguate that,
> given that these directories aren't really supposed to contain symlinks.
> (And on the third hand, if they aren't supposed to, then maybe these
> functions needn't look through any symlinks?  In which case just
> substituting lstat for stat would resolve the ambiguity.)

After looking at the callers of pg_ls_dir_files, and noticing that
it's already defined to ignore anything that's not a regular file,
I think switching to lstat makes sense.

I also grepped the other uses of ReadDir[Extended], and didn't see
any other ones that seemed desperately in need of changing.

So the attached seems like a sufficient fix.

			regards, tom lane


------- =_aaaaaaaaaa0
Content-Type: text/x-diff;
	name="ignore-concurrent-removals-in-pg_ls_dir_files.patch";
	charset="us-ascii"
Content-ID: <27037.1585499800.2@sss.pgh.pa.us>
Content-Description: ignore-concurrent-removals-in-pg_ls_dir_files.patch
Content-Transfer-Encoding: quoted-printable

diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfi=
le.c
index 01185f2..8429a12 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -596,10 +596,15 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char =
*dir, bool missing_ok)
 =

 		/* Get the file info */
 		snprintf(path, sizeof(path), "%s/%s", dir, de->d_name);
-		if (stat(path, &attrib) < 0)
+		if (lstat(path, &attrib) < 0)
+		{
+			/* Ignore concurrently-deleted files, else complain */
+			if (errno =3D=3D ENOENT)
+				continue;
 			ereport(ERROR,
 					(errcode_for_file_access(),
 					 errmsg("could not stat file \"%s\": %m", path)));
+		}
 =

 		/* Ignore anything but regular files */
 		if (!S_ISREG(attrib.st_mode))

------- =_aaaaaaaaaa0--