From: Fabien COELHO <coelho@cri.ensmp.fr>
To: Justin Pryzby <pryzby@telsasoft.com>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: David Steele <david@pgmasters.net>
Cc: pgsql-hackers@postgresql.org, "Bossart, Nathan" <bossartn@amazon.com>
Cc: Thomas Munro <thomas.munro@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Subject: Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)
Date: Mon, 16 Mar 2020 19:21:06 +0100 (CET)
Message-ID: <alpine.DEB.2.21.2003161903360.17290@pseudo> (raw)
In-Reply-To: <20200316154136.GK26184@telsasoft.com>
References: <20200303202313.GA28076@alvherre.pgsql>
<20200305161838.GJ684@telsasoft.com>
<20200306233507.GN684@telsasoft.com>
<alpine.DEB.2.21.2003071125340.21542@pseudo>
<20200307214010.GB1357@telsasoft.com>
<20200310183037.GA29065@telsasoft.com>
<20200313131232.GO29065@telsasoft.com>
<alpine.DEB.2.21.2003151146370.12715@pseudo>
<20200315212729.GC26184@telsasoft.com>
<alpine.DEB.2.21.2003161607590.19856@pseudo>
<20200316154136.GK26184@telsasoft.com>
Hello Justin,
>> psql> SELECT * FROM pg_ls_dir_recurse('.');
>> ERROR: could not stat file "./base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo": Too many levels of symbolic links
>> CONTEXT: SQL function "pg_ls_dir_recurse" statement 1
>>
>> This probably means using lstat instead of (in supplement to?) stat, and
>> probably tell if something is a link, and if so not recurse in them.
>
> Thanks for looking.
>
> I think that opens up a can of worms. I don't want to go into the business of
> re-implementing all of find(1) - I count ~128 flags (most of which take
> arguments). You're referring to find -L vs find -P, and some people would want
> one and some would want another. And don't forget about find -H...
This is not the point. The point is that a link can change a finite tree
into cyclic graph, and you do not want to delve into that, ever.
The "find" command, by default, does not recurse into a link because of
said problem, and the user *must* ask for it and assume the infinite loop
if any.
So if you implement one behavior, it should be not recursing into links.
Franckly, I would not provide the recurse into link alternative, but it
could be implemented if someone wants it, and the problem that come with
it.
> pg_stat_file doesn't expose the file type (I guess because it's not portable?),
You are right that Un*x and Windows are not the same wrt link. It seems
that there is already something about that in port:
"./src/port/dirmod.c:pgwin32_is_junction(const char *path)"
So most of the details are already hidden.
> and I think it's outside the scope of this patch to change that. Maybe it
> suggests that the pg_ls_dir_recurse patch should be excluded.
IMHO, I really think that it should be included. Dealing with links is no
big deal, but you need an additional column in _metadata to tell it is a
link, and there is a ifdef because testing is a little different between
unix and windows. I'd guess around 10-20 lines of code added.
> ISTM if someone wants to recursively list a directory, they should avoid
> putting cycles there, or permission errors, or similar.
Hmmm. I'd say the user should like to be able to call the function and
never have a bad experience with it such as a failure on an infinite loop.
> Or they should write their own C extension that borrows from
> pg_ls_dir_files but handles more arguments.
ISTM that the point of your patch is to provide the basic tool needed to
list directories contents, and handling links somehow is a necessary part
of that.
--
Fabien.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: coelho@cri.ensmp.fr, pryzby@telsasoft.com, alvherre@2ndquadrant.com, david@pgmasters.net, bossartn@amazon.com, thomas.munro@gmail.com, tgl@sss.pgh.pa.us
Subject: Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)
In-Reply-To: <alpine.DEB.2.21.2003161903360.17290@pseudo>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox