From: Justin Pryzby <pryzby@telsasoft.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-hackers@postgresql.org, Fabien COELHO <coelho@cri.ensmp.fr>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: David Steele <david@pgmasters.net>
Cc: Bossart, Nathan <bossartn@amazon.com>
Cc: Thomas Munro <thomas.munro@gmail.com>
Subject: Re: pg11+: pg_ls_*dir LIMIT 1: temporary files .. not closed at end-of-transaction
Date: Sun, 29 Mar 2020 15:12:15 -0500
Message-ID: <20200329201215.GM20103@telsasoft.com> (raw)
In-Reply-To: <29512.1585502524@sss.pgh.pa.us>
References: <20200312121156.GB29065@telsasoft.com>
<20200316155306.GM26184@telsasoft.com>
<3061.1584409130@sss.pgh.pa.us>
<20200317020017.GT26184@telsasoft.com>
<24244.1585415634@sss.pgh.pa.us>
<20200328183904.GI20103@telsasoft.com>
<28906.1585422475@sss.pgh.pa.us>
<27064.1585499825@sss.pgh.pa.us>
<20200329171415.GL20103@telsasoft.com>
<29512.1585502524@sss.pgh.pa.us>
On Sun, Mar 29, 2020 at 01:22:04PM -0400, Tom Lane wrote:
> Justin Pryzby <pryzby@telsasoft.com> writes:
> > On Sun, Mar 29, 2020 at 12:37:05PM -0400, Tom Lane wrote:
> >> After looking at the callers of pg_ls_dir_files, and noticing that
> >> it's already defined to ignore anything that's not a regular file,
> >> I think switching to lstat makes sense.
>
> > Yea, only pg_ls_dir() shows special file types (and currently the others even
> > hide dirs).
>
> > The essence of your patch is to ignore ENOENT, but you also changed to use
> > lstat(), which seems unrelated. That means we'll now hide (non-broken)
> > symlinks. Is that intentional/needed ?
>
> Well, the following comment says "ignore anything but regular files",
> so I'm supposing that that is the behavior that we actually want here
> and failed to implement correctly. There might be scope for
> additional directory-reading functions, but I'd think you'd want
> more information (such as the file type) returned from anything
> that doesn't act this way.
Maybe pg_stat_file() deserves similar attention ? Right now, it'll fail on a
broken link. If we changed it to lstat(), then it'd work, but it'd also show
metadata for the *link* rather than its target.
Patch proposed as v14-0001 patch here may be relevant:
https://www.postgresql.org/message-id/20200317190401.GY26184%40telsasoft.com
- indicating if it is a directory. Typical usages include:
+ indicating if it is a directory (or a symbolic link to a directory).
...
> In practice, since these directories shouldn't contain symlinks,
> it's likely moot. The only place in PG data directories where
> we actually expect symlinks is pg_tablespace ... and that contains
> symlinks to directories, so that this function would ignore them
> anyway.
I wouldn't hesitate to make symlinks, at least in log. It's surprising when
files are hidden, but I won't argue about the best behavior here.
I'm thinking of distributions or local configurations that use
/var/log/postgresql. I didn't remember or didn't realize, but it looks like
debian's packages use logging_collector=off and then launch postmaster with
2> /var/log/postgres/... It seems reasonable to do something like:
log/huge-querylog.csv => /zfs/compressed/...
--
Justin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: pryzby@telsasoft.com, tgl@sss.pgh.pa.us, coelho@cri.ensmp.fr, alvherre@2ndquadrant.com, david@pgmasters.net, bossartn@amazon.com, thomas.munro@gmail.com
Subject: Re: pg11+: pg_ls_*dir LIMIT 1: temporary files .. not closed at end-of-transaction
In-Reply-To: <20200329201215.GM20103@telsasoft.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox