agora inbox for pgsql-hackers@postgresql.orghelp / color / mirror / Atom feed
[PATCH v31 07/11] pg_ls_*dir to return all the metadata from pg_stat_file.. 23+ messages / 2 participants [nested] [flat]
* [PATCH v19 09/10] pg_ls_*dir to return all the metadata from pg_stat_file.. @ 2020-03-10 02:56 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 23+ messages in thread From: Justin Pryzby @ 2020-03-10 02:56 UTC (permalink / raw) ..but it doesn't seem worth factoring out the common bits, since stat_file doesn't return a name, so all the field numbers are off by one. NOTE, the atime is now shown where the mtime used to be. Need catversion bump --- doc/src/sgml/func.sgml | 37 ++++++++--- src/backend/utils/adt/genfile.c | 65 +++++++++----------- src/include/catalog/pg_proc.dat | 34 +++++----- src/test/regress/expected/misc_functions.out | 16 ++--- src/test/regress/output/tablespace.source | 8 +-- 5 files changed, 86 insertions(+), 74 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index c097977d9b..cee8f28ef7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25722,7 +25722,10 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> @@ -25742,7 +25745,10 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> @@ -25774,13 +25780,16 @@ LATERAL pg_ls_dir_recurse(dir) AS a; <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's log directory, - return the file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + return the file's name, along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -25799,13 +25808,16 @@ LATERAL pg_ls_dir_recurse(dir) AS a; <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's write-ahead log (WAL) directory, list the - file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + file's name along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special files types are excluded. </para> <para> @@ -25824,14 +25836,17 @@ LATERAL pg_ls_dir_recurse(dir) AS a; <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's WAL archive status directory - (<filename>pg_wal/archive_status</filename>), list the file's - name, size, last modification time (mtime), and a boolean indicating if - the file is a directory. + (<filename>pg_wal/archive_status</filename>), list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -25851,13 +25866,17 @@ LATERAL pg_ls_dir_recurse(dir) AS a; <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the temporary directory within the given - <parameter>tablespace</parameter>, return the file's name, size, last - modification time (mtime) and a boolean indicating if the file is a directory. + <parameter>tablespace</parameter>, list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Directories are used for temporary files shared by parallel processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is examined. diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 91bf8c69e9..10780d3fb1 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -36,6 +36,8 @@ #include "utils/syscache.h" #include "utils/timestamp.h" +static void tuple_from_stat(struct stat *fst, const char *path, Datum *values, + bool *isnull); static Datum pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags); #define LS_DIR_ISDIR (1<<0) /* Show column: isdir */ @@ -367,6 +369,28 @@ pg_read_binary_file_all(PG_FUNCTION_ARGS) return pg_read_binary_file(fcinfo); } +/* + * Populate values and isnull from fst and path. + * Used for pg_stat_file() and pg_stat_dir_files() + * isnull is assumed to have been zeroed. + */ +static void +tuple_from_stat(struct stat *fst, const char *path, Datum *values, bool *isnull) +{ + values[0] = Int64GetDatum((int64) fst->st_size); + values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_atime)); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_mtime)); + /* Unix has file status change time, while Win32 has creation time */ +#if !defined(WIN32) && !defined(__CYGWIN__) + values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); + isnull[4] = true; +#else + isnull[3] = true; + values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); +#endif + values[5] = BoolGetDatum(S_ISDIR(fst->st_mode)); +} + /* * stat a file */ @@ -418,25 +442,7 @@ pg_stat_file(PG_FUNCTION_ARGS) BlessTupleDesc(tupdesc); memset(isnull, false, sizeof(isnull)); - - values[0] = Int64GetDatum((int64) fst.st_size); - values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_atime)); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_mtime)); - /* Unix has file status change time, while Win32 has creation time */ -#if !defined(WIN32) && !defined(__CYGWIN__) - values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); - isnull[4] = true; -#else - isnull[3] = true; - values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); -#endif - values[5] = BoolGetDatum(S_ISDIR(fst.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(filename)) - values[5] = BoolGetDatum(false); -#endif - + tuple_from_stat(&fst, filename, values, isnull); tuple = heap_form_tuple(tupdesc, values, isnull); pfree(filename); @@ -578,8 +584,8 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) while ((de = ReadDir(dirdesc, dir)) != NULL) { - Datum values[4]; - bool nulls[4]; + Datum values[7]; + bool nulls[7]; char path[MAXPGPATH * 2]; struct stat attrib; @@ -618,23 +624,10 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) continue; } + memset(nulls, 0, sizeof(nulls)); values[0] = CStringGetTextDatum(de->d_name); if (flags & LS_DIR_METADATA) - { - values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); - if (flags & LS_DIR_ISDIR) - { - values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(path)) - values[3] = BoolGetDatum(false); -#endif - } - } - - memset(nulls, 0, sizeof(nulls)); + tuple_from_stat(&attrib, path, 1+values, 1+nulls); tuplestore_putvalues(tupstore, tupdesc, values, nulls); } diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index bac1226db7..c48c0ca2c8 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10878,48 +10878,48 @@ { oid => '3353', descr => 'list files in the log directory', proname => 'pg_ls_logdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_logdir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logdir' }, { oid => '3354', descr => 'list of files in the WAL directory', proname => 'pg_ls_waldir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_waldir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_waldir' }, { oid => '5031', descr => 'list of files in the archive_status directory', proname => 'pg_ls_archive_statusdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_archive_statusdir' }, { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{tablespace,name,size,modification,isdir}', + proallargtypes => '{oid,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{tablespace,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9979', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text bool bool', - proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}', - proargnames => '{dirname,missing_ok,include_dot_dirs,name,size,modification,isdir}', + proallargtypes => '{text,bool,bool,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o,o,o,o}', + proargnames => '{dirname,missing_ok,include_dot_dirs,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata' }, { oid => '9980', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{dirname,name,size,modification,isdir}', + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{dirname,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata_1arg' }, { oid => '9981', descr => 'list all files in a directory recursively', proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', - proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}', - prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false) as a where ls.isdir) select * from ls" }, + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargnames => '{dirname,name,size,access,modification,change,creation,isdir}', proargmodes => '{i,o,o,o,o,o,o,o}', + prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.access, a.modification, a.change, a.creation, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false) as a where ls.isdir) select * from ls" }, # hash partitioning constraint function { oid => '5028', descr => 'hash partition CHECK constraint', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 4188d684f0..cce84a60a9 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -157,8 +157,8 @@ select count(*) > 0 as ok from (select pg_ls_waldir()) ss; -- Test not-run-to-completion cases. select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select count(*) > 0 as ok from (select * from pg_ls_waldir() limit 1) ss; @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; @@ -239,8 +239,8 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; -- -- Check that expected columns are present select * from pg_ls_dir_metadata('.') limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix @@ -253,8 +253,8 @@ SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal'; -- Check that expected columns are present SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index ba9a3fe29a..1e1e02b589 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,15 +17,15 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet. -- The name='' condition is never true, so the function runs to completion but returns zero rows. SELECT * FROM pg_ls_logdir() WHERE name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --S0GG+JvAI2G0KxBG Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v19-0010-pg_ls_-to-show-file-type-and-show-special-files.patch" ^ permalink raw reply [nested|flat] 23+ messages in thread
* [PATCH v24 08/11] pg_ls_*dir to return all the metadata from pg_stat_file.. @ 2020-03-10 02:56 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 23+ messages in thread From: Justin Pryzby @ 2020-03-10 02:56 UTC (permalink / raw) ..but it doesn't seem worth factoring out the common bits, since stat_file doesn't return a name, so all the field numbers are off by one. NOTE, the atime is now shown where the mtime used to be. Need catversion bump --- doc/src/sgml/func.sgml | 34 +++++++--- src/backend/utils/adt/genfile.c | 71 +++++++++----------- src/include/catalog/pg_proc.dat | 28 ++++---- src/test/regress/expected/misc_functions.out | 12 ++-- src/test/regress/output/tablespace.source | 8 +-- 5 files changed, 81 insertions(+), 72 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 191cf7c9fa..e3d7df2dd7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25743,13 +25743,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's log directory, - return the file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + return the file's name, along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -25770,7 +25773,10 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>filename</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> @@ -25790,13 +25796,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's write-ahead log (WAL) directory, list the - file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + file's name along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special files types are excluded. </para> <para> @@ -25815,14 +25824,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's WAL archive status directory - (<filename>pg_wal/archive_status</filename>), list the file's - name, size, last modification time (mtime), and a boolean indicating if - the file is a directory. + (<filename>pg_wal/archive_status</filename>), list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -25842,13 +25854,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the temporary directory within the given - <parameter>tablespace</parameter>, return the file's name, size, last - modification time (mtime) and a boolean indicating if the file is a directory. + <parameter>tablespace</parameter>, list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Directories are used for temporary files shared by parallel processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is examined. diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 07fce2a9bc..9ff06f4158 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -36,6 +36,8 @@ #include "utils/syscache.h" #include "utils/timestamp.h" +static void tuple_from_stat(struct stat *fst, const char *path, Datum *values, + bool *nulls); static Datum pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags); #define LS_DIR_ISDIR (1<<0) /* Show column: isdir */ @@ -397,6 +399,28 @@ pg_read_binary_file_all(PG_FUNCTION_ARGS) return pg_read_binary_file(fcinfo); } +/* + * Populate values and nulls from fst and path. + * Used for pg_stat_file() and pg_ls_dir_files() + * nulls is assumed to have been zerod. + */ +static void +tuple_from_stat(struct stat *fst, const char *path, Datum *values, bool *nulls) +{ + values[0] = Int64GetDatum((int64) fst->st_size); + values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_atime)); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_mtime)); + /* Unix has file status change time, while Win32 has creation time */ +#if !defined(WIN32) && !defined(__CYGWIN__) + values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); + nulls[4] = true; +#else + nulls[3] = true; + values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); +#endif + values[5] = BoolGetDatum(S_ISDIR(fst->st_mode)); +} + /* * stat a file */ @@ -407,7 +431,7 @@ pg_stat_file(PG_FUNCTION_ARGS) char *filename; struct stat fst; Datum values[6]; - bool isnull[6]; + bool nulls[6]; HeapTuple tuple; TupleDesc tupdesc; bool missing_ok = false; @@ -447,27 +471,9 @@ pg_stat_file(PG_FUNCTION_ARGS) "isdir", BOOLOID, -1, 0); BlessTupleDesc(tupdesc); - memset(isnull, false, sizeof(isnull)); - - values[0] = Int64GetDatum((int64) fst.st_size); - values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_atime)); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_mtime)); - /* Unix has file status change time, while Win32 has creation time */ -#if !defined(WIN32) && !defined(__CYGWIN__) - values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); - isnull[4] = true; -#else - isnull[3] = true; - values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); -#endif - values[5] = BoolGetDatum(S_ISDIR(fst.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(filename)) - values[5] = BoolGetDatum(false); -#endif - - tuple = heap_form_tuple(tupdesc, values, isnull); + memset(nulls, false, sizeof(nulls)); + tuple_from_stat(&fst, filename, values, nulls); + tuple = heap_form_tuple(tupdesc, values, nulls); pfree(filename); @@ -608,8 +614,8 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) while ((de = ReadDir(dirdesc, dir)) != NULL) { - Datum values[4]; - bool nulls[4]; + Datum values[7]; + bool nulls[7]; char path[MAXPGPATH * 2]; struct stat attrib; @@ -648,23 +654,10 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) continue; } + memset(nulls, false, sizeof(nulls)); values[0] = CStringGetTextDatum(de->d_name); if (flags & LS_DIR_METADATA) - { - values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); - if (flags & LS_DIR_ISDIR) - { - values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(path)) - values[3] = BoolGetDatum(false); -#endif - } - } - - memset(nulls, 0, sizeof(nulls)); + tuple_from_stat(&attrib, path, 1+values, 1+nulls); tuplestore_putvalues(tupstore, tupdesc, values, nulls); } diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index bb57f3bc70..7fea405068 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10951,41 +10951,41 @@ { oid => '3353', descr => 'list files in the log directory', proname => 'pg_ls_logdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_logdir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logdir' }, { oid => '3354', descr => 'list of files in the WAL directory', proname => 'pg_ls_waldir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_waldir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_waldir' }, { oid => '5031', descr => 'list of files in the archive_status directory', proname => 'pg_ls_archive_statusdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_archive_statusdir' }, { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{tablespace,name,size,modification,isdir}', + proallargtypes => '{oid,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{tablespace,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9979', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text bool bool', - proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}', - proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,modification,isdir}', + proallargtypes => '{text,bool,bool,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o,o,o,o}', + proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata' }, { oid => '9980', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{dirname,filename,size,modification,isdir}', + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{dirname,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata_1arg' }, # hash partitioning constraint function diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 800d12b4b6..1f4106cd9c 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -157,8 +157,8 @@ select count(*) > 0 as ok from (select pg_ls_waldir()) ss; -- Test not-run-to-completion cases. select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select count(*) > 0 as ok from (select * from pg_ls_waldir() limit 1) ss; @@ -228,8 +228,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; @@ -245,8 +245,8 @@ select filename, isdir from pg_ls_dir_metadata('.', false, false) where filename -- Check that expected columns are present select * from pg_ls_dir_metadata('.') limit 0; - filename | size | modification | isdir -----------+------+--------------+------- + filename | size | access | modification | change | creation | isdir +----------+------+--------+--------------+--------+----------+------- (0 rows) -- diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index ba9a3fe29a..1e1e02b589 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,15 +17,15 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet. -- The name='' condition is never true, so the function runs to completion but returns zero rows. SELECT * FROM pg_ls_logdir() WHERE name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --mPTHnM80CEnHQ2WJ Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v24-0009-pg_ls_-pg_stat_file-to-show-file-type.patch" ^ permalink raw reply [nested|flat] 23+ messages in thread
* [PATCH v25 08/11] pg_ls_*dir to return all the metadata from pg_stat_file.. @ 2020-03-10 02:56 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 23+ messages in thread From: Justin Pryzby @ 2020-03-10 02:56 UTC (permalink / raw) ..but it doesn't seem worth factoring out the common bits, since stat_file doesn't return a name, so all the field numbers are off by one. NOTE, the atime is now shown where the mtime used to be. Need catversion bump --- doc/src/sgml/func.sgml | 34 +++++++--- src/backend/utils/adt/genfile.c | 71 +++++++++----------- src/include/catalog/pg_proc.dat | 28 ++++---- src/test/regress/expected/misc_functions.out | 12 ++-- src/test/regress/output/tablespace.source | 8 +-- 5 files changed, 81 insertions(+), 72 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index c4a673c8eb..f294730605 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25736,13 +25736,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's log directory, - return the file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + return the file's name, along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -25763,7 +25766,10 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>filename</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> @@ -25783,13 +25789,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's write-ahead log (WAL) directory, list the - file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + file's name along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special files types are excluded. </para> <para> @@ -25808,14 +25817,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's WAL archive status directory - (<filename>pg_wal/archive_status</filename>), list the file's - name, size, last modification time (mtime), and a boolean indicating if - the file is a directory. + (<filename>pg_wal/archive_status</filename>), list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -25835,13 +25847,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the temporary directory within the given - <parameter>tablespace</parameter>, return the file's name, size, last - modification time (mtime) and a boolean indicating if the file is a directory. + <parameter>tablespace</parameter>, list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Directories are used for temporary files shared by parallel processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is examined. diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 07fce2a9bc..9ff06f4158 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -36,6 +36,8 @@ #include "utils/syscache.h" #include "utils/timestamp.h" +static void tuple_from_stat(struct stat *fst, const char *path, Datum *values, + bool *nulls); static Datum pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags); #define LS_DIR_ISDIR (1<<0) /* Show column: isdir */ @@ -397,6 +399,28 @@ pg_read_binary_file_all(PG_FUNCTION_ARGS) return pg_read_binary_file(fcinfo); } +/* + * Populate values and nulls from fst and path. + * Used for pg_stat_file() and pg_ls_dir_files() + * nulls is assumed to have been zerod. + */ +static void +tuple_from_stat(struct stat *fst, const char *path, Datum *values, bool *nulls) +{ + values[0] = Int64GetDatum((int64) fst->st_size); + values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_atime)); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_mtime)); + /* Unix has file status change time, while Win32 has creation time */ +#if !defined(WIN32) && !defined(__CYGWIN__) + values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); + nulls[4] = true; +#else + nulls[3] = true; + values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); +#endif + values[5] = BoolGetDatum(S_ISDIR(fst->st_mode)); +} + /* * stat a file */ @@ -407,7 +431,7 @@ pg_stat_file(PG_FUNCTION_ARGS) char *filename; struct stat fst; Datum values[6]; - bool isnull[6]; + bool nulls[6]; HeapTuple tuple; TupleDesc tupdesc; bool missing_ok = false; @@ -447,27 +471,9 @@ pg_stat_file(PG_FUNCTION_ARGS) "isdir", BOOLOID, -1, 0); BlessTupleDesc(tupdesc); - memset(isnull, false, sizeof(isnull)); - - values[0] = Int64GetDatum((int64) fst.st_size); - values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_atime)); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_mtime)); - /* Unix has file status change time, while Win32 has creation time */ -#if !defined(WIN32) && !defined(__CYGWIN__) - values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); - isnull[4] = true; -#else - isnull[3] = true; - values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); -#endif - values[5] = BoolGetDatum(S_ISDIR(fst.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(filename)) - values[5] = BoolGetDatum(false); -#endif - - tuple = heap_form_tuple(tupdesc, values, isnull); + memset(nulls, false, sizeof(nulls)); + tuple_from_stat(&fst, filename, values, nulls); + tuple = heap_form_tuple(tupdesc, values, nulls); pfree(filename); @@ -608,8 +614,8 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) while ((de = ReadDir(dirdesc, dir)) != NULL) { - Datum values[4]; - bool nulls[4]; + Datum values[7]; + bool nulls[7]; char path[MAXPGPATH * 2]; struct stat attrib; @@ -648,23 +654,10 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) continue; } + memset(nulls, false, sizeof(nulls)); values[0] = CStringGetTextDatum(de->d_name); if (flags & LS_DIR_METADATA) - { - values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); - if (flags & LS_DIR_ISDIR) - { - values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(path)) - values[3] = BoolGetDatum(false); -#endif - } - } - - memset(nulls, 0, sizeof(nulls)); + tuple_from_stat(&attrib, path, 1+values, 1+nulls); tuplestore_putvalues(tupstore, tupdesc, values, nulls); } diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index fc6402d19b..3f43958423 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10952,41 +10952,41 @@ { oid => '3353', descr => 'list files in the log directory', proname => 'pg_ls_logdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_logdir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logdir' }, { oid => '3354', descr => 'list of files in the WAL directory', proname => 'pg_ls_waldir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_waldir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_waldir' }, { oid => '5031', descr => 'list of files in the archive_status directory', proname => 'pg_ls_archive_statusdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_archive_statusdir' }, { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{tablespace,name,size,modification,isdir}', + proallargtypes => '{oid,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{tablespace,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9979', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text bool bool', - proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}', - proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,modification,isdir}', + proallargtypes => '{text,bool,bool,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o,o,o,o}', + proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata' }, { oid => '9980', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{dirname,filename,size,modification,isdir}', + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{dirname,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata_1arg' }, # hash partitioning constraint function diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 800d12b4b6..1f4106cd9c 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -157,8 +157,8 @@ select count(*) > 0 as ok from (select pg_ls_waldir()) ss; -- Test not-run-to-completion cases. select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select count(*) > 0 as ok from (select * from pg_ls_waldir() limit 1) ss; @@ -228,8 +228,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; @@ -245,8 +245,8 @@ select filename, isdir from pg_ls_dir_metadata('.', false, false) where filename -- Check that expected columns are present select * from pg_ls_dir_metadata('.') limit 0; - filename | size | modification | isdir -----------+------+--------------+------- + filename | size | access | modification | change | creation | isdir +----------+------+--------+--------------+--------+----------+------- (0 rows) -- diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index ba9a3fe29a..1e1e02b589 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,15 +17,15 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet. -- The name='' condition is never true, so the function runs to completion but returns zero rows. SELECT * FROM pg_ls_logdir() WHERE name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --SBikYMzjhZGK9d4p Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v25-0009-pg_ls_-pg_stat_file-to-show-file-type.patch" ^ permalink raw reply [nested|flat] 23+ messages in thread
* [PATCH v22 09/10] pg_ls_*dir to return all the metadata from pg_stat_file.. @ 2020-03-10 02:56 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 23+ messages in thread From: Justin Pryzby @ 2020-03-10 02:56 UTC (permalink / raw) ..but it doesn't seem worth factoring out the common bits, since stat_file doesn't return a name, so all the field numbers are off by one. NOTE, the atime is now shown where the mtime used to be. Need catversion bump --- doc/src/sgml/func.sgml | 37 +++++++--- src/backend/utils/adt/genfile.c | 71 +++++++++----------- src/include/catalog/pg_proc.dat | 34 +++++----- src/test/regress/expected/misc_functions.out | 16 ++--- src/test/regress/output/tablespace.source | 8 +-- 5 files changed, 89 insertions(+), 77 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 7baf2b31e5..809483feed 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25758,7 +25758,10 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> @@ -25778,7 +25781,10 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> @@ -25810,13 +25816,16 @@ LATERAL pg_ls_dir_recurse(dir) AS a; <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's log directory, - return the file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + return the file's name, along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -25835,13 +25844,16 @@ LATERAL pg_ls_dir_recurse(dir) AS a; <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's write-ahead log (WAL) directory, list the - file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + file's name along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special files types are excluded. </para> <para> @@ -25860,14 +25872,17 @@ LATERAL pg_ls_dir_recurse(dir) AS a; <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's WAL archive status directory - (<filename>pg_wal/archive_status</filename>), list the file's - name, size, last modification time (mtime), and a boolean indicating if - the file is a directory. + (<filename>pg_wal/archive_status</filename>), list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -25887,13 +25902,17 @@ LATERAL pg_ls_dir_recurse(dir) AS a; <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the temporary directory within the given - <parameter>tablespace</parameter>, return the file's name, size, last - modification time (mtime) and a boolean indicating if the file is a directory. + <parameter>tablespace</parameter>, list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Directories are used for temporary files shared by parallel processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is examined. diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 35fe4d7cc1..20b948b746 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -36,6 +36,8 @@ #include "utils/syscache.h" #include "utils/timestamp.h" +static void tuple_from_stat(struct stat *fst, const char *path, Datum *values, + bool *nulls); static Datum pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags); #define LS_DIR_ISDIR (1<<0) /* Show column: isdir */ @@ -397,6 +399,28 @@ pg_read_binary_file_all(PG_FUNCTION_ARGS) return pg_read_binary_file(fcinfo); } +/* + * Populate values and nulls from fst and path. + * Used for pg_stat_file() and pg_stat_dir_files() + * nulls is assumed to have been zerod. + */ +static void +tuple_from_stat(struct stat *fst, const char *path, Datum *values, bool *nulls) +{ + values[0] = Int64GetDatum((int64) fst->st_size); + values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_atime)); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_mtime)); + /* Unix has file status change time, while Win32 has creation time */ +#if !defined(WIN32) && !defined(__CYGWIN__) + values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); + nulls[4] = true; +#else + nulls[3] = true; + values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); +#endif + values[5] = BoolGetDatum(S_ISDIR(fst->st_mode)); +} + /* * stat a file */ @@ -407,7 +431,7 @@ pg_stat_file(PG_FUNCTION_ARGS) char *filename; struct stat fst; Datum values[6]; - bool isnull[6]; + bool nulls[6]; HeapTuple tuple; TupleDesc tupdesc; bool missing_ok = false; @@ -447,27 +471,9 @@ pg_stat_file(PG_FUNCTION_ARGS) "isdir", BOOLOID, -1, 0); BlessTupleDesc(tupdesc); - memset(isnull, false, sizeof(isnull)); - - values[0] = Int64GetDatum((int64) fst.st_size); - values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_atime)); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_mtime)); - /* Unix has file status change time, while Win32 has creation time */ -#if !defined(WIN32) && !defined(__CYGWIN__) - values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); - isnull[4] = true; -#else - isnull[3] = true; - values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); -#endif - values[5] = BoolGetDatum(S_ISDIR(fst.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(filename)) - values[5] = BoolGetDatum(false); -#endif - - tuple = heap_form_tuple(tupdesc, values, isnull); + memset(nulls, false, sizeof(nulls)); + tuple_from_stat(&fst, filename, values, nulls); + tuple = heap_form_tuple(tupdesc, values, nulls); pfree(filename); @@ -608,8 +614,8 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) while ((de = ReadDir(dirdesc, dir)) != NULL) { - Datum values[4]; - bool nulls[4]; + Datum values[7]; + bool nulls[7]; char path[MAXPGPATH * 2]; struct stat attrib; @@ -648,23 +654,10 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) continue; } + memset(nulls, false, sizeof(nulls)); values[0] = CStringGetTextDatum(de->d_name); if (flags & LS_DIR_METADATA) - { - values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); - if (flags & LS_DIR_ISDIR) - { - values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(path)) - values[3] = BoolGetDatum(false); -#endif - } - } - - memset(nulls, 0, sizeof(nulls)); + tuple_from_stat(&attrib, path, 1+values, 1+nulls); tuplestore_putvalues(tupstore, tupdesc, values, nulls); } diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 0235042e26..3f62aea9c9 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10933,29 +10933,29 @@ { oid => '3353', descr => 'list files in the log directory', proname => 'pg_ls_logdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_logdir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logdir' }, { oid => '3354', descr => 'list of files in the WAL directory', proname => 'pg_ls_waldir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_waldir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_waldir' }, { oid => '5031', descr => 'list of files in the archive_status directory', proname => 'pg_ls_archive_statusdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_archive_statusdir' }, { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{tablespace,name,size,modification,isdir}', + proallargtypes => '{oid,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{tablespace,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9147', descr => 'check pages of a relation', proname => 'pg_relation_check_pages', procost => '10000', prorows => '20', @@ -10967,21 +10967,21 @@ { oid => '9979', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text bool bool', - proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}', - proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,modification,isdir}', + proallargtypes => '{text,bool,bool,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o,o,o,o}', + proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata' }, { oid => '9980', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{dirname,filename,size,modification,isdir}', + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{dirname,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata_1arg' }, { oid => '9981', descr => 'list all files in a directory recursively', proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,text,int8,timestamptz,bool}', - proargnames => '{dirname,path,filename,size,modification,isdir}', proargmodes => '{i,o,o,o,o,o}', - prolang => 'sql', prosrc => "with recursive ls as (select dirname as path, * from pg_ls_dir_metadata(dirname, false, false) union all select coalesce(nullif(parent.path,'.')||'/','')||parent.filename, a.filename, a.size, a.modification, a.isdir from ls as parent, lateral pg_ls_dir_metadata(parent.path||'/'||parent.filename, false, false) as a where parent.isdir) select * from ls" }, + proallargtypes => '{text,text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargnames => '{dirname,path,filename,size,access,modification,change,creation,isdir}', proargmodes => '{i,o,o,o,o,o,o,o,o}', + prolang => 'sql', prosrc => "with recursive ls as (select dirname as path, * from pg_ls_dir_metadata(dirname, false, false) union all select coalesce(nullif(parent.path,'.')||'/','')||parent.filename, a.filename, a.size, a.access, a.modification, a.change, a.creation, a.isdir from ls as parent, lateral pg_ls_dir_metadata(parent.path||'/'||parent.filename, false, false) as a where parent.isdir) select * from ls" }, # hash partitioning constraint function { oid => '5028', descr => 'hash partition CHECK constraint', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 302f348a6d..4be27fe21e 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -157,8 +157,8 @@ select count(*) > 0 as ok from (select pg_ls_waldir()) ss; -- Test not-run-to-completion cases. select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select count(*) > 0 as ok from (select * from pg_ls_waldir() limit 1) ss; @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; @@ -239,8 +239,8 @@ select filename, isdir from pg_ls_dir_metadata('.', false, false) where filename -- Check that expected columns are present select * from pg_ls_dir_metadata('.') limit 0; - filename | size | modification | isdir -----------+------+--------------+------- + filename | size | access | modification | change | creation | isdir +----------+------+--------+--------------+--------+----------+------- (0 rows) -- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix @@ -252,8 +252,8 @@ select path, filename, isdir from pg_ls_dir_recurse('.') where isdir and path='p -- Check that expected columns are present select * from pg_ls_dir_recurse('.') limit 0; - path | filename | size | modification | isdir -------+----------+------+--------------+------- + path | filename | size | access | modification | change | creation | isdir +------+----------+------+--------+--------------+--------+----------+------- (0 rows) -- diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index ba9a3fe29a..1e1e02b589 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,15 +17,15 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet. -- The name='' condition is never true, so the function runs to completion but returns zero rows. SELECT * FROM pg_ls_logdir() WHERE name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --d6Gm4EdcadzBjdND Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v22-0010-pg_ls_-pg_stat_file-to-show-file-type.patch" ^ permalink raw reply [nested|flat] 23+ messages in thread
* [PATCH v9 11/11] pg_ls_*dir to return all the metadata from pg_stat_file.. @ 2020-03-10 02:56 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 23+ messages in thread From: Justin Pryzby @ 2020-03-10 02:56 UTC (permalink / raw) ..but it doesn't seem worth factoring out the common bits, since stat_file doesn't return a name, so all the field numbers are off by one. Need catversion bump --- doc/src/sgml/func.sgml | 30 +++++++++------------ src/backend/utils/adt/genfile.c | 32 ++++++++++++++++------- src/include/catalog/pg_proc.dat | 30 ++++++++++----------- src/test/regress/output/tablespace.source | 8 +++--- 4 files changed, 55 insertions(+), 45 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index d0b782d803..ae6c004a6d 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -21348,8 +21348,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - For each file in the log directory, list the file's name, size, last - modification time, and a boolean indicating if it is a directory. + For each file in the log directory, list the file and its metadata. Access is granted to members of the <literal>pg_monitor</literal> role and may be granted to other non-superuser roles. </entry> @@ -21360,8 +21359,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - For each file in the WAL directory, list the file's name, size, last - modification time, and a boolean indicating if it is a directory. + For each file in the WAL directory, list the file and its metadata. Access is granted to members of the <literal>pg_monitor</literal> role and may be granted to other non-superuser roles. </entry> @@ -21372,9 +21370,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - For each file in the WAL archive status directory, list the file's - name, size, last modification time, and a boolean indicating if it is a - directory. Access is granted to members of the + For each file in the WAL archive status directory, list the file and its metadata. + Access is granted to members of the <literal>pg_monitor</literal> role and may be granted to other non-superuser roles. </entry> @@ -21386,8 +21383,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <entry><type>setof record</type></entry> <entry> For the temporary directory within <parameter>tablespace</parameter>, - list each file's name, size, last modification time, and a boolean - indicating if it is a directory. If <parameter>tablespace</parameter> + list each file and its metadata. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. Access is granted to members of the <literal>pg_monitor</literal> role and may be granted to other non-superuser roles. @@ -21463,8 +21459,8 @@ SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'') </indexterm> <para> <function>pg_ls_logdir</function> lists each file in the log directory, - along with file's size, last modification time, and a boolean - indicating if the file is a directory. By default, only superusers + along with the metadata columns returned by <function>pg_stat_file</function>. + By default, only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. Filenames beginning with a dot and special file types are not shown. @@ -21475,8 +21471,8 @@ SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'') </indexterm> <para> <function>pg_ls_waldir</function> lists each file in the WAL directory, - along with the file's size, last modification time, and a boolean - indicating if the file is a directory. By default, only superusers + along with the metadata columns returned by <function>pg_stat_file</function>. + By default, only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. @@ -21488,8 +21484,8 @@ SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'') </indexterm> <para> <function>pg_ls_archive_statusdir</function> lists each file in the WAL - archive status directory, along with the file's size, last modification - time, and a boolean indicating if the file is a directory. By default, only + archive status directory, along with the metadata columns returned by + <function>pg_stat_file</function>. By default, only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. @@ -21502,8 +21498,8 @@ SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'') <para> <function>pg_ls_tmpdir</function> lists each file in the temporary file directory for the specified <parameter>tablespace</parameter>, along with - its size, last modified time (mtime) and a boolean indicating if the file is a - directory. Directories are used for temporary files shared by parallel + the metadata columns returned by <function>pg_stat_file</function>. + Directories are used for temporary files shared by parallel processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. By default only superusers and members of the <literal>pg_monitor</literal> diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 387114d4ee..1fe7e0181d 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -510,16 +510,22 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) fctx = palloc(sizeof(directory_fctx)); - tupdesc = CreateTemplateTupleDesc((flags&FLAG_METADATA) ? 4 : 1); + tupdesc = CreateTemplateTupleDesc((flags&FLAG_METADATA) ? 7 : 1); TupleDescInitEntry(tupdesc, (AttrNumber) 1, "name", TEXTOID, -1, 0); if (flags&FLAG_METADATA) { TupleDescInitEntry(tupdesc, (AttrNumber) 2, "size", INT8OID, -1, 0); - TupleDescInitEntry(tupdesc, (AttrNumber) 3, "modification", + TupleDescInitEntry(tupdesc, (AttrNumber) 3, + "access", TIMESTAMPTZOID, -1, 0); + TupleDescInitEntry(tupdesc, (AttrNumber) 4, "modification", TIMESTAMPTZOID, -1, 0); - TupleDescInitEntry(tupdesc, (AttrNumber) 4, "isdir", + TupleDescInitEntry(tupdesc, (AttrNumber) 5, + "change", TIMESTAMPTZOID, -1, 0); + TupleDescInitEntry(tupdesc, (AttrNumber) 6, + "creation", TIMESTAMPTZOID, -1, 0); + TupleDescInitEntry(tupdesc, (AttrNumber) 7, "isdir", BOOLOID, -1, 0); } @@ -551,8 +557,8 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) while ((de = ReadDir(fctx->dirdesc, fctx->location)) != NULL) { - Datum values[4]; - bool nulls[4]; + Datum values[7]; + bool nulls[7]; char path[MAXPGPATH * 2]; struct stat attrib; HeapTuple tuple; @@ -579,17 +585,25 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) else if (!S_ISREG(attrib.st_mode)) continue; + memset(nulls, 0, sizeof(nulls)); if (flags & FLAG_METADATA) { values[0] = CStringGetTextDatum(de->d_name); values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); - values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_atime)); + values[3] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); + /* Unix has file status change time, while Win32 has creation time */ +#if !defined(WIN32) && !defined(__CYGWIN__) + values[4] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_ctime)); + nulls[5] = true; +#else + nulls[4] = true; + values[5] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_ctime)); +#endif + values[6] = BoolGetDatum(S_ISDIR(attrib.st_mode)); } else SRF_RETURN_NEXT(funcctx, CStringGetTextDatum(de->d_name)); - memset(nulls, 0, sizeof(nulls)); - tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls); SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple)); } diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 0e5a570285..b571666a79 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -6147,9 +6147,9 @@ { oid => '8511', descr => 'list all files in a directory recursively', proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', - proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}', - prolang => 'sql', prosrc => "WITH RECURSIVE x AS (SELECT * FROM pg_ls_dir_metadata(dirname, true, false) UNION ALL SELECT x.name||'/'||a.name, a.size, a.modification, a.isdir FROM x, pg_ls_dir_metadata(dirname||'/'||x.name, true, false)a WHERE x.isdir) SELECT * FROM x" }, + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargnames => '{dirname,name,size,access,modification,change,creation,isdir}', proargmodes => '{i,o,o,o,o,o,o,o}', + prolang => 'sql', prosrc => "WITH RECURSIVE x AS (SELECT * FROM pg_ls_dir_metadata(dirname, true, false) UNION ALL SELECT x.name||'/'||a.name, a.size,a.access,a.modification,a.change,a.creation,a.isdir FROM x, pg_ls_dir_metadata(dirname||'/'||x.name, true, false)a WHERE x.isdir) SELECT * FROM x" }, { oid => '2626', descr => 'sleep for the specified time in seconds', proname => 'pg_sleep', provolatile => 'v', prorettype => 'void', @@ -10725,35 +10725,35 @@ { oid => '3353', descr => 'list files in the log directory', proname => 'pg_ls_logdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_logdir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logdir' }, { oid => '3354', descr => 'list of files in the WAL directory', proname => 'pg_ls_waldir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_waldir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_waldir' }, { oid => '5031', descr => 'list of files in the archive_status directory', proname => 'pg_ls_archive_statusdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_archive_statusdir' }, { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{tablespace,name,size,modification,isdir}', + proallargtypes => '{oid,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{tablespace,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text bool bool', - proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}', - proargnames => '{dirname,missing_ok,include_dot_dirs,name,size,modification,isdir}', + proallargtypes => '{text,bool,bool,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o,o,o,o}', + proargnames => '{dirname,missing_ok,include_dot_dirs,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata' }, # hash partitioning constraint function diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index dacaafcae6..1e1e02b589 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,15 +17,15 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet. -- The name='' condition is never true, so the function runs to completion but returns zero rows. SELECT * FROM pg_ls_logdir() WHERE name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --32u276st3Jlj2kUU-- ^ permalink raw reply [nested|flat] 23+ messages in thread
* [PATCH v37 07/11] pg_ls_*dir to return all the metadata from pg_stat_file.. @ 2020-03-10 02:56 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 23+ messages in thread From: Justin Pryzby @ 2020-03-10 02:56 UTC (permalink / raw) ..but it doesn't seem worth factoring out the common bits, since stat_file doesn't return a name, so all the field numbers are off by one. NOTE, the atime is now shown where the mtime used to be. Need catversion bump --- doc/src/sgml/func.sgml | 34 ++++++--- src/backend/utils/adt/genfile.c | 72 +++++++++----------- src/include/catalog/pg_proc.dat | 42 ++++++------ src/test/regress/expected/misc_functions.out | 40 +++++------ src/test/regress/expected/tablespace.out | 8 +-- 5 files changed, 102 insertions(+), 94 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 3a4379058dd..59bbcb796a6 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25911,7 +25911,10 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560 <returnvalue>setof record</returnvalue> ( <parameter>filename</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> @@ -27486,13 +27489,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's log directory, - return the file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + return the file's name, along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -27511,13 +27517,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's write-ahead log (WAL) directory, list the - file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + file's name along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special files types are excluded. </para> <para> @@ -27609,14 +27618,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's WAL archive status directory - (<filename>pg_wal/archive_status</filename>), list the file's - name, size, last modification time (mtime), and a boolean indicating if - the file is a directory. + (<filename>pg_wal/archive_status</filename>), list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -27636,13 +27648,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the temporary directory within the given - <parameter>tablespace</parameter>, return the file's name, size, last - modification time (mtime) and a boolean indicating if the file is a directory. + <parameter>tablespace</parameter>, list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Directories are used for temporary files shared by parallel processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is examined. diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 5402ecb3d05..ce4a37a3301 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -37,6 +37,8 @@ #include "utils/syscache.h" #include "utils/timestamp.h" +static void values_from_stat(struct stat *fst, const char *path, Datum *values, + bool *nulls); static Datum pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags); #define LS_DIR_ISDIR (1<<0) /* Show column: isdir */ @@ -464,6 +466,28 @@ pg_read_binary_file_all_missing(PG_FUNCTION_ARGS) PG_RETURN_BYTEA_P(ret); } +/* + * Populate values and nulls from fst and path. + * Used for pg_stat_file() and pg_ls_dir_files() + * nulls is assumed to have been zerod. + */ +static void +values_from_stat(struct stat *fst, const char *path, Datum *values, bool *nulls) +{ + values[0] = Int64GetDatum((int64) fst->st_size); + values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_atime)); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_mtime)); + /* Unix has file status change time, while Win32 has creation time */ +#if !defined(WIN32) && !defined(__CYGWIN__) + values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); + nulls[4] = true; +#else + nulls[3] = true; + values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); +#endif + values[5] = BoolGetDatum(S_ISDIR(fst->st_mode)); +} + /* * stat a file */ @@ -474,7 +498,7 @@ pg_stat_file(PG_FUNCTION_ARGS) char *filename; struct stat fst; Datum values[6]; - bool isnull[6]; + bool nulls[6]; HeapTuple tuple; TupleDesc tupdesc; bool missing_ok = false; @@ -514,27 +538,9 @@ pg_stat_file(PG_FUNCTION_ARGS) "isdir", BOOLOID, -1, 0); BlessTupleDesc(tupdesc); - memset(isnull, false, sizeof(isnull)); - - values[0] = Int64GetDatum((int64) fst.st_size); - values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_atime)); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_mtime)); - /* Unix has file status change time, while Win32 has creation time */ -#if !defined(WIN32) && !defined(__CYGWIN__) - values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); - isnull[4] = true; -#else - isnull[3] = true; - values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); -#endif - values[5] = BoolGetDatum(S_ISDIR(fst.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(filename)) - values[5] = BoolGetDatum(false); -#endif - - tuple = heap_form_tuple(tupdesc, values, isnull); + memset(nulls, false, sizeof(nulls)); + values_from_stat(&fst, filename, values, nulls); + tuple = heap_form_tuple(tupdesc, values, nulls); pfree(filename); @@ -641,8 +647,8 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) while ((de = ReadDir(dirdesc, dir)) != NULL) { - Datum values[4]; - bool nulls[4]; + Datum values[7]; + bool nulls[7]; char path[MAXPGPATH * 2]; struct stat attrib; @@ -681,24 +687,10 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) continue; } + memset(nulls, false, sizeof(nulls)); values[0] = CStringGetTextDatum(de->d_name); if (flags & LS_DIR_METADATA) - { - values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); - if (flags & LS_DIR_ISDIR) - { -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(path)) - values[3] = BoolGetDatum(false); - else -#endif - values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); - } - } - - memset(nulls, 0, sizeof(nulls)); + values_from_stat(&attrib, path, 1+values, 1+nulls); tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls); } diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 3addb31e45b..29eec1e9045 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -11712,63 +11712,63 @@ { oid => '3353', descr => 'list files in the log directory', proname => 'pg_ls_logdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_logdir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logdir' }, { oid => '3354', descr => 'list of files in the WAL directory', proname => 'pg_ls_waldir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_waldir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_waldir' }, { oid => '5031', descr => 'list of files in the archive_status directory', proname => 'pg_ls_archive_statusdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_archive_statusdir' }, { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{tablespace,name,size,modification,isdir}', + proallargtypes => '{oid,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{tablespace,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '6270', descr => 'list of files in the pg_logical/snapshots directory', proname => 'pg_ls_logicalsnapdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logicalsnapdir' }, { oid => '6271', descr => 'list of files in the pg_logical/mappings directory', proname => 'pg_ls_logicalmapdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logicalmapdir' }, { oid => '6272', descr => 'list of files in the pg_replslot/slot_name directory', proname => 'pg_ls_replslotdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => 'text', proallargtypes => '{text,text,int8,timestamptz,bool}', - proargmodes => '{i,o,o,o,o}', - proargnames => '{slot_name,name,size,modification,isdir}', + proargtypes => 'text', proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{slot_name,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_replslotdir' }, { oid => '8450', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text bool bool', - proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}', - proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,modification,isdir}', + proallargtypes => '{text,bool,bool,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o,o,o,o}', + proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata' }, { oid => '8451', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{dirname,filename,size,modification,isdir}', + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{dirname,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata_1arg' }, # hash partitioning constraint function diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 4605fede3f2..fd67508e455 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -349,8 +349,8 @@ select count(*) > 0 as ok from (select pg_ls_waldir()) ss; -- Test not-run-to-completion cases. select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select count(*) > 0 as ok from (select * from pg_ls_waldir() limit 1) ss; @@ -500,38 +500,38 @@ select pg_ls_dir('does not exist'); -- fails with missingok=false ERROR: could not open directory "does not exist": No such file or directory -- Check that expected columns are present select * from pg_ls_archive_statusdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_logdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_logicalmapdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_logicalsnapdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_replslotdir('') limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_tmpdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_stat_file('.') limit 0; @@ -542,8 +542,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; @@ -559,8 +559,8 @@ select filename, isdir from pg_ls_dir_metadata('.', false, false) where filename -- Check that expected columns are present select * from pg_ls_dir_metadata('.') limit 0; - filename | size | modification | isdir -----------+------+--------------+------- + filename | size | access | modification | change | creation | isdir +----------+------+--------+--------------+--------+----------+------- (0 rows) -- diff --git a/src/test/regress/expected/tablespace.out b/src/test/regress/expected/tablespace.out index 53662fb0efd..ef64e7bfa8b 100644 --- a/src/test/regress/expected/tablespace.out +++ b/src/test/regress/expected/tablespace.out @@ -37,15 +37,15 @@ SELECT regexp_replace(pg_tablespace_location(oid), '(pg_tblspc)/(\d+)', '\1/NNN' -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet. -- The name='' condition is never true, so the function runs to completion but returns zero rows. SELECT * FROM pg_ls_logdir() WHERE name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.25.1 --Pk/CTwBz1VvfPIDp-- ^ permalink raw reply [nested|flat] 23+ messages in thread
* [PATCH v30 07/11] pg_ls_*dir to return all the metadata from pg_stat_file.. @ 2020-03-10 02:56 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 23+ messages in thread From: Justin Pryzby @ 2020-03-10 02:56 UTC (permalink / raw) ..but it doesn't seem worth factoring out the common bits, since stat_file doesn't return a name, so all the field numbers are off by one. NOTE, the atime is now shown where the mtime used to be. Need catversion bump --- doc/src/sgml/func.sgml | 34 +++++++--- src/backend/utils/adt/genfile.c | 71 +++++++++----------- src/include/catalog/pg_proc.dat | 28 ++++---- src/test/regress/expected/misc_functions.out | 12 ++-- src/test/regress/output/tablespace.source | 8 +-- 5 files changed, 81 insertions(+), 72 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 73fc6b9553..4c8faf7628 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25859,7 +25859,10 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <returnvalue>setof record</returnvalue> ( <parameter>filename</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> @@ -26865,13 +26868,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's log directory, - return the file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + return the file's name, along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -26890,13 +26896,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's write-ahead log (WAL) directory, list the - file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + file's name along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special files types are excluded. </para> <para> @@ -26915,14 +26924,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's WAL archive status directory - (<filename>pg_wal/archive_status</filename>), list the file's - name, size, last modification time (mtime), and a boolean indicating if - the file is a directory. + (<filename>pg_wal/archive_status</filename>), list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -26942,13 +26954,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the temporary directory within the given - <parameter>tablespace</parameter>, return the file's name, size, last - modification time (mtime) and a boolean indicating if the file is a directory. + <parameter>tablespace</parameter>, list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Directories are used for temporary files shared by parallel processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is examined. diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index f3437d306a..934e1d65e1 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -36,6 +36,8 @@ #include "utils/syscache.h" #include "utils/timestamp.h" +static void values_from_stat(struct stat *fst, const char *path, Datum *values, + bool *nulls); static Datum pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags); #define LS_DIR_ISDIR (1<<0) /* Show column: isdir */ @@ -399,6 +401,28 @@ pg_read_binary_file_all(PG_FUNCTION_ARGS) return pg_read_binary_file(fcinfo); } +/* + * Populate values and nulls from fst and path. + * Used for pg_stat_file() and pg_ls_dir_files() + * nulls is assumed to have been zerod. + */ +static void +values_from_stat(struct stat *fst, const char *path, Datum *values, bool *nulls) +{ + values[0] = Int64GetDatum((int64) fst->st_size); + values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_atime)); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_mtime)); + /* Unix has file status change time, while Win32 has creation time */ +#if !defined(WIN32) && !defined(__CYGWIN__) + values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); + nulls[4] = true; +#else + nulls[3] = true; + values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); +#endif + values[5] = BoolGetDatum(S_ISDIR(fst->st_mode)); +} + /* * stat a file */ @@ -409,7 +433,7 @@ pg_stat_file(PG_FUNCTION_ARGS) char *filename; struct stat fst; Datum values[6]; - bool isnull[6]; + bool nulls[6]; HeapTuple tuple; TupleDesc tupdesc; bool missing_ok = false; @@ -449,27 +473,9 @@ pg_stat_file(PG_FUNCTION_ARGS) "isdir", BOOLOID, -1, 0); BlessTupleDesc(tupdesc); - memset(isnull, false, sizeof(isnull)); - - values[0] = Int64GetDatum((int64) fst.st_size); - values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_atime)); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_mtime)); - /* Unix has file status change time, while Win32 has creation time */ -#if !defined(WIN32) && !defined(__CYGWIN__) - values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); - isnull[4] = true; -#else - isnull[3] = true; - values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); -#endif - values[5] = BoolGetDatum(S_ISDIR(fst.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(filename)) - values[5] = BoolGetDatum(false); -#endif - - tuple = heap_form_tuple(tupdesc, values, isnull); + memset(nulls, false, sizeof(nulls)); + values_from_stat(&fst, filename, values, nulls); + tuple = heap_form_tuple(tupdesc, values, nulls); pfree(filename); @@ -610,8 +616,8 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) while ((de = ReadDir(dirdesc, dir)) != NULL) { - Datum values[4]; - bool nulls[4]; + Datum values[7]; + bool nulls[7]; char path[MAXPGPATH * 2]; struct stat attrib; @@ -650,23 +656,10 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) continue; } + memset(nulls, false, sizeof(nulls)); values[0] = CStringGetTextDatum(de->d_name); if (flags & LS_DIR_METADATA) - { - values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); - if (flags & LS_DIR_ISDIR) - { - values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(path)) - values[3] = BoolGetDatum(false); -#endif - } - } - - memset(nulls, 0, sizeof(nulls)); + values_from_stat(&attrib, path, 1+values, 1+nulls); tuplestore_putvalues(tupstore, tupdesc, values, nulls); } diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index bc068415b4..282656ef82 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -11523,41 +11523,41 @@ { oid => '3353', descr => 'list files in the log directory', proname => 'pg_ls_logdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_logdir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logdir' }, { oid => '3354', descr => 'list of files in the WAL directory', proname => 'pg_ls_waldir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_waldir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_waldir' }, { oid => '5031', descr => 'list of files in the archive_status directory', proname => 'pg_ls_archive_statusdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_archive_statusdir' }, { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{tablespace,name,size,modification,isdir}', + proallargtypes => '{oid,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{tablespace,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '8450', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text bool bool', - proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}', - proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,modification,isdir}', + proallargtypes => '{text,bool,bool,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o,o,o,o}', + proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata' }, { oid => '8451', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{dirname,filename,size,modification,isdir}', + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{dirname,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata_1arg' }, # hash partitioning constraint function diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 96b662f316..ddd35b79f5 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -170,8 +170,8 @@ select count(*) > 0 as ok from (select pg_ls_waldir()) ss; -- Test not-run-to-completion cases. select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select count(*) > 0 as ok from (select * from pg_ls_waldir() limit 1) ss; @@ -241,8 +241,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; @@ -258,8 +258,8 @@ select filename, isdir from pg_ls_dir_metadata('.', false, false) where filename -- Check that expected columns are present select * from pg_ls_dir_metadata('.') limit 0; - filename | size | modification | isdir -----------+------+--------------+------- + filename | size | access | modification | change | creation | isdir +----------+------+--------+--------------+--------+----------+------- (0 rows) -- diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index 3571f14626..34f874d3f1 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,15 +17,15 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet. -- The name='' condition is never true, so the function runs to completion but returns zero rows. SELECT * FROM pg_ls_logdir() WHERE name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --ZwgA9U+XZDXt4+m+ Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v30-0008-pg_stat_file-and-pg_ls_dir_-to-use-lstat.patch" ^ permalink raw reply [nested|flat] 23+ messages in thread
* [PATCH v26 08/11] pg_ls_*dir to return all the metadata from pg_stat_file.. @ 2020-03-10 02:56 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 23+ messages in thread From: Justin Pryzby @ 2020-03-10 02:56 UTC (permalink / raw) ..but it doesn't seem worth factoring out the common bits, since stat_file doesn't return a name, so all the field numbers are off by one. NOTE, the atime is now shown where the mtime used to be. Need catversion bump --- doc/src/sgml/func.sgml | 34 +++++++--- src/backend/utils/adt/genfile.c | 71 +++++++++----------- src/include/catalog/pg_proc.dat | 28 ++++---- src/test/regress/expected/misc_functions.out | 12 ++-- src/test/regress/output/tablespace.source | 8 +-- 5 files changed, 81 insertions(+), 72 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index d154302172..017d761399 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25810,7 +25810,10 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <returnvalue>setof record</returnvalue> ( <parameter>filename</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> @@ -26420,13 +26423,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's log directory, - return the file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + return the file's name, along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -26445,13 +26451,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's write-ahead log (WAL) directory, list the - file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + file's name along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special files types are excluded. </para> <para> @@ -26470,14 +26479,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's WAL archive status directory - (<filename>pg_wal/archive_status</filename>), list the file's - name, size, last modification time (mtime), and a boolean indicating if - the file is a directory. + (<filename>pg_wal/archive_status</filename>), list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -26497,13 +26509,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the temporary directory within the given - <parameter>tablespace</parameter>, return the file's name, size, last - modification time (mtime) and a boolean indicating if the file is a directory. + <parameter>tablespace</parameter>, list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Directories are used for temporary files shared by parallel processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is examined. diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 07fce2a9bc..0bf148435f 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -36,6 +36,8 @@ #include "utils/syscache.h" #include "utils/timestamp.h" +static void values_from_stat(struct stat *fst, const char *path, Datum *values, + bool *nulls); static Datum pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags); #define LS_DIR_ISDIR (1<<0) /* Show column: isdir */ @@ -397,6 +399,28 @@ pg_read_binary_file_all(PG_FUNCTION_ARGS) return pg_read_binary_file(fcinfo); } +/* + * Populate values and nulls from fst and path. + * Used for pg_stat_file() and pg_ls_dir_files() + * nulls is assumed to have been zerod. + */ +static void +values_from_stat(struct stat *fst, const char *path, Datum *values, bool *nulls) +{ + values[0] = Int64GetDatum((int64) fst->st_size); + values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_atime)); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_mtime)); + /* Unix has file status change time, while Win32 has creation time */ +#if !defined(WIN32) && !defined(__CYGWIN__) + values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); + nulls[4] = true; +#else + nulls[3] = true; + values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); +#endif + values[5] = BoolGetDatum(S_ISDIR(fst->st_mode)); +} + /* * stat a file */ @@ -407,7 +431,7 @@ pg_stat_file(PG_FUNCTION_ARGS) char *filename; struct stat fst; Datum values[6]; - bool isnull[6]; + bool nulls[6]; HeapTuple tuple; TupleDesc tupdesc; bool missing_ok = false; @@ -447,27 +471,9 @@ pg_stat_file(PG_FUNCTION_ARGS) "isdir", BOOLOID, -1, 0); BlessTupleDesc(tupdesc); - memset(isnull, false, sizeof(isnull)); - - values[0] = Int64GetDatum((int64) fst.st_size); - values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_atime)); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_mtime)); - /* Unix has file status change time, while Win32 has creation time */ -#if !defined(WIN32) && !defined(__CYGWIN__) - values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); - isnull[4] = true; -#else - isnull[3] = true; - values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); -#endif - values[5] = BoolGetDatum(S_ISDIR(fst.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(filename)) - values[5] = BoolGetDatum(false); -#endif - - tuple = heap_form_tuple(tupdesc, values, isnull); + memset(nulls, false, sizeof(nulls)); + values_from_stat(&fst, filename, values, nulls); + tuple = heap_form_tuple(tupdesc, values, nulls); pfree(filename); @@ -608,8 +614,8 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) while ((de = ReadDir(dirdesc, dir)) != NULL) { - Datum values[4]; - bool nulls[4]; + Datum values[7]; + bool nulls[7]; char path[MAXPGPATH * 2]; struct stat attrib; @@ -648,23 +654,10 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) continue; } + memset(nulls, false, sizeof(nulls)); values[0] = CStringGetTextDatum(de->d_name); if (flags & LS_DIR_METADATA) - { - values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); - if (flags & LS_DIR_ISDIR) - { - values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(path)) - values[3] = BoolGetDatum(false); -#endif - } - } - - memset(nulls, 0, sizeof(nulls)); + values_from_stat(&attrib, path, 1+values, 1+nulls); tuplestore_putvalues(tupstore, tupdesc, values, nulls); } diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 908283bd19..30c21a7c6d 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -11242,41 +11242,41 @@ { oid => '3353', descr => 'list files in the log directory', proname => 'pg_ls_logdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_logdir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logdir' }, { oid => '3354', descr => 'list of files in the WAL directory', proname => 'pg_ls_waldir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_waldir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_waldir' }, { oid => '5031', descr => 'list of files in the archive_status directory', proname => 'pg_ls_archive_statusdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_archive_statusdir' }, { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{tablespace,name,size,modification,isdir}', + proallargtypes => '{oid,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{tablespace,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9979', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text bool bool', - proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}', - proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,modification,isdir}', + proallargtypes => '{text,bool,bool,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o,o,o,o}', + proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata' }, { oid => '9980', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{dirname,filename,size,modification,isdir}', + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{dirname,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata_1arg' }, # hash partitioning constraint function diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 800d12b4b6..1f4106cd9c 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -157,8 +157,8 @@ select count(*) > 0 as ok from (select pg_ls_waldir()) ss; -- Test not-run-to-completion cases. select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select count(*) > 0 as ok from (select * from pg_ls_waldir() limit 1) ss; @@ -228,8 +228,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; @@ -245,8 +245,8 @@ select filename, isdir from pg_ls_dir_metadata('.', false, false) where filename -- Check that expected columns are present select * from pg_ls_dir_metadata('.') limit 0; - filename | size | modification | isdir -----------+------+--------------+------- + filename | size | access | modification | change | creation | isdir +----------+------+--------+--------------+--------+----------+------- (0 rows) -- diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index ba9a3fe29a..1e1e02b589 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,15 +17,15 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet. -- The name='' condition is never true, so the function runs to completion but returns zero rows. SELECT * FROM pg_ls_logdir() WHERE name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --yKkOmjQZXRsvHRX8 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v26-0009-pg_ls_-pg_stat_file-to-show-file-type.patch" ^ permalink raw reply [nested|flat] 23+ messages in thread
* [PATCH v36 7/7] pg_ls_*dir to return all the metadata from pg_stat_file.. @ 2020-03-10 02:56 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 23+ messages in thread From: Justin Pryzby @ 2020-03-10 02:56 UTC (permalink / raw) ..but it doesn't seem worth factoring out the common bits, since stat_file doesn't return a name, so all the field numbers are off by one. NOTE, the atime is now shown where the mtime used to be. Need catversion bump --- doc/src/sgml/func.sgml | 34 ++++++--- src/backend/utils/adt/genfile.c | 72 +++++++++----------- src/include/catalog/pg_proc.dat | 42 ++++++------ src/test/regress/expected/misc_functions.out | 40 +++++------ src/test/regress/expected/tablespace.out | 8 +-- 5 files changed, 102 insertions(+), 94 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 8e724b4474f..fdc905c523d 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25951,7 +25951,10 @@ SELECT collation for ('foo' COLLATE "de_DE"); <returnvalue>setof record</returnvalue> ( <parameter>filename</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> @@ -28447,13 +28450,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's log directory, - return the file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + return the file's name, along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -28472,13 +28478,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's write-ahead log (WAL) directory, list the - file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + file's name along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special files types are excluded. </para> <para> @@ -28570,14 +28579,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's WAL archive status directory - (<filename>pg_wal/archive_status</filename>), list the file's - name, size, last modification time (mtime), and a boolean indicating if - the file is a directory. + (<filename>pg_wal/archive_status</filename>), list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -28597,13 +28609,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the temporary directory within the given - <parameter>tablespace</parameter>, return the file's name, size, last - modification time (mtime) and a boolean indicating if the file is a directory. + <parameter>tablespace</parameter>, list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Directories are used for temporary files shared by parallel processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is examined. diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index a01c7109f4c..a51cf29f095 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -37,6 +37,8 @@ #include "utils/syscache.h" #include "utils/timestamp.h" +static void values_from_stat(struct stat *fst, const char *path, Datum *values, + bool *nulls); static Datum pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags); #define LS_DIR_ISDIR (1<<0) /* Show column: isdir */ @@ -394,6 +396,28 @@ pg_read_binary_file_all(PG_FUNCTION_ARGS) return pg_read_binary_file(fcinfo); } +/* + * Populate values and nulls from fst and path. + * Used for pg_stat_file() and pg_ls_dir_files() + * nulls is assumed to have been zerod. + */ +static void +values_from_stat(struct stat *fst, const char *path, Datum *values, bool *nulls) +{ + values[0] = Int64GetDatum((int64) fst->st_size); + values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_atime)); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_mtime)); + /* Unix has file status change time, while Win32 has creation time */ +#if !defined(WIN32) && !defined(__CYGWIN__) + values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); + nulls[4] = true; +#else + nulls[3] = true; + values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); +#endif + values[5] = BoolGetDatum(S_ISDIR(fst->st_mode)); +} + /* * stat a file */ @@ -404,7 +428,7 @@ pg_stat_file(PG_FUNCTION_ARGS) char *filename; struct stat fst; Datum values[6]; - bool isnull[6]; + bool nulls[6]; HeapTuple tuple; TupleDesc tupdesc; bool missing_ok = false; @@ -444,27 +468,9 @@ pg_stat_file(PG_FUNCTION_ARGS) "isdir", BOOLOID, -1, 0); BlessTupleDesc(tupdesc); - memset(isnull, false, sizeof(isnull)); - - values[0] = Int64GetDatum((int64) fst.st_size); - values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_atime)); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_mtime)); - /* Unix has file status change time, while Win32 has creation time */ -#if !defined(WIN32) && !defined(__CYGWIN__) - values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); - isnull[4] = true; -#else - isnull[3] = true; - values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); -#endif - values[5] = BoolGetDatum(S_ISDIR(fst.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(filename)) - values[5] = BoolGetDatum(false); -#endif - - tuple = heap_form_tuple(tupdesc, values, isnull); + memset(nulls, false, sizeof(nulls)); + values_from_stat(&fst, filename, values, nulls); + tuple = heap_form_tuple(tupdesc, values, nulls); pfree(filename); @@ -571,8 +577,8 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) while ((de = ReadDir(dirdesc, dir)) != NULL) { - Datum values[4]; - bool nulls[4]; + Datum values[7]; + bool nulls[7]; char path[MAXPGPATH * 2]; struct stat attrib; @@ -611,24 +617,10 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) continue; } + memset(nulls, false, sizeof(nulls)); values[0] = CStringGetTextDatum(de->d_name); if (flags & LS_DIR_METADATA) - { - values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); - if (flags & LS_DIR_ISDIR) - { -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(path)) - values[3] = BoolGetDatum(false); - else -#endif - values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); - } - } - - memset(nulls, 0, sizeof(nulls)); + values_from_stat(&attrib, path, 1+values, 1+nulls); tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls); } diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index ec989867404..bbddb34ff6a 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -11769,63 +11769,63 @@ { oid => '3353', descr => 'list files in the log directory', proname => 'pg_ls_logdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_logdir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logdir' }, { oid => '3354', descr => 'list of files in the WAL directory', proname => 'pg_ls_waldir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_waldir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_waldir' }, { oid => '5031', descr => 'list of files in the archive_status directory', proname => 'pg_ls_archive_statusdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_archive_statusdir' }, { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{tablespace,name,size,modification,isdir}', + proallargtypes => '{oid,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{tablespace,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '6270', descr => 'list of files in the pg_logical/snapshots directory', proname => 'pg_ls_logicalsnapdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logicalsnapdir' }, { oid => '6271', descr => 'list of files in the pg_logical/mappings directory', proname => 'pg_ls_logicalmapdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logicalmapdir' }, { oid => '6272', descr => 'list of files in the pg_replslot/slot_name directory', proname => 'pg_ls_replslotdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => 'text', proallargtypes => '{text,text,int8,timestamptz,bool}', - proargmodes => '{i,o,o,o,o}', - proargnames => '{slot_name,name,size,modification,isdir}', + proargtypes => 'text', proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{slot_name,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_replslotdir' }, { oid => '8450', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text bool bool', - proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}', - proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,modification,isdir}', + proallargtypes => '{text,bool,bool,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o,o,o,o}', + proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata' }, { oid => '8451', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{dirname,filename,size,modification,isdir}', + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{dirname,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata_1arg' }, # hash partitioning constraint function diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 94b5eac47ea..afb1744bee7 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -349,8 +349,8 @@ select count(*) > 0 as ok from (select pg_ls_waldir()) ss; -- Test not-run-to-completion cases. select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select count(*) > 0 as ok from (select * from pg_ls_waldir() limit 1) ss; @@ -436,38 +436,38 @@ select pg_ls_dir('does not exist'); -- fails with missingok=false ERROR: could not open directory "does not exist": No such file or directory -- Check that expected columns are present select * from pg_ls_archive_statusdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_logdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_logicalmapdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_logicalsnapdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_replslotdir('') limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_tmpdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_stat_file('.') limit 0; @@ -478,8 +478,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; @@ -495,8 +495,8 @@ select filename, isdir from pg_ls_dir_metadata('.', false, false) where filename -- Check that expected columns are present select * from pg_ls_dir_metadata('.') limit 0; - filename | size | modification | isdir -----------+------+--------------+------- + filename | size | access | modification | change | creation | isdir +----------+------+--------+--------------+--------+----------+------- (0 rows) -- diff --git a/src/test/regress/expected/tablespace.out b/src/test/regress/expected/tablespace.out index 53662fb0efd..ef64e7bfa8b 100644 --- a/src/test/regress/expected/tablespace.out +++ b/src/test/regress/expected/tablespace.out @@ -37,15 +37,15 @@ SELECT regexp_replace(pg_tablespace_location(oid), '(pg_tblspc)/(\d+)', '\1/NNN' -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet. -- The name='' condition is never true, so the function runs to completion but returns zero rows. SELECT * FROM pg_ls_logdir() WHERE name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.1 --4ybNbZnZ8tziJ7D6-- ^ permalink raw reply [nested|flat] 23+ messages in thread
* [PATCH v20 09/10] pg_ls_*dir to return all the metadata from pg_stat_file.. @ 2020-03-10 02:56 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 23+ messages in thread From: Justin Pryzby @ 2020-03-10 02:56 UTC (permalink / raw) ..but it doesn't seem worth factoring out the common bits, since stat_file doesn't return a name, so all the field numbers are off by one. NOTE, the atime is now shown where the mtime used to be. Need catversion bump --- doc/src/sgml/func.sgml | 37 ++++++++--- src/backend/utils/adt/genfile.c | 65 +++++++++----------- src/include/catalog/pg_proc.dat | 34 +++++----- src/test/regress/expected/misc_functions.out | 16 ++--- src/test/regress/output/tablespace.source | 8 +-- 5 files changed, 86 insertions(+), 74 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index befd2ffca5..8c3e6f00dc 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25750,7 +25750,10 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> @@ -25784,7 +25787,10 @@ LATERAL pg_ls_dir_recurse(dir) AS a; <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> @@ -25804,13 +25810,16 @@ LATERAL pg_ls_dir_recurse(dir) AS a; <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's log directory, - return the file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + return the file's name, along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -25829,13 +25838,16 @@ LATERAL pg_ls_dir_recurse(dir) AS a; <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's write-ahead log (WAL) directory, list the - file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + file's name along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special files types are excluded. </para> <para> @@ -25854,14 +25866,17 @@ LATERAL pg_ls_dir_recurse(dir) AS a; <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's WAL archive status directory - (<filename>pg_wal/archive_status</filename>), list the file's - name, size, last modification time (mtime), and a boolean indicating if - the file is a directory. + (<filename>pg_wal/archive_status</filename>), list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -25881,13 +25896,17 @@ LATERAL pg_ls_dir_recurse(dir) AS a; <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the temporary directory within the given - <parameter>tablespace</parameter>, return the file's name, size, last - modification time (mtime) and a boolean indicating if the file is a directory. + <parameter>tablespace</parameter>, list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Directories are used for temporary files shared by parallel processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is examined. diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index e1fb160124..0095ad9621 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -36,6 +36,8 @@ #include "utils/syscache.h" #include "utils/timestamp.h" +static void tuple_from_stat(struct stat *fst, const char *path, Datum *values, + bool *isnull); static Datum pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags); #define LS_DIR_ISDIR (1<<0) /* Show column: isdir */ @@ -403,6 +405,28 @@ pg_read_binary_file_all(PG_FUNCTION_ARGS) return pg_read_binary_file(fcinfo); } +/* + * Populate values and isnull from fst and path. + * Used for pg_stat_file() and pg_stat_dir_files() + * isnull is assumed to have been zeroed. + */ +static void +tuple_from_stat(struct stat *fst, const char *path, Datum *values, bool *isnull) +{ + values[0] = Int64GetDatum((int64) fst->st_size); + values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_atime)); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_mtime)); + /* Unix has file status change time, while Win32 has creation time */ +#if !defined(WIN32) && !defined(__CYGWIN__) + values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); + isnull[4] = true; +#else + isnull[3] = true; + values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); +#endif + values[5] = BoolGetDatum(S_ISDIR(fst->st_mode)); +} + /* * stat a file */ @@ -454,25 +478,7 @@ pg_stat_file(PG_FUNCTION_ARGS) BlessTupleDesc(tupdesc); memset(isnull, false, sizeof(isnull)); - - values[0] = Int64GetDatum((int64) fst.st_size); - values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_atime)); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_mtime)); - /* Unix has file status change time, while Win32 has creation time */ -#if !defined(WIN32) && !defined(__CYGWIN__) - values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); - isnull[4] = true; -#else - isnull[3] = true; - values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); -#endif - values[5] = BoolGetDatum(S_ISDIR(fst.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(filename)) - values[5] = BoolGetDatum(false); -#endif - + tuple_from_stat(&fst, filename, values, isnull); tuple = heap_form_tuple(tupdesc, values, isnull); pfree(filename); @@ -614,8 +620,8 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) while ((de = ReadDir(dirdesc, dir)) != NULL) { - Datum values[4]; - bool nulls[4]; + Datum values[7]; + bool nulls[7]; char path[MAXPGPATH * 2]; struct stat attrib; @@ -654,23 +660,10 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) continue; } + memset(nulls, 0, sizeof(nulls)); values[0] = CStringGetTextDatum(de->d_name); if (flags & LS_DIR_METADATA) - { - values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); - if (flags & LS_DIR_ISDIR) - { - values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(path)) - values[3] = BoolGetDatum(false); -#endif - } - } - - memset(nulls, 0, sizeof(nulls)); + tuple_from_stat(&attrib, path, 1+values, 1+nulls); tuplestore_putvalues(tupstore, tupdesc, values, nulls); } diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index e60b6ca866..e1a9cf1d0a 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10895,48 +10895,48 @@ { oid => '3353', descr => 'list files in the log directory', proname => 'pg_ls_logdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_logdir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logdir' }, { oid => '3354', descr => 'list of files in the WAL directory', proname => 'pg_ls_waldir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_waldir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_waldir' }, { oid => '5031', descr => 'list of files in the archive_status directory', proname => 'pg_ls_archive_statusdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_archive_statusdir' }, { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{tablespace,name,size,modification,isdir}', + proallargtypes => '{oid,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{tablespace,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9979', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text bool bool', - proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}', - proargnames => '{dirname,missing_ok,include_dot_dirs,name,size,modification,isdir}', + proallargtypes => '{text,bool,bool,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o,o,o,o}', + proargnames => '{dirname,missing_ok,include_dot_dirs,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata' }, { oid => '9980', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{dirname,name,size,modification,isdir}', + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{dirname,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata_1arg' }, { oid => '9981', descr => 'list all files in a directory recursively', proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,text,int8,timestamptz,bool}', - proargnames => '{dirname,path,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o,o}', - prolang => 'sql', prosrc => "with recursive ls as (select dirname as path, * from pg_ls_dir_metadata(dirname, true, false) union all select parent.path||'/'||parent.name, a.name, a.size, a.modification, a.isdir from ls AS parent, lateral pg_ls_dir_metadata(parent.path||'/'||parent.name, false, false) as a where parent.isdir) select * from ls" }, + proallargtypes => '{text,text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargnames => '{dirname,path,name,size,access,modification,change,creation,isdir}', proargmodes => '{i,o,o,o,o,o,o,o,o}', + prolang => 'sql', prosrc => "with recursive ls as (select dirname as path, * from pg_ls_dir_metadata(dirname, true, false) union all select parent.path||'/'||parent.name, a.name, a.size, a.access, a.modification, a.change, a.creation, a.isdir from ls as parent, lateral pg_ls_dir_metadata(parent.path||'/'||parent.name, false, false) as a where parent.isdir) select * from ls" }, # hash partitioning constraint function { oid => '5028', descr => 'hash partition CHECK constraint', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 07c0a7f961..2880fca099 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -157,8 +157,8 @@ select count(*) > 0 as ok from (select pg_ls_waldir()) ss; -- Test not-run-to-completion cases. select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select count(*) > 0 as ok from (select * from pg_ls_waldir() limit 1) ss; @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; @@ -239,8 +239,8 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; -- -- Check that expected columns are present select * from pg_ls_dir_metadata('.') limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix @@ -252,8 +252,8 @@ SELECT path, name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND path='./pg_ -- Check that expected columns are present SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0; - path | name | size | modification | isdir -------+------+------+--------------+------- + path | name | size | access | modification | change | creation | isdir +------+------+------+--------+--------------+--------+----------+------- (0 rows) -- diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index ba9a3fe29a..1e1e02b589 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,15 +17,15 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet. -- The name='' condition is never true, so the function runs to completion but returns zero rows. SELECT * FROM pg_ls_logdir() WHERE name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --Z1Z8UV8BNhgCynIS Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v20-0010-pg_ls_-to-show-file-type-and-show-special-files.patch" ^ permalink raw reply [nested|flat] 23+ messages in thread
* [PATCH v32 07/11] pg_ls_*dir to return all the metadata from pg_stat_file.. @ 2020-03-10 02:56 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 23+ messages in thread From: Justin Pryzby @ 2020-03-10 02:56 UTC (permalink / raw) ..but it doesn't seem worth factoring out the common bits, since stat_file doesn't return a name, so all the field numbers are off by one. NOTE, the atime is now shown where the mtime used to be. Need catversion bump --- doc/src/sgml/func.sgml | 34 +++++++--- src/backend/utils/adt/genfile.c | 71 +++++++++----------- src/include/catalog/pg_proc.dat | 42 ++++++------ src/test/regress/expected/misc_functions.out | 12 ++-- src/test/regress/expected/tablespace.out | 8 +-- 5 files changed, 88 insertions(+), 79 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 99243fb1b1e..745faa1841f 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25956,7 +25956,10 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <returnvalue>setof record</returnvalue> ( <parameter>filename</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> @@ -27356,13 +27359,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's log directory, - return the file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + return the file's name, along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -27381,13 +27387,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's write-ahead log (WAL) directory, list the - file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + file's name along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special files types are excluded. </para> <para> @@ -27406,14 +27415,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's WAL archive status directory - (<filename>pg_wal/archive_status</filename>), list the file's - name, size, last modification time (mtime), and a boolean indicating if - the file is a directory. + (<filename>pg_wal/archive_status</filename>), list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -27433,13 +27445,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the temporary directory within the given - <parameter>tablespace</parameter>, return the file's name, size, last - modification time (mtime) and a boolean indicating if the file is a directory. + <parameter>tablespace</parameter>, list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Directories are used for temporary files shared by parallel processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is examined. diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index c381a170ad6..3c022cffd5a 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -37,6 +37,8 @@ #include "utils/syscache.h" #include "utils/timestamp.h" +static void values_from_stat(struct stat *fst, const char *path, Datum *values, + bool *nulls); static Datum pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags); #define LS_DIR_ISDIR (1<<0) /* Show column: isdir */ @@ -400,6 +402,28 @@ pg_read_binary_file_all(PG_FUNCTION_ARGS) return pg_read_binary_file(fcinfo); } +/* + * Populate values and nulls from fst and path. + * Used for pg_stat_file() and pg_ls_dir_files() + * nulls is assumed to have been zerod. + */ +static void +values_from_stat(struct stat *fst, const char *path, Datum *values, bool *nulls) +{ + values[0] = Int64GetDatum((int64) fst->st_size); + values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_atime)); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_mtime)); + /* Unix has file status change time, while Win32 has creation time */ +#if !defined(WIN32) && !defined(__CYGWIN__) + values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); + nulls[4] = true; +#else + nulls[3] = true; + values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); +#endif + values[5] = BoolGetDatum(S_ISDIR(fst->st_mode)); +} + /* * stat a file */ @@ -410,7 +434,7 @@ pg_stat_file(PG_FUNCTION_ARGS) char *filename; struct stat fst; Datum values[6]; - bool isnull[6]; + bool nulls[6]; HeapTuple tuple; TupleDesc tupdesc; bool missing_ok = false; @@ -450,27 +474,9 @@ pg_stat_file(PG_FUNCTION_ARGS) "isdir", BOOLOID, -1, 0); BlessTupleDesc(tupdesc); - memset(isnull, false, sizeof(isnull)); - - values[0] = Int64GetDatum((int64) fst.st_size); - values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_atime)); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_mtime)); - /* Unix has file status change time, while Win32 has creation time */ -#if !defined(WIN32) && !defined(__CYGWIN__) - values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); - isnull[4] = true; -#else - isnull[3] = true; - values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); -#endif - values[5] = BoolGetDatum(S_ISDIR(fst.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(filename)) - values[5] = BoolGetDatum(false); -#endif - - tuple = heap_form_tuple(tupdesc, values, isnull); + memset(nulls, false, sizeof(nulls)); + values_from_stat(&fst, filename, values, nulls); + tuple = heap_form_tuple(tupdesc, values, nulls); pfree(filename); @@ -611,8 +617,8 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) while ((de = ReadDir(dirdesc, dir)) != NULL) { - Datum values[4]; - bool nulls[4]; + Datum values[7]; + bool nulls[7]; char path[MAXPGPATH * 2]; struct stat attrib; @@ -651,23 +657,10 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) continue; } + memset(nulls, false, sizeof(nulls)); values[0] = CStringGetTextDatum(de->d_name); if (flags & LS_DIR_METADATA) - { - values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); - if (flags & LS_DIR_ISDIR) - { - values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(path)) - values[3] = BoolGetDatum(false); -#endif - } - } - - memset(nulls, 0, sizeof(nulls)); + values_from_stat(&attrib, path, 1+values, 1+nulls); tuplestore_putvalues(tupstore, tupdesc, values, nulls); } diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 1522ef041a2..f8ad1827cbd 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -11599,63 +11599,63 @@ { oid => '3353', descr => 'list files in the log directory', proname => 'pg_ls_logdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_logdir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logdir' }, { oid => '3354', descr => 'list of files in the WAL directory', proname => 'pg_ls_waldir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_waldir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_waldir' }, { oid => '5031', descr => 'list of files in the archive_status directory', proname => 'pg_ls_archive_statusdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_archive_statusdir' }, { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{tablespace,name,size,modification,isdir}', + proallargtypes => '{oid,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{tablespace,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9858', descr => 'list of files in the pg_logical/snapshots directory', proname => 'pg_ls_logicalsnapdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logicalsnapdir' }, { oid => '9859', descr => 'list of files in the pg_logical/mappings directory', proname => 'pg_ls_logicalmapdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logicalmapdir' }, { oid => '9860', descr => 'list of files in the pg_replslot/slot_name directory', proname => 'pg_ls_replslotdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => 'text', proallargtypes => '{text,text,int8,timestamptz,bool}', - proargmodes => '{i,o,o,o,o}', - proargnames => '{slot_name,name,size,modification,isdir}', + proargtypes => 'text', proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{slot_name,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_replslotdir' }, { oid => '8450', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text bool bool', - proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}', - proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,modification,isdir}', + proallargtypes => '{text,bool,bool,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o,o,o,o}', + proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata' }, { oid => '8451', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{dirname,filename,size,modification,isdir}', + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{dirname,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata_1arg' }, # hash partitioning constraint function diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 25492860bd8..dc4470fe3ee 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -199,8 +199,8 @@ select count(*) > 0 as ok from (select pg_ls_waldir()) ss; -- Test not-run-to-completion cases. select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select count(*) > 0 as ok from (select * from pg_ls_waldir() limit 1) ss; @@ -270,8 +270,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; @@ -287,8 +287,8 @@ select filename, isdir from pg_ls_dir_metadata('.', false, false) where filename -- Check that expected columns are present select * from pg_ls_dir_metadata('.') limit 0; - filename | size | modification | isdir -----------+------+--------------+------- + filename | size | access | modification | change | creation | isdir +----------+------+--------+--------------+--------+----------+------- (0 rows) -- diff --git a/src/test/regress/expected/tablespace.out b/src/test/regress/expected/tablespace.out index 549117256fc..db77360abf6 100644 --- a/src/test/regress/expected/tablespace.out +++ b/src/test/regress/expected/tablespace.out @@ -20,15 +20,15 @@ CREATE TABLESPACE regress_tblspace LOCATION :'testtablespace'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet. -- The name='' condition is never true, so the function runs to completion but returns zero rows. SELECT * FROM pg_ls_logdir() WHERE name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --Bne5rrxQd65beI7a Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v32-0008-pg_stat_file-and-pg_ls_dir_-to-use-lstat.patch" ^ permalink raw reply [nested|flat] 23+ messages in thread
* [PATCH v33 07/11] pg_ls_*dir to return all the metadata from pg_stat_file.. @ 2020-03-10 02:56 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 23+ messages in thread From: Justin Pryzby @ 2020-03-10 02:56 UTC (permalink / raw) ..but it doesn't seem worth factoring out the common bits, since stat_file doesn't return a name, so all the field numbers are off by one. NOTE, the atime is now shown where the mtime used to be. Need catversion bump --- doc/src/sgml/func.sgml | 34 ++++++--- src/backend/utils/adt/genfile.c | 72 +++++++++----------- src/include/catalog/pg_proc.dat | 42 ++++++------ src/test/regress/expected/misc_functions.out | 40 +++++------ src/test/regress/expected/tablespace.out | 8 +-- 5 files changed, 102 insertions(+), 94 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 53909e68ba3..a9106f13eae 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25969,7 +25969,10 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <returnvalue>setof record</returnvalue> ( <parameter>filename</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> @@ -27351,13 +27354,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's log directory, - return the file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + return the file's name, along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -27376,13 +27382,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's write-ahead log (WAL) directory, list the - file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + file's name along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special files types are excluded. </para> <para> @@ -27474,14 +27483,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's WAL archive status directory - (<filename>pg_wal/archive_status</filename>), list the file's - name, size, last modification time (mtime), and a boolean indicating if - the file is a directory. + (<filename>pg_wal/archive_status</filename>), list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -27501,13 +27513,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the temporary directory within the given - <parameter>tablespace</parameter>, return the file's name, size, last - modification time (mtime) and a boolean indicating if the file is a directory. + <parameter>tablespace</parameter>, list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Directories are used for temporary files shared by parallel processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is examined. diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index e49a15aa279..847d031af60 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -37,6 +37,8 @@ #include "utils/syscache.h" #include "utils/timestamp.h" +static void values_from_stat(struct stat *fst, const char *path, Datum *values, + bool *nulls); static Datum pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags); #define LS_DIR_ISDIR (1<<0) /* Show column: isdir */ @@ -400,6 +402,28 @@ pg_read_binary_file_all(PG_FUNCTION_ARGS) return pg_read_binary_file(fcinfo); } +/* + * Populate values and nulls from fst and path. + * Used for pg_stat_file() and pg_ls_dir_files() + * nulls is assumed to have been zerod. + */ +static void +values_from_stat(struct stat *fst, const char *path, Datum *values, bool *nulls) +{ + values[0] = Int64GetDatum((int64) fst->st_size); + values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_atime)); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_mtime)); + /* Unix has file status change time, while Win32 has creation time */ +#if !defined(WIN32) && !defined(__CYGWIN__) + values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); + nulls[4] = true; +#else + nulls[3] = true; + values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); +#endif + values[5] = BoolGetDatum(S_ISDIR(fst->st_mode)); +} + /* * stat a file */ @@ -410,7 +434,7 @@ pg_stat_file(PG_FUNCTION_ARGS) char *filename; struct stat fst; Datum values[6]; - bool isnull[6]; + bool nulls[6]; HeapTuple tuple; TupleDesc tupdesc; bool missing_ok = false; @@ -450,27 +474,9 @@ pg_stat_file(PG_FUNCTION_ARGS) "isdir", BOOLOID, -1, 0); BlessTupleDesc(tupdesc); - memset(isnull, false, sizeof(isnull)); - - values[0] = Int64GetDatum((int64) fst.st_size); - values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_atime)); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_mtime)); - /* Unix has file status change time, while Win32 has creation time */ -#if !defined(WIN32) && !defined(__CYGWIN__) - values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); - isnull[4] = true; -#else - isnull[3] = true; - values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); -#endif - values[5] = BoolGetDatum(S_ISDIR(fst.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(filename)) - values[5] = BoolGetDatum(false); -#endif - - tuple = heap_form_tuple(tupdesc, values, isnull); + memset(nulls, false, sizeof(nulls)); + values_from_stat(&fst, filename, values, nulls); + tuple = heap_form_tuple(tupdesc, values, nulls); pfree(filename); @@ -611,8 +617,8 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) while ((de = ReadDir(dirdesc, dir)) != NULL) { - Datum values[4]; - bool nulls[4]; + Datum values[7]; + bool nulls[7]; char path[MAXPGPATH * 2]; struct stat attrib; @@ -651,24 +657,10 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) continue; } + memset(nulls, false, sizeof(nulls)); values[0] = CStringGetTextDatum(de->d_name); if (flags & LS_DIR_METADATA) - { - values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); - if (flags & LS_DIR_ISDIR) - { -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(path)) - values[3] = BoolGetDatum(false); - else -#endif - values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); - } - } - - memset(nulls, 0, sizeof(nulls)); + values_from_stat(&attrib, path, 1+values, 1+nulls); tuplestore_putvalues(tupstore, tupdesc, values, nulls); } diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index c5bbae2408b..ee7793467d0 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -11615,63 +11615,63 @@ { oid => '3353', descr => 'list files in the log directory', proname => 'pg_ls_logdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_logdir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logdir' }, { oid => '3354', descr => 'list of files in the WAL directory', proname => 'pg_ls_waldir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_waldir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_waldir' }, { oid => '5031', descr => 'list of files in the archive_status directory', proname => 'pg_ls_archive_statusdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_archive_statusdir' }, { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{tablespace,name,size,modification,isdir}', + proallargtypes => '{oid,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{tablespace,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9858', descr => 'list of files in the pg_logical/snapshots directory', proname => 'pg_ls_logicalsnapdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logicalsnapdir' }, { oid => '9859', descr => 'list of files in the pg_logical/mappings directory', proname => 'pg_ls_logicalmapdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logicalmapdir' }, { oid => '9860', descr => 'list of files in the pg_replslot/slot_name directory', proname => 'pg_ls_replslotdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => 'text', proallargtypes => '{text,text,int8,timestamptz,bool}', - proargmodes => '{i,o,o,o,o}', - proargnames => '{slot_name,name,size,modification,isdir}', + proargtypes => 'text', proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{slot_name,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_replslotdir' }, { oid => '8450', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text bool bool', - proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}', - proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,modification,isdir}', + proallargtypes => '{text,bool,bool,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o,o,o,o}', + proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata' }, { oid => '8451', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{dirname,filename,size,modification,isdir}', + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{dirname,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata_1arg' }, # hash partitioning constraint function diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 91b6dba70b9..0db07209cc9 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -206,8 +206,8 @@ select count(*) > 0 as ok from (select pg_ls_waldir()) ss; -- Test not-run-to-completion cases. select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select count(*) > 0 as ok from (select * from pg_ls_waldir() limit 1) ss; @@ -270,38 +270,38 @@ select pg_ls_dir('does not exist'); -- fails with missingok=false ERROR: could not open directory "does not exist": No such file or directory -- Check that expected columns are present select * from pg_ls_archive_statusdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_logdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_logicalmapdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_logicalsnapdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_replslotdir('') limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_tmpdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_stat_file('.') limit 0; @@ -312,8 +312,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; @@ -329,8 +329,8 @@ select filename, isdir from pg_ls_dir_metadata('.', false, false) where filename -- Check that expected columns are present select * from pg_ls_dir_metadata('.') limit 0; - filename | size | modification | isdir -----------+------+--------------+------- + filename | size | access | modification | change | creation | isdir +----------+------+--------+--------------+--------+----------+------- (0 rows) -- diff --git a/src/test/regress/expected/tablespace.out b/src/test/regress/expected/tablespace.out index 50a3a13fbea..b01ca30c7ab 100644 --- a/src/test/regress/expected/tablespace.out +++ b/src/test/regress/expected/tablespace.out @@ -28,15 +28,15 @@ CREATE TABLESPACE regress_tblspace LOCATION ''; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet. -- The name='' condition is never true, so the function runs to completion but returns zero rows. SELECT * FROM pg_ls_logdir() WHERE name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.1 --9CzcV6dAFIr7O1Ie Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v33-0008-pg_stat_file-and-pg_ls_dir_-to-use-lstat.patch" ^ permalink raw reply [nested|flat] 23+ messages in thread
* [PATCH v28 07/11] pg_ls_*dir to return all the metadata from pg_stat_file.. @ 2020-03-10 02:56 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 23+ messages in thread From: Justin Pryzby @ 2020-03-10 02:56 UTC (permalink / raw) ..but it doesn't seem worth factoring out the common bits, since stat_file doesn't return a name, so all the field numbers are off by one. NOTE, the atime is now shown where the mtime used to be. Need catversion bump --- doc/src/sgml/func.sgml | 34 +++++++--- src/backend/utils/adt/genfile.c | 71 +++++++++----------- src/include/catalog/pg_proc.dat | 28 ++++---- src/test/regress/expected/misc_functions.out | 12 ++-- src/test/regress/output/tablespace.source | 8 +-- 5 files changed, 81 insertions(+), 72 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 0da1373c0f..727018826b 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25832,7 +25832,10 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <returnvalue>setof record</returnvalue> ( <parameter>filename</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> @@ -26879,13 +26882,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's log directory, - return the file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + return the file's name, along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -26904,13 +26910,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's write-ahead log (WAL) directory, list the - file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + file's name along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special files types are excluded. </para> <para> @@ -26929,14 +26938,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's WAL archive status directory - (<filename>pg_wal/archive_status</filename>), list the file's - name, size, last modification time (mtime), and a boolean indicating if - the file is a directory. + (<filename>pg_wal/archive_status</filename>), list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -26956,13 +26968,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the temporary directory within the given - <parameter>tablespace</parameter>, return the file's name, size, last - modification time (mtime) and a boolean indicating if the file is a directory. + <parameter>tablespace</parameter>, list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Directories are used for temporary files shared by parallel processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is examined. diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index e25010d40c..8f83ec08f6 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -36,6 +36,8 @@ #include "utils/syscache.h" #include "utils/timestamp.h" +static void values_from_stat(struct stat *fst, const char *path, Datum *values, + bool *nulls); static Datum pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags); #define LS_DIR_ISDIR (1<<0) /* Show column: isdir */ @@ -400,6 +402,28 @@ pg_read_binary_file_all(PG_FUNCTION_ARGS) return pg_read_binary_file(fcinfo); } +/* + * Populate values and nulls from fst and path. + * Used for pg_stat_file() and pg_ls_dir_files() + * nulls is assumed to have been zerod. + */ +static void +values_from_stat(struct stat *fst, const char *path, Datum *values, bool *nulls) +{ + values[0] = Int64GetDatum((int64) fst->st_size); + values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_atime)); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_mtime)); + /* Unix has file status change time, while Win32 has creation time */ +#if !defined(WIN32) && !defined(__CYGWIN__) + values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); + nulls[4] = true; +#else + nulls[3] = true; + values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); +#endif + values[5] = BoolGetDatum(S_ISDIR(fst->st_mode)); +} + /* * stat a file */ @@ -410,7 +434,7 @@ pg_stat_file(PG_FUNCTION_ARGS) char *filename; struct stat fst; Datum values[6]; - bool isnull[6]; + bool nulls[6]; HeapTuple tuple; TupleDesc tupdesc; bool missing_ok = false; @@ -450,27 +474,9 @@ pg_stat_file(PG_FUNCTION_ARGS) "isdir", BOOLOID, -1, 0); BlessTupleDesc(tupdesc); - memset(isnull, false, sizeof(isnull)); - - values[0] = Int64GetDatum((int64) fst.st_size); - values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_atime)); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_mtime)); - /* Unix has file status change time, while Win32 has creation time */ -#if !defined(WIN32) && !defined(__CYGWIN__) - values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); - isnull[4] = true; -#else - isnull[3] = true; - values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); -#endif - values[5] = BoolGetDatum(S_ISDIR(fst.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(filename)) - values[5] = BoolGetDatum(false); -#endif - - tuple = heap_form_tuple(tupdesc, values, isnull); + memset(nulls, false, sizeof(nulls)); + values_from_stat(&fst, filename, values, nulls); + tuple = heap_form_tuple(tupdesc, values, nulls); pfree(filename); @@ -611,8 +617,8 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) while ((de = ReadDir(dirdesc, dir)) != NULL) { - Datum values[4]; - bool nulls[4]; + Datum values[7]; + bool nulls[7]; char path[MAXPGPATH * 2]; struct stat attrib; @@ -651,23 +657,10 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) continue; } + memset(nulls, false, sizeof(nulls)); values[0] = CStringGetTextDatum(de->d_name); if (flags & LS_DIR_METADATA) - { - values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); - if (flags & LS_DIR_ISDIR) - { - values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(path)) - values[3] = BoolGetDatum(false); -#endif - } - } - - memset(nulls, 0, sizeof(nulls)); + values_from_stat(&attrib, path, 1+values, 1+nulls); tuplestore_putvalues(tupstore, tupdesc, values, nulls); } diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index b3c00d9077..08eea899d3 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -11510,41 +11510,41 @@ { oid => '3353', descr => 'list files in the log directory', proname => 'pg_ls_logdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_logdir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logdir' }, { oid => '3354', descr => 'list of files in the WAL directory', proname => 'pg_ls_waldir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_waldir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_waldir' }, { oid => '5031', descr => 'list of files in the archive_status directory', proname => 'pg_ls_archive_statusdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_archive_statusdir' }, { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{tablespace,name,size,modification,isdir}', + proallargtypes => '{oid,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{tablespace,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '8450', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text bool bool', - proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}', - proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,modification,isdir}', + proallargtypes => '{text,bool,bool,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o,o,o,o}', + proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata' }, { oid => '8451', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{dirname,filename,size,modification,isdir}', + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{dirname,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata_1arg' }, # hash partitioning constraint function diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 96b662f316..ddd35b79f5 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -170,8 +170,8 @@ select count(*) > 0 as ok from (select pg_ls_waldir()) ss; -- Test not-run-to-completion cases. select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select count(*) > 0 as ok from (select * from pg_ls_waldir() limit 1) ss; @@ -241,8 +241,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; @@ -258,8 +258,8 @@ select filename, isdir from pg_ls_dir_metadata('.', false, false) where filename -- Check that expected columns are present select * from pg_ls_dir_metadata('.') limit 0; - filename | size | modification | isdir -----------+------+--------------+------- + filename | size | access | modification | change | creation | isdir +----------+------+--------+--------------+--------+----------+------- (0 rows) -- diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index 3571f14626..34f874d3f1 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,15 +17,15 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet. -- The name='' condition is never true, so the function runs to completion but returns zero rows. SELECT * FROM pg_ls_logdir() WHERE name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --vk/v8fjDPiDepTtA Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v28-0008-pg_stat_file-and-pg_ls_dir_-to-use-lstat.patch" ^ permalink raw reply [nested|flat] 23+ messages in thread
* [PATCH v15 09/10] pg_ls_*dir to return all the metadata from pg_stat_file.. @ 2020-03-10 02:56 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 23+ messages in thread From: Justin Pryzby @ 2020-03-10 02:56 UTC (permalink / raw) ..but it doesn't seem worth factoring out the common bits, since stat_file doesn't return a name, so all the field numbers are off by one. NOTE, the atime is now shown where the mtime used to be. Need catversion bump --- doc/src/sgml/func.sgml | 30 ++++++++--------- src/backend/utils/adt/genfile.c | 29 ++++++++++------- src/include/catalog/pg_proc.dat | 34 ++++++++++---------- src/test/regress/expected/misc_functions.out | 16 ++++----- src/test/regress/output/tablespace.source | 8 ++--- 5 files changed, 59 insertions(+), 58 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 7c8b102610..88397ccddb 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -21397,8 +21397,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - For each file in the log directory, list the file's name, size, last - modification time, and a boolean indicating if it is a directory. + For each file in the log directory, list the file and its metadata. Access is granted to members of the <literal>pg_monitor</literal> role and may be granted to other non-superuser roles. </entry> @@ -21409,8 +21408,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - For each file in the WAL directory, list the file's name, size, last - modification time, and a boolean indicating if it is a directory. + For each file in the WAL directory, list the file and its metadata. Access is granted to members of the <literal>pg_monitor</literal> role and may be granted to other non-superuser roles. </entry> @@ -21421,9 +21419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - For each file in the WAL archive status directory, list the file's - name, size, last modification time, and a boolean indicating if it is a - directory. Access is granted to members of the + For each file in the WAL archive status directory, list the file and its metadata. + Access is granted to members of the <literal>pg_monitor</literal> role and may be granted to other non-superuser roles. </entry> @@ -21435,8 +21432,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <entry><type>setof record</type></entry> <entry> For the temporary directory within <parameter>tablespace</parameter>, - list each file's name, size, last modification time, and a boolean - indicating if it is a directory. If <parameter>tablespace</parameter> + list each file and its metadata. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. Access is granted to members of the <literal>pg_monitor</literal> role and may be granted to other non-superuser roles. @@ -21522,8 +21518,8 @@ lateral pg_ls_dir_recurse(dir) AS a; </indexterm> <para> <function>pg_ls_logdir</function> lists each file in the log directory, - along with file's size, last modification time, and a boolean - indicating if the file is a directory. By default, only superusers + along with the metadata columns returned by <function>pg_stat_file</function>. + By default, only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. Filenames beginning with a dot and special file types are not shown. @@ -21534,8 +21530,8 @@ lateral pg_ls_dir_recurse(dir) AS a; </indexterm> <para> <function>pg_ls_waldir</function> lists each file in the WAL directory, - along with the file's size, last modification time, and a boolean - indicating if the file is a directory. By default, only superusers + along with the metadata columns returned by <function>pg_stat_file</function>. + By default, only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. @@ -21547,8 +21543,8 @@ lateral pg_ls_dir_recurse(dir) AS a; </indexterm> <para> <function>pg_ls_archive_statusdir</function> lists each file in the WAL - archive status directory, along with the file's size, last modification - time, and a boolean indicating if the file is a directory. By default, only + archive status directory, along with the metadata columns returned by + <function>pg_stat_file</function>. By default, only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. @@ -21561,8 +21557,8 @@ lateral pg_ls_dir_recurse(dir) AS a; <para> <function>pg_ls_tmpdir</function> lists each file in the temporary file directory for the specified <parameter>tablespace</parameter>, along with - its size, last modified time (mtime) and a boolean indicating if the file is a - directory. Directories are used for temporary files shared by parallel + the metadata columns returned by <function>pg_stat_file</function>. + Directories are used for temporary files shared by parallel processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. By default only superusers and members of the <literal>pg_monitor</literal> diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index e8dc1d1be5..4d90f37a21 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -576,8 +576,8 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) while ((de = ReadDir(dirdesc, dir)) != NULL) { - Datum values[4]; - bool nulls[4]; + Datum values[7]; + bool nulls[7]; char path[MAXPGPATH * 2]; struct stat attrib; @@ -616,24 +616,29 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) continue; } + memset(nulls, 0, sizeof(nulls)); values[0] = CStringGetTextDatum(de->d_name); if (flags & LS_DIR_METADATA) { values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); - if (flags & LS_DIR_ISDIR) - { - values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_atime)); + values[3] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); + /* Unix has file status change time, while Win32 has creation time */ +#if !defined(WIN32) && !defined(__CYGWIN__) + values[4] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_ctime)); + nulls[5] = true; +#else + nulls[4] = true; + values[5] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_ctime)); +#endif + values[6] = BoolGetDatum(S_ISDIR(attrib.st_mode)); #ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(path)) - values[3] = BoolGetDatum(false); + /* Links should have isdir=false */ + if (pgwin32_is_junction(path)) + values[6] = BoolGetDatum(false); #endif - } } - memset(nulls, 0, sizeof(nulls)); - tuplestore_putvalues(tupstore, tupdesc, values, nulls); } diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 6c3b6f1f5b..d97ff8097f 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10777,48 +10777,48 @@ { oid => '3353', descr => 'list files in the log directory', proname => 'pg_ls_logdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_logdir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logdir' }, { oid => '3354', descr => 'list of files in the WAL directory', proname => 'pg_ls_waldir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_waldir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_waldir' }, { oid => '5031', descr => 'list of files in the archive_status directory', proname => 'pg_ls_archive_statusdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_archive_statusdir' }, { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{tablespace,name,size,modification,isdir}', + proallargtypes => '{oid,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{tablespace,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text bool bool', - proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}', - proargnames => '{dirname,missing_ok,include_dot_dirs,name,size,modification,isdir}', + proallargtypes => '{text,bool,bool,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o,o,o,o}', + proargnames => '{dirname,missing_ok,include_dot_dirs,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata' }, { oid => '5033', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{dirname,name,size,modification,isdir}', + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{dirname,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata_1arg' }, { oid => '5034', descr => 'list all files in a directory recursively', proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', - proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}', - prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" }, + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargnames => '{dirname,name,size,access,modification,change,creation,isdir}', proargmodes => '{i,o,o,o,o,o,o,o}', + prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.access, a.modification, a.change, a.creation, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" }, # hash partitioning constraint function { oid => '5028', descr => 'hash partition CHECK constraint', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 4188d684f0..cce84a60a9 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -157,8 +157,8 @@ select count(*) > 0 as ok from (select pg_ls_waldir()) ss; -- Test not-run-to-completion cases. select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select count(*) > 0 as ok from (select * from pg_ls_waldir() limit 1) ss; @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; @@ -239,8 +239,8 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; -- -- Check that expected columns are present select * from pg_ls_dir_metadata('.') limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix @@ -253,8 +253,8 @@ SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal'; -- Check that expected columns are present SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index ba9a3fe29a..1e1e02b589 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,15 +17,15 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet. -- The name='' condition is never true, so the function runs to completion but returns zero rows. SELECT * FROM pg_ls_logdir() WHERE name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --8w3uRX/HFJGApMzv Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v15-0010-pg_ls_-to-show-file-type-and-show-special-files.patch" ^ permalink raw reply [nested|flat] 23+ messages in thread
* [PATCH v35 7/7] pg_ls_*dir to return all the metadata from pg_stat_file.. @ 2020-03-10 02:56 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 23+ messages in thread From: Justin Pryzby @ 2020-03-10 02:56 UTC (permalink / raw) ..but it doesn't seem worth factoring out the common bits, since stat_file doesn't return a name, so all the field numbers are off by one. NOTE, the atime is now shown where the mtime used to be. Need catversion bump --- doc/src/sgml/func.sgml | 34 ++++++--- src/backend/utils/adt/genfile.c | 72 +++++++++----------- src/include/catalog/pg_proc.dat | 42 ++++++------ src/test/regress/expected/misc_functions.out | 40 +++++------ src/test/regress/expected/tablespace.out | 8 +-- 5 files changed, 102 insertions(+), 94 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index e7c63640d7e..a3004148a52 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25998,7 +25998,10 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <returnvalue>setof record</returnvalue> ( <parameter>filename</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> @@ -27417,13 +27420,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's log directory, - return the file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + return the file's name, along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -27442,13 +27448,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's write-ahead log (WAL) directory, list the - file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + file's name along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special files types are excluded. </para> <para> @@ -27540,14 +27549,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's WAL archive status directory - (<filename>pg_wal/archive_status</filename>), list the file's - name, size, last modification time (mtime), and a boolean indicating if - the file is a directory. + (<filename>pg_wal/archive_status</filename>), list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -27567,13 +27579,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the temporary directory within the given - <parameter>tablespace</parameter>, return the file's name, size, last - modification time (mtime) and a boolean indicating if the file is a directory. + <parameter>tablespace</parameter>, list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Directories are used for temporary files shared by parallel processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is examined. diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index ca5223be2ee..c289cf1ad85 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -37,6 +37,8 @@ #include "utils/syscache.h" #include "utils/timestamp.h" +static void values_from_stat(struct stat *fst, const char *path, Datum *values, + bool *nulls); static Datum pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags); #define LS_DIR_ISDIR (1<<0) /* Show column: isdir */ @@ -394,6 +396,28 @@ pg_read_binary_file_all(PG_FUNCTION_ARGS) return pg_read_binary_file(fcinfo); } +/* + * Populate values and nulls from fst and path. + * Used for pg_stat_file() and pg_ls_dir_files() + * nulls is assumed to have been zerod. + */ +static void +values_from_stat(struct stat *fst, const char *path, Datum *values, bool *nulls) +{ + values[0] = Int64GetDatum((int64) fst->st_size); + values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_atime)); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_mtime)); + /* Unix has file status change time, while Win32 has creation time */ +#if !defined(WIN32) && !defined(__CYGWIN__) + values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); + nulls[4] = true; +#else + nulls[3] = true; + values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); +#endif + values[5] = BoolGetDatum(S_ISDIR(fst->st_mode)); +} + /* * stat a file */ @@ -404,7 +428,7 @@ pg_stat_file(PG_FUNCTION_ARGS) char *filename; struct stat fst; Datum values[6]; - bool isnull[6]; + bool nulls[6]; HeapTuple tuple; TupleDesc tupdesc; bool missing_ok = false; @@ -444,27 +468,9 @@ pg_stat_file(PG_FUNCTION_ARGS) "isdir", BOOLOID, -1, 0); BlessTupleDesc(tupdesc); - memset(isnull, false, sizeof(isnull)); - - values[0] = Int64GetDatum((int64) fst.st_size); - values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_atime)); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_mtime)); - /* Unix has file status change time, while Win32 has creation time */ -#if !defined(WIN32) && !defined(__CYGWIN__) - values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); - isnull[4] = true; -#else - isnull[3] = true; - values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); -#endif - values[5] = BoolGetDatum(S_ISDIR(fst.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(filename)) - values[5] = BoolGetDatum(false); -#endif - - tuple = heap_form_tuple(tupdesc, values, isnull); + memset(nulls, false, sizeof(nulls)); + values_from_stat(&fst, filename, values, nulls); + tuple = heap_form_tuple(tupdesc, values, nulls); pfree(filename); @@ -571,8 +577,8 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) while ((de = ReadDir(dirdesc, dir)) != NULL) { - Datum values[4]; - bool nulls[4]; + Datum values[7]; + bool nulls[7]; char path[MAXPGPATH * 2]; struct stat attrib; @@ -611,24 +617,10 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) continue; } + memset(nulls, false, sizeof(nulls)); values[0] = CStringGetTextDatum(de->d_name); if (flags & LS_DIR_METADATA) - { - values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); - if (flags & LS_DIR_ISDIR) - { -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(path)) - values[3] = BoolGetDatum(false); - else -#endif - values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); - } - } - - memset(nulls, 0, sizeof(nulls)); + values_from_stat(&attrib, path, 1+values, 1+nulls); tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls); } diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index a68558cfed0..c7e96685e1b 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -11723,63 +11723,63 @@ { oid => '3353', descr => 'list files in the log directory', proname => 'pg_ls_logdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_logdir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logdir' }, { oid => '3354', descr => 'list of files in the WAL directory', proname => 'pg_ls_waldir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_waldir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_waldir' }, { oid => '5031', descr => 'list of files in the archive_status directory', proname => 'pg_ls_archive_statusdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_archive_statusdir' }, { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{tablespace,name,size,modification,isdir}', + proallargtypes => '{oid,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{tablespace,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9858', descr => 'list of files in the pg_logical/snapshots directory', proname => 'pg_ls_logicalsnapdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logicalsnapdir' }, { oid => '9859', descr => 'list of files in the pg_logical/mappings directory', proname => 'pg_ls_logicalmapdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logicalmapdir' }, { oid => '9860', descr => 'list of files in the pg_replslot/slot_name directory', proname => 'pg_ls_replslotdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => 'text', proallargtypes => '{text,text,int8,timestamptz,bool}', - proargmodes => '{i,o,o,o,o}', - proargnames => '{slot_name,name,size,modification,isdir}', + proargtypes => 'text', proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{slot_name,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_replslotdir' }, { oid => '8450', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text bool bool', - proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}', - proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,modification,isdir}', + proallargtypes => '{text,bool,bool,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o,o,o,o}', + proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata' }, { oid => '8451', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{dirname,filename,size,modification,isdir}', + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{dirname,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata_1arg' }, # hash partitioning constraint function diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 94b5eac47ea..afb1744bee7 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -349,8 +349,8 @@ select count(*) > 0 as ok from (select pg_ls_waldir()) ss; -- Test not-run-to-completion cases. select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select count(*) > 0 as ok from (select * from pg_ls_waldir() limit 1) ss; @@ -436,38 +436,38 @@ select pg_ls_dir('does not exist'); -- fails with missingok=false ERROR: could not open directory "does not exist": No such file or directory -- Check that expected columns are present select * from pg_ls_archive_statusdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_logdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_logicalmapdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_logicalsnapdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_replslotdir('') limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_tmpdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_stat_file('.') limit 0; @@ -478,8 +478,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; @@ -495,8 +495,8 @@ select filename, isdir from pg_ls_dir_metadata('.', false, false) where filename -- Check that expected columns are present select * from pg_ls_dir_metadata('.') limit 0; - filename | size | modification | isdir -----------+------+--------------+------- + filename | size | access | modification | change | creation | isdir +----------+------+--------+--------------+--------+----------+------- (0 rows) -- diff --git a/src/test/regress/expected/tablespace.out b/src/test/regress/expected/tablespace.out index 53662fb0efd..ef64e7bfa8b 100644 --- a/src/test/regress/expected/tablespace.out +++ b/src/test/regress/expected/tablespace.out @@ -37,15 +37,15 @@ SELECT regexp_replace(pg_tablespace_location(oid), '(pg_tblspc)/(\d+)', '\1/NNN' -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet. -- The name='' condition is never true, so the function runs to completion but returns zero rows. SELECT * FROM pg_ls_logdir() WHERE name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.1 --olLTNZSltDMg5Vbm-- ^ permalink raw reply [nested|flat] 23+ messages in thread
* [PATCH v31 07/11] pg_ls_*dir to return all the metadata from pg_stat_file.. @ 2020-03-10 02:56 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 23+ messages in thread From: Justin Pryzby @ 2020-03-10 02:56 UTC (permalink / raw) ..but it doesn't seem worth factoring out the common bits, since stat_file doesn't return a name, so all the field numbers are off by one. NOTE, the atime is now shown where the mtime used to be. Need catversion bump --- doc/src/sgml/func.sgml | 34 +++++++--- src/backend/utils/adt/genfile.c | 71 +++++++++----------- src/include/catalog/pg_proc.dat | 42 ++++++------ src/test/regress/expected/misc_functions.out | 12 ++-- src/test/regress/output/tablespace.source | 8 +-- 5 files changed, 88 insertions(+), 79 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 77547aa7cd..20e7f10e83 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25871,7 +25871,10 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <returnvalue>setof record</returnvalue> ( <parameter>filename</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> @@ -27271,13 +27274,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's log directory, - return the file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + return the file's name, along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -27296,13 +27302,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's write-ahead log (WAL) directory, list the - file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + file's name along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special files types are excluded. </para> <para> @@ -27321,14 +27330,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's WAL archive status directory - (<filename>pg_wal/archive_status</filename>), list the file's - name, size, last modification time (mtime), and a boolean indicating if - the file is a directory. + (<filename>pg_wal/archive_status</filename>), list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -27348,13 +27360,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the temporary directory within the given - <parameter>tablespace</parameter>, return the file's name, size, last - modification time (mtime) and a boolean indicating if the file is a directory. + <parameter>tablespace</parameter>, list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Directories are used for temporary files shared by parallel processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is examined. diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index c381a170ad..3c022cffd5 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -37,6 +37,8 @@ #include "utils/syscache.h" #include "utils/timestamp.h" +static void values_from_stat(struct stat *fst, const char *path, Datum *values, + bool *nulls); static Datum pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags); #define LS_DIR_ISDIR (1<<0) /* Show column: isdir */ @@ -400,6 +402,28 @@ pg_read_binary_file_all(PG_FUNCTION_ARGS) return pg_read_binary_file(fcinfo); } +/* + * Populate values and nulls from fst and path. + * Used for pg_stat_file() and pg_ls_dir_files() + * nulls is assumed to have been zerod. + */ +static void +values_from_stat(struct stat *fst, const char *path, Datum *values, bool *nulls) +{ + values[0] = Int64GetDatum((int64) fst->st_size); + values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_atime)); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_mtime)); + /* Unix has file status change time, while Win32 has creation time */ +#if !defined(WIN32) && !defined(__CYGWIN__) + values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); + nulls[4] = true; +#else + nulls[3] = true; + values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); +#endif + values[5] = BoolGetDatum(S_ISDIR(fst->st_mode)); +} + /* * stat a file */ @@ -410,7 +434,7 @@ pg_stat_file(PG_FUNCTION_ARGS) char *filename; struct stat fst; Datum values[6]; - bool isnull[6]; + bool nulls[6]; HeapTuple tuple; TupleDesc tupdesc; bool missing_ok = false; @@ -450,27 +474,9 @@ pg_stat_file(PG_FUNCTION_ARGS) "isdir", BOOLOID, -1, 0); BlessTupleDesc(tupdesc); - memset(isnull, false, sizeof(isnull)); - - values[0] = Int64GetDatum((int64) fst.st_size); - values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_atime)); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_mtime)); - /* Unix has file status change time, while Win32 has creation time */ -#if !defined(WIN32) && !defined(__CYGWIN__) - values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); - isnull[4] = true; -#else - isnull[3] = true; - values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); -#endif - values[5] = BoolGetDatum(S_ISDIR(fst.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(filename)) - values[5] = BoolGetDatum(false); -#endif - - tuple = heap_form_tuple(tupdesc, values, isnull); + memset(nulls, false, sizeof(nulls)); + values_from_stat(&fst, filename, values, nulls); + tuple = heap_form_tuple(tupdesc, values, nulls); pfree(filename); @@ -611,8 +617,8 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) while ((de = ReadDir(dirdesc, dir)) != NULL) { - Datum values[4]; - bool nulls[4]; + Datum values[7]; + bool nulls[7]; char path[MAXPGPATH * 2]; struct stat attrib; @@ -651,23 +657,10 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) continue; } + memset(nulls, false, sizeof(nulls)); values[0] = CStringGetTextDatum(de->d_name); if (flags & LS_DIR_METADATA) - { - values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); - if (flags & LS_DIR_ISDIR) - { - values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(path)) - values[3] = BoolGetDatum(false); -#endif - } - } - - memset(nulls, 0, sizeof(nulls)); + values_from_stat(&attrib, path, 1+values, 1+nulls); tuplestore_putvalues(tupstore, tupdesc, values, nulls); } diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index b442ca1130..b1944b32fb 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -11599,63 +11599,63 @@ { oid => '3353', descr => 'list files in the log directory', proname => 'pg_ls_logdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_logdir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logdir' }, { oid => '3354', descr => 'list of files in the WAL directory', proname => 'pg_ls_waldir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_waldir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_waldir' }, { oid => '5031', descr => 'list of files in the archive_status directory', proname => 'pg_ls_archive_statusdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_archive_statusdir' }, { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{tablespace,name,size,modification,isdir}', + proallargtypes => '{oid,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{tablespace,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9858', descr => 'list of files in the pg_logical/snapshots directory', proname => 'pg_ls_logicalsnapdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logicalsnapdir' }, { oid => '9859', descr => 'list of files in the pg_logical/mappings directory', proname => 'pg_ls_logicalmapdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logicalmapdir' }, { oid => '9860', descr => 'list of files in the pg_replslot/slot_name directory', proname => 'pg_ls_replslotdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => 'text', proallargtypes => '{text,text,int8,timestamptz,bool}', - proargmodes => '{i,o,o,o,o}', - proargnames => '{slot_name,name,size,modification,isdir}', + proargtypes => 'text', proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{slot_name,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_replslotdir' }, { oid => '8450', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text bool bool', - proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}', - proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,modification,isdir}', + proallargtypes => '{text,bool,bool,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o,o,o,o}', + proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata' }, { oid => '8451', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{dirname,filename,size,modification,isdir}', + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{dirname,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata_1arg' }, # hash partitioning constraint function diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 25492860bd..dc4470fe3e 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -199,8 +199,8 @@ select count(*) > 0 as ok from (select pg_ls_waldir()) ss; -- Test not-run-to-completion cases. select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select count(*) > 0 as ok from (select * from pg_ls_waldir() limit 1) ss; @@ -270,8 +270,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; @@ -287,8 +287,8 @@ select filename, isdir from pg_ls_dir_metadata('.', false, false) where filename -- Check that expected columns are present select * from pg_ls_dir_metadata('.') limit 0; - filename | size | modification | isdir -----------+------+--------------+------- + filename | size | access | modification | change | creation | isdir +----------+------+--------+--------------+--------+----------+------- (0 rows) -- diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index 3689bfac6d..1fc54528c7 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,15 +17,15 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet. -- The name='' condition is never true, so the function runs to completion but returns zero rows. SELECT * FROM pg_ls_logdir() WHERE name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --qZVVwWJgpX9Jzs7f Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v31-0008-pg_stat_file-and-pg_ls_dir_-to-use-lstat.patch" ^ permalink raw reply [nested|flat] 23+ messages in thread
* [PATCH v30 07/11] pg_ls_*dir to return all the metadata from pg_stat_file.. @ 2020-03-10 02:56 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 23+ messages in thread From: Justin Pryzby @ 2020-03-10 02:56 UTC (permalink / raw) ..but it doesn't seem worth factoring out the common bits, since stat_file doesn't return a name, so all the field numbers are off by one. NOTE, the atime is now shown where the mtime used to be. Need catversion bump --- doc/src/sgml/func.sgml | 34 +++++++--- src/backend/utils/adt/genfile.c | 71 +++++++++----------- src/include/catalog/pg_proc.dat | 28 ++++---- src/test/regress/expected/misc_functions.out | 12 ++-- src/test/regress/output/tablespace.source | 8 +-- 5 files changed, 81 insertions(+), 72 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 73fc6b9553..4c8faf7628 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25859,7 +25859,10 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <returnvalue>setof record</returnvalue> ( <parameter>filename</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> @@ -26865,13 +26868,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's log directory, - return the file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + return the file's name, along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -26890,13 +26896,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's write-ahead log (WAL) directory, list the - file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + file's name along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special files types are excluded. </para> <para> @@ -26915,14 +26924,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's WAL archive status directory - (<filename>pg_wal/archive_status</filename>), list the file's - name, size, last modification time (mtime), and a boolean indicating if - the file is a directory. + (<filename>pg_wal/archive_status</filename>), list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -26942,13 +26954,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the temporary directory within the given - <parameter>tablespace</parameter>, return the file's name, size, last - modification time (mtime) and a boolean indicating if the file is a directory. + <parameter>tablespace</parameter>, list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Directories are used for temporary files shared by parallel processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is examined. diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index f3437d306a..934e1d65e1 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -36,6 +36,8 @@ #include "utils/syscache.h" #include "utils/timestamp.h" +static void values_from_stat(struct stat *fst, const char *path, Datum *values, + bool *nulls); static Datum pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags); #define LS_DIR_ISDIR (1<<0) /* Show column: isdir */ @@ -399,6 +401,28 @@ pg_read_binary_file_all(PG_FUNCTION_ARGS) return pg_read_binary_file(fcinfo); } +/* + * Populate values and nulls from fst and path. + * Used for pg_stat_file() and pg_ls_dir_files() + * nulls is assumed to have been zerod. + */ +static void +values_from_stat(struct stat *fst, const char *path, Datum *values, bool *nulls) +{ + values[0] = Int64GetDatum((int64) fst->st_size); + values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_atime)); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_mtime)); + /* Unix has file status change time, while Win32 has creation time */ +#if !defined(WIN32) && !defined(__CYGWIN__) + values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); + nulls[4] = true; +#else + nulls[3] = true; + values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); +#endif + values[5] = BoolGetDatum(S_ISDIR(fst->st_mode)); +} + /* * stat a file */ @@ -409,7 +433,7 @@ pg_stat_file(PG_FUNCTION_ARGS) char *filename; struct stat fst; Datum values[6]; - bool isnull[6]; + bool nulls[6]; HeapTuple tuple; TupleDesc tupdesc; bool missing_ok = false; @@ -449,27 +473,9 @@ pg_stat_file(PG_FUNCTION_ARGS) "isdir", BOOLOID, -1, 0); BlessTupleDesc(tupdesc); - memset(isnull, false, sizeof(isnull)); - - values[0] = Int64GetDatum((int64) fst.st_size); - values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_atime)); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_mtime)); - /* Unix has file status change time, while Win32 has creation time */ -#if !defined(WIN32) && !defined(__CYGWIN__) - values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); - isnull[4] = true; -#else - isnull[3] = true; - values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); -#endif - values[5] = BoolGetDatum(S_ISDIR(fst.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(filename)) - values[5] = BoolGetDatum(false); -#endif - - tuple = heap_form_tuple(tupdesc, values, isnull); + memset(nulls, false, sizeof(nulls)); + values_from_stat(&fst, filename, values, nulls); + tuple = heap_form_tuple(tupdesc, values, nulls); pfree(filename); @@ -610,8 +616,8 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) while ((de = ReadDir(dirdesc, dir)) != NULL) { - Datum values[4]; - bool nulls[4]; + Datum values[7]; + bool nulls[7]; char path[MAXPGPATH * 2]; struct stat attrib; @@ -650,23 +656,10 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) continue; } + memset(nulls, false, sizeof(nulls)); values[0] = CStringGetTextDatum(de->d_name); if (flags & LS_DIR_METADATA) - { - values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); - if (flags & LS_DIR_ISDIR) - { - values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(path)) - values[3] = BoolGetDatum(false); -#endif - } - } - - memset(nulls, 0, sizeof(nulls)); + values_from_stat(&attrib, path, 1+values, 1+nulls); tuplestore_putvalues(tupstore, tupdesc, values, nulls); } diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index bc068415b4..282656ef82 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -11523,41 +11523,41 @@ { oid => '3353', descr => 'list files in the log directory', proname => 'pg_ls_logdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_logdir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logdir' }, { oid => '3354', descr => 'list of files in the WAL directory', proname => 'pg_ls_waldir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_waldir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_waldir' }, { oid => '5031', descr => 'list of files in the archive_status directory', proname => 'pg_ls_archive_statusdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_archive_statusdir' }, { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{tablespace,name,size,modification,isdir}', + proallargtypes => '{oid,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{tablespace,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '8450', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text bool bool', - proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}', - proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,modification,isdir}', + proallargtypes => '{text,bool,bool,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o,o,o,o}', + proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata' }, { oid => '8451', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{dirname,filename,size,modification,isdir}', + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{dirname,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata_1arg' }, # hash partitioning constraint function diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 96b662f316..ddd35b79f5 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -170,8 +170,8 @@ select count(*) > 0 as ok from (select pg_ls_waldir()) ss; -- Test not-run-to-completion cases. select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select count(*) > 0 as ok from (select * from pg_ls_waldir() limit 1) ss; @@ -241,8 +241,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; @@ -258,8 +258,8 @@ select filename, isdir from pg_ls_dir_metadata('.', false, false) where filename -- Check that expected columns are present select * from pg_ls_dir_metadata('.') limit 0; - filename | size | modification | isdir -----------+------+--------------+------- + filename | size | access | modification | change | creation | isdir +----------+------+--------+--------------+--------+----------+------- (0 rows) -- diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index 3571f14626..34f874d3f1 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,15 +17,15 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet. -- The name='' condition is never true, so the function runs to completion but returns zero rows. SELECT * FROM pg_ls_logdir() WHERE name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --ZwgA9U+XZDXt4+m+ Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v30-0008-pg_stat_file-and-pg_ls_dir_-to-use-lstat.patch" ^ permalink raw reply [nested|flat] 23+ messages in thread
* [PATCH v34 07/15] pg_ls_*dir to return all the metadata from pg_stat_file.. @ 2020-03-10 02:56 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 23+ messages in thread From: Justin Pryzby @ 2020-03-10 02:56 UTC (permalink / raw) ..but it doesn't seem worth factoring out the common bits, since stat_file doesn't return a name, so all the field numbers are off by one. NOTE, the atime is now shown where the mtime used to be. Need catversion bump --- doc/src/sgml/func.sgml | 34 ++++++--- src/backend/utils/adt/genfile.c | 72 +++++++++----------- src/include/catalog/pg_proc.dat | 42 ++++++------ src/test/regress/expected/misc_functions.out | 40 +++++------ src/test/regress/expected/tablespace.out | 8 +-- 5 files changed, 102 insertions(+), 94 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 88529184dd4..05d507f579f 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25987,7 +25987,10 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <returnvalue>setof record</returnvalue> ( <parameter>filename</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> @@ -27406,13 +27409,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's log directory, - return the file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + return the file's name, along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -27431,13 +27437,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's write-ahead log (WAL) directory, list the - file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + file's name along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special files types are excluded. </para> <para> @@ -27529,14 +27538,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's WAL archive status directory - (<filename>pg_wal/archive_status</filename>), list the file's - name, size, last modification time (mtime), and a boolean indicating if - the file is a directory. + (<filename>pg_wal/archive_status</filename>), list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -27556,13 +27568,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the temporary directory within the given - <parameter>tablespace</parameter>, return the file's name, size, last - modification time (mtime) and a boolean indicating if the file is a directory. + <parameter>tablespace</parameter>, list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Directories are used for temporary files shared by parallel processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is examined. diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 80516833e33..698468f9860 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -37,6 +37,8 @@ #include "utils/syscache.h" #include "utils/timestamp.h" +static void values_from_stat(struct stat *fst, const char *path, Datum *values, + bool *nulls); static Datum pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags); #define LS_DIR_ISDIR (1<<0) /* Show column: isdir */ @@ -394,6 +396,28 @@ pg_read_binary_file_all(PG_FUNCTION_ARGS) return pg_read_binary_file(fcinfo); } +/* + * Populate values and nulls from fst and path. + * Used for pg_stat_file() and pg_ls_dir_files() + * nulls is assumed to have been zerod. + */ +static void +values_from_stat(struct stat *fst, const char *path, Datum *values, bool *nulls) +{ + values[0] = Int64GetDatum((int64) fst->st_size); + values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_atime)); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_mtime)); + /* Unix has file status change time, while Win32 has creation time */ +#if !defined(WIN32) && !defined(__CYGWIN__) + values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); + nulls[4] = true; +#else + nulls[3] = true; + values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); +#endif + values[5] = BoolGetDatum(S_ISDIR(fst->st_mode)); +} + /* * stat a file */ @@ -404,7 +428,7 @@ pg_stat_file(PG_FUNCTION_ARGS) char *filename; struct stat fst; Datum values[6]; - bool isnull[6]; + bool nulls[6]; HeapTuple tuple; TupleDesc tupdesc; bool missing_ok = false; @@ -444,27 +468,9 @@ pg_stat_file(PG_FUNCTION_ARGS) "isdir", BOOLOID, -1, 0); BlessTupleDesc(tupdesc); - memset(isnull, false, sizeof(isnull)); - - values[0] = Int64GetDatum((int64) fst.st_size); - values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_atime)); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_mtime)); - /* Unix has file status change time, while Win32 has creation time */ -#if !defined(WIN32) && !defined(__CYGWIN__) - values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); - isnull[4] = true; -#else - isnull[3] = true; - values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); -#endif - values[5] = BoolGetDatum(S_ISDIR(fst.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(filename)) - values[5] = BoolGetDatum(false); -#endif - - tuple = heap_form_tuple(tupdesc, values, isnull); + memset(nulls, false, sizeof(nulls)); + values_from_stat(&fst, filename, values, nulls); + tuple = heap_form_tuple(tupdesc, values, nulls); pfree(filename); @@ -569,8 +575,8 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) while ((de = ReadDir(dirdesc, dir)) != NULL) { - Datum values[4]; - bool nulls[4]; + Datum values[7]; + bool nulls[7]; char path[MAXPGPATH * 2]; struct stat attrib; @@ -609,24 +615,10 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) continue; } + memset(nulls, false, sizeof(nulls)); values[0] = CStringGetTextDatum(de->d_name); if (flags & LS_DIR_METADATA) - { - values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); - if (flags & LS_DIR_ISDIR) - { -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(path)) - values[3] = BoolGetDatum(false); - else -#endif - values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); - } - } - - memset(nulls, 0, sizeof(nulls)); + values_from_stat(&attrib, path, 1+values, 1+nulls); tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls); } diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 6bacabc61ba..03839b35ea4 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -11631,63 +11631,63 @@ { oid => '3353', descr => 'list files in the log directory', proname => 'pg_ls_logdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_logdir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logdir' }, { oid => '3354', descr => 'list of files in the WAL directory', proname => 'pg_ls_waldir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_waldir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_waldir' }, { oid => '5031', descr => 'list of files in the archive_status directory', proname => 'pg_ls_archive_statusdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_archive_statusdir' }, { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{tablespace,name,size,modification,isdir}', + proallargtypes => '{oid,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{tablespace,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9858', descr => 'list of files in the pg_logical/snapshots directory', proname => 'pg_ls_logicalsnapdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logicalsnapdir' }, { oid => '9859', descr => 'list of files in the pg_logical/mappings directory', proname => 'pg_ls_logicalmapdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logicalmapdir' }, { oid => '9860', descr => 'list of files in the pg_replslot/slot_name directory', proname => 'pg_ls_replslotdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => 'text', proallargtypes => '{text,text,int8,timestamptz,bool}', - proargmodes => '{i,o,o,o,o}', - proargnames => '{slot_name,name,size,modification,isdir}', + proargtypes => 'text', proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{slot_name,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_replslotdir' }, { oid => '8450', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text bool bool', - proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}', - proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,modification,isdir}', + proallargtypes => '{text,bool,bool,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o,o,o,o}', + proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata' }, { oid => '8451', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{dirname,filename,size,modification,isdir}', + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{dirname,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata_1arg' }, # hash partitioning constraint function diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 0599a745270..cb784ece778 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -349,8 +349,8 @@ select count(*) > 0 as ok from (select pg_ls_waldir()) ss; -- Test not-run-to-completion cases. select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select count(*) > 0 as ok from (select * from pg_ls_waldir() limit 1) ss; @@ -413,38 +413,38 @@ select pg_ls_dir('does not exist'); -- fails with missingok=false ERROR: could not open directory "does not exist": No such file or directory -- Check that expected columns are present select * from pg_ls_archive_statusdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_logdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_logicalmapdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_logicalsnapdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_replslotdir('') limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_tmpdir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select * from pg_stat_file('.') limit 0; @@ -455,8 +455,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; @@ -472,8 +472,8 @@ select filename, isdir from pg_ls_dir_metadata('.', false, false) where filename -- Check that expected columns are present select * from pg_ls_dir_metadata('.') limit 0; - filename | size | modification | isdir -----------+------+--------------+------- + filename | size | access | modification | change | creation | isdir +----------+------+--------+--------------+--------+----------+------- (0 rows) -- diff --git a/src/test/regress/expected/tablespace.out b/src/test/regress/expected/tablespace.out index 50a3a13fbea..b01ca30c7ab 100644 --- a/src/test/regress/expected/tablespace.out +++ b/src/test/regress/expected/tablespace.out @@ -28,15 +28,15 @@ CREATE TABLESPACE regress_tblspace LOCATION ''; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet. -- The name='' condition is never true, so the function runs to completion but returns zero rows. SELECT * FROM pg_ls_logdir() WHERE name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.1 --smOfPzt+Qjm5bNGJ-- ^ permalink raw reply [nested|flat] 23+ messages in thread
* [PATCH v31 07/11] pg_ls_*dir to return all the metadata from pg_stat_file.. @ 2020-03-10 02:56 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 23+ messages in thread From: Justin Pryzby @ 2020-03-10 02:56 UTC (permalink / raw) ..but it doesn't seem worth factoring out the common bits, since stat_file doesn't return a name, so all the field numbers are off by one. NOTE, the atime is now shown where the mtime used to be. Need catversion bump --- doc/src/sgml/func.sgml | 34 +++++++--- src/backend/utils/adt/genfile.c | 71 +++++++++----------- src/include/catalog/pg_proc.dat | 42 ++++++------ src/test/regress/expected/misc_functions.out | 12 ++-- src/test/regress/output/tablespace.source | 8 +-- 5 files changed, 88 insertions(+), 79 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 77547aa7cd..20e7f10e83 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25871,7 +25871,10 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <returnvalue>setof record</returnvalue> ( <parameter>filename</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> @@ -27271,13 +27274,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's log directory, - return the file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + return the file's name, along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -27296,13 +27302,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's write-ahead log (WAL) directory, list the - file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + file's name along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special files types are excluded. </para> <para> @@ -27321,14 +27330,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's WAL archive status directory - (<filename>pg_wal/archive_status</filename>), list the file's - name, size, last modification time (mtime), and a boolean indicating if - the file is a directory. + (<filename>pg_wal/archive_status</filename>), list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -27348,13 +27360,17 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the temporary directory within the given - <parameter>tablespace</parameter>, return the file's name, size, last - modification time (mtime) and a boolean indicating if the file is a directory. + <parameter>tablespace</parameter>, list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Directories are used for temporary files shared by parallel processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is examined. diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index c381a170ad..3c022cffd5 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -37,6 +37,8 @@ #include "utils/syscache.h" #include "utils/timestamp.h" +static void values_from_stat(struct stat *fst, const char *path, Datum *values, + bool *nulls); static Datum pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags); #define LS_DIR_ISDIR (1<<0) /* Show column: isdir */ @@ -400,6 +402,28 @@ pg_read_binary_file_all(PG_FUNCTION_ARGS) return pg_read_binary_file(fcinfo); } +/* + * Populate values and nulls from fst and path. + * Used for pg_stat_file() and pg_ls_dir_files() + * nulls is assumed to have been zerod. + */ +static void +values_from_stat(struct stat *fst, const char *path, Datum *values, bool *nulls) +{ + values[0] = Int64GetDatum((int64) fst->st_size); + values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_atime)); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_mtime)); + /* Unix has file status change time, while Win32 has creation time */ +#if !defined(WIN32) && !defined(__CYGWIN__) + values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); + nulls[4] = true; +#else + nulls[3] = true; + values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); +#endif + values[5] = BoolGetDatum(S_ISDIR(fst->st_mode)); +} + /* * stat a file */ @@ -410,7 +434,7 @@ pg_stat_file(PG_FUNCTION_ARGS) char *filename; struct stat fst; Datum values[6]; - bool isnull[6]; + bool nulls[6]; HeapTuple tuple; TupleDesc tupdesc; bool missing_ok = false; @@ -450,27 +474,9 @@ pg_stat_file(PG_FUNCTION_ARGS) "isdir", BOOLOID, -1, 0); BlessTupleDesc(tupdesc); - memset(isnull, false, sizeof(isnull)); - - values[0] = Int64GetDatum((int64) fst.st_size); - values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_atime)); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_mtime)); - /* Unix has file status change time, while Win32 has creation time */ -#if !defined(WIN32) && !defined(__CYGWIN__) - values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); - isnull[4] = true; -#else - isnull[3] = true; - values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); -#endif - values[5] = BoolGetDatum(S_ISDIR(fst.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(filename)) - values[5] = BoolGetDatum(false); -#endif - - tuple = heap_form_tuple(tupdesc, values, isnull); + memset(nulls, false, sizeof(nulls)); + values_from_stat(&fst, filename, values, nulls); + tuple = heap_form_tuple(tupdesc, values, nulls); pfree(filename); @@ -611,8 +617,8 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) while ((de = ReadDir(dirdesc, dir)) != NULL) { - Datum values[4]; - bool nulls[4]; + Datum values[7]; + bool nulls[7]; char path[MAXPGPATH * 2]; struct stat attrib; @@ -651,23 +657,10 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) continue; } + memset(nulls, false, sizeof(nulls)); values[0] = CStringGetTextDatum(de->d_name); if (flags & LS_DIR_METADATA) - { - values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); - if (flags & LS_DIR_ISDIR) - { - values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(path)) - values[3] = BoolGetDatum(false); -#endif - } - } - - memset(nulls, 0, sizeof(nulls)); + values_from_stat(&attrib, path, 1+values, 1+nulls); tuplestore_putvalues(tupstore, tupdesc, values, nulls); } diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index b442ca1130..b1944b32fb 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -11599,63 +11599,63 @@ { oid => '3353', descr => 'list files in the log directory', proname => 'pg_ls_logdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_logdir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logdir' }, { oid => '3354', descr => 'list of files in the WAL directory', proname => 'pg_ls_waldir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_waldir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_waldir' }, { oid => '5031', descr => 'list of files in the archive_status directory', proname => 'pg_ls_archive_statusdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_archive_statusdir' }, { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{tablespace,name,size,modification,isdir}', + proallargtypes => '{oid,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{tablespace,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9858', descr => 'list of files in the pg_logical/snapshots directory', proname => 'pg_ls_logicalsnapdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logicalsnapdir' }, { oid => '9859', descr => 'list of files in the pg_logical/mappings directory', proname => 'pg_ls_logicalmapdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logicalmapdir' }, { oid => '9860', descr => 'list of files in the pg_replslot/slot_name directory', proname => 'pg_ls_replslotdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => 'text', proallargtypes => '{text,text,int8,timestamptz,bool}', - proargmodes => '{i,o,o,o,o}', - proargnames => '{slot_name,name,size,modification,isdir}', + proargtypes => 'text', proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{slot_name,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_replslotdir' }, { oid => '8450', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text bool bool', - proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}', - proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,modification,isdir}', + proallargtypes => '{text,bool,bool,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o,o,o,o}', + proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata' }, { oid => '8451', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{dirname,filename,size,modification,isdir}', + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{dirname,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata_1arg' }, # hash partitioning constraint function diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 25492860bd..dc4470fe3e 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -199,8 +199,8 @@ select count(*) > 0 as ok from (select pg_ls_waldir()) ss; -- Test not-run-to-completion cases. select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select count(*) > 0 as ok from (select * from pg_ls_waldir() limit 1) ss; @@ -270,8 +270,8 @@ select * from pg_stat_file('.') limit 0; -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; @@ -287,8 +287,8 @@ select filename, isdir from pg_ls_dir_metadata('.', false, false) where filename -- Check that expected columns are present select * from pg_ls_dir_metadata('.') limit 0; - filename | size | modification | isdir -----------+------+--------------+------- + filename | size | access | modification | change | creation | isdir +----------+------+--------+--------------+--------+----------+------- (0 rows) -- diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index 3689bfac6d..1fc54528c7 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,15 +17,15 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet. -- The name='' condition is never true, so the function runs to completion but returns zero rows. SELECT * FROM pg_ls_logdir() WHERE name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --qZVVwWJgpX9Jzs7f Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v31-0008-pg_stat_file-and-pg_ls_dir_-to-use-lstat.patch" ^ permalink raw reply [nested|flat] 23+ messages in thread
* [PATCH v21 09/10] pg_ls_*dir to return all the metadata from pg_stat_file.. @ 2020-03-10 02:56 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 23+ messages in thread From: Justin Pryzby @ 2020-03-10 02:56 UTC (permalink / raw) ..but it doesn't seem worth factoring out the common bits, since stat_file doesn't return a name, so all the field numbers are off by one. NOTE, the atime is now shown where the mtime used to be. Need catversion bump --- doc/src/sgml/func.sgml | 37 ++++++++--- src/backend/utils/adt/genfile.c | 65 +++++++++----------- src/include/catalog/pg_proc.dat | 34 +++++----- src/test/regress/expected/misc_functions.out | 16 ++--- src/test/regress/output/tablespace.source | 8 +-- 5 files changed, 86 insertions(+), 74 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index befd2ffca5..8c3e6f00dc 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25750,7 +25750,10 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> @@ -25784,7 +25787,10 @@ LATERAL pg_ls_dir_recurse(dir) AS a; <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> @@ -25804,13 +25810,16 @@ LATERAL pg_ls_dir_recurse(dir) AS a; <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's log directory, - return the file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + return the file's name, along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -25829,13 +25838,16 @@ LATERAL pg_ls_dir_recurse(dir) AS a; <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's write-ahead log (WAL) directory, list the - file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + file's name along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special files types are excluded. </para> <para> @@ -25854,14 +25866,17 @@ LATERAL pg_ls_dir_recurse(dir) AS a; <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's WAL archive status directory - (<filename>pg_wal/archive_status</filename>), list the file's - name, size, last modification time (mtime), and a boolean indicating if - the file is a directory. + (<filename>pg_wal/archive_status</filename>), list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -25881,13 +25896,17 @@ LATERAL pg_ls_dir_recurse(dir) AS a; <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the temporary directory within the given - <parameter>tablespace</parameter>, return the file's name, size, last - modification time (mtime) and a boolean indicating if the file is a directory. + <parameter>tablespace</parameter>, list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Directories are used for temporary files shared by parallel processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is examined. diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 16a929f5bb..d2f79d4ba9 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -36,6 +36,8 @@ #include "utils/syscache.h" #include "utils/timestamp.h" +static void tuple_from_stat(struct stat *fst, const char *path, Datum *values, + bool *isnull); static Datum pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags); #define LS_DIR_ISDIR (1<<0) /* Show column: isdir */ @@ -403,6 +405,28 @@ pg_read_binary_file_all(PG_FUNCTION_ARGS) return pg_read_binary_file(fcinfo); } +/* + * Populate values and isnull from fst and path. + * Used for pg_stat_file() and pg_stat_dir_files() + * isnull is assumed to have been zeroed. + */ +static void +tuple_from_stat(struct stat *fst, const char *path, Datum *values, bool *isnull) +{ + values[0] = Int64GetDatum((int64) fst->st_size); + values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_atime)); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_mtime)); + /* Unix has file status change time, while Win32 has creation time */ +#if !defined(WIN32) && !defined(__CYGWIN__) + values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); + isnull[4] = true; +#else + isnull[3] = true; + values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); +#endif + values[5] = BoolGetDatum(S_ISDIR(fst->st_mode)); +} + /* * stat a file */ @@ -454,25 +478,7 @@ pg_stat_file(PG_FUNCTION_ARGS) BlessTupleDesc(tupdesc); memset(isnull, false, sizeof(isnull)); - - values[0] = Int64GetDatum((int64) fst.st_size); - values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_atime)); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_mtime)); - /* Unix has file status change time, while Win32 has creation time */ -#if !defined(WIN32) && !defined(__CYGWIN__) - values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); - isnull[4] = true; -#else - isnull[3] = true; - values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); -#endif - values[5] = BoolGetDatum(S_ISDIR(fst.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(filename)) - values[5] = BoolGetDatum(false); -#endif - + tuple_from_stat(&fst, filename, values, isnull); tuple = heap_form_tuple(tupdesc, values, isnull); pfree(filename); @@ -614,8 +620,8 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) while ((de = ReadDir(dirdesc, dir)) != NULL) { - Datum values[4]; - bool nulls[4]; + Datum values[7]; + bool nulls[7]; char path[MAXPGPATH * 2]; struct stat attrib; @@ -654,23 +660,10 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) continue; } + memset(nulls, 0, sizeof(nulls)); values[0] = CStringGetTextDatum(de->d_name); if (flags & LS_DIR_METADATA) - { - values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); - if (flags & LS_DIR_ISDIR) - { - values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(path)) - values[3] = BoolGetDatum(false); -#endif - } - } - - memset(nulls, 0, sizeof(nulls)); + tuple_from_stat(&attrib, path, 1+values, 1+nulls); tuplestore_putvalues(tupstore, tupdesc, values, nulls); } diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 826a31c561..dfb2e7717e 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10895,48 +10895,48 @@ { oid => '3353', descr => 'list files in the log directory', proname => 'pg_ls_logdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_logdir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logdir' }, { oid => '3354', descr => 'list of files in the WAL directory', proname => 'pg_ls_waldir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_waldir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_waldir' }, { oid => '5031', descr => 'list of files in the archive_status directory', proname => 'pg_ls_archive_statusdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_archive_statusdir' }, { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{tablespace,name,size,modification,isdir}', + proallargtypes => '{oid,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{tablespace,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '9979', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text bool bool', - proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}', - proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,modification,isdir}', + proallargtypes => '{text,bool,bool,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o,o,o,o}', + proargnames => '{dirname,missing_ok,include_dot_dirs,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata' }, { oid => '9980', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{dirname,filename,size,modification,isdir}', + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{dirname,filename,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata_1arg' }, { oid => '9981', descr => 'list all files in a directory recursively', proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,text,int8,timestamptz,bool}', - proargnames => '{dirname,path,filename,size,modification,isdir}', proargmodes => '{i,o,o,o,o,o}', - prolang => 'sql', prosrc => "with recursive ls as (select dirname as path, * from pg_ls_dir_metadata(dirname, false, false) union all select coalesce(nullif(parent.path,'.')||'/','')||parent.filename, a.filename, a.size, a.modification, a.isdir from ls as parent, lateral pg_ls_dir_metadata(parent.path||'/'||parent.filename, false, false) as a where parent.isdir) select * from ls" }, + proallargtypes => '{text,text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargnames => '{dirname,path,filename,size,access,modification,change,creation,isdir}', proargmodes => '{i,o,o,o,o,o,o,o,o}', + prolang => 'sql', prosrc => "with recursive ls as (select dirname as path, * from pg_ls_dir_metadata(dirname, false, false) union all select coalesce(nullif(parent.path,'.')||'/','')||parent.filename, a.filename, a.size, a.access, a.modification, a.change, a.creation, a.isdir from ls as parent, lateral pg_ls_dir_metadata(parent.path||'/'||parent.filename, false, false) as a where parent.isdir) select * from ls" }, # hash partitioning constraint function { oid => '5028', descr => 'hash partition CHECK constraint', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 302f348a6d..4be27fe21e 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -157,8 +157,8 @@ select count(*) > 0 as ok from (select pg_ls_waldir()) ss; -- Test not-run-to-completion cases. select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select count(*) > 0 as ok from (select * from pg_ls_waldir() limit 1) ss; @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select filename, isdir from pg_ls_dir_metadata('.') where filename='.'; @@ -239,8 +239,8 @@ select filename, isdir from pg_ls_dir_metadata('.', false, false) where filename -- Check that expected columns are present select * from pg_ls_dir_metadata('.') limit 0; - filename | size | modification | isdir -----------+------+--------------+------- + filename | size | access | modification | change | creation | isdir +----------+------+--------+--------------+--------+----------+------- (0 rows) -- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix @@ -252,8 +252,8 @@ select path, filename, isdir from pg_ls_dir_recurse('.') where isdir and path='p -- Check that expected columns are present select * from pg_ls_dir_recurse('.') limit 0; - path | filename | size | modification | isdir -------+----------+------+--------------+------- + path | filename | size | access | modification | change | creation | isdir +------+----------+------+--------+--------------+--------+----------+------- (0 rows) -- diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index ba9a3fe29a..1e1e02b589 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,15 +17,15 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet. -- The name='' condition is never true, so the function runs to completion but returns zero rows. SELECT * FROM pg_ls_logdir() WHERE name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --Tcb1KvpfnM4LxW2s Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v21-0010-pg_ls_-to-show-file-type-and-show-special-files.patch" ^ permalink raw reply [nested|flat] 23+ messages in thread
* [PATCH v17 09/10] pg_ls_*dir to return all the metadata from pg_stat_file.. @ 2020-03-10 02:56 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 23+ messages in thread From: Justin Pryzby @ 2020-03-10 02:56 UTC (permalink / raw) ..but it doesn't seem worth factoring out the common bits, since stat_file doesn't return a name, so all the field numbers are off by one. NOTE, the atime is now shown where the mtime used to be. Need catversion bump --- doc/src/sgml/func.sgml | 36 ++++++++--- src/backend/utils/adt/genfile.c | 65 +++++++++----------- src/include/catalog/pg_proc.dat | 34 +++++----- src/test/regress/expected/misc_functions.out | 16 ++--- src/test/regress/output/tablespace.source | 8 +-- 5 files changed, 85 insertions(+), 74 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 334b6beb3a..c3ec56a5f6 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -25700,13 +25700,16 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's log directory, - return the file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + return the file's name, along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -25727,7 +25730,10 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> @@ -25747,7 +25753,10 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> @@ -25779,13 +25788,15 @@ LATERAL pg_ls_dir_recurse(dir) AS a; <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's write-ahead log (WAL) directory, list the - file's name, size, last modification time (mtime), and a boolean - indicating if the file is a directory. + filename along with the metadata columns returned by <function>pg_stat_file</function>. Filenames beginning with a dot and special files types are excluded. </para> <para> @@ -25804,14 +25815,17 @@ LATERAL pg_ls_dir_recurse(dir) AS a; <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the server's WAL archive status directory - (<filename>pg_wal/archive_status</filename>), list the file's - name, size, last modification time (mtime), and a boolean indicating if - the file is a directory. + (<filename>pg_wal/archive_status</filename>), list the filename + along with the metadata columns returned by + <function>pg_stat_file</function>. Filenames beginning with a dot and special file types are excluded. </para> <para> @@ -25831,13 +25845,17 @@ LATERAL pg_ls_dir_recurse(dir) AS a; <returnvalue>setof record</returnvalue> ( <parameter>name</parameter> <type>text</type>, <parameter>size</parameter> <type>bigint</type>, + <parameter>access</parameter> <type>timestamp with time zone</type>, <parameter>modification</parameter> <type>timestamp with time zone</type>, + <parameter>change</parameter> <type>timestamp with time zone</type>, + <parameter>creation</parameter> <type>timestamp with time zone</type>, <parameter>isdir</parameter> <type>boolean</type> ) </para> <para> For each file in the temporary directory within the given - <parameter>tablespace</parameter>, list the file's name, size, last - modification time (mtime) and a boolean indicating if the file is a directory. + <parameter>tablespace</parameter>, list the file's name + along with the metadata columns returned by + <function>pg_stat_file</function>. Directories are used for temporary files shared by parallel processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is examined. diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 91bf8c69e9..10780d3fb1 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -36,6 +36,8 @@ #include "utils/syscache.h" #include "utils/timestamp.h" +static void tuple_from_stat(struct stat *fst, const char *path, Datum *values, + bool *isnull); static Datum pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags); #define LS_DIR_ISDIR (1<<0) /* Show column: isdir */ @@ -367,6 +369,28 @@ pg_read_binary_file_all(PG_FUNCTION_ARGS) return pg_read_binary_file(fcinfo); } +/* + * Populate values and isnull from fst and path. + * Used for pg_stat_file() and pg_stat_dir_files() + * isnull is assumed to have been zeroed. + */ +static void +tuple_from_stat(struct stat *fst, const char *path, Datum *values, bool *isnull) +{ + values[0] = Int64GetDatum((int64) fst->st_size); + values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_atime)); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_mtime)); + /* Unix has file status change time, while Win32 has creation time */ +#if !defined(WIN32) && !defined(__CYGWIN__) + values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); + isnull[4] = true; +#else + isnull[3] = true; + values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst->st_ctime)); +#endif + values[5] = BoolGetDatum(S_ISDIR(fst->st_mode)); +} + /* * stat a file */ @@ -418,25 +442,7 @@ pg_stat_file(PG_FUNCTION_ARGS) BlessTupleDesc(tupdesc); memset(isnull, false, sizeof(isnull)); - - values[0] = Int64GetDatum((int64) fst.st_size); - values[1] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_atime)); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_mtime)); - /* Unix has file status change time, while Win32 has creation time */ -#if !defined(WIN32) && !defined(__CYGWIN__) - values[3] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); - isnull[4] = true; -#else - isnull[3] = true; - values[4] = TimestampTzGetDatum(time_t_to_timestamptz(fst.st_ctime)); -#endif - values[5] = BoolGetDatum(S_ISDIR(fst.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(filename)) - values[5] = BoolGetDatum(false); -#endif - + tuple_from_stat(&fst, filename, values, isnull); tuple = heap_form_tuple(tupdesc, values, isnull); pfree(filename); @@ -578,8 +584,8 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) while ((de = ReadDir(dirdesc, dir)) != NULL) { - Datum values[4]; - bool nulls[4]; + Datum values[7]; + bool nulls[7]; char path[MAXPGPATH * 2]; struct stat attrib; @@ -618,23 +624,10 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) continue; } + memset(nulls, 0, sizeof(nulls)); values[0] = CStringGetTextDatum(de->d_name); if (flags & LS_DIR_METADATA) - { - values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); - if (flags & LS_DIR_ISDIR) - { - values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); -#ifdef WIN32 - /* Links should have isdir=false */ - if (pgwin32_is_junction(path)) - values[3] = BoolGetDatum(false); -#endif - } - } - - memset(nulls, 0, sizeof(nulls)); + tuple_from_stat(&attrib, path, 1+values, 1+nulls); tuplestore_putvalues(tupstore, tupdesc, values, nulls); } diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index f94f403475..34ee5a8f97 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -10880,48 +10880,48 @@ { oid => '3353', descr => 'list files in the log directory', proname => 'pg_ls_logdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_logdir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logdir' }, { oid => '3354', descr => 'list of files in the WAL directory', proname => 'pg_ls_waldir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_waldir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_waldir' }, { oid => '5031', descr => 'list of files in the archive_status directory', proname => 'pg_ls_archive_statusdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_archive_statusdir' }, { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{tablespace,name,size,modification,isdir}', + proallargtypes => '{oid,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{tablespace,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text bool bool', - proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}', - proargnames => '{dirname,missing_ok,include_dot_dirs,name,size,modification,isdir}', + proallargtypes => '{text,bool,bool,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o,o,o,o}', + proargnames => '{dirname,missing_ok,include_dot_dirs,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata' }, { oid => '5033', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{dirname,name,size,modification,isdir}', + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{dirname,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata_1arg' }, { oid => '5034', descr => 'list all files in a directory recursively', proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', - proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}', - prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" }, + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargnames => '{dirname,name,size,access,modification,change,creation,isdir}', proargmodes => '{i,o,o,o,o,o,o,o}', + prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.access, a.modification, a.change, a.creation, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" }, # hash partitioning constraint function { oid => '5028', descr => 'hash partition CHECK constraint', diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 4188d684f0..cce84a60a9 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -157,8 +157,8 @@ select count(*) > 0 as ok from (select pg_ls_waldir()) ss; -- Test not-run-to-completion cases. select * from pg_ls_waldir() limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select count(*) > 0 as ok from (select * from pg_ls_waldir() limit 1) ss; @@ -222,8 +222,8 @@ ERROR: could not open directory "does not exist": No such file or directory -- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet -- The name='' condition is never true, so the function runs to completion but returns zero rows. select * from pg_ls_tmpdir() where name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) select name, isdir from pg_ls_dir_metadata('.') where name='.'; @@ -239,8 +239,8 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; -- -- Check that expected columns are present select * from pg_ls_dir_metadata('.') limit 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix @@ -253,8 +253,8 @@ SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal'; -- Check that expected columns are present SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index ba9a3fe29a..1e1e02b589 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,15 +17,15 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet. -- The name='' condition is never true, so the function runs to completion but returns zero rows. SELECT * FROM pg_ls_logdir() WHERE name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --4LFBTxd4L5NLO6ly Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v17-0010-pg_ls_-to-show-file-type-and-show-special-files.patch" ^ permalink raw reply [nested|flat] 23+ messages in thread
* [PATCH v9 11/11] pg_ls_*dir to return all the metadata from pg_stat_file.. @ 2020-03-10 02:56 Justin Pryzby <pryzbyj@telsasoft.com> 0 siblings, 0 replies; 23+ messages in thread From: Justin Pryzby @ 2020-03-10 02:56 UTC (permalink / raw) ..but it doesn't seem worth factoring out the common bits, since stat_file doesn't return a name, so all the field numbers are off by one. Need catversion bump --- doc/src/sgml/func.sgml | 30 +++++++++------------ src/backend/utils/adt/genfile.c | 32 ++++++++++++++++------- src/include/catalog/pg_proc.dat | 30 ++++++++++----------- src/test/regress/output/tablespace.source | 8 +++--- 4 files changed, 55 insertions(+), 45 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index d0b782d803..ae6c004a6d 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -21348,8 +21348,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - For each file in the log directory, list the file's name, size, last - modification time, and a boolean indicating if it is a directory. + For each file in the log directory, list the file and its metadata. Access is granted to members of the <literal>pg_monitor</literal> role and may be granted to other non-superuser roles. </entry> @@ -21360,8 +21359,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - For each file in the WAL directory, list the file's name, size, last - modification time, and a boolean indicating if it is a directory. + For each file in the WAL directory, list the file and its metadata. Access is granted to members of the <literal>pg_monitor</literal> role and may be granted to other non-superuser roles. </entry> @@ -21372,9 +21370,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); </entry> <entry><type>setof record</type></entry> <entry> - For each file in the WAL archive status directory, list the file's - name, size, last modification time, and a boolean indicating if it is a - directory. Access is granted to members of the + For each file in the WAL archive status directory, list the file and its metadata. + Access is granted to members of the <literal>pg_monitor</literal> role and may be granted to other non-superuser roles. </entry> @@ -21386,8 +21383,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup()); <entry><type>setof record</type></entry> <entry> For the temporary directory within <parameter>tablespace</parameter>, - list each file's name, size, last modification time, and a boolean - indicating if it is a directory. If <parameter>tablespace</parameter> + list each file and its metadata. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. Access is granted to members of the <literal>pg_monitor</literal> role and may be granted to other non-superuser roles. @@ -21463,8 +21459,8 @@ SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'') </indexterm> <para> <function>pg_ls_logdir</function> lists each file in the log directory, - along with file's size, last modification time, and a boolean - indicating if the file is a directory. By default, only superusers + along with the metadata columns returned by <function>pg_stat_file</function>. + By default, only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. Filenames beginning with a dot and special file types are not shown. @@ -21475,8 +21471,8 @@ SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'') </indexterm> <para> <function>pg_ls_waldir</function> lists each file in the WAL directory, - along with the file's size, last modification time, and a boolean - indicating if the file is a directory. By default, only superusers + along with the metadata columns returned by <function>pg_stat_file</function>. + By default, only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. @@ -21488,8 +21484,8 @@ SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'') </indexterm> <para> <function>pg_ls_archive_statusdir</function> lists each file in the WAL - archive status directory, along with the file's size, last modification - time, and a boolean indicating if the file is a directory. By default, only + archive status directory, along with the metadata columns returned by + <function>pg_stat_file</function>. By default, only superusers and members of the <literal>pg_monitor</literal> role can use this function. Access may be granted to others using <command>GRANT</command>. @@ -21502,8 +21498,8 @@ SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'') <para> <function>pg_ls_tmpdir</function> lists each file in the temporary file directory for the specified <parameter>tablespace</parameter>, along with - its size, last modified time (mtime) and a boolean indicating if the file is a - directory. Directories are used for temporary files shared by parallel + the metadata columns returned by <function>pg_stat_file</function>. + Directories are used for temporary files shared by parallel processes. If <parameter>tablespace</parameter> is not provided, the <literal>pg_default</literal> tablespace is used. By default only superusers and members of the <literal>pg_monitor</literal> diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c index 387114d4ee..1fe7e0181d 100644 --- a/src/backend/utils/adt/genfile.c +++ b/src/backend/utils/adt/genfile.c @@ -510,16 +510,22 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) fctx = palloc(sizeof(directory_fctx)); - tupdesc = CreateTemplateTupleDesc((flags&FLAG_METADATA) ? 4 : 1); + tupdesc = CreateTemplateTupleDesc((flags&FLAG_METADATA) ? 7 : 1); TupleDescInitEntry(tupdesc, (AttrNumber) 1, "name", TEXTOID, -1, 0); if (flags&FLAG_METADATA) { TupleDescInitEntry(tupdesc, (AttrNumber) 2, "size", INT8OID, -1, 0); - TupleDescInitEntry(tupdesc, (AttrNumber) 3, "modification", + TupleDescInitEntry(tupdesc, (AttrNumber) 3, + "access", TIMESTAMPTZOID, -1, 0); + TupleDescInitEntry(tupdesc, (AttrNumber) 4, "modification", TIMESTAMPTZOID, -1, 0); - TupleDescInitEntry(tupdesc, (AttrNumber) 4, "isdir", + TupleDescInitEntry(tupdesc, (AttrNumber) 5, + "change", TIMESTAMPTZOID, -1, 0); + TupleDescInitEntry(tupdesc, (AttrNumber) 6, + "creation", TIMESTAMPTZOID, -1, 0); + TupleDescInitEntry(tupdesc, (AttrNumber) 7, "isdir", BOOLOID, -1, 0); } @@ -551,8 +557,8 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) while ((de = ReadDir(fctx->dirdesc, fctx->location)) != NULL) { - Datum values[4]; - bool nulls[4]; + Datum values[7]; + bool nulls[7]; char path[MAXPGPATH * 2]; struct stat attrib; HeapTuple tuple; @@ -579,17 +585,25 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *dir, int flags) else if (!S_ISREG(attrib.st_mode)) continue; + memset(nulls, 0, sizeof(nulls)); if (flags & FLAG_METADATA) { values[0] = CStringGetTextDatum(de->d_name); values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); - values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_atime)); + values[3] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); + /* Unix has file status change time, while Win32 has creation time */ +#if !defined(WIN32) && !defined(__CYGWIN__) + values[4] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_ctime)); + nulls[5] = true; +#else + nulls[4] = true; + values[5] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_ctime)); +#endif + values[6] = BoolGetDatum(S_ISDIR(attrib.st_mode)); } else SRF_RETURN_NEXT(funcctx, CStringGetTextDatum(de->d_name)); - memset(nulls, 0, sizeof(nulls)); - tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls); SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple)); } diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 0e5a570285..b571666a79 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -6147,9 +6147,9 @@ { oid => '8511', descr => 'list all files in a directory recursively', proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text', - proallargtypes => '{text,text,int8,timestamptz,bool}', - proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}', - prolang => 'sql', prosrc => "WITH RECURSIVE x AS (SELECT * FROM pg_ls_dir_metadata(dirname, true, false) UNION ALL SELECT x.name||'/'||a.name, a.size, a.modification, a.isdir FROM x, pg_ls_dir_metadata(dirname||'/'||x.name, true, false)a WHERE x.isdir) SELECT * FROM x" }, + proallargtypes => '{text,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargnames => '{dirname,name,size,access,modification,change,creation,isdir}', proargmodes => '{i,o,o,o,o,o,o,o}', + prolang => 'sql', prosrc => "WITH RECURSIVE x AS (SELECT * FROM pg_ls_dir_metadata(dirname, true, false) UNION ALL SELECT x.name||'/'||a.name, a.size,a.access,a.modification,a.change,a.creation,a.isdir FROM x, pg_ls_dir_metadata(dirname||'/'||x.name, true, false)a WHERE x.isdir) SELECT * FROM x" }, { oid => '2626', descr => 'sleep for the specified time in seconds', proname => 'pg_sleep', provolatile => 'v', prorettype => 'void', @@ -10725,35 +10725,35 @@ { oid => '3353', descr => 'list files in the log directory', proname => 'pg_ls_logdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_logdir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_logdir' }, { oid => '3354', descr => 'list of files in the WAL directory', proname => 'pg_ls_waldir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_waldir' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_waldir' }, { oid => '5031', descr => 'list of files in the archive_status directory', proname => 'pg_ls_archive_statusdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', - proargtypes => '', proallargtypes => '{text,int8,timestamptz,bool}', - proargmodes => '{o,o,o,o}', proargnames => '{name,size,modification,isdir}', + proargtypes => '', proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', + proargmodes => '{o,o,o,o,o,o,o}', proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_archive_statusdir' }, { oid => '5029', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => '', - proallargtypes => '{text,int8,timestamptz,bool}', proargmodes => '{o,o,o,o}', - proargnames => '{name,size,modification,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, + proallargtypes => '{text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{o,o,o,o,o,o,o}', + proargnames => '{name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_noargs' }, { oid => '5030', descr => 'list files in the pgsql_tmp directory', proname => 'pg_ls_tmpdir', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'oid', - proallargtypes => '{oid,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}', - proargnames => '{tablespace,name,size,modification,isdir}', + proallargtypes => '{oid,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,o,o,o,o,o,o,o}', + proargnames => '{tablespace,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_tmpdir_1arg' }, { oid => '5032', descr => 'list directory with metadata', proname => 'pg_ls_dir_metadata', procost => '10', prorows => '20', proretset => 't', provolatile => 'v', prorettype => 'record', proargtypes => 'text bool bool', - proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}', - proargnames => '{dirname,missing_ok,include_dot_dirs,name,size,modification,isdir}', + proallargtypes => '{text,bool,bool,text,int8,timestamptz,timestamptz,timestamptz,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o,o,o,o}', + proargnames => '{dirname,missing_ok,include_dot_dirs,name,size,access,modification,change,creation,isdir}', prosrc => 'pg_ls_dir_metadata' }, # hash partitioning constraint function diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source index dacaafcae6..1e1e02b589 100644 --- a/src/test/regress/output/tablespace.source +++ b/src/test/regress/output/tablespace.source @@ -17,15 +17,15 @@ CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; -- The name='' condition is never true, so the function runs to completion but returns zero rows. -- The query is written to ERROR if the tablespace doesn't exist, rather than silently failing to call pg_ls_tmpdir() SELECT c.* FROM (SELECT oid FROM pg_tablespace b WHERE b.spcname='regress_tblspace' UNION SELECT 0 ORDER BY 1 DESC LIMIT 1) AS b , pg_ls_tmpdir(oid) AS c WHERE c.name='Does not exist'; - name | size | modification | isdir -------+------+--------------+------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- This tests the missing_ok parameter. If that's not functioning, this would ERROR if the logdir doesn't exist yet. -- The name='' condition is never true, so the function runs to completion but returns zero rows. SELECT * FROM pg_ls_logdir() WHERE name='Does not exist'; - name | size | modification -------+------+-------------- + name | size | access | modification | change | creation | isdir +------+------+--------+--------------+--------+----------+------- (0 rows) -- try setting and resetting some properties for the new tablespace -- 2.17.0 --32u276st3Jlj2kUU-- ^ permalink raw reply [nested|flat] 23+ messages in thread
* [PATCH v7 1/3] log_min_messages per process type @ 2025-11-04 20:18 Euler Taveira <euler@eulerto.com> 0 siblings, 0 replies; 23+ messages in thread From: Euler Taveira @ 2025-11-04 20:18 UTC (permalink / raw) Change log_min_messages from a single element to a comma-separated list of elements. Each element is type:level. The types are archiver, autovacuum (includes launcher and workers), backend, bgworker, bgwriter, checkpointer, ioworker, postmaster, syslogger, slotsyncworker, startup, walreceiver, walsender, walsummarizer and walwriter. A single log level should be part of this list and it is applied as a final step to the process types that were not informed. The old syntax (a single log level) is still accepted for backward compatibility. In this case, it means the informed log level is applied for all process types. The default log level is the same (WARNING) for all process types. --- doc/src/sgml/config.sgml | 42 +++- src/backend/commands/extension.c | 2 +- src/backend/commands/variable.c | 208 ++++++++++++++++++ src/backend/postmaster/launch_backend.c | 2 +- src/backend/utils/error/elog.c | 2 +- src/backend/utils/init/miscinit.c | 2 +- src/backend/utils/misc/guc_parameters.dat | 10 +- src/backend/utils/misc/guc_tables.c | 21 +- src/backend/utils/misc/postgresql.conf.sample | 2 + src/include/postmaster/proctypelist.h | 42 ++-- src/include/utils/guc.h | 6 +- src/include/utils/guc_hooks.h | 2 + src/test/regress/expected/guc.out | 54 +++++ src/test/regress/sql/guc.sql | 18 ++ 14 files changed, 372 insertions(+), 41 deletions(-) diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index 405c9689bd0..62d6c0d36a2 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -7069,7 +7069,7 @@ local0.* /var/log/postgresql <variablelist> <varlistentry id="guc-log-min-messages" xreflabel="log_min_messages"> - <term><varname>log_min_messages</varname> (<type>enum</type>) + <term><varname>log_min_messages</varname> (<type>string</type>) <indexterm> <primary><varname>log_min_messages</varname> configuration parameter</primary> </indexterm> @@ -7078,14 +7078,38 @@ local0.* /var/log/postgresql <para> Controls which <link linkend="runtime-config-severity-levels">message levels</link> are written to the server log. - Valid values are <literal>DEBUG5</literal>, <literal>DEBUG4</literal>, - <literal>DEBUG3</literal>, <literal>DEBUG2</literal>, <literal>DEBUG1</literal>, - <literal>INFO</literal>, <literal>NOTICE</literal>, <literal>WARNING</literal>, - <literal>ERROR</literal>, <literal>LOG</literal>, <literal>FATAL</literal>, and - <literal>PANIC</literal>. Each level includes all the levels that - follow it. The later the level, the fewer messages are sent - to the log. The default is <literal>WARNING</literal>. Note that - <literal>LOG</literal> has a different rank here than in + Valid values are a comma-separated list of <literal>type:level</literal> + and a single <literal>level</literal>. The list allows it to use + different levels per process type. Only the single <literal>level</literal> + is mandatory (order does not matter) and it is assigned to the process + types that are not specified in the list. + Valid process types are listed in the table below, each corresponding to + either postmaster, an auxiliary process type or a backend. + <simplelist type="vert" columns="4"> + <member><literal>archiver</literal></member> + <member><literal>autovacuum</literal></member> + <member><literal>backend</literal></member> + <member><literal>bgworker</literal></member> + <member><literal>bgwriter</literal></member> + <member><literal>checkpointer</literal></member> + <member><literal>ioworker</literal></member> + <member><literal>postmaster</literal></member> + <member><literal>syslogger</literal></member> + <member><literal>slotsyncworker</literal></member> + <member><literal>startup</literal></member> + <member><literal>walreceiver</literal></member> + <member><literal>walsender</literal></member> + <member><literal>walsummarizer</literal></member> + <member><literal>walwriter</literal></member> + </simplelist> + Valid <literal>level</literal> values are <literal>DEBUG5</literal>, + <literal>DEBUG4</literal>, <literal>DEBUG3</literal>, <literal>DEBUG2</literal>, + <literal>DEBUG1</literal>, <literal>INFO</literal>, <literal>NOTICE</literal>, + <literal>WARNING</literal>, <literal>ERROR</literal>, <literal>LOG</literal>, + <literal>FATAL</literal>, and <literal>PANIC</literal>. Each level includes + all the levels that follow it. The later the level, the fewer messages are sent + to the log. The default is <literal>WARNING</literal> for all process types. + Note that <literal>LOG</literal> has a different rank here than in <xref linkend="guc-client-min-messages"/>. Only superusers and users with the appropriate <literal>SET</literal> privilege can change this setting. diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c index ebc204c4462..d123b7ebb77 100644 --- a/src/backend/commands/extension.c +++ b/src/backend/commands/extension.c @@ -1143,7 +1143,7 @@ execute_extension_script(Oid extensionOid, ExtensionControlFile *control, (void) set_config_option("client_min_messages", "warning", PGC_USERSET, PGC_S_SESSION, GUC_ACTION_SAVE, true, 0, false); - if (log_min_messages < WARNING) + if (log_min_messages[MyBackendType] < WARNING) (void) set_config_option_ext("log_min_messages", "warning", PGC_SUSET, PGC_S_SESSION, BOOTSTRAP_SUPERUSERID, diff --git a/src/backend/commands/variable.c b/src/backend/commands/variable.c index 608f10d9412..2b309791ee7 100644 --- a/src/backend/commands/variable.c +++ b/src/backend/commands/variable.c @@ -34,6 +34,7 @@ #include "utils/backend_status.h" #include "utils/datetime.h" #include "utils/fmgrprotos.h" +#include "utils/guc.h" #include "utils/guc_hooks.h" #include "utils/snapmgr.h" #include "utils/syscache.h" @@ -1257,3 +1258,210 @@ check_ssl(bool *newval, void **extra, GucSource source) #endif return true; } + +/* + * GUC check_hook for log_min_messages + * + * The parsing consists of a comma-separated list of TYPE:LEVEL elements. TYPE + * is log_min_messages_process_types. LEVEL is server_message_level_options. A + * single LEVEL element should be part of this list and it is applied as a + * final step to the process types that are not specified. For backward + * compatibility, the old syntax is still accepted and it means to apply the + * LEVEL for all process types. + */ +bool +check_log_min_messages(char **newval, void **extra, GucSource source) +{ + char *rawstring; + List *elemlist; + ListCell *l; + int newlevel[BACKEND_NUM_TYPES]; + bool assigned[BACKEND_NUM_TYPES]; + int genericlevel = -1; /* -1 means not assigned */ + + /* Initialize the array. */ + for (int i = 0; i < BACKEND_NUM_TYPES; i++) + { + newlevel[i] = WARNING; + assigned[i] = false; + } + + /* Need a modifiable copy of string. */ + rawstring = guc_strdup(LOG, *newval); + + /* Parse string into list of identifiers. */ + if (!SplitGUCList(rawstring, ',', &elemlist)) + { + /* syntax error in list */ + GUC_check_errdetail("List syntax is invalid."); + list_free(elemlist); + guc_free(rawstring); + return false; + } + + /* Validate and assign log level and process type. */ + foreach(l, elemlist) + { + char *tok = (char *) lfirst(l); + char *sep; + const struct config_enum_entry *entry; + + /* + * Check whether there is a process type following the log level. If + * there is no separator, it means this is the generic log level. The + * generic log level will be assigned to the process types that were + * not informed. + */ + sep = strchr(tok, ':'); + if (sep == NULL) + { + bool found = false; + + /* Reject duplicates for generic log level. */ + if (genericlevel != -1) + { + GUC_check_errdetail("Generic log level was already assigned."); + guc_free(rawstring); + list_free(elemlist); + return false; + } + + /* Is the log level valid? */ + for (entry = server_message_level_options; entry && entry->name; entry++) + { + if (pg_strcasecmp(entry->name, tok) == 0) + { + genericlevel = entry->val; + found = true; + break; + } + } + + if (!found) + { + GUC_check_errdetail("Unrecognized log level: \"%s\".", tok); + guc_free(rawstring); + list_free(elemlist); + return false; + } + } + else + { + char *loglevel; + char *ptype; + bool found = false; + + ptype = guc_malloc(LOG, (sep - tok) + 1); + if (!ptype) + { + guc_free(rawstring); + list_free(elemlist); + return false; + } + memcpy(ptype, tok, sep - tok); + ptype[sep - tok] = '\0'; + loglevel = sep + 1; + + /* Is the log level valid? */ + for (entry = server_message_level_options; entry && entry->name; entry++) + { + if (pg_strcasecmp(entry->name, loglevel) == 0) + { + found = true; + break; + } + } + + if (!found) + { + GUC_check_errdetail("Unrecognized log level: \"%s\".", loglevel); + guc_free(ptype); + guc_free(rawstring); + list_free(elemlist); + return false; + } + + /* + * Is the process type name valid? There might be multiple entries + * per process type, don't bail out because it can assign the + * value for multiple entries. + */ + found = false; + for (int i = 0; i < BACKEND_NUM_TYPES; i++) + { + if (pg_strcasecmp(log_min_messages_process_types[i], ptype) == 0) + { + /* Reject duplicates for a process type. */ + if (assigned[i]) + { + GUC_check_errdetail("Process type \"%s\" was already assigned.", ptype); + guc_free(ptype); + guc_free(rawstring); + list_free(elemlist); + return false; + } + + newlevel[i] = entry->val; + assigned[i] = true; + found = true; + } + } + + if (!found) + { + GUC_check_errdetail("Unrecognized process type: \"%s\".", ptype); + guc_free(ptype); + guc_free(rawstring); + list_free(elemlist); + return false; + } + + guc_free(ptype); + } + } + + /* + * The generic log level must be specified. It is the fallback value. + */ + if (genericlevel == -1) + { + GUC_check_errdetail("Generic log level was not defined."); + guc_free(rawstring); + list_free(elemlist); + return false; + } + + /* + * Apply the generic log level after all of the specific process types + * have been assigned. Hence, it doesn't matter the order you specify the + * generic log level, the final result will be the same. + */ + for (int i = 0; i < BACKEND_NUM_TYPES; i++) + { + if (!assigned[i]) + newlevel[i] = genericlevel; + } + + guc_free(rawstring); + list_free(elemlist); + + /* + * Pass back data for assign_log_min_messages to use. + */ + *extra = guc_malloc(LOG, BACKEND_NUM_TYPES * sizeof(int)); + if (!*extra) + return false; + memcpy(*extra, newlevel, BACKEND_NUM_TYPES * sizeof(int)); + + return true; +} + +/* + * GUC assign_hook for log_min_messages + */ +void +assign_log_min_messages(const char *newval, void *extra) +{ + for (int i = 0; i < BACKEND_NUM_TYPES; i++) + log_min_messages[i] = ((int *) extra)[i]; +} diff --git a/src/backend/postmaster/launch_backend.c b/src/backend/postmaster/launch_backend.c index 976638a58ac..c2a044321d1 100644 --- a/src/backend/postmaster/launch_backend.c +++ b/src/backend/postmaster/launch_backend.c @@ -179,7 +179,7 @@ typedef struct } child_process_kind; static child_process_kind child_process_kinds[] = { -#define PG_PROCTYPE(bktype, description, main_func, shmem_attach) \ +#define PG_PROCTYPE(bktype, bkcategory, description, main_func, shmem_attach, log_min_messages) \ [bktype] = {description, main_func, shmem_attach}, #include "postmaster/proctypelist.h" #undef PG_PROCTYPE diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c index 29643c51439..15d771a16c9 100644 --- a/src/backend/utils/error/elog.c +++ b/src/backend/utils/error/elog.c @@ -235,7 +235,7 @@ is_log_level_output(int elevel, int log_min_level) static inline bool should_output_to_server(int elevel) { - return is_log_level_output(elevel, log_min_messages); + return is_log_level_output(elevel, log_min_messages[MyBackendType]); } /* diff --git a/src/backend/utils/init/miscinit.c b/src/backend/utils/init/miscinit.c index fec79992c8d..2c3926cbadf 100644 --- a/src/backend/utils/init/miscinit.c +++ b/src/backend/utils/init/miscinit.c @@ -266,7 +266,7 @@ GetBackendTypeDesc(BackendType backendType) switch (backendType) { -#define PG_PROCTYPE(bktype, description, main_func, shmem_attach) \ +#define PG_PROCTYPE(bktype, bkcategory, description, main_func, shmem_attach, log_min_messages) \ case bktype: backendDesc = description; break; #include "postmaster/proctypelist.h" #undef PG_PROCTYPE diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index 3b9d8349078..450180bee31 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -1683,12 +1683,14 @@ options => 'server_message_level_options', }, -{ name => 'log_min_messages', type => 'enum', context => 'PGC_SUSET', group => 'LOGGING_WHEN', +{ name => 'log_min_messages', type => 'string', context => 'PGC_SUSET', group => 'LOGGING_WHEN', short_desc => 'Sets the message levels that are logged.', long_desc => 'Each level includes all the levels that follow it. The later the level, the fewer messages are sent.', - variable => 'log_min_messages', - boot_val => 'WARNING', - options => 'server_message_level_options', + flags => 'GUC_LIST_INPUT', + variable => 'log_min_messages_string', + boot_val => '"WARNING"', + check_hook => 'check_log_min_messages', + assign_hook => 'assign_log_min_messages', }, { name => 'log_parameter_max_length', type => 'int', context => 'PGC_SUSET', group => 'LOGGING_WHAT', diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c index f87b558c2c6..a9ef447d1d4 100644 --- a/src/backend/utils/misc/guc_tables.c +++ b/src/backend/utils/misc/guc_tables.c @@ -146,7 +146,7 @@ static const struct config_enum_entry client_message_level_options[] = { {NULL, 0, false} }; -static const struct config_enum_entry server_message_level_options[] = { +const struct config_enum_entry server_message_level_options[] = { {"debug5", DEBUG5, false}, {"debug4", DEBUG4, false}, {"debug3", DEBUG3, false}, @@ -537,7 +537,6 @@ static bool default_with_oids = false; bool current_role_is_superuser; int log_min_error_statement = ERROR; -int log_min_messages = WARNING; int client_min_messages = NOTICE; int log_min_duration_sample = -1; int log_min_duration_statement = -1; @@ -595,6 +594,7 @@ static char *server_version_string; static int server_version_num; static char *debug_io_direct_string; static char *restrict_nonsystem_relation_kind_string; +static char *log_min_messages_string; #ifdef HAVE_SYSLOG #define DEFAULT_SYSLOG_FACILITY LOG_LOCAL0 @@ -646,6 +646,23 @@ char *role_string; /* should be static, but guc.c needs to get at this */ bool in_hot_standby_guc; +/* + * log_min_messages + */ +int log_min_messages[] = { +#define PG_PROCTYPE(bktype, bkcategory, description, main_func, shmem_attach, log_min_messages) \ + [bktype] = log_min_messages, +#include "postmaster/proctypelist.h" +#undef PG_PROCTYPE +}; + +const char *const log_min_messages_process_types[] = { +#define PG_PROCTYPE(bktype, bkcategory, description, main_func, shmem_attach, log_min_messages) \ + [bktype] = bkcategory, +#include "postmaster/proctypelist.h" +#undef PG_PROCTYPE +}; + /* * Displayable names for context types (enum GucContext) diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index dc9e2255f8a..24b5d983590 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -541,6 +541,8 @@ # log # fatal # panic + # and an optional comma-separated list + # of type:level #log_min_error_statement = error # values in order of decreasing detail: # debug5 diff --git a/src/include/postmaster/proctypelist.h b/src/include/postmaster/proctypelist.h index 242862451d8..5fbb8e7ab7c 100644 --- a/src/include/postmaster/proctypelist.h +++ b/src/include/postmaster/proctypelist.h @@ -25,27 +25,27 @@ */ /* - * List of process types (symbol, description, Main function, shmem_attach) - * entries. + * List of process types (symbol, category, description, Main function, + * shmem_attach, message level) entries. */ -/* bktype, description, main_func, shmem_attach */ -PG_PROCTYPE(B_ARCHIVER, gettext_noop("archiver"), PgArchiverMain, true) -PG_PROCTYPE(B_AUTOVAC_LAUNCHER, gettext_noop("autovacuum launcher"), AutoVacLauncherMain, true) -PG_PROCTYPE(B_AUTOVAC_WORKER, gettext_noop("autovacuum worker"), AutoVacWorkerMain, true) -PG_PROCTYPE(B_BACKEND, gettext_noop("client backend"), BackendMain, true) -PG_PROCTYPE(B_BG_WORKER, gettext_noop("background worker"), BackgroundWorkerMain, true) -PG_PROCTYPE(B_BG_WRITER, gettext_noop("background writer"), BackgroundWriterMain, true) -PG_PROCTYPE(B_CHECKPOINTER, gettext_noop("checkpointer"), CheckpointerMain, true) -PG_PROCTYPE(B_DEAD_END_BACKEND, gettext_noop("dead-end client backend"), BackendMain, true) -PG_PROCTYPE(B_INVALID, gettext_noop("unrecognized"), NULL, false) -PG_PROCTYPE(B_IO_WORKER, gettext_noop("io worker"), IoWorkerMain, true) -PG_PROCTYPE(B_LOGGER, gettext_noop("syslogger"), SysLoggerMain, false) -PG_PROCTYPE(B_SLOTSYNC_WORKER, gettext_noop("slotsync worker"), ReplSlotSyncWorkerMain, true) -PG_PROCTYPE(B_STANDALONE_BACKEND, gettext_noop("standalone backend"), NULL, false) -PG_PROCTYPE(B_STARTUP, gettext_noop("startup"), StartupProcessMain, true) -PG_PROCTYPE(B_WAL_RECEIVER, gettext_noop("walreceiver"), WalReceiverMain, true) -PG_PROCTYPE(B_WAL_SENDER, gettext_noop("walsender"), NULL, true) -PG_PROCTYPE(B_WAL_SUMMARIZER, gettext_noop("walsummarizer"), WalSummarizerMain, true) -PG_PROCTYPE(B_WAL_WRITER, gettext_noop("walwriter"), WalWriterMain, true) +/* bktype, bkcategory, description, main_func, shmem_attach, log_min_messages */ +PG_PROCTYPE(B_ARCHIVER, "archiver", gettext_noop("archiver"), PgArchiverMain, true, WARNING) +PG_PROCTYPE(B_AUTOVAC_LAUNCHER, "autovacuum", gettext_noop("autovacuum launcher"), AutoVacLauncherMain, true, WARNING) +PG_PROCTYPE(B_AUTOVAC_WORKER, "autovacuum", gettext_noop("autovacuum worker"), AutoVacWorkerMain, true, WARNING) +PG_PROCTYPE(B_BACKEND, "backend", gettext_noop("client backend"), BackendMain, true, WARNING) +PG_PROCTYPE(B_BG_WORKER, "bgworker", gettext_noop("background worker"), BackgroundWorkerMain, true, WARNING) +PG_PROCTYPE(B_BG_WRITER, "bgwriter", gettext_noop("background writer"), BackgroundWriterMain, true, WARNING) +PG_PROCTYPE(B_CHECKPOINTER, "checkpointer", gettext_noop("checkpointer"), CheckpointerMain, true, WARNING) +PG_PROCTYPE(B_DEAD_END_BACKEND, "backend", gettext_noop("dead-end client backend"), BackendMain, true, WARNING) +PG_PROCTYPE(B_INVALID, "postmaster", gettext_noop("unrecognized"), NULL, false, WARNING) +PG_PROCTYPE(B_IO_WORKER, "ioworker", gettext_noop("io worker"), IoWorkerMain, true, WARNING) +PG_PROCTYPE(B_LOGGER, "syslogger", gettext_noop("syslogger"), SysLoggerMain, false, WARNING) +PG_PROCTYPE(B_SLOTSYNC_WORKER, "slotsyncworker", gettext_noop("slotsync worker"), ReplSlotSyncWorkerMain, true, WARNING) +PG_PROCTYPE(B_STANDALONE_BACKEND, "backend", gettext_noop("standalone backend"), NULL, false, WARNING) +PG_PROCTYPE(B_STARTUP, "startup", gettext_noop("startup"), StartupProcessMain, true, WARNING) +PG_PROCTYPE(B_WAL_RECEIVER, "walreceiver", gettext_noop("walreceiver"), WalReceiverMain, true, WARNING) +PG_PROCTYPE(B_WAL_SENDER, "walsender", gettext_noop("walsender"), NULL, true, WARNING) +PG_PROCTYPE(B_WAL_SUMMARIZER, "walsummarizer", gettext_noop("walsummarizer"), WalSummarizerMain, true, WARNING) +PG_PROCTYPE(B_WAL_WRITER, "walwriter", gettext_noop("walwriter"), WalWriterMain, true, WARNING) diff --git a/src/include/utils/guc.h b/src/include/utils/guc.h index f21ec37da89..98b0ba08c0c 100644 --- a/src/include/utils/guc.h +++ b/src/include/utils/guc.h @@ -295,7 +295,7 @@ extern PGDLLIMPORT bool log_duration; extern PGDLLIMPORT int log_parameter_max_length; extern PGDLLIMPORT int log_parameter_max_length_on_error; extern PGDLLIMPORT int log_min_error_statement; -extern PGDLLIMPORT int log_min_messages; +extern PGDLLIMPORT int log_min_messages[]; extern PGDLLIMPORT int client_min_messages; extern PGDLLIMPORT int log_min_duration_sample; extern PGDLLIMPORT int log_min_duration_statement; @@ -329,6 +329,9 @@ extern PGDLLIMPORT bool trace_sort; extern PGDLLIMPORT bool optimize_bounded_sort; #endif +/* log_min_messages */ +extern PGDLLIMPORT const char *const log_min_messages_process_types[]; + /* * Declarations for options for enum values * @@ -344,6 +347,7 @@ extern PGDLLIMPORT const struct config_enum_entry archive_mode_options[]; extern PGDLLIMPORT const struct config_enum_entry dynamic_shared_memory_options[]; extern PGDLLIMPORT const struct config_enum_entry io_method_options[]; extern PGDLLIMPORT const struct config_enum_entry recovery_target_action_options[]; +extern PGDLLIMPORT const struct config_enum_entry server_message_level_options[]; extern PGDLLIMPORT const struct config_enum_entry wal_level_options[]; extern PGDLLIMPORT const struct config_enum_entry wal_sync_method_options[]; diff --git a/src/include/utils/guc_hooks.h b/src/include/utils/guc_hooks.h index 82ac8646a8d..e6ffc96468f 100644 --- a/src/include/utils/guc_hooks.h +++ b/src/include/utils/guc_hooks.h @@ -174,5 +174,7 @@ extern void assign_wal_sync_method(int new_wal_sync_method, void *extra); extern bool check_synchronized_standby_slots(char **newval, void **extra, GucSource source); extern void assign_synchronized_standby_slots(const char *newval, void *extra); +extern bool check_log_min_messages(char **newval, void **extra, GucSource source); +extern void assign_log_min_messages(const char *newval, void *extra); #endif /* GUC_HOOKS_H */ diff --git a/src/test/regress/expected/guc.out b/src/test/regress/expected/guc.out index d6fb879f500..3b8f44566b2 100644 --- a/src/test/regress/expected/guc.out +++ b/src/test/regress/expected/guc.out @@ -935,3 +935,57 @@ SELECT name FROM tab_settings_flags (0 rows) DROP TABLE tab_settings_flags; +-- Test log_min_messages +SET log_min_messages TO foo; -- fail +ERROR: invalid value for parameter "log_min_messages": "foo" +DETAIL: Unrecognized log level: "foo". +SET log_min_messages TO fatal; +SHOW log_min_messages; + log_min_messages +------------------ + fatal +(1 row) + +SET log_min_messages TO 'fatal'; +SHOW log_min_messages; + log_min_messages +------------------ + fatal +(1 row) + +SET log_min_messages TO 'checkpointer:debug2, autovacuum:debug1'; --fail +ERROR: invalid value for parameter "log_min_messages": "checkpointer:debug2, autovacuum:debug1" +DETAIL: Generic log level was not defined. +SET log_min_messages TO 'debug1, backend:error, fatal'; -- fail +ERROR: invalid value for parameter "log_min_messages": "debug1, backend:error, fatal" +DETAIL: Generic log level was already assigned. +SET log_min_messages TO 'backend:error, debug1, backend:warning'; -- fail +ERROR: invalid value for parameter "log_min_messages": "backend:error, debug1, backend:warning" +DETAIL: Process type "backend" was already assigned. +SET log_min_messages TO 'backend:error, foo:fatal, archiver:debug1'; -- fail +ERROR: invalid value for parameter "log_min_messages": "backend:error, foo:fatal, archiver:debug1" +DETAIL: Unrecognized process type: "foo". +SET log_min_messages TO 'backend:error, checkpointer:bar, archiver:debug1'; -- fail +ERROR: invalid value for parameter "log_min_messages": "backend:error, checkpointer:bar, archiver:debug1" +DETAIL: Unrecognized log level: "bar". +SET log_min_messages TO 'backend:error, checkpointer:debug3, fatal, archiver:debug2, autovacuum:debug1, walsender:debug3'; +SHOW log_min_messages; + log_min_messages +------------------------------------------------------------------------------------------------- + backend:error, checkpointer:debug3, fatal, archiver:debug2, autovacuum:debug1, walsender:debug3 +(1 row) + +SET log_min_messages TO 'warning, autovacuum:debug1'; +SHOW log_min_messages; + log_min_messages +---------------------------- + warning, autovacuum:debug1 +(1 row) + +SET log_min_messages TO 'autovacuum:debug1, warning'; +SHOW log_min_messages; + log_min_messages +---------------------------- + autovacuum:debug1, warning +(1 row) + diff --git a/src/test/regress/sql/guc.sql b/src/test/regress/sql/guc.sql index bafaf067e82..f7dfc909b42 100644 --- a/src/test/regress/sql/guc.sql +++ b/src/test/regress/sql/guc.sql @@ -377,3 +377,21 @@ SELECT name FROM tab_settings_flags WHERE no_reset AND NOT no_reset_all ORDER BY 1; DROP TABLE tab_settings_flags; + +-- Test log_min_messages +SET log_min_messages TO foo; -- fail +SET log_min_messages TO fatal; +SHOW log_min_messages; +SET log_min_messages TO 'fatal'; +SHOW log_min_messages; +SET log_min_messages TO 'checkpointer:debug2, autovacuum:debug1'; --fail +SET log_min_messages TO 'debug1, backend:error, fatal'; -- fail +SET log_min_messages TO 'backend:error, debug1, backend:warning'; -- fail +SET log_min_messages TO 'backend:error, foo:fatal, archiver:debug1'; -- fail +SET log_min_messages TO 'backend:error, checkpointer:bar, archiver:debug1'; -- fail +SET log_min_messages TO 'backend:error, checkpointer:debug3, fatal, archiver:debug2, autovacuum:debug1, walsender:debug3'; +SHOW log_min_messages; +SET log_min_messages TO 'warning, autovacuum:debug1'; +SHOW log_min_messages; +SET log_min_messages TO 'autovacuum:debug1, warning'; +SHOW log_min_messages; -- 2.47.3 --grg2fqqi6tkdt5me Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename="v7-0002-Assign-backend-type-earlier.patch" ^ permalink raw reply [nested|flat] 23+ messages in thread
end of thread, other threads:[~2025-11-04 20:18 UTC | newest] Thread overview: 23+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2020-03-10 02:56 [PATCH v19 09/10] pg_ls_*dir to return all the metadata from pg_stat_file.. Justin Pryzby <pryzbyj@telsasoft.com> 2020-03-10 02:56 [PATCH v24 08/11] pg_ls_*dir to return all the metadata from pg_stat_file.. Justin Pryzby <pryzbyj@telsasoft.com> 2020-03-10 02:56 [PATCH v25 08/11] pg_ls_*dir to return all the metadata from pg_stat_file.. Justin Pryzby <pryzbyj@telsasoft.com> 2020-03-10 02:56 [PATCH v22 09/10] pg_ls_*dir to return all the metadata from pg_stat_file.. Justin Pryzby <pryzbyj@telsasoft.com> 2020-03-10 02:56 [PATCH v9 11/11] pg_ls_*dir to return all the metadata from pg_stat_file.. Justin Pryzby <pryzbyj@telsasoft.com> 2020-03-10 02:56 [PATCH v37 07/11] pg_ls_*dir to return all the metadata from pg_stat_file.. Justin Pryzby <pryzbyj@telsasoft.com> 2020-03-10 02:56 [PATCH v30 07/11] pg_ls_*dir to return all the metadata from pg_stat_file.. Justin Pryzby <pryzbyj@telsasoft.com> 2020-03-10 02:56 [PATCH v26 08/11] pg_ls_*dir to return all the metadata from pg_stat_file.. Justin Pryzby <pryzbyj@telsasoft.com> 2020-03-10 02:56 [PATCH v36 7/7] pg_ls_*dir to return all the metadata from pg_stat_file.. Justin Pryzby <pryzbyj@telsasoft.com> 2020-03-10 02:56 [PATCH v20 09/10] pg_ls_*dir to return all the metadata from pg_stat_file.. Justin Pryzby <pryzbyj@telsasoft.com> 2020-03-10 02:56 [PATCH v32 07/11] pg_ls_*dir to return all the metadata from pg_stat_file.. Justin Pryzby <pryzbyj@telsasoft.com> 2020-03-10 02:56 [PATCH v33 07/11] pg_ls_*dir to return all the metadata from pg_stat_file.. Justin Pryzby <pryzbyj@telsasoft.com> 2020-03-10 02:56 [PATCH v28 07/11] pg_ls_*dir to return all the metadata from pg_stat_file.. Justin Pryzby <pryzbyj@telsasoft.com> 2020-03-10 02:56 [PATCH v15 09/10] pg_ls_*dir to return all the metadata from pg_stat_file.. Justin Pryzby <pryzbyj@telsasoft.com> 2020-03-10 02:56 [PATCH v35 7/7] pg_ls_*dir to return all the metadata from pg_stat_file.. Justin Pryzby <pryzbyj@telsasoft.com> 2020-03-10 02:56 [PATCH v31 07/11] pg_ls_*dir to return all the metadata from pg_stat_file.. Justin Pryzby <pryzbyj@telsasoft.com> 2020-03-10 02:56 [PATCH v30 07/11] pg_ls_*dir to return all the metadata from pg_stat_file.. Justin Pryzby <pryzbyj@telsasoft.com> 2020-03-10 02:56 [PATCH v34 07/15] pg_ls_*dir to return all the metadata from pg_stat_file.. Justin Pryzby <pryzbyj@telsasoft.com> 2020-03-10 02:56 [PATCH v31 07/11] pg_ls_*dir to return all the metadata from pg_stat_file.. Justin Pryzby <pryzbyj@telsasoft.com> 2020-03-10 02:56 [PATCH v21 09/10] pg_ls_*dir to return all the metadata from pg_stat_file.. Justin Pryzby <pryzbyj@telsasoft.com> 2020-03-10 02:56 [PATCH v17 09/10] pg_ls_*dir to return all the metadata from pg_stat_file.. Justin Pryzby <pryzbyj@telsasoft.com> 2020-03-10 02:56 [PATCH v9 11/11] pg_ls_*dir to return all the metadata from pg_stat_file.. Justin Pryzby <pryzbyj@telsasoft.com> 2025-11-04 20:18 [PATCH v7 1/3] log_min_messages per process type Euler Taveira <euler@eulerto.com>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox