public inbox for [email protected]
help / color / mirror / Atom feed[PATCH v14 6/8] Add pg_ls_dir_recurse to show dir recursively..
54+ messages / 2 participants
[nested] [flat]
* [PATCH v14 6/8] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
XXX: this test case is unstable, if backends/autovacuum remove FSM...
ERROR: could not stat file "./base/16384/20345_fsm": No such file or directory
CONTEXT: SQL function "pg_ls_dir_recurse" statement 1
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 25 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 +++++
src/test/regress/expected/misc_functions.out | 14 +++++++++++
src/test/regress/sql/misc_functions.sql | 6 +++++
5 files changed, 52 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 4cdd03610a..b30ac9f10d 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21351,6 +21351,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
For each file in a directory, list the file and its metadata. Restricted to superusers by default, but other users can be granted EXECUTE to run the function.
</entry>
</row>
+ <row>
+ <entry>
+ <literal><function>pg_ls_dir_recurse(<parameter>dirname</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry><type>setof text</type></entry>
+ <entry>
+ Call pg_ls_dir_metadata to recursively list the files in the specified directory, along with each file's metadata.
+ Restricted to superusers by default, but other users can be granted EXECUTE to run the function.
+ </entry>
+ </row>
<row>
<entry>
<literal><function>pg_ls_logdir()</function></literal>
@@ -21462,6 +21472,21 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
directory along with the file's metadata.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+lateral format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+lateral pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 05a644a7c9..e7295d8aaf 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1436,6 +1436,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 2c5829bca4..af2c2dd621 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10753,6 +10753,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--Kynn+LdAwU9N+JqL
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v14-0007-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v12 09/11] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
TODO:
src/backend/catalog/system_views.sql:REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 24 ++++++++++++++++++++
src/include/catalog/pg_proc.dat | 6 +++++
src/test/regress/expected/misc_functions.out | 14 ++++++++++++
src/test/regress/sql/misc_functions.sql | 6 +++++
4 files changed, 50 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 4cdd03610a..373e3b647b 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21351,6 +21351,15 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
For each file in a directory, list the file and its metadata. Restricted to superusers by default, but other users can be granted EXECUTE to run the function.
</entry>
</row>
+ <row>
+ <entry>
+ <literal><function>pg_ls_dir_recurse(<parameter>dirname</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry><type>setof text</type></entry>
+ <entry>
+ Call pg_ls_dir_metadata to recursively list the files in the specified directory, along with each file's metadata.
+ </entry>
+ </row>
<row>
<entry>
<literal><function>pg_ls_logdir()</function></literal>
@@ -21462,6 +21471,21 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
directory along with the file's metadata.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+format('/PG_%s_%s', left(current_setting('server_version_num'), 2), catalog_version_no) AS suffix) AS dir,
+pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b4c3d15495..667ea5721e 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10748,6 +10748,12 @@
proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}',
proargnames => '{dirname,missing_ok,include_dot_dirs,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata' },
+{ oid => '8511', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 448901d841..82d967a62d 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -187,6 +187,20 @@ select * from pg_ls_tmpdir() where name='Does not exist';
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 656aad93e6..2cfdac386d 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -55,6 +55,12 @@ select count(*) >= 0 as ok from pg_ls_archive_statusdir();
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--wIc/V6YLA2QdyfT4
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v12-0010-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v13 6/8] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
XXX: this test case is unstable, if backends/autovacuum remove FSM...
ERROR: could not stat file "./base/16384/20345_fsm": No such file or directory
CONTEXT: SQL function "pg_ls_dir_recurse" statement 1
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 25 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 +++++
src/test/regress/expected/misc_functions.out | 14 +++++++++++
src/test/regress/sql/misc_functions.sql | 6 +++++
5 files changed, 52 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 4cdd03610a..0763db3f50 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21351,6 +21351,16 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
For each file in a directory, list the file and its metadata. Restricted to superusers by default, but other users can be granted EXECUTE to run the function.
</entry>
</row>
+ <row>
+ <entry>
+ <literal><function>pg_ls_dir_recurse(<parameter>dirname</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry><type>setof text</type></entry>
+ <entry>
+ Call pg_ls_dir_metadata to recursively list the files in the specified directory, along with each file's metadata.
+ Restricted to superusers by default, but other users can be granted EXECUTE to run the function.
+ </entry>
+ </row>
<row>
<entry>
<literal><function>pg_ls_logdir()</function></literal>
@@ -21462,6 +21472,21 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
directory along with the file's metadata.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+format('/PG_%s_%s', left(current_setting('server_version_num'), 2), catalog_version_no) AS suffix) AS dir,
+pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 05a644a7c9..e7295d8aaf 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1436,6 +1436,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index be3e820a03..b3509f0e12 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10747,6 +10747,12 @@
proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}',
proargnames => '{dirname,missing_ok,include_dot_dirs,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata' },
+{ oid => '8511', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 3ca4b85dd9..ae8716a83f 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -208,6 +208,20 @@ select * from pg_ls_tmpdir() where name='Does not exist';
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index f09c890b5d..7038bcdb0f 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -64,6 +64,12 @@ select count(*) > 0 from
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--E0h0CbphJD8hN+Gf
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v13-0007-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v22 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/bin/pg_rewind/libpq_fetch.c | 22 +++-----------
src/bin/pg_rewind/t/RewindTest.pm | 7 ++++-
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 13 ++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
7 files changed, 69 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index f7ccf68ff4..7baf2b31e5 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25769,6 +25769,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 169602c1b6..55b700dadc 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1510,6 +1510,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/bin/pg_rewind/libpq_fetch.c b/src/bin/pg_rewind/libpq_fetch.c
index bf4dfc23b9..a9535d3dd5 100644
--- a/src/bin/pg_rewind/libpq_fetch.c
+++ b/src/bin/pg_rewind/libpq_fetch.c
@@ -176,30 +176,18 @@ libpqProcessFileList(void)
/*
* Create a recursive directory listing of the whole data directory.
*
- * The WITH RECURSIVE part does most of the work. The second part gets the
- * targets of the symlinks in pg_tblspc directory.
+ * Join to pg_tablespace to get the targets of the symlinks in
+ * pg_tblspc directory.
*
* XXX: There is no backend function to get a symbolic link's target in
* general, so if the admin has put any custom symbolic links in the data
* directory, they won't be copied correctly.
*/
sql =
- "WITH RECURSIVE files (path, filename, size, isdir) AS (\n"
- " SELECT '' AS path, filename, size, isdir FROM\n"
- " (SELECT pg_ls_dir('.', true, false) AS filename) AS fn,\n"
- " pg_stat_file(fn.filename, true) AS this\n"
- " UNION ALL\n"
- " SELECT parent.path || parent.filename || '/' AS path,\n"
- " fn, this.size, this.isdir\n"
- " FROM files AS parent,\n"
- " pg_ls_dir(parent.path || parent.filename, true, false) AS fn,\n"
- " pg_stat_file(parent.path || parent.filename || '/' || fn, true) AS this\n"
- " WHERE parent.isdir = 't'\n"
- ")\n"
- "SELECT path || filename, size, isdir,\n"
+ "SELECT COALESCE(NULLIF(path,'.')||'/','')||filename, size, isdir,\n"
" pg_tablespace_location(pg_tablespace.oid) AS link_target\n"
- "FROM files\n"
- "LEFT OUTER JOIN pg_tablespace ON files.path = 'pg_tblspc/'\n"
+ "FROM pg_ls_dir_recurse('.') files\n"
+ "LEFT OUTER JOIN pg_tablespace ON files.path = 'pg_tblspc'\n"
" AND oid::text = files.filename\n";
res = PQexec(conn, sql);
diff --git a/src/bin/pg_rewind/t/RewindTest.pm b/src/bin/pg_rewind/t/RewindTest.pm
index 41ed7d4b3b..cacd58651c 100644
--- a/src/bin/pg_rewind/t/RewindTest.pm
+++ b/src/bin/pg_rewind/t/RewindTest.pm
@@ -160,7 +160,12 @@ sub start_primary
GRANT EXECUTE ON function pg_catalog.pg_read_binary_file(text)
TO rewind_user;
GRANT EXECUTE ON function pg_catalog.pg_read_binary_file(text, bigint, bigint, boolean)
- TO rewind_user;");
+ TO rewind_user;
+ GRANT EXECUTE ON function pg_catalog.pg_ls_dir_metadata(text, bool, bool)
+ TO rewind_user;
+ GRANT EXECUTE ON function pg_catalog.pg_ls_dir_recurse(text)
+ TO rewind_user;
+ ");
#### Now run the test-specific parts to initialize the primary before setting
# up standby
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index c3564168e5..0235042e26 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10976,6 +10976,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,filename,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '9981', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,path,filename,size,modification,isdir}', proargmodes => '{i,o,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select dirname as path, * from pg_ls_dir_metadata(dirname, false, false) union all select coalesce(nullif(parent.path,'.')||'/','')||parent.filename, a.filename, a.size, a.modification, a.isdir from ls as parent, lateral pg_ls_dir_metadata(parent.path||'/'||parent.filename, false, false) as a where parent.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index bb926a2cf4..302f348a6d 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,19 @@ select * from pg_ls_dir_metadata('.') limit 0;
----------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+select path, filename, isdir from pg_ls_dir_recurse('.') where isdir and path='pg_wal';
+ path | filename | isdir
+--------+----------------+-------
+ pg_wal | archive_status | t
+(1 row)
+
+-- Check that expected columns are present
+select * from pg_ls_dir_recurse('.') limit 0;
+ path | filename | size | modification | isdir
+------+----------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 68e2bc6586..fda95828d7 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select filename, isdir from pg_ls_dir_metadata('.', false, false) where filename
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+select path, filename, isdir from pg_ls_dir_recurse('.') where isdir and path='pg_wal';
+
+-- Check that expected columns are present
+select * from pg_ls_dir_recurse('.') limit 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--d6Gm4EdcadzBjdND
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v22-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v23 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/bin/pg_rewind/libpq_source.c | 22 +++-----------
src/bin/pg_rewind/t/RewindTest.pm | 7 ++++-
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 13 ++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
7 files changed, 69 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 729d201249..c95dab6e1d 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25760,6 +25760,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index e7647787cf..148dd7a8a8 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1504,6 +1504,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/bin/pg_rewind/libpq_source.c b/src/bin/pg_rewind/libpq_source.c
index c73e8bf470..46c8b7fcd5 100644
--- a/src/bin/pg_rewind/libpq_source.c
+++ b/src/bin/pg_rewind/libpq_source.c
@@ -205,30 +205,18 @@ libpq_traverse_files(rewind_source *source, process_file_callback_t callback)
/*
* Create a recursive directory listing of the whole data directory.
*
- * The WITH RECURSIVE part does most of the work. The second part gets the
- * targets of the symlinks in pg_tblspc directory.
+ * Join to pg_tablespace to get the targets of the symlinks in
+ * pg_tblspc directory.
*
* XXX: There is no backend function to get a symbolic link's target in
* general, so if the admin has put any custom symbolic links in the data
* directory, they won't be copied correctly.
*/
sql =
- "WITH RECURSIVE files (path, filename, size, isdir) AS (\n"
- " SELECT '' AS path, filename, size, isdir FROM\n"
- " (SELECT pg_ls_dir('.', true, false) AS filename) AS fn,\n"
- " pg_stat_file(fn.filename, true) AS this\n"
- " UNION ALL\n"
- " SELECT parent.path || parent.filename || '/' AS path,\n"
- " fn, this.size, this.isdir\n"
- " FROM files AS parent,\n"
- " pg_ls_dir(parent.path || parent.filename, true, false) AS fn,\n"
- " pg_stat_file(parent.path || parent.filename || '/' || fn, true) AS this\n"
- " WHERE parent.isdir = 't'\n"
- ")\n"
- "SELECT path || filename, size, isdir,\n"
+ "SELECT COALESCE(NULLIF(path,'.')||'/','')||filename, size, isdir,\n"
" pg_tablespace_location(pg_tablespace.oid) AS link_target\n"
- "FROM files\n"
- "LEFT OUTER JOIN pg_tablespace ON files.path = 'pg_tblspc/'\n"
+ "FROM pg_ls_dir_recurse('.') files\n"
+ "LEFT OUTER JOIN pg_tablespace ON files.path = 'pg_tblspc'\n"
" AND oid::text = files.filename\n";
res = PQexec(conn, sql);
diff --git a/src/bin/pg_rewind/t/RewindTest.pm b/src/bin/pg_rewind/t/RewindTest.pm
index 41ed7d4b3b..cacd58651c 100644
--- a/src/bin/pg_rewind/t/RewindTest.pm
+++ b/src/bin/pg_rewind/t/RewindTest.pm
@@ -160,7 +160,12 @@ sub start_primary
GRANT EXECUTE ON function pg_catalog.pg_read_binary_file(text)
TO rewind_user;
GRANT EXECUTE ON function pg_catalog.pg_read_binary_file(text, bigint, bigint, boolean)
- TO rewind_user;");
+ TO rewind_user;
+ GRANT EXECUTE ON function pg_catalog.pg_ls_dir_metadata(text, bool, bool)
+ TO rewind_user;
+ GRANT EXECUTE ON function pg_catalog.pg_ls_dir_recurse(text)
+ TO rewind_user;
+ ");
#### Now run the test-specific parts to initialize the primary before setting
# up standby
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 344d6305e5..5e8f16d5b1 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10983,6 +10983,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,filename,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '9981', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,path,filename,size,modification,isdir}', proargmodes => '{i,o,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select dirname as path, * from pg_ls_dir_metadata(dirname, false, false) union all select coalesce(nullif(parent.path,'.')||'/','')||parent.filename, a.filename, a.size, a.modification, a.isdir from ls as parent, lateral pg_ls_dir_metadata(parent.path||'/'||parent.filename, false, false) as a where parent.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index bb926a2cf4..302f348a6d 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,19 @@ select * from pg_ls_dir_metadata('.') limit 0;
----------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+select path, filename, isdir from pg_ls_dir_recurse('.') where isdir and path='pg_wal';
+ path | filename | isdir
+--------+----------------+-------
+ pg_wal | archive_status | t
+(1 row)
+
+-- Check that expected columns are present
+select * from pg_ls_dir_recurse('.') limit 0;
+ path | filename | size | modification | isdir
+------+----------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 68e2bc6586..fda95828d7 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select filename, isdir from pg_ls_dir_metadata('.', false, false) where filename
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+select path, filename, isdir from pg_ls_dir_recurse('.') where isdir and path='pg_wal';
+
+-- Check that expected columns are present
+select * from pg_ls_dir_recurse('.') limit 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--mhjHhnbe5PrRcwjY
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v23-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v19 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 14 +++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
5 files changed, 59 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index d1d2f868d8..c097977d9b 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25733,6 +25733,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index f5217a0d93..297b0dcdb8 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1482,6 +1482,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b61f30247e..bac1226db7 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10914,6 +10914,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '9981', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false) as a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--S0GG+JvAI2G0KxBG
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v19-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 27 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 +++++
src/test/regress/expected/misc_functions.out | 14 ++++++++++
src/test/regress/sql/misc_functions.sql | 6 +++++
5 files changed, 54 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c3a6b3277f..3a2ad3eb50 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25311,6 +25311,18 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
EXECUTE to run the function.
</entry>
</row>
+ <row>
+ <entry>
+ <literal><function>pg_ls_dir_recurse(<parameter>dirname</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry><type>setof text</type></entry>
+ <entry>
+ Call pg_ls_dir_metadata to recursively list the files in the specified
+ directory, along with each file's metadata.
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </entry>
+ </row>
<row>
<entry>
<literal><function>pg_ls_logdir()</function></literal>
@@ -25422,6 +25434,21 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
directory along with the file's metadata.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 14 +++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
5 files changed, 59 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8e0de7c02d..334b6beb3a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25738,6 +25738,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 27 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 +++++
src/test/regress/expected/misc_functions.out | 14 ++++++++++
src/test/regress/sql/misc_functions.sql | 6 +++++
5 files changed, 54 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c3a6b3277f..3a2ad3eb50 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25311,6 +25311,18 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
EXECUTE to run the function.
</entry>
</row>
+ <row>
+ <entry>
+ <literal><function>pg_ls_dir_recurse(<parameter>dirname</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry><type>setof text</type></entry>
+ <entry>
+ Call pg_ls_dir_metadata to recursively list the files in the specified
+ directory, along with each file's metadata.
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </entry>
+ </row>
<row>
<entry>
<literal><function>pg_ls_logdir()</function></literal>
@@ -25422,6 +25434,21 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
directory along with the file's metadata.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 14 +++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
5 files changed, 59 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8e0de7c02d..334b6beb3a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25738,6 +25738,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 27 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 +++++
src/test/regress/expected/misc_functions.out | 14 ++++++++++
src/test/regress/sql/misc_functions.sql | 6 +++++
5 files changed, 54 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c3a6b3277f..3a2ad3eb50 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25311,6 +25311,18 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
EXECUTE to run the function.
</entry>
</row>
+ <row>
+ <entry>
+ <literal><function>pg_ls_dir_recurse(<parameter>dirname</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry><type>setof text</type></entry>
+ <entry>
+ Call pg_ls_dir_metadata to recursively list the files in the specified
+ directory, along with each file's metadata.
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </entry>
+ </row>
<row>
<entry>
<literal><function>pg_ls_logdir()</function></literal>
@@ -25422,6 +25434,21 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
directory along with the file's metadata.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 14 +++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
5 files changed, 59 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8e0de7c02d..334b6beb3a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25738,6 +25738,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 27 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 +++++
src/test/regress/expected/misc_functions.out | 14 ++++++++++
src/test/regress/sql/misc_functions.sql | 6 +++++
5 files changed, 54 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c3a6b3277f..3a2ad3eb50 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25311,6 +25311,18 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
EXECUTE to run the function.
</entry>
</row>
+ <row>
+ <entry>
+ <literal><function>pg_ls_dir_recurse(<parameter>dirname</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry><type>setof text</type></entry>
+ <entry>
+ Call pg_ls_dir_metadata to recursively list the files in the specified
+ directory, along with each file's metadata.
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </entry>
+ </row>
<row>
<entry>
<literal><function>pg_ls_logdir()</function></literal>
@@ -25422,6 +25434,21 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
directory along with the file's metadata.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 14 +++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
5 files changed, 59 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8e0de7c02d..334b6beb3a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25738,6 +25738,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 27 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 +++++
src/test/regress/expected/misc_functions.out | 14 ++++++++++
src/test/regress/sql/misc_functions.sql | 6 +++++
5 files changed, 54 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c3a6b3277f..3a2ad3eb50 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25311,6 +25311,18 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
EXECUTE to run the function.
</entry>
</row>
+ <row>
+ <entry>
+ <literal><function>pg_ls_dir_recurse(<parameter>dirname</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry><type>setof text</type></entry>
+ <entry>
+ Call pg_ls_dir_metadata to recursively list the files in the specified
+ directory, along with each file's metadata.
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </entry>
+ </row>
<row>
<entry>
<literal><function>pg_ls_logdir()</function></literal>
@@ -25422,6 +25434,21 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
directory along with the file's metadata.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 14 +++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
5 files changed, 59 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8e0de7c02d..334b6beb3a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25738,6 +25738,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 27 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 +++++
src/test/regress/expected/misc_functions.out | 14 ++++++++++
src/test/regress/sql/misc_functions.sql | 6 +++++
5 files changed, 54 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c3a6b3277f..3a2ad3eb50 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25311,6 +25311,18 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
EXECUTE to run the function.
</entry>
</row>
+ <row>
+ <entry>
+ <literal><function>pg_ls_dir_recurse(<parameter>dirname</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry><type>setof text</type></entry>
+ <entry>
+ Call pg_ls_dir_metadata to recursively list the files in the specified
+ directory, along with each file's metadata.
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </entry>
+ </row>
<row>
<entry>
<literal><function>pg_ls_logdir()</function></literal>
@@ -25422,6 +25434,21 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
directory along with the file's metadata.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 14 +++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
5 files changed, 59 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8e0de7c02d..334b6beb3a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25738,6 +25738,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 27 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 +++++
src/test/regress/expected/misc_functions.out | 14 ++++++++++
src/test/regress/sql/misc_functions.sql | 6 +++++
5 files changed, 54 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c3a6b3277f..3a2ad3eb50 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25311,6 +25311,18 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
EXECUTE to run the function.
</entry>
</row>
+ <row>
+ <entry>
+ <literal><function>pg_ls_dir_recurse(<parameter>dirname</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry><type>setof text</type></entry>
+ <entry>
+ Call pg_ls_dir_metadata to recursively list the files in the specified
+ directory, along with each file's metadata.
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </entry>
+ </row>
<row>
<entry>
<literal><function>pg_ls_logdir()</function></literal>
@@ -25422,6 +25434,21 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
directory along with the file's metadata.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 14 +++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
5 files changed, 59 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8e0de7c02d..334b6beb3a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25738,6 +25738,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 27 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 +++++
src/test/regress/expected/misc_functions.out | 14 ++++++++++
src/test/regress/sql/misc_functions.sql | 6 +++++
5 files changed, 54 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c3a6b3277f..3a2ad3eb50 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25311,6 +25311,18 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
EXECUTE to run the function.
</entry>
</row>
+ <row>
+ <entry>
+ <literal><function>pg_ls_dir_recurse(<parameter>dirname</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry><type>setof text</type></entry>
+ <entry>
+ Call pg_ls_dir_metadata to recursively list the files in the specified
+ directory, along with each file's metadata.
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </entry>
+ </row>
<row>
<entry>
<literal><function>pg_ls_logdir()</function></literal>
@@ -25422,6 +25434,21 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
directory along with the file's metadata.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 14 +++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
5 files changed, 59 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8e0de7c02d..334b6beb3a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25738,6 +25738,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 27 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 +++++
src/test/regress/expected/misc_functions.out | 14 ++++++++++
src/test/regress/sql/misc_functions.sql | 6 +++++
5 files changed, 54 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c3a6b3277f..3a2ad3eb50 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25311,6 +25311,18 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
EXECUTE to run the function.
</entry>
</row>
+ <row>
+ <entry>
+ <literal><function>pg_ls_dir_recurse(<parameter>dirname</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry><type>setof text</type></entry>
+ <entry>
+ Call pg_ls_dir_metadata to recursively list the files in the specified
+ directory, along with each file's metadata.
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </entry>
+ </row>
<row>
<entry>
<literal><function>pg_ls_logdir()</function></literal>
@@ -25422,6 +25434,21 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
directory along with the file's metadata.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 14 +++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
5 files changed, 59 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8e0de7c02d..334b6beb3a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25738,6 +25738,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 27 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 +++++
src/test/regress/expected/misc_functions.out | 14 ++++++++++
src/test/regress/sql/misc_functions.sql | 6 +++++
5 files changed, 54 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c3a6b3277f..3a2ad3eb50 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25311,6 +25311,18 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
EXECUTE to run the function.
</entry>
</row>
+ <row>
+ <entry>
+ <literal><function>pg_ls_dir_recurse(<parameter>dirname</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry><type>setof text</type></entry>
+ <entry>
+ Call pg_ls_dir_metadata to recursively list the files in the specified
+ directory, along with each file's metadata.
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </entry>
+ </row>
<row>
<entry>
<literal><function>pg_ls_logdir()</function></literal>
@@ -25422,6 +25434,21 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
directory along with the file's metadata.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 14 +++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
5 files changed, 59 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8e0de7c02d..334b6beb3a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25738,6 +25738,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 27 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 +++++
src/test/regress/expected/misc_functions.out | 14 ++++++++++
src/test/regress/sql/misc_functions.sql | 6 +++++
5 files changed, 54 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c3a6b3277f..3a2ad3eb50 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25311,6 +25311,18 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
EXECUTE to run the function.
</entry>
</row>
+ <row>
+ <entry>
+ <literal><function>pg_ls_dir_recurse(<parameter>dirname</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry><type>setof text</type></entry>
+ <entry>
+ Call pg_ls_dir_metadata to recursively list the files in the specified
+ directory, along with each file's metadata.
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </entry>
+ </row>
<row>
<entry>
<literal><function>pg_ls_logdir()</function></literal>
@@ -25422,6 +25434,21 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
directory along with the file's metadata.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 14 +++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
5 files changed, 59 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8e0de7c02d..334b6beb3a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25738,6 +25738,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 27 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 +++++
src/test/regress/expected/misc_functions.out | 14 ++++++++++
src/test/regress/sql/misc_functions.sql | 6 +++++
5 files changed, 54 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c3a6b3277f..3a2ad3eb50 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25311,6 +25311,18 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
EXECUTE to run the function.
</entry>
</row>
+ <row>
+ <entry>
+ <literal><function>pg_ls_dir_recurse(<parameter>dirname</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry><type>setof text</type></entry>
+ <entry>
+ Call pg_ls_dir_metadata to recursively list the files in the specified
+ directory, along with each file's metadata.
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </entry>
+ </row>
<row>
<entry>
<literal><function>pg_ls_logdir()</function></literal>
@@ -25422,6 +25434,21 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
directory along with the file's metadata.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 14 +++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
5 files changed, 59 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8e0de7c02d..334b6beb3a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25738,6 +25738,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 27 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 +++++
src/test/regress/expected/misc_functions.out | 14 ++++++++++
src/test/regress/sql/misc_functions.sql | 6 +++++
5 files changed, 54 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c3a6b3277f..3a2ad3eb50 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25311,6 +25311,18 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
EXECUTE to run the function.
</entry>
</row>
+ <row>
+ <entry>
+ <literal><function>pg_ls_dir_recurse(<parameter>dirname</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry><type>setof text</type></entry>
+ <entry>
+ Call pg_ls_dir_metadata to recursively list the files in the specified
+ directory, along with each file's metadata.
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </entry>
+ </row>
<row>
<entry>
<literal><function>pg_ls_logdir()</function></literal>
@@ -25422,6 +25434,21 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
directory along with the file's metadata.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 14 +++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
5 files changed, 59 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8e0de7c02d..334b6beb3a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25738,6 +25738,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 27 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 +++++
src/test/regress/expected/misc_functions.out | 14 ++++++++++
src/test/regress/sql/misc_functions.sql | 6 +++++
5 files changed, 54 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c3a6b3277f..3a2ad3eb50 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25311,6 +25311,18 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
EXECUTE to run the function.
</entry>
</row>
+ <row>
+ <entry>
+ <literal><function>pg_ls_dir_recurse(<parameter>dirname</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry><type>setof text</type></entry>
+ <entry>
+ Call pg_ls_dir_metadata to recursively list the files in the specified
+ directory, along with each file's metadata.
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </entry>
+ </row>
<row>
<entry>
<literal><function>pg_ls_logdir()</function></literal>
@@ -25422,6 +25434,21 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
directory along with the file's metadata.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 14 +++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
5 files changed, 59 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8e0de7c02d..334b6beb3a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25738,6 +25738,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 27 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 +++++
src/test/regress/expected/misc_functions.out | 14 ++++++++++
src/test/regress/sql/misc_functions.sql | 6 +++++
5 files changed, 54 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c3a6b3277f..3a2ad3eb50 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25311,6 +25311,18 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
EXECUTE to run the function.
</entry>
</row>
+ <row>
+ <entry>
+ <literal><function>pg_ls_dir_recurse(<parameter>dirname</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry><type>setof text</type></entry>
+ <entry>
+ Call pg_ls_dir_metadata to recursively list the files in the specified
+ directory, along with each file's metadata.
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </entry>
+ </row>
<row>
<entry>
<literal><function>pg_ls_logdir()</function></literal>
@@ -25422,6 +25434,21 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
directory along with the file's metadata.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 14 +++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
5 files changed, 59 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8e0de7c02d..334b6beb3a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25738,6 +25738,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 27 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 +++++
src/test/regress/expected/misc_functions.out | 14 ++++++++++
src/test/regress/sql/misc_functions.sql | 6 +++++
5 files changed, 54 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c3a6b3277f..3a2ad3eb50 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25311,6 +25311,18 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
EXECUTE to run the function.
</entry>
</row>
+ <row>
+ <entry>
+ <literal><function>pg_ls_dir_recurse(<parameter>dirname</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry><type>setof text</type></entry>
+ <entry>
+ Call pg_ls_dir_metadata to recursively list the files in the specified
+ directory, along with each file's metadata.
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </entry>
+ </row>
<row>
<entry>
<literal><function>pg_ls_logdir()</function></literal>
@@ -25422,6 +25434,21 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
directory along with the file's metadata.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 14 +++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
5 files changed, 59 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8e0de7c02d..334b6beb3a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25738,6 +25738,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 27 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 +++++
src/test/regress/expected/misc_functions.out | 14 ++++++++++
src/test/regress/sql/misc_functions.sql | 6 +++++
5 files changed, 54 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c3a6b3277f..3a2ad3eb50 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25311,6 +25311,18 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
EXECUTE to run the function.
</entry>
</row>
+ <row>
+ <entry>
+ <literal><function>pg_ls_dir_recurse(<parameter>dirname</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry><type>setof text</type></entry>
+ <entry>
+ Call pg_ls_dir_metadata to recursively list the files in the specified
+ directory, along with each file's metadata.
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </entry>
+ </row>
<row>
<entry>
<literal><function>pg_ls_logdir()</function></literal>
@@ -25422,6 +25434,21 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
directory along with the file's metadata.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 14 +++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
5 files changed, 59 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8e0de7c02d..334b6beb3a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25738,6 +25738,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 27 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 +++++
src/test/regress/expected/misc_functions.out | 14 ++++++++++
src/test/regress/sql/misc_functions.sql | 6 +++++
5 files changed, 54 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c3a6b3277f..3a2ad3eb50 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25311,6 +25311,18 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
EXECUTE to run the function.
</entry>
</row>
+ <row>
+ <entry>
+ <literal><function>pg_ls_dir_recurse(<parameter>dirname</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry><type>setof text</type></entry>
+ <entry>
+ Call pg_ls_dir_metadata to recursively list the files in the specified
+ directory, along with each file's metadata.
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </entry>
+ </row>
<row>
<entry>
<literal><function>pg_ls_logdir()</function></literal>
@@ -25422,6 +25434,21 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
directory along with the file's metadata.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 14 +++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
5 files changed, 59 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8e0de7c02d..334b6beb3a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25738,6 +25738,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 27 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 +++++
src/test/regress/expected/misc_functions.out | 14 ++++++++++
src/test/regress/sql/misc_functions.sql | 6 +++++
5 files changed, 54 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c3a6b3277f..3a2ad3eb50 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25311,6 +25311,18 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
EXECUTE to run the function.
</entry>
</row>
+ <row>
+ <entry>
+ <literal><function>pg_ls_dir_recurse(<parameter>dirname</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry><type>setof text</type></entry>
+ <entry>
+ Call pg_ls_dir_metadata to recursively list the files in the specified
+ directory, along with each file's metadata.
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </entry>
+ </row>
<row>
<entry>
<literal><function>pg_ls_logdir()</function></literal>
@@ -25422,6 +25434,21 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
directory along with the file's metadata.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 14 +++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
5 files changed, 59 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8e0de7c02d..334b6beb3a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25738,6 +25738,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 27 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 +++++
src/test/regress/expected/misc_functions.out | 14 ++++++++++
src/test/regress/sql/misc_functions.sql | 6 +++++
5 files changed, 54 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index c3a6b3277f..3a2ad3eb50 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25311,6 +25311,18 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
EXECUTE to run the function.
</entry>
</row>
+ <row>
+ <entry>
+ <literal><function>pg_ls_dir_recurse(<parameter>dirname</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry><type>setof text</type></entry>
+ <entry>
+ Call pg_ls_dir_metadata to recursively list the files in the specified
+ directory, along with each file's metadata.
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </entry>
+ </row>
<row>
<entry>
<literal><function>pg_ls_logdir()</function></literal>
@@ -25422,6 +25434,21 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
directory along with the file's metadata.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--2FkSFaIQeDFoAt0B
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 14 +++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
5 files changed, 59 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8e0de7c02d..334b6beb3a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25738,6 +25738,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 1c77430f0c..215fba3b7d 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1468,6 +1468,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index adfce45d1a..f94f403475 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10916,6 +10916,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '5034', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--4LFBTxd4L5NLO6ly
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v18 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 14 +++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
5 files changed, 59 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 4aa9cdb18c..683f533392 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25738,6 +25738,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index cc8e870740..58d535a9f6 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1469,6 +1469,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b61f30247e..bac1226db7 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10914,6 +10914,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '9981', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, lateral pg_ls_dir_metadata(dirname||'/'||ls.name, false, false) as a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..4188d684f0 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,20 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..6041c4f3dc 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--5I6of5zJg18YgZEa
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v18-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v20 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 13 ++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
5 files changed, 58 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9652409581..befd2ffca5 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25741,6 +25741,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 360b6bda26..44690be916 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1479,6 +1479,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 0b5716525c..e60b6ca866 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10931,6 +10931,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '9981', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,path,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select dirname as path, * from pg_ls_dir_metadata(dirname, true, false) union all select parent.path||'/'||parent.name, a.name, a.size, a.modification, a.isdir from ls AS parent, lateral pg_ls_dir_metadata(parent.path||'/'||parent.name, false, false) as a where parent.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 64b1417fb8..07c0a7f961 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,19 @@ select * from pg_ls_dir_metadata('.') limit 0;
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT path, name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND path='./pg_wal';
+ path | name | isdir
+----------+----------------+-------
+ ./pg_wal | archive_status | t
+(1 row)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ path | name | size | modification | isdir
+------+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 372345720d..349752da94 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select name, isdir from pg_ls_dir_metadata('.', false, false) where name='.'; --
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT path, name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND path='./pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--Z1Z8UV8BNhgCynIS
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v20-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v21 07/10] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 32 ++++++++++++++++++++
src/backend/catalog/system_views.sql | 1 +
src/bin/pg_rewind/libpq_fetch.c | 22 +++-----------
src/bin/pg_rewind/t/RewindTest.pm | 7 ++++-
src/include/catalog/pg_proc.dat | 6 ++++
src/test/regress/expected/misc_functions.out | 13 ++++++++
src/test/regress/sql/misc_functions.sql | 6 ++++
7 files changed, 69 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 9652409581..befd2ffca5 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25741,6 +25741,38 @@ SELECT pg_size_pretty(sum(pg_relation_size(relid))) AS total_size
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <function>pg_ls_dir_recurse</function> ( <parameter>dirname</parameter> <type>text</type> )
+ <returnvalue>setof record</returnvalue>
+ ( <parameter>name</parameter> <type>text</type>,
+ <parameter>size</parameter> <type>bigint</type>,
+ <parameter>modification</parameter> <type>timestamp with time zone</type>,
+ <parameter>isdir</parameter> <type>boolean</type> )
+ </para>
+ <para>
+ Recursively list each file in the specified directory, along with the
+ files' metadata.
+ </para>
+ <para>
+ Restricted to superusers by default, but other users can be granted
+ EXECUTE to run the function.
+ </para></entry>
+ </row>
+
+ <!--para>
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+LATERAL format('/PG_%s_%s', left(current_setting('server_version_num'), 2), pcs.catalog_version_no) AS suffix) AS dir,
+LATERAL pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para-->
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index b9b2a6aa20..cdf105ffac 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1479,6 +1479,7 @@ REVOKE EXECUTE ON FUNCTION pg_stat_file(text,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_ls_dir_metadata(text,boolean,boolean) FROM public;
+REVOKE EXECUTE ON FUNCTION pg_ls_dir_recurse(text) FROM public;
--
-- We also set up some things as accessible to standard roles.
diff --git a/src/bin/pg_rewind/libpq_fetch.c b/src/bin/pg_rewind/libpq_fetch.c
index c44648f823..58f2bb9fd1 100644
--- a/src/bin/pg_rewind/libpq_fetch.c
+++ b/src/bin/pg_rewind/libpq_fetch.c
@@ -176,30 +176,18 @@ libpqProcessFileList(void)
/*
* Create a recursive directory listing of the whole data directory.
*
- * The WITH RECURSIVE part does most of the work. The second part gets the
- * targets of the symlinks in pg_tblspc directory.
+ * Join to pg_tablespace to get the targets of the symlinks in
+ * pg_tblspc directory.
*
* XXX: There is no backend function to get a symbolic link's target in
* general, so if the admin has put any custom symbolic links in the data
* directory, they won't be copied correctly.
*/
sql =
- "WITH RECURSIVE files (path, filename, size, isdir) AS (\n"
- " SELECT '' AS path, filename, size, isdir FROM\n"
- " (SELECT pg_ls_dir('.', true, false) AS filename) AS fn,\n"
- " pg_stat_file(fn.filename, true) AS this\n"
- " UNION ALL\n"
- " SELECT parent.path || parent.filename || '/' AS path,\n"
- " fn, this.size, this.isdir\n"
- " FROM files AS parent,\n"
- " pg_ls_dir(parent.path || parent.filename, true, false) AS fn,\n"
- " pg_stat_file(parent.path || parent.filename || '/' || fn, true) AS this\n"
- " WHERE parent.isdir = 't'\n"
- ")\n"
- "SELECT path || filename, size, isdir,\n"
+ "SELECT COALESCE(NULLIF(path,'.')||'/','')||filename, size, isdir,\n"
" pg_tablespace_location(pg_tablespace.oid) AS link_target\n"
- "FROM files\n"
- "LEFT OUTER JOIN pg_tablespace ON files.path = 'pg_tblspc/'\n"
+ "FROM pg_ls_dir_recurse('.') files\n"
+ "LEFT OUTER JOIN pg_tablespace ON files.path = 'pg_tblspc'\n"
" AND oid::text = files.filename\n";
res = PQexec(conn, sql);
diff --git a/src/bin/pg_rewind/t/RewindTest.pm b/src/bin/pg_rewind/t/RewindTest.pm
index 7516af7a01..bc8dad074e 100644
--- a/src/bin/pg_rewind/t/RewindTest.pm
+++ b/src/bin/pg_rewind/t/RewindTest.pm
@@ -160,7 +160,12 @@ sub start_primary
GRANT EXECUTE ON function pg_catalog.pg_read_binary_file(text)
TO rewind_user;
GRANT EXECUTE ON function pg_catalog.pg_read_binary_file(text, bigint, bigint, boolean)
- TO rewind_user;");
+ TO rewind_user;
+ GRANT EXECUTE ON function pg_catalog.pg_ls_dir_metadata(text, bool, bool)
+ TO rewind_user;
+ GRANT EXECUTE ON function pg_catalog.pg_ls_dir_recurse(text)
+ TO rewind_user;
+ ");
#### Now run the test-specific parts to initialize the primary before setting
# up standby
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 58d1c74b52..826a31c561 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10931,6 +10931,12 @@
proallargtypes => '{text,text,int8,timestamptz,bool}', proargmodes => '{i,o,o,o,o}',
proargnames => '{dirname,filename,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata_1arg' },
+{ oid => '9981', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,path,filename,size,modification,isdir}', proargmodes => '{i,o,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select dirname as path, * from pg_ls_dir_metadata(dirname, false, false) union all select coalesce(nullif(parent.path,'.')||'/','')||parent.filename, a.filename, a.size, a.modification, a.isdir from ls as parent, lateral pg_ls_dir_metadata(parent.path||'/'||parent.filename, false, false) as a where parent.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index bb926a2cf4..302f348a6d 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -243,6 +243,19 @@ select * from pg_ls_dir_metadata('.') limit 0;
----------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+select path, filename, isdir from pg_ls_dir_recurse('.') where isdir and path='pg_wal';
+ path | filename | isdir
+--------+----------------+-------
+ pg_wal | archive_status | t
+(1 row)
+
+-- Check that expected columns are present
+select * from pg_ls_dir_recurse('.') limit 0;
+ path | filename | size | modification | isdir
+------+----------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 68e2bc6586..fda95828d7 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -76,6 +76,12 @@ select filename, isdir from pg_ls_dir_metadata('.', false, false) where filename
-- Check that expected columns are present
select * from pg_ls_dir_metadata('.') limit 0;
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+select path, filename, isdir from pg_ls_dir_recurse('.') where isdir and path='pg_wal';
+
+-- Check that expected columns are present
+select * from pg_ls_dir_recurse('.') limit 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--Tcb1KvpfnM4LxW2s
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v21-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v9 07/11] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
TODO:
src/backend/catalog/system_views.sql:REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 13 +++++++++++++
src/include/catalog/pg_proc.dat | 8 ++++++++
2 files changed, 21 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 7c73d3b82a..672cbab7b9 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21442,6 +21442,19 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
empty directory from an non-existent directory.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ <parameter>missing_ok</parameter> ???
+ To recusively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp')dir FROM pg_tablespace b, pg_control_system() pcs, format('/PG_%s_%s', left(current_setting('server_version_num'), 2), catalog_version_no) suffix)dir, pg_ls_dir_recurse(dir)a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 7789d029ea..cc2c6f6571 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6143,6 +6143,14 @@
proname => 'pg_ls_dir', prorows => '1000', proretset => 't',
provolatile => 'v', prorettype => 'text', proargtypes => 'text bool bool',
prosrc => 'pg_ls_dir' },
+
+{ oid => '8511', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "WITH RECURSIVE x AS (SELECT * FROM pg_ls_dir_metadata(dirname, true, false, true) UNION ALL SELECT x.name||'/'||a.name, a.size, a.modification, a.isdir FROM x, pg_ls_dir_metadata(dirname||'/'||x.name, true, false, true)a WHERE x.isdir) SELECT * FROM x" },
+
{ oid => '2626', descr => 'sleep for the specified time in seconds',
proname => 'pg_sleep', provolatile => 'v', prorettype => 'void',
proargtypes => 'float8', prosrc => 'pg_sleep' },
--
2.17.0
--32u276st3Jlj2kUU
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v9-0008-generalize-pg_ls_dir_files-and-retire-pg_ls_dir.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v10 7/9] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
TODO:
src/backend/catalog/system_views.sql:REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 13 +++++++++++++
src/include/catalog/pg_proc.dat | 7 +++++++
2 files changed, 20 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 3d8f7dff96..d0b782d803 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21445,6 +21445,19 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
empty directory from an non-existent directory.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ <parameter>missing_ok</parameter> ???
+ To recusively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp')dir FROM pg_tablespace b, pg_control_system() pcs, format('/PG_%s_%s', left(current_setting('server_version_num'), 2), catalog_version_no) suffix)dir, pg_ls_dir_recurse(dir)a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b4c3d15495..75423db22e 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6144,6 +6144,13 @@
provolatile => 'v', prorettype => 'text', proargtypes => 'text bool bool',
prosrc => 'pg_ls_dir' },
+{ oid => '8511', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "WITH RECURSIVE x AS (SELECT * FROM pg_ls_dir_metadata(dirname, true, false, true) UNION ALL SELECT x.name||'/'||a.name, a.size, a.modification, a.isdir FROM x, pg_ls_dir_metadata(dirname||'/'||x.name, true, false, true)a WHERE x.isdir) SELECT * FROM x" },
+
{ oid => '2626', descr => 'sleep for the specified time in seconds',
proname => 'pg_sleep', provolatile => 'v', prorettype => 'void',
proargtypes => 'float8', prosrc => 'pg_sleep' },
--
2.17.0
--rCwQ2Y43eQY6RBgR
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v10-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* [PATCH v11 7/9] Add pg_ls_dir_recurse to show dir recursively..
@ 2020-03-09 03:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: Justin Pryzby @ 2020-03-09 03:52 UTC (permalink / raw)
..possibly there's a better place to put this, like maybe a doc-only example ?
TODO:
src/backend/catalog/system_views.sql:REVOKE EXECUTE ON FUNCTION pg_ls_dir(text) FROM public;
Need catversion bumped ?
---
doc/src/sgml/func.sgml | 24 ++++++++++++++++++++
src/include/catalog/pg_proc.dat | 6 +++++
src/test/regress/expected/misc_functions.out | 14 ++++++++++++
src/test/regress/sql/misc_functions.sql | 6 +++++
4 files changed, 50 insertions(+)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index e4ba3fee40..f3422808d9 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21351,6 +21351,15 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
For each file in a directory, list the file and its metadata. Restricted to superusers by default, but other users can be granted EXECUTE to run the function.
</entry>
</row>
+ <row>
+ <entry>
+ <literal><function>pg_ls_dir_recurse(<parameter>dirname</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry><type>setof text</type></entry>
+ <entry>
+ Call pg_ls_dir_metadata to recursively list the files in the specified directory, along with each file's metadata.
+ </entry>
+ </row>
<row>
<entry>
<literal><function>pg_ls_logdir()</function></literal>
@@ -21462,6 +21471,21 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
directory along with the file's metadata.
</para>
+ <indexterm>
+ <primary>pg_ls_dir_recurse</primary>
+ </indexterm>
+ <para>
+ <function>pg_ls_dir_recurse</function> recursively lists the files
+ in the specified directory.
+ To recursively list temporary directories in all tablespaces:
+<programlisting>
+SELECT * FROM (SELECT DISTINCT COALESCE(NULLIF(pg_tablespace_location(b.oid),'')||suffix, 'base/pgsql_tmp') AS dir
+FROM pg_tablespace b, pg_control_system() pcs,
+format('/PG_%s_%s', left(current_setting('server_version_num'), 2), catalog_version_no) AS suffix) AS dir,
+pg_ls_dir_recurse(dir) AS a;
+</programlisting>
+ </para>
+
<indexterm>
<primary>pg_ls_logdir</primary>
</indexterm>
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b4c3d15495..667ea5721e 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10748,6 +10748,12 @@
proallargtypes => '{text,bool,bool,text,int8,timestamptz,bool}', proargmodes => '{i,i,i,o,o,o,o}',
proargnames => '{dirname,missing_ok,include_dot_dirs,name,size,modification,isdir}',
prosrc => 'pg_ls_dir_metadata' },
+{ oid => '8511', descr => 'list all files in a directory recursively',
+ proname => 'pg_ls_dir_recurse', prorows => '10000', proretset => 't',
+ provolatile => 'v', prorettype => 'record', proargtypes => 'text',
+ proallargtypes => '{text,text,int8,timestamptz,bool}',
+ proargnames => '{dirname,name,size,modification,isdir}', proargmodes => '{i,o,o,o,o}',
+ prolang => 'sql', prosrc => "with recursive ls as (select * from pg_ls_dir_metadata(dirname, true, false) union all select ls.name||'/'||a.name, a.size, a.modification, a.isdir from ls, pg_ls_dir_metadata(dirname||'/'||ls.name, false, false)a where ls.isdir) select * from ls" },
# hash partitioning constraint function
{ oid => '5028', descr => 'hash partition CHECK constraint',
diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out
index 448901d841..82d967a62d 100644
--- a/src/test/regress/expected/misc_functions.out
+++ b/src/test/regress/expected/misc_functions.out
@@ -187,6 +187,20 @@ select * from pg_ls_tmpdir() where name='Does not exist';
------+------+--------------+-------
(0 rows)
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+ name | isdir
+-----------------------+-------
+ pg_wal | t
+ pg_wal/archive_status | t
+(2 rows)
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+ name | size | modification | isdir
+------+------+--------------+-------
+(0 rows)
+
--
-- Test adding a support function to a subject function
--
diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql
index 656aad93e6..2cfdac386d 100644
--- a/src/test/regress/sql/misc_functions.sql
+++ b/src/test/regress/sql/misc_functions.sql
@@ -55,6 +55,12 @@ select count(*) >= 0 as ok from pg_ls_archive_statusdir();
-- The name='' condition is never true, so the function runs to completion but returns zero rows.
select * from pg_ls_tmpdir() where name='Does not exist';
+-- Check that we at least succeed in recursing once, and that we don't show the leading dir prefix
+SELECT name, isdir FROM pg_ls_dir_recurse('.') WHERE isdir AND name~'^pg_wal';
+
+-- Check that expected columns are present
+SELECT * FROM pg_ls_dir_recurse('.') LIMIT 0;
+
--
-- Test adding a support function to a subject function
--
--
2.17.0
--oTHb8nViIGeoXxdp
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v11-0008-pg_ls_logdir-to-ignore-error-if-initial-top-dir-.patch"
^ permalink raw reply [nested|flat] 54+ messages in thread
* Re: speed up text_position() for utf-8
@ 2022-01-19 19:24 John Naylor <[email protected]>
0 siblings, 1 reply; 54+ messages in thread
From: John Naylor @ 2022-01-19 19:24 UTC (permalink / raw)
To: PostgreSQL Hackers <[email protected]>
I wrote:
> 0001 puts the main implementation of pg_utf_mblen() into an inline
> function and uses this in pg_mblen(). This is somewhat faster in the
> strpos tests, so that gives some measure of the speedup expected for
> other callers. Text search seems to call this a lot, so this might
> have noticeable benefit.
>
> 0002 refactors text_position_get_match_pos() to use
> pg_mbstrlen_with_len(). This itself is significantly faster when
> combined with 0001, likely because the latter can inline the call to
> pg_mblen(). The intention is to speed up more than just text_position.
>
> 0003 explicitly specializes for the inline version of pg_utf_mblen()
> into pg_mbstrlen_with_len(), but turns out to be almost as slow as
> master for ascii. It doesn't help if I undo the previous change in
> pg_mblen(), and I haven't investigated why yet.
>
> 0002 looks good now, but the experience with 0003 makes me hesitant to
> propose this seriously until I can figure out what's going on there.
>
> The test is as earlier, a worst-case substring search, times in milliseconds.
>
> patch | no match | ascii | multibyte
> --------+----------+-------+-----------
> PG11 | 1220 | 1220 | 1150
> master | 385 | 2420 | 1980
> 0001 | 390 | 2180 | 1670
> 0002 | 389 | 1330 | 1100
> 0003 | 391 | 2100 | 1360
I tried this test on a newer CPU, and 0003 had no regression. Both
systems used gcc 11.2. Rather than try to figure out why an experiment
had unexpected behavior, I plan to test 0001 and 0002 on a couple
different compilers/architectures and call it a day. It's also worth
noting that 0002 by itself seemed to be decently faster on the newer
machine, but not as fast as 0001 and 0002 together.
Looking at the assembly, pg_mblen is inlined into
pg_mbstrlen_[with_len] and pg_mbcliplen, so the specialization for
utf-8 in 0001 would be inlined in the other 3 as well. That's only a
few bytes, so I think it's fine.
--
John Naylor
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 54+ messages in thread
* Re: speed up text_position() for utf-8
@ 2022-02-02 20:20 John Naylor <[email protected]>
parent: John Naylor <[email protected]>
0 siblings, 0 replies; 54+ messages in thread
From: John Naylor @ 2022-02-02 20:20 UTC (permalink / raw)
To: PostgreSQL Hackers <[email protected]>
I wrote:
> I tried this test on a newer CPU, and 0003 had no regression. Both
> systems used gcc 11.2. Rather than try to figure out why an experiment
> had unexpected behavior, I plan to test 0001 and 0002 on a couple
> different compilers/architectures and call it a day. It's also worth
> noting that 0002 by itself seemed to be decently faster on the newer
> machine, but not as fast as 0001 and 0002 together.
I tested four machines with various combinations of patches, and it
seems the only thing they all agree on is that 0002 is a decent
improvement (full results attached). The others can be faster or
slower. 0002 also simplifies things, so it has that going for it. I
plan to commit that this week unless there are objections.
--
John Naylor
EDB: http://www.enterprisedb.com
haswell xeon / gcc 8
master:
Time: 608.047 ms
Time: 2859.939 ms (00:02.860)
Time: 2255.402 ms (00:02.255)
0001 only:
Time: 603.026 ms
Time: 3077.163 ms (00:03.077)
Time: 2377.189 ms (00:02.377)
0002 only:
Time: 615.444 ms
Time: 2388.939 ms (00:02.389)
Time: 2227.665 ms (00:02.228)
0001-0002:
Time: 608.757 ms
Time: 2746.711 ms (00:02.747)
Time: 2156.545 ms (00:02.157)
0001-0003:
Time: 609.236 ms
Time: 1466.130 ms (00:01.466)
Time: 1353.658 ms (00:01.354)
power8 / gcc 4.8
master:
Time: 559.375 ms
Time: 7336.079 ms (00:07.336)
Time: 5049.222 ms (00:05.049)
0001 only:
Time: 559.466 ms
Time: 7272.719 ms (00:07.273)
Time: 4878.817 ms (00:04.879)
0002 only:
Time: 555.255 ms
Time: 6061.435 ms (00:06.061)
Time: 4440.829 ms (00:04.441)
0001-0002:
Time: 555.144 ms
Time: 5378.270 ms (00:05.378)
Time: 5745.331 ms (00:05.745)
0001-0003:
Time: 554.707 ms
Time: 2189.187 ms (00:02.189)
Time: 2193.870 ms (00:02.194)
comet lake / gcc 11
master:
Time: 383.196 ms
Time: 2323.875 ms (00:02.324)
Time: 1866.594 ms (00:01.867)
0001 only:
Time: 379.193 ms
Time: 2046.540 ms (00:02.047)
Time: 1720.601 ms (00:01.721)
0002 only:
Time: 374.072 ms
Time: 1769.163 ms (00:01.769)
Time: 1485.612 ms (00:01.486)
0001-0002:
Time: 374.454 ms
Time: 1208.933 ms (00:01.209)
Time: 1018.236 ms (00:01.018)
0001-0003:
Time: 376.698 ms
Time: 1278.807 ms (00:01.279)
Time: 1177.005 ms (00:01.177)
3-year old core i5 / gcc 11
master:
Time: 378.955 ms
Time: 2329.763 ms (00:02.330)
Time: 1891.931 ms (00:01.892)
0001 only:
Time: 384.547 ms
Time: 2342.837 ms (00:02.343)
Time: 1906.436 ms (00:01.906)
0002 only:
Time: 379.096 ms
Time: 2051.755 ms (00:02.052)
Time: 1641.376 ms (00:01.641)
0001-0002:
Time: 377.758 ms
Time: 1293.624 ms (00:01.294)
Time: 1096.991 ms (00:01.097)
0001-0003:
Time: 377.767 ms
Time: 2052.826 ms (00:02.053)
Time: 1329.185 ms (00:01.329)
Attachments:
[text/plain] test-textpos-20220202.txt (1.8K, ../../CAFBsxsH3Yw61WUfARYo518-ABNYbA6AYsdGpCScHAbenBBh4ng@mail.gmail.com/2-test-textpos-20220202.txt)
download | inline:
haswell xeon / gcc 8
master:
Time: 608.047 ms
Time: 2859.939 ms (00:02.860)
Time: 2255.402 ms (00:02.255)
0001 only:
Time: 603.026 ms
Time: 3077.163 ms (00:03.077)
Time: 2377.189 ms (00:02.377)
0002 only:
Time: 615.444 ms
Time: 2388.939 ms (00:02.389)
Time: 2227.665 ms (00:02.228)
0001-0002:
Time: 608.757 ms
Time: 2746.711 ms (00:02.747)
Time: 2156.545 ms (00:02.157)
0001-0003:
Time: 609.236 ms
Time: 1466.130 ms (00:01.466)
Time: 1353.658 ms (00:01.354)
power8 / gcc 4.8
master:
Time: 559.375 ms
Time: 7336.079 ms (00:07.336)
Time: 5049.222 ms (00:05.049)
0001 only:
Time: 559.466 ms
Time: 7272.719 ms (00:07.273)
Time: 4878.817 ms (00:04.879)
0002 only:
Time: 555.255 ms
Time: 6061.435 ms (00:06.061)
Time: 4440.829 ms (00:04.441)
0001-0002:
Time: 555.144 ms
Time: 5378.270 ms (00:05.378)
Time: 5745.331 ms (00:05.745)
0001-0003:
Time: 554.707 ms
Time: 2189.187 ms (00:02.189)
Time: 2193.870 ms (00:02.194)
comet lake / gcc 11
master:
Time: 383.196 ms
Time: 2323.875 ms (00:02.324)
Time: 1866.594 ms (00:01.867)
0001 only:
Time: 379.193 ms
Time: 2046.540 ms (00:02.047)
Time: 1720.601 ms (00:01.721)
0002 only:
Time: 374.072 ms
Time: 1769.163 ms (00:01.769)
Time: 1485.612 ms (00:01.486)
0001-0002:
Time: 374.454 ms
Time: 1208.933 ms (00:01.209)
Time: 1018.236 ms (00:01.018)
0001-0003:
Time: 376.698 ms
Time: 1278.807 ms (00:01.279)
Time: 1177.005 ms (00:01.177)
3-year old core i5 / gcc 11
master:
Time: 378.955 ms
Time: 2329.763 ms (00:02.330)
Time: 1891.931 ms (00:01.892)
0001 only:
Time: 384.547 ms
Time: 2342.837 ms (00:02.343)
Time: 1906.436 ms (00:01.906)
0002 only:
Time: 379.096 ms
Time: 2051.755 ms (00:02.052)
Time: 1641.376 ms (00:01.641)
0001-0002:
Time: 377.758 ms
Time: 1293.624 ms (00:01.294)
Time: 1096.991 ms (00:01.097)
0001-0003:
Time: 377.767 ms
Time: 2052.826 ms (00:02.053)
Time: 1329.185 ms (00:01.329)
^ permalink raw reply [nested|flat] 54+ messages in thread
end of thread, other threads:[~2022-02-02 20:20 UTC | newest]
Thread overview: 54+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-03-09 03:52 [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v12 09/11] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v13 6/8] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v11 7/9] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v20 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v18 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v19 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v14 6/8] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v9 07/11] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v22 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v23 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v21 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v16 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v10 7/9] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2020-03-09 03:52 [PATCH v17 07/10] Add pg_ls_dir_recurse to show dir recursively.. Justin Pryzby <[email protected]>
2022-01-19 19:24 Re: speed up text_position() for utf-8 John Naylor <[email protected]>
2022-02-02 20:20 ` Re: speed up text_position() for utf-8 John Naylor <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox